Understanding how your test framework works is a good start to avoid making mistakes. In this post, I share my basic mental model of how most test frameworks work. This is not 100% accurate, but it is sufficient to understand a few things about writing better tests.

In the following posts, we will continue growing that mental model and see why some features of testing frameworks exist.

A loop over functions...

You can view most test frameworks as a loop over functions. Because it’s a loop, some tests are run before others. That’s not a super impressive statement, but it has some consequences we’ll see later. For now, remember that we’ve got something that loops and that it’s the first component of our test framework.

In pseudo-code, here is the main loop for a test framework:

tests = [
	test1() {},
	test2() {},
	test3() {},
];

foreach(tests as test) {
	test()
}

... throwing exceptions...

The second part of the test framework is assertions. We use assertions to validate that everything went as expected while running the code.
To communicate if the result was ok or not, assertions use exceptions. If something is okay, assertions do nothing; if something goes wrong, they throw an exception.

Here is some pseudo-code for the superstar assertEqual assertion :

assertEqual($a, $b) {
	if($a != $b) {
		throw TestAssertionException("$a and $b are not equal")
	}
}

You get a green bar if no assertions throw an exception inside the loop, meaning that your system behaved according to your tests. Hurray!

The basics are as simple as that: a loop over functions that may throw exceptions.

But keeping it like this would not provide a great developer experience because the loop would stop at the first exception thrown. You can only know how many of your tests are failing by fixing them individually and each time relaunching your test runner to discover if you still have errors or have just fixed the last problem.

... and a try/catch.

To solve that problem, the test framework encapsulates the test function inside a try/catch block. Now, when something doesn’t go as expected, the assertion still throws an exception, which bubbles up the test function and is caught by the try/catch block inside the loop.

There, the exception is stored in a list of exceptions caught for every test, and that list will later be used to display all error messages.

Even if one or more tests fail, all tests are run. Also, the test framework is now able to display one error for each of the failing tests.

The pseudo-code for the enhanced test framework is as follows. We’ve added a try/catch encapsulating the test execution and a second loop for exception display.

tests = [
	test1() {},
	test2() {},
	test3() {},
];

failedTestExceptions = []

// Test loop
foreach(tests as test) {
	try {
		test()
	} catch(exception) {
		failedTestExceptions[] = exception
	}
}

// Error display
if(count(failedTestExceptions) === 0) {
	displayGreenBar()
}
else {
	foreach(failedTestExceptions as exception) {
		displayErrorFor(exception)
	}
}

And now, we have an understanding of how basic test frameworks work. Obviously, real test frameworks are more complex than that simple mental model, and they offer more functionalities.

In the following posts, we will see why following the Arrange-Act-Assert pattern is impossible when we verify that the system under tests throws an exceptionopen . We will also understand why the tearDown functionality is more important than the setup one and discuss the "one assertion per test" rule from a technical point of view.

As you can see, I enjoy teaching about testing. If you are looking to improve your test suites I think I can help. Have a look at my video course in French or let's have a chat and see what we can do together.