In the previous article, we built a basic mental model of how test frameworks work. Now, it’s time to expand on that model to understand why it’s not possible to follow the Arrange-Act-Assert test arrangement pattern for tests that expect an exception to be thrown.

Let’s look at the code of our simple test framework in the state we left it previously.

tests = [
	test1() {},
	test2() {},
	test3() {},
];

failedTestExceptions = []

// Test loop
foreach(tests as test) {
	try {
		test()
	} catch(exception) {
		failedTestExceptions[] = exception
	}
}

// Error display
if(count(failedTestExceptions) === 0) {
	displayGreenBar()
}
else {
	foreach(failedTestExceptions as exception) {
		displayErrorFor(exception)
	}
}

Let’s throw some exceptions.

Imagine now that you want to assert that the system under test throws an exception under certain conditions. We can add a new test to the test list, which throws an exception to mimic the fact that the system under test throws one, like so:

test4() {

	throws MySuperException
	
}

and add it to the test list

tests = [
	test1() {},
	test2() {},
	test3() {},
	test4() {

		throws MySuperException
	
	}
];

When the test framework executes the test, the exception is thrown. This is the behavior we expect. The exception is caught by the test framework and added to the collection of failedTestException. Even if the system under test behaved as expected and correctly threw an exception, the test is marked as failing.

OK, so we need to tell the framework that we expect this to happen.

We need to find a way to tell the test framework that we expect an exception so that he can avoid considering the test as failing, and we need to do this before the exception is thrown. Otherwise, the test will fail. We can achieve this by storing the expected exception in a variable. Now, when the SUT throws the exception, we can compare it with the expected one, if set, and avoid storing the exception as an evidence of a failing test.

tests = [
	test1() {},
	test2() {},
	test3() {},
	test4() {
		expectedException = MySuperException

		throws MySuperException
	}
];

failedTestExceptions = []

// Test loop

foreach(tests as test) {
	expectedException = null
	try {
		test()
	} catch(exception) {
		if(exception != expectedException) {
			failedTestExceptions[] = exception
		}
	}
}

// Error display
if(count(failedTestExceptions) === 0) {
	displayGreenBar()
}
else {
	foreach(failedTestExceptions as exception) {
		displayErrorFor(exception)
	}
}

Note that expectedException variable is reset to null before every test to ensure that we will stop tracking that that exception is thrown for the following tests to be run. Without that part, the error would be masked if MySuperException is thrown by the SUT incorrectly in a subsequent test. Also, we are storing information about what exception is expected and not only that some exception is expected because we want to avoid masking failure if an exception that is not MySuperException is thrown.

As you can see, we have to tell the framework that an exception is thrown before the moment it will be thrown because it wouldn’t be able to compare them otherwise and decide what to do. This explains why you, unfortunately, have to stay away from the AAA pattern when testing that the SUT throws an exception.

The other important information you get with that mental of model of test frameworks expect exception is that you have to be super suspicious when you see a test where the expected exception is declared after the Act part of the test. The test is probably not testing what you think and will likely never fail because the exception goes missing ^[1].

And what if the expected exception is not thrown after all?

So far, our basic test framework doesn’t fail when the expected exception is thrown. That’s only the first half of the problem. Suppose now that the exception is not thrown. That would mean that our SUT doesn’t behave as expected, and this is something we would like to know.

We need to complement our test framework so that it can tell us that an expected exception wasn’t thrown. To achieve this, we add a continue statement in the try/catch block. If an exception is thrown, expected or not, we deal with it in the catch block and move on to the next test—nothing more to do here.

If no exception was thrown, we must ensure we weren’t expecting one. After the try/catch block, a place we should not reach if an exception was thrown, thanks to the continue instruction, we can check if an exception was expected. If we expected an exception, if expectedException variable is not null, we add to the list of errors that we never received the expected exception.

The pseudo-code looks like this:

tests = [
	test1() {},
	test2() {},
	test3() {},
	test4() {
		expectedException = MySuperException

		throws MySuperException
	
	}
];

failedTestExceptions = []

// Test loop

foreach(tests as test) {
	expectedException = null
	try {
		test()
	} catch(exception) {
		if(exception != expectedException) {
			failedTestExceptions[] = exception
		}
		continue;
	}
	// Did we get our expected exception?
	if(expectedException) {
		failedTestExceptions[] = NeverReceivedExpectedException(expectedException)
	}
}

// Error display
if(count(failedTestExceptions) === 0) {
	displayGreenBar()
}
else {
	foreach(failedTestExceptions as exception) {
		displayErrorFor(exception)
	}
}

Our mental model of the inner workings of test frameworks continues growing as we add more features to it. We now understand how choosing exceptions as the communication mechanism to tell that an assertion failed blocks us from using the Arrange-Act-Assert pattern when we want to assert that an exception is thrown.

That suite of articles on building a mental model about test frameworks will continue. In the next one, we will dive into the necessity of the tearDown method.