Cet article est disponible en français.

There is a feature of test runners that we use far too little, myself included: tags.

With a bit of imagination, they can provide great services, and even change some discussions around testing and test cases organization.

What are tags?

Usually, to create groups of tests, we rely on the position of the test within our test suite. Tests are grouped because they belong to the same test class, the same describe method, the same file, or the same file hierarchy.

Tags are another way to create groups of tests without needing to know where they are stored. Some tools even refer to them as groups rather than tags.

Generally, tags are implemented through the annotation mechanism of the language in which the tests are written.

Once tags are in place, we have a new way to decide which tests we want to run or not. We will see examples of usage in the rest of the article.

My test runner doesn't support tags

That's unfortunate...

But chances are, the community offers a plugin that provides a similar mechanism^[1].

If not, all is not lost. There is a good chance the test runner allows filtering tests based on their name^[2]. We can then decide to include tags at the end of test names and filter on the pattern that interests us. Adding a tag to the test name adds noise, information that is not relevant in some cases, so it's important to have a convention that makes it easy to know if information is a tag or not. Starting all tags with a hashtag (#) is probably a good idea.

Usage examples

Let's look at some ideas of what tags can do for us.

Test type

The first idea is the most obvious, so obvious that it may seem unnecessary, yet...

Tags can indicate the type of the test. Is it a unit test, an integration test, an end-to-end test, an acceptance test, or another type of test?

With tags, we can then decide to run only certain types of tests at certain times.

This idea seems unnecessary because you may already be using test suites arranged in different folders to separate test types, and you can already decide which types of tests to run.

The downside of sorting test types by folder is that it forces us to maintain one or more file hierarchies mirroring that of the production code. And let's be honest, most of the time this synchronization is not done. We end up with completely different folder hierarchies everywhere.

Thanks to tags, we can abandon this constraint and place all tests in the same hierarchy, greatly facilitating maintainability.

We can even consider going further and placing tests directly next to the code and still have a way to choose which types of tests to run.

And now, no more synchronized folder hierarchies to maintain, yet we can easily select which types of tests to run.

Dependencies

The second idea is a continuation of the previous one.

But first, let me ask you a few questions:

How many times have you had a debate about whether a test is a unit test? Or rather an integration test? Or perhaps an acceptance test? Maybe an end-to-end test?
How much time have you wasted arguing as a team about how to classify a test and which folder to put it in?

What if the most important thing is not the type of test but its characteristics?

Think about it for 2 minutes: why do we care so much about the type of a test?

Isn't being able to separate slow tests from others a big part of it?

If your workflow relies heavily on tests, for example if you practice TDD, you probably don't want to wait a long time to find out if your changes had an unexpected impact on the system. Chances are you frequently run the fastest tests as a first safety net, and occasionally run all the tests, even the slowest ones, for a more comprehensive safety net.

So rather than debating the type of tests, let's ask ourselves why they are slow. Generally, tests are slow when they involve I/O operations. Tests that touch a database, a file system, or make calls to external services via the network are much slower than other tests^[3].

We then have a new idea for tag categories: tags that indicate dependencies.

When a test needs a particular dependency, we can indicate it with a tag, such as database, external-call, network, filesystem, etc.

With this type of tag we no longer have to try to precisely categorize tests; we just describe the dependencies they need, avoiding endless debates.

We can now select or exclude certain tests based on what they depend on. When we need to run only the fastest tests, it's easy to exclude troublesome dependencies.

As a bonus, if a dependency is missing in an environment, for example if we can't make calls to external services because we're on a train, we can easily exclude all tests that won't pass anyway.

This idea of selection by dependency, which is unmanageable with the file system, becomes possible with tags.

More precise documentation

I often talk about using tests as documentation, for developers, but also for the rest of the team, and even for the business.

In PHP, with PHPUnit, the '--testdox' option allows generating documentation in text or HTML format, except...

Different people are interested in different aspects of the system. Rather than generating huge documents that no one will read, it is possible to add tags for each topic of interest.

With these tags, we can now generate specific documentation for everyone:

• documentation about payment features by filtering on the feat-payment tag
• documentation on the backoffice operation with tool-bo
• ...

Again, we can manage with the file system, but there are also more transversal topics, such as security (security), translation (i18n), or business rules that concern only customers from certain countries (market-es)...

By using tags, we can mark topics that interest us on a recurring basis.

Link with the ticketing system

Tags can also link to a ticketing system.

With the ticket number as a tag, we can easily find a non-regression test for a bug bug-3457 or find all tests related to a user story US-89.

I don't think these tags are very useful in the long term because after a while knowing that a behavior was created in connection with a ticket is no longer very relevant in my opinion, but they can still be useful in the short term.

For example, the build tool could generate specific documentation based on the tests for each user story and display it directly in a pull request comment^[4] or check that a non-regression test has been added before resolving the bug.

Nomenclature

As with any tool, overuse can have unpleasant consequences.

To prevent the use of tags from becoming a huge mess it is important to think about a nomenclature. To do this, ask yourself what you can get from using tags, whether to select tests to run or to extract information from them.

Once you have defined what tags will be used for and identified categories you can use prefixes to easily identify tags.

We encountered some ideas in this article:

• feat-payment
• US-89
• bug-3457
• tool-backoffice
• market-es
• domain-delivery

Also, remember to clean up occasionally and remove tags that are no longer useful - especially if you use tags linked to the ticketing system.

For this, let your test runner help you. If it allows filtering tests by group, it probably has a command to list tags. For example, PHPUnit has a --list-groups command. If the test runner doesn't help, you can probably manage with grep.

Some tags will have more or less long lifespans by their nature, your usage will likely change as well, and that's okay.

I hope this article has given you ideas, whether the ones presented or others, on how you can incorporate the use of tags into your daily routine to get the most out of your tests.

And if you want to make sure you have tests you truly enjoy working with, you can access my video training on improving automated tests, in French. You can also schedule a meeting so we can work together to take your team to the next level.