TL;DR: Steer clear of non-deterministic tests..

Problem

• Lack of determinism

• Eroding confidence

• Time squandered

Solution

1. Ensure that the test is fully deterministic. Eradicate any potential for unpredictable behavior.

2. Eliminate test coupling.

Examples

The terms "Fragile," "Intermittent," "Sporadic," and "Erratic" are often used interchangeably when discussing problematic tests in many development environments.

However, such tests erode the trust developers place in their test suites.

Such tests are best avoided.

Sample Code

Wrong

importorg.junit.jupiter.api.Assertions.assertEqualsimportorg.junit.jupiter.api.TestabstractclassSetTest{protectedabstractfunconstructor():MutableSet<String>@Testfun`testaddempty`(){valset=constructor()set.add("green")set.add("blue")assertEquals("[green, blue]",set.toString())// This is fragile since the outcome hinges on the set's// ordering, which isn't predefined and can vary based on// the set's implementation.}}

Right

importorg.junit.jupiter.api.Testimportkotlin.test.assertEqualsimportkotlin.test.assertTrueabstractclassSetTest{protectedabstractfunconstructor():MutableSet<String>@Testfun`testaddempty`(){valset=constructor()set.add("green")assertEquals("[green]",set.toString())}@Testfun`testentryatsingleentry`(){valset=createFromArgs("red")valresult=set.contains("red")assertTrue(result)}}

Conclusion

Fragile tests often indicate system coupling and manifest as non-deterministic or unpredictable behaviors.

Addressing and resolving these erratic tests can drain considerable developer time and energy.

I hope you enjoyed this journey and learned something new. If you want to stay updated with my latest thoughts and ideas, feel free to register for mynewsletter. You can also find me onLinkedIn orTwitter. Let's stay connected and keep the conversation going!

Credits

• Code Smell 51 - Fragile Tests byMaximiliano Contieri