AtPotloc we have a test stack which is pretty standard in the Rails ecosystem. We run tests withRSpec, we useFactoryBot for setting up our test data,Capybara for user interactions,Github Actions as a CI etc..

These great tools allow us to code at a fast pace with good test coverage. But this pace comes at a cost. The more the team grows, the bigger the codebase gets and the more tests get written.

🤕 The issue

As developer we usually take care of optimizing our own code and queries, we are used to test new implementations with all the edge cases. But tests optimization is likely a topic that we putat the bottom of the list and that’s if we ever even think about it.
Up until the moment where quietly but surely you will end up with a CI that take ages to run and a whole test suite to speed up.

Let’s see how we took on this challenge atPotloc.

For the purpose of demonstration we will rely on this simple test file:

RSpec.describePurging::QuestionnaireWorker,type: :workerdolet(:questionnaire){create(:questionnaire)}describe"#perform"doit"destroys a survey form"doexpect{subject.perform(questionnaire.id)}.tochange(Questionnaire,:count).by(-1)endcontext"given associations"doit"destroys a questionnaire and its associations"docreate(:question,:postal_code,questionnaire:questionnaire)expect{subject.perform(questionnaire.id)}.tochange(SurveyQuestion,:count).by(-1)endendendend

🧑‍🚒 The solutions

Thefactory-bot gem is used in almost in all of our spec files and it make our set up much more easier than when we use fixtures.
Here is the tradeoff, the easier the gem is to use, the more likely you’ll end up with some pain to control its usage. And when the times come to tackle slow tests,the best bet you can take is to start digging into you factories because it’s likely they are the primary reason why your test suite is slowing down

• Avoiding the factory cascades

To quoteEvil Martian a factory cascade is an

uncontrollable process of generating excess data through nested factory invocations.

In our test we use two factories aquestionnaire andquestion who could be represented like this as a tree:

questionnaire||---- surveyquestion||---- survey

In this simple example each factory calls a nested factory. This means that every time we create aquestionnaire factory, we also create asurvey factory.

To have a better vision of what objects are created in our spec file we can usetest-prof, a powerful gem that provides a collection of different tools to analyse your test suite performance. One of this tool is really useful toidentify a factory cascade, let’s introducefactory profiler.

If we runFPROF=1 bundle exec rspec the factory profiler, it will generate the following report:

[TEST PROF INFO] Factories usage Total: 6 Total top-level: 3 Total time: 00:01.005 (out of 00:26.087) Total uniq factories: 3   total   top-level     total time      time per call      top-level time               name       3           0        0.6911s            0.2304s             0.0000s             survey       2           2        0.6397s            0.3199s             0.6397s             questionnaire       1           1        0.3658s            0.3658s             0.3658s             question

The most interesting insight is thedifference between thetotal and thetop-level. The more this difference is important, the more you end up with factory cascade, meaning thatyou are creating useless factories.

Let’s take thesurvey , we don’t instantiate any surveys in our test suite but during the execution we create 4.

The easiest workaround it is to instantiate asurvey at the top-level and to associate the factories to it.

RSpec.describePurging::QuestionnaireWorker,type: :workerdolet(:survey){create(:survey)}let(:questionnaire){create(:questionnaire,survey:survey)}....it"destroys a questionnaire and its associations"docreate(:question,:postal_code,questionnaire:questionnaire,survey:survey)....

If we run the factory profiler we now have a different report:

[TEST PROF INFO] Factories usage Total: 5 Total top-level: 5 Total time: 00:00.868 (out of 01:09.067) Total uniq factories: 3   total   top-level     total time      time per call      top-level time               name       2           2        0.6986s            0.3493s             0.6986s             survey       2           2        0.0973s            0.0487s             0.0973s             questionnaire       1           1        0.0729s            0.0729s             0.0729s             question

Nice! No more factory cascades. Thetotal and thetop-level columns are the same.We now create 5 factories instead of 8. We have decreased the time spent creating factoriesby 30%.

The caveat of this method it that it could bea heavy process to maintain. Thankfully,test-prof as a recipe calledFactoryDefault. Removing factory cascades manually could be good enough most of the time but if you want to go further you can follow thedocumentation.

That being said,test-prof has even more to offer, it’s time to introduce an awesome helper namedlet_it_be.

• Reuse the factory you need

Let's bring a little bit of magic and introduce a new way to set up a shared test data.

let_it_be is ahelper that allows you to reuse the same factory for all your spec file. In our example we don’t need to create 2survey and 2questionnaire we could re-use the same ones for all our file.

RSpec.describePurging::QuestionnaireWorker,type: :workerdolet_it_be(:survey){create(:survey)}let_it_be(:questionnaire){create(:questionnaire,survey:survey)}...end

If we run the factory profiler we now have a different report:

[TEST PROF INFO] Factories usage Total: 3 Total top-level: 3 Total time: 00:00.272 (out of 00:24.264) Total uniq factories: 3   total   top-level     total time      time per call      top-level time               name       1           1        0.2024s            0.2024s             0.2024s             survey       1           1        0.0323s            0.0323s             0.0323s             questionnaire       1           1        0.0375s            0.0375s             0.0375s             question

So now we only create the factories we need, by reusing the same ones throughout our file.

Be aware thatlet_it_be come with acaveat section. I strongly encourage you to read the documentation and use this powerful helper in accordance with your needs.

🚀 Conclusion

Let’s take a step back and relish our improvements:

Numbers look nice for this simple example. But what is the impact in real life atPotloc?

So far we just applied this recipe for a specific folder of our codebase. Below the result by profiling locally that folder:

Before we spent3.50 min in factories creation, now2 min. (~ -50%)
Before we created6824 factories, now4378. (~ -35%)

test-prof is the swiss army knife we needed to speed up our test suite. It’s still a long journey but by embracing this topic we have already taken an important step!

Want to go further? Watch this99 problems of slow test talk byVladimir Dementyev

Interested in what we do at Potloc? Come join us! We are hiring 🚀