RSpec 3.3 has just been released! Given our commitment tosemantic versioning, this should be a trivialupgrade for anyone already using RSpec 3.0, 3.1 or 3.2, but if we did introduceany regressions, please let us know, and we’ll get a patch releaseout with a fix ASAP.
RSpec continues to be a community-driven project with contributorsfrom all over the world. This release includes 769 commits and 200merged pull requests from nearly 50 different contributors!
Thank you to everyone who helped make this release happen!
Historically, RSpec examples have been identified primarily byfilelocation. For example, this command:
$ rspec spec/unit/baseball_spec.rb:23…would run an example or example group defined on line 23 ofspec/unit/baseball_spec.rb. Location-based identification generallyworks well, but does not always uniquely identity a particular example.For example, if you use shared examples, your spec suite mayhave multiple copies of the example defined atspec/support/shared_examples.rb:47.
RSpec 3.3 introduces a new way to identify examples and example groups:unique IDs. The IDs are scoped to a particular file and are based on theindex of the example or group. For example, this command:
$ rspec spec/unit/baseball_spec.rb[1:2,1:4]…would run the 2nd and 4th example or group defined under the 1sttop-level group defined inspec/unit/baseball_spec.rb.
For the most part, the new example IDs are primarily used internally byRSpec to support some of the new 3.3 features, but you’re free to usethem from the command line. The re-run command printed by RSpec foreach failure will use them if the file location does not uniquelyidentify the failed example. Copying and pasting the re-run commandfor a particular failed example will always run just that example now!
--only-failures optionNow that RSpec has a robust way to uniquely identify every example, we’veadded new filtering capabilities to allow you to run only the failures. Toenable this feature, you first have to configure RSpec so it knows whereto persist the status of each example:
RSpec.configuredo|c|c.example_status_persistence_file_path="./spec/examples.txt"endOnce you’ve configured that, RSpec will begin persisting the status of eachexample after every run of your suite. You’ll probably want to add this fileto.gitignore (or whatever the equivalent for your source control system is),as it’s not intended to be kept under source control. With that configurationin place, you can use the new CLI option:
$ rspec --only-failures# or apply it to a specific file or directory:$ rspec spec/unit --only-failuresIt’s worth noting that this option filters to the examples that failedthe last time they ran, not just to the failures from the last run ofrspec. That means, for example, that if there’s a slow acceptance specthat you don’t generally run locally (leaving it to your CI server torun it), and it failed the last time it ran locally, even if it was weeks ago,it’ll be included when you runrspec --only-failures.
See ourrelishdocsfor an end-to-end example of this feature.
--next-failure optionWhen making a change that causes many failures across my spec suite—suchas renaming a commonly used method—I’ve often used a specific work flow:
This allows me to systematically work through all of the failures,without paying the cost of repeatedly running the entire suite. RSpec3.3 includes a new option that vastly simplifies this work flow:
$ rspec --next-failure# or apply it to a specific file or directory:$ rspec spec/unit --next-failureThis option is the equivalent of--only-failures --fail-fast --order defined.It filters to only failures, and will abort as soon as one fails. Itapplies--order defined in order to ensure that you keep getting the samefailing example when you run it multiple times in a row without fixingthe example.
RSpec’s random ordering has always allowed you to pass a particular--seed to run the suite in the same order as a prior run. However,this only worked if you ran the same set of examples as the originalrun. If you apply the seed to a subset of examples, their orderingwouldn’t necessarily be consistent relative to each other. This isa consequence of howArray#shuffle works:%w[ a b c d ].shuffle mayorderc beforeb, but%w[ b c d ].shuffle may ordercafterb even if you use the same random seed.
This may not seem like a big deal, but it makes--seed far lessuseful. When you are trying to track down the source of an orderingdependency, you have to keep running the entire suite to reproducethe failure.
RSpec 3.3 addresses this. We no longer useshuffle for randomordering. Instead, we combine the--seed value with the id of eachexample, hash it, and sort by the produced values. This ensures thatif a particular seed orders examplec before exampleb,c willalways come beforeb when you re-use that seed no matter what subsetof the suite you are running.
While stable random ordering is useful in its own right, the big winhere is a new feature that it enables: bisect.
RSpec has supported random ordering (with a--seed option to reproducea particular ordering) since RSpec 2.8. These features help surfaceordering dependencies between specs, which you’ll want to quicklyisolate and fix.
Unfortunately, RSpec provided little to help with isolatingan ordering dependency. That’s changing in RSpec 3.3. We now providea--bisect option that narrows an ordering dependency down to aminimal reproduction case. The new bisect flag repeatedly runs subsetsof your suite in order to isolate the minimal set of examples thatreproduce the same failures when you run your whole suite.
Stable random ordering makes it possible for you to run various subsetsof the suite to try to narrow an ordering dependency down to a minimal reproduction case.
See ourrelishdocsfor an end-to-end example of this feature.
let andsubjectHistorically,let andsubject havenever beenthread safe.That’s changing in RSpec 3.3, thanks to thegreatwork of Josh Cheek.
Note that the thread safety synchronization does add a bit of overhead,as you’d expect. If you’re not spinning up any threads in your examplesand want to avoid that overhead, you can configure RSpec todisable the thread safety.
aggregate_failures APIWhen you’ve got multiple independent expectations to make about aparticular result, there’s generally two routes you can take. One way isto define a separate example for each expectation:
RSpec.describeClientdolet(:response){Client.make_request}it"returns a 200 response"doexpect(response.status).toeq(200)endit"indicates the response body is JSON"doexpect(response.headers).toinclude("Content-Type"=>"application/json")endit"returns a success message"doexpect(response.body).toeq('{"message":"Success"}')endendThis follows the “one expectation per example” guideline that encouragesyou to keep each spec focused on a single facet of behavior. Alternately,you can put all the expectations in a single example:
RSpec.describeClientdoit"returns a successful JSON response"doresponse=Client.make_requestexpect(response.status).toeq(200)expect(response.headers).toinclude("Content-Type"=>"application/json")expect(response.body).toeq('{"message":"Success"}')endendThis latter approach is going to be faster, as the request is only made oncerather than three times. However, if the status code is not a 200, the examplewill abort on the first expectation and you won’t be able to see whether or notthe latter two expectations passed or not.
RSpec 3.3 has a new feature that helps when, for speed or other reasons, youwant to put multiple expectations in a single example. Simply wrap yourexpectations in anaggregate_failures block:
RSpec.describeClientdoit"returns a successful JSON response"doresponse=Client.make_requestaggregate_failures"testing response"doexpect(response.status).toeq(200)expect(response.headers).toinclude("Content-Type"=>"application/json")expect(response.body).toeq('{"message":"Success"}')endendendWithin theaggregate_failures block, expectations failures do not cause theexample to abort. Instead, a singleaggregate exception will beraised at the end containing multiple sub-failures which RSpec willformat nicely for you:
1) Client returns a successful response Got 3 failures from failure aggregation block "testing reponse". # ./spec/client_spec.rb:5 1.1) Failure/Error: expect(response.status).to eq(200) expected: 200 got: 404 (compared using ==) # ./spec/client_spec.rb:6 1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json") expected {"Content-Type" => "text/plain"} to include {"Content-Type" => "application/json"} Diff: @@ -1,2 +1,2 @@ -[{"Content-Type"=>"application/json"}] +"Content-Type" => "text/plain", # ./spec/client_spec.rb:7 1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}') expected: "{\"message\":\"Success\"}" got: "Not Found" (compared using ==) # ./spec/client_spec.rb:8RSpec::Core provides improved support for this feature through the use ofmetadata. Instead of wrapping the expectations withaggregate_failures,simply tag the example or group with:aggregate_failures:
RSpec.describeClient,:aggregate_failuresdoit"returns a successful JSON response"doresponse=Client.make_requestexpect(response.status).toeq(200)expect(response.headers).toinclude("Content-Type"=>"application/json")expect(response.body).toeq('{"message":"Success"}')endendIf you want to enable this feature everywhere, you can usedefine_derived_metadata:
RSpec.configuredo|c|c.define_derived_metadatado|meta|meta[:aggregate_failures]=trueunlessmeta.key?(:aggregate_failures)endendOf course, you may not want this enabled by default everywhere. Theunless meta.key?(:aggregate_failures) bit allows you to opt out individualexamples or groups by tagging them withaggregate_failures: false. Whenyou’ve gotdependent expectations (e.g. where an expectation only makessense if the prior expectation passed), or if you’re using expectations toexpress a pre-condition, you’ll probably want the example to immediately abortwhen the expectation fails.
RSpec 3.3 includes improved failure messages across the board for all matchers.Test doubles now look prettier in our failure messages:
Failure/Error: expect([foo]).to include(bar) expected [#<Double "Foo">] to include #<Double "Bar"> Diff: @@ -1,2 +1,2 @@ -[#<Double "Bar">] +[#<Double "Foo">]In addition, RSpec’s improved formatting forTime and other objects willnow be used wherever those objects are inspected, regardless of whichbuilt-in matcher you used. So, for example, where you used to get this:
Failure/Error: expect([Time.now]).to include(Time.now) expected [2015-06-09 07:48:06 -0700] to include 2015-06-09 07:48:06 -0700…you’ll now get this instead:
Failure/Error: expect([Time.now]).to include(Time.now) expected [2015-06-09 07:49:16.610635000 -0700] to include 2015-06-09 07:49:16.610644000 -0700 Diff: @@ -1,2 +1,2 @@ -[2015-06-09 07:49:16.610644000 -0700] +[2015-06-09 07:49:16.610635000 -0700]…which makes it much clearer that the time objects differ at the subsecond level.
Thanks to Gavin Miller, Nicholas Chmielewski and Siva Gollapalli for contributing tothese improvements!
RSpec::Mocks has also received some nice improvements to its failure output. RSpec’simproved formatting forTime and other objects is now applied to mock expectationfailures as well:
Failure/Error: dbl.foo(Time.now) #<Double (anonymous)> received :foo with unexpected arguments expected: (2015-06-09 08:33:36.865827000 -0700) got: (2015-06-09 08:33:36.874197000 -0700) Diff: @@ -1,2 +1,2 @@ -[2015-06-09 08:33:36.865827000 -0700] +[2015-06-09 08:33:36.874197000 -0700]In addition, the failure output forhave_received has been much improved so that whenthe expected args do not match, it lists each set of actual args, and the number oftimes the message was received with those args:
Failure/Error: expect(dbl).to have_received(:foo).with(3) #<Double (anonymous)> received :foo with unexpected arguments expected: (3) got: (1) (2 times) (2) (1 time)Thanks to John Ceh for implementing the latter improvement!
MyClass.new Verifies AgainstMyClass#initializeRSpec’sverifying doublesuse the metadata that Ruby’s reflection capabilities provide to verify,among other things, that passed arguments are valid according to theoriginal method’s signature. However, when a method is defined usingjust an arg splat:
defmethod_1(*args)method_2(*args)enddefmethod_2(a,b)end…then the verifying double is going to allowany arguments, evenif the method simply delegates toanother method that does have astrong signature. Unfortunately,Class#new is one of these methods.It’s defined by Ruby to delegate to#initialize, and will only acceptarguments that the signature ofinitialize can handle, but the metadataprovided byMyClass.method(:new).parameters indicates it can handle anyarguments, even if it can’t.
In RSpec 3.3, we’ve improved verifying doubles so that when you stubnew on a class, it uses the method signature of#initialize toverify arguments, unless you’ve redefinednew to do somethingdifferent. This allows verifying doubles to give you an error whenyou pass arguments to a stubbedMyClass#new method that the realclass would not allow.
Rails 4 added support forPATCH as the primary HTTP method forupdates. The routing matchers in rspec-rails have likewise supportedPATCH since 2.14, but the generated scaffold spec were not updatedto match. This has beenaddressed inrspec-rails 3.3.
Thanks to Igor Zubkov for this improvement!
:job spec typeNow thatActiveJob is part of Rails, rspec-rails has a new:job spectype that you can opt into by either tagging your example group with:type => :job or putting the spec file inspec/jobs if you’veenabledinfer_spec_type_from_file_location!.
Thanks to Gabe Martin-Dempesy for this improvement!
Enhancements:
RSpec::Core::Example#reporter.(Jon Rowe, #1866)RSpec::Core::Reporter#message a public supported API. (Jon Rowe, #1866)RSpec::Core::Reporter#publish(event_name, hash_of_attributes). (Jon Rowe, #1869)Set and replace withRSpec::Core::Set.(Jon Rowe, #1870)config.example_status_persistence_file_path option, which isused to persist the last run status of each example. (Myron Marston, #1888):last_run_status metadata to each example, which indicates whathappened the last time an example ran. (Myron Marston, #1888)--only-failures CLI option which filters to only the examplesthat failed the last time they ran. (Myron Marston, #1888)--next-failure CLI option which allows you to repeatedly focuson just one of the currently failing examples, then move on to thenext failure, etc. (Myron Marston, #1888)--order random ordering stable, so that when you rerun asubset with a given seed, the examples will be order consistentlyrelative to each other. (Myron Marston, #1908)let andsubject threadsafe. (Josh Cheek, #1858)--bisect CLI option, which will repeatedly run your suite inorder to isolate the failures to the smallest reproducible case.(Myron Marston, #1917)config.include,config.extend andconfig.prepend, apply themodule to previously defined matching example groups. (Eugene Kenny, #1935).rspec or~/.rspec orENV['SPEC_OPTS']) so they caneasily find the source of the problem. (Myron Marston, #1940):aggregate_failures metadata. Tag an example orgroup with this metadata and it’ll use rspec-expectations’aggregate_failures feature to allow multiple failures in an exampleand list them all, rather than aborting on the first failure. (MyronMarston, #1946)before(:context)hooks. (Denis Laliberté, Jon Rowe, #1971)it block and one from anafter block. (Myron Marston, #1985)Bug Fixes:
RSpec::Core::RakeTask#failure_message so that it gets printedwhen the task failed. (Myron Marston, #1905)let work properly when defined in a shared context that is appliedto an individual example via metadata. (Myron Marston, #1912)rspec/autorun respects configuration defaults. (Jon Rowe, #1933)Enhancements:
RSpec::Matchers::EnglishPhrasing to make it easier to writenice failure messages in custom matchers. (Jared Beck, #736)RSpec::Matchers::FailMatchers, a mixin which providesfail,fail_with andfail_including matchers for use inspecifying that an expectation fails for use byextension/plugin authors. (Charlie Rudolph, #729)tempfile (and its dependencies) unlessit is absolutely needed. (Myron Marston, #735)be_true orbe_false.(Tim Wade, #744)RSpec::Matchers#respond_to_missing? so thatRSpec::Matchers#respond_to? andRSpec::Matchers#method handledynamic predicate matchers. (Andrei Botalov, #751)raise_error matcher that you may be subject tofalse positives. (Jon Rowe, #768)raise_error matcher in negativeexpectations that may be subject to false positives. (Jon Rowe, #775)include(a, b, c) so that ifa andbare included the failure message only mentionsc. (Chris Arcand, #780)satisfy matcher to take an optional description argumentthat will be used in thedescription,failure_message andfailure_message_when_negated in place of the undescriptive“sastify block”. (Chris Arcand, #783)aggregate_failures API that allows multiple independentexpectations to all fail and be listed in the failure output, ratherthan the example aborting on the first failure. (Myron Marston, #776)raise_error matcher so that it can accept a matcher as a single argumentthat matches the message. (Time Wade, #782)Bug Fixes:
contain_exactly /match_array work with strict test doublesthat have not defined<=>. (Myron Marston, #758)include matcher so that it omits the diff when it wouldconfusingly highlight items that are actually included but are notan exact match in a line-by-line diff. (Tim Wade, #763)match matcher so that it does not blow up when matching a stringor regex against another matcher (rather than a string or regex).(Myron Marston, #772)Enhancements:
new onMyClass orclass_double(MyClass), use themethod signature fromMyClass#initialize to verify arguments.(Myron Marston, #886)stringio unnecessarily. (Myron Marston, #894)RSpec::Mocks::Configuration#when_declaring_verifying_double toRSpec::Mocks::Configuration#before_verifying_doubles and utilise whenverifying partial doubles. (Jon Rowe, #940)ObjectFormatter for improved formatting ofarguments in failure messages so that, for example, full timeprecisions is displayed for time objects. (Gavin Miller, Myron Marston, #955)Bug Fixes:
have_received(msg).with(args).exactly(n).times andreceive(msg).with(args).exactly(n).times failure messagesfor when the message was received the wrong number of times withthe specified args, and also received additional times with otherarguments. Previously it confusingly listed the arguments as beingmis-matched (even when the double was allowed to receive with anyargs) rather than listing the count. (John Ceh, #918)any_args/anything support so that we avoid callingobj == anythingon user objects that may have improperly implemented== in a way thatraises errors. (Myron Marston, #924)NoMethodError internally in RSpec. (Myron Marston #954)Enhancements:
routes by usingyieldinstead ofcall. (Anton Davydov, #1308)ActiveJob specs as standardRSpec::Rails::RailsExampleGoupsvia both:type => :job and inferring type from spec directoryspec/jobs.(Gabe Martin-Dempesy, #1361)RSpec::Rails::FixtureSupport into example groups using metadata:use_fixtures => true. (Aaron Kromer, #1372)rspec:request generator for generating request specs; this is analias ofrspec:integration (Aaron Kromer, #1378)rails_helper generator with a default check to abort the spec runwhen the Rails environment is production. (Aaron Kromer, #1383)Enhancements:
Bug Fixes:
FuzzyMatcher so that it checksexpected == actual rather thanactual == expected, which avoids errors in situations where theactual object’s== is improperly implemented to assume that onlyobjects of the same type will be given. This allows rspec-mocks’anything to match against objects with buggy== definitions.(Myron Marston, #193)