Repository files navigation

A collection of tools to keep mocks in line with real objects.

Based on the RubyConf 2022 talk"Weaving and seaming mocks"

• Installation
• Typed doubles
  • Using with RBS
  • Typed doubles limitations
• Mock context
• Auto-generated type signatures and post-run checks
• Mock contracts verification
• Tracking stubbed method calls
• Tracking real method calls
• Configuration
• Future development

# Gemfilegroup:testdogem"mock-suey"end

Then, drop"require 'mock-suey' to yourspec_helper.rb /rails_helper.rb /whatever_helper.rb.

MockSuey enhances verified doubles by adding type-checking support: every mocked method call is checked against the corresponding method signature (if present), and an exception is raised if types mismatch.

Consider an example:

let(:array_double){instance_double("Array")}specify"#take"doallow(array_double).toreceive(:take).and_return([1,2,3])expect(array_double.take("three")).toeq([1,2,3])end

This test passes with plain RSpec, because from the verified double perspective everything is valid. However, calling[].take("string") raises a TypeError in runtime.

With MockSuey andRBS, we can make verified doubles stricter and ensure that the types we use in method stubs are correct.

To enable typed verified doubles, you must explicitly configure a type checker.

NOTE: RBS 3.0+ is not supported yet. We're working on it.

To use MockSuey with RBS, configure it as follows:

MockSuey.configuredo |config|config.type_check=:ruby# Optional: specify signature directries to use ("sig" is used by default)# config.signature_load_dirs = ["sig"]# Optional: specify whether to raise an exception if no signature found# config.raise_on_missing_types = falseend

Make sure thatrbs gem is present in the bundle (MockSuey doesn't require it as a runtime dependency).

That's it! Now all mocked methods are type-checked.

Typed doubles rely on the type signatures being defined. What if you don't have types (or don't want to add them)? There are two options:

1. Adding type signatures only for the objects being mocked. You don't even need to type check your code or cover it with types. Instead, you can rely on runtime checks made in tests for real objects and use typed doubles for mocked objects.

2. Auto-generating types on-the-fly from the real call traces (see below).

Mock context is a re-usable mocking/stubbing configuration. Keeping alibrary of mockshelps to keep fake objects under control. The idea is similar to data fixtures (and heavily inspired by thefixturama gem).

Technically, mock contexts areshared contexts (in RSpec sense) thatknow which objects and methods are being mocked at the boot time, not at the run time. We use this knowledge to collect calls made onreal objects (so we can use them for the mocked calls verification later).

The API is similar to shared contexts:

# Define a contextRSpec.mock_context"Anyway::Env"dolet(:testo_env)do{"a"=>"x","data"=>{"key"=>"value"}}endbeforedoenv_double=instance_double("Anyway::Env")allow(::Anyway::Env).toreceive(:new).and_return(env_double)allow(env_double).toreceive(:fetch).with("UNKNOWN",any_args).and_return(Anyway::Env::Parsed.new({},nil))allow(env_double).toreceive(:fetch).with("TESTO",any_args).and_return(Anyway::Env::Parsed.new(testo_env,nil))allow(env_double).toreceive(:fetch).with("",any_args).and_return(nil)endend# Include in a test filedescribeAnyway::Loaders::Envdoinclude_mock_context"Anyway::Env"# ...end

It's recommended to keep mock contexts underspec/mocks orspec/fixtures/mocks and load them in the RSpec configuration file:

Dir["#{__dir__}/mocks/**/*.rb"].sort.each{ |f|requiref}

You can get the registry of mocked objects and methods after all tests were loaded,for example, in thebefore(:suite) hook:

RSpec.configuredo |config|config.before(:suite)doMockSuey::RSpec::MockContext.registry.eachdo |klass,methods|methods.eachdo |method_name,stub_calls|# Stub calls is a method call object,# containing the information about the stubbed call:# - receiver_class == klass# - method_name == method_name# - arguments: expected arguments (empty if expects no arguments)# - return_value: stubbed return valueendendendend

We can combine typed doubles and mock contexts to provide type-checking capabilities to codebase not using type signatures. For that, we can generate type signatures automatically by tracingreal object calls.

You must opt-in to use this feature:

MockSuey.configuredo |config|# Make sure type checker is configuredconfig.type_check=:rubyconfig.auto_type_check=true# Choose the real objects tracing methodconfig.trace_real_calls_via=:prepend# or :trace_point# Whether to raise if type is missing in a post-check# (i.e., hasn't been generated)config.raise_on_missing_auto_types=trueend

Under the hood, we use theTracking real method calls feature described below.

IMPORTANT: Only objects declared within mock contexts could be type-checked.

Currently, this feature only works if both real objects and mock objects calls are made during the same test run. Thus, tests could fail when running tests in parallel.

Types drastically increase mocks/stubs stability (or consistency), but even they do not guarantee that mocks behave the same way as real objects. For example, if your method returns completely different results depending on the values (not types) of the input.

The only way to provide ~100% confidence to mocks is enforcing a contract. One way to enforce mock contracts is to require having a unit/functional tests where a real object receives the same input and returns the same result as the mock. For example, consider the following tests:

describeAccountantdolet(:calculator){instance_double("TaxCalculator")}# Declaring a mock == declaring a contract (input/output correspondance)beforedoallow(calculator).toreceive(:tax_for_income).with(2020).and_return(202)allow(calculator).toreceive(:tax_for_income).with(0).and_return(0)endsubject{described_class.new(calculator)}specify"#after_taxes"do# Assuming the #after_taxes method calls calculator.tax_for_incomeexpect(subject.after_taxes(2020)).toeq(1818)expect(subject.after_taxes(0)).tobe_nilendenddescribeTaxCalculatordosubject{described_class.new}# Adding a unit-test using the same input# verifies the contractspecify"#tax_for_income"doexpect(subject.tax_for_income(2020)).toeq(202)expect(subject.tax_for_income(0)).toeq(0)endend

We need a way to enforce mock contract verification. In other words, if the dependency behaviour changes and the corresponding unit-test reflects this change, our mock should be marked as invalid and result into a test suit failure.

One way to do this is to introduce explicit contract verification (via custom mocking mechanisms or DSL or whatever, seebogus orcompact, for example).

Mock Suey chooses another way: automatically infer mock contracts (via mock contexts) and verify them by collecting real object calls during the test run. You can enable this feature via the following configuration options:

MockSuey.configuredo |config|config.verify_mock_contracts=true# Choose the real objects tracing methodconfig.trace_real_calls_via=:prepend# or :trace_pointend

Each method stub represents a contract. For example:

allow(calculator).toreceive(:tax_for_income).with(2020).and_return(202)allow(calculator).toreceive(:tax_for_income).with(0).and_return(0)#=> TaxCalculator#tax_for_income: (2020) -> Integer#=> TaxCalculator#tax_for_income: (0) -> Integer

If the method behaviours changes, running tests would result in a failure if mock doesn't reflect the change:

# Assuming we decided to return nil for non-positive integersspecify"#tax_for_income"doexpect(subject.tax_for_income(0)).tobe_nilend

The test suite will fail with the following exception:

$ rspec accountant_spec.rb........1) Mock contract verification failed:   No matching call found for:     TaxCalculator#tax_for_income: (0) -> Integer   Captured calls:     (0) -> NilClass

The contract describes which explicit input values result in a particular output type (not value). Such verification can help to verify boundary conditions (e.g., when some inputs result in nil results or exceptions).

1. Currently, verification takes into account only primitive values (String, Number, Booleans, plain Ruby hashes and arrays, etc.). Custom classes are not yet supported.

2. Similarly to auto-type checks, this feature does not yet support parallel tests execution.

The core functionality of this gem is the ability to hook into mocked method invocations to perform custom checks.

NOTE: Currently, only RSpec is supported.

You can add an after mock call callback as follows:

MockSuey.on_mocked_calldo |call_obj|# Do whatever you want with the method call object,# containing the following fields:# - receiver_class# - method_name# - arguments# - return_valueend

By default, MockSuey doesn't keep mocked calls, but if you want to analyze themat the end of the test suite run, you can configure MockSuey to keep all the calls andaccess them later:

MockSuey.configuredo |config|config.store_mocked_calls=trueend# Them, you can access them in the after(:suite) hook, for exampleRSpec.configuredo |config|config.after(:suite)dopMockSuey.stored_mocked_callsendend

This gem provides a method call tracking functionality for non-double objects.You can use it separately (not as a part of auto-type-checking or contract verification features). For example:

RSpec.configuredo |config|tracer=MockSuey::Tracer.new(via::trace_point)# or via: :prependconfig.before(:suite)dotracer.collect(SomeClass,%i[some_methodanother_method])tracer.start!endconfig.after(:suite)docalls=traces.stop# where call is a call object# similar to a mocked call object described aboveendend

Additional configuration options could be set:

MockSuey.configuredo |config|# A logger instanceconfig.logger=Logger.new($stdout)# You can also specify log level and whether to colorize logs# config.log_level = :info# config.color = ? # Depends on the logging device# Debug mode is a shortcut to setup an STDOUT logger with the debug levelconfig.debug=ENV["MOCK_SUEY_DEBUG"] =="true"# or 1, y, yes, or tend

I'm interested in the following contributions/discussions:

• Figure out parallel builds
• Sorbet support
• Minitest support
• Advanced mock contracts (custom rules, custom classes support, etc.)
• Methods delegation (e.g.,.perform_async -> #perform,.call -> #call)
• Exceptions support in contracts verification

Bug reports and pull requests are welcome on GitHub athttps://github.com/test-prof/mock-suey.

This gem is generated vianew-gem-generator.

The gem is available as open source under the terms of theMIT License.