Repository files navigation

Visual regression testing framework, comparing screenshots from multiple builds.

Table of Contents

• Installation
• Getting Started
• Command-Line
• Examples
• API-Documentation
• Tests
• Project Naming
• Third-party libraries
• License

##Installation

Install this module with the following command:

npm install kobold

Install this module globally with the following command:

npm install -g kobold

This will make sure that you don't have to enter the whole path when running Kobold.

Add the module to yourpackage.json dependencies:

npm install --save kobold

Add the module to yourpackage.json dev-dependencies:

npm install --save-dev kobold

##Getting Started

Kobold can be executed by runningkobold script in thebin directory of the package. When you installed the module globally, then you can us onlykobold to run it.

Only one parameter is required to run the tests:

kobold test/ui/regression

With this parameter, Kobold starts running tests on thetest/ui/regression directory. It analyzes what screens were already approved, what screens are available in the current build, and what screens it has to compare. Then, it creates the test-code and executes the image-comparison, creating the test results in thehighlight directory.

Instead of implicitly using the last argument for the test-path, you can also be more specific and supply the--test-path parameter:

kobold --test-path test/ui/regression

By default, Kobold will look for four directories:

• approved - All approved screens should be available in this folder. These are the "golden standard" images that will be used to compare them to build screens.
• build - Screens that were just taken are saved in this folder. This is most likely done by some Selenium tests, taking screenshots and saving them in this directory. These images will be compared to the images in theapproved folder, creating comparison result images inhighlight.
• highlight - Image comparison results of the latest test-run are saved in this folder. These images will highlight the differences between theapproved-image and thebuild-image.
• config - Custom configuration files (.js or .json) for image specific comparison options. Sometimes, it is helpful to tune the comparison of a specific image. These options overwrite globally set comparison options. They are also optional. See the Blink-Diff Project WebSite for more information on these options.

The folder names forapproved,build,highlight, andconfig can be changed by supplying the parameters--approved-folder,--build-folder,--highlight-folder, and--config-folder respectively:

kobold --approved-folder"golden" --highlight-folder"differences" /test/ui/regression

This test-run will look forgolden anddifferences for theapproved andhighlight folder respectively instead of using the default folder names. In this specific case,build was not changed.

Sometimes, you want the tests to fail when previously approved screens are missing from thebuild directory. This can be the case when for example the tests that created these screenshots didn't fully run. Use the--fail-orphans flag to make these tests fail:

kobold --fail-orphans test/ui/regression

The--fail-additions flag will fail tests for screens that are new, screens that were never approved before.

kobold --fail-orphans --fail-additions test/ui/regression

Kobold also supports all Mocha parameters which can be interspersed with the Kobold parameters:

kobold --slow 3000 --fail-orphans --fail-additions --reporter dot --test-path test/ui/regression

##Command-LineSince Kobold is built on top of Mocha, it supports all of its parameters in addition to the following:

• approved-folder - Name of the approved folder (default: 'approved')
• build-folder - Name of the build folder (default: 'build')
• highlight-folder - Name of the highlight folder (default: 'highlight')
• config-folder - Name of the config folder (default: 'config')
• fail-orphans - Flag that determines that tests should fail when screens that were previously approved, being in theapprove folder, are not found in thebuild folder.
• fail-additions - Flag that determines that tests should fail when unapproved screens are found in thebuild folder, meaning screens that are missing from theapprove folder.
• test-path - Path to directory with all test related folders as mentioned above (required). Uses implicitly the last parameter element when none is given.
• config - Path to a config file (.js or .json) that is used to overwrite comparison options. (default: none) See the Blink-Diff Project WebSite for more information on these options.

##ConfigurationThe global configuration file that can be selected by supplying--config to the command-line describes default-values for the comparison.

Here is an example of the file (config.js - can also be a JSON file):

module.exports={"delta":35// Changing the distance of pixel-comparison};

This file then can be called as follows:

kobold --config config.js test/ui/regression

Configuration files for specific screens on the other hand describe only the comparison options for that screen.Here is an example for a screen that is named "YDN_Missing"; the path isconfig/YDN_Missing.json (could also be a.js):

{"outputBackgroundOpacity":0.3,"outputBackgroundGreen":100}

This example will make the background for the comparison image look greener, and lightens-up the whole comparison by reducing the opacity of the background-mask.

##ExamplesKobold creates a regular test report as if the tests were manually written, and it also creates the following images, comparing the approved (left) with the build screenshot (right), pointing out the differences (middle).

• Changed color
• Missing items
• Messed-up sorting
• Image swap
• Text formatting

These examples can be found in theexamples folder. The above results were produced by running the following command in the project root with a globally installed Kobold:

kobold test/ui/regression

##API-Documentation

Generate the API-documentation with following command:

npm run docs

This will generate the source-code documentation in thedocs folder of the module root.

##Tests

Run the tests with the following command:

npm runtest

The code-coverage will be written to thecoverage folder in the module root.

##Project NameA Kobold is a gnome that can make itself invisible and is often the source of nuisances - so, pretty much like visual regressions.

##Third-party libraries

The following third-party libraries are used by this module:

###Dependencies

• blink-diff:https://github.com/yahoo/blink-diff
• preceptor-core:https://github.com/yahoo/preceptor-core
• kobold-core:https://github.com/yahoo/kobold-core
• log4js:https://github.com/nomiddlename/log4js-node
• promise:https://github.com/then/promise
• mocha:https://github.com/visionmedia/mocha

###Dev-Dependencies

• chai:http://chaijs.com
• istanbul:https://github.com/gotwarlost/istanbul
• yuidocjs:https://github.com/yui/yuidoc

##License

The MIT License

Copyright 2014 Yahoo Inc.