Repository files navigation

Yeti automates tests written for various test frameworks.Yeti scales from your dev box (where it works by itself)to your CI system (where it launches browsers with Selenium)without changing your existing tests.

• Works with the frameworks you already use.
• Automates tests without any additional software. Selenium not required!
• Built-in code coverage provided byIstanbul.
• Works with IE 6+, Android 4+, Firefox, Safari, Chrome, iOS 4+.
• Server-side AJAX testing withechoecho.
• JUnit XML output makes Yeti play nice with Jenkins.
• Workload can be split across multiple instances of the same browser.
• Optional Selenium/WebDriver browser launching. Works great with Sauce Labs.

You can use any of these test frameworks with Yeti.

• QUnit
• Jasmine
• Mocha withExpect.js assertions
• Dojo Objective Harness
• YUI Test
• Your framework here. Submit a pull request!

npm install -g yeti

Yeti requires Node.js, which provides thenpm command for installation.You candownload Node.js source or pre-builtinstallers from their website.

Just run Yeti with the HTML files containing your tests.

$ yeti test/*.htmlCreating a Hub at http://localhost:9000Waiting for agents to connect at http://localhost:9000.When ready, press Enter to begin testing.

Point your browsers at that URL, then come back and press Enter.

[Open some browsers...]  Agent connected: Safari (6.0) / Mac OS  Agent connected: Chrome (22.0.1221.0) / Mac OS[Come back, press Enter]✔ Testing started on Safari (6.0) / Mac OS, Chrome (22.0.1221.0) / Mac OS✔ Agent completed: Safari (6.0) / Mac OS✔ Agent completed: Chrome (22.0.1221.0) / Mac OS504 tests passed! (9.1 seconds)$

Yeti exits automatically when all tests complete. If test failures occur, Yeti will exit with a non-zero status code.

• 0 indicates testing was completed and all tests passed.
• 1 indicates an error occurred that prevented testing from running to completion.
• 3 indicated testing was completed but some tests failed.

These codes are useful for using Yeti programmatically. The codes0 and3 indicate testing completion.If Yeti ended in code1, you may wish to retry the command to workaround transient errors. This is usefulwhen using Yeti to launch browsers in CI, which sometimes may fail due to temporary Selenium issues.

Yeti can output machine-readable JUnit XML suitable for use inJenkins with the--junit option.

$ yeti --junit test/*.html > yeti.xml

Yeti will output XML on stdout and status messages on stderr.

When using Yeti several times in the same Jenkins job, it's useful to label tests with a prefixto distinguish between different Yeti runs after Jenkins merges the reports together. You canassign this prefix with the--name option.

$ yeti --name stable --junit test/stable/*.html > stable.xml$ yeti --name flaky --junit test/flaky/*.html > flaky.xml

Yeti can generate a code coverage report with the-c or--coverage option.

$ yeti -cFound 1 file to test.  Agent connected: Chrome (34.0.1825.4) / Mac OS from 127.0.0.1✓ Testing started on Chrome (34.0.1825.4) / Mac OS✓ Agent completed: Chrome (34.0.1825.4) / Mac OSCoverage summary:Statements   : 96.02% ( 1110/1156 )Branches     : 88.53% ( 772/872 )Functions    : 97.45% ( 229/235 )Lines        : 96.27% ( 1110/1153 )✓ 397 tests passed! (7 seconds)

Use the--coverage-report option to generate reports written to./coverageor another directory defined by--coverage-dir. Values for--coverage-report include:

• lcov for LCOV and HTML reports.
• lcovonly for LCOV without HTML.
• html for HTML only.
• json for Istanbul JSON.
• summary for the default summary shown above.
• Any otherreport supported by Istanbul.

Yeti instruments all JavaScript files automatically withIstanbul. If you'd prefer to instrumentyour JavaScript code with Istanbul as a part of a build step, you can pass--no-instrumentto skip all instrumenting. Yeti will still generate code coverage reports if used withthe--coverage option.

If you'd like to exclude some JavaScript files from instrumenting, you can pass multiple--instrument-exclude options definingminimatch patterns to exclude.

$ yeti -c --instrument-exclude "**/vendor/**" --instrument-exclude "**.nocover.js"

To prevent needing to specify these verbose options every time, you can put acoverageOptions object in your project's.yeti.json file containing configurationthat is only used with--coverage.

$ cd yui3/src/app$ cat ../../.yeti.json{    "basedir": ".",    "glob": "**/tests/unit/*.html",    "coverageOptions": {        "instrument": false,        "query": "filter=coverage"    }}$ yeti -c # will run with --no-instrument --query "filter=coverage"Found 1 file to test.  Agent connected: Chrome (34.0.1825.4) / Mac OS from 127.0.0.1✓ Testing started on Chrome (34.0.1825.4) / Mac OS✓ Agent completed: Chrome (34.0.1825.4) / Mac OSCoverage summary:Statements   : 96.02% ( 1110/1156 )Branches     : 88.53% ( 772/872 )Functions    : 97.45% ( 229/235 )Lines        : 96.27% ( 1110/1153 )✓ 397 tests passed! (7 seconds)

Yeti provides server-side AJAX routes withechoecho. Your test canmake relative HTTP requests to test your code against server-side HTTPGET, POST, PUT, DELETE, OPTIONS, GET with delay, JSON or JSONP responses via POST,or any HTTP status code.

Example supported routes:

• echo/status/500 returns a 500 response.
• echo/delay/3 returns a 200 response after 3 seconds.
• echo/jsonp?callback=foo returns a JSONP response with the givenPOST data wrapped in a call tofoo.

Note these routes are intentionally relative paths.See theechoecho README for more details.

Yeti will move on to the next test if a test takes longer than 5 minutes (300 seconds).You can adjust this interval with the--timeout option.

This will run Yeti with a 120 second timeout:

$ yeti --timeout 120 test.html

There isn't a general timeout setting. Yeti actively pings browsers about every2-5 seconds and disconnects them if they fail to respond to a ping three times.

You can specify query string parameters to add to your test URLs.This can be used to pass information to your tests that control its behavior.

This will append?filter=coverage to your tests, which is used by the testsfor theYUI Library to trigger loading instrumented code.

$ yeti --query 'filter=coverage' test/*.html

Yeti will report an uncaught exceptions as Script Errors.

Yeti enforcesNo-Quirks Mode in your tests because it may impact DOM-related APIs.Add a DOCTYPE to your test document to fix this.

When combined withlocaltunnel, mobile testing is simple. If you're not dealing with sensitive information, startup your Yeti Hub and then run:

$ localtunnel 9000   Port 9000 is now publicly accessible from http://3z48.localtunnel.com ...

You can then visit that URL on your mobile (or any other) device and have it run new tests.

To save time, start a Yeti Hub.

$ yeti --serverYeti Hub listening on port 9000.

Point browsers at your local Yeti on port 9000. Now, you're ready to run tests without having to reconnect browsers each time.

Starting Yeti in another terminal will connect to that Hub instead of starting a new oneand will begin testing immediately if browsers are already connected.

$ yeti test/*.htmlConnected to http://localhost:9000  Agent connected: Chrome (22.0.1221.0) / Mac OS  Agent connected: Safari (6.0) / Mac OS✔ Testing started on Chrome (22.0.1221.0) / Mac OS, Safari (6.0) / Mac OS✔ Agent completed: Chrome (22.0.1221.0) / Mac OS✔ Agent completed: Safari (6.0) / Mac OS504 tests passed! (11.5 seconds)$

Your Yeti Hub can be shared with other developers.

First, I'll start a Hub on test.yeti.cx on port 80.

$ yeti --server --port 80

Go ahead and point a few browsers there.

Now, others can connect to it from their computer like so:

$ yeti --hub http://test.yeti.cx/ test/*.htmlConnected to http://test.yeti.cx/Waiting for agents to connect at http://test.yeti.cx/.When ready, press Enter to begin testing.

Yourpwd and your test file will be served through the Hub. Like magic.

[Hit Enter]  Agent connected: Chrome (22.0.1221.0) / Mac OS  Agent connected: Safari (6.0) / Mac OS✔ Testing started on Chrome (22.0.1221.0) / Mac OS, Safari (6.0) / Mac OS✔ Agent completed: Safari (6.0) / Mac OS✔ Agent completed: Chrome (22.0.1221.0) / Mac OS504 tests passed! (8.7 seconds)

This makes it really simple to setup an ad-hoc testing lab shared with your team.

You can specify thewd-url option to connect Yeti to a Selenium 2 Hub using theWebDriver protocol. Specifying one or morecaps options will cause Yeti to launchbrowsers for the given capabilities over WebDriver.

For example, launching 2 Firefox browsers to test multiple files:

yeti --hub http://example.com --wd-url http://localhost:4444 \     --caps "browserName=firefox;version=26;platform=Windows XP" \     --caps "browserName=firefox;version=26;platform=Windows XP" test/*.html

Or launching an iPhone simulator on Sauce Labs:

yeti --caps "platform=OS X 10.8;version=7;device-orientation=portrait;app=safari;device=iPhone Simulator" test.html

Here's a breakdown of all available CLI options.

• query (String) Query string parameters to pass to tests.
• timeout (Number in seconds) Test timeout.
• hub (URL) Location of the Yeti Hub to use.Set tofalse or specify--no-hub to override a configuration file.
• server Starts a Yeti Hub.
• port (Number) Yeti Hub will listen to this port.
• loglevel (debug orinfo) Print debugging information.
• caps (String) Browser capabilities to launch with WebDriver.Should be used withwd-url.
• wd-url (URL) WebDriver Hub URL. May contain a username and password.
• help Print usage.
• version Print the Yeti version.

You may use place JSON in a.yeti.json file to set project or user specific configuration.

Yeti will look for.yeti.json in these places:

• Recursively starting in the directory you start Yeti
• In your home folder

Here is an example.yeti.json for the YUI project, which is placed in the repository root:

{    "hub": "http://test.yeti.cx/",    "basedir": ".",    "glob": "**/tests/unit/*.html"}

Here is the breakdown of these settings:

• Thehub option defines a Yeti Hub URL to use.
• Thebasedir option indicates that the directory where.yeti.json lives ispermitted to serve files to the Yeti Hub.
• Theglob option defines a pattern to search for test files.

These settings let YUI developers simply runyeti inside of the project directoryto run tests. Since all tests in the project match the glob pattern, theyeticommand works for specific components as well as for the entire project.

This configuration can be overridden on the command line. For example, to ignore thehub setting, you can run Yeti with--no-hub.

You canrequire("yeti") inside your application to script Yeti for your own use.

For API documentation:

• Runmake html to build HTML documentation to./build_docs.
• Review code comments insidelib/yeti.js,lib/client.js andlib/hub/index.js.

Yeti followsSemantic Versioning but is currently at a 0.x.y release.The public API is not stable. There will be changes.

Yeti typically automates test frameworks, but you can integrate any client-side test or performance frameworkinto Yeti. Combined with the Yeti API, you can easily build your own automation tools. YUI uses Yeti in thisway to automate performance benchmarks.

Normally Yeti will scan pages in order to find test frameworks. When serving a page to Yeti, you can setwindow.stopYetiScan to true to signal that your page will explicitly submit results to Yeti.

When your framework has results and is ready to move to the next page, you can callwindow.sendYetiResultswith an object containing data to report. This data will be passed through verbatim to the Node.js Yeti APIfor further processing in your tool.

Yeti should work on all platforms supported by Node.js.It's tested on Linux and OS X.

You must start Yeti's client in the directory you'll be serving tests from. For security reasons, Yeti will reject requests that try to access files outside of the directory you start Yeti in.

You can install the latest development snapshot of Yeti easily:

npm install -g http://latest.yeti.cx

This will install Yeti as it exists on theyui/yeti GitHub repository.You can check the stability of the Yeti snapshot by checkingyui/yeti on Travis.

Do you want to add new features or fix bugs in Yeti itself? We made it easy for you to hack on Yeti.

After runningnpm install, replace themake commands below with.\jake.bat to use the experimental Jake tasks that are Windows ready.

Clone Yeti.

git clone https://github.com/yui/yeti.gitcd yeti

Install Yeti's dependencies.

npm install

Yeti's automated tests require PhantomJS.You candownload PhantomJS source or pre-builtbinaries from their website. Make sure thephantomjs binary is installed in your PATH.

make testmake coverage

The latter command usesJSCoverage for Node.js,which will be built and installed to./tools/jscoverage.

make lint

You may also run the linter on individual files with./go lint:

./go lint test/blizzard.js

Yeti usesJSHint to analyze code for problems. See.jshintrc for options used by Yeti.

RequiresGoogle Chrome Canary and OS X.

Profile the Yeti Hub:

./go profile --server

Using./go profile without--server to profile the Yeti clientrequires an interactive terminal, which does not yet work.

Yeti usesSelleck to generate its website. Selleck files are located indoc/.

make html

Documentation will be built tobuild_docs/.

Yeti usesYUIDocJS to generate API documentation from inline JSDoc comment blocks.

make html-api

Documentation will be built tobuild_docs/api/everything/.

SeeCONTRIBUTING.md.

Open a ticket onYUILibrary.com's Yeti Issue Tracker to report bugs or feature requests.

Yeti is free to use under YUI's BSD license.See the LICENSE file or theYUI license pagefor license text and copyright information.