Repository files navigation

xctool is an extension for Apple'sxcodebuild which makes iteasier to test iOS and Mac products. It's especially helpfulfor continuous integration.

[Features •Requirements •Usage•Continuous Integration•Reporters •Configuration •Contributing •Known Issues & Tips •License ]

xctool is drop-in replacement forxcodebuild test that adds a fewextra features:

• Faster, parallelized test runs.

  xctool can optionally run all of your test bundles in parallel,speeding up your test runs significantly. At Facebook, we've seen 2xand 3x speed ups by parallelizing our runs.

  Use the-parallelize option withrun-tests ortest to enable.SeeParallelizing Test Runs for more info.

• Structured output of test results.

  xctool captures all test results as structured JSON objects. Ifyou're building a continuous integration system, this means you don'thave to regex parsexcodebuild output anymore.

  Try one of theReporters to customize the output or getthe full event stream with the-reporter json-stream option.

• Human-friendly, ANSI-colored output.

  xcodebuild is incredibly verbose, printing the full compile commandand output for every source file. By default,xctool is only verboseif something goes wrong, making it much easier to identify where theproblems are.

  Example:

  

• Written in Objective-C.

  xctool is written in Objective-C. Mac OS X and iOS developers caneasily submit new features and fix any bugs they may encounter withoutlearning a new language. We very much welcome pull requests!

Note: Support for building projects with xctool is deprecated and willnot be updated to support future versions of Xcode. We suggest moving toxcodebuild (withxcpretty) forsimple needs, orxcbuild for moreinvolved requirements. xctool will continue to support testing (see above).

• Xcode 7 or higher
• You'll need Xcode's Command Line Tools installed. From Xcode, installviaXcode → Preferences → Downloads.

xctool can be installed from homebrew via

brew install xctool

or can be downloaded and run via the xctool.sh command.

xctool's commands and options are mostly a superset of xcodebuild's. Inmost cases, you can just swapxcodebuild withxctool and things willrun as expected but with more attractive output.

You can always get help and a full list of options with:

path/to/xctool.sh -help

xctool has arun-tests action which knows how to run thetests in your scheme. You can optionally limit what tests are runor change the SDK they're run against.

To run all tests in your scheme, you would use:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests

To run just the tests in a specific target, use the-only option:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -only SomeTestTarget

You can go further and just run a specific test class:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -only SomeTestTarget:SomeTestClass

Or, even further and run just a single test method:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -only SomeTestTarget:SomeTestClass/testSomeMethod

You can also specify prefix matching for classes or test methods:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -only SomeTestTarget:SomeTestClassPrefix*,SomeTestClass/testSomeMethodPrefix*

Alternatively, you can omit a specific item by prefix matching for classes or test methods:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -omit SomeTestTarget:SomeTestClass/testSomeMethodPrefix*

You can also run tests against a different SDK:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -test-sdk iphonesimulator5.1

Optionally you can specify-testTimeout when running tests. When an individualtest hits this timeout, it is considered a failure rather than waiting indefinitely.This can prevent your test run from deadlocking forever due to misbehaving tests.

By default application tests will wait at most 30 seconds for the simulatorto launch. If you need to change this timeout, use the-launch-timeout option.

Before running tests you need to build them. You can usexcodebuild,xcbuild orBuck to do that.

For example:

xcodebuild \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  build-for-testing

If you are using Xcode 7 for building you can continue using xctool to build tests usingbuild-tests or just usetest actions to run tests.

For example:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  build-tests

You can optionally just build a single test target with the-only option:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  build-tests -only SomeTestTarget

xctool can optionally run unit tests in parallel, making better use ofotherwise idle CPU cores. At Facebook, we've seen 2x and 3x gains byparallelizing our test runs.

To allow test bundles to run concurrently, use the-parallelizeoption:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -parallelize

The above gives you parallelism, but you're bounded by your slowest testbundle. For example, if you had two test bundles ('A' and 'B'), but 'B'took 10 times as long to run because it contained 10 times as manytests, then the above parallelism won't help much.

You can get further gains by breaking your test execution into bucketsusing the-logicTestBucketSize option:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  run-tests -parallelize -logicTestBucketSize 20

The above will break your test execution into buckets of20 testcases each, and those bundles will be run concurrently. If some of yourtest bundles are much larger than others, this will help even things outand speed up the overall test run.

Note: Support for building projects with xctool is deprecated and isn'tsupported in Xcode 8 and later. We suggest moving toxcodebuild (withxcpretty) forsimple needs, orxcbuild for moreinvolved requirements. Alternatively, you can useBuck.

Building products withxctool is the same as building them withxcodebuild.

If you use workspaces and schemes:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  build

If you use projects and schemes:

path/to/xctool.sh \  -project YourProject.xcodeproj \  -scheme YourScheme \  build

All of the common options like-configuration,-sdk,-arch workjust as they do withxcodebuild.

NOTE:xctool doesn't support directly building targets using-target; you must use schemes.

xctool is an excellent choice for running your tests under a continuousintegration server such asTravis CI orJenkins.To run tests within a continuous integration environment,you must createShared Schemes for your application target and ensure that all dependencies (such as CocoaPods) are added explicitly to the Scheme. To do so:

1. Open up theManage Schemes sheet by selecting theProduct menu >Schemes >Manage Schemes...
2. Locate your application target in the list. Ensure that theShared checkbox in far right hand column of the sheet is checked.
3. If your application or test targets include cross-project dependencies such as CocoaPods, then you will need to ensure that they have beenconfigured as explicit dependencies. To do so:
   1. Highlight your application target and hit theEdit... button to open the Scheme editing sheet.
   2. Click theBuild tab in the left-hand panel of the Scheme editor.
   3. Click the+ button and add each dependency to the project. CocoaPods will appear as a static library namedPods.
   4. Drag the dependency above your application target so that it is built first.

You will now have a new file in thexcshareddata/xcschemes directory underneath your Xcode project. This is theshared Scheme that you just configured. Check this file into your repository and xctool will be able to find and executeyour tests on the next CI build.

Travis CI is a very popular continuousintegration system offered for free to Open Source projects. Itintegrates well with Github, and it now usesxctool as the defaultbuild and test tool for Objective-C projects. Once you have set up yourshared Scheme for use with xctool, you will need to configure a.travis.yml file.

If you're using workspaces, your.travis.yml might be:

language:objective-cxcode_workspace:path/to/YourApp.xcworkspacexcode_scheme:YourApp

If you're using projects, your.travis.yml might be:

language:objective-cxcode_project:path/to/YourApp.xcodeprojxcode_scheme:YourApp

For more flexibility, you can also control how Travis installs andinvokes xctool:

language:objective-cbefore_install:    -brew update    -brew install xctoolscript:xctool -workspace MyApp.xcworkspace -scheme MyApp test

You can learn more about the Travis CI environment for iOS and OS Xapplication by referring to theAbout OS X Travis CIEnvironmentdocument and find in-depth documentation for configuring your project byconsulting theGettingStarted page.

xctool has reporters that output build and test results in differentformats. If you do not specify any reporters yourself, xctool usesthepretty anduser-notifications reporters by default. xctool alsohas these special rules:

• Overwrite is disabled on thepretty reporter when xctool does notdetect a TTY. This can be overridden by settingXCTOOL_FORCE_TTY inthe environment.
• Theuser-notifications reporter will not be usedif xctool detects that the build is being run by Travis CI, CircleCI, TeamCity,or Jenkins (i.e.TRAVIS=true,CIRCLECI=true,TEAMCITY_VERSION, orJENKINS_URL in the environment).

You can choose your own reporters with the-reporter option:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  -reporter plain \  build

By default, reporters output to standard out, but you can also directthe output to a file by adding:OUTPUT_PATH after the reporter name:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  -reporter plain:/path/to/plain-output.txt \  build

You can use as many reporters as you like; just use the-reporteroption multiple times.

• pretty: a text-based reporter that uses ANSI colors and unicodesymbols for pretty output (the default).
• plain: likepretty, but with no colors or unicode.
• phabricator: outputs a JSON array of build/test results which canbe fed into thePhabricator code-review tool.
• junit: produces a JUnit/xUnit compatible XML file with testresults.
• json-stream: a stream of build/test events as JSON dictionaries,one per line(exampleoutput).
• json-compilation-database: outputs aJSON Compilation Database of build events which can be used byClang Tooling based tools, e.g.OCLint.
• user-notifications: sends notification to Notification Center when action is completed(example notifications).
• teamcity: sends service messages toTeamCity Continuous Integration Server

You can also implement your own reporters using whatever language youlike. Reporters in xctool are separate executables that read JSONobjects from STDIN and write formatted results to STDOUT.

You can invoke reporters by passing their full path via the-reporteroption:

path/to/xctool.sh \  -workspace YourWorkspace.xcworkspace \  -scheme YourScheme \  -reporter /path/to/your/reporter \test

For example, here's a simple reporter in Python that outputs aperiodfor every passing test and anexclamation mark for every failing test:

#!/usr/bin/pythonimportfileinputimportjsonimportsysforlineinfileinput.input():obj=json.loads(line)ifobj['event']=='end-test':ifobj['succeeded']:sys.stdout.write('.')else:sys.stdout.write('!')sys.stdout.write('\n')

If you're writing a reporter in Objective-C, you'll find theReporter class helpful - seeReporter.h.

If you routinely need to pass many arguments toxctool on thecommand-line, you can use an.xctool-args file to speed up your workflow.Ifxctool finds an.xctool-args file in the current directory, itwill automatically pre-populate its arguments from there.

The format is just a JSON array of arguments:

["-workspace","YourWorkspace.xcworkspace","-scheme","YourScheme","-configuration","Debug","-sdk","iphonesimulator","-arch","i386"]

Any extra arguments you pass on the command-line will take precedenceover those in the.xctool-args file.

Bug fixes, improvements, and especially newReporterimplementations are welcome. Before submitting apullrequest, pleasebe sure to sign theFacebookContributor LicenseAgreement. We can'taccept pull requests unless it's been signed.

1. Fork.
2. Make a feature branch:git checkout -b my-feature
3. Make your feature. Keep things tidy so you have one commit per self-contained change (squashing can help).
4. Push your branch to your fork:git push -u origin my-feature
5. Open GitHub, under "Your recently pushed branches", clickPullRequest formy-feature.

Be sure to use a separate feature branch and pull request for everyself-contained feature. If you need to make changes from feedback, makethe changes in place rather than layering on commits (use interactiverebase to edit your earlier commits). Then usegit push --forceorigin my-feature to update your pull request.

Mostly the same, but use branches in the main xctool repo if you prefer.It's a nice way to keep things together.

1. Make a feature branch:git checkout -b myusername/my-feature
2. Push your branch:git push -u origin myusername/my-feature
3. Open GitHub tofacebook/xctool,under "Your recently pushed branches", clickPull Request formyusername/my-feature.

• Use shared schemes and disable the Autocreate Schemes feature.

  Xcode has two kinds of schemes: shared, and user. User schemes arethe default, and they're stored under a folder calledUSERNAME.xcuserdatad,which most people correctly add to their.gitignore.

  Use shared schemes instead, and commit them to your repo. This wayeveryone on your team (and your build server) are working from thesame information, and are building in the same way.

  

• Make sure simulators run in a GUI context.

  If you are runningxctool in continuous integration, the user accountcallingxctoolmust have an active GUI context.If not, the simulator will fail to start with cryptic warnings like:

  Tried to install the test host app 'com.myapp.test' but failed.Preparing test environment failed.-[TEST_BUNDLE FAILED_TO_START] There was a problem starting the test bundle: Simulator 'iPhone 6' was not prepared: Failed for unknown reason.Test did not run: Simulator 'iPhone 6' was not prepared: Failed for unknown reason.2015-01-21 12:02:19.296 xcodebuild[35135:875297]  iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.Testing failed:Test target MyProjectTests encountered an error (Timed out waiting 120 seconds for simulator to boot, current state is 1.

  Note that thesame holds true withxcodebuild...this is notxctool specific.

  For more information, seethis post by Jason Jarrett.

Copyright 2014-present Facebook

Licensed under the Apache License, Version 2.0 (the "License"); you maynot use this work except in compliance with the License. You may obtaina copy of the License in the LICENSE file, or at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.