gcloud beta firebase test ios run - invoke a test in Firebase Test Lab for iOS and view test results

gcloud beta firebase test ios run[ARGSPEC][--device=DIMENSION=VALUE,[…]][--test=XCTEST_ZIP][--timeout=TIMEOUT][--type=TYPE][--xcode-version=XCODE_VERSION][--xctestrun-file=XCTESTRUN_FILE][--additional-ipas=IPA,[IPA,…]][--app=APP][--async][--client-details=[KEY=VALUE,…]][--directories-to-pull=[DIR_TO_PULL,…]][--num-flaky-test-attempts=int][--other-files=DEVICE_PATH=FILE_PATH,[…]][--record-video][--results-bucket=RESULTS_BUCKET][--results-dir=RESULTS_DIR][--results-history-name=RESULTS_HISTORY_NAME][--robo-script=ROBO_SCRIPT][--scenario-numbers=int,[int,…]][--test-special-entitlements][--filter=EXPRESSION][--limit=LIMIT][--page-size=PAGE_SIZE][--sort-by=[FIELD,…]][GCLOUD_WIDE_FLAG …]

Two types of iOS tests are currently supported:

• xctest: corresponds to the XCTest and XCUITest frameworks. OtheriOS testing frameworks which are built upon XCTest and XCUITest should alsowork. The XCTEST_ZIP test package is a zip file built using Apple's Xcode andsupporting tools. For a detailed description of the process to create yourXCTEST_ZIP file, seehttps://firebase.google.com/docs/test-lab/ios/command-line.
• game-loop: launches the game app through a custom URL scheme toexecute a "demo mode" built into the game app that simulates actions of a realplayer. This test type can include multiple game loops (also called "scenarios")indicated by positive numbers.

The type of test to run can be specified with the--type flag,which defaults toxctest.

All arguments forgcloud beta firebase test ios run may bespecified on the command line and/or within an argument file. Run$gcloud topic arg-filesfor more information about argument files.

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--client-details=matrixLabel="Example matrix label"

To invoke an XCTest lasting up to five minutes against the default deviceenvironment, run:

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--timeout=5m

To invoke an XCTest against an iPad 5 running iOS 11.2, run:

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--device=model=ipad5,version=11.2

To run your tests against multiple iOS devices simultaneously, specify the--device flag more than once:

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--device=model=iphone7--device=model=ipadmini4,version=11.2--device=model=iphonese

To run your XCTest using a specific version of Xcode, say 9.4.1, run:

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--xcode-version=9.4.1

To help you identify and locate your test matrix in the Firebase console, run:

gcloudbetafirebasetestiosrun--test=XCTEST_ZIP--client-details=matrixLabel="Example matrix label"

To run an iOS game loop, specify the--type and--appflags:

gcloudbetafirebasetestiosrun--type=game-loop--app=app.ipa

To run an iOS game loop with specific scenario(s), use the--scenario-numbers flag:

gcloudbetafirebasetestiosrun--type=game-loop--app=app.ipa--scenario-numbers=1,2,3

To run a test that pushes a local file onto the device before testing, use the--other-files flag:

gcloudbetafirebasetestiosrun--type=game-loop--app=app.ipa--scenario-numbers=1--other-files=/private/var/mobile/Media/file.txt=/path/to/file.txt

All test arguments for a given test may alternatively be stored in an argumentgroup within a YAML-formatted argument file. TheARG_FILE may contain one or more named argument groups,and argument groups may be combined using theinclude: attribute(Run$gcloud topicarg-files for more information). The ARG_FILE can easily be shared withcolleagues or placed under source control to ensure consistent test executions.

To run a test using arguments loaded from an ARG_FILE namedexcelsior_app_args, which contains an argument group namedios-args:, use the following syntax:

gcloudbetafirebasetestiosrunpath/to/excelsior_app_args:ios-args

[ARGSPEC]

An ARG_FILE:ARG_GROUP_NAME pair, where ARG_FILE is the path to a file containinggroups of test arguments in yaml format, and ARG_GROUP_NAME is the particularyaml object holding a group of arg:value pairs to use. Run$gcloud topic arg-filesfor more information and examples.

--device=DIMENSION=VALUE,[…]

A list ofDIMENSION=VALUE pairs whichspecify a target device to test against. This flag may be repeated to specifymultiple devices. The device dimensions are:model,version,locale, andorientation. If anydimensions are omitted, they will use a default value. The default value, andall possible values, for each dimension can be found with thelist command for that dimension, such as$gcloudbeta firebase test ios models list. Omitting this flag entirely willrun tests against a single device using defaults for every dimension.

Examples:

--devicemodel=iphone8plus--deviceversion=11.2--devicemodel=ipadmini4,version=11.2,locale=zh_CN,orientation=landscape

--test=XCTEST_ZIP

The path to the test package (a zip file containing the iOS app and XCTestfiles). The given path may be in the local filesystem or in Google Cloud Storageusing a URL beginning withgs://. Note: any .xctestrun file in thiszip file will be ignored if--xctestrun-file is specified.

--timeout=TIMEOUT

The max time this test execution can run before it is cancelled (default: 15m).It does not include any time necessary to prepare and clean up the targetdevice. The maximum possible testing time is 45m on physical devices and 60m onvirtual devices. TheTIMEOUT units can be h, m, or s. Ifno unit is given, seconds are assumed. Examples:

• --timeout 1h is 1 hour
• --timeout 5m is 5 minutes
• --timeout 200s is 200 seconds
• --timeout 100 is 100 seconds

--type=TYPE

The type of iOS test to run.TYPE must be one of:xctest,game-loop,robo.

--xcode-version=XCODE_VERSION

The version of Xcode that should be used to run an XCTest. Defaults to thelatest Xcode version supported in Firebase Test Lab. This Xcode version must besupported by all iOS versions selected in the test matrix. The list of Xcodeversions supported by each version of iOS can be viewed by running$gcloud betafirebase test ios versions list.

--xctestrun-file=XCTESTRUN_FILE

The path to an .xctestrun file that will override any .xctestrun file containedin the--test package. Because the .xctestrun file containsenvironment variables along with test methods to run and/or ignore, this can beuseful for customizing or sharding test suites. The given path may be in thelocal filesystem or in Google Cloud Storage using a URL beginning withgs://.

--additional-ipas=IPA,[IPA,…]

List of up to 100 additional IPAs to install, in addition to the one beingdirectly tested. The path may be in the local filesystem or in Google CloudStorage using gs:// notation.

--app=APP

The path to the application archive (.ipa file) for game-loop testing. The pathmay be in the local filesystem or in Google Cloud Storage using gs:// notation.This flag is only valid when--type isgame-loop orrobo.

--async

Invoke a test asynchronously without waiting for test results.

--client-details=[KEY=VALUE,…]

Comma-separated, KEY=VALUE map of additional details to attach to the testmatrix. Arbitrary KEY=VALUE pairs may be attached to a test matrix to provideadditional context about the tests being run. When consuming the test results,such as in Cloud Functions or a CI system, these details can add additionalcontext such as a link to the corresponding pull request.

Example:

--client-details=buildNumber=1234,pullRequest=https://example.com/link/to/pull-request

To help you identify and locate your test matrix in the Firebase console, usethe matrixLabel key.

Example:

--client-details=matrixLabel="Example matrix label"

--directories-to-pull=[DIR_TO_PULL,…]

A list of paths that will be copied from the device's storage to the designatedresults bucket after the test is complete. These must be absolute paths under/private/var/mobile/Media or/Documents of the appunder test. If the path is under an app's/Documents, it must beprefixed with the app's bundle id and a colon.

Example:

--directories-to-pull=com.my.app:/Documents/output,/private/var/mobile/Media/output

--num-flaky-test-attempts=int

Specifies the number of times a test execution should be reattempted if one ormore of its test cases fail for any reason. An execution that initially failsbut succeeds on any reattempt is reported as FLAKY.

The maximum number of reruns allowed is 10. (Default: 0, which implies noreruns.) All additional attempts are executed in parallel.

--other-files=DEVICE_PATH=FILE_PATH,[…]

A list of device-path=file-path pairs that specify the paths of the test deviceand the files you want pushed to the device prior to testing.

Device paths should either be under the Media shared folder (e.g. prefixed with/private/var/mobile/Media) or within the documents directory of the filesystemof an app under test (e.g. /Documents). Device paths to app filesystems shouldbe prefixed by the bundle ID and a colon. Source file paths may be in the localfilesystem or in Google Cloud Storage (gs://…).

Examples:

--other-filescom.my.app:/Documents/file.txt=local/file.txt,/private/var/mobile/Media/file.jpg=gs://bucket/file.jpg

--record-video

Enable video recording during the test. Enabled by default, use--no-record-video to disable.

--results-bucket=RESULTS_BUCKET

The name of a Google Cloud Storage bucket where raw test results will be stored(default: "test-lab-<random-UUID>"). Note that the bucket must be owned bya billing-enabled project, and that using a non-default bucket will result inbilling charges for the storage used.

--results-dir=RESULTS_DIR

The name of aunique Google Cloud Storage object within the resultsbucket where raw test results will be stored (default: a timestamp with a randomsuffix). Caution: if specified, this argumentmust be unique foreach test matrix you create, otherwise results from multiple test matrices willbe overwritten or intermingled.

--results-history-name=RESULTS_HISTORY_NAME

The history name for your test results (an arbitrary string label; default: thebundle ID for the iOS application). All tests which use the same history namewill have their results grouped together in the Firebase console in atime-ordered test history list.

--robo-script=ROBO_SCRIPT

The path to a Robo Script JSON file. The path may be in the local filesystem orin Google Cloud Storage using gs:// notation. You can guide the Robo test toperform specific actions by specifying a Robo Script with this argument. Learnmore athttps://firebase.google.com/docs/test-lab/robo-ux-test#scripting.This flag is only valid when--type=robo is also set.

--scenario-numbers=int,[int,…]

A list of game-loop scenario numbers which will be run as part of the test(default: scenario 1). A maximum of 1024 scenarios may be specified in one testmatrix, but the maximum number may also be limited by the overall test--timeout setting. This flag is only valid when--type=game-loop is also set.

--test-special-entitlements

Enables testing special app entitlements. Re-signs an app having specialentitlements with a new application-identifier. This currently supports testingPush Notifications (aps-environment) entitlement for up to one app in a project.

Note: Because this changes the app's identifier, make sure none of the resourcesin your zip file contain direct references to the test app's bundle id.

--filter=EXPRESSION

Apply a Boolean filterEXPRESSION to each resource itemto be listed. If the expression evaluatesTrue, then that item islisted. For more details and examples of filter expressions, run $gcloud topic filters. This flaginteracts with other flags that are applied in this order:--flatten,--sort-by,--filter,--limit.

--limit=LIMIT

Maximum number of resources to list. The default isunlimited. Thisflag interacts with other flags that are applied in this order:--flatten,--sort-by,--filter,--limit.

--page-size=PAGE_SIZE

Some services group resource list output into pages. This flag specifies themaximum number of resources per page. The default is determined by the serviceif it supports paging, otherwise it isunlimited (no paging).Paging may be applied before or after--filter and--limit depending on the service.

--sort-by=[FIELD,…]

Comma-separated list of resource field key names to sort by. The default orderis ascending. Prefix a field with ``~´´ for descending order on thatfield. This flag interacts with other flags that are applied in this order:--flatten,--sort-by,--filter,--limit.

Run$gcloud help for details.