In order to use the App Testing agent,you will need to opt-in to the feature from theFirebase console.For information, seeAccessing the App Testing agent preview.

The App Testing agent is a test case generation, management, and execution agentpowered by Gemini inFirebase. You define test goals in natural language,and the agent uses AI to understand and navigate your app, simulate userinteractions, and provide detailed test results.

How App Testing agent uses your data

App Testing agent is provided byGemini inFirebase and is governed under thesame terms. SeeHow Gemini inFirebase uses yourdata for moreinformation about how Gemini inFirebase uses your data.

Before you begin

If you haven't already,register your app withFirebase.

If you aren't using any other Firebase products, you only have to create aproject and register your app. You don't need to add any SDKs to your app.However, if you decide to use additional products in the future, be sure tocomplete all of the steps inAdd Firebase using theFirebase console.

When you're ready to distribute a pre-release version of your app to testers,build your APK or AAB, using your normal process, and upload it to AppDistribution in theFirebase console. You mustsign the APK with your debug key or app signing key.

Create a test case

To run AI-guided tests, the App Testing agent uses your natural language testcases to execute tests against your app.

Each test case is broken down into steps, which will be executed in sequence.Steps allow you to break up your test case into phases, each with their ownsuccess criteria. The agent may take many actions during any one step.

To create a test case, open theApp Distribution page of theFirebase console and takethe following steps:

1. On theTest Cases tab, clickNew test case. If you don't want tocreate your own test case, you can modify or use the providedexample testcase.
2. In theAdd test case dialog, give the test case a name. This is used toidentify the test, but is ignored by the agent.
3. (Optional) Select aPrerequisite test case that contains set-up steps torun before the main test. If the prerequisite test fails, the entire testwill be marked as a failure. Steps and outcomes from the prerequisite andmain tests will be shown together in the test results.
4. Consider breaking your test into multiple steps, by clicking theAddanother step button.
5. Give each step aGoal that describes what the App Testing agent shoulddo during that step.
6. (Optional) Add aHint to provide additional information to help the AppTesting agent understand and navigate your app during that step.
7. (Optional) AddSuccess Criteria to help the App Testing agent determinewhen the step has been successfully completed.
8. ClickSave once you're done customizing your test.

Example test case

The following is an example of how to create a test case using the App Testingagent:

Run a test

The App Testing agent lets you runAI-guided tests in the console byclicking theRun tests button from either the Releases or Test Cases page.This opens the App Testing agent customization screen, where you can choose oneor more of your existing test cases for the agent to execute. You can alsochoose the devices you want to test against, and whether to provide any logincredentials.

You can also choose to run aRandom crawl test by changing the test type.Random crawl tests use theAutomatedTester feature.

You can view the results of your tests from theReleases page in theApp Testing agent tab of a release. TheView details button will openthe Test Results dialog and show you any issues, screenshots of the app, and theactions that Gemini took during the test.

Automatically test your builds

To automatically run App Testing agent tests on your new builds, for examplefrom CI/CD pipelines, you can distribute your builds to the agent usingApp Distribution's Gradle or fastlane plugins, or the Firebase CLI.

testPasswordFile="/path/to/testPassword.txt"

Import & export test cases with YAML files

Importing test cases from YAML files is useful when you want to manage testcases outside of theFirebase console. It can also be useful to export testcases to move them between projects. You can use an LLM to refine existingtest cases or create new test cases. You can import and export test cases fromthe Test Cases page in theFirebase console or programmatically using theFirebase CLI. An example:

-displayName:Setupid:setupsteps:-goal:Log inhint:Any username and password will work-displayName:Smoke testid:smoke_testprerequisiteTestCaseId:setupsteps:-goal:Go through the onboarding flowhint:Tap the next button until you reach the home screensuccessCriteria:The main app home page is visible-goal:Open the settings pagehint:The settings button is in the top right cornersuccessCriteria:The settings page is visible

Debug your test results

If your test results are different from what you expected, you can debug yourtest using theShow agent view toggle underView details on the TestResults page. The agent view shows you the on screen elements that the AppTesting agent was able to detect when using the app's accessibility information.If you would like to take a closer look at what the agent saw, you can downloadthat information from the action overflow menu.

You can also use theView artifacts button on the Test Results page to lookat all of the videos, logs, and other Cloud artifacts for your test results.

Known issues and limitations

The App Testing agent preview has some known limitations:

• Because the App Testing agent uses generative AI to test your app, it willsometimes take different actions while still following the sameinstructions.
• The App Testing agent only supports the following actions: tap, enter text,swipe up/down/left/right, long press, drag-and-drop, back, and wait.
• The App Testing agent has trouble executing tests containing only a singlestep that takes many actions to accomplish. It performs better when complex tasks are broken up into multiple shorter steps.
• The App Testing agent sometimes won't scroll to expose other elements offscreen. This happens more often when there is no visual indication ofscrollability. As a workaround, the "hints" field can be used to suggestscrolling.
• The App Testing agent sometimes has trouble counting, for example performingan action a specific number of times.
• The App Testing agent cannot navigate your app ifFLAG_SECURE is enabled. Instead of screenshots of your app, it will only see a blankscreen.

Testing quotas

During the preview, the AI-guided tests will be offered at no cost within aquota limit. The default quota limit is 200 tests per month, per Firebaseproject.

Note that if you choose to run multiple test cases, or run the same test case,on multiple devices, this counts as multiple tests. For example, if you run 2test cases on 2 devices, this counts as a total of 4 tests.

To increase your quota above the default limit, contactFirebaseSupport with your use case.