- Notifications
You must be signed in to change notification settings - Fork5
Deep-GUI is a tool for generating intelligent inputs to test UI-based applications, such as Android or web applications.
License
Feri73/deep-gui
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Deep-GUI is a tool for generating intelligent inputs to test UI-based applications, such as Android or web applications.This document provides instruction on how to train a Deep-GUI model, how to use it in order to automaticallyinteract with an application, and how to quantify the quality of the generated inputs in terms of line coverage.
System requirements:
- Ubuntu 18.04
- Python 3.6.8
- Java 8
Before getting started:
- Install Android SDK
- Download and extractAndroid SDK commandline tool in
~/android-sdk
. Locatesdkmanager
in~/android-sdk/cmdline-tools/bin
- Install build-tools 30.0.3 using
sdkmanager
- Install system image
system-images;android-10;google_apis;x86
usingsdkmanager
- Download and extractAndroid SDK commandline tool in
- Install Mozilla Firefox for testing web applications
- installgeckodriver v0.27.0 and add it to
PATH
- installgeckodriver v0.27.0 and add it to
- Clone this repository in
/home/$USER
- Install Packages in requirements.txt
Note: it is important to install the requirements in the exact paths mentioned above.
deep-gui │└───apks: a sample set of apk files with their respective emma file.│└───configs: configuration templates for different tasks│ └───models: pre-trained deep-gui models│ └───scripts: bash scripts used by different parts of the tool│ └───src: python source files
You can use this software to perform these tasks:
In order to train deep-gui, you need to first collect data.
Create emulator template
cd scripts./create_tester_ref./clone_avd.sh tester_ref collector_ref
Set the configurations:
In
configs/collect-configs.yaml
, set these mandatory values:collectors: The first number is the number of parallel data collection agentsdata_file_dir: The directory where the data is stored in (change this dircetory to collect both training and validation data)logs_dir: The directory where the tensorboard logs are written tocollectors_apks_path: The directory containing training/validation apkscollector_configs.version_start: If you need to append to existing data, set this number accordnigly
Copy
configs/collect-configs.yaml
tosrc/configs.yaml
Run the code:
cd src; python main.py
You can also monitor the progress:
cd <logs_dir> # same as logs_dir used in configs.yaml filetensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to
localhost:6006
to see tensorboard logs.
To train using the collected data:
Set the configurations:
In
configs/train-configs.yaml
, set these mandatory values:data_file_dir: The directory containing the training datalearner_configs.save_dir: The directory where the trained models are stored. You need to create this directory manually.learner_configs.validation_dir: The directory containing the validation datacollector_configs.version_start: Set this to a large number
Copy
configs/train-configs.yaml
tosrc/configs.yaml
Run the code:
cd src; python main.py
Note: A pre-trained model is availabe inmodels
To run the experiments:
Create emulator template
cd scripts./create_tester_ref./clone_avd.sh tester_ref collector_ref
Set the configurations:
In
configs/monkey-test-configs.yaml
, set these mandatory values:testers: The first number is the number of parallel agents.weights_file.e10: The path to the model that is to be used (.hdf5)
Copy
configs/monkey-test-configs.yaml
tosrc/configs.yaml
Run the code:
- If you want to run monkey without deep-gui:
cd scripts ./run_all_monkies.sh <experiment-name> monkey 0 <num-agents> <num-rounds> <apk-dir> <experiment-dir>
- If you want to run deep-gui:
cd scripts ./run_all_monkies.sh <experiment-name> deep 1 <num-agents> 1 <apk-dir> <experiment-dir>
where:
<experiment-name>: An arbitrary name for the experiment<num-agents>: Number of parallel agents (must match the configs.yaml file)<apk-dir>: The directory containing test apks. Each apk named app.apk needs to have a emma file in the same directory named app.apk.em<experiment-dir>: The directory containing the experiment files
- If you want to run monkey without deep-gui:
After the experiment is completed, run this:
cd ../srcpython update_tb.py <experiment-dir> <experiment-name> <experiment-dir>/tb_otest_logs <apk-dir>
Look at the logs:
cd <experiment-dir>/tb_otest_logstensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to
localhost:6006
to see tensorboard logs.
To run the experiments:
Set the configurations:
In
configs/web-configs.yaml
, set these mandatory values:- If you want to run a random agent:
testers: Set to [<num-parallel-agents>, [0, 0, 0, 0, 0, 1], monkey] reward_predictor: [RandomRewardPredictor, random]
- If you want to use deep-gui:
testers: Set to [<num-parallel-agents>, [.7, .3], deep, c99s, e10] reward_predictor: [UNetRewardPredictor, unet]
logs_dir: The directory where the tensorboard logs are written toweights_file.e10: The path to the model that is to be used (.hdf5)browser_configs.apps: The list of websites to be explored
- If you want to run a random agent:
Copy
configs/web-configs.yaml
tosrc/configs.yaml
Run the code:
cd src; python main.py
You can also monitor the progress:
cd <logs_dir> # same as logs_dir used in configs.yaml filetensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to
localhost:6006
to see tensorboard logs.
To analyze the results:
Uncomment the appropriate section in
scripts/analysis.sh
Run the analysis:
cd scripts./run_all_analyses.sh <experiment-dir> <apk-dir> <num-agents>
Check the results:
cd <experiment-dir>/tb_otest_logs/analysistensorboard --logdir=. --reload_interval 1 --samples_per_plugin "images=0"
Connect to
localhost:6006
to see tensorboard logs.
If you use the code or data, please cite the following paper:
@INPROCEEDINGS{9678778, author={YazdaniBanafsheDaragh, Faraz and Malek, Sam},title={Deep GUI: Black-box GUI Input Generation with Deep Learning},booktitle={2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)},year={2021}, volume={}, number={}, pages={905-916}, doi={10.1109/ASE51524.2021.9678778}}
About
Deep-GUI is a tool for generating intelligent inputs to test UI-based applications, such as Android or web applications.
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.