Movatterモバイル変換

[0]ホーム
URL:

Skip to main content

Advertisement

Springer Nature Link
Log in

VisiDroid: An Approach for Generating Test Scripts from Task Descriptions for Mobile Testing

  • Conference paper
  • First Online:

Abstract

Testing user interface and system-level functionality of a mobile app is crucial for ensuring its quality. However, it is becoming increasingly costly due to the complexity of modern applications and the diverse range of devices. Recent approaches have focused on exploring entire applications to test and detect defects in mobile apps. Additionally, they do not consider the ability to guide and restrict large language models (LLMs) based on user-defined rules. This paper introduces VisiDroid, an approach to generating scripts for mobile testing from task goals or natural language descriptions by leveraging the capabilities of LLMs. We evaluate the approach using an open-source dataset consisting of 131 tasks on 11 mobile apps. The results show that VisiDroid can accurately generate actions and achieves a task completion rate of 72.2%, outperforming the state-of-the-art approach. It also successfully generates valid test scripts with an 80.05% success rate overall.

This is a preview of subscription content,log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 6634
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 8293
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

References

  1. Fdroid: Free and open source android app repository.https://f-droid.org/en/

  2. Amalfitano, D., Fasolino, A.R., Tramontana, P., Ta, B.D., Memon, A.M.: Mobiguitar: automated model-based testing of mobile apps. IEEE Softw.32(5), 53–59 (2014)

    Article  Google Scholar 

  3. Google (2022).https://developer.android.com/studio/test/other-testing-tools/monkey

  4. Hao, W., Wang, H., Liu, J., Li, Y.: Droidbot-gpt: Gpt-powered ui automation for android.arXiv:2304.07061v5 (2024)

  5. Hao, W., et al.: Empowering llm to use smartphone for intelligent task automation.arXiv:2308.15272 (2023)

  6. Juyeon, Y., Feldt, R., Yoo, S.: Autonomous large language model agents enabling intent-driven mobile gui testing.arXiv:2311.08649v1 (2023)

  7. Li, Y., Yang, Z., Guo, Y., Chen, X.: Humanoid: a deep learning-based approach to automated black-box android app testing. In: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 1070–1073. IEEE (2019)

    Google Scholar 

  8. Lin, J.W., Malek, S.: Gui test transfer from web to android. In: 2022 IEEE Conference on Software Testing, Verification and Validation, pp. 1–11. IEEE (2022)

    Google Scholar 

  9. Liu, Z., et al.: Chatting with gpt-3 for zero-shot human-like mobile automated gui testing. arXiv preprintarXiv:2305.09434 (2023)

  10. Mario, L.-V., Bernal-Cardenas, C., Moran, K., Poshyvany, D.: How do developers test android applications. In: 2017 IEEE International Conference on Software Maintenance and Evolution (2017)

    Google Scholar 

  11. OpenAI: Openai vision api (2024).https://platform.openai.com/docs/guides/vision

  12. Qin, X., Zhong, H., Wang, X.: Testmig: migrating gui test cases from ios to android. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 284–295 (2019)

    Google Scholar 

  13. Shengcheng, Y., Fang, C., Ling, Y., Wu, C., Chen, Z.: Llm for test script generation and migration: Challenges, capabilities, and opportunities.arXiv:2309.13574 (2023)

  14. Shunyu, Y., et al.: Tree of thoughts: Deliberate problem solving with large language models.arXiv:2305.10601 (2023)

  15. Taori, R., Gulrajani, I., Zhang, T., et al.: Stanford alpaca: An instruction-following llama model. GitHub repository (2023)

    Google Scholar 

  16. Wang, L., et al.: A survey on large language model based autonomous agents. Front. Comp. Sci.18(6), 1–26 (2024)

    Google Scholar 

  17. Yu, S., Fang, C., Yun, Y., Feng, Y.: Layout and image recognition driving cross-platform automated mobile testing. In: IEEE/ACM 43rd International Conference on Software Engineering, pp. 1561–1571. IEEE (2021)

    Google Scholar 

  18. Zhe, L., et al.: Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions.arXiv:2310.15780v1 (2023)

Download references

Acknowledgements

This research is partially supported by research funding from the Faculty of Information Technology, University of Science, VNU-HCM, Vietnam.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam

    Hai Phung, Hao Pham & Vu Nguyen

  2. Vietnam National Univerisity, Ho Chi Minh City, Vietnam

    Hai Phung, Hao Pham & Vu Nguyen

  3. Katalon Inc., Atlanta, Georgia

    Vu Nguyen

  4. University of Texas at Dallas, Texas, USA

    Tien Nguyen

Authors
  1. Hai Phung

    You can also search for this author inPubMed Google Scholar

  2. Hao Pham

    You can also search for this author inPubMed Google Scholar

  3. Tien Nguyen

    You can also search for this author inPubMed Google Scholar

  4. Vu Nguyen

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence toVu Nguyen.

Editor information

Editors and Affiliations

  1. Kyoto University, Kyoto, Japan

    Rafik Hadfi

  2. Lincoln University, Christchurch, New Zealand

    Patricia Anthony

  3. RIKEN Center for Integrative Medical Sciences, Yokohama, Japan

    Alok Sharma

  4. Kyoto University, Kyoto, Japan

    Takayuki Ito

  5. University of Tasmania, Tasmania, TAS, Australia

    Quan Bai

Rights and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Phung, H., Pham, H., Nguyen, T., Nguyen, V. (2025). VisiDroid: An Approach for Generating Test Scripts from Task Descriptions for Mobile Testing. In: Hadfi, R., Anthony, P., Sharma, A., Ito, T., Bai, Q. (eds) PRICAI 2024: Trends in Artificial Intelligence. PRICAI 2024. Lecture Notes in Computer Science(), vol 15285. Springer, Singapore. https://doi.org/10.1007/978-981-96-0128-8_6

Download citation

Publish with us

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 6634
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 8293
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide -see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

[8]ページ先頭
©2009-2025 Movatter.jp