Movatterモバイル変換


[0]ホーム

URL:


CN100472472C - GUI test driving method based on MSAA under Windows platform - Google Patents

GUI test driving method based on MSAA under Windows platform
Download PDF

Info

Publication number
CN100472472C
CN100472472CCNB2007100253749ACN200710025374ACN100472472CCN 100472472 CCN100472472 CCN 100472472CCN B2007100253749 ACNB2007100253749 ACN B2007100253749ACN 200710025374 ACN200710025374 ACN 200710025374ACN 100472472 CCN100472472 CCN 100472472C
Authority
CN
China
Prior art keywords
script
window
environment
msaa
test
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB2007100253749A
Other languages
Chinese (zh)
Other versions
CN101105768A (en
Inventor
吕建
马晓星
曹春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing UniversityfiledCriticalNanjing University
Priority to CNB2007100253749ApriorityCriticalpatent/CN100472472C/en
Publication of CN101105768ApublicationCriticalpatent/CN101105768A/en
Application grantedgrantedCritical
Publication of CN100472472CpublicationCriticalpatent/CN100472472C/en
Expired - Fee Relatedlegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Images

Landscapes

Abstract

Translated fromChinese

本发明公开了一种Windows平台下基于MSAA的GUI测试驱动方法。其步骤是:由脚本解释器读取脚本文件至内存中;再根据测试脚本进行环境感知,脚本解释器通过环境感知器执行脚本中指定的感知要求,寻找符合要求的操作点,环境感知器使用Windows提供的MSAA接口,并按照脚本中指定条件搜索当前环境中的对象的IAccessible接口,当找到满足条件的窗口时,向脚本返回窗口句柄;如发现符合需求的操作点,由脚本解释器控制动作执行器产生相应的动作,动作执行器把输入焦点设定在特定窗口对象上,然后产生鼠标或键盘输入。本发明使程序对于被操作对象有识别和判断能力,测试过程中减少了人工干预,提高了劳动生产率。

The invention discloses an MSAA-based GUI test driving method under the Windows platform. The steps are: the script interpreter reads the script file into the memory; then performs environment perception according to the test script, the script interpreter executes the perception requirements specified in the script through the environment sensor, and finds an operation point that meets the requirements, and the environment sensor uses The MSAA interface provided by Windows, and search the IAccessible interface of the object in the current environment according to the conditions specified in the script. When a window that meets the conditions is found, the window handle is returned to the script; if an operation point that meets the requirements is found, the action is controlled by the script interpreter The executor generates corresponding actions, the action executor sets the input focus on a specific window object, and then generates mouse or keyboard input. The invention enables the program to have the ability to identify and judge the operated object, reduces manual intervention in the test process, and improves labor productivity.

Description

GUI based on MSAA under the windows platform tests driving method
One, technical field
The present invention relates to the GUI test driving method under a kind of windows platform, used GUI operative technique especially, realized the gui program test drive system of a robotization based on MSAA.
Two, background technology
Along with the computer system man-machine interface is more and more graphical, the application of GUI (graphical user interface) becomes more and more widely.But exploitation GUI interactive program great challenge is the problem of how the GUI interface being tested.The GUI test request test driver of robotization can possess certain judgement, can judge the operation that selection should be carried out according to the current environment situation.
Traditional gui program is tested, or carries out actual operation by employing a large amount of test mans, or uses some elementary GUI operation tools, produces inlet flow blindly.Traditional test driving method or expend a large amount of human resources, otherwise anti-jamming capacity is very poor, and one has disturbance will cause entering incorrect state, still needing finally to cause the distribution personnel on duty.
An outstanding automatic GUI test Solution should be able to have the certain reaction ability for performance and the incident of GUI, can judge according to current state, produces corresponding input then.This scheme should require manual intervention as few as possible.
Three, summary of the invention
Technical matters to be solved by this invention provides a kind of GUI test drive system with sight reaction capacity, and it can reduce the manual intervention in the test process of the software with GUI, finishes test case automatically.
GUI test drive system based on MSAA of the present invention, it comprises test script interpreter, environment sensing device and actuator.Script interpreter is used for the test case script that interpreting user is write, and the perceptron that controls environment identification current environment and control action actuator produce input.The step that this program is carried out is as follows:
A) read test drives script: this is written as concrete test script to abstract test case by the tester according to concrete linguistic norm.Script interpreter reads script file in internal memory;
B) carry out environment sensing according to test script: script interpreter is sought satisfactory operating point by the perception requirement of appointment in the environment sensing device execution script;
C) carry out specific action:, produce corresponding action by script interpreter control action actuator if find to meet the operating point of demand.
At above-mentioned steps b) in, the MSAA interface that the environment sensing device uses Windows to provide, and, when finding the window that satisfies condition, return window handle to script according to the IAccessible interface of the object in the conditional search current environment of appointment in the script.
In step c), actuator is set in input focus on the specific window object, and the API that calls Windows then produces mouse or keyboard input.
When writing test script, can use following statement at least:
Statement 1 environment of observation
Find topwnd " title " |
Childwnd " name " " role " " class " |
Anywnd " name " " role " " class " |
Topwnd represents the window of top layer, application program of ordinary representation.As long as header parameter; Childwnd is illustrated under the current window environment, searches the IAccessible of qualified subwindow, three parameters; Anywnd so begin to search the IAccessible interface of subwindow from desktop, three parameters because whom bad definite parent window is
Statement 2 actions
Do kbinput " character string " |
(click|rclick|dbclick)?x?y
Wherein kbinput represents to produce the keyboard input, and " character string " is the string that will import.Following click represents to produce the left mouse button single-click operation, rclick represents to produce the right mouse button single-click operation, dbclick represents to produce the left mouse button double click operation, and last x y represents that this coordinate of coordinate of clicking operation is based on the upper left corner that a find orders the window that finds.The present invention carries out flow process by script interpreter control, obtains qualified environment position by the environment sensing device, produces corresponding input by actuator.Can carry out the operation of robotization to the program in the system of being loaded into by the test script of writing as required.
The present invention compared with prior art, its remarkable advantage is: program for operated object have identification and judgement.So in the process of test, just need manual intervention seldom, improved labour productivity greatly.
Four, description of drawings
Fig. 1 has shown system architecture of the present invention.
Fig. 2-1,2-2,2-3 have shown three groundwork flow processs respectively.
Five, embodiment
1, running environment:
(1) PC of operation Windows system, they are as the platform of program run.
(2) this program is installed and tests drive system on the PC device as GUI.
(3) tested software is loaded on this platform.
(4) the auxiliary test script of writing of the Inspect32 of installation Microsoft.
(5) write test script for test process.
System architecture as shown in Figure 1, it comprises script interpreter, environment sensing device, actuator, described script interpreter is explained the test script program of being write by test procedure person, interface object in running environment by the environment for use perceptron, when the object that satisfies condition occurs, the input that script interpreter again can the usage operation actuator produces keyboard or mouse.
2, scene:
Fig. 2-1 has shown the workflow of the script interpreter of native system.The test that interpretive routine at first reads in appointment drives script, begins to explain sentence by sentence script then.If current statement is the script sentence (promptly seek qualified window object, and return the handle of window object) of a perception environment, carry out corresponding judgment with regard to the transfer environment perceptron so.If one is carried out imperative statement (following usually after a series of environment sensing statements), that is carried out corresponding script with regard to the invoke script actuator and produces input action.
On the PC of mounted Windows and native system, load the application program that to test.Then, be the programming test script.Then the input boldface type partly moves under control desk
C: PCompiler script file name
Then, the script interpretive routine can be carried out test script, seeks qualified window object, produces corresponding operation then.
The flow process of seeking qualified window object is shown in Fig. 2-2.The window system of Windows is to be that root tissue becomes tree structure with the desktop.Seeking qualified window object is to begin search from desktop, obtain the IAccessible interface of current window, obtain the name (IAccessible::get_accName) of window by the method for this IAccessible interface, role (IAccessible::get_accRole), class information such as (WindowFromAccessibleObject and GetObjectClass), the information conforms Rule of judgment, then return the handle of current window, otherwise all the subwindow () IAccessible::get_accChild and the IEnumVARIANT interface of recursive search current window) judge, up to finding window or the exhaustive intact all windows that will look for.
The flow process of carrying out corresponding operating is shown in Fig. 2-3.What judge at first that this statement will carry out is keyboard input or mouse input.If the keyboard input is read input of character string, one by one described all key-press event of input of character string.If mouse input, read the type (left button is clicked, right-click, double-click) of mouse input, import corresponding mouse event then.
3, technical description:
(1)MSAA:
Microsoft Active Accessibility technology has been used in the identification of window elements.This technology exposes unified com interface by the window writing routine below windows platform.Make the programmer can discern and operate standard interface element on the Windows by this interface.
Here window has three important attribute: Name, Class, Role.Name represents the name of window, and Class represents the class name that window is registered in system, and Role represents that the developer gives the role of window.The parent window information that these three attributes and window have been arranged just can uniquely be determined a window.
(2) windows messaging mechanism
On produce action, use the message mechanism of Windows, under the desktop environment of Windows, produce keyboard or mouse input.This process is the API SendInput realization by Windows.

Claims (3)

CNB2007100253749A2007-07-262007-07-26 GUI test driving method based on MSAA under Windows platformExpired - Fee RelatedCN100472472C (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CNB2007100253749ACN100472472C (en)2007-07-262007-07-26 GUI test driving method based on MSAA under Windows platform

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CNB2007100253749ACN100472472C (en)2007-07-262007-07-26 GUI test driving method based on MSAA under Windows platform

Publications (2)

Publication NumberPublication Date
CN101105768A CN101105768A (en)2008-01-16
CN100472472Ctrue CN100472472C (en)2009-03-25

Family

ID=38999672

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CNB2007100253749AExpired - Fee RelatedCN100472472C (en)2007-07-262007-07-26 GUI test driving method based on MSAA under Windows platform

Country Status (1)

CountryLink
CN (1)CN100472472C (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101751329B (en)*2008-12-182012-01-25大唐移动通信设备有限公司Method and system for realizing automatic testing
JP5490036B2 (en)*2011-02-142014-05-14株式会社Nttドコモ Test equipment
CN109542440A (en)*2018-11-292019-03-29金蝶软件(中国)有限公司To the operating method and device of window, computer installation and readable storage medium storing program for executing
CN110287088B (en)*2019-01-142021-08-31南京大学 An automated testing method based on dynamic UI model

Citations (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US5943048A (en)*1997-11-191999-08-24Microsoft CorporationMethod and apparatus for testing a graphic control area
CN1851667A (en)*2006-02-152006-10-25华为技术有限公司Graphic user interface test method and system
CN1866219A (en)*2005-12-092006-11-22华为技术有限公司System and method for testing measured object based on automatic test script

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US5943048A (en)*1997-11-191999-08-24Microsoft CorporationMethod and apparatus for testing a graphic control area
CN1866219A (en)*2005-12-092006-11-22华为技术有限公司System and method for testing measured object based on automatic test script
CN1851667A (en)*2006-02-152006-10-25华为技术有限公司Graphic user interface test method and system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
GUI自动测试工具设计与实现. 杨凡德,李小将,闫舒.装备指挥技术学院学报,第14卷第3期. 2003
GUI自动测试工具设计与实现. 杨凡德,李小将,闫舒.装备指挥技术学院学报,第14卷第3期. 2003*
一种基于GUI的测试脚本开发环境. 阚红星,龚育昌.计算机系统应用,第1期. 2005
一种基于GUI的测试脚本开发环境. 阚红星,龚育昌.计算机系统应用,第1期. 2005*

Also Published As

Publication numberPublication date
CN101105768A (en)2008-01-16

Similar Documents

PublicationPublication DateTitle
Cheng et al.Seeclick: Harnessing gui grounding for advanced visual gui agents
KR102565455B1 (en) Domain-specific language interpreter and interactive visual interface for rapid screening
Xu et al.Aguvis: Unified pure vision agents for autonomous gui interaction
CN113015956B (en) System and method for activity target selection for robotic process automation
US9189254B2 (en)Translating text to, merging, and optimizing graphical user interface tasks
US6762776B2 (en)Mouse input panel windows class list
US7587668B2 (en)Using existing content to generate active content wizard executables for execution of tasks
Chai et al.Amex: Android multi-annotation expo dataset for mobile gui agents
CN112720452A (en)Naming robot process automation activities according to automatically detected target tags
EP3398031B1 (en)Hand gesture api using finite state machine and gesture language discrete values
US20110197124A1 (en)Automatic Creation And Management Of Dynamic Content
JP2009506465A5 (en)
CN1530817A (en)Guide of effective content: execution of task and structural content
KR20040063837A (en)Method and system for recording macros in a language independent syntax
CN100472472C (en) GUI test driving method based on MSAA under Windows platform
CN118467032A (en) Automated UI interaction exploration method based on multimodal large model
Inala et al.Data analysis in the era of generative AI
Cheng et al.YOLOv5‐MGC: GUI Element Identification for Mobile Applications Based on Improved YOLOv5
CN115390720A (en) Robotic Process Automation (RPA) including automatic document scrolling
CN113807077A (en)Natural language test script parsing processing method and device and electronic equipment
Xue et al.[Retracted] Visual Identification of Mobile App GUI Elements for Automated Robotic Testing
CN113703637A (en)Inspection task coding method and device, electronic equipment and computer storage medium
CN106371818B (en)Method for processing user-level events to program application program
Rose et al.Modechart toolset user's guide
Yang et al.macOSWorld: A Multilingual Interactive Benchmark for GUI Agents

Legal Events

DateCodeTitleDescription
C06Publication
PB01Publication
C10Entry into substantive examination
SE01Entry into force of request for substantive examination
C14Grant of patent or utility model
GR01Patent grant
CF01Termination of patent right due to non-payment of annual fee

Granted publication date:20090325

Termination date:20160726

CF01Termination of patent right due to non-payment of annual fee

[8]ページ先頭

©2009-2025 Movatter.jp