Disclosure of Invention
In view of the shortcomings of the prior art, the invention aims to provide a voice interaction method, a storage medium and terminal equipment.
The technical scheme adopted by the invention is as follows:
a method of voice interaction, comprising:
registering a voice packet in each interface starting function of an application program in advance through a decompilation mode to receive broadcasting;
receiving a voice control instruction, and searching a control corresponding to the voice control instruction in a current interface of the application program according to instruction information corresponding to the voice control instruction;
and calling a response function corresponding to the control to respond to the voice control instruction.
The voice interaction method, wherein registering voice packets in each interface start function of an application program in advance through decompilation mode specifically comprises the following steps:
for each application program loaded by a terminal device, acquiring the interface name of each interface of the application program;
decompiling the application program, determining interface starting functions of all interfaces according to all interface names, and dynamically registering voice packets in all interface starting functions to receive broadcasting.
The voice interaction method, wherein for each application program loaded on the terminal device, the obtaining the interface name of each interface of the application program specifically includes:
for each application program loaded by a terminal device, analyzing an installation package of the application program to obtain a configuration file of the application program;
and determining the interface names of the interfaces contained in the application program according to the configuration file.
The voice interaction method, wherein decompiling the application program, determining an interface starting function of each interface according to each interface name, and dynamically registering a voice packet in each interface starting function to receive broadcasting specifically comprises the following steps:
decompiling the application program, and obtaining a smali file obtained by decompiling;
searching a code file containing an interface starting function corresponding to each interface name in the smali file, and registering the voice packet receiving broadcast in the interface starting function.
The voice interaction method, wherein decompiling the application program, determining an interface starting function of each interface according to each interface name, and dynamically registering a voice packet in each interface starting function to receive broadcast further comprises:
and determining an interface exit function of each interface according to each interface name, and registering voice destruction broadcasting in each interface exit function.
The voice interaction method, wherein the determining the interface exit function of each interface according to each interface name, and registering voice package destruction broadcast in each interface exit function specifically comprises the following steps:
acquiring a smali file obtained by decompilation, and searching code files of all interfaces in the smali file;
searching an interface exit function of each interface in the searched code file of each interface;
if the interface exit function is found, registering voice destruction broadcasting in the interface exit function;
if the interface exit function is not found, the interface exit function is created in the code file, and the voice destruction broadcast is registered in the created interface exit function.
The voice interaction method, wherein registering the voice packet in each interface start function of the application program in advance through decompilation mode further comprises:
reconstructing the application program so that voice packet receiving broadcast is registered in each interface starting function of the application program.
The voice interaction method, wherein the receiving the voice control instruction and searching the control corresponding to the voice control instruction in the current interface of the application program according to the instruction information corresponding to the voice control instruction specifically comprises:
receiving a voice control instruction and identifying instruction information corresponding to the voice control instruction;
traversing each node contained in the current interface, and matching node instructions contained in each node with the instruction information;
and when traversing to the node successfully matched, determining the control corresponding to the voice control instruction according to the node.
A computer readable storage medium storing one or more programs executable by one or more processors to implement steps in a voice interaction method as described in any of the above.
A terminal device, comprising: a processor, a memory, and a communication bus; the memory has stored thereon a computer readable program executable by the processor;
the communication bus realizes connection communication between the processor and the memory;
the processor, when executing the computer readable program, implements the steps in the voice interaction method as described in any of the above.
The beneficial effects are that: compared with the prior art, the invention provides a voice interaction method, a storage medium and terminal equipment, wherein the method comprises the following steps: registering a voice packet in each interface starting function of an application program in advance through a decompilation mode to receive broadcasting; receiving a voice control instruction, and searching a control corresponding to the voice control instruction in a current interface of the application program according to instruction information corresponding to the voice control instruction; and calling a response function corresponding to the control to respond to the voice control instruction. According to the voice packet receiving and broadcasting method and device, the voice packet receiving and broadcasting is registered into the interface starting function through decompiling of the application program, so that when the application program is started, the voice packet receiving and broadcasting is started to receive the voice control instruction, the butt joint of the application program and the voice application program is avoided, and the response speed of voice interaction is improved.
Detailed Description
The invention provides a voice interaction method, a storage medium and a terminal device, and in order to make the purposes, technical schemes and effects of the invention clearer and more definite, the invention is further described in detail below by referring to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The invention will be further described by the description of embodiments with reference to the accompanying drawings.
The embodiment provides a voice interaction method, as shown in fig. 1, including:
s10, registering the voice packet in each interface starting function of the application program in advance through a decompilation mode to receive the broadcast.
Specifically, the interface starting function is configured in the application program, and each interface of the application program triggers the corresponding interface function when being started. That is, each interface of an application corresponds to an interface function, and the interface function is activated or triggered when the interface is started. Thus, the broadcasting of the voice packet is dynamically registered in the interface starting function, and the broadcasting of the voice packet is registered when the interface is started, so that the interface can receive the voice control instruction. In addition, in the android system, each interface of the application program inherits the Activity in the android native, and from the life cycle of the Activity, each interface start triggers an interface function first. The interface function may be an onCreate function, and in the smali code obtained by decompilation, the code of the interface starting function may be:
#virutal methods
.method protected onCreate(Landroid/os/Bundle；)V
where, # virus methods are method declaration instructions, hereafter introduced method protected onCreate (Landrid/os/Bundle) V denotes the onCreate function.
Further, in one embodiment of the present invention, as shown in fig. 2, the registering, in advance, the voice packet receiving broadcast in each interface start function of the application program through the decompilation manner specifically includes:
s11, for each application program loaded by a terminal device, acquiring the interface name of each interface of the application program;
s12, decompiling the application program, determining interface starting functions of the interfaces according to the names of the interfaces, and dynamically registering voice packets in the interface starting functions to receive broadcasting.
Specifically, the interface name is an interface unique identifier of each interface of the application program, according to the interface name, one interface of the application program can be used, and the interface name can be found in a configuration file of the application program when the application program installation package is analyzed, wherein the interface name is a tag value of a < activity > node in the configuration file. That is, for each application program loaded on the terminal device, the installation package of the application program is parsed, the configuration file android management. Xml in the application member is obtained, and when the < activity > node in the configuration file android management. Xml in the application is parsed, the android: name tag value of the application member is obtained and stored, wherein the android: name tag value is the interface name of each interface. Correspondingly, for each application program loaded by the terminal device, the acquiring the interface name of each interface of the application program specifically includes:
for each application program loaded by a terminal device, analyzing an installation package of the application program to obtain a configuration file of the application program;
and determining the interface names of the interfaces contained in the application program according to the configuration file.
Specifically, the determining, according to the configuration file, the interface name of each interface included in the application program is searching the interface name of each interface in the configuration file. The interface names of the interfaces in the configuration file can be stored in the same format, for example, in the form of android name, wherein the tag value of the android name is the interface name. That is, extracting tag values of all android: names in the configuration file, namely obtaining interface names of all interfaces contained in the application program.
Further, the decompilation is performed by a decompilation tool in the parsing of the application installation package, wherein the decompilation tool may be an apktool tool or the like. Correspondingly, as shown in fig. 3, decompiling the application program, determining an interface starting function of each interface according to each interface name, and dynamically registering a voice packet in each interface starting function to receive a broadcast specifically includes:
s121, decompiling the application program, and acquiring a smali file obtained by decompiling;
s122, searching a code file which contains an interface starting function and corresponds to each interface name in the smali file, and registering the voice packet receiving broadcast in the interface starting function.
Specifically, when decompiling is performed on an application program, a smali folder is generated under a decompiling engineering directory, all the decompiled smali files are stored in the smali files, all the smali files generate corresponding directories according to the hierarchical structure of the program package, and all the classes in the program generate independent smali files under the corresponding directories. For example, assuming that an application interface name is com.supxz.test.MainActivlty, a com/supxz/test/directory structure is sequentially generated under the smali directory, and then a MainActivlty.smali file is generated under this directory.
Further, in the smali folder, the smali code file with the same interface name as that obtained in the configuration file can be searched, and the following is continuously taken as an example of the MainActivlty.
Wherein the first row of the class instruction specifies the class name of the current class, the second row of the class instruction specifies the parent class of the current class, and the third row of the source instruction specifies the source file name of the current class. The #virus methods are method declaration instructions, parameter is parameter instruction, program is code start instruction, invoke-virtual is method call instruction.
Further, in order to avoid that the application program still continues to receive the voice control instruction when exiting from the foreground operation, the voice packet cancellation broadcast is registered in the interface exit function of each interface of the application program. Correspondingly, decompiling the application program, determining an interface starting function of each interface according to each interface name, and dynamically registering a voice packet in each interface starting function to receive the broadcast further comprises:
and determining an interface exit function of each interface according to each interface name, and registering voice destruction broadcasting in each interface exit function.
Specifically, the interface exit function is stored in a smali code file corresponding to the application interface, that is, the interface exit function is searched in the smali code file corresponding to each interface of the application program, and registered voice packet broadcast is dynamically destroyed in the interface exit function. When the application program comprises a plurality of interfaces, the smali code files corresponding to the interfaces are modified in sequence, namely after the interface starting function and the interface exiting function of the smali code file corresponding to one interface are registered, the registration of the interface starting function and the interface exiting function is repeated until all the interfaces of the application program are configured.
In addition, the interface exit function may not be carried in the smali code file corresponding to each interface of the application program, and at this time, the interface exit function needs to be added in the smali code file, and the voice destruction broadcast is registered in the interface exit function. Correspondingly, determining the interface exit function of each interface according to each interface name, and registering the voice destruction broadcast in each interface exit function specifically comprises the following steps:
acquiring a smali file obtained by decompilation, and searching code files of all interfaces in the smali file;
searching an interface exit function of each interface in the searched code file of each interface;
if the interface exit function is found, registering voice destruction broadcasting in the interface exit function;
if the interface exit function is not found, the interface exit function is created in the code file, and the voice destruction broadcast is registered in the created interface exit function.
Specifically, the code file is a smali code file of the interface, and an interface starting function and an interface exiting function of the interface are stored in the code file. That is, the interface exit function is searched in the smali code file of the interface, if the interface exit function is not searched, the interface exit function is added in the smali code file, and meanwhile, the registered voice packet broadcast is dynamically destroyed in the interface exit function.
Further, after decompiling an application and registering a voice package broadcast within an interface launch function, the application needs to be reconfigured. Correspondingly, registering the voice packet receiving broadcast in each interface starting function of the application program in advance through a decompilation mode further comprises: reconstructing the application program so that voice packet receiving broadcast is registered in each interface starting function of the application program. And the reconstruction process is to reinstall the application program by adopting a decompiled installation package.
In addition, in order to ensure that each application program in the terminal equipment is decompiled, the terminal equipment monitors whether the application program is installed or not, when the application program is monitored to be installed, the installed application program can be decompiled, a voice packet is registered in an interface starting function in a smali code file of each interface obtained by decompiling to receive broadcasting, and a voice packet destruction broadcasting is registered in an interface exiting function, so that all the application programs loaded by the terminal equipment do not need to be in butt joint with the voice application program, and a voice control instruction can be received. Of course, the application program in the terminal device may be a pre-assembled program or a third application program, and the voice packet broadcast is registered by adopting a decompilation mode for all third party application programs, and the voice packet broadcast may be registered by adopting a decompilation mode for the pre-assembled application program or by adopting a mode of modifying an Activity component in the android system in advance and registering the voice packet broadcast to the Activity component.
S20, receiving a voice control instruction, and searching a control corresponding to the voice control instruction in a current interface of the application program according to instruction information corresponding to the voice control instruction.
Specifically, the voice control instruction may be received by the terminal device through a sound pickup, or may be sent by an external device connected to the terminal device. The voice control instruction carries instruction information, and when the voice control instruction is received, the voice control instruction needs to be identified so as to obtain the instruction carrying information. The voice control instruction recognition method may be an existing voice recognition method, and will not be described herein.
Further, after the instruction information is obtained, the instruction information is searched in a display interface of the application program, which is referred to as a current interface, and a control carrying the instruction information is searched for, so that the response is achieved through the searched control. Correspondingly, as shown in fig. 4, the receiving the voice control instruction, and searching the control corresponding to the voice control instruction in the current interface of the application program according to the instruction information corresponding to the voice control instruction specifically includes:
s21, receiving a voice control instruction and identifying instruction information corresponding to the voice control instruction;
s22, traversing each node contained in the current interface, and matching node instructions contained in each node with the instruction information;
s23, when traversing to the node successfully matched, determining a control corresponding to the voice control instruction according to the node.
Specifically, an application program receives a voice control instruction through a registered voice packet, obtains instruction information corresponding to the voice control instruction, inherits services of the accessibilitiyService, sends a notification through the services of the accessibilitiyService, triggers an oneaccessibilitiyEvent callback function through the notification, traverses each node in a current interface through the oneaccessibilitiyEvent callback function, judges whether each node carries characters or not, and matches the node instructions with the instruction information when the characters are carried; and triggering the control corresponding to the node if the matching is successful. Of course, when no text exists or matching is unsuccessful, whether all nodes in the current interface are traversed is detected, if all nodes are traversed, the feedback voice control instruction cannot respond, and if all nodes are not traversed, the process is repeated until all nodes are traversed.
S30, calling a response function corresponding to the control to respond to the voice control instruction.
Specifically, the calling the response function corresponding to the control may be obtaining the position of the text node corresponding to the control, and generating a touch event according to the position to trigger the response function corresponding to the control, and responding to the voice control instruction through the response function.
Based on the above voice interaction method, the present invention further provides a computer readable storage medium, wherein the computer readable storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the steps in the voice interaction method as described in any one of the above.
Based on the above voice interaction method, the present invention further provides a terminal device, as shown in fig. 5, which includes at least one processor (processor) 20; a display screen 21; and a memory (memory) 22, which may also include a communication interface (Communications Interface) 23 and a bus 24. Wherein the processor 20, the display 21, the memory 22 and the communication interface 23 may communicate with each other via a bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may invoke logic instructions in the memory 22 to perform the methods of the embodiments described above.
Further, the logic instructions in the memory 22 described above may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand alone product.
The memory 22, as a computer readable storage medium, may be configured to store a software program, a computer executable program, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 performs functional applications and data processing, i.e. implements the methods of the embodiments described above, by running software programs, instructions or modules stored in the memory 22.
The memory 22 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include high-speed random access memory, and may also include nonvolatile memory. For example, a plurality of media capable of storing program codes such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or a transitory storage medium may be used.
In addition, the specific processes that the storage medium and the plurality of instruction processors in the terminal device load and execute are described in detail in the above method, and are not stated here.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.