CN110119322B - Data acquisition method and device, computer equipment and readable storage medium - Google Patents
Data acquisition method and device, computer equipment and readable storage mediumDownload PDFInfo
- Publication number
- CN110119322B CN110119322BCN201910381001.8ACN201910381001ACN110119322BCN 110119322 BCN110119322 BCN 110119322BCN 201910381001 ACN201910381001 ACN 201910381001ACN 110119322 BCN110119322 BCN 110119322B
- Authority
- CN
- China
- Prior art keywords
- service
- target
- data
- target service
- code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Technical Field
The present invention relates to the field of data management technologies, and in particular, to a data acquisition method and apparatus, a computer device, and a readable storage medium.
Background
With the increase of the demand of people, the related aspects of internet services are more extensive, so that the internet services are more and more complex, due to the introduction of new architectures such as micro-services, a plurality of application servers can realize a complex internet service through the micro-services, and in order to maintain the normal operation of the internet service, when the internet service has an error in operation, the data related to the service can be acquired to determine the location of the error.
Currently, data acquisition may be the following process: an internet service can be realized by a plurality of Remote Procedure Call (RPC) requests, each RPC request can be realized by calling a plurality of application servers, when any application server is called by any RPC request, the application server records log data for realizing the RPC request, the log data comprises a global unique identifier, request parameters and return results of the internet service, the log data on each application server in the service system is stored in a data center according to the global unique identifier, the data center can adopt different colors to identify different return results when storing the log data, for example, if the return results are successful, the return results in the log data are marked as black, if the return results are failed, the return results in the log data are marked as red, after the computer device obtains the log data corresponding to the global unique identifier in the data center, the user can determine the call with the problem according to the color of the returned result in the log data.
In the data acquisition process, according to the color of the returned result of the RPC request, the call causing the operation error can be determined, but the specific wrong data in the call process cannot be determined, so that the acquired data is incomplete.
Disclosure of Invention
The embodiment of the invention provides a data acquisition method, a data acquisition device, computer equipment and a readable storage medium, which can solve the problem of incomplete data acquisition in the data acquisition process. The technical scheme is as follows:
in a first aspect, a data acquisition method is provided, and the method includes:
acquiring a webpage address and a service identifier of a target service according to a problem troubleshooting instruction aiming at the target service;
generating a first data acquisition request according to the service identifier and the webpage address, wherein the first data acquisition request is used for indicating target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center;
sending the first data acquisition request to the target equipment, wherein the service data comprises input parameters and return results of a plurality of Remote Procedure Call (RPC) requests carrying the service identification;
and acquiring the service data through the data center.
In a second aspect, a data acquisition method is provided, the method comprising:
receiving a first data acquisition request, wherein the first data acquisition request is used for indicating target equipment to execute a target service based on a webpage address and sending service data in the target service execution process to a data center;
executing the target service based on the first data acquisition request to obtain service data in the target service execution process, wherein the service data comprises input parameters and return results of a plurality of RPC requests carrying service identifiers of the target service;
and sending the service data and the service identifier to a data center.
In a third aspect, a data acquisition apparatus is provided, the apparatus comprising:
the first acquisition module is used for acquiring a webpage address and a service identifier of a target service according to a problem troubleshooting instruction aiming at the target service;
the generating module is used for generating a first data acquisition request according to the service identifier and the webpage address, wherein the first data acquisition request is used for indicating target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center;
a sending module, configured to send the first data obtaining request to the target device, where the service data includes input parameters and return results of multiple Remote Procedure Call (RPC) requests that carry the service identifier;
and the second acquisition module is used for acquiring the service data through the data center.
In a possible implementation manner, the first obtaining module is configured to:
extracting a webpage address of the target service from the problem troubleshooting instruction aiming at the target service;
and generating a service identifier based on the webpage address.
In a possible implementation manner, the first data obtaining request includes the service identifier, the web address, and a parameter identifier, where the parameter identifier is used to instruct a target device to execute the target service based on the web address, and send service data in a target service execution process to a data center.
In a possible implementation manner, the second obtaining module is configured to receive the service data sent by the data center.
In a possible implementation manner, the second obtaining module is further configured to send a second data obtaining request to the data center, where the second data obtaining request carries the service identifier.
In one possible implementation manner, the device further comprises a display module for displaying the input parameters and the return result of the RPC request on a line for each RPC request.
In a fourth aspect, there is provided a data acquisition apparatus, the apparatus comprising:
the system comprises a receiving module, a data center and a data processing module, wherein the receiving module is used for receiving a first data acquisition request, and the first data acquisition request is used for indicating target equipment to execute a target service based on a webpage address and sending service data in the target service execution process to the data center;
the execution module is used for executing the target service based on the first data acquisition request to obtain service data in the target service execution process, wherein the service data comprises input parameters and return results of a plurality of RPC requests carrying service identifiers of the target service;
and the sending module is used for sending the service data and the service identifier to a data center.
In one possible implementation manner, the execution module includes:
a generating unit, configured to generate an agent based on the web address and the service identifier of the target service in the first data acquisition request, where the agent is configured to execute the target service based on the web address of the target service and send service data in the target service execution process to a data center;
an obtaining unit, configured to obtain service data in the target service execution process based on the agent program;
correspondingly, the sending module is configured to send the service data and the service identifier to a data center based on the agent program.
In one possible implementation manner, the generating unit includes:
a determining subunit, configured to determine, based on a web address of a target service in the data acquisition request, a first code, where the first code is a method code for implementing an original RPC request;
the first generating subunit is configured to generate a first target bytecode based on the first code and the parameter identifier, where the first target bytecode is configured to execute a target service based on a web address of the target service, and send service data in a target service execution process to a data center;
an implementation subunit, configured to implement dynamic instrumentation based on a first target class, where the first target class is used to implement modifying bytecode;
and the second generation subunit is used for generating the agent program based on the first target byte code and the instrumentation.
In a possible implementation manner, the first generating subunit is configured to:
generating a first bytecode based on the first code;
when detecting that the first data acquisition request carries a parameter identifier, adding a target code in the first code according to a calling mode in the first code to obtain a second code, wherein the target code is used for executing the target service based on a webpage address of the target service and sending service data in the target service execution process to a data center;
generating a second bytecode based on the second code;
and adding a second target bytecode to the first bytecode to obtain a first target bytecode, wherein the second target bytecode is a bytecode, which is different from the first bytecode, in the second bytecode.
In a possible implementation manner, the adding, according to a calling manner in the first code, object code in the first code to obtain second code includes:
when the calling mode in the first code is asynchronous calling, defining a callback class to realize an asynchronous method callback interface, obtaining a first callback class, rewriting a callback function executed when the request is completed and a function for processing errors so as to realize that the target service is executed based on the webpage address of the target service, and sending service data in the target service execution process to a data center;
adding a target code in the first code to obtain a second code;
and replacing the original callback class with the first callback class.
In a possible implementation manner, the adding, according to a calling manner in the first code, object code in the first code to obtain second code includes:
and when the calling mode in the first code is synchronous calling, adding a target code in the first code to obtain a second code.
In a fifth aspect, there is provided a computer device comprising: a processor; a memory for storing a computer program; the processor is used for executing the computer program stored in the memory to realize the operation executed by the data acquisition method.
In a sixth aspect, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the operations performed by the data acquisition method.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
the target service is executed through the target equipment, the service data for executing the target service are obtained, and the service data are sent to the data center, so that the first equipment can obtain the target data of the target service in the data center.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic illustration of an implementation environment provided by an embodiment of the invention;
fig. 2 is a schematic structural diagram of a data acquisition system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention;
fig. 4 is a flowchart of a data acquisition method provided in an embodiment of the present application;
fig. 5 is a flowchart of an agent generation method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the invention. Referring to fig. 1, the implementation environment includes a first device, a target device, and a data center, where the first device is configured to receive a problem troubleshooting instruction of a target service and display service data of the target service, so that a technician may troubleshoot a problem occurring in an execution process of the target service according to the service data displayed by the first device. The first device may be any one of the computer devices having the above functions, and the first device may also be a dedicated device used when the problem is solved for the target service. The embodiment of the present invention does not specifically limit the first device.
And the target device is used for generating the agent program, acquiring the service data in the process of executing the target service based on the agent program, and uploading the service data to the data center so that the data center stores the service data and returns the service data to the first device, and the target device can be any application server for executing the target service, and the application server can call other application servers through RPC (remote procedure call) requests in the process of executing the target service so as to complete the target service.
And the data center is used for receiving and storing the service data sent by the target equipment and sending the service data to the first equipment so that the first equipment can display the service data.
In some embodiments, the functions of the first device and the target device may be embodied in a data acquisition system, referring to fig. 2, fig. 2 is a schematic structural diagram of a data acquisition system according to an embodiment of the present invention, where the data acquisition system may include a sending request module, an executing agent module, a bytecode modification and data collection reporting module, and a data presentation module.
The system comprises a request sending module, an agent program executing module, a bytecode modification and data collection reporting module and a data display module, wherein the request sending module is used for completing the function of sending a request to a target device by a first device, the agent program executing module is used for completing the function of executing an agent program on the target device, the bytecode modification and data collection reporting module is used for completing the functions of generating the agent program, collecting data and reporting data, and the data display module is used for completing the function of displaying service data of a target service by the first device.
The first device may be a computer device, and in order to further embody the hardware structure of the computer device, referring to fig. 3, fig. 3 is a schematic structural diagram of a computer device provided in an embodiment of the present invention, thecomputer device 300 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 301 and one ormore memories 302, where thememories 302 store at least one instruction, and the at least one instruction is loaded and executed by theprocessors 301 to implement the methods provided in the following method embodiments. Certainly, the computer device may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the computer device may further include other components for implementing the functions of the device, which is not described herein again.
In an exemplary embodiment, a computer-readable storage medium, such as a memory, including instructions executable by a processor in a terminal to perform the data acquisition method in the following embodiments is also provided. For example, the computer-readable storage medium may be a read-only memory (ROM), a Random Access Memory (RAM), a compact disc-read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and so on.
The above is a description of an implementation environment of the present invention, and further, in order to embody a process of acquiring data by a first device, a specific embodiment is described herein, referring to fig. 4, where fig. 4 is a flowchart of a data acquisition method provided in an embodiment of the present application, and a method flow provided in an embodiment of the present invention includes:
401. the first device obtains the webpage address and the service identification of the target service according to the problem troubleshooting instruction aiming at the target service.
The target service may be a service provided by any web page, a web page address of the web page, that is, a web page address of the target service, or may be understood that the web page indicated by the web page address is used to provide the target service, and information related to the target service may be displayed on the web page, for example, the web page is a web page of a web store, the target service provided by the web page is a commodity in a selling store, and the web page may also display sales promotion information, commodity type, and the like of the store. The customer may then display the web page on the client and may purchase the items within the store on the web page.
The service identifier is used to indicate the service, the service identifier may be represented by a character string, and services indicated by different character strings are different, for example, the service identifier may be (UUID).
The problem troubleshooting instruction may include a web address of the target service, which is used to indicate to obtain service data in the execution process of the target service, so that a technician may troubleshoot the reason of the error information displayed on the web page according to the service data obtained by the first device. The problem troubleshooting instruction may be a hypertext transfer protocol (HTTP) request or an RPC request. The problem troubleshooting instruction is not particularly limited in the embodiment of the present invention.
In a possible implementation manner, when the user finds that the information on the webpage of the target service is displayed incorrectly, the user may input the webpage address of the target service on the query interface of the first device, and click the query, so as to trigger the first device to receive the problem troubleshooting instruction, where the problem troubleshooting instruction carries the webpage address of the target service.
The first device may perform thisstep 401 by extracting the content of the problem troubleshooting instruction, and in one possible implementation, the first device may perform thisstep 401 by the process shown in steps 401A-401B described below.
Step 401A, the first device extracts a web page address of the target service from the problem troubleshooting instruction for the target service.
When the first device receives the problem troubleshooting instruction, the web page address of the target service may be extracted from a target field of the problem troubleshooting instruction, where the target field is used to store the web page address of the target service.
Step 401B, the first device generates a service identifier based on the web page address.
The first device may generate the service identifier of the target service according to a preset service identifier generation algorithm, where the service identifier generation algorithm is used to generate the service identifier, for example, a web address is http:// aaa. bbb.com, and according to the web address and an algorithm used to generate the UUID, the UUID ═ 28B35FC5548DDCDA15E generated by the first device is the service identifier of the target service. It should be noted that, the embodiment of the present invention does not specifically limit the service identifier generation algorithm.
402. And the first equipment generates a first data acquisition request according to the service identifier and the webpage address, wherein the first data acquisition request is used for indicating the target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center.
The first data acquisition request comprises the service identifier, the webpage address and a parameter identifier, wherein the parameter identifier is used for indicating target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center. The parameter identifier may be represented by a character string, for example, the parameter identifier is dataMark ═ true, and the embodiment of the present invention does not specifically limit the parameter identifier.
In a possible implementation manner, after the first device generates the service identifier according to the web address, the first device combines the service identifier, the web address, and the parameter identifier according to a combination rule to obtain the first data request, where the combination rule is used to generate the first data request.
In a possible implementation manner, after the first device receives the problem troubleshooting instruction and generates the service identifier, the first device adds the service identifier and the parameter identifier to the problem troubleshooting instruction to obtain the first data acquisition request.
For example, when the first device generates the service identifier uuid ═ 28B35FC5548 ddda 15E, the service identifier uuid ═ 28B35FC5548 ddda 15E and the parameter identifier dataMark ═ true are added to the problem troubleshooting instruction http:// aaa.bbb.com, and the obtained first data acquisition request is:
http://aaa.bbb.comuuid＝28B35FC5548DDCDA15E&dataMark＝true。
403. and the first equipment sends the first data acquisition request to the target equipment, wherein the service data comprises input parameters and return results of a plurality of Remote Procedure Call (RPC) requests carrying the service identifier.
The target device can complete the target service by executing the RPC request, and the target device can call other target devices by other RPC requests to complete the target service in the process of executing the RPC request, so that a plurality of RPC requests can appear in the process of completing the target service, each RPC request carries input parameters and a return result, the input parameters are parameters input by any target device when executing the RPC request, and the return result is a result returned by any target device after executing the RPC request.
It should be noted that the service data may further include method names of a plurality of remote procedure call RPC requests, and the method name of each RPC request is used to indicate the RPC request.
404. The target equipment receives a first data acquisition request, wherein the first data acquisition request is used for indicating the target equipment to execute the target service based on the webpage address, and the service data in the target service execution process is sent to the data center.
405. The target device executes the target service based on the first data acquisition request to obtain service data in the target service execution process, wherein the service data comprises input parameters and return results of a plurality of RPC requests carrying service identifiers of the target service.
The target device may implement thisstep 405 through an agent program, where the agent program is a program independent from the application program, and is mainly used to execute the target service, collect service data of the target service, and upload the service data, and when the agent program requests to execute the target service through the PRC, the agent program may add the service identifier in the first data acquisition request to the PRC request related to the target service, so that the target device may collect the service data of the target service through the agent program.
In one possible implementation, thisstep 405 may be implemented by the process illustrated by steps 405A-405B described below.
Step 405A, the target device generates an agent program based on the web page address and the service identifier of the target service in the first data acquisition request, where the agent program is configured to execute the target service based on the web page address of the target service, and send service data in the target service execution process to a data center.
The target device may dynamically modify the bytecode of the method of implementing the original RPC request when the target device executes the target service, based on the web address and the service identifier of the target service in the first data acquisition request, thereby generating the agent program. The method for realizing the original RPC request is a method for initiating the RPC request by an RPC frame before target equipment executes a target service.
In a possible implementation manner, referring to fig. 5, fig. 5 is a flowchart of an agent generation method provided in an embodiment of the present invention, where the method flow may include the followingsteps
When the target device executes the target service, other target devices can be called through the RPC request to complete a task indicated by the RPC request, and when a method depending on a service is called, the RPC request on the target device can be initiated through the RPC frame, wherein the method of the service refers to a calling method in the service provided by the RPC frame, so that the target device can determine the RPC request when the target device executes the target service before according to a webpage address of the target service in the data acquisition request, and further can determine a first code in the RPC frame through the RPC request.
In order to implement the problem troubleshooting, the target device needs to execute the target service once according to the agent program, and then can collect the service data of the target service and upload the service data to the data center.
In one possible implementation, this step 502A may be implemented by the process shown in steps 21-24 below.
Step 21, the target device generates a first bytecode based on the first code.
The target device may generate a first Bytecode from the first code by using a Java Bytecode manipulation framework-Bytecode (ASM-Bytecode) plug-in, where the first Bytecode is a Bytecode generated based on ASM, that is, an ASM Bytecode, and the ASM-Bytecode plug-in is an Eclipse (open source code and Java-based extensible development platform) plug-in. Specifically, the target device is installed with an ASM-byte code plug-in, and a user can open a first code in Eclipse and click an "ASM" button in the upper right corner of the byte code view to trigger the ASM to generate the first Bytecode.
The Java bytecode manipulation framework (ASM) refers to ObjectWeb ASM, that is, a lightweight Java bytecode processing framework, which can dynamically generate stub (stub) classes or other proxy classes in binary format, and can dynamically modify the classes before the classes are loaded into the memory by the Java virtual machine.
Step 22, when detecting that the first data acquisition request carries a parameter identifier, the target device adds a target code to the first code according to a calling mode in the first code to obtain a second code, wherein the target code is used for executing the target service based on a webpage address of the target service, and sending service data in the target service execution process to a data center.
The calling mode in the first code may include any one of a synchronous calling mode and an asynchronous calling mode, the calling mode in the first code is different, and the target device adds the target code in the first code in a different process. The object code may be pre-written code by a technician so that the object device may add the object code directly to the first code when generating the agent.
In one possible implementation, the step 22 can be implemented by the process shown in steps 2201-2204.
Step 2201, when the calling mode in the first code is synchronous calling, the target device adds a target code to the first code to obtain a second code.
When the calling mode in the first code is synchronous calling, the target device needs to wait for a return result after the RPC request is sent, and the target device can send the RPC request only after the target device obtains the return result. Therefore, when the calling mode in the first code is synchronous calling, the target code can be directly and continuously added in the first code to implement the target service based on the webpage address of the target service, and the service data in the target service execution process is sent to the data center.
The target device may directly add the target code to the first code according to a preset rule, so as to obtain the second code, where the preset rule is used to add the target code to the first code.
Step 2202, when the calling mode in the first code is asynchronous calling, the target device defines a callback class to implement an asynchronous method callback interface, obtains the first callback class, rewrites a callback function executed when the request is completed and a function handling an error, to implement the target service based on the web address of the target service, and sends service data in the target service execution process to the data center.
When the call mode in the first code is asynchronous call, if the RPC request is sent, the target device may perform the next RPC request without waiting for a return result, so that in the case of asynchronous call, the target device may not immediately obtain the RPC request return result after sending the RPC request, and in order to implement the function of the broker, the target device may redefine an asynchronous method Callback (asynchronous method Callback) interface, that is, the process shown in this step 2202.
Since the first callback class is used for executing the target service when asynchronous call is implemented, and sending service data in the process of executing the target service to the data center, when the asynchronous method callback interface is redefined, a method (onComplete method) executed when an RPC request in the asynchronous method callback interface is completed and a method (onError method) used when an error occurs need to be rewritten to implement the above functions. The onComplete method is used for processing a return result of the RPC request, and the oneerror method is used for processing an exception occurring in the process of calling the service.
Step 2203, the target device adds a target code to the first code to obtain a second code.
Step 2204, the target device replaces the original callback class with the first callback class.
The original callback class is the callback class in the first code. The target device may delete the original callback class in the first code and add the first callback class in the first code to implement this step 2204.
It should be noted that, when step 2201 is a synchronous call, the target device adds the target code to the first code. Step 2202 and 2204 are processes for adding the target code to the first code by the target device when asynchronous calling.
Step 23, the target device generates a second bytecode based on the second code.
The target device may generate a first Bytecode from the first code by using an ASM-Bytecode plug-in, where the second Bytecode is an ASM Bytecode.
Step 24, the target device adds a second target bytecode to the first bytecode to obtain a first target bytecode, where the second target bytecode is a bytecode of the second bytecode different from the first bytecode.
Since the second target bytecode is a bytecode of the second bytecode that is different from the first bytecode, it can be understood that the second target bytecode can implement execution of the target service and send service data of the target service to the data center, and then the function implemented by the first target bytecode includes a function of the second target bytecode.
The first target class is a subclass of a class access (ClassVisitor), and is used for adding a second target bytecode to a first bytecode in a compiled class to obtain a first target bytecode so as to modify the first bytecode. The ClassVisitor class is an important class in an ASM framework, is used for accessing all elements of a Java class, is an abstract class, and can complete reading and writing of compiled classes after the subclass realizes the method. The ClassVisitor is an interface in the Java bytecode engine library (ASM3.0), and the ClassVisitor class is also an interface class.
Dynamic Instrumentation, thisstep 503 may be implemented by the process shown in steps 31-33 below, in one possible implementation.
Step 31, the target device defines a second target class, and the second target class is used for realizing the proxy main function.
The agent main function is an agentmain function, technical personnel can write codes of a second object class in advance, the object equipment compiles the codes of the second object class to generate a file of the second object class, and a program in the file of the second object class is used for realizing the agentmain function.
Step 32, the target device implements a Class File Transformer (Class File Transformer) interface and rewrites a transform (transform) function based on the first target bytecode.
The target device may implement this step 32 by a write class (ClassReader) and a read class (ClassWriter) in the ASM. In one possible implementation, this step 32 may be implemented by the process shown in steps 3201-3205.
Step 3201, the target device reads a class file of the class containing the first code through a ClassReader class.
Since dynamic Instrumentation is to modify the bytecode, when implementing dynamic Instrumentation, the target device needs to read the class file of the class containing the first code first to modify the first bytecode in the class file.
Step 3202, the first target class overwrites an access method (visitMethod) of the class visitor class by inheriting the class visitor class to add the second target bytecode to the first bytecode, thereby implementing modification of the first bytecode.
Step 3203, the target device sets ClassWriter such that ClassWriter has the functionality of the first target class.
That is, the set ClassWriter may add the second target bytecode to the first bytecode in the compiled class to obtain the first target bytecode, so as to modify the first bytecode.
Step 3204, the ClassReader receives the instance of the first target class as a parameter by calling an accept method of the ClassReader, so that the ClassReader can call the rewritten visitMethod method of the first target class when traversing different nodes in the class (class) structure.
And 3205, the target device returns a byte stream of the ClassWriter to realize the output work of the modified first byte code.
The byte stream is also the first target bytecode obtained by the final target device, i.e. the modified first bytecode.
Step 33, the target device sets the Agent Class (Agent-Class) in the application Java package (manifest) file to specify that the second target Class is included.
Specifically, the target device opens a manifest file, and adds a code in the manifest: "Agent-Class name of second target Class" to enable setting of Agent-Class in manifest file to specify inclusion of the second target Class.
The target device may package all the related programs instep 503 into jar packages, and the contents in the jar packages are the agents. The associated program includes a first target bytecode and a program that implements dynamic instrumentation.
Step 405B, the target device obtains the service data in the target service execution process based on the agent program.
After the target device generates the agent program, the agent program is operated, and the agent program re-executes the target service in the operation process, so that the target device can acquire service data in the target service execution process.
406. And the target equipment sends the service data and the service identification to a data center.
The target device may send the service data and the service identification to a data center based on an agent. In a possible implementation manner, the target device may obtain service data carrying the target service in running the agent program, so that service data collection may be implemented, and the target device may send the service data and the service identifier of the target service to the data center while continuing to run the agent program.
The target device generates the agent program only when receiving the first data request, collects the service data of the target service based on the agent program and uploads the service data to the data center, so that the target device does not need to collect the service data all the time and send the service data to the data center all the time when executing the target service, and the service pressure of the target device and the data center is reduced.
It should be noted that the target device may also send the service data, the service identifier, and the device identifier of the first device to the data center, so that the data center may directly send the service data to the first device indicated by the device identifier according to the device identifier. The device identifier is used to indicate the first device, and may be an address of the first device. The device identifier is not specifically limited in the embodiments of the present invention.
407. When the data center receives the service data and the service identifier, the data center stores the service data and the service identifier in an associated manner.
The data center may use the service identifier as an index of the service data, and store the service data and the service identifier in association.
408. And the first equipment acquires the service data through the data center.
The first device may directly obtain the service data, or may obtain the service data by requesting, see the following manner 1-2.
In the mode 1, the first device receives the service data sent by the data center.
When the target device sends the device identifier of the first device, the service data, and the service identifier to the data center, after receiving the device identifier, the data center sends the service data to the first device indicated by the device identifier, and then the first device may receive the service data.
In the mode 2, the first device requests the data center for service data to obtain the service data.
The first device may implement mode 2 by following the process shown in steps 408A-408C.
Step 408A, the first device sends a second data acquisition request to the data center, where the second data acquisition request carries the service identifier.
And the second data acquisition request is used for requesting to acquire the service data corresponding to the service identifier.
Step 408B, the data center sends the service data corresponding to the service identifier to the first device based on the service identifier in the second data obtaining request.
The data center may query, using the service identifier as an index, service data stored in the data center and corresponding to the service identifier, and after the service data is queried, the data center sends the service data to the first device.
Step 408C, the first device receives the service data.
409. For each RPC request, the first device displays the input parameters and the returned results of the RPC request on a line.
Since the returned service data are the data input parameters and the returned results of the plurality of RPC requests, of course, the service data may also include the name of the method of each RPC request, and the first device may display the data in each RPC request in a manner of being striped through thisstep 409, so that a technician may perform problem troubleshooting according to each displayed content, thereby improving troubleshooting efficiency.
For example, the promotion information on the website of the target service should be a list of "new customer minus 15 yuan", but the promotion information displayed on the client is "new customer minus 12 yuan", so that the promotion information is wrong, and through the process shown insteps
{NewOrOldUser.getUser TypeById:{{request:{id:1234}},{response:{type:‘newUser’}}}；
{Promotion.GetPromotionById:{{request:{id:2345}},{response:[{PromotionPrice:45,user Type:‘oldUser’},{PromotionPrice:48,userType:‘newUser’}]}}；
{Price.GetPriceById:{{request:{id:2345}},{response:{price:60}}}。
from the above displayed results, the skilled person can see that: the original price of the commodity is 60 yuan, if the user is a new user, the promotion price of the new user is 48 yuan, and the promotion price of the old user is 45 yuan. And performing logical splicing according to the service data, namely the new user can reduce 60-48 to 12 yuan, and the new user actually needs to display 'new customer reduces 15 yuan', so that the situation that the promotion service data is in question is judged, and the promotion prices of the new user and the old user are reversed.
The display is convenient for technicians to read, so that the troubleshooting efficiency of the technicians can be improved. It should be noted that, since the return result is actual return data in the target service execution process, when a person performs problem troubleshooting according to the service data displayed by the first device, a technician determines an incorrect return result according to a logical relationship between the service data, and can further determine a problematic RPC request, and further can accurately locate problematic calls.
According to the method provided by the embodiment of the invention, the target service is executed by the target equipment, the service data for executing the target service is acquired, and the service data is sent to the data center, so that the first equipment can acquire the target data of the target service in the data center. And only when the target device receives the first data request, the agent program is generated, the service data of the target service is collected based on the agent program, and the service data is uploaded to the data center, so that the target device does not need to collect the service data all the time and send the service data to the data center all the time when executing the target service, and the service pressure of the target device and the data center is reduced. And, this first equipment can realize that the data of every RPC request carries out the stripe display to can make, the technical staff carries out the problem investigation according to every content that shows, thereby can improve investigation efficiency. And because the return result is the actual return data in the target service execution process, when a person checks problems according to the service data displayed by the first device, a technician determines the wrong return result according to the logical relationship between the service data, and then can determine the RPC request with problems, and then can accurately position the call with problems. And the first equipment only needs to send a problem troubleshooting instruction to the first equipment before displaying, so that the time for positioning the problem by the technician is saved.
Fig. 6 is a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention, where the apparatus includes:
a first obtainingmodule 601, configured to obtain a web address and a service identifier of a target service according to a problem troubleshooting instruction for the target service;
agenerating module 602, configured to generate a first data obtaining request according to the service identifier and the web address, where the first data obtaining request is used to instruct a target device to execute the target service based on the web address, and send service data in a target service execution process to a data center;
a sendingmodule 603, configured to send the first data obtaining request to the target device, where the service data includes input parameters and return results of multiple Remote Procedure Call (RPC) requests that carry the service identifier;
a second obtainingmodule 604, configured to obtain the service data through the data center.
Optionally, the first obtainingmodule 601 is configured to:
extracting a webpage address of the target service from the problem troubleshooting instruction aiming at the target service;
and generating a service identifier based on the webpage address.
Optionally, the first data obtaining request includes the service identifier, the web address, and a parameter identifier, where the parameter identifier is used to instruct a target device to execute the target service based on the web address, and send service data in the target service execution process to a data center.
Optionally, the second obtainingmodule 604 is configured to receive the service data sent by the data center.
Optionally, the second obtainingmodule 604 is further configured to send a second data obtaining request to the data center, where the second data obtaining request carries the service identifier.
Optionally, the apparatus further comprises a display module for displaying, for each RPC request, the input parameters and the return result of the RPC request on a line.
Fig. 7 is a schematic structural diagram of a data acquisition apparatus according to an embodiment of the present invention, where the apparatus includes:
areceiving module 701, configured to receive a first data obtaining request, where the first data obtaining request is used to instruct a target device to execute a target service based on a web address, and send service data in a target service execution process to a data center;
an executingmodule 702, configured to execute the target service based on the first data obtaining request, to obtain service data in the target service executing process, where the service data includes input parameters and return results of multiple RPC requests carrying service identifiers of the target service;
a sendingmodule 703, configured to send the service data and the service identifier to a data center.
Optionally, the executingmodule 702 includes:
a generating unit, configured to generate an agent based on the web address and the service identifier of the target service in the first data acquisition request, where the agent is configured to execute the target service based on the web address of the target service and send service data in the target service execution process to a data center;
an obtaining unit, configured to obtain service data in the target service execution process based on the agent program;
correspondingly, the sendingmodule 703 is configured to send the service data and the service identifier to a data center based on the agent program.
Optionally, the generating unit includes:
a determining subunit, configured to determine, based on the web address of the target service in the data acquisition request, a first code, where the first code is a method code for implementing an original RPC request;
the first generating subunit is configured to generate a first target bytecode based on the first code and the parameter identifier, where the first target bytecode is configured to execute a target service based on a web address of the target service, and send service data in a target service execution process to a data center;
an implementation subunit, configured to implement dynamic instrumentation based on a first target class, where the first target class is used to implement modifying bytecode;
and the second generation subunit is used for generating the agent program based on the first target byte code and the instrumentation.
Optionally, the first generating subunit is configured to:
generating a first bytecode based on the first code;
when detecting that the first data acquisition request carries a parameter identifier, adding a target code in the first code according to a calling mode in the first code to obtain a second code, wherein the target code is used for executing the target service based on a webpage address of the target service and sending service data in the target service execution process to a data center;
generating a second bytecode based on the second code;
adding a second target bytecode to the first bytecode to obtain a first target bytecode, wherein the second target bytecode is a bytecode different from the first bytecode in the second bytecode.
Optionally, the adding a target code in the first code according to the calling mode in the first code to obtain a second code includes:
when the calling mode in the first code is asynchronous calling, defining a callback class to realize an asynchronous method callback interface, obtaining the first callback class, rewriting a callback function executed when the request is completed and a function for processing errors so as to realize that the target service is executed based on the webpage address of the target service, and sending service data in the target service execution process to a data center;
adding a target code in the first code to obtain a second code;
and replacing the original callback class with the first callback class.
Optionally, the adding a target code in the first code according to the calling mode in the first code to obtain a second code includes:
and when the calling mode in the first code is synchronous calling, adding a target code in the first code to obtain a second code.
All the above optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described herein again.
It should be noted that: in the data acquisition apparatus provided in the above embodiment, when acquiring data, only the division of the above functional modules is taken as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the above described functions. In addition, the data acquisition device and the data acquisition method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (14)
1. A data acquisition method is applied to a first device, and the method comprises the following steps:
acquiring a webpage address and a service identifier of a target service according to a problem troubleshooting instruction aiming at the target service, wherein the problem troubleshooting instruction is used for indicating to acquire service data in the target service execution process, and the target service is a commodity displayed in a webpage indicated by the webpage address;
generating a first data acquisition request according to the service identifier and the webpage address, wherein the first data acquisition request is used for indicating target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center;
sending the first data acquisition request to the target device, generating an agent program by the target device based on the webpage address and the service identifier of the target service in the first data acquisition request, running the agent program, the agent is an application-independent program, the agent re-executes the target service during execution, the agent program adds the service identifier to a plurality of Remote Procedure Call (RPC) requests related to the target service when the target service is re-executed, calling other target equipment through the RPC requests to complete the target service, and sending the service data obtained when the agent program re-executes the target service to the data center, the service data comprises input parameters and return results of the RPC requests carrying the service identifiers;
and acquiring the service data through the data center.
2. The method according to claim 1, wherein the obtaining the web page address and the service identifier of the target service according to the problem troubleshooting instruction for the target service comprises:
extracting a webpage address of the target service from the problem troubleshooting instruction aiming at the target service;
and generating a service identifier based on the webpage address.
3. The method according to claim 1, wherein the first data acquisition request includes the service identifier, the web address, and a parameter identifier, and the parameter identifier is used to instruct a target device to execute the target service based on the web address, and send service data in the target service execution process to a data center.
4. The method of claim 1, further comprising:
and sending a second data acquisition request to the data center, wherein the second data acquisition request carries the service identifier.
5. A data acquisition method is applied to a target device, and the method comprises the following steps:
receiving a first data acquisition request, wherein the first data acquisition request is used for indicating the target equipment to execute a target service based on a webpage address and sending service data in the target service execution process to a data center, the first data acquisition request is generated by the first equipment according to the webpage address and a service identifier of the target service, which are acquired by a problem troubleshooting instruction aiming at the target service, the problem troubleshooting instruction is used for indicating to acquire the service data in the target service execution process, and the target service is a commodity displayed in a webpage indicated by the webpage address;
generating an agent program based on the webpage address and the service identifier of the target service in the first data acquisition request, wherein the agent program is a program independent of an application program and is used for executing the target service based on the webpage address of the target service and sending service data in the target service execution process to a data center;
the agent program is operated, the target service is re-executed in the operation process of the agent program, when the target service is re-executed, the service identifier is added into a plurality of RPC requests related to the target service, other target equipment is called through the RPC requests to complete the target service, and service data in the target service execution process are obtained, wherein the service data comprise input parameters and return results of the RPC requests carrying the service identifier;
and sending the service data and the service identifier to a data center.
6. The method of claim 5, wherein sending the service data and the service identifier to a data center comprises:
and sending the service data and the service identification to a data center based on the agent program.
7. The method according to claim 6, wherein the first data acquisition request includes a parameter identifier, and the parameter identifier is used to instruct a target device to execute the target service based on the web page address, and to send service data in the target service execution process to a data center;
generating an agent program based on the web page address and the service identifier of the target service in the first data acquisition request, including:
determining a first code based on a webpage address of a target service in the data acquisition request, wherein the first code is a method code for realizing an original RPC request;
generating a first target bytecode based on the first code and the parameter identifier, wherein the first target bytecode is used for executing the target service based on a webpage address of the target service and sending service data in the target service execution process to a data center;
implementing dynamic instrumentation based on a first target class, the first target class being used to implement modifying bytecode;
and generating an agent program based on the first target byte code and the instrumentation.
8. The method of claim 7, wherein generating a first target bytecode based on the first code and the parameter identification comprises:
generating a first bytecode based on the first code;
when detecting that the first data acquisition request carries a parameter identifier, adding a target code in the first code according to a calling mode in the first code to obtain a second code, wherein the target code is used for executing the target service based on a webpage address of the target service and sending service data in the target service execution process to a data center;
generating a second bytecode based on the second code;
adding a second target bytecode to the first bytecode to obtain a first target bytecode, wherein the second target bytecode is a bytecode different from the first bytecode in the second bytecode.
9. The method of claim 8, wherein adding object code to the first code according to the calling mode in the first code to obtain a second code comprises:
when the calling mode in the first code is asynchronous calling, defining a callback class to realize an asynchronous method callback interface, obtaining the first callback class, rewriting a callback function executed when the request is completed and a function for processing errors so as to realize that the target service is executed based on the webpage address of the target service, and sending service data in the target service execution process to a data center;
adding a target code in the first code to obtain a second code;
and replacing the original callback class with the first callback class.
10. The method of claim 8, wherein adding object code to the first code according to the calling mode in the first code to obtain a second code comprises:
and when the calling mode in the first code is synchronous calling, adding a target code in the first code to obtain a second code.
11. A data acquisition apparatus, characterized in that the apparatus comprises:
the system comprises a first acquisition module, a first processing module and a second acquisition module, wherein the first acquisition module is used for acquiring a webpage address and a service identifier of a target service according to a problem troubleshooting instruction aiming at the target service, the problem troubleshooting instruction is used for indicating to acquire service data in the execution process of the target service, and the target service is a commodity displayed in a webpage indicated by the webpage address;
the generating module is used for generating a first data acquisition request according to the service identifier and the webpage address, wherein the first data acquisition request is used for indicating target equipment to execute the target service based on the webpage address and sending service data in the target service execution process to a data center;
a sending module, configured to send the first data obtaining request to the target device, generate, by the target device, an agent based on the web address and the service identifier of the target service in the first data obtaining request, and run the agent, the agent is an application-independent program, the agent re-executes the target service during execution, the agent program adds the service identification to a plurality of Remote Procedure Call (RPC) requests related to the target service when the target service is re-executed, calling other target equipment through the RPC requests to complete the target service, and sending the service data obtained when the agent program re-executes the target service to the data center, the service data comprises input parameters and return results of a plurality of Remote Procedure Call (RPC) requests carrying the service identification;
and the second acquisition module is used for acquiring the service data through the data center.
12. A data acquisition apparatus, characterized in that the apparatus comprises:
the system comprises a receiving module, a data center and a processing module, wherein the receiving module is used for receiving a first data acquisition request, the first data acquisition request is used for indicating target equipment to execute a target service based on a webpage address and sending service data in the target service execution process to the data center, the first data acquisition request is generated by the first equipment according to the webpage address and a service identifier of the target service acquired by a problem troubleshooting instruction aiming at the target service, the problem troubleshooting instruction is used for indicating to acquire the service data in the target service execution process, and the target service is a commodity displayed in a webpage indicated by the webpage address;
the execution module is used for generating an agent program based on the webpage address and the service identifier of the target service in the first data acquisition request, wherein the agent program is a program independent of an application program and is used for executing the target service based on the webpage address of the target service and sending service data in the target service execution process to a data center;
the agent program is operated, the target service is re-executed in the operation process of the agent program, when the target service is re-executed, the service identifier is added into a plurality of RPC requests related to the target service, other target equipment is called through the RPC requests to complete the target service, and service data in the target service execution process are obtained, wherein the service data comprise input parameters and return results of the RPC requests carrying the service identifier;
and the sending module is used for sending the service data and the service identifier to a data center.
13. A computer device comprising one or more processors and one or more memories having stored therein at least one instruction that is loaded and executed by the one or more processors to perform operations performed by the data acquisition method of any one of claims 1 to 10.
14. A computer-readable storage medium having stored therein at least one instruction, which is loaded and executed by a processor to perform operations performed by a data acquisition method as claimed in any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910381001.8ACN110119322B (en) | 2019-05-08 | 2019-05-08 | Data acquisition method and device, computer equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910381001.8ACN110119322B (en) | 2019-05-08 | 2019-05-08 | Data acquisition method and device, computer equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110119322A CN110119322A (en) | 2019-08-13 |
| CN110119322Btrue CN110119322B (en) | 2022-05-10 |
Family
ID=67521913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910381001.8AActiveCN110119322B (en) | 2019-05-08 | 2019-05-08 | Data acquisition method and device, computer equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110119322B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110933075B (en)* | 2019-11-28 | 2022-01-11 | 腾讯音乐娱乐科技（深圳）有限公司 | Service calling method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462943A (en)* | 2014-11-21 | 2015-03-25 | 用友软件股份有限公司 | Non-intrusive performance monitoring device and method for service system |
| US9465684B1 (en)* | 2013-03-15 | 2016-10-11 | Emc Corporation | Managing logs of storage systems |
| CN108228147A (en)* | 2016-12-15 | 2018-06-29 | 中国移动通信集团公司 | A kind of performance data journal obtaining method and device |
| CN108228461A (en)* | 2018-01-08 | 2018-06-29 | 平安科技（深圳）有限公司 | A kind of daily record generation method, device, computer equipment and storage medium |
| CN109471845A (en)* | 2018-10-25 | 2019-03-15 | 深圳壹账通智能科技有限公司 | Log management method, server, and computer-readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100361456C (en)* | 2005-10-13 | 2008-01-09 | 华为技术有限公司 | Terminal device management method and terminal device thereof |
| WO2008103324A1 (en)* | 2007-02-19 | 2008-08-28 | Wms Gaming Inc. | Network diagnostics in a wagering game system |
| CN101888318B (en)* | 2010-06-30 | 2015-09-16 | 中兴通讯股份有限公司 | A kind of system and method for testing performance index of communication system |
| US10216621B1 (en)* | 2017-11-16 | 2019-02-26 | Servicenow, Inc. | Automated diagnostic testing of databases and configurations for performance analytics visualization software |
- 2019
- 2019-05-08CNCN201910381001.8Apatent/CN110119322B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9465684B1 (en)* | 2013-03-15 | 2016-10-11 | Emc Corporation | Managing logs of storage systems |
| CN104462943A (en)* | 2014-11-21 | 2015-03-25 | 用友软件股份有限公司 | Non-intrusive performance monitoring device and method for service system |
| CN108228147A (en)* | 2016-12-15 | 2018-06-29 | 中国移动通信集团公司 | A kind of performance data journal obtaining method and device |
| CN108228461A (en)* | 2018-01-08 | 2018-06-29 | 平安科技（深圳）有限公司 | A kind of daily record generation method, device, computer equipment and storage medium |
| CN109471845A (en)* | 2018-10-25 | 2019-03-15 | 深圳壹账通智能科技有限公司 | Log management method, server, and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110119322A (en) | 2019-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6755954B2 (en) | Interface data presentation method and equipment | |
| WO2021088724A1 (en) | Testing method and apparatus | |
| CN114416481B (en) | Log analysis method, device, equipment and storage medium | |
| CN112817817B (en) | Buried point information query method, buried point information query device, computer equipment and storage medium | |
| CN105468779A (en) | Browser compatibility detection oriented client Web application capture and playback system and method | |
| CN113703785A (en) | Component-based platform deployment method, device, equipment and storage medium | |
| US11741002B2 (en) | Test automation systems and methods using logical identifiers | |
| CN113868126A (en) | Storage medium for application debugging method, device and equipment | |
| US11366704B2 (en) | Configurable analytics for microservices performance analysis | |
| CN110263070A (en) | Event report method and device | |
| CN110851756A (en) | Page loading method and device, computer readable storage medium and terminal equipment | |
| CN109814863A (en) | A processing method, device, computer equipment and computer storage medium for requesting returned data | |
| CN114201615B (en) | Scientific research data change review method and server based on data snapshot | |
| CN112306594A (en) | Rendering method and device based on mobile terminal and storage medium | |
| CN110059278A (en) | Web page configuration method, server and computer readable storage medium | |
| CN115576974A (en) | Data processing method, device, equipment and medium | |
| CN110334302B (en) | Complex Web application front-end runtime analysis method | |
| CN115617780A (en) | Data import method, device, equipment and storage medium | |
| CN110119322B (en) | Data acquisition method and device, computer equipment and readable storage medium | |
| CN108153655A (en) | The detection method and storage medium of the draw call quantity of virtual reality software | |
| CN111488144A (en) | Data processing method and equipment | |
| CN110716804A (en) | Method and device for automatically deleting useless resources, storage medium and electronic equipment | |
| CN118502735A (en) | Editing assistance method, system and electronic device for code editor | |
| CN115033634B (en) | Data collection method, device, electronic device and medium | |
| CN114928566B (en) | Function test method and device of client, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |