CN110659435A - Page data acquisition processing method and device, computer equipment and storage medium - Google Patents
Page data acquisition processing method and device, computer equipment and storage mediumDownload PDFInfo
- Publication number
- CN110659435A CN110659435ACN201910748859.3ACN201910748859ACN110659435ACN 110659435 ACN110659435 ACN 110659435ACN 201910748859 ACN201910748859 ACN 201910748859ACN 110659435 ACN110659435 ACN 110659435A
- Authority
- CN
- China
- Prior art keywords
- page
- data
- event
- collection plug
- identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及计算机技术领域，特别是涉及一种页面数据采集处理方法、装置、计算机设备和存储介质。The present application relates to the field of computer technology, and in particular, to a method, apparatus, computer device and storage medium for collecting and processing page data.
背景技术Background technique
随着计算机技术的迅速发展，出现了许多web应用等互联网产品。在web应用等互联网产品运行的过程中，通常是通过接入在线收集插件，收集用户在操作过程中的页面数据。通常情况下各个应用都有对应的先后执行率。当需要收集web应用中用户操作的最后页面状态时，在线收集插件已经对运行代码进行了第一次收集，当用户未发生操作事件的时候，收集页面内容不被更新为最终状态，当用户结束操作时，这时候无法获取到页面的最终状态，导致页面数据的收集效率较低。With the rapid development of computer technology, many Internet products such as web applications have appeared. In the process of running Internet products such as web applications, the online collection plug-in is usually connected to collect the page data of the user during the operation. Usually, each application has a corresponding sequential execution rate. When it is necessary to collect the last page status of the user operation in the web application, the online collection plug-in has already collected the running code for the first time. When the user does not have an operation event, the content of the collected page is not updated to the final status. When the user ends During operation, the final state of the page cannot be obtained at this time, resulting in low efficiency of page data collection.
现有的网页数据采集方式通常是实时监听网页数据，通过判断标记的元素是否发生变化，以判断网页数据是否发生变化。这种方式需要实时监听数据是否发生变化，计算量较大，占用的计算资源较多，导致网页页面数据的监听和采集的效率较低。The existing webpage data collection method usually monitors the webpage data in real time, and judges whether the webpage data changes by judging whether the marked element changes. This method needs to monitor whether the data changes in real time, which requires a large amount of calculation and occupies a lot of computing resources, resulting in low efficiency of monitoring and collection of web page data.
发明内容SUMMARY OF THE INVENTION
基于此，有必要针对上述技术问题，提供一种能够有效提高页面数据采集效率的页面数据采集处理方法、装置、计算机设备和存储介质。Based on this, it is necessary to provide a method, apparatus, computer equipment and storage medium for collecting and processing page data which can effectively improve the efficiency of collecting page data, aiming at the above technical problems.
一种页面数据采集处理方法，所述方法包括：A method for collecting and processing page data, the method comprising:
发送访问请求至应用服务器，使得所述应用服务器根据所述访问请求获取对应的页面数据并返回至终端；sending an access request to the application server, so that the application server obtains the corresponding page data according to the access request and returns it to the terminal;
接收所述应用服务器返回的页面数据，所述页面数据携带页面标识；receiving page data returned by the application server, where the page data carries a page identifier;
调用初始化的页面收集插件，利用所述页面收集插件监听页面变化数据；Call the initialized page collection plug-in, and use the page collection plug-in to monitor page change data;
当检测到所述页面数据发生变化时，通过所述页面收集插件采集第一页面状态数据；关闭所述页面收集插件的监听端口；When it is detected that the page data changes, collect the first page status data through the page collection plug-in; close the listening port of the page collection plug-in;
当检测到用户触发的操作事件时，获取所述操作事件对应的操作页面数据，将所述操作页面数据与所述第一页面状态数据进行比对；When an operation event triggered by the user is detected, obtain operation page data corresponding to the operation event, and compare the operation page data with the first page status data;
若所述操作页面数据与所述第一页面状态数据不一致，调用页面收集插件收集所述页面操作数据对应的第二页面状态数据；If the operation page data is inconsistent with the first page state data, calling a page collection plug-in to collect the second page state data corresponding to the page operation data;
将所述第一页面状态数据和所述第二页面状态数据根据所述页面标识上传至运维服务器，使得所述运维服务器根据页面状态数据对所述页面数据进行分析。Uploading the first page status data and the second page status data to the operation and maintenance server according to the page identifier, so that the operation and maintenance server analyzes the page data according to the page status data.
在其中一个实施例中，所述通过所述页面收集插件采集第一页面状态数据的步骤包括：通过所述页面收集插件读取所述页面数据的文档结构数据；根据所述访问请求对所述文档结构数据和相应的元素节点进行遍历采集，并对所述文档结构数据添加第一读取标签；所述第一读取标签包括多个元素属性；获取所述页面数据的初始元素标签，根据多个元素属性将所述第一读取标签与所述初始元素标签进行比对；当所述第一读取标签与所述初始元素标签不一致时，则确定所述页面数据存在变化，并根据所述第一读取标签生成第一页面状态数据。In one embodiment, the step of collecting the first page status data through the page collection plug-in includes: reading the document structure data of the page data through the page collection plug-in; The document structure data and corresponding element nodes are traversed and collected, and a first read tag is added to the document structure data; the first read tag includes a plurality of element attributes; the initial element tag of the page data is obtained, according to Multiple element attributes compare the first read tag with the initial element tag; when the first read tag is inconsistent with the initial element tag, it is determined that the page data has changed, and according to The first read tag generates first page status data.
在其中一个实施例中，所述将所述页面操作数据与所述第一页面状态数据进行比对的步骤包括：检测所述操作事件的事件类型，获取所述操作页面数据的操作页面标识；将所述页面数据的页面标识与所述操作页面数据的操作页面标识进行比对；当所述事件类型为动态事件，且所述页面标识与所述操作页面标识不一致时，确定所述页面数据存在变化。In one embodiment, the step of comparing the page operation data with the first page status data includes: detecting an event type of the operation event, and acquiring an operation page identifier of the operation page data; Compare the page identifier of the page data with the operation page identifier of the operation page data; when the event type is a dynamic event and the page identifier is inconsistent with the operation page identifier, determine the page data There are changes.
在其中一个实施例中，所述第一页面状态数据包括状态属性信息，所述方法还包括：当所述事件类型为动态事件，且所述页面标识与所述操作页面标识一致时，根据所述操作事件获取当前帧的界面显示数据；所述界面显示数据包括时间标识；获取上一帧的界面显示数据，根据上一帧的界面显示数据和当前帧的界面显示数据的时间标识计算上一帧的界面显示数据的访问时长；将所述访问时长添加至所述第一页面状态数据的状态属性信息中。In one embodiment, the first page status data includes status attribute information, and the method further includes: when the event type is a dynamic event and the page identifier is consistent with the operation page identifier, according to the The operation event obtains the interface display data of the current frame; the interface display data includes a time stamp; obtains the interface display data of the previous frame, and calculates the previous frame according to the interface display data of the previous frame and the time stamp of the interface display data of the current frame. The interface of the frame displays the access duration of the data; the access duration is added to the state attribute information of the first page state data.
在其中一个实施例中，所述方法还包括：检测所述页面数据中页面数据类型；当所述页面数据中存在动画数据时，通过所述页面收集插件采集第一页面状态数据后，关闭所述页面收集插件的监听端口；当检测到用户触发的操作事件时，获取所述操作事件的操作类型；若所述操作类型为第一预设类型，保持所述页面收集插件的监听端口关闭；若所述操作类型为第二预设类型，获取所述操作事件对应的第二页面状态数据。In one embodiment, the method further includes: detecting the page data type in the page data; when there is animation data in the page data, after collecting the first page status data through the page collection plug-in, closing all When detecting the operation event triggered by the user, obtain the operation type of the operation event; if the operation type is the first preset type, keep the listening port of the page collection plug-in closed; If the operation type is the second preset type, acquire second page status data corresponding to the operation event.
一种页面数据采集处理装置，所述装置包括：A device for collecting and processing page data, the device comprising:
请求发送模块，用于发送访问请求至应用服务器，使得所述应用服务器根据所述访问请求获取对应的页面数据并返回至终端；a request sending module, configured to send an access request to an application server, so that the application server obtains corresponding page data according to the access request and returns it to the terminal;
数据接收模块，用于接收所述应用服务器返回的页面数据，所述页面数据携带页面标识；a data receiving module, configured to receive page data returned by the application server, where the page data carries a page identifier;
页面采集模块，用于调用初始化的页面收集插件，利用所述页面收集插件监听页面变化数据；当检测到所述页面数据发生变化时，通过所述页面收集插件采集第一页面状态数据；关闭所述页面收集插件的监听端口；The page collection module is used to call the initialized page collection plug-in, and use the page collection plug-in to monitor page change data; when it is detected that the page data changes, collect the first page status data through the page collection plug-in; close all The listening port of the above-mentioned page collection plug-in;
数据比对模块，用于当检测到用户触发的操作事件时，获取所述操作事件对应的操作页面数据，将所述操作页面数据与所述第一页面状态数据进行比对；a data comparison module, configured to acquire operation page data corresponding to the operation event when an operation event triggered by the user is detected, and compare the operation page data with the first page status data;
所述页面采集模块还用于若所述操作页面数据与所述第一页面状态数据不一致，调用页面收集插件收集所述页面操作数据对应的第二页面状态数据；The page collection module is further configured to call a page collection plug-in to collect the second page state data corresponding to the page operation data if the operation page data is inconsistent with the first page state data;
数据上传模块，用于将所述第一页面状态数据和所述第二页面状态数据根据所述页面标识上传至运维服务器，使得所述运维服务器根据页面状态数据对所述页面数据进行分析。A data uploading module, configured to upload the first page status data and the second page status data to the operation and maintenance server according to the page identifier, so that the operation and maintenance server analyzes the page data according to the page status data .
在其中一个实施例中，所述页面采集模块还用于通过所述页面收集插件读取所述页面数据的文档结构数据；根据所述访问请求对所述文档结构数据和相应的元素节点进行遍历采集，并对所述文档结构数据添加第一读取标签；所述第一读取标签包括多个元素属性；获取所述页面数据的初始元素标签，根据多个元素属性将所述第一读取标签与所述初始元素标签进行比对；当所述第一读取标签与所述初始元素标签不一致时，则确定所述页面数据存在变化，并根据所述第一读取标签生成第一页面状态数据。In one embodiment, the page collection module is further configured to read the document structure data of the page data through the page collection plug-in; traverse the document structure data and corresponding element nodes according to the access request collecting, and adding a first read tag to the document structure data; the first read tag includes a plurality of element attributes; acquiring the initial element tag of the page data, and assigning the first read tag according to the plurality of element attributes Compare the label with the initial element label; when the first read label is inconsistent with the initial element label, determine that the page data has changed, and generate a first read label according to the first read label. Page state data.
在其中一个实施例中，所述页面采集模块还用于检测所述页面数据中页面数据类型；当所述页面数据中存在动画数据时，通过所述页面收集插件采集第一页面状态数据后，关闭所述页面收集插件的监听端口；当检测到用户触发的操作事件时，获取所述操作事件的操作类型；若所述操作类型为第一预设类型，保持所述页面收集插件的监听端口关闭；若所述操作类型为第二预设类型，获取所述操作事件对应的第二页面状态数据。In one embodiment, the page collection module is further configured to detect the page data type in the page data; when there is animation data in the page data, after collecting the first page status data through the page collection plug-in, Close the listening port of the page collection plug-in; when detecting an operation event triggered by the user, obtain the operation type of the operation event; if the operation type is the first preset type, keep the listening port of the page collection plug-in Close; if the operation type is the second preset type, acquire the second page status data corresponding to the operation event.
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现本申请任意一个实施例中提供的页面数据采集处理方法的步骤。A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the processor implements the steps of the method for collecting and processing page data provided in any embodiment of the present application.
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现本申请任意一个实施例中提供的页面数据采集处理方法的步骤。A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method for collecting and processing page data provided in any embodiment of the present application.
上述页面数据采集处理方法、装置、计算机设备和存储介质，终端接收应用服务器根据访问请求至返回的页面数据后，则调用初始化的页面收集插件，利用页面收集插件监听页面变化数据，通过预设的插件监听页面的操作事件和变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态。当检测到页面数据发生变化时，通过页面收集插件采集第一页面状态数据后，则关闭页面收集插件的监听端口，通过关闭页面收集插件的监听端口，能够有效避免一直监听，从而能够有效节省页面的资源占用率。当检测到用户触发的操作事件时，获取操作事件对应的操作页面数据，将操作页面数据与第一页面状态数据进行比对。若操作页面数据与第一页面状态数据不一致，再调用页面收集插件收集页面操作数据对应的第二页面状态数据，并将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器，使得运维服务器根据页面状态数据对页面数据进行分析。通过预设的页面收集插件监听页面的操作事件和采集变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态，同时也优化了收集过程的计算量，从而有效地提高了页面数据采集效率。In the above method, device, computer equipment and storage medium for collecting and processing page data, after receiving the page data returned by the application server according to the access request, the terminal calls the initialized page collection plug-in, uses the page collection plug-in to monitor page change data, and uses the preset page collection plug-in to monitor page change data. The plug-in monitors the operation events and change data of the page, and can effectively collect the page data and the final state of the page during the operation of the user. When it is detected that the page data has changed, after collecting the first page status data through the page collection plug-in, the monitoring port of the page collection plug-in is closed. resource occupancy rate. When an operation event triggered by the user is detected, the operation page data corresponding to the operation event is obtained, and the operation page data is compared with the first page state data. If the operation page data is inconsistent with the first page status data, then call the page collection plug-in to collect the second page status data corresponding to the page operation data, and upload the first page status data and the second page status data to the operation and maintenance server according to the page ID , so that the operation and maintenance server analyzes the page data according to the page status data. The preset page collection plug-in monitors the operation events of the page and collects the change data, which can effectively collect the page data and the final state of the page during the operation of the user, and also optimize the calculation amount of the collection process, thereby effectively improving the Page data collection efficiency.
附图说明Description of drawings
图1为一个实施例中页面数据采集处理方法的应用场景图；1 is an application scenario diagram of a method for collecting and processing page data in one embodiment;
图2为一个实施例中页面数据采集处理方法的流程示意图；2 is a schematic flowchart of a method for collecting and processing page data in one embodiment;
图3为一个实施例中通过页面收集插件采集页面状态数据的步骤的流程示意图；3 is a schematic flowchart of a step of collecting page status data through a page collection plug-in in one embodiment;
图4为一个实施例中页面数据采集处理装置的结构框图；4 is a structural block diagram of an apparatus for collecting and processing page data in one embodiment;
图5为一个实施例中计算机设备的内部结构图。FIG. 5 is a diagram of the internal structure of a computer device in one embodiment.
具体实施方式Detailed ways
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.
本申请提供的页面数据采集处理方法，可以应用于如图1所示的应用环境中。其中，终端102通过网络与应用服务器104进行通信，终端102通过网络与运维服务器106进行通信，应用服务器104通过网络与运维服务器106进行通信。其中，终端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，应用服务器104以及运维服务器106可以用独立的服务器或者是多个服务器组成的服务器集群来实现。The method for collecting and processing page data provided by the present application can be applied to the application environment shown in FIG. 1 . The terminal 102 communicates with the application server 104 through the network, the terminal 102 communicates with the operation and maintenance server 106 through the network, and the application server 104 communicates with the operation and maintenance server 106 through the network. Wherein, the terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the application server 104 and the operation and maintenance server 106 can be an independent server or a server cluster composed of multiple servers to fulfill.
在一个实施例中，如图2所示，提供了一种页面数据采集处理方法，以该方法应用于图1中的终端为例进行说明，包括以下步骤：In one embodiment, as shown in FIG. 2 , a method for collecting and processing page data is provided, and the method is applied to the terminal in FIG. 1 as an example for description, including the following steps:
步骤202，发送访问请求至应用服务器，使得应用服务器根据访问请求获取对应的页面数据并返回至终端。Step 202: Send an access request to the application server, so that the application server acquires corresponding page data according to the access request and returns it to the terminal.
步骤204，接收应用服务器返回的页面数据，页面数据携带页面标识。Step 204: Receive page data returned by the application server, where the page data carries the page identifier.
在Web应用、APP(Application，应用程序)以及网页等应用中，可以预先部署页面采集插件。当用户首次访问web网页时，用户可以通过对应的终端向应用服务器发送访问请求，访问请求中携带了页面标识和用户标识。应用服务器接收到用户终端发送的访问请求后，根据访问请求中携带的页面标识和用户标识向用户终端返回对应的页面数据。其中，页面数据可以是网页数据、应用数据等，页面数据中可以包括多种类型的数据，例如可以包括文本、图片、音频以及视频等多种形式的页面数据。In applications such as a web application, an APP (Application, application program), and a web page, a page collection plug-in may be pre-deployed. When a user accesses a web page for the first time, the user may send an access request to the application server through a corresponding terminal, and the access request carries the page identifier and the user identifier. After receiving the access request sent by the user terminal, the application server returns corresponding page data to the user terminal according to the page identifier and the user identifier carried in the access request. The page data may be web page data, application data, etc. The page data may include various types of data, for example, may include page data in various forms such as text, pictures, audio, and video.
步骤206，调用初始化的页面收集插件，利用页面收集插件监听页面变化数据。
其中，插件是一种遵循一定规范的应用程序接口编写的程序，会随应用的启动自动执行。页面收集插件可以是预先部署在应用中的插件。例如在浏览器中安装相关的插件后，WEB浏览器能够直接调用插件程序，用于处理特定类型的文件。Among them, a plug-in is a program written in accordance with a certain specification of an application program interface, and will be automatically executed when the application is started. The page collection plugin can be a plugin that is pre-deployed in the application. For example, after installing a related plug-in in the browser, the WEB browser can directly call the plug-in program to process files of a specific type.
终端在发送访问请求至应用服务器后，应用服务器则初始化对应的页面收集插件，并获取访问请求对应的页面数据返回至终端。终端接收应用服务器返回的页面数据后，则调用初始化的页面收集插件，通过初始化页面收集插件调用监听方法采集并监听页面变化数据。例如监听页面的Dom(Document Object Model，文档对象模型)数据是否发生变更。After the terminal sends the access request to the application server, the application server initializes the corresponding page collection plug-in, and obtains the page data corresponding to the access request and returns it to the terminal. After receiving the page data returned by the application server, the terminal calls the initialized page collection plug-in, and calls the monitoring method through the initialized page collection plug-in to collect and monitor page change data. For example, monitor whether the Dom (Document Object Model, document object model) data of the page is changed.
其中，终端接收应用服务器返回的页面数据中包括状态数据，终端可以获取页面数据中的初始状态数据。The page data returned by the terminal receiving the application server includes status data, and the terminal may acquire initial status data in the page data.
步骤208，当检测到页面数据发生变化时，通过页面收集插件采集第一页面状态数据；关闭页面收集插件的监听端口。
当检测到页面发生变化时，通过页面收集插件获取第一当前页面数据。例如可以是MutationObserver监听方法，监听内容可以包括页面url，页面源代码、页面停留时间和页面内容等数据。通过页面收集插件获取当前页面状态数据后，终端则通过关闭页面收集插件的监听端口，以停止自动监听。When it is detected that the page has changed, the first current page data is acquired through the page collection plug-in. For example, it can be a MutationObserver monitoring method, and the monitoring content can include data such as page url, page source code, page dwell time, and page content. After obtaining the current page status data through the page collection plug-in, the terminal closes the listening port of the page collection plug-in to stop automatic monitoring.
其中，当页面数据未发生变化时，终端可以将初始状态数据作为第一页面状态数据。第一页面状态数据中包括了对应的状态属性信息，例如用户访问时间、访问时长以及操作事件类型等状态属性信息。Wherein, when the page data does not change, the terminal may use the initial state data as the first page state data. The first page state data includes corresponding state attribute information, such as user access time, access duration, and operation event type and other state attribute information.
步骤210，当检测到用户触发的操作事件时，获取操作事件对应的操作页面数据，将操作页面数据与第一页面状态数据进行比对。
其中，操作页面数据可以是用户通过终端触发操作事件后基于操作事件所产生的页面数据。The operation page data may be page data generated based on the operation event after the user triggers the operation event through the terminal.
终端获取页面的操作事件，当检测到操作事件后，获取操作事件对应的页面操作数据，将页面操作数据和第一当前页面数据进行比较。具体地，终端检测操作事件的事件类型，获取操作页面数据的操作页面标识，将页面数据的页面标识与操作页面数据的操作页面标识进行比对。当事件类型为动态事件，且页面标识与操作页面标识不一致时，确定页面数据存在变化。当页面标识与操作页面标识一致时，表示页面数据没有发生变化。The terminal acquires the operation event of the page, and after detecting the operation event, acquires the page operation data corresponding to the operation event, and compares the page operation data with the first current page data. Specifically, the terminal detects the event type of the operation event, obtains the operation page identifier of the operation page data, and compares the page identifier of the page data with the operation page identifier of the operation page data. When the event type is a dynamic event and the page identifier is inconsistent with the operation page identifier, it is determined that there is a change in the page data. When the page ID is consistent with the operation page ID, it means that the page data has not changed.
步骤212，若操作页面数据与第一页面状态数据不一致，调用页面收集插件收集页面操作数据对应的第二页面状态数据。
步骤214，将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器，使得运维服务器根据页面状态数据对页面数据进行分析。Step 214: Upload the first page status data and the second page status data to the operation and maintenance server according to the page identifier, so that the operation and maintenance server analyzes the page data according to the page status data.
若操作页面数据与第一页面状态数据不一致，表示页面数据发生了变化，终端则调用页面收集插件收集页面操作数据对应的第二页面状态数据。并将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器。If the operation page data is inconsistent with the first page state data, it means that the page data has changed, and the terminal calls the page collection plug-in to collect the second page state data corresponding to the page operation data. And upload the first page status data and the second page status data to the operation and maintenance server according to the page identifier.
进一步地，终端则持续根据用户的操作采集相应的页面状态数据，直到检测到终端关闭或退出页面访问，终端则停止页采集面状态数据，并将采集到的页面状态数据实时上传至运维服务器。通过预设的页面收集插件监听页面的操作事件和采集变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态，同时也优化了收集过程的计算量。Further, the terminal continues to collect the corresponding page status data according to the user's operation, until it detects that the terminal is closed or exits the page access, the terminal stops collecting the page status data, and uploads the collected page status data to the operation and maintenance server in real time. . The preset page collection plug-in monitors the operation events of the page and collects the change data, which can effectively collect the page data and the final state of the page during the operation of the user, and also optimize the calculation amount of the collection process.
终端将收集的页面数据上传至运维服务器后，运维服务器可以对页面数据进行页面还原分析，分析用户的操作流程是否存在问题，当分析出用户操作中存在问题是，则根据操作问题生成对应的操作提示信息，并将操作提示信息发送至用户终端，以使得用户终端根据操作提示信息进行进一步操作。运维服务器还可以获取多个用户终端的操作数据和页面数据，并对大量的操作数据和页面数据进行分析，分析该互联网产品或页面中在运行过程中的操作指标数据，根据操作指标数据进一步分析该互联网产品或页面存在的问题以及可以优化的地方，以使得开发人员进一步对该互联网产品或页面进行优化。After the terminal uploads the collected page data to the operation and maintenance server, the operation and maintenance server can perform page restoration analysis on the page data, and analyze whether there are problems in the user's operation process. and send the operation prompt information to the user terminal, so that the user terminal can perform further operations according to the operation prompt information. The operation and maintenance server can also obtain the operation data and page data of multiple user terminals, analyze a large amount of operation data and page data, analyze the operation index data in the operation process of the Internet product or page, and further analyze the operation index data according to the operation index data. Analyze the problems existing in the Internet product or page and places that can be optimized, so that developers can further optimize the Internet product or page.
上述页面数据采集处理方法中，终端接收应用服务器根据访问请求至返回的页面数据后，则调用初始化的页面收集插件，利用页面收集插件监听页面变化数据，通过预设的插件监听页面的操作事件和变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态。当检测到页面数据发生变化时，通过页面收集插件采集第一页面状态数据后，则关闭页面收集插件的监听端口，通过关闭页面收集插件的监听端口，能够有效避免一直监听，从而能够有效节省页面的资源占用率。当检测到用户触发的操作事件时，获取操作事件对应的操作页面数据，将操作页面数据与第一页面状态数据进行比对。若操作页面数据与第一页面状态数据不一致，再调用页面收集插件收集页面操作数据对应的第二页面状态数据，并将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器，使得运维服务器根据页面状态数据对页面数据进行监控分析。通过预设的页面收集插件监听页面的操作事件和采集变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态，同时也优化了收集过程的计算量，从而有效地提高了页面数据采集效率。In the above page data collection and processing method, after receiving the page data returned by the application server according to the access request, the terminal calls the initialized page collection plug-in, uses the page collection plug-in to monitor page change data, and uses the preset plug-in to monitor page operation events and page changes. The change data can effectively collect the page data and the final state of the page during the operation of the user. When it is detected that the page data has changed, after collecting the first page status data through the page collection plug-in, the monitoring port of the page collection plug-in is closed. resource occupancy rate. When an operation event triggered by the user is detected, the operation page data corresponding to the operation event is obtained, and the operation page data is compared with the first page state data. If the operation page data is inconsistent with the first page status data, then call the page collection plug-in to collect the second page status data corresponding to the page operation data, and upload the first page status data and the second page status data to the operation and maintenance server according to the page ID , so that the operation and maintenance server monitors and analyzes the page data according to the page status data. The preset page collection plug-in monitors the operation events of the page and collects the change data, which can effectively collect the page data and the final state of the page during the operation of the user, and also optimize the calculation amount of the collection process, thereby effectively improving the Page data collection efficiency.
在一个实施例中，如图3所示，通过页面收集插件采集第一页面状态数据的步骤包括：In one embodiment, as shown in FIG. 3 , the step of collecting the state data of the first page through the page collection plug-in includes:
步骤302，通过页面收集插件读取页面数据的文档结构数据。Step 302: Read the document structure data of the page data through the page collection plug-in.
步骤304，根据访问请求对文档结构数据和相应的元素节点进行遍历采集，并对文档结构数据添加第一读取标签；第一读取标签包括多个元素属性。Step 304 , traverse and collect the document structure data and corresponding element nodes according to the access request, and add a first read tag to the document structure data; the first read tag includes a plurality of element attributes.
步骤306，获取页面数据的初始元素标签，根据多个元素属性将第一读取标签与初始元素标签进行比对。Step 306: Obtain the initial element label of the page data, and compare the first read label with the initial element label according to multiple element attributes.
步骤308，当第一读取标签与初始元素标签不一致时，则确定页面数据存在变化，并根据第一读取标签生成第一页面状态数据。Step 308 , when the first read tag is inconsistent with the initial element tag, it is determined that there is a change in the page data, and first page status data is generated according to the first read tag.
其中，文档结构数据可以是指页面数据中的文档对象模型，即DOM(DocumentObject Model，文档对象模型)，文档结构数据中包括多个元素节点。The document structure data may refer to a document object model in the page data, that is, a DOM (Document Object Model, document object model), and the document structure data includes a plurality of element nodes.
终端在发送访问请求至应用服务器后，应用服务器则初始化对应的页面收集插件，并获取访问请求对应的页面数据返回至终端。终端接收应用服务器返回的页面数据后，则调用初始化的页面收集插件，通过初始化页面收集插件调用监听方法采集并监听页面变化数据。After the terminal sends the access request to the application server, the application server initializes the corresponding page collection plug-in, and obtains the page data corresponding to the access request and returns it to the terminal. After receiving the page data returned by the application server, the terminal calls the initialized page collection plug-in, and calls the monitoring method through the initialized page collection plug-in to collect and monitor page change data.
具体地，终端通过页面收集插件读取页面数据的文档结构数据，根据访问请求对文档结构数据和相应的元素节点进行遍历采集，并对文档结构数据添加第一读取标签；第一读取标签包括多个元素属性。终端进而获取页面数据的初始元素标签，根据多个元素属性将第一读取标签与初始元素标签进行比对。例如可以采用MutationObserver监听方法，通过检测页面的DOM文档结构数据，对网页中的文档结构数据和元素节点进行遍历采集，将采集的页面数据添加页面读取标识，页面读取标识可以包括元素标识、终端标识和读取时间标识等。终端则根据元素标识、终端标识和读取时间标识等多个元素将所第一读取标签与初始元素标签进行比对。Specifically, the terminal reads the document structure data of the page data through the page collection plug-in, traverses and collects the document structure data and corresponding element nodes according to the access request, and adds a first read tag to the document structure data; the first read tag Include multiple element attributes. The terminal further obtains the initial element label of the page data, and compares the first read label with the initial element label according to a plurality of element attributes. For example, the MutationObserver monitoring method can be used to traverse and collect the document structure data and element nodes in the webpage by detecting the DOM document structure data of the page, and add the page read mark to the collected page data. The page read mark can include element identification, Terminal identification and reading time stamp, etc. The terminal compares the first read tag with the initial element tag according to multiple elements such as the element identifier, the terminal identifier, and the read time identifier.
终端进一步的，当第一读取标签与初始元素标签不一致时，表示当前的页面数据已经发生了变化，则确定该页面数据存在变化，终端则根据第一读取标签生成第一页面状态数据。其中，当第一读取标签与初始元素标签一致时，表示页面数据未发生变化时，终端可以将初始状态数据作为第一状态数据。通过利用页面收集插件监听页面变化数据，能够监听页面的有效地监听页面中的操作事件和变化数据，从而能够有效地收集用户在操作过程中的页面数据和页面的最终状态。The terminal further, when the first read tag is inconsistent with the initial element tag, indicating that the current page data has changed, it is determined that the page data has changed, and the terminal generates the first page status data according to the first read tag. Wherein, when the first read tag is consistent with the initial element tag, indicating that the page data has not changed, the terminal may use the initial state data as the first state data. By using the page collection plug-in to monitor the page change data, the page can be effectively monitored for operation events and change data in the page, so that the page data and the final state of the page during the user's operation can be effectively collected.
在一个实施例中，将页面操作数据与第一页面状态数据进行比对的步骤包括：检测操作事件的事件类型，获取操作页面数据的操作页面标识；将页面数据的页面标识与操作页面数据的操作页面标识进行比对；当事件类型为动态事件，且页面标识与操作页面标识不一致时，确定页面数据存在变化。In one embodiment, the step of comparing the page operation data with the first page status data includes: detecting the event type of the operation event, obtaining the operation page identifier of the operation page data; comparing the page identifier of the page data with the operation page data identifier The operation page identifier is compared; when the event type is a dynamic event and the page identifier is inconsistent with the operation page identifier, it is determined that there is a change in the page data.
操作事件是指用户通过终端的用输入设备触发一些操作，会影响到窗口的状态，触发窗口发生相应的事件响应，例如常见的鼠标、键盘操作，如鼠标点击事件触发进入某个元素。其中，动态操作事件可以指使窗口数据发生变化的操作，例如鼠标点击按钮操作、表单事件、页面滑动操作等；静态操作事件则指对窗口数据不会产生影响的无执行操作，例如鼠标点击空白区域的操作等。终端可以预先配置操作事件的类型，例如将按钮点击事件、元素点击事件、表单触发事件以及页面滑动事件等配置为动态操作事件的类型，将空白区域点击事件等配置为静态操作事件的类型，并根据配置操作事件类型生成事件类型配置表。An operation event means that the user triggers some operations through the input device of the terminal, which will affect the state of the window and trigger the corresponding event response in the window, such as common mouse and keyboard operations, such as mouse click events that trigger entry to an element. Among them, dynamic operation events can refer to operations that change window data, such as mouse clicks on buttons, form events, page sliding operations, etc.; static operation events refer to no-execution operations that do not affect window data, such as mouse clicks on blank areas operation, etc. The terminal can pre-configure the types of operation events, for example, configure button click events, element click events, form trigger events, and page sliding events as dynamic operation event types, and configure blank area click events as static operation event types. Generates an event type configuration table based on the configuration action event type.
终端在发送访问请求至应用服务器后，应用服务器则初始化对应的页面收集插件，并获取访问请求对应的页面数据返回至终端。终端接收应用服务器返回的页面数据后，则调用初始化的页面收集插件，通过初始化页面收集插件调用监听方法采集并监听页面变化数据。当检测到页面发生变化时，通过页面收集插件获取第一页面状态数据。通过页面收集插件获取当前页面状态数据后，终端则通过关闭页面收集插件的监听端口，以停止自动监听。After the terminal sends the access request to the application server, the application server initializes the corresponding page collection plug-in, and obtains the page data corresponding to the access request and returns it to the terminal. After receiving the page data returned by the application server, the terminal calls the initialized page collection plug-in, and calls the monitoring method through the initialized page collection plug-in to collect and monitor page change data. When a page change is detected, the first page state data is obtained through the page collection plug-in. After obtaining the current page status data through the page collection plug-in, the terminal closes the listening port of the page collection plug-in to stop automatic monitoring.
终端在访问页面数据的过程中，会触发一些操作行为。当终端检测到用户触发了操作事件时，终端获取用户基于该页面触发的操作事件，并获取操作事件对应的页面操作数据。其中，操作页面数据可以是用户通过终端触发操作事件后基于操作事件所产生的页面数据。终端则将页面操作数据和第一当前页面数据进行比较。In the process of accessing page data, the terminal will trigger some operation behaviors. When the terminal detects that the user triggers the operation event, the terminal obtains the operation event triggered by the user based on the page, and obtains the page operation data corresponding to the operation event. The operation page data may be page data generated based on the operation event after the user triggers the operation event through the terminal. The terminal compares the page operation data with the first current page data.
具体地，终端检测操作事件的事件类型，并获取预置的事件类型配置表，根据事件类型配置表匹配该操作事件的类型属性，即根据事件类型配置表判断该操作事件为动态操作事件还是静态操作事件。终端进一步获取操作页面数据的操作页面标识，将页面数据的页面标识与操作页面数据的操作页面标识进行比对，即判断操作事件之后的当前页面数据与操作事件之前的上一页面数据相比较是否发生了变化。当事件类型为动态事件，且页面标识与操作页面标识不一致时，表示用户的操作事件执行使得页面数据发生了变化，则确定页面数据存在变化。Specifically, the terminal detects the event type of the operation event, obtains a preset event type configuration table, matches the type attribute of the operation event according to the event type configuration table, that is, determines whether the operation event is a dynamic operation event or a static operation event according to the event type configuration table Action events. The terminal further obtains the operation page identifier of the operation page data, and compares the page identifier of the page data with the operation page identifier of the operation page data, that is, to determine whether the current page data after the operation event is compared with the previous page data before the operation event. has changed. When the event type is a dynamic event and the page identifier is inconsistent with the operation page identifier, it means that the page data has changed due to the user's operation event execution, and it is determined that the page data has changed.
若操作页面数据与第一页面状态数据不一致，表示页面数据发生了变化，终端则调用页面收集插件收集页面操作数据对应的第二页面状态数据，并将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器。通过预设的页面收集插件监听页面的操作事件和采集变化数据，能够有效地收集用户在操作过程中的页面数据和页面的最终状态，同时也优化了收集过程的计算量。If the operation page data is inconsistent with the first page status data, it means that the page data has changed. The terminal will call the page collection plug-in to collect the second page status data corresponding to the page operation data, and combine the first page status data with the second page status data. Upload to the operation and maintenance server according to the page ID. The preset page collection plug-in monitors the operation events of the page and collects the change data, which can effectively collect the page data and the final state of the page during the operation of the user, and also optimize the calculation amount of the collection process.
在一个实施例中，第一页面状态数据包括状态属性信息，该方法还包括：当事件类型为动态事件，且页面标识与操作页面标识一致时，根据操作事件获取当前帧的界面显示数据；界面显示数据包括时间标识；获取上一帧的界面显示数据，根据上一帧的界面显示数据和当前帧的界面显示数据的时间标识计算上一帧的界面显示数据的访问时长；将访问时长添加至第一页面状态数据的状态属性信息中。In one embodiment, the first page state data includes state attribute information, and the method further includes: when the event type is a dynamic event and the page identifier is consistent with the operation page identifier, acquiring the interface display data of the current frame according to the operation event; the interface The display data includes a time stamp; obtain the interface display data of the previous frame, and calculate the access duration of the interface display data of the previous frame according to the interface display data of the previous frame and the time stamp of the interface display data of the current frame; add the access duration to in the state attribute information of the state data of the first page.
其中，用户在访问页面数据的过程中，当访问同一个当前的页面数据时，可能会触发多次页面滑动操作，例如包括多个文本、图片的页面数据。终端每次执行用户触发的页面滑动操作后，显示在终端界面的页面数据会发生变化，用户停留在每一帧的界面显示数据则不同。Among them, in the process of accessing page data, when a user accesses the same current page data, multiple page sliding operations may be triggered, such as page data including multiple texts and pictures. Each time the terminal performs a page sliding operation triggered by the user, the page data displayed on the terminal interface will change, and the interface display data where the user stays in each frame will be different.
终端接收应用服务器返回的页面数据后，则调用初始化的页面收集插件，通过初始化页面收集插件调用监听方法采集并监听页面变化数据。当检测到页面发生变化时，通过页面收集插件获取第一页面状态数据。通过页面收集插件获取当前页面状态数据后，终端则通过关闭页面收集插件的监听端口，以停止自动监听。After receiving the page data returned by the application server, the terminal calls the initialized page collection plug-in, and calls the monitoring method through the initialized page collection plug-in to collect and monitor page change data. When a page change is detected, the first page state data is obtained through the page collection plug-in. After obtaining the current page status data through the page collection plug-in, the terminal closes the listening port of the page collection plug-in to stop automatic monitoring.
终端在访问页面数据的过程中，会触发一些操作行为。当终端检测到用户触发了操作事件时，终端获取用户基于该页面触发的操作事件，并获取操作事件对应的页面操作数据。其中，操作页面数据可以是用户通过终端触发操作事件后基于操作事件所产生的页面数据。终端则将页面操作数据和第一当前页面数据进行比较。In the process of accessing page data, the terminal will trigger some operation behaviors. When the terminal detects that the user triggers the operation event, the terminal obtains the operation event triggered by the user based on the page, and obtains the page operation data corresponding to the operation event. The operation page data may be page data generated based on the operation event after the user triggers the operation event through the terminal. The terminal compares the page operation data with the first current page data.
具体地，终端检测操作事件的事件类型，并获取预置的事件类型配置表，根据事件类型配置表匹配该操作事件的类型属性，即根据事件类型配置表判断该操作事件为动态操作事件还是静态操作事件。终端进一步获取操作页面数据的操作页面标识，将页面数据的页面标识与操作页面数据的操作页面标识进行比对，即判断操作事件之后的当前页面数据与操作事件之前的上一页面数据相比较是否发生了变化。Specifically, the terminal detects the event type of the operation event, obtains a preset event type configuration table, matches the type attribute of the operation event according to the event type configuration table, that is, determines whether the operation event is a dynamic operation event or a static operation event according to the event type configuration table Action events. The terminal further obtains the operation page identifier of the operation page data, and compares the page identifier of the page data with the operation page identifier of the operation page data, that is, to determine whether the current page data after the operation event is compared with the previous page data before the operation event. has changed.
当事件类型为动态事件中的页面滑动操作，且当页面标识与操作页面标识一致时，表示用户的操作事件执行并未导致页面数据发生变化，但可能显示在终端界面的界面显示数据发生了变化，如基于用户的页面滑动操作响应的页面显示数据，终端可以收集用户滑动的界面显示数据。具体地，当检测到操作事件为页面滑动操作，且页面标识与操作页面标识一致时，终端根据页面滑动事件获取当前页面帧的界面显示数据，界面显示数据包括页面显示区域和时间标识。终端获取上一帧的界面显示数据，根据上一帧的界面显示数据和当前帧的界面显示数据的时间标识计算上一帧的界面显示数据的访问时长，即计算出当前帧的页面显示数据和上一帧页面显示数据之间的间隔时间。终端则根据当前帧的页面数据生成相应的第一页面状态数据，并将页面显示区域和访问时长添加至第一页面状态数据的状态属性信息中。由此能够有效地获取用户滑动的界面显示数据和访问时长，能够有效地收集用户在操作过程中的页面数据和访问状态信息，进而使得运维服务器根据页面状态数据有效地对页面数据进行监控分析。When the event type is the page sliding operation in the dynamic event, and when the page ID is the same as the operation page ID, it means that the user's operation event execution does not cause the page data to change, but the interface display data that may be displayed on the terminal interface has changed. For example, based on the page display data responded to by the user's page sliding operation, the terminal may collect the interface display data of the user's sliding operation. Specifically, when it is detected that the operation event is a page sliding operation and the page identifier is consistent with the operation page identifier, the terminal obtains the interface display data of the current page frame according to the page sliding event, and the interface display data includes the page display area and the time identifier. The terminal obtains the interface display data of the previous frame, and calculates the access duration of the interface display data of the previous frame according to the interface display data of the previous frame and the time stamp of the interface display data of the current frame, that is, the page display data and The previous frame page displays the interval time between data. The terminal generates corresponding first page status data according to the page data of the current frame, and adds the page display area and the access duration to the status attribute information of the first page status data. In this way, the user's sliding interface display data and access duration can be effectively obtained, and the user's page data and access status information during the operation process can be effectively collected, so that the operation and maintenance server can effectively monitor and analyze the page data according to the page status data. .
在一个实施例中，该方法还包括：检测页面数据中页面数据类型；当页面数据中存在动画数据时，通过页面收集插件采集第一页面状态数据后，关闭页面收集插件的监听端口；当检测到用户触发的操作事件时，获取操作事件的操作类型；若操作类型为第一预设类型，保持页面收集插件的监听端口关闭；若操作类型为第二预设类型，获取操作事件对应的第二页面状态数据。In one embodiment, the method further includes: detecting the page data type in the page data; when there is animation data in the page data, after collecting the first page status data through the page collection plug-in, closing the listening port of the page collection plug-in; when detecting When the operation event triggered by the user is reached, obtain the operation type of the operation event; if the operation type is the first preset type, keep the listening port of the page collection plug-in closed; if the operation type is the second preset type, obtain the first preset type corresponding to the operation event. Two pages of status data.
其中，页面数据中可能包括动态的动画数据，例如音频、视频以及动图等动态数据，动画数据可以在页面中自动执行播放。通常动画数据在自动播放的过程中，页面数据中的其他元素数据不会发生变化。The page data may include dynamic animation data, such as audio, video, and dynamic data, and the animation data can be automatically played on the page. Usually, during the automatic playback of animation data, other element data in the page data will not change.
终端向应用服务器发送访问请求，并接收到应用服务器根据访问请求返回的页面数据后，检测页面数据中页面数据类型。当检测到页面数据中存在动画数据时，终端则调用初始化的页面收集插件，通过初始化页面收集插件调用监听方法采集并监听页面变化数据，并收集包含动画的初始页面数据，即第一页面状态数据。通过页面收集插件获取当前页面状态数据后，终端则通过关闭页面收集插件的监听端口，以停止自动监听。The terminal sends an access request to the application server, and after receiving the page data returned by the application server according to the access request, detects the page data type in the page data. When it is detected that there is animation data in the page data, the terminal calls the initialized page collection plug-in, calls the monitoring method through the initialized page collection plug-in to collect and monitor the page change data, and collects the initial page data including the animation, that is, the first page status data . After obtaining the current page status data through the page collection plug-in, the terminal closes the listening port of the page collection plug-in to stop automatic monitoring.
终端实时检测用户的操作事件，当检测到用户触发的操作事件时，获取操作事件之后的页面数据，将页面数据与包含动画的初始页面数据进行比较，当不一致时，表示页面数据发生变更。The terminal detects the user's operation event in real time. When an operation event triggered by the user is detected, it obtains the page data after the operation event, and compares the page data with the initial page data including animations. If they are inconsistent, it means that the page data has changed.
进一步的，终端还可以检测操作事件的类型，例如操作事件的类型包括暂停、播放、滑动、关闭等类型。终端则可以根据操作事件的类型判断页面数据是否发生变化，例如当操作事件类型为暂停、播放，标识页面数据未发生变化，例如当操作事件类型为滑动、关闭等，表示页面数据发生了变化。终端则直接获取操作事件后对应的页面数据。通过识别检测页面中自动执行的动画数据，能够在减少监听计算量的同时有效地对包含自执行动画的页面进行监听。Further, the terminal can also detect the type of the operation event, for example, the type of the operation event includes pause, play, slide, close and so on. The terminal can determine whether the page data has changed according to the type of the operation event. For example, when the operation event type is pause or play, it indicates that the page data has not changed. For example, when the operation event type is slide or close, it means that the page data has changed. The terminal directly obtains the corresponding page data after the operation event. By identifying and detecting the automatically executed animation data in the page, it is possible to effectively monitor the page containing the self-executing animation while reducing the monitoring calculation amount.
应该理解的是，虽然图2-3的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2-3中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the steps in the flowcharts of FIGS. 2-3 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2-3 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. These sub-steps or stages are not necessarily completed at the same time. The order of execution of the steps is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of sub-steps or stages of other steps.
在一个实施例中，如图4所示，提供了一种页面数据采集处理装置，包括：请求发送模块402、数据接收模块404、页面采集模块406、数据比对模块408和数据上传模块410，其中：In one embodiment, as shown in FIG. 4, a page data collection and processing apparatus is provided, including: a
请求发送模块402，用于发送访问请求至应用服务器，使得应用服务器根据访问请求获取对应的页面数据并返回至终端；a
数据接收模块404，用于接收应用服务器返回的页面数据，页面数据携带页面标识；A
页面采集模块406，用于调用初始化的页面收集插件，利用页面收集插件监听页面变化数据；当检测到页面数据发生变化时，通过页面收集插件采集第一页面状态数据；关闭页面收集插件的监听端口；The
数据比对模块408，用于当检测到用户触发的操作事件时，获取操作事件对应的操作页面数据，将操作页面数据与第一页面状态数据进行比对；The
页面采集模块406还用于若操作页面数据与第一页面状态数据不一致，调用页面收集插件收集页面操作数据对应的第二页面状态数据；The
数据上传模块410，用于将第一页面状态数据和第二页面状态数据根据页面标识上传至运维服务器，使得运维服务器根据页面状态数据对页面数据进行分析。The
在一个实施例中，页面采集模块406还用于通过页面收集插件读取页面数据的文档结构数据；根据访问请求对文档结构数据和相应的元素节点进行遍历采集，并对文档结构数据添加第一读取标签；第一读取标签包括多个元素属性；获取页面数据的初始元素标签，根据多个元素属性将第一读取标签与初始元素标签进行比对；当第一读取标签与初始元素标签不一致时，则确定页面数据存在变化，并根据第一读取标签生成第一页面状态数据。In one embodiment, the
在一个实施例中，数据比对模块408还用于检测操作事件的事件类型，获取操作页面数据的操作页面标识；将页面数据的页面标识与操作页面数据的操作页面标识进行比对；当事件类型为动态事件，且页面标识与操作页面标识不一致时，确定页面数据存在变化。In one embodiment, the
在一个实施例中，第一页面状态数据包括状态属性信息，页面采集模块406还用于当事件类型为动态事件，且页面标识与操作页面标识一致时，根据操作事件获取当前帧的界面显示数据；界面显示数据包括时间标识；获取上一帧的界面显示数据，根据上一帧的界面显示数据和当前帧的界面显示数据的时间标识计算上一帧的界面显示数据的访问时长；将访问时长添加至第一页面状态数据的状态属性信息中。In one embodiment, the first page state data includes state attribute information, and the
在一个实施例中，页面采集模块还用于检测页面数据中页面数据类型；当页面数据中存在动画数据时，通过页面收集插件采集第一页面状态数据后，关闭页面收集插件的监听端口；当检测到用户触发的操作事件时，获取操作事件的操作类型；若操作类型为第一预设类型，保持页面收集插件的监听端口关闭；若操作类型为第二预设类型，获取操作事件对应的第二页面状态数据。In one embodiment, the page collection module is also used to detect the page data type in the page data; when there is animation data in the page data, after collecting the first page status data through the page collection plug-in, the monitoring port of the page collection plug-in is closed; When detecting an operation event triggered by the user, obtain the operation type of the operation event; if the operation type is the first preset type, keep the listening port of the page collection plug-in closed; if the operation type is the second preset type, obtain the corresponding operation event. Second page status data.
关于页面数据采集处理装置的具体限定可以参见上文中对于页面数据采集处理方法的限定，在此不再赘述。上述页面数据采集处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。For the specific limitation of the page data collection and processing apparatus, please refer to the limitation of the page data collection and processing method above, which will not be repeated here. Each module in the above-mentioned page data collection and processing apparatus may be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图5所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种页面数据采集处理方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。In one embodiment, a computer device is provided, and the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 5 . The computer equipment includes a processor, memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for collecting and processing page data is realized. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.
本领域技术人员可以理解，图5中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。Those skilled in the art can understand that the structure shown in FIG. 5 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现本申请任意一个实施例中提供的页面数据采集处理方法的步骤。In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method for collecting and processing page data provided in any embodiment of the present application.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.
Claims (10)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910748859.3ACN110659435A (en) | 2019-08-14 | 2019-08-14 | Page data acquisition processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910748859.3ACN110659435A (en) | 2019-08-14 | 2019-08-14 | Page data acquisition processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110659435Atrue CN110659435A (en) | 2020-01-07 |
Family
ID=69037472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910748859.3APendingCN110659435A (en) | 2019-08-14 | 2019-08-14 | Page data acquisition processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110659435A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309547A (en)* | 2020-01-21 | 2020-06-19 | 北京百度网讯科技有限公司 | Web page information acquisition method, device and electronic device |
| CN111931102A (en)* | 2020-07-01 | 2020-11-13 | 华为技术有限公司 | Method, device and system for constructing page |
| CN112035722A (en)* | 2020-08-04 | 2020-12-04 | 北京启明星辰信息安全技术有限公司 | Method and device for extracting dynamic webpage information and computer readable storage medium |
| CN113377634A (en)* | 2021-05-25 | 2021-09-10 | 北京猿力教育科技有限公司 | Monitoring method for backtracking user operation, user operation backtracking method and device |
| CN114138863A (en)* | 2021-11-29 | 2022-03-04 | 挂号网（杭州）科技有限公司 | Troubleshooting method, device, electronic device and storage medium |
| CN115033213A (en)* | 2022-06-24 | 2022-09-09 | 金蝶软件（中国）有限公司 | Data acquisition method, device, computer equipment and storage medium |
| CN115941866A (en)* | 2022-12-22 | 2023-04-07 | 北斗星通智联科技有限责任公司 | Multi-camera page switching system and method and vehicle navigation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080046562A1 (en)* | 2006-08-21 | 2008-02-21 | Crazy Egg, Inc. | Visual web page analytics |
| CN103139276A (en)* | 2011-12-05 | 2013-06-05 | 金蝶软件（中国）有限公司 | Method and system of both-way communication between Web page and graphical user interface (GUI) client |
| WO2017084508A1 (en)* | 2015-11-17 | 2017-05-26 | 阿里巴巴集团控股有限公司 | Method and device for automatically burying points |
| CN107547502A (en)* | 2017-06-07 | 2018-01-05 | 新华三技术有限公司 | A kind of information monitoring system, method and device |
| CN108089856A (en)* | 2016-11-22 | 2018-05-29 | 北京国双科技有限公司 | The monitor method and device of a kind of page elements |
| WO2018121501A1 (en)* | 2016-12-29 | 2018-07-05 | 北京国双科技有限公司 | Method and apparatus for monitoring network visitor behavior |
| CN108846116A (en)* | 2018-06-26 | 2018-11-20 | 北京京东金融科技控股有限公司 | Page Impression collecting method, system, electronic equipment and storage medium |
| CN109710354A (en)* | 2018-12-13 | 2019-05-03 | 平安普惠企业管理有限公司 | Page monitor method, page restoring method, device, equipment and medium |
- 2019
- 2019-08-14CNCN201910748859.3Apatent/CN110659435A/enactivePending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080046562A1 (en)* | 2006-08-21 | 2008-02-21 | Crazy Egg, Inc. | Visual web page analytics |
| CN103139276A (en)* | 2011-12-05 | 2013-06-05 | 金蝶软件（中国）有限公司 | Method and system of both-way communication between Web page and graphical user interface (GUI) client |
| WO2017084508A1 (en)* | 2015-11-17 | 2017-05-26 | 阿里巴巴集团控股有限公司 | Method and device for automatically burying points |
| CN108089856A (en)* | 2016-11-22 | 2018-05-29 | 北京国双科技有限公司 | The monitor method and device of a kind of page elements |
| WO2018121501A1 (en)* | 2016-12-29 | 2018-07-05 | 北京国双科技有限公司 | Method and apparatus for monitoring network visitor behavior |
| CN107547502A (en)* | 2017-06-07 | 2018-01-05 | 新华三技术有限公司 | A kind of information monitoring system, method and device |
| CN108846116A (en)* | 2018-06-26 | 2018-11-20 | 北京京东金融科技控股有限公司 | Page Impression collecting method, system, electronic equipment and storage medium |
| CN109710354A (en)* | 2018-12-13 | 2019-05-03 | 平安普惠企业管理有限公司 | Page monitor method, page restoring method, device, equipment and medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309547A (en)* | 2020-01-21 | 2020-06-19 | 北京百度网讯科技有限公司 | Web page information acquisition method, device and electronic device |
| CN111931102A (en)* | 2020-07-01 | 2020-11-13 | 华为技术有限公司 | Method, device and system for constructing page |
| CN112035722A (en)* | 2020-08-04 | 2020-12-04 | 北京启明星辰信息安全技术有限公司 | Method and device for extracting dynamic webpage information and computer readable storage medium |
| CN113377634A (en)* | 2021-05-25 | 2021-09-10 | 北京猿力教育科技有限公司 | Monitoring method for backtracking user operation, user operation backtracking method and device |
| CN114138863A (en)* | 2021-11-29 | 2022-03-04 | 挂号网（杭州）科技有限公司 | Troubleshooting method, device, electronic device and storage medium |
| CN115033213A (en)* | 2022-06-24 | 2022-09-09 | 金蝶软件（中国）有限公司 | Data acquisition method, device, computer equipment and storage medium |
| CN115941866A (en)* | 2022-12-22 | 2023-04-07 | 北斗星通智联科技有限责任公司 | Multi-camera page switching system and method and vehicle navigation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110659435A (en) | Page data acquisition processing method and device, computer equipment and storage medium | |
| CN110688598B (en) | Service parameter acquisition method and device, computer equipment and storage medium | |
| CN108304498B (en) | Webpage data acquisition method and device, computer equipment and storage medium | |
| CN109766256B (en) | Method and device for testing H5 page performance in application program and computer equipment | |
| CN109885624B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112394908A (en) | Method and device for automatically generating embedded point page, computer equipment and storage medium | |
| CN112491602A (en) | Behavior data monitoring method and device, computer equipment and medium | |
| CN110688168B (en) | Method, device, equipment and storage medium for improving application startup speed | |
| CN107085549B (en) | Method and device for generating fault information | |
| CN110659092B (en) | Webpage screenshot method and device, computer equipment and storage medium | |
| CN109542764B (en) | Webpage automatic testing method and device, computer equipment and storage medium | |
| CN110659188B (en) | Page portrait data processing method and device, computer equipment and storage medium | |
| CN112445757A (en) | Visual log storage and backtracking method and device and computer equipment | |
| CN111666298A (en) | Method and device for detecting user service class based on flink, and computer equipment | |
| CN110750443A (en) | Webpage testing method and device, computer equipment and storage medium | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN111459577A (en) | Application installation source tracking method, device, equipment and storage medium | |
| US10140377B2 (en) | Data processing, data collection | |
| CN112181794A (en) | Page monitoring method and device, computer equipment and storage medium | |
| CN112016027A (en) | Method and device for monitoring access data operation platform | |
| CN111552643A (en) | Multi-browser web page access synchronization method and system | |
| CN112817816B (en) | Embedded point processing method and device, computer equipment and storage medium | |
| CN112905935A (en) | Page recording method, page recording animation generation method, equipment and storage medium | |
| CN113688022A (en) | Browser performance monitoring method, device, device and medium | |
| CN111046240A (en) | Gateway flow statistical method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20200107 |