技术领域Technical Field
本申请涉及智能终端技术领域，尤其涉及一种数据处理方法、服务器及终端设备。The present application relates to the field of intelligent terminal technology, and in particular to a data processing method, a server and a terminal device.
背景技术Background Art
差分还原技术应用在电子设备的软件升级场景中。差分还原技术包括差分算法和还原算法。服务器通过差分算法，可以得到未升级与升级后的软件之间的差异数据，客户终端通过还原算法以及差异数据，可以将未升级的软件进行升级。Differential restoration technology is used in software upgrade scenarios of electronic devices. Differential restoration technology includes differential algorithms and restoration algorithms. The server can obtain the difference data between the unupgraded and upgraded software through the differential algorithm, and the client terminal can upgrade the unupgraded software through the restoration algorithm and the differential data.
执行差分算法时，通常会得到数据量较大的差异数据，导致用于软件升级的补丁文件数据量较大，这样会加大客户终端下载差异数据的流量，增加下载时间和加大存储空间的占用等，不利于客户终端对未升级的软件进行升级。When executing the differential algorithm, a large amount of differential data is usually obtained, resulting in a large amount of patch file data for software upgrades. This will increase the traffic of client terminals downloading differential data, increase download time and increase storage space usage, etc., which is not conducive to client terminals upgrading non-upgraded software.
因此，如何通过差分算法尽可能减小补丁文件大小是目前亟待解决的一个问题。Therefore, how to reduce the size of patch files as much as possible through differential algorithms is a problem that needs to be solved urgently.
发明内容Summary of the invention
本申请提供一种数据处理方法、服务器及终端设备，通过为保留数据长度设置数据串长度阈值，可以减小保留数据的数量和新增数据的大小，以压缩补丁文件的大小，可以减小终端设备下载补丁文件的流量，缩短对目标应用的升级时间。The present application provides a data processing method, a server and a terminal device. By setting a data string length threshold for the retained data length, the amount of retained data and the size of new data can be reduced to compress the size of the patch file, which can reduce the traffic of the terminal device downloading the patch file and shorten the upgrade time for the target application.
本申请第一方面提供一种数据处理方法，应用于服务器，方法包括：接收来自第一设备的第一文件更新请求，第一文件更新请求包括第一目标应用的第一新文件数据；获取第一新文件数据对应的第一旧文件数据，以及第一数据串集合，第一数据串集合是基于第一旧文件数据和第一新文件数据创建的不同长度的数据串；根据第一数据串集合以及预设的数据串长度阈值，确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集，保留数据集包括至少一条保留数据，新增数据集包括至少一条新增数据，每条保留数据的数据长度大于数据串长度阈值；根据至少一条保留数据的数据长度、至少一条新增数据在第一新文件数据的位置信息、以及至少一条保留数据分别在第一新文件数据和第一旧文件数据的位置信息，生成第一新文件数据的补丁文件，补丁文件用于对第一目标应用进行数据更新。In a first aspect, the present application provides a data processing method, which is applied to a server, and the method includes: receiving a first file update request from a first device, the first file update request including first new file data of a first target application; obtaining first old file data corresponding to the first new file data, and a first data string set, the first data string set being data strings of different lengths created based on the first old file data and the first new file data; determining a retained data set and a newly added data set of the first new file data compared to the first old file data according to the first data string set and a preset data string length threshold, the retained data set including at least one retained data, the newly added data set including at least one newly added data, and the data length of each retained data being greater than the data string length threshold; generating a patch file for the first new file data according to the data length of at least one retained data, the position information of at least one newly added data in the first new file data, and the position information of at least one retained data in the first new file data and the first old file data respectively, the patch file being used to update data of the first target application.
在第一方面的一些实施例中，根据第一数据串集合以及预设的数据串长度阈值，确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集，包括：从第一新文件数据的第一个字节开始，在第一数据串集合中查找以第一个字节开始的最长数据串；通过比较最长数据串的长度与数据串长度阈值的大小关系，确定是否将最长数据串作为一条保留数据；在遍历完第一新文件数据的所有字节后，得到保留数据集；根据保留数据集中保留数据在第一新文件数据的位置，确定新增数据集。In some embodiments of the first aspect, based on the first data string set and a preset data string length threshold, determining a retained data set and a newly added data set of the first new file data compared to the first old file data, including: starting from the first byte of the first new file data, searching the first data string set for the longest data string starting with the first byte; determining whether to take the longest data string as a retained data by comparing the length of the longest data string with the data string length threshold; obtaining a retained data set after traversing all bytes of the first new file data; and determining a newly added data set based on the position of the retained data in the retained data set in the first new file data.
在第一方面的一些实施例中，通过比较最长数据串的长度与数据串长度阈值的大小关系，确定是否将最长数据串作为一条保留数据，包括：若最长数据串的长度大于数据串长度阈值，确定将最长数据串作为一条保留数据，且从第一新文件数据的目标字节开始，在第一数据串集合中查找是否有以目标字节开始的最长数据串，其中，目标字节为第一个字节累加第一个字节开始的最长数据串的长度后确定的字节；若最长数据串的长度小于或等于数据串长度阈值，从第一个字节的下一个字节开始，在第一数据串集合中继续查找是否有以下一个字节开始的最长数据串。In some embodiments of the first aspect, by comparing the length of the longest data string with a data string length threshold, determining whether to take the longest data string as a retained data, including: if the length of the longest data string is greater than the data string length threshold, determining to take the longest data string as a retained data, and starting from the target byte of the first new file data, searching in the first data string set whether there is a longest data string starting with the target byte, wherein the target byte is the byte determined by adding the length of the longest data string starting with the first byte to the first byte; if the length of the longest data string is less than or equal to the data string length threshold, starting from the byte next to the first byte, continuing to search in the first data string set whether there is a longest data string starting with the next byte.
在第一方面的一些实施例中，根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块，N为大于1且小于或等于M的正整数，M为并行执行的处理线程的最大值；创建N个第一处理线程，N个第一处理线程与N对新旧文件数据块一一对应；通过N个第一处理线程并行执行对N对新旧文件数据块的数据比对，得到N对新旧文件数据块的补丁文件；根据N对新旧文件数据块的补丁文件，生成第一新文件数据的补丁文件。In some embodiments of the first aspect, according to a preset data volume threshold, the first new file data and the first old file data are respectively segmented to obtain N pairs of new and old file data blocks, where N is a positive integer greater than 1 and less than or equal to M, and M is the maximum number of processing threads executed in parallel; N first processing threads are created, and the N first processing threads correspond one-to-one to the N pairs of new and old file data blocks; data comparison of the N pairs of new and old file data blocks is performed in parallel by the N first processing threads to obtain patch files of the N pairs of new and old file data blocks; based on the patch files of the N pairs of new and old file data blocks, a patch file of the first new file data is generated.
在第一方面的一些实施例中，方法还包括：通过第一目标应用的第一应用线程调用预设算法库初始化预设参数，预设参数包括数据量阈值和M；根据第一文件更新请求，通过第一应用线程创建第一管理线程；根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块，包括：通过第一管理线程执行根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块的步骤；创建N个第一处理线程，包括：通过第一管理线程创建N个第一处理线程；根据N对新旧文件数据块的补丁文件，生成新文件数据的补丁文件，包括：通过第一管理线程执行根据N对新旧文件数据块的补丁文件，生成第一新文件数据的补丁文件的步骤。In some embodiments of the first aspect, the method also includes: calling a preset algorithm library through a first application thread of a first target application to initialize preset parameters, the preset parameters including a data volume threshold and M; creating a first management thread through the first application thread according to a first file update request; performing data segmentation on the first new file data and the first old file data according to the preset data volume threshold, respectively, to obtain N pairs of new and old file data blocks, including: executing the step of performing data segmentation on the first new file data and the first old file data according to the preset data volume threshold, respectively, to obtain N pairs of new and old file data blocks through the first management thread; creating N first processing threads, including: creating N first processing threads through the first management thread; generating a patch file for new file data according to patch files of N pairs of new and old file data blocks, including: executing the step of generating a patch file for the first new file data according to patch files of N pairs of new and old file data blocks through the first management thread.
在第一方面的一些实施例中，第一应用线程还用于在第一存储空间中写入第一文件更新请求对应的N个第一处理线程的信息；第一管理线程还用于在生成第一新文件数据的补丁文件后，删除N个第一处理线程的信息；第一应用线程在第一存储空间内写入数据时，禁止第一管理线程在第一存储空间删除数据。In some embodiments of the first aspect, the first application thread is also used to write information of N first processing threads corresponding to the first file update request in the first storage space; the first management thread is also used to delete the information of N first processing threads after generating a patch file for the first new file data; when the first application thread writes data in the first storage space, the first management thread is prohibited from deleting data in the first storage space.
在第一方面的一些实施例中，第一处理线程用于在第二存储空间写入第一处理线程的处理状态和/或处理进度，第一管理线程用于从第二存储空间读取第一处理线程的处理状态和/或处理进度；第一处理线程在第二存储空间写入数据时，禁止第一管理线程在第二存储空间读取数据。In some embodiments of the first aspect, the first processing thread is used to write the processing status and/or processing progress of the first processing thread in the second storage space, and the first management thread is used to read the processing status and/or processing progress of the first processing thread from the second storage space; when the first processing thread writes data in the second storage space, the first management thread is prohibited from reading data in the second storage space.
在第一方面的一些实施例中，若第一文件更新请求包括第一目标应用的第一新文件数据和第二新文件数据，第一新文件数据和第二新文件数据均对应第一旧文件数据；通过第一应用线程创建第一管理线程，包括：通过第一应用程序创建两个第一管理线程；两个第一管理线程的其中一个第一管理线程在读取第一旧文件数据时，禁止两个第一管理线程的另一个第一管理线程读取第一旧文件数据。In some embodiments of the first aspect, if the first file update request includes first new file data and second new file data of a first target application, the first new file data and the second new file data both correspond to first old file data; creating a first management thread through a first application thread comprises: creating two first management threads through a first application program; when one of the two first management threads reads the first old file data, prohibiting the other first management thread of the two first management threads from reading the first old file data.
在第一方面的一些实施例中，在接收第一文件更新请求的同时，接收来自第一设备的第二文件更新请求，第二文件更新请求包括第二目标应用的第三新文件数据；根据第二文件更新请求，创建第二目标应用的第二应用线程，第二应用线程用于通过调用预设算法库初始化预设参数；通过第二应用线程创建第二管理线程；第一应用线程在调用预设算法库时，禁止第二应用线程调用预设算法库。In some embodiments of the first aspect, while receiving a first file update request, a second file update request is received from the first device, the second file update request including third new file data of a second target application; based on the second file update request, a second application thread of the second target application is created, the second application thread being used to initialize preset parameters by calling a preset algorithm library; a second management thread is created through the second application thread; and when the first application thread calls the preset algorithm library, the second application thread is prohibited from calling the preset algorithm library.
在第一方面的一些实施例中，方法还包括：接收来自第二设备的升级请求，升级请求用于请求升级第一目标应用；响应于升级请求，获取第一目标应用的补丁文件；向第二设备发送升级响应，升级响应包括第一目标应用的补丁文件。In some embodiments of the first aspect, the method also includes: receiving an upgrade request from a second device, the upgrade request being used to request an upgrade of a first target application; obtaining a patch file of the first target application in response to the upgrade request; and sending an upgrade response to the second device, the upgrade response including the patch file of the first target application.
本申请第二方面还提供一种数据处理方法，应用于终端设备，方法包括：接收来自服务器的升级响应，升级响应包括第一目标应用的补丁文件；根据补丁文件的至少一条保留数据在第一旧文件数据的位置信息，以及至少一条保留数据的数据长度，从第一旧文件数据中读取至少一条保留数据；根据至少一条保留数据中的相邻保留数据之间的间隔长度，从补丁文件中读取与间隔长度相同长度的新增数据，以及至少一条保留数据在第一新文件数据的位置信息；根据新增数据、至少一条保留数据在第一新文件数据的位置信息以及至少一条保留数据块生成第一目标应用的第一新文件数据。The second aspect of the present application also provides a data processing method, which is applied to a terminal device, and the method includes: receiving an upgrade response from a server, the upgrade response including a patch file of a first target application; reading at least one piece of retained data from the first old file data based on the position information of at least one piece of retained data in the patch file in the first old file data, and the data length of at least one piece of retained data; reading new data of the same length as the interval length and the position information of at least one piece of retained data in the first new file data from the patch file based on the interval length between adjacent retained data in the at least one piece of retained data; generating the first new file data of the first target application based on the new data, the position information of at least one piece of retained data in the first new file data, and at least one retained data block.
本申请第三方面还提供一种服务器，包括：处理器以及存储器，存储器与处理器耦合，存储器用于存储计算机程序代码，处理器调用计算机程序代码以使得服务器执行第一方面的方法。The third aspect of the present application also provides a server, including: a processor and a memory, the memory is coupled to the processor, the memory is used to store computer program code, and the processor calls the computer program code to enable the server to execute the method of the first aspect.
本申请第四方面还提供一种终端设备，包括：处理器以及存储器，存储器与处理器耦合，存储器用于存储计算机程序代码，处理器调用计算机程序代码以使得终端设备执行第二方面的方法。The fourth aspect of the present application also provides a terminal device, including: a processor and a memory, the memory is coupled to the processor, the memory is used to store computer program code, and the processor calls the computer program code to enable the terminal device to execute the method of the second aspect.
本申请实施例还提供一种可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如第一方面或第二方面的方法。An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method of the first aspect or the second aspect is implemented.
本申请实施例还提供一种芯片系统，芯片系统应用于电子设备或车辆，芯片系统包括一个或多个处理器，一个或多个处理器用于调用计算机指令以使得第一方面或第二方面的方法。An embodiment of the present application also provides a chip system, which is applied to an electronic device or a vehicle. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to implement the method of the first aspect or the second aspect.
本申请实施例还提供一种计算机程序产品，包括计算机程序，计算机程序被处理器执行时用于实现如第一方面或第二方面的方法。An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the method of the first aspect or the second aspect when executed by a processor.
本申请提供的一种数据处理方法、服务器及终端设备，该方法包括：通过为目标应用的旧文件数据创建数据串集合，根据数据串集合和预设的数据串长度阈值，可以得到目标应用的新文件数据相较于旧文件数据的保留数据集和新增数据集；根据保留数据集中至少一条保留数据的相关信息以及新增数据集中至少一条新增数据在新文件数据的位置信息，生成目标应用的补丁文件，用于对目标应用进行数据更新。上述方法中，通过为保留数据长度设置数据串长度阈值，可以减小保留数据的数量和新增数据的大小，以压缩补丁文件的大小，可以减小终端设备下载补丁文件的流量，缩短对目标应用的升级时间。The present application provides a data processing method, server and terminal device, the method comprising: by creating a data string set for the old file data of the target application, according to the data string set and the preset data string length threshold, the retained data set and the newly added data set of the new file data of the target application compared with the old file data can be obtained; according to the relevant information of at least one retained data in the retained data set and the position information of at least one newly added data in the newly added data set in the new file data, a patch file of the target application is generated for updating the data of the target application. In the above method, by setting the data string length threshold for the retained data length, the amount of retained data and the size of newly added data can be reduced to compress the size of the patch file, which can reduce the traffic of the terminal device downloading the patch file and shorten the upgrade time of the target application.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
图1为本申请一实施例提供的数据处理方法的一应用场景示意图；FIG1 is a schematic diagram of an application scenario of a data processing method provided by an embodiment of the present application;
图2为本申请一实施例提供的数据处理方法的一流程示意图；FIG2 is a flow chart of a data processing method provided by an embodiment of the present application;
图3为本申请一实施例提供的数据处理方法的另一应用场景示意图；FIG3 is a schematic diagram of another application scenario of the data processing method provided by an embodiment of the present application;
图4为本申请一实施例提供的数据处理方法的另一流程示意图；FIG4 is another schematic diagram of a flow chart of a data processing method provided by an embodiment of the present application;
图5为本申请一实施例提供的数据处理方法的再一应用场景示意图；FIG5 is a schematic diagram of another application scenario of the data processing method provided in one embodiment of the present application;
图6为本申请一实施例提供的数据处理方法的再一流程示意图；FIG6 is another schematic diagram of a flow chart of a data processing method provided in an embodiment of the present application;
图7为本申请一实施例提供的数据处理方法的多线程场景示意图；FIG7 is a schematic diagram of a multi-threaded scenario of a data processing method provided by an embodiment of the present application;
图8为本申请又一实施例提供的数据处理方法的一应用场景示意图；FIG8 is a schematic diagram of an application scenario of a data processing method provided by another embodiment of the present application;
图9为本申请另一实施例提供的数据处理方法的一流程示意图；FIG9 is a flow chart of a data processing method provided by another embodiment of the present application;
图10为本申请另一实施例提供的数据处理方法的又一流程示意图；FIG10 is another schematic flow chart of a data processing method provided by another embodiment of the present application;
图11为本申请又一实施例提供的数据处理方法的另一应用场景示意图；FIG11 is a schematic diagram of another application scenario of a data processing method provided by yet another embodiment of the present application;
图12为本申请又一实施例提供的数据处理方法的再一应用场景示意图；FIG12 is a schematic diagram of yet another application scenario of a data processing method provided by yet another embodiment of the present application;
图13为本申请一实施例提供的数据处理装置的一结构示意图；FIG13 is a schematic diagram of a structure of a data processing device provided by an embodiment of the present application;
图14为本申请一实施例提供的数据处理装置的另一结构示意图；FIG14 is another schematic diagram of the structure of a data processing device provided by an embodiment of the present application;
图15为本申请一实施例提供的服务器的结构示意图；FIG15 is a schematic diagram of the structure of a server provided in an embodiment of the present application;
图16为本申请一实施例提供的终端设备的结构示意图。FIG. 16 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application.
通过上述附图，已示出本申请明确的实施例，后文中将有更详细的描述。这些附图和文字描述并不是为了通过任何方式限制本申请构思的范围，而是通过参考特定实施例为本领域技术人员说明本申请的概念。The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments.
具体实施方式DETAILED DESCRIPTION
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
本申请所涉及的术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。在以下各实施例的描述中，“多个”的含义是两个以上，除非另有明确具体的限定。The terms "first", "second", etc. used in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
差分还原技术应用在电子设备的软件升级场景中。差分还原技术包括差分算法和还原算法。服务器通过差分算法，可以得到未升级与升级后的软件之间的差异数据，客户终端通过还原算法以及差异数据，可以将未升级的软件进行升级。Differential restoration technology is used in software upgrade scenarios of electronic devices. Differential restoration technology includes differential algorithms and restoration algorithms. The server can obtain the difference data between the non-upgraded and upgraded software through the differential algorithm, and the client terminal can upgrade the non-upgraded software through the restoration algorithm and the differential data.
执行差分算法时，通常会得到数据量较大的差异数据，导致用于软件升级的补丁文件数据量较大，这样会加大客户终端下载差异数据的流量，增加下载时间和加大存储空间的占用等，不利于客户终端对未升级的软件进行升级。When executing the differential algorithm, a large amount of differential data is usually obtained, resulting in a large amount of patch file data for software upgrades. This will increase the traffic of client terminals downloading differential data, increase download time and increase storage space usage, etc., which is not conducive to client terminals upgrading non-upgraded software.
因此，如何通过差分算法尽可能减小补丁文件大小是目前亟待解决的一个问题。Therefore, how to reduce the size of patch files as much as possible through differential algorithms is a problem that needs to be solved urgently.
鉴于此，本申请提供了一种数据处理方法、服务器、终端设备、存储介质、芯片系统和程序产品，通过为目标应用的旧文件数据创建数据串集合，根据数据串集合和预设的数据串长度阈值，可以得到目标应用的新文件数据相较于旧文件数据的保留数据集和新增数据集，根据保留数据集中至少一条保留数据的相关信息以及新增数据集中至少一条新增数据在新文件数据的位置信息，生成目标应用的补丁文件，用于对目标应用进行数据更新。上述方法中，通过为保留数据长度设置数据串长度阈值，可以减小保留数据的数量和新增数据的大小，以压缩补丁文件的大小，可以减小终端设备下载补丁文件的流量，缩短对目标应用的升级时间。In view of this, the present application provides a data processing method, a server, a terminal device, a storage medium, a chip system and a program product, by creating a data string set for the old file data of the target application, according to the data string set and the preset data string length threshold, the new file data of the target application can be obtained compared with the old file data. The retained data set and the newly added data set are obtained, and the patch file of the target application is generated according to the relevant information of at least one retained data in the retained data set and the position information of at least one newly added data in the newly added data set in the new file data, which is used to update the data of the target application. In the above method, by setting the data string length threshold for the retained data length, the amount of retained data and the size of the newly added data can be reduced to compress the size of the patch file, which can reduce the traffic of the terminal device downloading the patch file and shorten the upgrade time of the target application.
请参阅图1，图1为本申请数据处理方法的一应用场景图。应用场景包括服务器10、第一设备20和第二设备30。服务器10接收来自第一设备20的文件更新请求，文件更新请求包括目标应用(Application，APP)的新文件数据(新文件数据为升级后的目标应用数据)，获取旧文件数据(旧文件数据为未升级的目标应用数据)，接着，根据本申请的数据处理方法生成补丁文件，补丁文件的内容包括新增数据集中的新增数据，以及保留数据集中的保留数据的相关信息，相关信息包括保留数据的数据长度、保留数据分别在新文件数据和旧文件数据的位置信息。Please refer to Figure 1, which is an application scenario diagram of the data processing method of the present application. The application scenario includes a server 10, a first device 20 and a second device 30. The server 10 receives a file update request from the first device 20, the file update request includes new file data of the target application (Application, APP) (the new file data is the upgraded target application data), obtains the old file data (the old file data is the non-upgraded target application data), and then generates a patch file according to the data processing method of the present application. The content of the patch file includes the newly added data in the newly added data set, and the relevant information of the retained data in the retained data set, and the relevant information includes the data length of the retained data and the location information of the retained data in the new file data and the old file data, respectively.
服务器10接收第二设备30的升级请求，将补丁文件发送至第二设备30，第二设备30根据补丁文件升级部署在第二设备30的目标应用。The server 10 receives the upgrade request of the second device 30 , and sends the patch file to the second device 30 . The second device 30 upgrades the target application deployed on the second device 30 according to the patch file.
在一些实施例中，第一设备为开发目标应用的新文件数据和/或旧文件数据的设备终端。第一设备包括平板电脑和台式电脑等。In some embodiments, the first device is a device terminal for developing new file data and/or old file data of a target application. The first device includes a tablet computer and a desktop computer.
在一些实施例中，第二设备包括车辆、无人机、智慧手表、手机和平板电脑等用户使用设备。In some embodiments, the second device includes a user device such as a vehicle, a drone, a smart watch, a mobile phone, and a tablet computer.
需要说明的是，图1仅是本申请实施例提供的一种应用场景的示意图，本申请实施例不对图1中包括的各种设备的实际形态进行限定，也不对图1中设备之间的交互方式进行限定，在方案的具体应用中，可以根据实际需求设定。It should be noted that Figure 1 is only a schematic diagram of an application scenario provided by an embodiment of the present application. The embodiment of the present application does not limit the actual forms of the various devices included in Figure 1, nor does it limit the interaction method between the devices in Figure 1. In the specific application of the solution, it can be set according to actual needs.
下面以具体地实施例对本申请的技术方案，以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本申请的实施例进行描述。The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
图2为本申请提供的数据处理方法的一实施例的流程示意图，本申请实施例的执行主体可以为服务器，本实施例中的方法可以通过软件、硬件或者软硬件结合的方式来实现。如图2所示，该数据处理方法，可以包括以下步骤：FIG2 is a flow chart of an embodiment of the data processing method provided by the present application. The execution subject of the embodiment of the present application may be a server. The method in the embodiment may be implemented by software, hardware, or a combination of software and hardware. As shown in FIG2, the data processing method may include the following steps:
步骤S110：接收来自第一设备的第一文件更新请求。Step S110: receiving a first file update request from a first device.
第一文件更新请求包括第一目标应用的第一新文件数据。第一目标应用包括导航和地图应用程序、娱乐应用程序、通讯应用程序、智能驾驶辅助应用程序或停车或充电应用程序等。第一新文件数据为更新后(更新包括升级)的第一目标应用数据。The first file update request includes first new file data of a first target application. The first target application includes a navigation and map application, an entertainment application, a communication application, an intelligent driving assistance application, or a parking or charging application, etc. The first new file data is updated (update includes upgrade) data of the first target application.
可理解，为了升级第一目标应用，研发人员开发第一目标应用的第一新文件数据后，通过第一设备生成第一文件更新请求，以将第一新文件数据发送至服务器。It is understandable that in order to upgrade the first target application, after the R&D personnel develop the first new file data of the first target application, they generate a first file update request through the first device to send the first new file data to the server.
步骤S120：获取第一新文件数据对应的第一旧文件数据，以及第一数据串集合。Step S120: Acquire the first old file data corresponding to the first new file data and the first data string set.
可以从服务器的存储装置中获取第一旧文件数据，以及第一数据串集合。第一旧文件数据可以为更新前的第一目标应用数据。第一数据串集合是基于第一旧文件数据和第一新文件数据创建的不同长度的数据串，将不同长度的数据串按数据串长度从小到大的顺序排序后，存储至第一数据串集合中，用于确定后续步骤中的保留数据。在一些实施例中，第一数据串集合也可以描述为后缀数组，数据串也可以描述为字节串。The first old file data and the first data string set can be obtained from the storage device of the server. The first old file data can be the first target application data before the update. The first data string set is a data string of different lengths created based on the first old file data and the first new file data. The data strings of different lengths are sorted in order of the length of the data string from small to large, and then stored in the first data string set for determining the retained data in the subsequent steps. In some embodiments, the first data string set can also be described as a suffix array, and the data string can also be described as a byte string.
示例性地，如图5所示，第一新文件数据中包括“abcde”，基于差分算法可以在第一旧文件数据中查找到以a开头的不同长度的数据串，如图5中的“a”、“abc”、“a”和“abcd”。同理，还可以在第一旧文件数据中查找以b开头的不同长度的数据串，或者以c开头的不同长度的数据串，或者以d开头的不同长度的数据串，或者以e开头的不同长度的数据串。For example, as shown in FIG5 , the first new file data includes “abcde”, and based on the differential algorithm, data strings of different lengths starting with a can be found in the first old file data, such as “a”, “abc”, “a”, and “abcd” in FIG5 . Similarly, data strings of different lengths starting with b, or data strings of different lengths starting with c, or data strings of different lengths starting with d, or data strings of different lengths starting with e can also be found in the first old file data.
步骤S130：根据第一数据串集合以及预设的数据串长度阈值，确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集。Step S130: Determine a retained data set and a newly added data set of the first new file data compared to the first old file data according to the first data string set and a preset data string length threshold.
具体地，选中第一新文件数据中的某一字节，在第一数据串集合中，读取以该字节开头的数据串，将数据串与预设的数据串长度阈值进行比较，将大于数据串长度阈值的数据串作为一条保留数据。示例性地，如图5所示，保留数据是第一新文件数据和第一旧文件数据中都有的数据。其中，在第一新文件数据中可以找到与第一旧文件数据相同的保留数据，或者第一旧文件数据的保留数据的子集。通常，保留数据集包括至少一条保留数据。Specifically, a certain byte in the first new file data is selected, and in the first data string set, a data string starting with the byte is read, and the data string is compared with a preset data string length threshold, and a data string greater than the data string length threshold is used as a reserved data. Exemplarily, as shown in FIG5, the reserved data is data in both the first new file data and the first old file data. Among them, the same reserved data as the first old file data, or a subset of the reserved data of the first old file data can be found in the first new file data. Usually, the reserved data set includes at least one reserved data.
示例性地，如图3所示，第一新文件数据包括4条保留数据1、2、3和4。第一旧文件数据包括3条保留数据1、2和3，其中保留数据3中包含保留数据4，也即保留数据4是保留数据3的子集，其中每条保留数据的数据长度大于数据串长度阈值。Exemplarily, as shown in Fig. 3, the first new file data includes four pieces of reserved data 1, 2, 3 and 4. The first old file data includes three pieces of reserved data 1, 2 and 3, wherein reserved data 3 includes reserved data 4, that is, reserved data 4 is a subset of reserved data 3, wherein the data length of each piece of reserved data is greater than the data string length threshold.
在确定多条保留数据后，可以读取相邻两条保留数据之间的新增数据，由于有多个保留数据，可以得到包括至少一条新增数据的新增数据集。如图3所示，新增数据集包括第一新文件数据的5条新增数据A、B、C、D和E。After determining multiple pieces of retained data, the newly added data between two adjacent pieces of retained data can be read. Since there are multiple pieces of retained data, a newly added data set including at least one piece of newly added data can be obtained. As shown in FIG3 , the newly added data set includes five pieces of newly added data A, B, C, D and E of the first new file data.
应理解，对于同一大小的新文件数据，若每条保留数据的数据长度均大于数据串长度阈值，那么保留数据的数量就会越少(例如部分保留数据可以复用，如图3中的保留数据3和保留数据4)，新增数据的数量也会越小。It should be understood that for new file data of the same size, if the data length of each retained data is greater than the data string length threshold, then the amount of retained data will be smaller (for example, part of the retained data can be reused, such as retained data 3 and retained data 4 in Figure 3), and the amount of new data will also be smaller.
步骤S140：根据至少一条保留数据的数据长度、至少一条新增数据在第一新文件数据的位置信息、以及至少一条保留数据分别在第一新文件数据和第一旧文件数据的位置信息，生成第一新文件数据的补丁文件。Step S140: Generate a patch file for the first new file data according to the data length of at least one piece of retained data, the position information of at least one piece of newly added data in the first new file data, and the position information of at least one piece of retained data in the first new file data and the first old file data respectively.
补丁文件用于对第一目标应用进行数据更新，示例性地，可以根据补丁文件对第一目标应用进行升级。补丁文件包括新增数据集和保留数据集的相关信息，保留数据集的相关信息包括保留数据的数据长度和保留数据分别在第一新文件数据和第一旧文件数据的位置信息，位置信息包括起始位置。如图3所示，补丁文件包括3条保留数据的相关信息，以及第一新文件数据的5条新增数据A、B、C、D和E。举例说明，保留数据2的相关信息包括保留数据2在第一新文件数据的起始位置、保留数据2在第一旧文件数据的起始位置和保留数据2的数据长度。The patch file is used to update the data of the first target application. Exemplarily, the first target application can be upgraded according to the patch file. The patch file includes relevant information of the newly added data set and the retained data set. The relevant information of the retained data set includes the data length of the retained data and the location information of the retained data in the first new file data and the first old file data respectively. The location information includes the starting position. As shown in FIG3 , the patch file includes relevant information of 3 retained data and 5 newly added data A, B, C, D and E of the first new file data. For example, the relevant information of retained data 2 includes the starting position of retained data 2 in the first new file data, the starting position of retained data 2 in the first old file data and the data length of retained data 2.
可理解，每条保留数据越长，保留数据的整体数量越少，相邻保留数据块之间确定的新增数据就越小，那么补丁文件就越小，因此，通过设置数据串长度阈值，以尽可能在旧文件数据中查找到最长的保留数据，最终达到压缩补丁文件的目的。It can be understood that the longer each retained data is, the smaller the overall amount of retained data is, the smaller the new data determined between adjacent retained data blocks is, and the smaller the patch file is. Therefore, by setting the data string length threshold, the longest retained data can be found in the old file data as much as possible, ultimately achieving the purpose of compressing the patch file.
请再参阅图1，在一个应用场景中，服务器10接收来自第一设备20的文件更新请求，文件更新请求包括智能驾驶辅助应用程序的新文件数据，获取智能驾驶辅助应用程序的旧文件数据，以及数据串集合。接着，根据数据串集合以及预设的数据串长度阈值，确定新文件数据相较于旧文件数据的保留数据集和新增数据集。最后，根据至少一条保留数据的数据长度、至少一条新增数据在新文件数据的位置信息、以及至少一条保留数据在智能驾驶辅助应用程序的新文件数据和旧文件数据的位置信息，生成新文件数据的补丁文件。Please refer to Figure 1 again. In an application scenario, the server 10 receives a file update request from the first device 20. The file update request includes new file data of the intelligent driving assistance application, obtains old file data of the intelligent driving assistance application, and a data string set. Then, based on the data string set and the preset data string length threshold, the retained data set and the newly added data set of the new file data compared to the old file data are determined. Finally, based on the data length of at least one retained data, the location information of at least one newly added data in the new file data, and the location information of at least one retained data in the new file data and the old file data of the intelligent driving assistance application, a patch file for the new file data is generated.
服务器10接收车辆30的升级请求，将补丁文件发送至车辆30，车辆30根据补丁文件升级部署在车辆30的智能驾驶辅助应用程序。The server 10 receives the upgrade request of the vehicle 30 and sends the patch file to the vehicle 30 . The vehicle 30 upgrades the intelligent driving assistance application deployed in the vehicle 30 according to the patch file.
上述技术方案中，服务器接收来自第一设备的第一文件更新请求，可以获取第一新文件数据对应的第一旧文件数据，以及第一数据串集合。接着，根据第一数据串集合以及预设的数据串长度阈值，可以确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集。其中，保留数据集的保留数据长度大于数据串长度阈值，可以减小新增数据集中新增数据的大小和保留数据集的保留数据的数量。最后，根据至少一条保留数据的数据长度、至少一条新增数据在第一新文件数据的位置信息、以及至少一条保留数据分别在第一新文件数据和第一旧文件数据的位置信息，生成第一新文件数据的补丁文件。通过为保留数据长度设置数据串长度阈值，可以减小保留数据的数量和新增数据的大小，以压缩补丁文件的大小，可以减小终端设备下载补丁文件的流量，缩短对目标应用的升级时间。In the above technical solution, the server receives a first file update request from the first device, and can obtain the first old file data corresponding to the first new file data, as well as the first data string set. Then, based on the first data string set and the preset data string length threshold, the retained data set and the newly added data set of the first new file data compared to the first old file data can be determined. Among them, the retained data length of the retained data set is greater than the data string length threshold, which can reduce the size of the newly added data in the newly added data set and the number of retained data in the retained data set. Finally, based on the data length of at least one retained data, the location information of at least one newly added data in the first new file data, and the location information of at least one retained data in the first new file data and the first old file data, respectively, a patch file for the first new file data is generated. By setting a data string length threshold for the retained data length, the number of retained data and the size of newly added data can be reduced to compress the size of the patch file, which can reduce the traffic of the terminal device downloading the patch file and shorten the upgrade time for the target application.
在一些实施例中，请参阅图4，数据处理方法还包括下述步骤：In some embodiments, referring to FIG. 4 , the data processing method further includes the following steps:
步骤S210：接收来自第二设备的升级请求。Step S210: receiving an upgrade request from a second device.
升级请求用于请求升级第一目标应用。当第二设备监测到第一目标应用需要升级后，生成升级请求，发送至服务器。The upgrade request is used to request an upgrade of the first target application. When the second device detects that the first target application needs to be upgraded, it generates an upgrade request and sends it to the server.
步骤S220：响应于升级请求，获取第一目标应用的补丁文件。Step S220: In response to the upgrade request, obtain a patch file of the first target application.
服务器响应于升级请求，从存储装置中获取根据第一新文件数据和第一旧文件数据得到的第一目标应用的补丁文件。In response to the upgrade request, the server obtains a patch file of the first target application obtained according to the first new file data and the first old file data from the storage device.
步骤S230：向第二设备发送升级响应。Step S230: Send an upgrade response to the second device.
服务器向第二设备发送包括第一目标应用的补丁文件的升级响应。The server sends an upgrade response including the patch file of the first target application to the second device.
在一些实施例中，根据第一数据串集合以及预设的数据串长度阈值，确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集，包括下述步骤：In some embodiments, determining the retained data set and the newly added data set of the first new file data compared with the first old file data according to the first data string set and a preset data string length threshold includes the following steps:
步骤S310：从第一新文件数据的第一个字节开始，在第一数据串集合中查找以第一个字节开始的最长数据串。Step S310: Starting from the first byte of the first new file data, search the first data string set for the longest data string starting with the first byte.
步骤S320：通过比较最长数据串的长度与数据串长度阈值的大小关系，确定是否将最长数据串作为一条保留数据。Step S320: Determine whether to treat the longest data string as a reserved data by comparing the length of the longest data string with the data string length threshold.
具体地，若最长数据串的长度大于数据串长度阈值，确定将最长数据串作为一条保留数据。若最长数据串的长度小于或等于数据串长度阈值，从第一个字节的下一个字节开始，在第一数据串集合中继续查找是否有以下一个字节开始的最长数据串。Specifically, if the length of the longest data string is greater than the data string length threshold, the longest data string is determined to be a reserved data. If the length of the longest data string is less than or equal to the data string length threshold, starting from the next byte of the first byte, continue to search in the first data string set whether there is a longest data string starting with the next byte.
步骤S330：在遍历完第一新文件数据的所有字节后，得到保留数据集。Step S330: After traversing all bytes of the first new file data, a reserved data set is obtained.
通过下述示例的步骤a1至a3，对步骤S310至步骤S330，进一步说明。Steps S310 to S330 are further explained through steps a1 to a3 of the following examples.
示例性地，数据串长度阈值为12字节。服务器执行下列步骤：Exemplarily, the data string length threshold is 12 bytes. The server performs the following steps:
步骤a1：从第一新文件数据的第1个字节开始，在第一数据串集合中查找以第1个字节开始的最长数据串。Step a1: Starting from the first byte of the first new file data, search for the longest data string starting with the first byte in the first data string set.
步骤a2：若最长数据串的长度Len大于12字节，则最长数据串作为保留数据，并记录保留数据的相关信息。且从第一新文件数据的第Len+1个字节开始，继续在第一数据串集合中查找以第Len+1个字节开始的最长数据串，直至第一新文件数据的所有字节都遍历完。Step a2: If the length Len of the longest data string is greater than 12 bytes, the longest data string is used as the reserved data, and the relevant information of the reserved data is recorded. Starting from the Len+1th byte of the first new file data, continue to search for the longest data string starting with the Len+1th byte in the first data string set until all bytes of the first new file data are traversed.
步骤a3：若最长数据串的长度Len小于或等于12字节，从第一新文件数据的第2个字节开始，继续在第一数据串集合中查找以第2个字节开始的最长数据串，直至第一新文件数据的所有字节都遍历完。Step a3: If the length Len of the longest data string is less than or equal to 12 bytes, starting from the second byte of the first new file data, continue to search for the longest data string starting with the second byte in the first data string set until all bytes of the first new file data are traversed.
下面结合图5示例对步骤S310和步骤S320进行说明。The steps S310 and S320 are described below with reference to the example of FIG. 5 .
示例性地，如图5所示，第一新文件数据包括“abcde…”，第一新文件数据的第一个字节为“a”，一种示例中，假设第一数据串集合中包括3种以“a”开头的数据串，分别为“a”、“abc”，“abcde”。从第一新文件数据的“a”开始，在前述第一数据串集合中可以查找到以“a”开始的最长数据串为“abcde”。若数据串长度阈值为4，最长数据串“abcde”的长度大于数据串长度阈值4，则将该最长数据串“abcde”作为保留数据。另一种示例中，假设第一数据串集合中包括3种以“a”开头的数据串，分别为“a”、“ab”和“abc”，从第一新文件数据的“a”开始，在前述第一数据串集合中可以查找到以“a”开始的最长数据串为“abc”，若数据串长度阈值为4，最长数据串“abc”的长度小于数据串长度阈值4，则“abc”不作为保留数据，从第一新文件数据的第二个字节“b”开始，在第一数据串集合(可能包括以“b”开始的数据串)中查找以第二个字节开始的最长数据串，并通过比较该最长数据串的长度与数据串长度阈值的大小关系，确定是否将该最长数据串作为一条保留数据。Exemplarily, as shown in FIG5 , the first new file data includes "abcde...", and the first byte of the first new file data is "a". In one example, it is assumed that the first data string set includes three data strings starting with "a", namely "a", "abc", and "abcde". Starting from "a" of the first new file data, the longest data string starting with "a" that can be found in the aforementioned first data string set is "abcde". If the data string length threshold is 4, and the length of the longest data string "abcde" is greater than the data string length threshold 4, then the longest data string "abcde" is retained as data. In another example, assume that the first data string set includes three data strings starting with "a", namely "a", "ab" and "abc". Starting from "a" of the first new file data, the longest data string starting with "a" can be found in the aforementioned first data string set as "abc". If the data string length threshold is 4, and the length of the longest data string "abc" is less than the data string length threshold 4, then "abc" is not used as retained data. Starting from the second byte "b" of the first new file data, the longest data string starting with the second byte is searched in the first data string set (which may include a data string starting with "b"), and by comparing the length of the longest data string with the data string length threshold, it is determined whether to use the longest data string as a retained data.
步骤S340：根据保留数据集中保留数据在第一新文件数据的位置，确定新增数据集。Step S340: determining a newly added data set according to the position of the reserved data in the first new file data in the reserved data set.
在一种实施方式中，根据保留数据在第一新文件数据的位置，读取保留数据集中相邻两条保留数据之间的新增数据，得到新增数据集。In one implementation, according to the position of the reserved data in the first new file data, the newly added data between two adjacent reserved data in the reserved data set is read to obtain the newly added data set.
可理解，从第一新文件数据的第一个字节开始，在第一数据串集合中查找以第一个字节开始的最长数据串，通过比较最长数据串的长度与数据串长度阈值的大小关系，确定是否将最长数据串作为一条保留数据。按照第一个字节的查找方式，在遍历完第一新文件数据的所有字节后，得到保留数据集，以及保留数据集的相关信息。根据保留数据集中保留数据在第一新文件数据的位置，可以确定新增数据集。如此，完成了对一对第一新文件数据和第一旧文件数据的差分处理，可以根据保留数据集的相关信息和新增数据集生成补丁文件。It can be understood that starting from the first byte of the first new file data, the longest data string starting with the first byte is searched in the first data string set, and by comparing the length of the longest data string with the data string length threshold, it is determined whether to use the longest data string as a reserved data. According to the first byte search method, after traversing all bytes of the first new file data, a reserved data set and related information of the reserved data set are obtained. According to the position of the reserved data in the reserved data set in the first new file data, the newly added data set can be determined. In this way, the differential processing of a pair of the first new file data and the first old file data is completed, and a patch file can be generated according to the related information of the reserved data set and the newly added data set.
在一些实施例中，服务器在对一对新文件数据和旧文件数据进行差分处理，生成补丁文件的过程中，服务器可以通过多线程并行处理一对新文件数据和旧文件数据生成的多个新旧文件数据块，以提高差分处理速度。以处理第一新文件数据和第一旧文件数据为例，进行说明，具体地，请参阅图6和图7，数据处理方法还包括下述步骤：In some embodiments, when the server performs differential processing on a pair of new file data and old file data to generate a patch file, the server can process multiple new and old file data blocks generated by a pair of new file data and old file data in parallel through multi-threading to improve the differential processing speed. Taking the processing of the first new file data and the first old file data as an example, specifically, please refer to Figures 6 and 7, the data processing method also includes the following steps:
步骤S410：通过第一目标应用的第一应用线程调用预设算法库初始化预设参数。Step S410: Initializing preset parameters by calling a preset algorithm library through a first application thread of a first target application.
预设算法库储存有差分算法，可以为差分算法定义操作系统的无关性接口，通过无关性接口使差分算法适配不同的操作系统。第一应用线程通过无关性接口调用差分算法，初始化预设参数。预设参数包括数据量阈值、并行执行的处理线程的最大值M。数据量阈值指新旧文件数据块的大小阈值。The preset algorithm library stores the differential algorithm, and an operating system-independent interface can be defined for the differential algorithm, so that the differential algorithm can be adapted to different operating systems through the independent interface. The first application thread calls the differential algorithm through the independent interface to initialize the preset parameters. The preset parameters include a data volume threshold and a maximum value M of parallel processing threads. The data volume threshold refers to the size threshold of the new and old file data blocks.
可理解，在得到一对新文件数据和旧文件数据的大小后，通过预设新旧文件数据块的大小阈值，和并行执行的处理线程的最大值M，可以控制新旧文件数据块的大小和第一处理线程的数量等指标参数。工作人员可以通过修改预设参数，控制指标参数。It can be understood that after obtaining the size of a pair of new file data and old file data, by presetting the size threshold of the new and old file data blocks and the maximum value M of the processing threads executed in parallel, the index parameters such as the size of the new and old file data blocks and the number of the first processing threads can be controlled. The staff can control the index parameters by modifying the preset parameters.
步骤S420：根据第一文件更新请求，通过第一应用线程创建第一管理线程。Step S420: creating a first management thread through a first application thread according to the first file update request.
在接收第一文件更新请求后，服务器通过第一应用线程创建第一管理线程。第一管理线程用于根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块。N为大于1且小于或等于M的正整数。如图8所示，第一文件数据和第一旧文件数据可以被切分成至少3对新旧文件数据块。每对新旧文件数据块包括新文件数据块和旧文件数据块。After receiving the first file update request, the server creates a first management thread through the first application thread. The first management thread is used to split the first new file data and the first old file data according to a preset data volume threshold, respectively, to obtain N pairs of new and old file data blocks. N is a positive integer greater than 1 and less than or equal to M. As shown in FIG8 , the first file data and the first old file data can be split into at least 3 pairs of new and old file data blocks. Each pair of new and old file data blocks includes a new file data block and an old file data block.
步骤S430：通过第一管理线程创建N个第一处理线程。N个第一处理线程与N对新旧文件数据块一一对应。Step S430: Create N first processing threads through the first management thread. The N first processing threads correspond one-to-one to N pairs of new and old file data blocks.
可理解，通过第一管理线程按照根据预设的数据量阈值对一对新文件数据和旧文件数据进行切分，得到N对新旧文件数据块，并为N对新旧文件数据块创建N个第一处理线程，N个第一处理线程同时处理N对新旧文件数据块，实现了对N对新旧文件数据块的并行处理，缩短了对一对新文件数据和旧文件数据处理的差分时间。同时，通过调整并行执行的处理线程的最大值M，可以调整并行处理的线程数量，有效调整差分时间的长短。It can be understood that the first management thread divides a pair of new file data and old file data according to a preset data volume threshold to obtain N pairs of new and old file data blocks, and creates N first processing threads for the N pairs of new and old file data blocks. The N first processing threads simultaneously process the N pairs of new and old file data blocks, thereby realizing parallel processing of the N pairs of new and old file data blocks and shortening the differential time for processing a pair of new file data and old file data. At the same time, by adjusting the maximum value M of the processing threads executed in parallel, the number of threads for parallel processing can be adjusted, and the length of the differential time can be effectively adjusted.
步骤S440：通过N个第一处理线程并行执行对N对新旧文件数据块的数据比对，得到N对新旧文件数据块的补丁文件。Step S440: using N first processing threads to perform data comparison on N pairs of new and old file data blocks in parallel, to obtain patch files of the N pairs of new and old file data blocks.
每个第一处理线程单独将一块新旧文件数据块读入内存，进行数据对比，得到每块新旧文件数据块对应的一个补丁信息。具体方式与上述步骤S120至S130的技术原理一致，区别在于，本申请实施例的差分处理对象为新旧文件数据块，差分处理生成的为补丁信息。具体地，具体方式步骤包括：Each first processing thread reads a new and old file data block into the memory separately, performs data comparison, and obtains a patch information corresponding to each new and old file data block. The specific method is consistent with the technical principle of the above steps S120 to S130, except that the differential processing object of the embodiment of the present application is the new and old file data block, and the differential processing generates the patch information. Specifically, the specific method steps include:
步骤b1：第一处理线程为新旧文件数据块的旧文件数据块创建多个后缀。Step b1: the first processing thread creates multiple suffixes for the old file data blocks of the new and old file data blocks.
步骤b2：将多个后缀按从小到大的顺序后，得到后缀数组。Step b2: Arrange multiple suffixes in ascending order to obtain a suffix array.
步骤b3：从新旧文件数据块的新文件数据块的第1个字节开始，在后缀数组中查找以第1个字节开始的最长数据串。Step b3: Starting from the first byte of the new file data block of the new and old file data blocks, search the suffix array for the longest data string starting with the first byte.
步骤b4：若最长数据串的长度Len大于12字节，则最长数据串作为保留数据，并记录保留数据的相关信息。且从新文件数据块的第Len+1个字节开始，继续在后缀数组中查找以第Len+1个字节开始的最长数据串，直至新文件数据块的所有字节都遍历完。Step b4: If the length Len of the longest data string is greater than 12 bytes, the longest data string is used as the reserved data, and the relevant information of the reserved data is recorded. Starting from the Len+1th byte of the new file data block, continue to search for the longest data string starting with the Len+1th byte in the suffix array until all bytes of the new file data block are traversed.
步骤b5：若最长数据串的长度Len小于或等于12字节，从新文件数据块的第2个字节开始，继续在后缀数组中查找以第2个字节开始的最长数据串，直至新文件数据块的所有字节都遍历完。Step b5: If the length Len of the longest data string is less than or equal to 12 bytes, start from the second byte of the new file data block and continue to search the suffix array for the longest data string starting with the second byte until all bytes of the new file data block are traversed.
步骤b6：根据保留数据的相关信息和相邻保留数据之间的新增数据生成补丁信息。Step b6: Generate patch information according to the relevant information of the reserved data and the newly added data between adjacent reserved data.
步骤b1至b6中的未详细展开阐述的技术特征，参见步骤S120至S130以及其细化步骤。For the technical features in steps b1 to b6 that are not elaborated in detail, please refer to steps S120 to S130 and their detailed steps.
步骤S450：通过第一管理线程根据N对新旧文件数据块的补丁文件，生成第一新文件数据的补丁文件。Step S450: Generate a patch file for the first new file data according to N pairs of patch files of new and old file data blocks through the first management thread.
根据N对新旧文件数据块对应的N个补丁信息可以得到一个补丁文件，至此完成对一对新文件数据和旧文件数据的差分过程。A patch file can be obtained according to the N patch information corresponding to the N pairs of new and old file data blocks, thus completing the difference process of a pair of new file data and old file data.
在一种实施方式中，通过第一管理线程将一对新文件数据和旧文件数据切分成多个新旧文件数据块，分几次创建多个第一处理线程，每次多个第一处理线程将多个新旧文件数据块中的几个新旧文件数据块读入内存进行差分处理，生成补丁信息后，删除新旧文件数据块，从而有效减小内存的占用。In one embodiment, a pair of new file data and old file data are divided into multiple new and old file data blocks through a first management thread, and multiple first processing threads are created several times. Each time, the multiple first processing threads read several new and old file data blocks from the multiple new and old file data blocks into the memory for differential processing. After the patch information is generated, the new and old file data blocks are deleted, thereby effectively reducing memory occupancy.
在一些实施例中，还可以同时并行处理多对新文件数据和旧文件数据，得到多对新文件数据和旧文件数对应的多个补丁文件，请再参阅图7，以同时处理两对新文件数据和旧文件数据进行说明。数据处理方法还包括：In some embodiments, multiple pairs of new file data and old file data may be processed in parallel to obtain multiple patch files corresponding to the multiple pairs of new file data and old file data. Please refer to FIG. 7 again to illustrate the processing of two pairs of new file data and old file data at the same time. The data processing method also includes:
步骤S510：在接收第一文件更新请求的同时，接收来自第一设备的第二文件更新请求，第二文件更新请求包括第二目标应用的第三新文件数据。Step S510: while receiving the first file update request, receiving a second file update request from the first device, the second file update request including third new file data of the second target application.
第二目标应用的功能区别于第一目标应用，第二目标应用也可以包括导航和地图应用程序、娱乐应用程序、通讯应用程序、智能驾驶辅助应用程序或停车或充电应用程序等。示例性地，第一目标应用为智能驾驶辅助应用程序，第二目标应用为通讯应用程序。The function of the second target application is different from that of the first target application, and the second target application may also include a navigation and map application, an entertainment application, a communication application, an intelligent driving assistance application, or a parking or charging application, etc. Exemplarily, the first target application is an intelligent driving assistance application, and the second target application is a communication application.
步骤S520：根据第二文件更新请求，创建第二目标应用的第二应用线程。Step S520: creating a second application thread of a second target application according to the second file update request.
第二应用线程用于通过调用预设算法库初始化预设参数。The second application thread is used to initialize preset parameters by calling a preset algorithm library.
步骤S530：通过第二应用线程创建第二管理线程。Step S530: Create a second management thread through the second application thread.
步骤S540：通过第二管理线程创建N个第二处理线程。N个第二处理线程与N对新旧文件数据块一一对应。Step S540: Create N second processing threads through the second management thread. The N second processing threads correspond one-to-one to the N pairs of new and old file data blocks.
步骤S550：通过N个第二处理线程并行执行对N对新旧文件数据块的数据比对，得到N对新旧文件数据块的补丁文件。Step S550: using N second processing threads to perform data comparison on N pairs of new and old file data blocks in parallel, to obtain patch files of the N pairs of new and old file data blocks.
在一些实施例中，差分处理过程中，若多个应用线程之间、多个管理线程之间、应用线程与管理线程之间或管理线程与处理线程之间同时对同一数据进行操作，先操作的线程获取互斥锁，以防止线程之间读取数据功能紊乱。具体地，请参阅图9，数据处理方法还包括下述步骤：In some embodiments, during the differential processing, if multiple application threads, multiple management threads, application threads and management threads, or management threads and processing threads simultaneously operate on the same data, the thread that operates first acquires a mutex lock to prevent the data reading function between threads from being disrupted. Specifically, referring to FIG9 , the data processing method further includes the following steps:
步骤S610：第一应用线程在第一存储空间内写入数据时，禁止第一管理线程在第一存储空间删除数据。Step S610: When the first application thread writes data in the first storage space, the first management thread is prohibited from deleting data in the first storage space.
具体地，第一存储空间用于储存差分处理过程的信息，包括基于第一文件更新请求产生的N个第一处理线程的信息。在差分处理过程中，根据第一文件更新请求，第一应用线程在第一存储空间中写入第一文件更新请求对应的N个第一处理线程的信息。第一管理线程从第一存储空间中读取第一文件更新请求对应的N个第一处理线程的信息，并创建N个第一处理线程。在差分处理完成后，第一管理线程还用于在生成第一新文件数据的补丁文件后，删除对应N个第一处理线程的信息。因此，在整个差分处理过程中，第一应用线程和第一管理线程可能同时对第一存储空间的数据进行操作。在第一应用线程在第一存储空间内写入数据(例如N个第一处理线程的信息)时，禁止第一管理线程在第一存储空间删除数据(例如N个第一处理线程的信息)，也即第一应用线程获得互斥锁，以防止第一应用线程和第一管理线程之间读取数据的功能紊乱。Specifically, the first storage space is used to store information of the differential processing process, including information of N first processing threads generated based on the first file update request. During the differential processing process, according to the first file update request, the first application thread writes the information of the N first processing threads corresponding to the first file update request in the first storage space. The first management thread reads the information of the N first processing threads corresponding to the first file update request from the first storage space and creates N first processing threads. After the differential processing is completed, the first management thread is also used to delete the information corresponding to the N first processing threads after generating the patch file of the first new file data. Therefore, during the entire differential processing process, the first application thread and the first management thread may operate on the data in the first storage space at the same time. When the first application thread writes data (such as information of N first processing threads) in the first storage space, the first management thread is prohibited from deleting data (such as information of N first processing threads) in the first storage space, that is, the first application thread obtains a mutex lock to prevent the function of reading data between the first application thread and the first management thread from being disordered.
步骤S620：第一处理线程在第二存储空间写入数据时，禁止第一管理线程在第二存储空间读取数据。Step S620: When the first processing thread writes data in the second storage space, the first management thread is prohibited from reading data in the second storage space.
具体地，第二存储空间用于存储第一处理线程的处理状态和/或处理进度。处理状态指补丁信息是否成功生成，或者一对新旧文件数据块是否成功差分。处理进度是指一对新旧文件数据块的差分进度，例如差分进度为5％和10％等。Specifically, the second storage space is used to store the processing status and/or processing progress of the first processing thread. The processing status refers to whether the patch information is successfully generated, or whether a pair of new and old file data blocks are successfully differentiated. The processing progress refers to the differential progress of a pair of new and old file data blocks, for example, the differential progress is 5% and 10%, etc.
在差分处理过程中，第一处理线程用于在第二存储空间写入第一处理线程的处理状态和/或处理进度。第一管理线程用于从第二存储空间读取第一处理线程的处理状态和/或处理进度。从而第一管理线程可以根据处理状态和/或处理进度，管理第一处理线程。因此，第一管理线程和第一处理线程也可能同时对第二存储空间的数据进行操作。在第一处理线程在第二存储空间写入数据时，禁止第一管理线程在第二存储空间读取数据，也即第一处理线程获得互斥锁，以防止第一管理线程和第一处理线程之间读取数据的功能紊乱。During the differential processing, the first processing thread is used to write the processing status and/or processing progress of the first processing thread into the second storage space. The first management thread is used to read the processing status and/or processing progress of the first processing thread from the second storage space. Thus, the first management thread can manage the first processing thread according to the processing status and/or processing progress. Therefore, the first management thread and the first processing thread may also operate on the data in the second storage space at the same time. When the first processing thread writes data in the second storage space, the first management thread is prohibited from reading data in the second storage space, that is, the first processing thread obtains a mutex lock to prevent the function of reading data between the first management thread and the first processing thread from being disordered.
步骤S630：两个第一管理线程的其中一个第一管理线程在读取第一旧文件数据时，禁止两个第一管理线程的另一个第一管理线程读取第一旧文件数据。Step S630: when one of the two first management threads reads the first old file data, the other first management thread of the two first management threads is prohibited from reading the first old file data.
具体地，若第一文件更新请求包括第一目标应用的第一新文件数据和第二新文件数据，其中，第一新文件数据和第二新文件数据均对应第一旧文件数据，则通过第一应用程序创建两个第一管理线程。Specifically, if the first file update request includes first new file data and second new file data of the first target application, wherein the first new file data and the second new file data both correspond to first old file data, two first management threads are created through the first application program.
也即，请再参阅图7，本申请的数据处理方法，还可以同时将第一目标应用升级成不同版本，第一新文件数据和第二新文件数据分别为不同版本的第一目标应用数据。通过第一应用程序创建两个第一管理线程。其中，一个第一管理线程1根据第一旧文件数据和第一新文件数据得到一个版本的目标应用的补丁文件。另一个第一管理线程2根据第一旧文件数据和第二新文件数据得到另一个版本的目标应用的补丁文件。因此，两个第一管理线程可能同时读取第一旧文件数据，通过第一管理线程1在读取第一旧文件数据时，禁止第一管理线程2读取第一旧文件数据，也即先读取第一旧文件数的第一管理线程获取互斥锁，以防止两个管理线程之间读取数据的功能紊乱。That is, please refer to Figure 7 again. The data processing method of the present application can also upgrade the first target application to different versions at the same time, and the first new file data and the second new file data are different versions of the first target application data. Two first management threads are created through the first application. Among them, one first management thread 1 obtains a patch file of a version of the target application based on the first old file data and the first new file data. Another first management thread 2 obtains a patch file of another version of the target application based on the first old file data and the second new file data. Therefore, the two first management threads may read the first old file data at the same time. When the first management thread 1 reads the first old file data, the first management thread 2 is prohibited from reading the first old file data, that is, the first management thread that reads the first old file data first obtains a mutex lock to prevent functional disorder of reading data between the two management threads.
步骤S640：第一应用线程在调用预设算法库时，禁止第二应用线程调用预设算法库。Step S640: When the first application thread calls the preset algorithm library, the second application thread is prohibited from calling the preset algorithm library.
可理解，第一应用线程和第二应用线程同时调用预设算法库，初始化预设参数，且第一应用线程和第二应用线程初始化的预设参数不一样时，允许先调用预设算法库的第一应用线程调用，禁止第二应用线程调用。也即第一应用线程获得互斥锁。以防止两个应用线程之间读取数据的功能紊乱。It can be understood that when the first application thread and the second application thread call the preset algorithm library at the same time to initialize the preset parameters, and the preset parameters initialized by the first application thread and the second application thread are different, the first application thread that calls the preset algorithm library first is allowed to call, and the second application thread is prohibited from calling. That is, the first application thread obtains a mutex lock to prevent the function of reading data between the two application threads from being disordered.
可理解，步骤S610至步骤S640之间的顺序不做限定。It is understandable that the order between step S610 to step S640 is not limited.
需要注意的是，相关技术中的多种差分算法的开源实现方案，有的注重补丁文件的大小，有的注重差分还原的时间长短，有的注重占用内存或CPU的多少，还有的是针对特定格式的文件。注重的特性单一，无法适应复杂多变的应用场景。It should be noted that some open source implementations of various differential algorithms in related technologies focus on the size of patch files, some focus on the length of time for differential restoration, some focus on the amount of memory or CPU occupied, and some are for files in specific formats. They focus on a single feature and cannot adapt to complex and changing application scenarios.
相比相关技术的开源实现方案，上述部分实施例中，应用线程、处理线程和管理线程的设计，通过无关性接口适配不同的操作系统，通过初始化预设参数，可以控制新旧文件数据块的大小和处理线程的数量等指标参数，N个第一处理线程同时处理N对新旧文件数据块，实现了对N对新旧文件数据块的并行处理，缩短了对一对新文件数据和旧文件数据处理的差分时间，有效减小内存的占用。通过互斥锁，防止线程之间读取数据功能紊乱，能够兼顾多种特性。Compared with the open source implementation scheme of the related technology, in the above-mentioned embodiments, the design of the application thread, the processing thread and the management thread can adapt to different operating systems through an irrelevant interface, and by initializing preset parameters, the size of the new and old file data blocks and the number of processing threads and other index parameters can be controlled. N first processing threads simultaneously process N pairs of new and old file data blocks, realizing parallel processing of N pairs of new and old file data blocks, shortening the differential time for processing a pair of new file data and old file data, and effectively reducing memory usage. Mutex locks are used to prevent the disorder of data reading functions between threads, and multiple characteristics can be taken into account.
本申请还提供一种数据处理方法，数据处理方法应用于终端设备，请参阅图10和图11，数据处理方法包括下述步骤：The present application also provides a data processing method, which is applied to a terminal device. Please refer to FIG. 10 and FIG. 11 . The data processing method includes the following steps:
S710：接收来自服务器的升级响应。S710: Receive an upgrade response from the server.
S720：根据补丁文件的至少一条保留数据在第一旧文件数据的位置信息，以及至少一条保留数据的数据长度，从第一旧文件数据中读取至少一条保留数据。S720: Read at least one piece of reserved data from the first old file data according to the location information of the at least one piece of reserved data of the patch file in the first old file data and the data length of the at least one piece of reserved data.
S730：根据至少一条保留数据中的相邻保留数据之间的间隔长度，从补丁文件中读取与间隔长度相同长度的新增数据，以及至少一条保留数据在第一新文件数据的位置信息。S730: Reading, from the patch file, new data having the same length as the interval length between adjacent reserved data in the at least one piece of reserved data, and position information of the at least one piece of reserved data in the first new file data.
S740：根据新增数据、至少一条保留数据在第一新文件数据的位置信息以及至少一条保留数据块生成第一目标应用的第一新文件数据。S740: Generate first new file data of a first target application according to the newly added data, the location information of the at least one reserved data in the first new file data, and the at least one reserved data block.
具体地，为还原算法定义操作系统的无关性接口，通过无关性接口还原算法适配不同的操作系统。请再参阅图11，终端设备接收包括第一目标应用的补丁文件的升级响应，通过无关性接口调用还原算法初始化预设参数，执行下列步骤：Specifically, an operating system-independent interface is defined for the restoration algorithm, and the restoration algorithm is adapted to different operating systems through the irrelevant interface. Please refer to Figure 11 again, the terminal device receives an upgrade response including a patch file of the first target application, calls the restoration algorithm through the irrelevant interface to initialize preset parameters, and executes the following steps:
步骤c1:将补丁文件加载至内存，从补丁文件中读出1个新旧文件数据块的补丁信息。Step c1: Load the patch file into the memory, and read the patch information of a new and old file data block from the patch file.
步骤c2：从补丁信息中，取出一个保留数据的相关信息，然后根据相关信息中记录的旧文件的起始位置，以及保留数据的数据长度，从磁盘中的旧文件中读取保留数据。Step c2: Retrieve relevant information of the reserved data from the patch information, and then read the reserved data from the old file in the disk according to the starting position of the old file recorded in the relevant information and the data length of the reserved data.
步骤c3：根据相邻保留数据之间的间隔长度，从补丁信息中，读取与间隔长度相同长度的新增数据。Step c3: According to the interval length between adjacent retained data, read the newly added data with the same length as the interval length from the patch information.
步骤c4：依次处理所有保留数据，从而还原出所有新增数据，并根据新增数据、至少一条保留数据在第一新文件数据的位置信息以及至少一条保留数据生成新文件数据块。Step c4: Process all the reserved data in sequence to restore all the new data, and generate a new file data block according to the new data, the position information of the at least one reserved data in the first new file data, and the at least one reserved data.
步骤c5：重复c1-c4步骤，直到根据新文件数据块还原出整个第一新文件数据为止。Step c5: Repeat steps c1-c4 until the entire first new file data is restored based on the new file data block.
可理解，还原算法存在大量输入/输出(I/O)操作，所以可以把第一旧文件数据的保留数据、补丁文件一次性读入内存，并把还原出的第一新文件数据暂存到内存，直到还原完成后再一次写入磁盘。It can be understood that the restoration algorithm involves a large number of input/output (I/O) operations, so the retained data and patch files of the first old file data can be read into the memory at one time, and the restored first new file data can be temporarily stored in the memory until it is written to the disk again after the restoration is completed.
可理解，考虑到还原算法经常运行在内存较少的环境中，所以在差分处理时，通过第一管理线程将新旧文件数据块切分的成指定大小，可以满足还原时的内存需求。It is understandable that, considering that the restoration algorithm often runs in an environment with less memory, during differential processing, the first management thread divides the new and old file data blocks into specified sizes to meet the memory requirements during restoration.
请结合图12和图1，在本申请的再一应用场景中，服务器10接收来自第一设备20的文件更新请求，文件更新请求包括智能驾驶辅助应用程序的新文件数据，获取智能驾驶辅助应用程序的旧文件数据，以及数据串集合。接着，根据数据串集合以及预设的数据串长度阈值，确定新文件数据相较于旧文件数据的保留数据集和新增数据集。最后，根据至少一条保留数据的数据长度、至少一条新增数据在新文件数据的位置信息、以及至少一条保留数据在智能驾驶辅助应用程序的新文件数据和旧文件数据的位置信息，生成新文件数据的补丁文件。Please refer to Figure 12 and Figure 1. In another application scenario of the present application, the server 10 receives a file update request from the first device 20, and the file update request includes new file data of the intelligent driving assistance application, obtains old file data of the intelligent driving assistance application, and a data string set. Then, based on the data string set and the preset data string length threshold, the retained data set and the newly added data set of the new file data compared to the old file data are determined. Finally, based on the data length of at least one retained data, the location information of at least one newly added data in the new file data, and the location information of at least one retained data in the new file data and the old file data of the intelligent driving assistance application, a patch file for the new file data is generated.
服务器10接收车辆30的升级请求，发送升级响应至车辆30。车辆30根据升级响应的补丁文件中，至少一条保留数据在智能驾驶辅助应用程序的旧文件数据的位置信息，以及至少一条保留数据的数据长度，从旧文件数据中读取至少一条保留数据。接着，根据至少一条保留数据中的相邻保留数据之间的间隔长度，从补丁文件中读取与间隔长度相同长度的新增数据，以及至少一条保留数据在智能驾驶辅助应用程序的新文件数据的位置信息。The server 10 receives the upgrade request of the vehicle 30 and sends an upgrade response to the vehicle 30. The vehicle 30 reads at least one piece of reserved data from the old file data according to the location information of at least one piece of reserved data in the old file data of the intelligent driving assistance application in the patch file of the upgrade response and the data length of at least one piece of reserved data. Then, according to the interval length between adjacent pieces of reserved data in the at least one piece of reserved data, the vehicle 30 reads new data with the same length as the interval length from the patch file and the location information of at least one piece of reserved data in the new file data of the intelligent driving assistance application.
根据新增数据、至少一条保留数据在新文件数据的位置信息以及至少一条保留数据块生成智能驾驶辅助应用程序的新文件数据，完成部署在车辆30的智能驾驶辅助应用程序的升级。New file data of the intelligent driving assistance application is generated according to the newly added data, the location information of at least one retained data in the new file data, and at least one retained data block, thereby completing the upgrade of the intelligent driving assistance application deployed in the vehicle 30.
图13为本申请提供的数据处理方法的数据处理装置的一实施例的结构示意图。该装置40可以集成于上述方法实施例中的服务器中，也可以通过上述方法实施例中的服务器实现。如图13所示，该数据处理装置40包括第一接收模块41、获取模块42、确定模块43和补丁文件生成模块44。FIG13 is a schematic diagram of the structure of an embodiment of a data processing device of the data processing method provided by the present application. The device 40 can be integrated into the server in the above method embodiment, or can be implemented by the server in the above method embodiment. As shown in FIG13 , the data processing device 40 includes a first receiving module 41, an acquisition module 42, a determination module 43, and a patch file generation module 44.
第一接收模块41用于，接收来自第一设备的第一文件更新请求，第一文件更新请求包括第一目标应用的第一新文件数据。The first receiving module 41 is used to receive a first file update request from a first device, where the first file update request includes first new file data of a first target application.
获取模块42用于，获取第一新文件数据对应的第一旧文件数据，以及第一数据串集合，第一数据串集合是基于第一旧文件数据创建的不同长度的数据串。The acquisition module 42 is used to acquire the first old file data corresponding to the first new file data and a first data string set, where the first data string set is data strings of different lengths created based on the first old file data.
确定模块43用于，根据第一数据串集合以及预设的数据串长度阈值，确定第一新文件数据相较于第一旧文件数据的保留数据集和新增数据集，保留数据集包括至少一条保留数据，新增数据集包括至少一条新增数据，每条保留数据的数据长度大于数据串长度阈值。The determination module 43 is used to determine, based on the first data string set and a preset data string length threshold, a retained data set and a newly added data set of the first new file data compared to the first old file data, the retained data set includes at least one retained data, the newly added data set includes at least one newly added data, and the data length of each retained data is greater than the data string length threshold.
补丁文件生成模块44，用于根据至少一条保留数据的数据长度、至少一条新增数据在第一新文件数据的位置信息、以及至少一条保留数据分别在第一新文件数据和第一旧文件数据的位置信息，生成第一新文件数据的补丁文件，补丁文件用于对第一目标应用进行数据更新。The patch file generation module 44 is used to generate a patch file for the first new file data based on the data length of at least one retained data, the position information of at least one newly added data in the first new file data, and the position information of at least one retained data in the first new file data and the first old file data respectively. The patch file is used to update the data of the first target application.
在一些实施例中，确定模块43还用于，从第一新文件数据的第一个字节开始，在第一数据串集合中查找以第一个字节开始的最长数据串。通过比较最长数据串的长度与数据串长度阈值的大小关系，确定是否将最长数据串作为一条保留数据。在遍历完第一新文件数据的所有字节后，得到保留数据集。根据保留数据集中保留数据在第一新文件数据的位置，确定新增数据集。In some embodiments, the determination module 43 is further used to search the first data string set for the longest data string starting with the first byte starting from the first byte of the first new file data. By comparing the length of the longest data string with the data string length threshold, it is determined whether to use the longest data string as a reserved data. After traversing all bytes of the first new file data, a reserved data set is obtained. According to the position of the reserved data in the reserved data set in the first new file data, a newly added data set is determined.
在一些实施例中，确定模块43还用于，若最长数据串的长度大于数据串长度阈值，确定将最长数据串作为一条保留数据，且从第一新文件数据的目标字节开始，在第一数据串集合中查找是否有以目标字节开始的最长数据串，其中，目标字节为第一个字节累加第一个字节开始的最长数据串的长度后确定的字节；若最长数据串的长度小于或等于数据串长度阈值，从第一个字节的下一个字节开始，在第一数据串集合中继续查找是否有以下一个字节开始的最长数据串。In some embodiments, the determination module 43 is also used to, if the length of the longest data string is greater than a data string length threshold, determine that the longest data string is to be retained as a piece of data, and start from the target byte of the first new file data, search in the first data string set whether there is a longest data string starting with the target byte, wherein the target byte is the byte determined by adding the length of the longest data string starting with the first byte to the first byte; if the length of the longest data string is less than or equal to the data string length threshold, start from the byte after the first byte, and continue to search in the first data string set whether there is a longest data string starting with the next byte.
在一些实施例中，数据处理装置40还包括线程处理模块45，线程处理模块45用于根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块，N为大于1且小于或等于M的正整数，M为并行执行的处理线程的最大值；创建N个第一处理线程，N个第一处理线程与N对新旧文件数据块一一对应；通过N个第一处理线程并行执行对N对新旧文件数据块的数据比对，得到N对新旧文件数据块的补丁文件；根据N对新旧文件数据块的补丁文件，生成第一新文件数据的补丁文件。In some embodiments, the data processing device 40 also includes a thread processing module 45, which is used to perform data segmentation on the first new file data and the first old file data according to a preset data volume threshold, respectively, to obtain N pairs of new and old file data blocks, where N is a positive integer greater than 1 and less than or equal to M, and M is the maximum value of processing threads executed in parallel; create N first processing threads, and the N first processing threads correspond one-to-one to the N pairs of new and old file data blocks; obtain patch files of N pairs of new and old file data blocks by performing data comparison on N pairs of new and old file data blocks in parallel through the N first processing threads; and generate a patch file of the first new file data based on the patch files of the N pairs of new and old file data blocks.
在一些实施例中，线程处理模块45还用于，通过第一目标应用的第一应用线程调用预设算法库初始化预设参数，预设参数包括数据量阈值和M；根据第一文件更新请求，通过第一应用线程创建第一管理线程。以及通过第一管理线程执行根据预设的数据量阈值，分别对第一新文件数据和第一旧文件数据进行数据切分，得到N对新旧文件数据块的步骤；以及通过第一管理线程执行根据N对新旧文件数据块的补丁文件，生成第一新文件数据的补丁文件的步骤。In some embodiments, the thread processing module 45 is further used to call the preset algorithm library through the first application thread of the first target application to initialize the preset parameters, the preset parameters including the data volume threshold and M; create the first management thread through the first application thread according to the first file update request. And execute the step of splitting the first new file data and the first old file data according to the preset data volume threshold respectively through the first management thread to obtain N pairs of new and old file data blocks; and execute the step of generating the patch file of the first new file data according to the patch files of the N pairs of new and old file data blocks through the first management thread.
在一些实施例中，线程处理模块45还用于，第一应用线程还用于在第一存储空间中写入第一文件更新请求对应的N个第一处理线程的信息；第一管理线程还用于在生成第一新文件数据的补丁文件后，删除N个第一处理线程的信息；第一应用线程在第一存储空间内写入数据时，禁止第一管理线程在第一存储空间删除数据。In some embodiments, the thread processing module 45 is also used for the first application thread to write information of N first processing threads corresponding to the first file update request in the first storage space; the first management thread is also used to delete the information of N first processing threads after generating a patch file for the first new file data; when the first application thread writes data in the first storage space, the first management thread is prohibited from deleting data in the first storage space.
在一些实施例中，线程处理模块45还用于，第一处理线程用于在第二存储空间写入第一处理线程的处理状态和/或处理进度，第一管理线程用于从第二存储空间读取第一处理线程的处理状态和/或处理进度；第一处理线程在第二存储空间写入数据时，禁止第一管理线程在第二存储空间读取数据。In some embodiments, the thread processing module 45 is also used for the first processing thread to write the processing status and/or processing progress of the first processing thread in the second storage space, and the first management thread to read the processing status and/or processing progress of the first processing thread from the second storage space; when the first processing thread writes data in the second storage space, the first management thread is prohibited from reading data in the second storage space.
在一些实施例中，线程处理模块45还用于，通过第一应用程序创建两个第一管理线程；两个第一管理线程的其中一个第一管理线程在读取第一旧文件数据时，禁止两个第一管理线程的另一个第一管理线程读取第一旧文件数据。In some embodiments, the thread processing module 45 is further used to create two first management threads through the first application; when one of the two first management threads reads the first old file data, the other first management thread of the two first management threads is prohibited from reading the first old file data.
在一些实施例中，线程处理模块45还用于，在接收第一文件更新请求的同时，接收来自第一设备的第二文件更新请求，第二文件更新请求包括第二目标应用的第三新文件数据；根据第二文件更新请求，创建第二目标应用的第二应用线程，第二应用线程用于通过调用预设算法库初始化预设参数；通过第二应用线程创建第二管理线程；第一应用线程在调用预设算法库时，禁止第二应用线程调用预设算法库。In some embodiments, the thread processing module 45 is also used to receive a second file update request from the first device while receiving the first file update request, the second file update request including third new file data of the second target application; create a second application thread of the second target application according to the second file update request, the second application thread being used to initialize preset parameters by calling a preset algorithm library; create a second management thread through the second application thread; and prohibit the second application thread from calling the preset algorithm library when the first application thread calls the preset algorithm library.
在一些实施例中，第一接收模块41还用于，接收来自第二设备的升级请求，升级请求用于请求升级第一目标应用；响应于升级请求，获取第一目标应用的补丁文件；向第二设备发送升级响应，升级响应包括第一目标应用的补丁文件。In some embodiments, the first receiving module 41 is also used to receive an upgrade request from the second device, the upgrade request is used to request an upgrade of the first target application; in response to the upgrade request, obtain a patch file of the first target application; and send an upgrade response to the second device, the upgrade response including the patch file of the first target application.
本实施例提供的数据处理装置40，用于执行前述服务器执行的方法的实施例中的技术方案，其实现原理和技术效果类似，在此不再赘述。The data processing device 40 provided in this embodiment is used to execute the technical solution in the embodiment of the method executed by the aforementioned server. Its implementation principle and technical effect are similar and will not be repeated here.
图14为本申请提供的数据处理方法的数据处理装置的另一实施例的结构示意图。该装置50可以集成于上述方法实施例中的终端设备中，也可以通过上述方法实施例中的终端设备实现。如图14所示，该数据处理装置50包括第二接收模块51、第一读取模块52、第二读取模块53和新文件数据生成模块54。FIG14 is a schematic diagram of the structure of another embodiment of the data processing device of the data processing method provided by the present application. The device 50 can be integrated into the terminal device in the above method embodiment, or can be implemented by the terminal device in the above method embodiment. As shown in FIG14, the data processing device 50 includes a second receiving module 51, a first reading module 52, a second reading module 53 and a new file data generating module 54.
第二接收模块51用于，接收来自服务器的升级响应，升级响应包括第一目标应用的补丁文件。The second receiving module 51 is used to receive an upgrade response from the server, where the upgrade response includes a patch file of the first target application.
第一读取模块52用于，根据补丁文件的至少一条保留数据在第一旧文件数据的位置信息，以及至少一条保留数据的数据长度，从第一旧文件数据中读取至少一条保留数据。The first reading module 52 is used for reading at least one piece of reserved data from the first old file data according to the position information of the at least one piece of reserved data of the patch file in the first old file data and the data length of the at least one piece of reserved data.
第二读取模块53用于，根据至少一条保留数据中的相邻保留数据之间的间隔长度，从补丁文件中读取与间隔长度相同长度的新增数据，以及至少一条保留数据在第一新文件数据的位置信息。The second reading module 53 is used to read, according to the interval length between adjacent reserved data in at least one piece of reserved data, new data having the same length as the interval length and position information of at least one piece of reserved data in the first new file data from the patch file.
新文件数据生成模块54用于，根据新增数据、至少一条保留数据在第一新文件数据的位置信息以及至少一条保留数据块生成第一目标应用的第一新文件数据。The new file data generating module 54 is used to generate first new file data of a first target application according to the newly added data, the position information of at least one reserved data in the first new file data, and at least one reserved data block.
图15为本申请提供的一种服务器的结构示意图。如图15所示，该服务器60包括：处理器61，存储器62和总线63；Fig. 15 is a schematic diagram of the structure of a server provided by the present application. As shown in Fig. 15, the server 60 includes: a processor 61, a memory 62 and a bus 63;
所述存储器62用于存储所述处理器61的计算机程序代码；The memory 62 is used to store the computer program code of the processor 61;
其中，所述处理器61配置为经由执行所述计算机程序代码来执行前述任一方法实施例中服务器的技术方案。The processor 61 is configured to execute the technical solution of the server in any of the aforementioned method embodiments by executing the computer program code.
可选的，存储器62既可以是独立的，也可以跟处理器61集成在一起。Optionally, the memory 62 may be independent or integrated with the processor 61 .
存储器62、通过总线63与处理器61连接并完成相互间的通信。The memory 62 is connected to the processor 61 via a bus 63 and completes communication with each other.
可选的，存储器62可能包含随机存取存储器(random access memory，RAM)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一条磁盘存储器。Optionally, the memory 62 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
总线63可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。The bus 63 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
上述的处理器可以是通用处理器，包括中央处理器CPU、网络处理器(networkprocessor，NP)等；还可以是数字信号处理器DSP、专用集成电路ASIC、现场可编程门阵列FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
该服务器用于执行前述任一方法实施例中服务器的技术方案，其实现原理和技术效果类似，在此不再赘述。The server is used to execute the technical solution of the server in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.
图16为本申请提供的一种终端设备的结构示意图。如图16所示，该终端设备70包括：处理器71，存储器72和总线73；Fig. 16 is a schematic diagram of the structure of a terminal device provided by the present application. As shown in Fig. 16, the terminal device 70 includes: a processor 71, a memory 72 and a bus 73;
所述存储器72用于存储所述处理器71的计算机程序代码；The memory 72 is used to store the computer program code of the processor 71;
其中，所述处理器71配置为经由执行所述计算机程序代码来执行前述任一方法实施例中服务器的技术方案。The processor 71 is configured to execute the technical solution of the server in any of the aforementioned method embodiments by executing the computer program code.
可选的，存储器72既可以是独立的，也可以跟处理器71集成在一起。Optionally, the memory 72 can be independent or integrated with the processor 71.
存储器72、通过总线73与处理器71连接并完成相互间的通信。The memory 72 is connected to the processor 71 via a bus 73 and completes communication with each other.
可选的，存储器72可能包含随机存取存储器(random access memory，RAM)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一条磁盘存储器。Optionally, the memory 72 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
总线73可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。The bus 73 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
上述的处理器可以是通用处理器，包括中央处理器CPU、网络处理器(networkprocessor，NP)等；还可以是数字信号处理器DSP、专用集成电路ASIC、现场可编程门阵列FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
该终端设备用于执行前述任一方法实施例中终端设备的技术方案，其实现原理和技术效果类似，在此不再赘述。The terminal device is used to execute the technical solution of the terminal device in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.
本申请实施例还提供一种可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现前述任一实施例提供的技术方案。An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided in any of the aforementioned embodiments is implemented.
本申请实施例还提供一种芯片系统，芯片系统应用于服务器或终端设备，芯片系统包括一个或多个处理器，一个或多个处理器用于调用计算机指令以使得服务器或终端设备实现前述任一实施例提供的技术方案。An embodiment of the present application also provides a chip system, which is applied to a server or a terminal device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the server or the terminal device implements the technical solution provided by any of the aforementioned embodiments.
本申请实施例还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时用于实现前述任一方法实施例提供的技术方案。An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned method embodiments when executed by a processor.
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或对其中部分或全部技术特征进行等同替换；而这些修改或替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411153163.3ACN118916058A (en) | 2024-08-20 | 2024-08-20 | Data processing method, server and terminal equipment |
| PCT/CN2025/071337WO2025148943A1 (en) | 2024-01-08 | 2025-01-08 | Data processing method, server, and terminal device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411153163.3ACN118916058A (en) | 2024-08-20 | 2024-08-20 | Data processing method, server and terminal equipment |
| Publication Number | Publication Date |
|---|---|
| CN118916058Atrue CN118916058A (en) | 2024-11-08 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411153163.3APendingCN118916058A (en) | 2024-01-08 | 2024-08-20 | Data processing method, server and terminal equipment |
| Country | Link |
|---|---|
| CN (1) | CN118916058A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025148943A1 (en)* | 2024-01-08 | 2025-07-17 | 浙江极氪智能科技有限公司 | Data processing method, server, and terminal device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1726476A (en)* | 2002-10-31 | 2006-01-25 | 松下电器产业株式会社 | Data update system, differential data creation device and program for data update system, file restoration device and program after update |
| KR102088167B1 (en)* | 2019-08-27 | 2020-03-12 | 루나 주식회사 | Apparatus for updating software as an agent and patch method thereof |
| CN112416386A (en)* | 2020-10-26 | 2021-02-26 | 北京一亩田新农网络科技有限公司 | Method, device, system and medium for software upgrading |
| CN116028095A (en)* | 2022-06-30 | 2023-04-28 | 荣耀终端有限公司 | Generation method, restoration method and electronic equipment of differential file |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1726476A (en)* | 2002-10-31 | 2006-01-25 | 松下电器产业株式会社 | Data update system, differential data creation device and program for data update system, file restoration device and program after update |
| KR102088167B1 (en)* | 2019-08-27 | 2020-03-12 | 루나 주식회사 | Apparatus for updating software as an agent and patch method thereof |
| CN112416386A (en)* | 2020-10-26 | 2021-02-26 | 北京一亩田新农网络科技有限公司 | Method, device, system and medium for software upgrading |
| CN116028095A (en)* | 2022-06-30 | 2023-04-28 | 荣耀终端有限公司 | Generation method, restoration method and electronic equipment of differential file |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025148943A1 (en)* | 2024-01-08 | 2025-07-17 | 浙江极氪智能科技有限公司 | Data processing method, server, and terminal device |
| Publication | Publication Date | Title |
|---|---|---|
| CN112286939B (en) | Method, device and equipment for generating hash of global state in block chain type account book | |
| CN111444192B (en) | Method, device and equipment for generating Hash of global state in block chain type account book | |
| CN118916058A (en) | Data processing method, server and terminal equipment | |
| CN113934437B (en) | Method, system and client cloud phone for installing applications on cloud phone | |
| CN113961510A (en) | A file processing method, device, device and storage medium | |
| CN112202834A (en) | Data processing method, data processing device and node server | |
| CN116850598A (en) | Game resource updating method, system, equipment and storage medium | |
| CN114610366A (en) | A differential package upgrade method, system, intelligent electronic device and storage medium | |
| CN115114232A (en) | Method, device and medium for enumerating historical version objects | |
| WO2023060893A1 (en) | Storage space management method and apparatus, and device and storage medium | |
| CN111488483B (en) | Method, device, terminal and non-transitory computer readable storage medium for updating a library | |
| CN114356386A (en) | Block differential upgrading method, terminal equipment and computer readable storage medium | |
| CN115695521B (en) | Distributed session method and device | |
| CN111796845B (en) | Database upgrading method, state data query method, device and storage medium | |
| CN115344550A (en) | Method, device and medium for cloning directories of distributed file system | |
| CN109597683B (en) | Memory management method, device, equipment and storage medium | |
| CN111625254A (en) | File processing method, device, terminal and storage medium | |
| WO2025148943A1 (en) | Data processing method, server, and terminal device | |
| CN114969429B (en) | High-frame-rate video recording method and device, computer equipment and storage medium | |
| US12190176B2 (en) | Updating a state of a client device based on an event that includes a subset of records that identify binaries associated with the client device | |
| CN118152418B (en) | Index database dynamic updating method and system based on inotify | |
| CN114116732B (en) | Transaction processing method and device, storage device and server | |
| CN116775136B (en) | Method, device and medium for implementing compression storage and guide of embedded operating system | |
| CN117421141A (en) | Inter-process data parameter sharing method, device, equipment and medium | |
| CN118524152A (en) | Memory data processing method, memory data processing device, computer equipment, storage medium and memory data processing product |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |