技术领域technical field
本发明属于数据库接入、访问技术领域，尤其涉及一种任务处理系统和方法。The invention belongs to the technical field of database access and access, and in particular relates to a task processing system and method.
背景技术Background technique
历史数据的查询性能是实时数据库的重要性能指标。The query performance of historical data is an important performance index of real-time database.
目前，针对用户请求的数据查询任务，实时数据库不提供任务拆分功能或仅提供简单的基于任务量的平均拆分功能，且采用非共享机制管理其历史数据，基于此，对于用户请求的数据查询任务，实时数据库只能由固定的服务程序接入、并访问其历史数据实现对相应任务进行处理，或由多个服务程序以轮询方式接入、访问其历史数据实现对通过简单拆分所得的相应子任务进行处理。此种处理方式大大影响了任务请求的响应速率，会导致大数据集查询任务(例如千万级以上测点数量或长时间段的历史数据查询任务)的响应延迟较高，进而降低了实时数据库的查询性能。At present, for the data query tasks requested by users, the real-time database does not provide task splitting function or only provides a simple average splitting function based on task volume, and uses a non-sharing mechanism to manage its historical data. Based on this, for the data requested by users For query tasks, the real-time database can only be accessed by a fixed service program and access its historical data to process the corresponding task, or multiple service programs can access and access its historical data in a polling manner to achieve simple split The resulting corresponding subtasks are processed. This processing method greatly affects the response rate of task requests, which will lead to a high response delay for large data set query tasks (such as the number of measurement points above tens of millions or long-term historical data query tasks), which in turn reduces the real-time database. query performance.
发明内容Contents of the invention
有鉴于此，本发明的目的在于提供一种任务处理系统和方法，以克服上述问题，提高实时数据库的查询性能。In view of this, the object of the present invention is to provide a task processing system and method to overcome the above problems and improve the query performance of the real-time database.
为此，本发明公开如下技术方案：For this reason, the present invention discloses following technical scheme:
一种任务处理系统，包括请求接收模块、任务拆分模块、任务处理模块、结果整合模块和结果返回模块，其中：A task processing system, including a request receiving module, a task splitting module, a task processing module, a result integration module and a result return module, wherein:
所述请求接收模块，用于接收用户的请求信息，所述请求信息包含对数据源中的目标数据集进行查询的查询任务；The request receiving module is configured to receive user request information, and the request information includes a query task for querying a target data set in a data source;
所述任务拆分模块，用于基于预先设定的第一任务拆分策略对所述查询任务进行拆分，得到所述查询任务的N个子任务，其中，所述N为大于1的自然数；The task splitting module is configured to split the query task based on a preset first task splitting strategy to obtain N subtasks of the query task, where N is a natural number greater than 1;
所述任务处理模块，用于基于数据源共享机制，对所述N个子任务进行并行处理，得到相应的N个查询子结果；The task processing module is configured to process the N subtasks in parallel based on a data source sharing mechanism to obtain corresponding N query subresults;
所述结果整合模块，用于利用预先设定的汇总规则对所述N个查询子结果进行汇总、整合，得到用户所需的查询结果集；The result integration module is used to summarize and integrate the N query sub-results by using preset summary rules to obtain the query result set required by the user;
所述结果返回模块，用于将所述查询结果集返回至用户。The result returning module is used to return the query result set to the user.
上述系统，优选的，所述数据源具体为实时数据库。In the above system, preferably, the data source is a real-time database.
上述系统，优选的，所述任务处理模块具体包括任务分配单元和并行处理单元，其中：In the above system, preferably, the task processing module specifically includes a task allocation unit and a parallel processing unit, wherein:
所述任务分配单元，用于将所述N个子任务以一对一的映射关系分配至N个数据库服务器；The task allocation unit is configured to allocate the N subtasks to N database servers in a one-to-one mapping relationship;
所述并行处理单元，用于调度所述N个数据库服务器并行接入、访问所述数据源，得到与所述N个子任务相对应的N个查询子结果。The parallel processing unit is configured to schedule the N database servers to access and access the data source in parallel to obtain N query sub-results corresponding to the N sub-tasks.
上述系统，优选的，还包括：The above system, preferably, also includes:
子任务拆分单元，用于基于预先设定的第二任务拆分策略对每个所述子任务进行二次拆分，得到所述子任务的M个二次子任务，并触发所述并行处理单元执行如下操作：调度所述N个数据库服务器并行访问所述数据源，且使每个数据库服务器以多线程并发处理方式处理其所负责的M个二次子任务，其中，所述M为大于1的自然数。The subtask splitting unit is configured to split each of the subtasks twice based on a preset second task splitting strategy, obtain M secondary subtasks of the subtasks, and trigger the parallel The processing unit performs the following operations: scheduling the N database servers to access the data source in parallel, and making each database server process M secondary subtasks it is responsible for in a multi-threaded concurrent processing manner, wherein the M is A natural number greater than 1.
上述系统，优选的，还包括：The above system, preferably, also includes:
故障处理模块，用于在所述数据库服务器发生故障时，将发生故障的数据库服务器负责的子任务转交至未发生故障的数据库服务器进行处理。The failure processing module is configured to transfer the subtasks in charge of the failed database server to the non-failed database server for processing when the database server fails.
上述系统，优选的，所述第一任务拆分策略具体基于所述查询任务的任务量、查询任务对应目标数据的时间属性以及数据库服务器集群当前的并行处理能力制定；所述第二任务拆分策略具体基于相应子任务的任务量、子任务对应目标数据的时间属性以及相应数据库服务器当前的多线程并发处理能力制定。In the above system, preferably, the first task splitting strategy is specifically formulated based on the task amount of the query task, the time attribute of the target data corresponding to the query task, and the current parallel processing capability of the database server cluster; the second task splitting strategy The strategy is specifically formulated based on the task amount of the corresponding subtask, the time attribute of the target data corresponding to the subtask, and the current multi-thread concurrent processing capability of the corresponding database server.
一种任务处理方法，包括：A task processing method, comprising:
接收用户的请求信息，所述请求信息包含对数据源中的目标数据集进行查询的查询任务；Receive request information from the user, the request information includes a query task for querying the target data set in the data source;
基于预先设定的第一任务拆分策略对所述查询任务进行拆分，得到所述查询任务的N个子任务，其中，所述N为大于1的自然数；Splitting the query task based on a preset first task splitting strategy to obtain N subtasks of the query task, where N is a natural number greater than 1;
基于数据源共享机制，对所述N个子任务进行并行处理，得到相应的N个查询子结果；Based on the data source sharing mechanism, the N subtasks are processed in parallel to obtain corresponding N query subresults;
利用预先设定的汇总规则对所述N个查询子结果进行汇总、整合，得到用户所需的查询结果集；Summarize and integrate the N query sub-results by using preset summary rules to obtain the query result set required by the user;
将所述查询结果集返回至用户。The query result set is returned to the user.
上述方法，优选的，所述基于所述数据源对所述N个子任务进行并行处理，得到相应的N个查询子结果，具体包括：In the above method, preferably, the N subtasks are processed in parallel based on the data source to obtain corresponding N query subresults, specifically including:
将所述N个子任务以一对一的映射关系分配至N个数据库服务器；Distributing the N subtasks to N database servers in a one-to-one mapping relationship;
调度所述N个数据库服务器并行接入、访问所述数据源，得到与所述N个子任务相对应的N个查询子结果。Scheduling the N database servers to access and access the data source in parallel to obtain N query sub-results corresponding to the N sub-tasks.
上述方法，优选的，基于所述数据源对所述N个子任务进行并行处理，得到相应的N个查询子结果，还包括：In the above method, preferably, the N sub-tasks are processed in parallel based on the data source to obtain corresponding N query sub-results, further comprising:
基于预先设定的第二任务拆分策略对每个所述子任务进行二次拆分，得到所述子任务的M个二次子任务，其中，所述M为大于1的自然数；Perform secondary splitting on each of the subtasks based on a preset second task splitting strategy to obtain M secondary subtasks of the subtasks, wherein the M is a natural number greater than 1;
调度所述N个数据库服务器并行访问所述数据源，并使每个数据库服务器以多线程处理方式处理其所负责的M个二次子任务。Scheduling the N database servers to access the data source in parallel, and making each database server process the M secondary subtasks it is responsible for in a multi-threaded manner.
上述方法，优选的，还包括：The above method, preferably, also includes:
当所述数据库服务器发生故障时，将发生故障的所述数据库服务器负责的子任务转交至未发生故障的数据库服务器进行处理。When the database server fails, the subtasks responsible for the failed database server are transferred to the non-failed database server for processing.
本发明的任务处理系统包括请求接收模块、任务拆分模块、任务处理模块、结果整合模块和结果返回模块。针对现有技术中实时数据库采用非共享机制管理其历史数据的这一特点，本发明系统采用共享管理机制，通过其包括的各功能模块将用户请求的查询任务基于预先设定的拆分策略分为多个子任务，并基于数据源共享机制进行多个子任务的并行处理，具体实施时，可布设多个数据库服务器作为对等节点接入实时数据库，以共享其数据访问权，并通过对实时数据库进行并行访问实现对多个子任务的并行处理。从而，针对用户提交的大数据集任务请求，本发明可通过以上的任务拆分、分配以及子任务的并行处理过程大大提高任务请求的响应速率，因此，相较于现有基于非共享管理机制的任务处理方式，本发明大幅提升了实时数据库的查询性能。The task processing system of the present invention includes a request receiving module, a task splitting module, a task processing module, a result integrating module and a result returning module. In view of the fact that the real-time database in the prior art uses a non-shared mechanism to manage its historical data, the system of the present invention adopts a shared management mechanism, and divides the query tasks requested by users based on a preset splitting strategy through various functional modules included in it. For multiple subtasks, and based on the data source sharing mechanism, multiple subtasks are processed in parallel. During specific implementation, multiple database servers can be deployed as peer nodes to access the real-time database to share their data access rights, and through the real-time database Perform parallel access to implement parallel processing of multiple subtasks. Therefore, for the large data set task request submitted by the user, the present invention can greatly improve the response rate of the task request through the above task splitting, allocation and parallel processing of subtasks. Therefore, compared with the existing non-shared management mechanism The task processing mode of the invention greatly improves the query performance of the real-time database.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without creative work.
图1是本发明实施例一公开的任务处理系统的一种结构示意图；FIG. 1 is a schematic structural diagram of a task processing system disclosed in Embodiment 1 of the present invention;
图2是本发明实施例一公开的任务处理模块的一种结构示意图；FIG. 2 is a schematic structural diagram of a task processing module disclosed in Embodiment 1 of the present invention;
图3是本发明实施例二公开的任务处理模块的另一种结构示意图；Fig. 3 is another schematic structural diagram of the task processing module disclosed in Embodiment 2 of the present invention;
图4是本发明实施例三公开的任务处理系统的另一种结构示意图；Fig. 4 is another schematic structural diagram of the task processing system disclosed in Embodiment 3 of the present invention;
图5是本发明实施例四公开的任务处理方法的一种流程图；FIG. 5 is a flow chart of the task processing method disclosed in Embodiment 4 of the present invention;
图6是本发明实施例四公开的任务处理方法的另一种流程图；FIG. 6 is another flow chart of the task processing method disclosed in Embodiment 4 of the present invention;
图7是本发明实施例四公开的应用实例中分布式实时数据库系统的组成结构示意图。Fig. 7 is a schematic diagram of the composition and structure of the distributed real-time database system in the application example disclosed in the fourth embodiment of the present invention.
具体实施方式Detailed ways
为了引用和清楚起见，下文中使用的技术名词、简写或缩写总结解释如下：For the sake of reference and clarity, the technical terms, abbreviations or abbreviations used in the following text are summarized as follows:
实时数据库：指目前电力信息化行业所指的实时数据库。Real-time database: refers to the real-time database referred to in the current electric power information industry.
测点：指实时数据库中的数据组织单位，也称标签点、Tag。Measuring point: refers to the data organization unit in the real-time database, also known as tag point or Tag.
任务：指进行的某个操作请求。Task: Refers to an operation request to be performed.
单点：指单个测点。Single point: refers to a single measuring point.
断面：指部分(或所有)测点的同一时刻的数据。Section: Refers to the data of some (or all) measuring points at the same time.
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
实施例一Embodiment one
本发明实施例一公开一种任务处理系统，请参见图1，该系统包括请求接收模块100、任务拆分模块200、任务处理模块300、结果整合模块400和结果返回模块500。Embodiment 1 of the present invention discloses a task processing system. Please refer to FIG. 1 .
请求接收模块100，用于接收用户的请求信息，所述请求信息包含对数据源中的目标数据集进行查询的查询任务。The request receiving module 100 is configured to receive request information from a user, and the request information includes a query task for querying a target data set in a data source.
本实施例中，所述数据源具体为实时数据库，以下将通过对用户提交的查询实时数据库中历史数据的任务请求进行处理来对本发明进行详细说明。In this embodiment, the data source is specifically a real-time database. The present invention will be described in detail below by processing a task request submitted by a user for querying historical data in the real-time database.
其中，用户提交的任务请求具体可以是请求对实时数据库进行单点历史查询，也可以是请求对实时数据库进行断面历史查询。Wherein, the task request submitted by the user may specifically be a request for a single-point historical query of the real-time database, or a request for a cross-sectional historical query of the real-time database.
任务拆分模块200，用于基于预先设定的第一任务拆分策略对所述查询任务进行拆分，得到所述查询任务的N个子任务，其中，所述N为大于1的自然数。The task splitting module 200 is configured to split the query task based on a preset first task splitting strategy to obtain N subtasks of the query task, where N is a natural number greater than 1.
任务处理模块300，用于基于数据源共享机制，对所述N个子任务进行并行处理，得到相应的N个查询子结果。The task processing module 300 is configured to process the N subtasks in parallel based on the data source sharing mechanism to obtain corresponding N query subresults.
其中，如图2所示，任务处理模块300具体包括任务分配单元310和并行处理单元320。任务分配单元310，用于将所述N个子任务以一对一的映射关系分配至N个数据库服务器；并行处理单元320，用于调度所述N个数据库服务器并行接入、访问所述数据源，得到与所述N个子任务相对应的N个查询子结果。Wherein, as shown in FIG. 2 , the task processing module 300 specifically includes a task allocation unit 310 and a parallel processing unit 320 . A task assignment unit 310, configured to assign the N subtasks to N database servers in a one-to-one mapping relationship; a parallel processing unit 320, configured to schedule the N database servers to access and access the data source in parallel , to obtain N query sub-results corresponding to the N sub-tasks.
本实施例基于数据共享管理形式，将分布式数据库服务器集群中的各数据库服务器作为对等节点同时接入实时数据库，各节点共享对实时数据库中历史数据的访问权，并采用分布式并发处理机制实现对各子任务的并发处理。In this embodiment, based on the data sharing management form, each database server in the distributed database server cluster is used as a peer node to access the real-time database at the same time, and each node shares the access right to the historical data in the real-time database, and adopts a distributed concurrent processing mechanism Realize the concurrent processing of each subtask.
区别于现有技术中对查询任务仅采用基于任务量的平均拆分策略，本实施例中，获取各子任务所使用的拆分策略具体基于用户提交的查询任务的任务量、查询任务对应目标数据的时间属性以及数据库服务器集群当前的并行处理能力制定。Different from the prior art that only uses an average splitting strategy based on the amount of tasks for query tasks, in this embodiment, the splitting strategy used to obtain each subtask is specifically based on the amount of tasks submitted by the user and the corresponding target of the query task. The temporal properties of the data and the current parallel processing capabilities of the database server cluster are specified.
例如，若当前数据库服务器集群中共有N个数据库服务器空闲，则拆分策略基于任务量、目标数据的时间属性以及各数据库服务器的繁忙状态、并行处理能力将用户请求的查询任务拆分为N个子任务，最终使每个数据库服务器负责执行一个子任务，实现各子任务的优化分配。For example, if there are a total of N database servers idle in the current database server cluster, the splitting strategy splits the query task requested by the user into N subgroups based on the task amount, the time attribute of the target data, the busy status of each database server, and the parallel processing capability. Tasks, and finally each database server is responsible for executing a subtask, so as to realize the optimal allocation of each subtask.
现实应用场景中，拆分策略的制定不限于以上三个维度，具体可由本领域技术人员根据实际需求进行相关拆分算法的设定，例如还可将目标数据集中数据间的逻辑相关性作为拆分策略制定的参考依据，将逻辑关联强度较大的数据请求任务划分在同一子任务中，以实现后续对子任务更为有效、快速地处理。In real application scenarios, the formulation of the splitting strategy is not limited to the above three dimensions. Specifically, relevant splitting algorithms can be set by those skilled in the art according to actual needs. For example, the logical correlation between data in the target data set can also be used as a split Based on the reference basis for sub-strategy formulation, data request tasks with strong logical correlations are divided into the same subtask, so as to achieve more effective and rapid processing of subsequent subtasks.
结果整合模块400，用于利用预先设定的汇总规则对所述N个查询子结果进行汇总、整合，得到用户所需的查询结果集。The result integration module 400 is configured to aggregate and integrate the N query sub-results by using a preset aggregation rule to obtain the query result set required by the user.
结果返回模块500，用于将所述查询结果集返回至用户。The result returning module 500 is configured to return the query result set to the user.
任务执行完毕，得到用户所需的查询结果集后，结果返回模块500唤醒用户程序，并将查询结果集返回至用户程序供用户使用，至此，本发明系统的任务处理过程结束。After the task is executed and the query result set required by the user is obtained, the result return module 500 wakes up the user program and returns the query result set to the user program for the user to use. So far, the task processing process of the system of the present invention ends.
综上，本发明的任务处理系统包括请求接收模块100、任务拆分模块200、任务处理模块300、结果整合模块400和结果返回模块500。针对现有技术中实时数据库采用非共享机制管理其历史数据的这一特点，本发明系统采用共享管理机制，通过其包括的各功能模块将用户请求的查询任务基于预先设定的拆分策略分为多个子任务，并基于数据源共享机制进行多个子任务的并行处理，具体实施时，可布设多个数据库服务器作为对等节点接入实时数据库，以共享其数据访问权，并通过对实时数据库进行并行访问实现对多个子任务的并行处理。从而，针对用户提交的大数据集任务请求，本发明可通过以上的任务拆分、分配以及子任务的并行处理过程大大提高任务请求的响应速率，因此，相较于现有基于非共享管理机制的任务处理方式，本发明大幅提升了实时数据库的查询性能。To sum up, the task processing system of the present invention includes a request receiving module 100 , a task splitting module 200 , a task processing module 300 , a result integrating module 400 and a result returning module 500 . In view of the fact that the real-time database in the prior art uses a non-shared mechanism to manage its historical data, the system of the present invention adopts a shared management mechanism, and divides the query tasks requested by users based on a preset splitting strategy through various functional modules included in it. For multiple subtasks, and based on the data source sharing mechanism, multiple subtasks are processed in parallel. During specific implementation, multiple database servers can be deployed as peer nodes to access the real-time database to share their data access rights, and through the real-time database Perform parallel access to implement parallel processing of multiple subtasks. Therefore, for the large data set task request submitted by the user, the present invention can greatly improve the response rate of the task request through the above task splitting, allocation and parallel processing of subtasks. Therefore, compared with the existing non-shared management mechanism The task processing mode of the invention greatly improves the query performance of the real-time database.
实施例二Embodiment two
本发明实施例二继续对实施例一中公开的任务处理系统进行优化，请参见图3，本实施例中，任务处理模块300还包括子任务拆分单元330，该单元具体在任务分配单元310和并行处理单元320之间，与所述两个单元逻辑衔接。Embodiment 2 of the present invention continues to optimize the task processing system disclosed in Embodiment 1. Please refer to FIG. 3 . Between the parallel processing unit 320 and the two units are logically connected.
子任务拆分单元330，用于基于预先设定的第二任务拆分策略对每个所述子任务进行二次拆分，得到所述子任务的M个二次子任务，其中，所述M为大于1的自然数。The subtask splitting unit 330 is configured to split each of the subtasks twice based on a preset second task splitting strategy to obtain M secondary subtasks of the subtasks, wherein the M is a natural number greater than 1.
其中，所述第二任务拆分策略具体基于相应子任务的任务量、子任务对应目标数据的时间属性以及相应数据库服务器当前的多线程并发处理能力制定。同样地，该拆分策略的制订不局限于以上三个维度，现实应用场景中还可依据实际需求将其他参考因素作为拆分依据。Wherein, the second task splitting strategy is specifically formulated based on the task amount of the corresponding subtask, the time attribute of the target data corresponding to the subtask, and the current multi-thread concurrent processing capability of the corresponding database server. Similarly, the formulation of the splitting strategy is not limited to the above three dimensions, and other reference factors can also be used as splitting basis according to actual needs in actual application scenarios.
除此之外，并行处理单元调度320也进行了相应功能完善，即该单元除了调度所述N个数据库服务器并行访问实时数据库，还同时保证每个数据库服务器以多线程并发处理方式处理其所负责的M个二次子任务。In addition, the parallel processing unit scheduling 320 has also been improved correspondingly, that is, this unit not only schedules the N database servers to access the real-time database in parallel, but also ensures that each database server handles the tasks it is responsible for in a multi-threaded concurrent processing manner. M secondary subtasks of .
数据库服务器将其负责的M个二次子任务分配给M个线程去同时处理，并将M个线程返回的M个子结果首先进行本地汇总，得到查询任务的中间结果。N个服务器共产生N份中间结果，进而对N个中间结果进行汇总即可得到用户所需的结果数据集。The database server assigns the M secondary subtasks it is responsible for to M threads to process simultaneously, and first summarizes the M subresults returned by the M threads locally to obtain the intermediate results of the query task. N servers generate N intermediate results in total, and then aggregate the N intermediate results to obtain the result data set required by the user.
本实施例提供了本地任务的二次拆分功能，并对二次拆分所得的各二次子任务采用多线程并发处理机制进行处理，提高了各个子任务的处理效率，从而，进一步提升了用户任务请求的响应速率。This embodiment provides the secondary splitting function of the local task, and adopts the multi-thread concurrent processing mechanism to process each secondary subtask obtained from the secondary splitting, which improves the processing efficiency of each subtask, thereby further improving the Response rate for user task requests.
实施例三Embodiment three
本实施例继续对以上两个实施例中公开的任务处理系统进行补充、完善。This embodiment continues to supplement and improve the task processing systems disclosed in the above two embodiments.
实施例一及实施例二的任务处理过程具体需基于各数据库服务器正常工作这一前提，而在实际的集群系统中，任何一个或多个服务器节点因出现故障导致其临时下线属于常见现象，针对此种情况，请参见图4，本实施例在原有各功能模块的基础上，为任务处理系统添加故障处理模块600。The task processing process of Embodiment 1 and Embodiment 2 needs to be based on the premise that each database server works normally. In an actual cluster system, it is common for any one or more server nodes to go offline temporarily due to failure. For this situation, please refer to FIG. 4 , this embodiment adds a fault processing module 600 to the task processing system on the basis of the original functional modules.
故障处理模块600，用于在数据库服务器发生故障时，将发生故障的数据库服务器负责的子任务转交至未发生故障的数据库服务器进行处理。The failure processing module 600 is configured to transfer the subtasks in charge of the failed database server to the non-faulted database server for processing when the database server fails.
具体地，故障处理模块600以周期轮询的方式监控各数据库服务器。当某一数据库服务器接收到该模块发送的轮询命令，且在设定的时间阈值内一直没有响应时，则该模块判定所述数据库服务器失效，为了不对查询任务的处理造成影响，本模块将失效服务器节点负责的子任务置为未处理状态，并重新为其分配数据库服务器进行处理。Specifically, the failure processing module 600 monitors each database server in a periodic polling manner. When a database server receives the polling command sent by the module and has not responded within the set time threshold, the module determines that the database server is invalid. In order not to affect the processing of query tasks, this module will The subtasks responsible for the failed server node are set as unprocessed, and the database server is reassigned for processing.
由于失效服务器上的相应子任务处理结果无法访问，因此，在失效机器上即使相应子任务已经执行完成，同样需要重新执行该子任务。Since the processing result of the corresponding subtask on the failed server cannot be accessed, even if the corresponding subtask has been executed on the failed machine, the subtask needs to be re-executed.
本实施例为任务处理系统提供了故障处理机制，从而在服务器节点出现故障时，仍能保证系统对用户的任务请求进行正常地、有效地处理，提高了系统任务处理的健壮性。This embodiment provides a fault handling mechanism for the task processing system, so that when a server node fails, the system can still ensure that the system can normally and effectively process the user's task request, improving the robustness of the system task processing.
实施例四Embodiment Four
本实施例四公开一种任务处理方法，该方法与以上三个实施例公开的任务处理系统相对应。Embodiment 4 discloses a task processing method, which corresponds to the task processing systems disclosed in the above three embodiments.
首先，相应于实施例一中系统的结构，本实施例公开任务处理方法的一种流程，请参见图5，该方法包括如下步骤：First of all, corresponding to the structure of the system in the first embodiment, this embodiment discloses a flow of the task processing method, please refer to Figure 5, the method includes the following steps:
S501：接收用户的请求信息，所述请求信息包含对数据源中的目标数据集进行查询的查询任务。S501: Receive request information from a user, where the request information includes a query task for querying a target dataset in a data source.
S502：基于预先设定的第一任务拆分策略对所述查询任务进行拆分，得到所述查询任务的N个子任务，其中，所述N为大于1的自然数。S502: Split the query task based on a preset first task splitting strategy to obtain N subtasks of the query task, where N is a natural number greater than 1.
S503：基于数据源共享机制，对所述N个子任务进行并行处理，得到相应的N个查询子结果。S503: Based on the data source sharing mechanism, perform parallel processing on the N subtasks to obtain corresponding N query subresults.
其中，步骤S503具体包括：Wherein, step S503 specifically includes:
将所述N个子任务以一对一的映射关系分配至N个数据库服务器；Distributing the N subtasks to N database servers in a one-to-one mapping relationship;
调度所述N个数据库服务器并行接入、访问所述数据源，得到与所述N个子任务相对应的N个查询子结果。Scheduling the N database servers to access and access the data source in parallel to obtain N query sub-results corresponding to the N sub-tasks.
S504：利用预先设定的汇总规则对所述N个查询子结果进行汇总、整合，得到用户所需的查询结果集。S504: Summarize and integrate the N query sub-results by using a preset summarization rule to obtain a query result set required by the user.
S505：将所述查询结果集返回至用户。S505: Return the query result set to the user.
相应于实施例二中任务处理系统的结构，本实施例继续公开任务处理方法的另一种流程，本流程中步骤S503还包括：Corresponding to the structure of the task processing system in Embodiment 2, this embodiment continues to disclose another flow of the task processing method. Step S503 in this flow also includes:
基于预先设定的第二任务拆分策略对每个所述子任务进行二次拆分，得到所述子任务的M个二次子任务，其中，所述M为大于1的自然数；Perform secondary splitting on each of the subtasks based on a preset second task splitting strategy to obtain M secondary subtasks of the subtasks, wherein the M is a natural number greater than 1;
调度所述N个数据库服务器并行访问所述数据源，并使每个数据库服务器以多线程处理方式处理其所负责的M个二次子任务。Scheduling the N database servers to access the data source in parallel, and making each database server process the M secondary subtasks it is responsible for in a multi-threaded manner.
对应于实施例三中任务处理系统的结构，如图6所示，任务处理方法还包括如下步骤：Corresponding to the structure of the task processing system in Embodiment 3, as shown in Figure 6, the task processing method further includes the following steps:
S506：当所述数据库服务器发生故障时，将发生故障的所述数据库服务器负责的子任务转交至未发生故障的数据库服务器进行处理。S506: When the database server fails, transfer the subtasks in charge of the failed database server to the non-faulted database server for processing.
对于本发明实施例四公开的任务处理方法而言，由于其与以上各实施例公开的任务处理系统相对应，所以描述的比较简单，相关相似之处请参见以上各实施例中任务处理系统部分的说明即可，此处不再详述。For the task processing method disclosed in the fourth embodiment of the present invention, since it corresponds to the task processing system disclosed in the above embodiments, the description is relatively simple. For related similarities, please refer to the task processing system in the above embodiments The description is sufficient and will not be described in detail here.
接下来，公开本发明的一具体应用实例。Next, a specific application example of the present invention is disclosed.
本实例提供一分布式实时数据库系统，如图7所示，该系统是由实时数据库、任务管理/拆分服务器、调度服务器以及P(P为自然数，且P≥N)个数据库服务器组成的集群系统，在用户程序向该系统请求查询任务时，该系统对请求任务进行处理的过程如下：This example provides a distributed real-time database system, as shown in Figure 7, the system is a cluster composed of real-time databases, task management/splitting servers, scheduling servers, and P (P is a natural number, and P≥N) database servers system, when a user program requests a query task from the system, the system processes the requested task as follows:
1)当用户程序向该系统提交任务请求时，任务管理/拆分服务器调用任务拆分策略将用户提交的大任务拆分成N个子任务，然后将包含相应信息的子任务请求发送给调度服务器。1) When the user program submits a task request to the system, the task management/splitting server invokes the task splitting strategy to split the large task submitted by the user into N subtasks, and then sends the subtask request containing the corresponding information to the scheduling server .
2)调度服务器把N个子任务分配给当前系统中空闲的N个数据库服务器，每个数据库服务器负责执行一个子任务。2) The scheduling server assigns N subtasks to N idle database servers in the current system, and each database server is responsible for executing a subtask.
3)数据库服务器收到子任务后调用二次拆分策略，对子任务进行二次拆分，将所得的M个二次子任务分配给M个线程去同时处理。并对M个线程返回的M个子结果进行本地汇总，得到中间结果值。3) After receiving the subtasks, the database server invokes the secondary splitting strategy, performs secondary splitting on the subtasks, and distributes the obtained M secondary subtasks to M threads for simultaneous processing. And locally summarize the M sub-results returned by the M threads to obtain intermediate result values.
4)所有的子任务都执行完毕后，调度服务器把N份中间结果值按一定的规则汇总为一个查询结果集。4) After all the subtasks are executed, the scheduling server summarizes the N intermediate result values into a query result set according to certain rules.
5)调度服务器唤醒用户程序，将查询结果集返回给用户程序，本次任务执行完毕。5) The scheduling server wakes up the user program, returns the query result set to the user program, and the execution of this task is completed.
对于处理过程中，某一个或多个服务器出现故障的情况，该系统依据本发明的故障处理机制，针对故障服务器负责的任务，由调度服务器调度其他正常的服务器节点对其进行处理。For the situation that one or more servers fail during the processing, the system, according to the fault handling mechanism of the present invention, dispatches other normal server nodes to process the tasks responsible for the faulty server by the dispatching server.
综上所述，本发明提供了基于多维度策略的任务拆分功能以及本地任务二次拆分功能，采用数据共享管理形式，通过分布式集群系统实现了各子任务的并行处理，大幅提升了实时数据库单点历史查询性能及断面历史查询性能，任务处理延迟小、速率快，且硬件配置要求低。In summary, the present invention provides a task splitting function based on a multi-dimensional strategy and a local task secondary splitting function, adopts a data sharing management form, realizes parallel processing of each subtask through a distributed cluster system, and greatly improves Real-time database single-point historical query performance and section historical query performance, task processing delay is small, the speed is fast, and the hardware configuration requirements are low.
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts in each embodiment, refer to each other, that is, Can.
为了描述的方便，描述以上装置时以功能分为各种模块或单元分别描述。当然，在实施本申请时可以把各模块、单元的功能在同一个或多个软件和/或硬件中实现。For the convenience of description, when describing the above devices, the functions are divided into various modules or units and described separately. Of course, when implementing the present application, the functions of each module and unit can be implemented in one or more software and/or hardware.
通过以上的实施方式的描述可知，本领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例或者实施例的某些部分所述的方法。It can be known from the above description of the implementation manners that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general-purpose hardware platform. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in storage media, such as ROM/RAM, disk , CD, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present application.
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410177684.2ACN103942098A (en) | 2014-04-29 | 2014-04-29 | System and method for task processing |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410177684.2ACN103942098A (en) | 2014-04-29 | 2014-04-29 | System and method for task processing |
| Publication Number | Publication Date |
|---|---|
| CN103942098Atrue CN103942098A (en) | 2014-07-23 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410177684.2APendingCN103942098A (en) | 2014-04-29 | 2014-04-29 | System and method for task processing |
| Country | Link |
|---|---|
| CN (1) | CN103942098A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104331255A (en)* | 2014-11-17 | 2015-02-04 | 中国科学院声学研究所 | Embedded file system-based reading method for streaming data |
| CN104699542A (en)* | 2015-03-31 | 2015-06-10 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104731647A (en)* | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104731951A (en)* | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Data query method and device |
| CN105117283A (en)* | 2015-08-26 | 2015-12-02 | 深圳市华验防伪科技有限公司 | Task splitting method and system |
| CN105335143A (en)* | 2014-07-30 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Business processing method and apparatus |
| CN105488134A (en)* | 2015-11-25 | 2016-04-13 | 用友网络科技股份有限公司 | Big data processing method and big data processing device |
| CN105843886A (en)* | 2016-03-21 | 2016-08-10 | 国电南瑞科技股份有限公司 | Multi-thread based power grid offline model data query method |
| CN106022908A (en)* | 2016-05-17 | 2016-10-12 | 中国建设银行股份有限公司 | Method and system for querying information of assets and liabilities |
| CN106330987A (en)* | 2015-06-15 | 2017-01-11 | 交通银行股份有限公司 | Dynamic load balancing method |
| CN106339265A (en)* | 2016-08-30 | 2017-01-18 | 中国银行股份有限公司 | Method and device for processing combined task |
| CN106407190A (en)* | 2015-07-27 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Event record querying method and device |
| CN106570038A (en)* | 2015-10-12 | 2017-04-19 | 中国联合网络通信集团有限公司 | Distributed data processing method and system |
| CN106873957A (en)* | 2016-06-23 | 2017-06-20 | 阿里巴巴集团控股有限公司 | The processing method and equipment of a kind of operation flow |
| CN106874080A (en)* | 2016-07-07 | 2017-06-20 | 阿里巴巴集团控股有限公司 | Method for computing data and system based on distributed server cluster |
| CN106934027A (en)* | 2017-03-14 | 2017-07-07 | 深圳市博信诺达经贸咨询有限公司 | Distributed reptile realization method and system |
| CN107203645A (en)* | 2017-06-27 | 2017-09-26 | 浪潮软件集团有限公司 | A method and Eclipse platform for querying multiple databases concurrently |
| CN107291720A (en)* | 2016-03-30 | 2017-10-24 | 阿里巴巴集团控股有限公司 | A kind of method, system and computer cluster for realizing batch data processing |
| CN107707328A (en)* | 2016-08-08 | 2018-02-16 | 北京京东尚科信息技术有限公司 | Summary info transmission method and device |
| CN108052646A (en)* | 2017-12-25 | 2018-05-18 | 北京车联天下信息技术有限公司 | Big data system and method are calculated in real time |
| CN108172299A (en)* | 2017-12-25 | 2018-06-15 | 华中科技大学同济医学院附属协和医院 | A remote computing system and method for medical data |
| CN108229908A (en)* | 2017-12-08 | 2018-06-29 | 泰康保险集团股份有限公司 | Reward appraisal method and apparatus |
| CN108241529A (en)* | 2017-10-13 | 2018-07-03 | 平安科技（深圳）有限公司 | Wages computational methods, application server and computer readable storage medium |
| CN108389121A (en)* | 2018-02-07 | 2018-08-10 | 平安普惠企业管理有限公司 | Loan data processing method, device, computer equipment and storage medium |
| WO2018188498A1 (en)* | 2017-04-12 | 2018-10-18 | 梅特勒-托利多(常州)精密仪器有限公司 | Collaborative weighing and measuring system and metering system |
| CN108846763A (en)* | 2018-06-05 | 2018-11-20 | 中国平安人寿保险股份有限公司 | Core protects request processing method, device, computer equipment and storage medium |
| CN109271243A (en)* | 2018-08-31 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of cluster task management system |
| CN109558237A (en)* | 2017-09-27 | 2019-04-02 | 北京国双科技有限公司 | A kind of task status management method and device |
| CN109901919A (en)* | 2017-12-08 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Information output method and device |
| CN110119269A (en)* | 2019-04-19 | 2019-08-13 | 北京大米科技有限公司 | Method, apparatus, server and the storage medium of control task object |
| CN110209496A (en)* | 2019-05-20 | 2019-09-06 | 中国平安财产保险股份有限公司 | Task sharding method, device and sliced service device based on data processing |
| CN110765157A (en)* | 2019-09-06 | 2020-02-07 | 中国平安财产保险股份有限公司 | Data query method and device, computer equipment and storage medium |
| CN110780977A (en)* | 2019-10-25 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Cloud computing-based task delivery method, device, system and readable storage medium |
| CN111274052A (en)* | 2020-01-19 | 2020-06-12 | 中国平安人寿保险股份有限公司 | Data distribution method, server, and computer-readable storage medium |
| CN112597338A (en)* | 2020-10-09 | 2021-04-02 | 腾讯科技（深圳）有限公司 | Video understanding method and related device |
| CN113438304A (en)* | 2021-06-23 | 2021-09-24 | 平安消费金融有限公司 | Data query method, device, server and medium based on database cluster |
| CN114756732A (en)* | 2022-04-11 | 2022-07-15 | 平安国际智慧城市科技股份有限公司 | Data query method, device, equipment and storage medium |
| CN114819490A (en)* | 2022-03-16 | 2022-07-29 | 深圳市工易付电子科技有限公司 | Task release method, device, device and storage medium |
| CN115941660A (en)* | 2022-11-23 | 2023-04-07 | 内蒙古欣荣惠信息技术有限公司 | Minor Protection System and Method |
| CN116069488A (en)* | 2021-11-01 | 2023-05-05 | 第四范式（北京）技术有限公司 | Parallel computing method and device for distributed data |
| CN117112650A (en)* | 2023-09-08 | 2023-11-24 | 浙江省自然资源征收中心 | Process query method and system based on digital migration |
| CN119088764A (en)* | 2024-08-19 | 2024-12-06 | 成都融见软件科技有限公司 | Signal searching system |
| CN116069488B (en)* | 2021-11-01 | 2025-10-17 | 第四范式（北京）技术有限公司 | Parallel computing method and device for distributed data |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6112225A (en)* | 1998-03-30 | 2000-08-29 | International Business Machines Corporation | Task distribution processing system and the method for subscribing computers to perform computing tasks during idle time |
| US20090070773A1 (en)* | 2007-09-10 | 2009-03-12 | Novell, Inc. | Method for efficient thread usage for hierarchically structured tasks |
| CN101441557A (en)* | 2008-11-08 | 2009-05-27 | 腾讯科技（深圳）有限公司 | Distributed parallel calculating system and method based on dynamic data division |
| CN202058147U (en)* | 2011-05-23 | 2011-11-30 | 北京六所和瑞科技发展有限公司 | Distribution type real-time database management system |
| CN102630316A (en)* | 2011-12-22 | 2012-08-08 | 华为技术有限公司 | Processing method and apparatus of concurrent tasks |
| CN103235835A (en)* | 2013-05-22 | 2013-08-07 | 曙光信息产业(北京)有限公司 | Inquiry implementation method for database cluster and device |
| CN103246749A (en)* | 2013-05-24 | 2013-08-14 | 北京立新盈企信息技术有限公司 | Matrix data base system for distributed computing and query method thereof |
| CN103577938A (en)* | 2013-11-15 | 2014-02-12 | 国家电网公司 | Power grid dispatching automation main-and-standby system model synchronizing method and synchronizing system thereof |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6112225A (en)* | 1998-03-30 | 2000-08-29 | International Business Machines Corporation | Task distribution processing system and the method for subscribing computers to perform computing tasks during idle time |
| US20090070773A1 (en)* | 2007-09-10 | 2009-03-12 | Novell, Inc. | Method for efficient thread usage for hierarchically structured tasks |
| CN101441557A (en)* | 2008-11-08 | 2009-05-27 | 腾讯科技（深圳）有限公司 | Distributed parallel calculating system and method based on dynamic data division |
| CN202058147U (en)* | 2011-05-23 | 2011-11-30 | 北京六所和瑞科技发展有限公司 | Distribution type real-time database management system |
| CN102630316A (en)* | 2011-12-22 | 2012-08-08 | 华为技术有限公司 | Processing method and apparatus of concurrent tasks |
| CN103235835A (en)* | 2013-05-22 | 2013-08-07 | 曙光信息产业(北京)有限公司 | Inquiry implementation method for database cluster and device |
| CN103246749A (en)* | 2013-05-24 | 2013-08-14 | 北京立新盈企信息技术有限公司 | Matrix data base system for distributed computing and query method thereof |
| CN103577938A (en)* | 2013-11-15 | 2014-02-12 | 国家电网公司 | Power grid dispatching automation main-and-standby system model synchronizing method and synchronizing system thereof |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335143A (en)* | 2014-07-30 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Business processing method and apparatus |
| CN104331255A (en)* | 2014-11-17 | 2015-02-04 | 中国科学院声学研究所 | Embedded file system-based reading method for streaming data |
| CN104331255B (en)* | 2014-11-17 | 2018-04-17 | 中国科学院声学研究所 | A kind of stream data read method based on embedded file system |
| CN104699542B (en)* | 2015-03-31 | 2018-02-09 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104699542A (en)* | 2015-03-31 | 2015-06-10 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104731647A (en)* | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104731951A (en)* | 2015-03-31 | 2015-06-24 | 北京奇艺世纪科技有限公司 | Data query method and device |
| CN104731951B (en)* | 2015-03-31 | 2018-08-07 | 北京奇艺世纪科技有限公司 | A kind of data query method and device |
| CN104731647B (en)* | 2015-03-31 | 2018-02-09 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN106330987A (en)* | 2015-06-15 | 2017-01-11 | 交通银行股份有限公司 | Dynamic load balancing method |
| CN106407190A (en)* | 2015-07-27 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Event record querying method and device |
| CN106407190B (en)* | 2015-07-27 | 2020-01-14 | 阿里巴巴集团控股有限公司 | Event record query method and device |
| US11113276B2 (en) | 2015-07-27 | 2021-09-07 | Advanced New Technologies Co., Ltd. | Querying a database |
| CN105117283A (en)* | 2015-08-26 | 2015-12-02 | 深圳市华验防伪科技有限公司 | Task splitting method and system |
| CN106570038A (en)* | 2015-10-12 | 2017-04-19 | 中国联合网络通信集团有限公司 | Distributed data processing method and system |
| CN106570038B (en)* | 2015-10-12 | 2020-05-22 | 中国联合网络通信集团有限公司 | A distributed data processing method and system |
| CN105488134A (en)* | 2015-11-25 | 2016-04-13 | 用友网络科技股份有限公司 | Big data processing method and big data processing device |
| CN105843886A (en)* | 2016-03-21 | 2016-08-10 | 国电南瑞科技股份有限公司 | Multi-thread based power grid offline model data query method |
| CN107291720A (en)* | 2016-03-30 | 2017-10-24 | 阿里巴巴集团控股有限公司 | A kind of method, system and computer cluster for realizing batch data processing |
| CN107291720B (en)* | 2016-03-30 | 2020-10-02 | 阿里巴巴集团控股有限公司 | Method, system and computer cluster for realizing batch data processing |
| CN106022908A (en)* | 2016-05-17 | 2016-10-12 | 中国建设银行股份有限公司 | Method and system for querying information of assets and liabilities |
| CN106873957A (en)* | 2016-06-23 | 2017-06-20 | 阿里巴巴集团控股有限公司 | The processing method and equipment of a kind of operation flow |
| CN106874080B (en)* | 2016-07-07 | 2020-05-12 | 阿里巴巴集团控股有限公司 | Data calculation method and system based on distributed server cluster |
| CN106874080A (en)* | 2016-07-07 | 2017-06-20 | 阿里巴巴集团控股有限公司 | Method for computing data and system based on distributed server cluster |
| CN107707328A (en)* | 2016-08-08 | 2018-02-16 | 北京京东尚科信息技术有限公司 | Summary info transmission method and device |
| CN107707328B (en)* | 2016-08-08 | 2020-11-24 | 北京京东尚科信息技术有限公司 | Abstract information transmission method and device |
| CN106339265A (en)* | 2016-08-30 | 2017-01-18 | 中国银行股份有限公司 | Method and device for processing combined task |
| CN106934027A (en)* | 2017-03-14 | 2017-07-07 | 深圳市博信诺达经贸咨询有限公司 | Distributed reptile realization method and system |
| US11378441B2 (en) | 2017-04-12 | 2022-07-05 | Mettler-Toledo (Changzhou) Precision Instruments Co., Ltd. | Collaborative weighing and measuring system and metering system |
| WO2018188498A1 (en)* | 2017-04-12 | 2018-10-18 | 梅特勒-托利多(常州)精密仪器有限公司 | Collaborative weighing and measuring system and metering system |
| CN107203645A (en)* | 2017-06-27 | 2017-09-26 | 浪潮软件集团有限公司 | A method and Eclipse platform for querying multiple databases concurrently |
| CN109558237A (en)* | 2017-09-27 | 2019-04-02 | 北京国双科技有限公司 | A kind of task status management method and device |
| CN108241529A (en)* | 2017-10-13 | 2018-07-03 | 平安科技（深圳）有限公司 | Wages computational methods, application server and computer readable storage medium |
| CN108241529B (en)* | 2017-10-13 | 2021-11-09 | 平安科技（深圳）有限公司 | Salary calculation method, application server and computer readable storage medium |
| CN109901919A (en)* | 2017-12-08 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Information output method and device |
| CN109901919B (en)* | 2017-12-08 | 2021-09-03 | 北京京东尚科信息技术有限公司 | Information output method and device |
| CN108229908B (en)* | 2017-12-08 | 2021-10-08 | 泰康保险集团股份有限公司 | Salary assessment method and device |
| CN108229908A (en)* | 2017-12-08 | 2018-06-29 | 泰康保险集团股份有限公司 | Reward appraisal method and apparatus |
| CN108172299B (en)* | 2017-12-25 | 2021-04-27 | 华中科技大学同济医学院附属协和医院 | A system and method for remote computing of medical data |
| CN108052646A (en)* | 2017-12-25 | 2018-05-18 | 北京车联天下信息技术有限公司 | Big data system and method are calculated in real time |
| CN108172299A (en)* | 2017-12-25 | 2018-06-15 | 华中科技大学同济医学院附属协和医院 | A remote computing system and method for medical data |
| CN108389121B (en)* | 2018-02-07 | 2021-06-22 | 平安普惠企业管理有限公司 | Loan data processing method, loan data processing device, loan data processing program, and computer device and storage medium |
| CN108389121A (en)* | 2018-02-07 | 2018-08-10 | 平安普惠企业管理有限公司 | Loan data processing method, device, computer equipment and storage medium |
| CN108846763A (en)* | 2018-06-05 | 2018-11-20 | 中国平安人寿保险股份有限公司 | Core protects request processing method, device, computer equipment and storage medium |
| CN109271243B (en)* | 2018-08-31 | 2021-09-17 | 郑州云海信息技术有限公司 | Cluster task management system |
| CN109271243A (en)* | 2018-08-31 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of cluster task management system |
| CN110119269A (en)* | 2019-04-19 | 2019-08-13 | 北京大米科技有限公司 | Method, apparatus, server and the storage medium of control task object |
| CN110209496A (en)* | 2019-05-20 | 2019-09-06 | 中国平安财产保险股份有限公司 | Task sharding method, device and sliced service device based on data processing |
| CN110765157B (en)* | 2019-09-06 | 2024-02-02 | 中国平安财产保险股份有限公司 | Data query method, device, computer equipment and storage medium |
| CN110765157A (en)* | 2019-09-06 | 2020-02-07 | 中国平安财产保险股份有限公司 | Data query method and device, computer equipment and storage medium |
| CN110780977A (en)* | 2019-10-25 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Cloud computing-based task delivery method, device, system and readable storage medium |
| CN111274052A (en)* | 2020-01-19 | 2020-06-12 | 中国平安人寿保险股份有限公司 | Data distribution method, server, and computer-readable storage medium |
| CN112597338A (en)* | 2020-10-09 | 2021-04-02 | 腾讯科技（深圳）有限公司 | Video understanding method and related device |
| CN113438304A (en)* | 2021-06-23 | 2021-09-24 | 平安消费金融有限公司 | Data query method, device, server and medium based on database cluster |
| CN116069488A (en)* | 2021-11-01 | 2023-05-05 | 第四范式（北京）技术有限公司 | Parallel computing method and device for distributed data |
| CN116069488B (en)* | 2021-11-01 | 2025-10-17 | 第四范式（北京）技术有限公司 | Parallel computing method and device for distributed data |
| CN114819490A (en)* | 2022-03-16 | 2022-07-29 | 深圳市工易付电子科技有限公司 | Task release method, device, device and storage medium |
| CN114756732A (en)* | 2022-04-11 | 2022-07-15 | 平安国际智慧城市科技股份有限公司 | Data query method, device, equipment and storage medium |
| CN115941660A (en)* | 2022-11-23 | 2023-04-07 | 内蒙古欣荣惠信息技术有限公司 | Minor Protection System and Method |
| CN117112650A (en)* | 2023-09-08 | 2023-11-24 | 浙江省自然资源征收中心 | Process query method and system based on digital migration |
| CN117112650B (en)* | 2023-09-08 | 2025-09-12 | 浙江省自然资源征收中心 | Process query method and system based on digital relocation |
| CN119088764A (en)* | 2024-08-19 | 2024-12-06 | 成都融见软件科技有限公司 | Signal searching system |
| Publication | Publication Date | Title |
|---|---|---|
| CN103942098A (en) | System and method for task processing | |
| US9916183B2 (en) | Scheduling mapreduce jobs in a cluster of dynamically available servers | |
| US20200174838A1 (en) | Utilizing accelerators to accelerate data analytic workloads in disaggregated systems | |
| WO2021254135A1 (en) | Task execution method and storage device | |
| CN103336808B (en) | A kind of real-time diagram data processing system and method based on BSP models | |
| CN103135943B (en) | Self-adaptive IO (Input Output) scheduling method of multi-control storage system | |
| CN106909451A (en) | A kind of distributed task dispatching system and method | |
| US11256547B2 (en) | Efficient allocation of cloud computing resources to job requests | |
| US9092272B2 (en) | Preparing parallel tasks to use a synchronization register | |
| CN105471985A (en) | Load balance method, cloud platform computing method and cloud platform | |
| US8903981B2 (en) | Method and system for achieving better efficiency in a client grid using node resource usage and tracking | |
| CN105045607A (en) | Method for achieving uniform interface of multiple big data calculation frames | |
| WO2015130262A1 (en) | Multiple pools in a multi-core system | |
| WO2019223596A1 (en) | Method, device, and apparatus for event processing, and storage medium | |
| CN103761146B (en) | A kind of method that MapReduce dynamically sets slots quantity | |
| US10554782B2 (en) | Agile hostpool allocator | |
| CN109257399B (en) | Cloud platform application program management method, management platform and storage medium | |
| CN102521265B (en) | Dynamic consistency control method in massive data management | |
| CN103036946A (en) | Method and system for processing file backup on cloud platform | |
| CN103412786A (en) | High performance server architecture system and data processing method thereof | |
| CN108021435A (en) | A kind of cloud computing task stream scheduling method with fault-tolerant ability based on deadline | |
| CN105740085A (en) | Fault tolerance processing method and device | |
| WO2019024235A1 (en) | Electronic device, server allocation control method and computer readable storage medium | |
| CN104765644A (en) | Resource collaboration evolution system and method based on intelligent agent | |
| CN104468710A (en) | Mixed big data processing system and method |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20140723 |