Content of the invention
In order to overcome above-mentioned deficiency of the prior art, the technical problem to be solved is to provide a kind of task-setGroup's schedule management method and device, it can enter row order according to the execution dependence between task and task to task and holdOK, improve the efficiency of tasks carrying.
The present invention one better embodiment provides a kind of task cluster schedule management method, is applied to and tasks carrying terminalThe task management terminal of communication is it is characterised in that methods described includes:
Workflow schedule step, is scheduling to predefined workflow, and described predefined workflow includes manyThere is the task of execution dependence order, a group task of wherein no preposition dependence is to need execute at first to appoint between groupBusiness；
Task distributing step, a group task of described no preposition dependence is distributed at least one tasks carrying terminal and holdsOK；
Monitor step, the group task monitoring described no preposition dependence is in described at least one tasks carrying terminalPractice condition；
Task scheduling step, when the group task execution listening to described no preposition dependence completes, according to taskExecution relies on sequential scheduling residue task；
Repeat described task distributing step, monitor step and task scheduling step, all of in described workflowBusiness execution completes.
Another preferred embodiment of the present invention also provides a kind of task cluster schedule management method, is applied to communicate each otherThe tasks carrying terminal connecting and task management terminal, methods described includes:
Workflow schedule step, the predefined workflow of described task management terminal-pair is scheduling, described fixed in advanceThe workflow of justice include multigroup between there is the task of execution dependence order, a group task of wherein no preposition dependence is needExecuting at first of task；
Task distributing step, a group task of described no preposition dependence is distributed at least by described task management terminalOne tasks carrying terminal execution；
Task execution step, described at least one tasks carrying terminal executes a group task of described no preposition dependence；
Monitor step, a group task of no preposition dependence described in described task management terminal monitoring is described at least onePractice condition in tasks carrying terminal；
Task scheduling step, described task management terminal is listening to a group task execution of described no preposition dependenceWhen completing, rely on sequential scheduling residue task according to tasks carrying；
Iterative task distributing step, task execution step, monitoring step and task scheduling step, until described workflow is heldRow completes.
Another preferred embodiment of the present invention also provides a kind of task cluster dispatching managing device, is applied to tasks carrying eventuallyIt is characterised in that described device includes in the task management terminal of end communication:
Workflow-deployment module, for being scheduling to predefined workflow, described predefined workflow packagesInclude multigroup between there is the task of execution dependence order, a group task of wherein no preposition dependence needs to execute at firstTask；
Task distribution module, for being distributed at least one tasks carrying eventually by a group task of described no preposition dependenceEnd execution；
Monitor module, for monitoring a group task of described no preposition dependence in described at least one tasks carrying terminalOn practice condition；
Task scheduling modules, during for completing in the group task execution listening to described no preposition dependence, according toTasks carrying relies on sequential scheduling residue task；
Wherein, described task distribution module, monitoring module and task scheduling modules circulate execution, successively until described workIn flowing, all of tasks carrying completes.
In terms of existing technologies, the method have the advantages that
Set up the sequencing of tasks carrying by the execution dependence between setting task, appoint first carrying outAfter the completion of business execution, just the other tasks depending on this task can be distributed to execution terminal and be executed, can be prevented effectively fromThe wasting of resources that causes and the low technical problem of tasks carrying efficiency because of the unordered distribution of task.
Specific embodiment
Below in conjunction with accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, completeGround description is it is clear that described embodiment is only a part of embodiment of the present invention, rather than whole embodiments.Generally existThe assembly of the embodiment of the present invention described and illustrated in accompanying drawing can be arranged with various different configurations and design herein.CauseThis, be not intended to limit claimed invention to the detailed description of the embodiments of the invention providing in the accompanying drawings belowScope, but it is merely representative of the selected embodiment of the present invention.Based on embodiments of the invention, those skilled in the art are not doingThe every other embodiment being obtained on the premise of going out creative work, broadly falls into the scope of protection of the invention.
It should also be noted that similar label and letter expression similar terms in following accompanying drawing, therefore, once a certain Xiang YiIt is defined in individual accompanying drawing, then do not need it to be defined further and explains in subsequent accompanying drawing.Meanwhile, the present invention'sIn description, term " first ", " second " etc. are only used for distinguishing description, and it is not intended that indicating or hint relative importance.
Refer to Fig. 1, be that the task management terminal 100 that present pre-ferred embodiments provide is led to tasks carrying terminal 200The interactive schematic diagram of letter.Described task management terminal 100 can be communicated with described tasks carrying terminal 200 by network 300,With the data cube computation realized between described task management terminal 100 and described tasks carrying terminal 200 or interact.
Refer to Fig. 2, be the block diagram that Fig. 1 is a kind of task management terminal 100 provided in an embodiment of the present invention.InstituteState task management terminal 100 and include task cluster dispatching managing device 110, memorizer 111, storage control 112, processor113 and communication unit 114.
Described memorizer 111, storage control 112, processor 113 and each element of communication unit 114 are direct each otherOr be indirectly electrically connected with, to realize transmission or the interaction of data.For example, these elements can pass through one or more each otherCommunication bus or holding wire are realized being electrically connected with.Described task cluster dispatching managing device 110 includes at least one can be with softwareOr the form of firmware (firmware) is stored in described memorizer 111 or is solidificated in the operation of described task management terminal 100Software function module in system (operating system, os).Described memorizer 111 stores application program.Described processDevice 113 is used for executing the executable module of storage in described memorizer 111, for example described task cluster dispatching managing device 110Included software function module and computer program etc..
Wherein, described memorizer 111 may be, but not limited to, random access memory (random accessMemory, ram), read only memory (read only memory, rom), programmable read only memory (programmableRead-only memory, prom), erasable read-only memory (erasable programmable read-onlyMemory, eprom), electricallyerasable ROM (EEROM) (electric erasable programmable read-onlyMemory, eeprom) etc..Wherein, memorizer 111 be used for storage program, described processor 113 after receiving execute instruction,Execution described program.Described processor 113 and other possible assemblies can control in described storage to the access of memorizer 111Carry out under the control of device 112.
Described processor 113 is probably a kind of IC chip, has the disposal ability of signal.Above-mentioned processor113 can be general processor, including central processing unit (central processing unit, cpu), network processing unit(network processor, np) etc.；Can also be digital signal processor (dsp)), special IC (asic), ready-madeProgrammable gate array (fpga) or other PLDs, discrete gate or transistor logic, discrete hardware groupPart.Can realize or execute disclosed each method in the embodiment of the present invention, step and logic diagram.General processor is permissibleIt is microprocessor or this processor can also be any conventional processor etc..
Described communication unit 114 is used for setting up described task management terminal 100 by described network 300 and tasks carrying is wholeEnd 200 communication connections between the two, thus realize described task management terminal 100 hold in described tasks carrying terminal 200The management of row task.
First embodiment
Refer to Fig. 3, Fig. 3 is task cluster dispatching managing device shown in Fig. 2 110 that present pre-ferred embodiments provideFunctional block diagram.Described task cluster dispatching managing device 110 includes workflow-deployment module 1101, task distribution module1102nd, module 1103 and task scheduling modules 1104 are monitored.Fig. 4 is the task colony dispatching pipe that present pre-ferred embodiments provideThe flow chart of reason method.The method and step of the related flow process of methods described can be by described workflow-deployment module 1101, taskDistribution module 1102, monitor module 1103 and task scheduling modules 1104 to realize.Below how modules are specifically realizedIdiographic flow is described in detail.
Step s111, described workflow-deployment module 1101 is scheduling to predefined workflow.
In the present embodiment, described predefined workflow include multigroup between exist execution dependence order task.
Specifically, described task management terminal can be pre-defined to workflow by visualization interface, itsIn, the execution sequencing between definition task and task is included, but not limited to the predefined content of workflow.LiftExample illustrates it is assumed that it is now desired to being processed the information of trade company for the use of other platforms, wherein, the information of trade companyIncluding but not limited to merchant transaction report data statistics, activity data report form statistics and trade company's member's report data statistics.In order toMake data compatible or consistent between different platform, Data Control can be carried out in the example shown by way of white list.ThisSample just develops and a data work flow: obtains white list trade company -- > and proposes basic data -- > data secondary operations -- > and rawBecome business datum.The course of processing of data is referred to Fig. 5, according to Fig. 5, it is known that above-mentioned workflow is by 8 tasksComposition.Task dependence is: first carries out task [extracting white list trade company id], treats that this tasks carrying executes after terminating again[newly-increased white list trade company id inserts other platform service forms], after tasks carrying terminates, more simultaneously and concurrently executes task [trade companyTransaction report data statistics], [activity data report form statistics] and [trade company member's report data count], treat task activity datagramAfter list processing terminates, then execution task [scene marketing report form statistics], execution terminates followed by execution [scene marketing form knotFruit is put in storage].Task [terminating data mart modeling, rehabilitation] needs to wait merchant transaction report data statistics, trade company's member's formData statisticss, scene marketing form result are put in storage could start to process after the whole process of these three tasks terminates.Understanding workAfter mutual relation between task in stream, you can to define task on the visualization interface in described task management terminal, andSet up the execution dependence between task.
In the present embodiment, described visualization interface can also arrange following functions.Resource distribution function, above-mentioned to appointBusiness definition adopt this resource distribution functional realiey, described resource distribution function can also include configuration timer-triggered scheduler,Real time inspection server registration situation etc..Control of authority function, realizes function privilege, data permission by user, role, authorityConfiguration management.Elastic dilatation function, completes dilatation by automation services login mechanism, and the dilatation of execution terminal needs two steps completeBecome, first pass through resources configuration management interface editing tasks carrying terminal quantity, then increase tasks carrying terminal disposition nodeRegister machine by automation services and complete dilatation.Real-time status monitoring function, the running status of monitor in real time workflow and task,The process progress of Real-time Feedback task, run succeeded or failure situation and run time-out warning situations such as.
In the present embodiment, after user carries out to workflow predefining, task cluster dispatching managing device 110 basisBy figure whether this dependence is assembled into a figure by workflow, and whether the workflow of definition is correct, whether there is closed loop,Meet the feature of directed acyclic graph to be judged.Specific evaluation algorithm can be using Depth Priority Algorithm and topology rowSequence algorithm, wherein workflow in the process of implementation, are processed by directed acyclic graph is changed into a topological queue.
Step s112, task is distributed at least one tasks carrying terminal execution by task distribution module 1102.
In the present embodiment, a group task of no preposition dependence is distributed to by described task distribution module 1102 firstAt least one tasks carrying terminal execution.
Specifically, a group task of described no preposition dependence refers to that the execution of this group task needs not rely on otherBusiness implementing result it is however generally that, in the present embodiment, a group task of described no preposition dependence refers in workflowFirst task of needing execution.Such as, the task in the example above " extracting white list trade company id ".
Task, when carrying out task distribution, is also carried out burst and processes to obtain number of tasks by described task distribution module 1102According to burst, and by described task data burst be sent to described at least one tasks carrying terminal execution.
In the present embodiment, the quantity of task data burst be execute this task execution terminal data and registration execute endSmaller value in terminal number amount.
Wherein, the execution terminal quantity of task can be defined by the visualization interface in described task management terminal 100:M is it is also possible to obtain registration execution terminal quantity: r by reading zookeeper.Described data fragmentation piece number p=min m,r}.
Step s113, monitors module 1103 and monitors execution shape in described at least one tasks carrying terminal for the described taskCondition.
In the present embodiment, log system holding to the data fragmentation in described at least one tasks carrying terminal can be passed throughRow daily record is collected, to realize the monitoring to the practice condition stated at least one tasks carrying terminal.
Specifically, by being transformed to log system so that the daily record producing in described execution terminal is according to workflowExecution address carry out packet aggregation, the daily record collected is arranged by final log system, make one execution address correspond toOne journal file.
In the present embodiment, described task management terminal adopts the scheduling mechanism based on event, by the thing of zookeeperPart monitors and achieves event synchronization in cluster, by the encapsulation again to event, it is to avoid same event is many in clusterSecondary response.
Step s114, task scheduling modules 1104, when listening to described tasks carrying and completing, judge whether also in needThe task of execution, relies on sequential scheduling residue task when also sometimes entering step s112 according to tasks carrying, iterative task is distributedStep s112, monitoring step s113 and task scheduling step s114, in described workflow, all of tasks carrying completes.WhenWhen not needing executing of task, terminate flow process.
In the case of there is execution dependence between multiple tasks, according to the sequencing executing in workflowTask carries out order and carries out distributed scheduling execution.
In the present embodiment, described task cluster schedule management method can also include: choose master for each workflow and holdRow terminal, described main execution terminal has workflow described in right scheduling.
Specifically, judge the weighted value size of all online execution terminals, if weighted value is different, select weighted value maximumExecution terminal based on execute terminal；If weighted value is identical, randomly chooses one from all online execution terminals and execute endEnd is as main execution terminal；If cannot randomly choosing from all online execution terminals and obtaining main execution terminal, by workThe execution terminal flowing down first registration is as described main execution terminal.
In the present embodiment, an execution terminal is randomly choosed as main execution terminal using random function, wherein, describedRandom function can be realized using conventional random distribution algorithm (such as, concordance hash algorithm).
Specifically, described execution terminal carries out intersecting registration under each workflow, by lower for workflow first registrationExecution terminal is as described main execution terminal, or the weighted value according to registration execution terminal under each workflow, selects weighted valueMaximum execution terminal is as described main execution terminal.
In the present embodiment, the execution terminal only executing terminal based on one-tenth just has this workflow of right scheduling, in order to avoid, by same execution terminal as main execution terminal, described task cluster dispatching managing device is to execution terminal for most workflowsOn scheduler carry out load balance process.Specifically, refer to Fig. 6, be to carry out selecting main execution using intersection logon modeThe schematic diagram of terminal, carries out main execution terminal by the way of intersecting registration for workflow flow1, flow2 and flow3 and dividesJoin.By intersecting login mechanism, the load balancing of the preliminary scheduler realized, it is to avoid the scheduling of all working stream falls and holds atIn row terminal.
In the present embodiment, can smoothly timely execute to ensure task, described task cluster schedule management method is adoptedWith the mode of failover, the task of execution failure is shifted.Specifically, described monitoring module 1103 is listening to taskWhen task data burst in actuating station executes unsuccessfully, the task data burst of described execution failure is scheduled to other tasks and holdsRow terminal executes.
Second embodiment
The present embodiment provides a kind of task cluster schedule management method, and this task cluster schedule management method is applied to mutuallyBetween communication connection tasks carrying terminal 200 and task management terminal 100, methods described includes:
Step s211, described task management terminal 100 is scheduling to predefined workflow, described predefinedWorkflow include multigroup between exist execution dependence order task, a group task of wherein no preposition dependence is to needFirst carrying out of task.
Step s212, task is distributed at least one tasks carrying terminal execution by described task management terminal 100.
In the present embodiment, a group task of no preposition dependence is distributed to by described task management terminal 100 firstAt least one tasks carrying terminal execution.
Step s213, described at least one tasks carrying terminal 200 executes described task.
Step s214, described task management terminal 100 monitors described task in described at least one tasks carrying terminal 200Practice condition.
Step s215, described task management terminal 100, when listening to described tasks carrying and completing, judges whether also to needTask to be executed, relies on sequential scheduling residue task when also sometimes entering step s212 according to tasks carrying, iterative task dividesSend out step s212, task execution step s213, monitor step s214 and step s215, all of task in described workflowExecution completes.When the task of not needing to execute, terminate flow process.
In sum, the present invention implements task cluster schedule management method and the device providing, between setting taskExecution dependence set up the sequencing of tasks carrying, after the completion of should first carrying out tasks carrying, just can will rely onOther tasks in this task are distributed to execution terminal and are executed, and can be prevented effectively from the money that causes because of the unordered distribution of taskThe technical problem that source wastes and tasks carrying efficiency is low.
It should be understood that disclosed equipment and method are it is also possible to pass through in several embodiments provided hereinOther modes are realized.Apparatus embodiments described above are only schematically, for example, the flow chart in accompanying drawing and block diagramShow the equipment of multiple embodiments according to the present invention, the architectural framework in the cards of method and computer program product,Function and operation.At this point, each square frame in flow chart or block diagram can represent the one of a module, program segment or codePart, a part for described module, program segment or code comprises holding of one or more logic function for realizing regulationRow instruction.It should also be noted that at some as in the implementation replaced, the function of being marked in square frame can also be to be different fromThe order being marked in accompanying drawing occurs.For example, two continuous square frames can essentially execute substantially in parallel, and they are sometimesCan execute in the opposite order, this is depending on involved function.It is also noted that it is every in block diagram and/or flow chartThe combination of the square frame in individual square frame and block diagram and/or flow chart, can be with the special base of the function of execution regulation or actionSystem in hardware to be realized, or can be realized with combining of computer instruction with specialized hardware.
In addition, each functional module in each embodiment of the present invention can integrate one independent portion of formationDivide or modules individualism is it is also possible to two or more modules are integrated to form an independent part.
If described function realized using in the form of software function module and as independent production marketing or use when, permissibleIt is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other wordsPartly being embodied in the form of software product of part that prior art is contributed or this technical scheme, this meterCalculation machine software product is stored in a storage medium, including some instructions with so that a computer equipment (can be individualPeople's computer, server, or network equipment etc.) execution each embodiment methods described of the present invention all or part of step.And aforesaid storage medium includes: u disk, portable hard drive, read only memory (rom, read-only memory), random access memory are depositedReservoir (ram, random access memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
It should be noted that herein, such as first and second or the like relational terms are used merely to a realityBody or operation are made a distinction with another entity or operation, and not necessarily require or imply these entities or deposit between operatingIn any this actual relation or order.And, term " inclusion ", "comprising" or its any other variant are intended toComprising of nonexcludability, wants so that including a series of process of key elements, method, article or equipment and not only including thoseElement, but also include other key elements being not expressly set out, or also include for this process, method, article or equipmentIntrinsic key element.In the absence of more restrictions, the key element that limited by sentence "including a ..." it is not excluded thatAlso there is other identical element including in the process of described key element, method, article or equipment.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for the skill of this areaFor art personnel, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, made any repairChange, equivalent, improvement etc., should be included within the scope of the present invention.It should also be noted that similar label and letter existRepresenting similar terms in figure below, therefore, once being defined in a certain Xiang Yi accompanying drawing, being then not required in subsequent accompanying drawingIt is defined further and to be explained.
The above, the only specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, and anyThose familiar with the art the invention discloses technical scope in, change or replacement can be readily occurred in, all should containCover within protection scope of the present invention.Therefore, protection scope of the present invention should described be defined by scope of the claims.