Disclosure of Invention
In order to solve the technical problem, the application provides a method and a system for realizing task scheduling, which can greatly enhance the expandability of a trigger event, thereby improving the working efficiency and reducing the working cost.
In order to achieve the object of the present application, the present application provides a method for implementing task scheduling, including: creating a task by using a task template realized by adopting a plug-in;
and scheduling the tasks in the real-time queue according to the real-time state of the tasks.
The task template at least comprises: the task template identification ID is used for uniquely identifying the task template;
task template names, wherein different task template IDs correspond to different task template names; and the number of the first and second groups,
and the plug-in is used for dynamically loading the tasks through task scheduling.
The task module further comprises:
template configuration with contents being a combination of commands; and/or the presence of a gas in the gas,
when the task is executed, a parameter list is transmitted through an execution function of the plug-in; and/or the presence of a gas in the gas,
a strategy for analyzing the output of the plug-in operation; and/or the presence of a gas in the gas,
and a result response for driving completion of a next link of the task according to the execution result of the task.
The task scheduling according to the real-time state of the task comprises the following steps of gradually loading and applying the components of the task template: starting a working thread according to the created task instance, and dynamically loading the plug-in; and after the task is successfully executed, finishing the working thread and destroying the successfully executed task.
The step-by-step loading and application of the components of the task template further comprises the following steps: calling an execution function of the plug-in, transferring template configuration parameters and/or parameters in a parameter list and/or strategy information, and analyzing and processing the parameters.
The interface of the plug-in comprises an execution function, a pause function, a retry function and an interrupt function.
The task template comprises a template configuration; the template configuration uses macros built-in to the task scheduling engine, and variable external macros.
The task template comprises a parameter list; the parameters in the parameter list are parameters with fixed values and/or parameters with variable values;
the variable value parameter includes parameter macro and parameter sentence.
The task template comprises a strategy; the format of the policy is: the XML format, the initialization file ini format, the data configuration file dat format, the lightweight data exchange json format or the binary format.
The task template comprises result responses; the resulting response is an operation function.
The method further comprises the following steps: monitoring the state of the task being executed, and recording the progress and completion of the task:
if the task in the real-time queue is successfully executed, deleting the task from the real-time queue;
and if the task in the real-time queue fails to be executed, determining whether to be executed again according to the interval time of the task failure and the limit of the retry times.
The re-executing specifically includes: judging whether the retry times exceed a preset retry time threshold, judging whether the interval time exceeds a preset retry interval threshold when the retry times do not exceed the retry time threshold, and if the interval time exceeds the retry interval threshold, re-executing the task;
if the retry number is judged to exceed the retry number threshold value, the task is not executed any more;
and if the retry times are judged not to exceed the retry times threshold value, and the interval time does not exceed the retry interval threshold value, the task is not executed temporarily.
The method comprises the following steps: and carrying out mirror image processing on the tasks which are not executed any more, and deleting the tasks from the real-time queue.
The deleting task comprises: and recording the task and the execution result thereof, destroying the task instance, deleting the task from the real-time task queue, and updating a counter for recording the execution times of the task.
The method also comprises the following steps: and determining whether to restart the task scheduling according to the state of the pre-configured watchdog.
When performing the task, the method further comprises: the length of the task queue currently being executed is queried,
if the current queue length is smaller than a preset queue length threshold value, the task is directly executed; if the length of the current queue has exceeded the queue length threshold, the task is not executed.
The method also comprises the following steps: judging whether the task scheduling engine is suspended or not, and if the task scheduling engine is suspended, not checking the automatic task queue;
otherwise, checking the automatic task queue and executing all tasks in the schedule, wherein the time is not later than the current time.
In the task scheduling engine, a continuous failure task queue for storing continuous failure tasks, a time threshold value for automatically switching to a pause state after the tasks are continuously failed, and a preset retry time window for retrial detection after the tasks are paused are preset; the method further comprises the following steps:
before the current task is to be executed, if the current task is judged to exist in the continuous failure task queue, acquiring the continuous failure times and the final execution time of the current task to be executed;
when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the last execution time is smaller than a retry time window, exiting the current task;
when the continuous failure times are smaller than a preset time threshold value, executing the task to be executed at present;
and executing the task which is currently prepared to be executed when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the last execution time is larger than a retry time window.
After the task is successfully executed, the method further comprises the following steps: resetting the continuous failure times of the task to zero, and finally setting the execution time as the current time.
After the task fails to execute, the method further comprises: and adding one to the continuous failure times of the task, and setting the final execution time as the current time.
The present application further provides a system for implementing task scheduling, including: an interface and adaptation unit, a task scheduling engine, and a task instance unit, wherein,
the interface and adaptation unit is used for acquiring an external manual task;
the task instance unit is used for completing the work content given to the task according to the parameters transmitted when the task is started and the execution-response sequence; reporting the real-time state and progress information in the task execution process to a task scheduling engine;
and the task scheduling engine is used for creating a task by using a task template realized by adopting the plug-in and scheduling the task according to the real-time state of the task.
The task scheduling engine comprises a task creating module and a task executing module, wherein,
the task creating module is used for creating tasks by utilizing a task template realized by adopting a plug-in;
and the task execution module is specifically used for starting a working thread according to the created task instance, dynamically loading the plug-in, finishing the working thread after the task is successfully executed, and destroying the successfully executed task.
The task template at least comprises:
the task template ID is used for uniquely identifying the task template;
task template names, wherein different task template IDs correspond to different task template names; and the number of the first and second groups,
and the plug-in is used for dynamically loading the tasks through task scheduling.
The task template further comprises:
a template configuration for containing a combination of commands; and/or the presence of a gas in the gas,
when the task is executed, a parameter list is transmitted by an execution function; and/or the presence of a gas in the gas,
a policy for analyzing the output of the plug-in operation; and/or the presence of a gas in the gas,
and a result response for driving completion of a next link of the task according to the execution result of the task.
The task execution module is also used for calling an execution function of the plug-in to transmit the template configuration parameters and/or the parameters in the parameter list and/or the strategy information, and analyzing and processing the parameters.
The task execution module is configured with a watchdog and is further used for entering task scheduling when the status of the watchdog is true.
The interface and adaptation unit is also used for receiving a control instruction from the outside to the task scheduling engine and acquiring the state of the task scheduling engine;
the task execution module is also used for not checking the automatic task queue when the received control instruction from the interface and the adaptation unit is pause;
when the received control instruction from the interface and the adaptation unit is continued, the automatic task queue is checked and all tasks in the schedule, which are not later than the current time, are executed.
The task execution module is also used for inquiring the length of a task queue currently executed when a new task is executed, and directly executing the task when the length of the current queue is smaller than a preset queue length threshold; when the length of the current queue has exceeded the queue length threshold, the task is not executed.
The task scheduling engine also comprises a task monitoring module which is used for monitoring the state of the task being executed, recording the progress and the completion condition of the task, and deleting the task from the real-time queue when the task in the real-time queue is successfully executed;
and when the task in the real-time queue fails to be executed, determining whether to be executed again according to the interval time of the task failure and the limit of the retry times.
The task monitoring module is specifically configured to: in the event of a failure in the execution of a task,
judging whether the retry times exceed a preset retry time threshold value, further judging whether the interval time exceeds a retry interval threshold value when the retry times do not exceed the retry time threshold value, and if the interval time exceeds the retry interval, re-executing the task; if the retry times of the task are judged to exceed the retry time threshold value, the task is not executed any more; if the retry times of the task do not exceed the retry times threshold and the interval time does not exceed the retry interval threshold, the task is not executed temporarily;
and for the task which is successfully executed, recording the task and the execution result thereof, destroying the task instance, deleting the task from the real-time task queue, and updating a counter for recording the execution times of the task.
The task execution module is further used for carrying out mirror image processing on the information of the tasks which are not executed any more and deleting the tasks.
The task scheduling engine also comprises an automatic operation and maintenance module; in the automatic operation and maintenance module, a continuous failure task queue for storing continuous failure tasks, a time threshold value for automatically switching to a pause state after the tasks are continuously failed, and a preset retry time window for retrial detection after the tasks are paused are preset;
the automatic operation and maintenance module is used for operating,
before the current task is to be executed, if the current task exists in the continuous failure task queue, acquiring the continuous failure times and the last execution time of the current task to be executed,
when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the obtained last execution time is smaller than a retry time window, exiting the current task;
when the continuous failure times are smaller than a preset time threshold value, executing the task to be executed at present;
and executing the task which is currently prepared to be executed when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the obtained last execution time is larger than a retry time window.
And the automatic operation and maintenance module is also used for resetting the continuous failure times of the tasks to zero after the tasks are successfully executed, and finally, the execution time is set as the current time.
And the automatic operation and maintenance module is also used for adding one to the continuous failure times of the task after the task fails to be executed, and finally setting the execution time as the current time.
The method comprises the steps of creating a task by using a task template realized by adopting a plug-in and scheduling the task according to the real-time state of the task. The task template is created by using the plug-in, so that the expandability of the task is enhanced, and the method is a universal task creating mode; moreover, because the plug-ins in the task template, the template configuration, and/or the parameter list, and/or the strategy, and/or the mutual independence and expandability of the result response can be realized, and the plug-ins and the template configuration, and/or the parameter list, and/or the strategy, and/or the result response can be arranged and combined, the reuse of each component is greatly realized, the expandability of the task is greatly enhanced, and the universality of the method for realizing task scheduling is ensured, so that the expandability of the trigger event is greatly enhanced, the working efficiency is improved, and the working cost is reduced.
In addition, through the automatic operation and maintenance process, when the task execution fails, the task scheduling engine automatically controls the tasks, so that the influence of accumulation of the failed tasks on normal task operation is avoided; moreover, the failed task reenters the normal scheduling process through the attempt of the periodic execution of the failed task, so that the resources are saved, the efficiency is improved, and the manual intervention is avoided.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Detailed Description
To make the objects, technical solutions and advantages of the present application more apparent, embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
In one exemplary configuration of the present application, a computing device includes one or more processors (CPUs), input/output interfaces, a network interface, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Currently, in practical applications, a time-based and periodic task scheduling trigger point is used in most cases. The tasks are triggered according to a time standard, and one task can be separately scheduled to synchronize time according to a period so as to ensure that a trigger point has an accurate time standard. One common type of task scheduling is Quartz, which is an open source job scheduling framework, written entirely in Java and designed for use in J2SE and J2EE applications. Quartz provides great flexibility without sacrificing simplicity. Quartz can be used to create simple or complex schedules for executing a job. Quartz has many features, such as: database support, clustering, plug-in, EJB job pre-construction, JavaMail and others, support cron-like expressions, and the like. In addition, cron4j, cron4j is a Java task scheduling framework, similar to crontab under UNIX system.
The existing technical scheme for realizing task scheduling mostly triggers events by various time rules and supports scheduling schemes of various time rules or frequencies. However, the triggering event types are narrow, and can be driven only by some preset and fixed triggering events. Even for a general task scheduler such as Quartz, the expansibility of trigger events is still not ideal enough, which undoubtedly reduces the working efficiency and increases the working cost. In addition, in some existing solutions for implementing task scheduling, once a large number of tasks fail due to errors in the process of running a large number of tasks, the tasks can be continuously run only after manual intervention and troubleshooting are performed; without human intervention, a number of normal tasks will be affected. The problems are solved with low efficiency, high cost and the like.
Fig. 1 is a flowchart of a method for implementing task scheduling according to the present application, as shown in fig. 1, including:
step 100: tasks are created using task templates implemented using plug-ins.
The task in this application is derived from the task template and a new task is created, meaning that it inherits all the properties of the task template. The task template at least comprises: a task template Identification (ID) for uniquely identifying the task template; task template names, wherein different task template IDs correspond to different task template names; and the plug-in is used for dynamically loading the tasks through task scheduling, and the interface of the plug-in meets the preset requirement.
Because the plug-in is a dynamically loaded module, the plug-in can be written temporarily as required, and only needs to be written according to the preset interface requirement. The interface of the plug-in may be four functions of execute (work), pause (pause), retry (resume), interrupt (terminate). Different plug-ins, the functions that appear when a task is executed, are completely different. According to the method and the device, the trigger event can be conveniently expanded according to the writing of the plug-in. It should be noted that creating tasks by using plug-ins is a conventional technical means for those skilled in the art, and the specific implementation method is not used to limit the scope of the present application, and it should be emphasized in the present application that in task scheduling, tasks are created by using task templates implemented by using plug-ins, so that the extensibility of tasks is enhanced.
Further, the task template may further include: template configuration, and/or parameter lists, and/or policies, and/or outcome responses; wherein,
the contents of the template configuration are a combination of commands, and the template configuration may be a file with tpl as an extension. In actual use, the contents of the template configuration file can be written as required. That is, the functions of the tasks of the same plug-in and different template configurations are different, so that the expandability of the tasks is further enhanced.
In the template configuration, several macros built into the task scheduling engine may be used, as shown in Table 1:
TABLE 1
In addition to the macros shown in table 1, variable external macros can be used in the template configuration, where the external macros are used to transfer parameters in the task template to the x. The format of the outlaid macro may be: % [ N ]%; wherein N is 0, 1, 2 …. In the parameter list, the first parameter is: % [0 ]%, the second parameter is: percent 1, and so on. When the plug-in uses the tpl file, the external macros at different positions are automatically replaced by the real parameter values in the parameter list.
The parameter list is a list of parameters that are passed in through the work function when the task is executed. The parameter list may continue to pass into the tpl file to control and instantiate the tpl file, or may be completely independent of the tpl file. How the parameters in the parameter list are used is entirely determined by the plug-in, and the parameters may be divided into fixed-value parameters and/or variable-value parameters. Wherein,
a fixed value parameter means that the value of the parameter is fixed, such as: -h 127.0.0.1-p 8080 "ab c", etc., wherein the values of the five parameters are fixed, respectively: -h, 127.0.0.1, -p, 8080, "a b c". At this time, the input value is not changed in the transmission process;
the variable-value parameter refers to a parameter that includes a parameter macro and a parameter statement. At this time, the input value is converted from the formal parameter to the substantial parameter during the transmission process, and is instantiated into a different value. Such as:
| macro name | Function of |
| ％[RANDOM]％ | Random number of 0-100 |
| ％[NOWS]％ | Time of yyyy-MM-dd HH: MM: SS format |
| ％[NOWU]％ | Time in integer format |
| ％[Y]％ | Year in the current time |
| ％[m]％ | Month in current time |
| ％[d]％ | Day of the current time |
| ％[H]％ | Hour in the current time |
| ％[M]％ | Minutes of the current time |
| ％[S]％ | Second in current time |
TABLE 2
When the parameters in the parameter list include the parameters related to time in table 2, the task scheduling engine will automatically extract the corresponding values from the scheduled time of the task. It can be seen that even if the plug-in and template configurations are fixed, the parameter list can generate many different changes, so that completely different tasks are formed, and the expandability of the tasks is further enhanced.
And the strategy is used for analyzing the output of the plug-in. In the plug-in development process, the policy may be in any format and in any form, such as extensible markup language (xml) format, initialization file (ini) format, data configuration file (dat) format, lightweight data interchange (json, JavaScript object) format, binary system, or the like. No matter what form the strategy takes, the strategy scheduling and management of the task scheduling engine on the strategy is not influenced, because the plug-in is really understood and can analyze the strategy, and the plug-in is dynamically developed and added. And the expandability of the task is further enhanced through the intervention of the strategy in the whole task processing flow.
And the result response is called at the end stage of the task, and the completion of the next link of the task can be driven according to the execution result of the task. The result response is also a dynamically loaded module, but the result response has only one Action function. The result response can be newly added or adapted according to the actual needs, and any required action is realized, so that the expandability of the task is further enhanced.
In the step, the expandability of the task is enhanced by using the task created by the task template realized by the plug-in, and the method is a universal task creation mode; in addition, due to the mutual independence and expandability of the plug-ins, the template configuration, the parameter list, the strategy and the result response in the task template, and the permutation and combination among the plug-ins, the template configuration, the parameter list, the strategy and the result response, the reuse of each component is greatly realized, the expandability of the task is greatly enhanced, and the universality of the method for realizing task scheduling is ensured.
Step 101: and scheduling the tasks in the real-time queue according to the real-time state of the tasks.
Scheduling a task, that is, executing the task, which is actually a process in which each component of a task template is gradually loaded and applied, substantially includes: starting a working thread according to a task instance contained in the task created in the step 100; and dynamically loading the plug-in, finishing the working thread after the task is successfully executed, and destroying the successfully executed task. It should be noted that a task may be composed of one or more task instances, as is well known to those skilled in the art.
The start of scheduling may be triggered according to a schedule created according to a scheduled task table for scheduling scheduled tasks, or may be triggered manually; the manually triggered task is obtained by the external interface through the underlying communication.
Further, still include: calling a work function of the plug-in to transfer template configuration parameters and/or parameters in a parameter list and/or strategy information, and analyzing and processing the parameters; reading template configuration and use strategies, and calling an Action function to load result responses; and after the task is successfully executed, ending the working thread and destroying the successfully executed task.
In the process of task scheduling, the method of the application further comprises the following steps: monitoring the status of the executing task, recording the progress (such as retry times, etc.) and completion (such as execution success or execution failure, etc.) of the executing task, and if the task in the real-time queue is successfully executed and completed, deleting the task from the real-time queue, i.e. performing task cleaning, mainly comprising: recording the task and the execution result thereof to a database, destroying a task instance, deleting the task from a real-time task queue, and then updating a counter for recording the execution times of the task;
if the task fails to be executed, whether to execute the task again is determined according to the interval time of the task failure (namely the interval time of the last time of the failed execution and the current time) and the limit of the retry number. The method specifically comprises the following steps: judging whether the retry times exceed a preset retry time threshold, further judging whether the interval time exceeds a preset retry interval threshold when the retry times do not exceed the retry time threshold, and if the interval time exceeds the retry interval threshold, immediately retrying to re-execute the task; if it is determined that the threshold number of retries has been exceeded, then the task is no longer performed. If the threshold of the retry times is not exceeded and the threshold of the retry interval is not exceeded, the task is not executed for the moment and is put back into the real-time queue to continue waiting.
For memo or future testing, the method further comprises the following steps: all information dump of tasks which are not executed any more (namely, mirroring) is put into a file, and then task cleaning is carried out on the tasks, which mainly comprises the following steps: and recording the task and the execution result thereof in a database, destroying a task instance, deleting the task from the real-time task queue, and updating a counter for recording the execution times of the task.
During the task execution process, the task scheduling engine further performs concurrent task control according to the current length of the real-time task queue, that is, controls the total number of tasks executed simultaneously. When a new task is executed, the length of a task queue currently being executed is always inquired, and if the length of the current queue is smaller than a preset queue length threshold (the maximum length value allowed by the task queue), the task is directly executed; if the length of the current queue has exceeded the queue length threshold, then the task will not be executed in this round.
The method also comprises the following steps: and determining whether to reload the scheduled task table or not according to the state of the configuration watchdog, namely True/False (True/False), and generating the latest time table according to the currently loaded scheduled task table. The application of the watchdog belongs to the conventional technical means of technicians in the field, the specific implementation can be realized through hardware or software, and details are not repeated, in the method, the watchdog is used, when the state of the watchdog is true, the watchdog is reloaded into the scheduled task table, the latest time schedule is generated according to the currently loaded scheduled task table, and the start of scheduling is triggered according to the latest time schedule so as to restart the scheduling of the tasks.
Further, the method of the present application further comprises: judging whether the task scheduling engine is suspended according to a control instruction from the outside, and if the task scheduling engine is suspended, not checking the automatic task queue; otherwise, checking the automatic task queue and executing all tasks in the schedule, wherein the time is not later than the current time.
It should be noted that the task scheduling engine may preferentially check the manual task queue, i.e., execute immediately as long as there is a task in the manual task queue.
As a task scheduling engine, the automatic tasks scheduled at any moment are necessarily various in quantity and variety. Various problems or failures of the task are almost unavoidable. Once a large number of tasks fail, the failed tasks will be retried continuously, occupying the queue of real-time tasks. In view of the fact that the total number of tasks which can be executed concurrently at the same time is limited, a large number of tasks are inevitably queued up, and task congestion is caused; even if some types of tasks do not fail, they cannot be executed because other types of failed tasks crowd real-time tasks. Therefore, the method of the present application further comprises: an automated operation and maintenance process is determined whether to perform a task to be performed.
Fig. 2 is a schematic flow diagram of an automatic operation and maintenance process implemented by the present application, where a continuous failure task queue for storing continuous failure tasks, a time threshold for automatically switching to a suspended state after a task is continuously failed, and a preset retry time window for retrial detection after a task is suspended are preset in a task scheduling engine. These two thresholds may be set in a configuration file; as shown in fig. 2, the method comprises the following steps:
step 200: before the current task (the task in the automatic task queue) is executed, whether a continuous failure task with consistent task ID exists in the continuous failure task queue is searched according to the task ID, and if the continuous failure task exists, the step 201 is executed; otherwise step 205 is entered.
Step 201: and acquiring the continuous failure times and the final execution time of the task to be executed currently.
Step 202: judging whether the obtained continuous failure times are larger than a preset time threshold value or not, and if so, entering a step 203; otherwise step 205 is entered.
Step 203: judging whether the difference between the current time and the obtained last execution time is smaller than a retry time window or not, and entering a step 204 if the difference is smaller than the retry time window; otherwise step 205 is entered.
It should be noted that the task executed at this time is a task that performs tentative detection execution after entering the automatic operation and maintenance process. Therefore, when the number of continuous failures of a task is greater than the preset number threshold, but the difference between the current time and the last execution time of the task is greater than the retry time window, the task needs to be executed in a detection manner in the automatic operation and maintenance process, so that the problem of task congestion caused by the reason that the failed task always processes a queue waiting state is solved.
Step 204: exiting the current task means not executing the task currently ready to be executed. The process is exited.
Step 205 to step 206: the task that is currently ready to be executed is executed, and the process proceeds to step 208 when the task is successfully executed, and proceeds to step 207 when the task is failed to be executed.
Step 207: and updating the mark of the task in the continuous failure task queue, namely adding one to the continuous failure times, and setting the final execution time as the current time. The flow is ended.
Step 208: and resetting the mark of the task in the continuous failure task queue, namely clearing the continuous failure times, and finally setting the execution time as the current time.
In addition, in order to better realize the automatic operation and maintenance of the task, the method further comprises the following steps: and starting or closing the automatic operation and maintenance process. After the task scheduling engine is initialized, the default automatic operation and maintenance process is started.
Through the automatic operation and maintenance process, when the task execution fails, the task scheduling engine automatically controls the tasks, and the influence of accumulation of the failed tasks on normal task operation is avoided; moreover, the failed task reenters the normal scheduling process through the regular attempt of the failed task, so that the resources are saved, the efficiency is improved, and the manual intervention is avoided.
The present application further provides a system for implementing task scheduling, as shown in fig. 3, including: an interface and adaptation unit, a task scheduling engine, and a task instance unit, wherein,
the interface and adaptation unit is used for acquiring an external manual task by the task scheduling engine; and the system is further used for receiving control instructions such as pause and continue from the outside to the task scheduling engine and acquiring the state of the task scheduling engine.
A task instance unit, which is used for completing the work content given to the task according to the parameters transmitted when the task is started and the sequence of execution (work) -response (response); and reporting the real-time state and progress information in the task execution process to a task scheduling engine. The task instances are the components of tasks (also called running tasks) which are scheduled by the task engine one by one, and when one task is scheduled and started, the task instances are both automatic tasks and manual tasks.
And the task scheduling engine is used for creating a task by using a task template realized by adopting the plug-in and scheduling the task according to the real-time state of the task. Specifically, the task scheduling engine includes a task creation module and a task execution module, wherein,
and the task creating module is used for creating the task by utilizing the task template realized by the plug-in. Wherein,
the task template at least comprises: the task template ID is used for uniquely identifying the task template; task template names, wherein different task template IDs correspond to different task template names; the plug-in is used for dynamically loading the tasks through task scheduling, and an interface of the plug-in meets the preset requirement;
the task template further comprises: a template configuration for containing a combination of commands; and/or a parameter list which is transmitted through a work function when the task is executed; and/or a policy for analyzing the output of the plug-in; and/or a result response for driving completion of a next link of the task according to the execution result of the task.
The task execution module is specifically used for gradually loading and applying the components of the task template: and starting a working thread according to the created task instance, dynamically loading the plug-in, finishing the working thread after the task is successfully executed, and destroying the successfully executed task.
Further, the task execution module is further configured to call a work function of the plug-in to transfer the template configuration parameter, and/or the parameter in the parameter list, and/or the policy information, and parse and process the parameter.
Further, the task execution module is further configured to configure a watchdog and enter task scheduling when the watchdog state is true.
Further, the task execution module is also used for not checking the automatic task queue when receiving the pause instruction from the interface and the adaptation unit; upon receiving a continue instruction from the interface and adaptation unit, the automatic task queue is checked and all tasks in the schedule are executed that are not later than the current time.
The task scheduling engine of the present application further comprises: the task monitoring module is used for monitoring the state of the task being executed, recording the progress and the completion of the task, and particularly,
when the task is successfully executed, deleting the task from the real-time queue;
when the task fails to execute, whether to execute the task again is determined according to the interval time of the task failure and the limit of the retry number. Specifically, the method comprises the following steps: judging whether the retry times exceed a preset retry time threshold value, further judging whether the interval time exceeds a retry interval threshold value when the retry times do not exceed the retry time threshold value, and if the interval time exceeds the retry interval, immediately re-executing the task; if the retry times of the task are judged to exceed the retry time threshold value, the task is not executed any more; if the retry times of the task do not exceed the retry times threshold and the interval time does not exceed the retry interval threshold, the task is not executed temporarily and is put back into the real-time queue to continue waiting.
At this time, the task execution module is further configured to perform mirroring on information of the task that is no longer executed, and delete the task.
And the task execution module is also used for controlling the concurrent tasks according to the current length of the real-time task queue: when a new task is executed, inquiring the length of a task queue currently being executed, and when the length of the current queue is smaller than a preset queue length threshold value, directly executing the task; when the length of the current queue has exceeded the queue length threshold, the task is not executed.
The task scheduling engine of the present application further comprises: and an automatic operation and maintenance module.
In the automatic operation and maintenance module, a continuous failure task queue for storing continuous failure tasks, a time threshold value for automatically switching to a pause state after the tasks are continuously failed, and a preset retry time window for retrial detection after the tasks are paused are preset; before the current task is to be executed, the following steps are specifically performed:
if the current task exists in the continuous failure task queue, acquiring the continuous failure times and the last execution time of the current task to be executed, and exiting the current task when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the acquired last execution time is smaller than a retry time window; when the continuous failure times are smaller than a preset time threshold value, executing the task to be executed at present; when the continuous failure times are larger than a preset time threshold value and the difference between the current time and the obtained last execution time is larger than a retry time window, executing the task to be executed currently;
the automatic operation and maintenance module is further used for resetting the continuous failure times of the task to zero after the task is successfully executed, and finally setting the execution time as the current time; and after the execution of the task fails, adding one to the continuous failure times of the task, and finally setting the execution time as the current time.
It will be appreciated by those skilled in the art that the components of the apparatus and steps of the method provided in the embodiments of the present application described above may be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented in program code executable by a computing device. Thus, they may be stored in a memory device for execution by a computing device, or they may be separately fabricated as individual integrated circuit modules, or multiple modules or steps thereof may be fabricated as a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
Although the embodiments disclosed in the present application are described above, the descriptions are only for the convenience of understanding the present application, and are not intended to limit the present application. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims.