Invention content
In view of this, it is necessary to for the Mission Scheduling that can only be set in the prior art using hour as rank, provideA kind of high frequency method for scheduling task, system, computer equipment and storage medium.
A kind of high frequency method for scheduling task, the high frequency method for scheduling task include：
Dispatch server calls default timed task frame in the database, is timed task scheduling, the timingTask framework is preset with timed task, and the timed task is configured with task definition, scheduling rule and scheduling information, scheduling informationStorage is in the database；
Dispatch server configuration schedules routine interface appoints any timing in timed task frame by dispatcher interfaceThe scheduling rule and scheduling information of business are modified, and dispatcher interface is encapsulated as api interface and is placed in django engineerings；
Dispatch server configuration monitoring device, the monitor is according to scheduled monitor mode to being stored with the number of scheduling informationSQL query is carried out according to library, is notified after obtaining monitored results.
The dispatch server is by calling configure tools to install python automatically in one of the embodiments,Program, the timed task frame are included in the python programs, i.e. the setting of completion timing task framework.
When scheduling information stores in one of the embodiments, ORM tools, scheduling clothes are preset in dispatch serverDevice be engaged in by calling ORM tools to read scheduling information, is written in database in the form of scheduled digitization, that is, completes scheduling informationStorage.
The monitor mode is monitored using setting polling time interval in one of the embodiments,；
The monitored results are the exception information in database, including error information when scheduler task, time-out information, instituteMonitored results are stated to be notified by way of mail or phone.
Also include in one of the embodiments, scheduler modules, the dispatch server in the python programsUsing the reschedule_job interfaces of scheduler modules as dispatcher interface, complete to scheduling rule and scheduling informationModification.
The database uses relevant database, non-relational database or memory number in one of the embodiments,According to library.
The scheduling rule includes scheduling mode in one of the embodiments, and the scheduling mode includes blocking modelOr asynchronous mode, the dispatch server complete the modification to scheduling mode by the dispatcher interface.
A kind of high frequency task scheduling system, the high frequency task scheduling system include：
Timed task dispensing unit calls default timed task frame in the database for dispatch server, carries outTimed task is dispatched, and the timed task frame is preset with timed task, and the timed task is configured with task definition, scheduling ruleThen stored in the database with scheduling information, scheduling information；
Interface dispensing unit is used for dispatch server configuration schedules routine interface, is appointed to timing by dispatcher interfaceThe scheduling rule and scheduling information of any timed task are modified in business frame, and dispatcher interface is encapsulated as api interfaceIt is placed in django engineerings；
Monitoring unit is used for dispatch server configuration monitoring device, and the monitor is according to scheduled monitor mode to storageThere is the database of scheduling information to carry out SQL query, is notified after obtaining monitored results.
A kind of computer equipment, including database and processor are stored with computer-readable instruction in the database, instituteWhen stating computer-readable instruction and being executed by the processor so that the processor executes the step of above-mentioned high frequency method for scheduling taskSuddenly.
A kind of storage medium being stored with computer-readable instruction, the computer-readable instruction are handled by one or moreWhen device executes so that one or more processors execute the step of above-mentioned high frequency method for scheduling task.
Above-mentioned high frequency method for scheduling task, device, computer equipment and storage medium, including dispatch server are called and are presetTimed task frame in the database, is timed task scheduling, and timed task frame is preset with timed task, timed taskConfigured with task definition, scheduling rule and scheduling information, scheduling information stores in the database；Dispatch server configuration schedules journeySequence interface repaiies the scheduling rule and scheduling information of any timed task in timed task frame by dispatcher interfaceChange, dispatcher interface is encapsulated as api interface and is placed in django engineerings；Dispatch server configuration monitoring device, monitorSQL query is carried out to the database for being stored with scheduling information according to scheduled monitor mode, is notified after obtaining monitored results.The technical program is used as new scheduling engine by introducing timed task frame, is configured according to timed task and starts task scheduling,The high frequency task scheduling of support minute, even second rank；Database realizing scheduling information persistence is written into scheduling information；Dynamic modification task scheduling frequency is realized by api interface, by the task scheduling in a manner of support timing, time interval etc.；It introducesMonitor is realized to the real time monitoring of scheduling process, finds in time and change the problem in task scheduling.
Specific implementation mode
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, rightThe present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, andIt is not used in the restriction present invention.
Those skilled in the art of the present technique are appreciated that unless expressly stated, singulative " one " used herein, " oneIt is a ", " described " and "the" may also comprise plural form.It is to be further understood that is used in the specification of the present invention arrangesIt refers to there are the feature, integer, step, operation, element and/or component, but it is not excluded that presence or addition to take leave " comprising "Other one or more features, integer, step, operation, element, component and/or their group.
Fig. 1 is the flow chart of the high frequency method for scheduling task in one embodiment of the invention, as shown in Figure 1, a kind of high frequencyMethod for scheduling task includes the following steps：
Step S1, dispatch server calls default timed task frame in the database, is timed task scheduling, fixedWhen task framework be preset with timed task, timed task is configured with task definition, scheduling rule and scheduling information, and scheduling information is depositedStorage is in the database.
Dispatch server in the present embodiment can select Linux server.Linux server is a kind of using LinuxThe network server of system is used for business processing application, such as network and system management, data base administration and Web service, is toolA kind of server of standby high-performance and increasing income property.Linux server has splendid system stability, safety, due to usingMulti-user operating system, therefore the flexibility having had and scalability, no matter processor or machine architecture can well intoReceiving in row resource.Software in Linux server is free, thus Linux server can more save in application and developmentCost.Most of customer service can carry out more free exploitation with linux system to business.
New scheduling engine is introduced in Linux server, i.e., has preset being used for timed task tune in Linux serverThe timed task frame of degree.It, can be fixed by calling the automatic installation kit of configure tools to contain when Linux server is presetWhen task framework python programs, it is automatic that timed task frame is installed.Linux server starts timed task framework functionsBefore, several timed tasks can be increased newly as needed in Linux server, its corresponding is configured for each timed taskBusiness content, scheduling rule and scheduling information.Wherein, task definition is task execution code, and scheduling rule is task triggering mode,Scheduling information is the Scheduling content after task triggering.After having configured timed task, Linux server starts timed task frame,Each timed task can execute corresponding scheduling information according to its scheduling rule by task definition.
When as different such as dispatch server selection, when timed task frame is installed, it need to only pass through the ring of modification operating systemTimed task frame is installed to selected dispatch server and mapping out the work for timed task frame can be completed by border variable.
Timed task frame acquiescence storage be simply scheduling information preserve in memory, when program midway stop orWhen person's system runs quickly routed, the scheduling information of all operations can all lose.For example a certain timed task needs to execute 100 times, if heldRow is to system crash at 50 times, and the execution counter of task will be since 0 when system reboot.Therefore the present embodiment believes schedulingIn the database, after database preserves scheduling information, even if being restarted after system crash, the scheduling of timed task is believed for breath storageBreath can still be restored, and in previous example, task needs to execute 100 times, if heavy after system crash when going to 50 timesNew operation, counter will be started counting up from 51, and the present embodiment has used database preservation scheduling information to realize scheduling information and heldLongization.
Preserve scheduling information database can select relevant database PostgreSQL, PostgreSQL be one fromBy Object-relational Database server (data base management system), PostgreSQL provides other opposite open source codesDatabase Systems (such as MySQL and Firebird) and proprietary system (such as Oracle, Sybase and Microsoft SQLServer another selection except).PostgreSQL can be used for C/S (client/server) environment, PostgreSQL installationsCore is database service end process, need to access storage data in the database application program must by database intoJourney, such client-side program can not directly access data.PostgreSQL support space querying, support all size text,Support server side scripts, included full-text search function.The scheduling information storages of PostgreSQL in this present embodiment, after beingIt is convenient that continuous modification scheduling rule and scheduling information and SQL query provide.
Non-relational database MongoDB can be selected by preserving the database of scheduling information.MongoDB can quick, peaceComplete and automation realization node or data center's failure transfer.Although MongoDB does not support affairs, its insertion speed is non-Often fast, in view of the weak data structure pattern of MongoDB, 1 newer field of addition will not have old table any influence, therefore entireProcess can very quickly.When application program changes, does not need special database administrator yet and remove modification database mouldFormula.In addition, MongoDB is suitble to the persistence of class, class that can be serialized into JSON and be stored in MongoDB.Therefore, in this realityIt applies in example, when timed task needs to load the business datum of a large amount of low values, for example when log collection, selects MongoDB that can obtainTo efficient storage speed.
Memory database Redis can also be selected by preserving the database of scheduling information.Redis is a use increased incomeANSI C languages write, support network, can based on memory also can persistence log type, Key-Value databases, and provide moreThe API of kind language.It supports numerous value storage classes, including string (character string), list (chained list), set (collectionClose), zset (sorted set-- ordered sets) and hash (Hash type).Redis supports different unordered, orderly lists,The advanced server end atomic operation such as intersection, union between unordered, orderly set.Redis also supports that master-slave synchronisation, data canCan be associated with other main services from server from server to be synchronized from server from master server to any number ofDevice, this makes Redis can perform single layer tree copying.Timed task in the present embodiment is when for example storage class is complex, selectionRedis can obtain preferably storing operation.
Step S2, dispatch server configuration schedules routine interface, by dispatcher interface to appointing in timed task frameThe scheduling rule and scheduling information of one timed task are modified, and dispatcher interface is encapsulated as api interface and is placed onIn django engineerings.
In order to realize dynamic modification task scheduling rule and scheduling information, appointing in a manner of support timing, time interval etc.Business scheduling, the present embodiment are additionally arranged dispatcher interface, complete the modification to scheduling rule and scheduling information by the interface, such asThe primary task of scheduling in original 3 minutes, can be modified as 5 minutes once by dispatcher interface.
Api interface is a kind of open interface, and foreground can realize inquiry, the additions and deletions to database by calling api interfaceChange, reduce the coupling of intermodule, improve the flexibility of database, changed convenient for system upgrade, therefore for the ease of modification, adjustsDegree routine interface is encapsulated as api interface.
Django is the Web application frameworks of an open source code, and the api interface of encapsulation is placed in django engineeringsAfter calling, the opening of api interface is realized, when needing to change scheduling rule or scheduling information, is by Web application callsIt can complete to change, change and its conveniently.
Step S3, dispatch server configuration monitoring device, monitor is according to scheduled monitor mode to being stored with scheduling informationDatabase carry out SQL query, obtain monitored results after notified.
Using developed application supervision platform and third party's monitoring tools, wherein third party supervises monitor in this stepControl tool can be configured by web interface, realize monitoring purpose.For example, SQL may be used in the monitor in this stepServer database monitoring softwares are realized the real time monitoring to data in database and are divided in order to use SQL query databaseAnalysis.Monitor mode may be used setting polling time interval and be monitored.Time interval can be as needed, set with hour asUnit is monitored for one day for unit.
SQL is a kind of abbreviation to structured query language, is a kind of data base querying and programming language, for depositingAccess according to this and inquiry, update and administrative relationships Database Systems, while be also database script file extension name.It allowsUser works in high level data structure, does not require the specified deposit method to data of user, does not also need user and understands specificallyData location mode, so with entirely different fabric disparate databases system, identical structuring can be usedQuery language inputs the interface with management as data.Structured query language sentence can be nested, and it is great that this makes it haveFlexibility and powerful function.The present embodiment monitors database in real time using SQL query, can find and change to appoint in timeThe exception occurred in business scheduling.
The present embodiment realizes higher frequency task scheduling, database is written in scheduling information by introducing new scheduling engineMode, realize scheduling information persistence, by api interface realize dynamic change scheduling rule and scheduling information work(Can, it introduces and monitors it, can find that scheduling is abnormal and carries out time update by api interface in time.
In one embodiment, step S1 includes default timed task frame：
By calling configure tools to install python programs automatically, timed task frame is included in dispatch serverIn python programs, i.e. the setting of completion timing task framework.
Python (computer programming language) is a kind of explanation type computer programming language of object-oriented.Python grammer simple and clears have abundant and powerful library, can be various module (the especially C/ made of other languageC++ it) is easily bound up very much.Its coding of timed task frame based on python programs is more succinct, can it is convenient withThe web frames such as django, which combine, carries out restful API exploitations, is combined more closely with existing dispatching platform function.
Timed task frame can select Quartz modules or apscheduler modules, preferably apscheduler modules.A Python timed task frame of the apscheduler modules based on Quartz, the institute for realizing Quartz is functional, and makesWith conveniently.Apscheduler provides the calling rule based on date, Fixed Time Interval and crontab types, and canWith persistence task.Based on these functions, using apscheduler modules as new scheduling engine, the timed task of realizationMeets the needs of high frequency task.
By taking apscheduler modules are arranged in Linux dispatch servers as an example, apscheduler modules are python packets,The deployment of python packets takes the strategy being integrally packaged with python files to be configured.And configure kits contain greatlyThe command-line option of amount, dispatch server call configure tools that can realize Python by changing command-line optionThe automatic installation of program, automatically according to after the completion, you can use apscheduler modules.
The present embodiment disposes apscheduler modules as timed task frame in such a way that python files are integrally packagedFrame, functions of modules is abundant, easy to use, can be compatible with existing dispatching platform well.
In one embodiment, as shown in Fig. 2, step S1 further includes newly-increased timed task：
Step S102A：Instantiate a scheduler class；
Step S102B：Add task definition and scheduling information；
Step S102C：Calling rule is configured, and triggers task in a manner of calling.
Such as newly-increased timed task is：
It indicates to execute a my_job function, output hello world every 5s.
In one embodiment, in step S102B, also in the database by scheduling information storage：In dispatch serverIn be preset with ORM tools, dispatch server is written by calling ORM tools to read scheduling information in the form of scheduled digitizationIn database, that is, complete the storage of scheduling information.
ORM tools are the abbreviations of Object Relation Mapping, for realizing different type system in Object-Oriented Programming LanguageConversion between data, the method that it provides conceptual, understandable modeled data.ORM toolsThe SQLAlchemy module shields grammatical differences of underlying database, only need the link information of configuration database can to write dataEnter database.ORM tools and SQLAlchemy modules can support relevant database and non-relational database and memory numberAccording to library.
The present embodiment is written in database scheduling information by the SQLAlchemy modules of ORM tools in real time, ORM toolsCompatible all kinds of existing databases can preferably be applied in existing dispatching platform, realize the function of scheduling information persistence.
Specifically, only needing the link information of configuration database using ORM tools, you can realize the write-in work of information, ORMThe relevant database that the SQLAlchemy of tool is supported, when database uses relevant database postgresql, an allusion quotationThe configuration of type is as follows：
When using non-relational database MongoDB and memory database Redis such as database, ablation process is and above-mentionedRelevant database PostgreSQL is similar.
In one embodiment, also include scheduler modules in step S2, in python programs, dispatch server willThe reschedule_job interfaces of scheduler modules are completed to scheduling rule and scheduling information as dispatcher interfaceModification.Reschedule_job interfaces, which are placed on, to be encapsulated as api interface and is placed in django engineerings.
Specifically, such as the primary task of scheduling in original 3 minutes is modified as scheduling in 5 minutes once, may be used such as lower sectionFormula：
Scheduler.reschedule_job (' my_job_id', trigger='cron', minute='*/5')
In one embodiment, scheduling rule includes scheduling mode, and scheduling mode includes blocking model or asynchronous mode, is adjustedSpend modification of the server by dispatcher interface completion to scheduling mode.
By api interface when modifying, in addition to timing, time interval, crontab expression formula modes scheduling intoRow modification is outer, can also modify to scheduling mode, such as selects blocking model or asynchronous mode.
Under blocking model, scheduling meeting next time waits for the end of last scheduling.Specifically, such as the frequency of some schedulingIt is primary for 3 minutes, if it has been more than 3 minutes that wherein certain primary this, which dispatches corresponding Runtime, for blocking model, underIt is primary call can wait for this time task run be over and just start to dispatch.
It is then finished in asynchronous mode without waiting for the last time, carries out asynchronous schedule, cover in practical application oftenThe scene seen.Specifically, if the frequency of some scheduling is 3 minutes primary, if wherein certain primary this dispatches corresponding task fortuneThe row time has been more than 3 minutes, for asynchronous mode, to after 3 minutes, starts to reschedule the task but regardless of last taskOperating condition how.
In one embodiment, in step S3, monitor mode is monitored using setting polling time interval.
Developed application supervision platform and third party's monitoring tools, wherein third party's monitoring tools may be used in monitorIt can be configured by web interface, realize monitoring purpose.The monitor mode of monitor is carried out using setting polling time intervalWhen monitoring, according to time interval, SQL query periodically is carried out to the database for being stored with scheduling information.
In one embodiment, in step S3, monitored results are the exception information in database, including when scheduler taskError information, time-out information, monitored results are notified by way of mail or phone.
After monitor carries out SQL query by polling time interval, the result of inquiry is real-time scheduling information, and monitoring is normalHave in seeing, in task scheduling with the presence or absence of the situation that reports an error, whether scheduler task run time is also not complete more than a certain threshold valueAt etc. abnormal conditions.Monitor is exactly that whether will produce these abnormal conditions in monitoring data library, when there are abnormal conditions,Monitored results are notified into related personnel by modes such as mails or phone, related personnel has found simultaneously in time according to monitored resultsChange the problems in task scheduling.
In one embodiment it is proposed that a kind of high frequency task scheduling system, as shown in figure 3, high frequency task scheduling systemIncluding：
Timed task dispensing unit calls default timed task frame in the database for dispatch server, carries outTimed task is dispatched, and timed task frame is preset with timed task, and timed task is configured with task definition, scheduling rule and schedulingInformation, scheduling information store in the database；
Interface dispensing unit is used for dispatch server configuration schedules routine interface, is appointed to timing by dispatcher interfaceThe scheduling rule and scheduling information of any timed task are modified in business frame, and dispatcher interface is encapsulated as api interfaceIt is placed in django engineerings；
Monitoring unit, is used for dispatch server configuration monitoring device, and monitor is adjusted according to scheduled monitor mode to being stored withThe database for spending information carries out SQL query, is notified after obtaining monitored results.
In one embodiment, for timed task dispensing unit in default timed task frame, dispatch server passes through tunePython programs are installed automatically with configure tools, timed task frame is included in python programs, i.e., completion timing is appointedThe deployment of business frame.
In one embodiment, timed task dispensing unit presets ORM when scheduling information is stored in dispatch serverTool, dispatch server are written in the form of scheduled digitization in database, i.e., by calling ORM tools to read scheduling informationComplete the storage of scheduling information.
In one embodiment, monitor mode is monitored using setting polling time interval.
In one embodiment, monitored results are the exception information in database, including error information when scheduler task,Time-out information, monitored results are notified by way of mail or phone.
In one embodiment it is proposed that a kind of computer equipment, including database and processor, it is stored in databaseComputer-readable instruction, when computer-readable instruction is executed by processor so that processor executes real when computer-readable instructionExisting following steps：The timed task frame dispatched for timed task is preset in dispatch server, presets timed task frameWhen, for dispatch server by calling configure tools to install python programs automatically, timed task frame is included in pythonIn program.Timed task frame is preset with timed task, and timed task is configured with task definition, scheduling rule and scheduling information,Scheduling information stores in the database, when scheduling information stores, ORM tools is preset in dispatch server, dispatch server is logicalIt crosses calling ORM tools and reads scheduling information, be written in database in the form of scheduled digitization.Dispatch server preset schedule journeySequence interface repaiies the scheduling rule and scheduling information of any timed task in timed task frame by dispatcher interfaceChange, dispatcher interface is encapsulated as api interface and is placed in django engineerings.Dispatch server configuration monitoring device, monitorSQL query is carried out to the database for being stored with scheduling information according to scheduled polling time interval, postal is crossed after obtaining monitored resultsThe mode of part or phone is notified.
In one embodiment it is proposed that a kind of storage medium being stored with computer-readable instruction, computer-readable fingerWhen order is executed by one or more processors so that one or more processors execute following steps：It is pre- in dispatch serverIf for the timed task frame of timed task scheduling, when presetting timed task frame, dispatch server passes through callingConfigure tools install python programs automatically, and timed task frame is included in python programs.Timed task frame is pre-Equipped with timed task, timed task is configured with task definition, scheduling rule and scheduling information, and scheduling information is stored in databaseIn, when scheduling information stores, ORM tools are preset in dispatch server, dispatch server is by calling ORM tools to read schedulingInformation is written in the form of scheduled digitization in database.Dispatch server preset schedule routine interface, is connect by scheduler programMouth modifies to the scheduling rule and scheduling information of any timed task in timed task frame, and dispatcher interface is packedIt is placed in django engineerings for api interface.Dispatch server configuration monitoring device, monitor is according to scheduled polling time intervalDatabase to being stored with scheduling information carries out SQL query, crosses mail after obtaining monitored results or the mode of phone is notified.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is canIt is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storageMedium may include：Read-only data library (ROM, Read Only Memory), Random Access Data library (RAM, RandomAccess Memory), disk or CD etc..
Each technical characteristic of embodiment described above can be combined arbitrarily, to keep description succinct, not to above-mentioned realityIt applies all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not depositedIn contradiction, it is all considered to be the range of this specification record.
Some exemplary embodiments of the invention above described embodiment only expresses, the description thereof is more specific and detailed, butIt cannot be construed as a limitation to the scope of the present invention.It should be pointed out that for the ordinary skill people of this fieldFor member, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the present invention'sProtection domain.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.