[detailed description of the invention]
For making the purpose of the embodiment of the present application, technical scheme and advantage clearer, below in conjunction with this ShenPlease accompanying drawing in embodiment, the technical scheme in the embodiment of the present application is clearly and completely described,Obviously, described embodiment is some embodiments of the present application rather than whole embodiments.Based onEmbodiment in the application, those of ordinary skill in the art are obtained under not making creative work premiseThe every other embodiment obtained, broadly falls into the scope of the application protection.
Before introducing the application method flow, first simply introduce the framework of the application operation system.As it is shown in figure 1, a kind of framework of operation system includes at least one headend equipment 10, proxy server20 and at least one application server 30.Headend equipment 10 is by proxy server 20 and application serviceDevice 30 is set up and is connected.Headend equipment 10 can be PC, desktop computer, panel computer or mobile phoneDeng.Proxy server 20 can be realized by the equipment arbitrarily with agent functionality.
In the another kind of framework of operation system, as in figure 2 it is shown, headend equipment 10 and application server30 do not set up connection by proxy server 20, but directly set up connection.It is to say, in businessSystem realize in framework, proxy server 20 is an optional equipment.
What deserves to be explained is, a headend equipment 10 can be set up with one or more application server 30Connecting, be only a kind of example shown in Fig. 2, the connection between headend equipment 10 and application server 30 is closedSystem is not limited to shown in Fig. 2.
Wherein, client is typically deployed on front end end equipment 10, and service end is deployed in application serverOn 30.Client matches with service end and realizes technical scheme.
The schematic flow sheet of the data processing method that Fig. 3 provides for the application one embodiment.As it is shown on figure 3,The method includes:
301, service end is after receiving the data processing request that client sends, and returns number to clientAccording to processing response, these data process response and are used for indicating service end can perform number in asynchronous task scheduling modeThe data processing task corresponding according to processing request.
302, service end is after receiving the result queries request that client sends, and returns number to clientAccording to the implementation progress processing data processing task corresponding to request.
In the present embodiment, when client needs service end to carry out data process, number is sent to service endAccording to processing request；Service end receives the data processing request that client sends.
Unlike the prior art, service end is after receiving the data processing request of client, notAs prior art, proceed by data according to data processing request process, but return number to clientAccording to processing response, process response by these data and tell client: service end can be with asynchronous task scheduling sideFormula performs the data processing task that this data processing request is corresponding.Client receives the data that service end returnsProcess response, process response according to these data and know that service end has received data processing request.So far,Being once connected between client with service end just finishes.Afterwards, service end is with asynchronous task scheduling sideFormula performs the data processing task that data processing request is corresponding.
From above-mentioned, in the present embodiment, if the connection retention time between service end and clientEnsure that reception data processing request and return data process and respond, the retention time is shorter, quiteIn the connection resource having saved service end；It addition, during service end asynchronous execution data processing task,Connection between client and service end is released, and during this period, service end can use and be releasedThe connection resource bled off is set up with other client and is connected, and is conducive to improving the utilization rate of connection resource.
It addition, the most complete data handling procedure should include that client obtains data processed result.ExamineConsidering to this demand, service end provides query function to client, it is allowed to client query data process appointsThe implementation progress of business, in order to obtain and perform result.
Based on this, client is after submitting data processing request to service end, it is also possible to send out to service endResult queries is sent to ask, to inquire about the implementation progress of data processing task.Service end receives client and sendsResult queries request, according to the implementation progress of result queries requesting query data processing task, and will holdTraveling degree returns to client.
Optionally, determining for the ease of client and need the data processing task of inquiry, service end is by relevantThe information of data processing task is sent to client.The information of data processing task includes: data process appointsThe mark (ID) of business, the file name creating time, type, pending data place etc..Optional, the information of data processing task can be carried in data process response by service end, at dataReason response returns to client.
Based on above-mentioned, client can carry the information of data processing task, example in result queries is askedAs being the ID of data processing task, establishment time, type etc., in order to service end determines clientEnd to inquire about the implementation progress of which data processing task.
Illustrate at this, client can after the time to be specified such as submitting after data processing request to, then toService end sends result queries request.This appointment time enough service end has performed data processing task.ThisSample, client only need to send a result queries to service end and ask to obtain finally to perform result.OrPerson, this appointment time is shorter, and service end is not enough to perform data processing task within this appointment time,Based on this, service end returns to the implementation progress of client and does not include finally performing result, therefore, and visitorFamily end can send result queries request at least twice to service end, in order to obtains and finally performs result.
Wherein, each result queries is required for client and is connected with service end foundation, and returns in service endAfter implementation progress, discharge this connection.Connection during result queries, between client and service endAs long as the retention time ensure that reception result queries is asked and returns implementation progress, this retention timeShorter, be equivalent to save connection resource.
What deserves to be explained is, above-mentioned data processing request can be that request carries out asking of various data processAsk.Optionally, this data processing request can ask that some data volumes are relatively big, service logic is more complicatedData process business, such as, can be that batch data processes request.Concrete, can be that batch uploads numberAccording to request, the request of bulk-download data, batch check data request or batch correct data pleaseAsk.
According to the difference of application scenarios, the data processing function that service end is realized can be different.Such as,Uploading in data scene at batch, service end is mainly according to the data processing request of client, to uploadData carry out resolving and storing.The most such as, in bulk-download data scene, the main root of service endAccording to the data processing request of client, determine data resource to be downloaded, from Resource Server, download instituteState data resource to be downloaded and return to client.
From above-mentioned, in the present embodiment, service end after receiving the data processing request of client,Return data to client and process response, be used for indicating service end can perform this in asynchronous task scheduling modeThe data processing task that data processing request is corresponding, the connection between such client and service end can be brokenOpening, the connection retention time between client and service end is shorter, and in service end with asynchronous task schedulingDuring mode processes response data process task, client is released with the connection of service end, hasIt is beneficial to save the connection resource of service end；Further, during this period, service end can also be by being releasedThe connection resource fallen is set up with other client and is connected, and is conducive to improving the utilization rate of connection resource；AgainPerson, service end provides query function to client, it is allowed to client sends result queries request and obtains correspondingThe implementation progress of data processing task, it is ensured that the integrity of data handling procedure.
What deserves to be explained is, although from the point of view of a data handling procedure, current embodiment require that client withService end is set up to connect at least twice and (is sent data processing request and result queries request is respectively necessary for setting upConnect), but shorter owing to connecting the retention time every time, may be appreciated for a person skilled in the artIt is: the retention time sum of all connections in the present embodiment is also than in prior art data handling procedureConnection Time short, the most still there is the advantage of connection resource saving service end.
For service end, after returning data process response to client, in addition it is also necessary to asynchronous taskScheduling mode performs the data processing task that data processing request is corresponding.
In an optional embodiment, service end performs data processing request pair in asynchronous task scheduling modeThe flow process of the data processing task answered includes:
Service end, according to the data processing request received, creates data corresponding to data processing request and processesTask, and generate the data directory of pending data；It is being dispatched at the data that data processing request is correspondingDuring reason task, obtain pending data according to data directory, perform the process logic of this data processing task,So that pending data are processed.
The effect of above-mentioned data directory is directed to pending data.According to the difference of application scenarios, above-mentioned numberWould also vary from according to index.Such as, in batch data uploads scene, data directory can be to uploadThe path of file, this document can be found by this path.The most such as, scene is downloaded at batch dataIn, data directory can be download condition, such as, can include downloading the title of resource, URL etc., logicalCross this download condition and may determine that download resource.
Optionally, service end can manage data processing task by asynchronous task list, based on this,Service end specifically can create the data processing task that data processing request is corresponding in asynchronous task list.
Optionally, service end can also pass through index list, the number that corresponding management data processing task is correspondingAccording to index.Based on this, data directory, after generating data directory, can be stored index column by service endIn table.
What deserves to be explained is, between index list and asynchronous task list, there is corresponding relation, in order to willData processing task is mapped with data directory, such as, by same task ID, or can store positionPut etc. and both are mapped.
For service end, can be according to the data in certain scheduling strategy dispatching asynchronous task list atReason task.For example, it is possible to according to the establishment time of data processing task, or can process according to dataData processing task in the dispatching asynchronous task lists such as the priority of task.
When being dispatched to data processing task corresponding to data processing request, perform this data processing taskProcess logic, thus pending data are processed.Such as, service end can start one or moreAsynchronous thread performs the process logic of data processing task.
According to the difference of application scenarios, the process logic of data processing task would also vary from.Such as,Batch uploading in data scene, the process logic of data processing task mainly according to the data of client atThe data uploaded are resolved and store by reason request.The most such as, in bulk-download data sceneIn, the process logic of data processing task, mainly according to the data processing request of client, determines to be downloadedData resource, from Resource Server, download described data resource to be downloaded and return to client.
In actual applications, it may occur that various faults, data handling procedure is caused to be interrupted.In faultAfter recovery, if re-executing data handling procedure, the wasting of resources can be caused.In order to economize on resources, in eventAfter barrier recovers, can be to use in the way of breakpoint succession performs data processing task.But, to breakpointDetermine relatively difficult.For this problem, the present embodiment provides a kind of solution, particularly as follows:
After the above-mentioned data processing task of successful execution, service end record data processing task is in asynchronous taskPositional information in list.
Optionally, service end can be by the data processing task of successful execution position in asynchronous task listConfidence breath storage in external file, restarts the loss of rear internal storage data to prevent from hindering for some reason.
The data processing task of successful execution based on above-mentioned record position letter in asynchronous task listBreath, it may be determined that breakpoint location.So, when service end is after fault recovery, last record can be obtainedPositional information, from the beginning of the positional information obtained, continue the data in dispatching asynchronous task list and processTask, solves aborted problem.
Finally what deserves to be explained is, can pass through between client and service end webpage, mail, short message,The various mode such as instant communication information interacts.It is to say, client can pass through webpage, mail,The mode such as short message or instant communication message sends data processing request and result queries request to service end；Accordingly, service end can be by modes such as hairnet page, mail, short message or instant communication messages to visitorFamily end returns data and processes the implementation progress of response and data processing task.
The schematic flow sheet of the data processing method that Fig. 4 provides for the application one embodiment.These data processMethod can be applicable to batch data upload, batch data download, batch check data, batch correct data etc.Scene carries out batch data process, but is not limited to batch data and processes.As shown in Figure 4, the method includes:
401, data processing request is sent to service end.
402, receiving the data process response that service end returns, these data process response and are used for indicating serviceEnd can perform, in asynchronous task scheduling mode, the data processing task that data processing request is corresponding.
403, result queries request is sent to service end.
404, the execution of the data processing task that the described data processing request of reception service end return is correspondingProgress.
The executive agent of above-mentioned steps 401-step 404 is specially client, can be with above-mentioned steps 301-step302 are implemented in combination in data processes.
Detailed description about the present embodiment method can be found in previous embodiment, does not repeats them here.
In the present embodiment, client matches with service end, supports service end asynchronous process clientRequest, and the implementation progress of result queries requesting query data processing task is sent to service end, so existDuring the request of service end asynchronous process client, the connection between client and service end can be releasedFall, be conducive to saving the connection resource of service end, be conducive to improving the utilization rate of connection resource, Jin ErkeImprove the treatment effeciency of service end.
It should be noted that for aforesaid each method embodiment, in order to be briefly described, therefore by its all tableStating as a series of combination of actions, but those skilled in the art should know, the application is by being retouchedThe restriction of the sequence of movement stated because according to the application, some step can use other orders or withShi Jinhang.Secondly, those skilled in the art also should know, embodiment described in this description all belongs toIn preferred embodiment, necessary to involved action and module not necessarily the application.
In the above-described embodiments, the description to each embodiment all emphasizes particularly on different fields, and does not has in certain embodimentThe part described in detail, may refer to the associated description of other embodiments.
The structural representation of the service end that Fig. 5 provides for the application one embodiment.As it is shown in figure 5, these clothesBusiness end includes: receiver module 51 and sending module 52.
Receiver module 51, for receiving the data processing request that client sends, and receives what client sentResult queries is asked.
Sending module 5, processes response for returning data to client, and data process response and are used for indicatingService end can perform, in asynchronous task scheduling mode, the data processing task that data processing request is corresponding, and toClient returns the implementation progress of data processing task corresponding to data processing request.
Further, as it is shown in figure 5, this service end also includes: task processing module 53.
Task processing module 53, for performing, in asynchronous task scheduling mode, the data that receiver module 51 receivesProcess the data processing task that request is corresponding.
In an optional embodiment, task processing module 53 specifically for:
According to data processing request, create the data processing task that data processing request is corresponding, and generate and wait to locateThe data directory of reason data；
When being dispatched to data processing task, obtain pending data according to data directory, perform data and processThe process logic of task, to process pending data.
In an optional embodiment, as it is shown in figure 5, this service end also includes: logging modle 54.
Logging modle 54, for after task processing module 53 successful execution data processing task, records numberAccording to processing task positional information in asynchronous task list.
In an optional embodiment, as it is shown in figure 5, this service end also includes: fault processing module 55.
Fault processing module 55, for after service end fault recovery, obtains logging modle 54 postscriptThe positional information of record, from the beginning of the positional information obtained, continues at the data in dispatching asynchronous task listReason task.
Optionally, above-mentioned data processing request can be that batch data processes request, but is not limited to this.
The service end that the present embodiment provides, after receiving the data processing request of client, to clientReturn data and process response, be used for indicating service end can perform these data in asynchronous task scheduling mode and processThe data processing task that request is corresponding, the connection between such client and service end can disconnect, clientThe connection retention time between end and service end is shorter, and processes in asynchronous task scheduling mode in service endDuring response data process task, client is released with the connection of service end, is conducive to savingThe connection resource of service end；Further, during this period, the service end that the present embodiment provides can also be passed throughThe connection resource being released is set up with other client and is connected, and is conducive to improving the utilization of connection resourceRate；Furthermore, the service end that the present embodiment provides provides query function to client, it is allowed to client sendsResult queries request obtains corresponding data and processes the implementation progress of task, it is ensured that data handling procedureIntegrity.
What deserves to be explained is, although from the point of view of a data handling procedure, the service end that the present embodiment providesNeed to set up with client to be connected at least twice and (send data processing request and result queries request is respectively required forConnection to be set up), but shorter owing to connecting the retention time every time, the most permissibleBe understood by: the retention time sum of all connections in the present embodiment also than a secondary data in prior art atThe Connection Time of reason process is short, the most still has the advantage of the connection resource saving service end.
The structural representation of the client that Fig. 6 provides for the application one embodiment.As shown in Figure 6, this visitorFamily end includes: sending module 61 and receiver module 62.
Sending module 61, for sending data processing request to service end, and sends result to described service endInquiry request.
Receiver module 62, the data returned for receiving described service end process response, and described data processIt is corresponding that response is used for indicating described service end can perform described data processing request in asynchronous task scheduling modeData processing task, and receive at data corresponding to described data processing request that described service end returnsThe implementation progress of reason task.
Optionally, above-mentioned data processing request can be that batch data processes request, but is not limited to this.
The client that the present embodiment provides, matches with service end, supports service end asynchronous process clientRequest, and to service end send result queries requesting query data processing task implementation progress, soDuring the request of service end asynchronous process client, the present embodiment client can disconnect with service end itBetween connection, be conducive to save service end connection resource, be conducive to improve connection resource utilization rate,And then the treatment effeciency of service end can be improved.
Those skilled in the art is it can be understood that arrive, and for convenience and simplicity of description, above-mentioned retouchesThe specific works process of the system stated, device and unit, is referred to the correspondence in preceding method embodimentProcess, does not repeats them here.
In several embodiments provided herein, it should be understood that disclosed system, device andMethod, can realize by another way.Such as, device embodiment described above is only shownMeaning property, such as, the division of described unit, be only a kind of logic function and divide, actual can when realizingThere to be other dividing mode, the most multiple unit or assembly can in conjunction with or be desirably integrated into anotherSystem, or some features can ignore, or do not perform.Another point, shown or discussed each otherCoupling direct-coupling or communication connection can be the INDIRECT COUPLING by some interfaces, device or unitOr communication connection, can be electrical, machinery or other form.
The described unit illustrated as separating component can be or may not be physically separate, makeesThe parts shown for unit can be or may not be physical location, i.e. may be located at a place,Or can also be distributed on multiple NE.Can select according to the actual needs part therein orThe whole unit of person realizes the purpose of the present embodiment scheme.
It addition, each functional unit in each embodiment of the application can be integrated in a processing unitIn, it is also possible to it is that unit is individually physically present, it is also possible to two or more unit are integrated in oneIn individual unit.Above-mentioned integrated unit both can realize to use the form of hardware, it would however also be possible to employ hardware addsThe form of SFU software functional unit realizes.
The above-mentioned integrated unit realized with the form of SFU software functional unit, can be stored in a computerIn read/write memory medium.Above-mentioned SFU software functional unit is stored in a storage medium, including some fingersMake with so that a computer equipment (can be personal computer, server, or the network equipment etc.)Or processor (processor) performs the part steps of method described in each embodiment of the application.And it is aforementionedStorage medium include: USB flash disk, portable hard drive, read only memory (Read-Only Memory, ROM),Random access memory (Random Access Memory, RAM), magnetic disc or CD etc. are variousThe medium of program code can be stored.
Last it is noted that above example is only in order to illustrate the technical scheme of the application, rather than to itLimit；Although the application being described in detail with reference to previous embodiment, the ordinary skill of this areaPersonnel it is understood that the technical scheme described in foregoing embodiments still can be modified by it, orPerson carries out equivalent to wherein portion of techniques feature；And these amendments or replacement, do not make corresponding skillThe essence of art scheme departs from the spirit and scope of the application each embodiment technical scheme.