Embodiment
The schematic flow sheet of the data processing method based on distributed file system that Fig. 1 provides for one embodiment of the invention, as shown in Figure 1, comprising:
101, server receives the file manipulation command that the first client computer sends.
Wherein, file manipulation command be used to indicate the data of required processing, to the second client computer under the operation of the required execution of data and data; Data comprise at least one in file and file directory; File directory is used to indicate the memory location of file in the second client computer.
The property cache of the data of 102, server lookup file manipulation command indication.
Wherein, property cache is used to indicate the metadata of data; Metadata, comprising: at least one in data volume, last modification time, last access time and creation-time.
Judge whether the pre-stored property cache that has described data, pre-stored described property cache obtains in advance from described the second client computer; If pre-stored, there is described property cache, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed described first threshold the settling time of pre-stored property cache, using described pre-stored property cache as the described property cache inquiring; The settling time of described property cache be from get in advance described property cache time be carved into the time between current time; If do not store in advance the property cache of described data, or exceed described first threshold the settling time of determining described pre-stored property cache, from described the second client computer, obtain the property cache of described data, using the property cache of the described data that get as the described property cache inquiring.
103, server judges that whether the property cache of data corresponding data copy is identical with the property cache inquiring.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition.
If 104 is identical, the operation of server to the indication of data trnascription execute file processing command, generates the first operating result, and sends the first operating result to the first client computer.
Further, the data processing method based on distributed file system that the present embodiment provides also comprises, if different, the second client computer Transmit message processing instruction to file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result; From the second client computer, obtain the second operating result, described the second operating result getting is joined in message queue; Described server is processed described the second operating result in described message queue according to first in first out; During the second operating result in carrying out described message queue, described server judges that the second operating result in described message queue joins the time in described message queue; If the time that described the second operating result joins in described message queue exceeds Second Threshold, described server abandons described the second operating result in described message queue; If the time that described the second operating result joins in described message queue does not exceed described Second Threshold, described server sends described the second operating result to described the first client computer.
Further, before 101, also comprise the connection request sending according to the first client computer, described the first client computer is carried out to identity and Authority Verification, connection request comprises username and password, if be verified, set up the service processes corresponding with the first client computer, the described file manipulation command that described service processes sends for receiving described the first client computer.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The schematic flow sheet of the data processing method based on distributed file system that Fig. 2 provides for another embodiment of the present invention, the data processing method based on distributed file system that the present embodiment provides, can be applicable to the server in distributed file system, wherein, distributed file system, comprise at least two client computer and the server that is positioned at high in the clouds, as shown in Figure 2, comprising:
201, the connection request that server sends according to the first client computer, carries out identity and Authority Verification to the first client computer.
Wherein, connection request comprises username and password.
Server, the first client computer and the second client computer can be used for providing software to serve (Software as a Service), SaaS is a kind of Software Delivery pattern, software supplier's unified management is deployed in the application software in server, user is by above-mentioned client computer access network, thereby the application software in RAS, that is to say, the execution of application software is being positioned on the server in high in the clouds, utilizes the application software in high in the clouds to process user's file.User's file can be stored in above-mentioned client computer, also can be stored in server.When the file of the required processing of user is stored in the second client computer, and it is while utilizing the first client computer RAS, user need to send connection request by the first client computer, to open the file directory in the second client computer by server, know the file under this document catalogue, and then therefrom obtain the file of required processing.
User's state file system (Filesystem in Userapace in server, FUSE) allow to set up a user class file system for each user, FUSE moves to user's state by the run mode of application program from kernel mode, reduced complexity, the workload that has reduced exploitation, has improved operation stability.
If 202 are verified, server is set up the service processes corresponding with the first client computer.
Wherein, the file manipulation command that service processes sends for receiving the first client computer.
203, server receives the file manipulation command that the first client computer sends.
Wherein, file manipulation command is used to indicate data, second client computer to the operation of the required execution of data and storage data of required processing; Data comprise file and file directory; File directory is used to indicate the memory location of file in the second client computer; Operation comprises: open, close, at least one in read operation and write operation.
Server by utilizing service processes receives file manipulation command, the file manipulation command receiving is analyzed, know the first client computer that sends this document processing command, this document processing command is joined in service processes corresponding to this first client computer, this document processing command is carried out to buffer memory, with etc. pending this document processing command.
204, server judges whether the pre-stored property cache that has the data of file manipulation command indication, and does not exceed first threshold the settling time of pre-stored property cache, if so, carries out 205, otherwise carries out 206.
Wherein, pre-stored property cache obtains in advance from the second client computer.The settling time of property cache be from get in advance property cache time be carved into time of current time.
The property cache of server data of inquiry file processing command indication in buffer memory, if inquire, current time is deducted to the moment that gets in advance property cache, the settling time of calculating pre-stored property cache, the settling time of pre-stored property cache and first threshold are judged mutually.The moment that wherein gets in advance property cache can be carried out record for the form of this property cache interpolation timestamp.Wherein, first threshold can be set to the shorter time, effective to guarantee it, for example 3 seconds.
If 205 pre-storedly have property cache, and do not exceed first threshold the settling time of pre-stored property cache, server is determined using pre-stored property cache as the property cache inquiring.
If 206 do not store the property cache of data in advance, or exceed first threshold the settling time of pre-stored property cache, server obtains the property cache of data from the second client computer, determines using the property cache of the data that get as the property cache inquiring.
If exceed first threshold the settling time of pre-stored property cache, illustrate that this pre-stored property cache lost efficacy, abandon this pre-stored property cache in buffer memory, from the second client computer, again obtain the property cache of data, determine using the property cache of the data that get as the property cache inquiring.If do not store in advance the property cache of data, from the second client computer, obtain equally the property cache of data, determine using the property cache of the data that get as the property cache inquiring.Because property cache is used to indicate metadata, and the data volume of metadata is very little, and therefore, the impact that is subject to network state is very little, has avoided because good the do not brought server performance of network state declines.
207, server judges that whether the property cache of data corresponding data copy is identical with the property cache inquiring, if identical, carries out 208, if different, carries out 209-211.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition.
The property cache of the property cache inquiring and data corresponding data copy is judged, according to the metadata of property cache indication, know that whether copy in advance the data trnascription of acquisition identical with the data of current storage in the second client computer, whether the data trnascription that copies in advance acquisition is effective.
It should be noted that, server in the present embodiment need to utilize disk or buffer memory to carry out pre-stored to data trnascription, if while there is idle storage space in disk or buffer memory, can directly store, if while there is not idle storage space in disk or buffer memory, need to use, recent minimum use (Least Recently Used, LRU) algorithm is stored after the data trnascription of having stored in disk or buffer memory is replaced.
If 208 is identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, and send the first operating result to the second client computer, so that the second client computer is upgraded the data of storing in the second client computer according to the first operating result.
If 209 is different, server is to the second client computer Transmit message processing instruction of file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result.
210, server obtains the second operating result from the second client computer, and the second operating result getting is joined in message queue.
Server obtains the second operating result from the second client computer, the second operating result getting is joined in message queue, server is processed the second operating result in described message queue according to first in first out (First Input First Output, FIFO) principle.
211,, during second operating result of server in carrying out message queue, to the first client computer, send the second operating result in message queue.
During the second operating result in carrying out described message queue, described server judges that the second operating result in described message queue joins the time in described message queue; If the time that described the second operating result joins in described message queue exceeds Second Threshold, described server abandons described the second operating result in described message queue, thereby has reduced the propagation delay time of data; If the time that described the second operating result joins in described message queue does not exceed described Second Threshold, described server sends described the second operating result to described the first client computer.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The structural representation of the server that Fig. 3 provides for one embodiment of the invention, as shown in Figure 3, comprising:receiver module 31,enquiry module 32,judge module 33 and thefirst sending module 34.
Receiver module 31, the file manipulation command sending for receiving the first client computer.
Wherein, file manipulation command be used to indicate the data of required processing, to the second client computer under the operation of the required execution of data and data; Data comprise file and file directory; File directory is used to indicate the memory location of file in the second client computer.
Enquiry module 32, is connected withreceiver module 31, the property cache of the data of indicating for inquiry file processing command.
Wherein, property cache is used to indicate the metadata of data.Metadata, comprising: at least one in data volume, last modification time, last access time and creation-time.
Whetherjudge module 33 is identical with the property cache inquiring for judging the property cache of data corresponding data copy.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition;
Thefirst sending module 34, if be identical for the result of judge module judgement, the operation to the indication of data trnascription execute file processing command, generates the first operating result, and sends the first operating result to the first client computer.
Wherein, operation, comprising: open, close, at least one in read operation and write operation.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The structural representation of the server that Fig. 4 provides for another embodiment of the present invention, as shown in Figure 4, on the basis of a upper embodiment, theenquiry module 32 in the present embodiment further comprises:
Judgingunit 321, for judging whether the pre-stored property cache that has data.
Wherein, pre-stored property cache obtains in advance from the second client computer.
The first determiningunit 322, be connected with judgingunit 321, if be used for the pre-stored property cache that has, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed first threshold the settling time of pre-stored property cache, using pre-stored property cache as the property cache inquiring.
Wherein, the settling time of property cache be from get in advance property cache time be carved into the time between current time.
The second determiningunit 323, be connected with the first determiningunit 322 with judgingunit 321 respectively, if for not storing in advance the property cache of data, or exceed first threshold the settling time of determining pre-stored property cache, from the second client computer, obtain the property cache of data, using the property cache of the data that get as the property cache inquiring.
Further, the server that the present embodiment provides, also comprises: thesecond sending module 35, identity andAuthority Verification module 36 and set upmodule 37.
Thesecond sending module 35, be connected withjudge module 33, if the result forjudge module 33 judgements is different, the second client computer Transmit message processing instruction to file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result; From the second client computer, obtain the second operating result, the second operating result getting is joined in message queue; Real-Time Monitoring the second operating result joins the time in message queue, when the second operating result joins time in message queue while exceeding Second Threshold, abandons the second operating result in message queue; During the second operating result in carrying out message queue, to the first client computer, send the second operating result in message queue.
Identity andAuthority Verification module 36, the connection request for sending according to the first client computer, carries out identity and Authority Verification to the first client computer; Connection request comprises username and password;
Set upmodule 37, be connected withreceiver module 31 withAuthority Verification module 36 with identity respectively, if for being verified, set up the service processes corresponding with the first client computer, the file manipulation command that service processes sends for receiving the first client computer.
Based on this, receiver module is specifically for utilizing described service processes to receive the described file manipulation command that described the first client computer sends.
Server in the present embodiment can be the server in distributed file system, and this distributed file system comprises at least two client computer and the server that is positioned at high in the clouds.Server, the first client computer and the second client computer can be used for providing software to serve (Software as a Service), SaaS is a kind of Software Delivery pattern, software supplier's unified management is deployed in the application software in server, user is by above-mentioned client computer access network, thereby the application software in RAS, that is to say, the execution of application software is being positioned on the server in high in the clouds, utilizes the application software in high in the clouds to process user's file.User's file can be stored in above-mentioned client computer, also can be stored in server.When the file of the required processing of user is stored in the second client computer, and it is while utilizing the first client computer RAS, user need to send connection request by the first client computer, to open the file directory in the second client computer by server, know the file under this document catalogue, and then therefrom obtain the file of required processing.
In the present embodiment, by reception, be used to indicate the file manipulation command of the data of required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each embodiment of the method can complete by the relevant hardware of program command.Aforesaid program can be stored in a computer read/write memory medium.This program, when carrying out, is carried out the step that comprises above-mentioned each embodiment of the method; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CDs.
Finally it should be noted that: each embodiment, only in order to technical scheme of the present invention to be described, is not intended to limit above; Although the present invention is had been described in detail with reference to aforementioned each embodiment, those of ordinary skill in the art is to be understood that: its technical scheme that still can record aforementioned each embodiment is modified, or some or all of technical characterictic is wherein equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.