Movatterモバイル変換


[0]ホーム

URL:


CN103595761A - Data processing method based on distributed file system and server - Google Patents

Data processing method based on distributed file system and server
Download PDF

Info

Publication number
CN103595761A
CN103595761ACN201310482041.4ACN201310482041ACN103595761ACN 103595761 ACN103595761 ACN 103595761ACN 201310482041 ACN201310482041 ACN 201310482041ACN 103595761 ACN103595761 ACN 103595761A
Authority
CN
China
Prior art keywords
data
client computer
property cache
server
file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310482041.4A
Other languages
Chinese (zh)
Other versions
CN103595761B (en
Inventor
沃天宇
康俊彬
王聿达
张本龙
马晓航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang UniversityfiledCriticalBeihang University
Priority to CN201310482041.4ApriorityCriticalpatent/CN103595761B/en
Publication of CN103595761ApublicationCriticalpatent/CN103595761A/en
Application grantedgrantedCritical
Publication of CN103595761BpublicationCriticalpatent/CN103595761B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Images

Landscapes

Abstract

Translated fromChinese

本发明提供一种基于分布式文件系统的数据处理方法和服务器,通过接收用于指示所需处理的数据的文件处理命令,查询文件处理命令指示的数据的属性缓存,判断数据对应数据副本的属性缓存与查询到的属性缓存是否相同,若相同,对所述数据副本执行所述文件处理命令指示的所述操作,生成第一操作结果,并向所述第一客户机发送所述第一操作结果,由于预先存储数据的属性缓存,以及预先存储数据对应的数据副本,减少了第一客户机利用网络对服务器进行访问,再通过服务器与第二客户机之间的网络对第二客户机中的包括文件或目录在内的数据进行处理的次数,从而解决在分布式文件系统中,数据处理的性能受到网络状况的影响较大的技术问题。

Figure 201310482041

The present invention provides a data processing method and server based on a distributed file system, by receiving a file processing command indicating the data to be processed, querying the attribute cache of the data indicated by the file processing command, and judging the attribute of the corresponding data copy of the data Whether the cache is the same as the queried attribute cache, and if they are the same, perform the operation indicated by the file processing command on the data copy, generate a first operation result, and send the first operation to the first client As a result, due to the attribute cache of the pre-stored data and the data copy corresponding to the pre-stored data, it is reduced that the first client computer accesses the server through the network, and then accesses the second client computer through the network between the server and the second client computer. The number of times the data including files or directories is processed, so as to solve the technical problem that in the distributed file system, the performance of data processing is greatly affected by the network conditions.

Figure 201310482041

Description

Data processing method based on distributed file system and server
Technical field
The present invention relates to the communication technology, relate in particular to a kind of data processing method and server based on distributed file system.
Background technology
Distributed file system (Distributed File System), comprises server and at least two client computer.Servers installed, in high in the clouds, is connected with each client computer by network, between each client computer, also can mutually conduct interviews by this network.In distributed file system, the second client computer is delivered the file directory that allows the first client access, the memory location of the file that this document catalogue is indicated is positioned on the second client computer place physical node, thereby the first client computer can comprise to the file under this document catalogue the processing of read operation and write operation and the processing that this document catalogue is opened and closed by server.
In prior art, because the first client computer is utilized network server is conducted interviews, by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer are processed again, and often the data volume of file is larger, therefore, in existing distributed file system, the impact that the performance of data processing is subject to network condition is larger.
Summary of the invention
The invention provides a kind of data processing method and server based on distributed file system, for solving in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
One aspect of the present invention is to provide a kind of data processing method based on distributed file system, comprising:
Server receives the file manipulation command that the first client computer sends; Described file manipulation command is used to indicate the data of required processing, to the second client computer under the operation of the required execution of described data and described data; Described data comprise at least one in file and file directory; Described file directory is used to indicate the memory location of described file in described the second client computer;
The property cache of the described data of file manipulation command indication described in described server lookup; Described property cache is used to indicate the metadata of described data;
Described server judges that whether the property cache of described data corresponding data copy is identical with the property cache inquiring; Described data trnascription is in advance data described in described the second client computer to be copied to acquisition;
If the property cache of described data corresponding data copy with described in the property cache that inquires identical, described server is carried out the described operation of described file manipulation command indication to described data trnascription, generate the first operating result, to described the first client computer, send described the first operating result.
Another aspect of the present invention is to provide a kind of server, is arranged in distributed file system, comprising:
Receiver module, the file manipulation command sending for receiving the first client computer; Described file manipulation command is used to indicate the data of required processing, to the second client computer under the operation of the required execution of described data and described data; Described data comprise at least one in file and file directory; Described file directory is used to indicate the memory location of described file in described the second client computer;
Enquiry module, for inquiring about the property cache of the described data of described file manipulation command indication; Described property cache is used to indicate the metadata of described data;
Whether judge module is identical with the property cache inquiring for judging the property cache of described data corresponding data copy; Described data trnascription is in advance data described in described the second client computer to be copied to acquisition;
The first sending module, if be identical for the result of described judge module judgement, carries out the described operation of described file manipulation command indication to described data trnascription, generate the first operating result, to described the first client computer, sends described the first operating result.
Data processing method and server based on distributed file system provided by the invention, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, described data trnascription is carried out to the described operation of described file manipulation command indication, generate the first operating result, and send described the first operating result to described the first client computer, due to pre-stored property cache and the data trnascription corresponding to data of data, can directly on server, to data, process, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
Accompanying drawing explanation
The schematic flow sheet of the data processing method based on distributed file system that Fig. 1 provides for one embodiment of the invention;
The schematic flow sheet of the data processing method based on distributed file system that Fig. 2 provides for another embodiment of the present invention;
The structural representation of the server that Fig. 3 provides for one embodiment of the invention;
The structural representation of the server that Fig. 4 provides for another embodiment of the present invention.
Embodiment
The schematic flow sheet of the data processing method based on distributed file system that Fig. 1 provides for one embodiment of the invention, as shown in Figure 1, comprising:
101, server receives the file manipulation command that the first client computer sends.
Wherein, file manipulation command be used to indicate the data of required processing, to the second client computer under the operation of the required execution of data and data; Data comprise at least one in file and file directory; File directory is used to indicate the memory location of file in the second client computer.
The property cache of the data of 102, server lookup file manipulation command indication.
Wherein, property cache is used to indicate the metadata of data; Metadata, comprising: at least one in data volume, last modification time, last access time and creation-time.
Judge whether the pre-stored property cache that has described data, pre-stored described property cache obtains in advance from described the second client computer; If pre-stored, there is described property cache, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed described first threshold the settling time of pre-stored property cache, using described pre-stored property cache as the described property cache inquiring; The settling time of described property cache be from get in advance described property cache time be carved into the time between current time; If do not store in advance the property cache of described data, or exceed described first threshold the settling time of determining described pre-stored property cache, from described the second client computer, obtain the property cache of described data, using the property cache of the described data that get as the described property cache inquiring.
103, server judges that whether the property cache of data corresponding data copy is identical with the property cache inquiring.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition.
If 104 is identical, the operation of server to the indication of data trnascription execute file processing command, generates the first operating result, and sends the first operating result to the first client computer.
Further, the data processing method based on distributed file system that the present embodiment provides also comprises, if different, the second client computer Transmit message processing instruction to file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result; From the second client computer, obtain the second operating result, described the second operating result getting is joined in message queue; Described server is processed described the second operating result in described message queue according to first in first out; During the second operating result in carrying out described message queue, described server judges that the second operating result in described message queue joins the time in described message queue; If the time that described the second operating result joins in described message queue exceeds Second Threshold, described server abandons described the second operating result in described message queue; If the time that described the second operating result joins in described message queue does not exceed described Second Threshold, described server sends described the second operating result to described the first client computer.
Further, before 101, also comprise the connection request sending according to the first client computer, described the first client computer is carried out to identity and Authority Verification, connection request comprises username and password, if be verified, set up the service processes corresponding with the first client computer, the described file manipulation command that described service processes sends for receiving described the first client computer.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The schematic flow sheet of the data processing method based on distributed file system that Fig. 2 provides for another embodiment of the present invention, the data processing method based on distributed file system that the present embodiment provides, can be applicable to the server in distributed file system, wherein, distributed file system, comprise at least two client computer and the server that is positioned at high in the clouds, as shown in Figure 2, comprising:
201, the connection request that server sends according to the first client computer, carries out identity and Authority Verification to the first client computer.
Wherein, connection request comprises username and password.
Server, the first client computer and the second client computer can be used for providing software to serve (Software as a Service), SaaS is a kind of Software Delivery pattern, software supplier's unified management is deployed in the application software in server, user is by above-mentioned client computer access network, thereby the application software in RAS, that is to say, the execution of application software is being positioned on the server in high in the clouds, utilizes the application software in high in the clouds to process user's file.User's file can be stored in above-mentioned client computer, also can be stored in server.When the file of the required processing of user is stored in the second client computer, and it is while utilizing the first client computer RAS, user need to send connection request by the first client computer, to open the file directory in the second client computer by server, know the file under this document catalogue, and then therefrom obtain the file of required processing.
User's state file system (Filesystem in Userapace in server, FUSE) allow to set up a user class file system for each user, FUSE moves to user's state by the run mode of application program from kernel mode, reduced complexity, the workload that has reduced exploitation, has improved operation stability.
If 202 are verified, server is set up the service processes corresponding with the first client computer.
Wherein, the file manipulation command that service processes sends for receiving the first client computer.
203, server receives the file manipulation command that the first client computer sends.
Wherein, file manipulation command is used to indicate data, second client computer to the operation of the required execution of data and storage data of required processing; Data comprise file and file directory; File directory is used to indicate the memory location of file in the second client computer; Operation comprises: open, close, at least one in read operation and write operation.
Server by utilizing service processes receives file manipulation command, the file manipulation command receiving is analyzed, know the first client computer that sends this document processing command, this document processing command is joined in service processes corresponding to this first client computer, this document processing command is carried out to buffer memory, with etc. pending this document processing command.
204, server judges whether the pre-stored property cache that has the data of file manipulation command indication, and does not exceed first threshold the settling time of pre-stored property cache, if so, carries out 205, otherwise carries out 206.
Wherein, pre-stored property cache obtains in advance from the second client computer.The settling time of property cache be from get in advance property cache time be carved into time of current time.
The property cache of server data of inquiry file processing command indication in buffer memory, if inquire, current time is deducted to the moment that gets in advance property cache, the settling time of calculating pre-stored property cache, the settling time of pre-stored property cache and first threshold are judged mutually.The moment that wherein gets in advance property cache can be carried out record for the form of this property cache interpolation timestamp.Wherein, first threshold can be set to the shorter time, effective to guarantee it, for example 3 seconds.
If 205 pre-storedly have property cache, and do not exceed first threshold the settling time of pre-stored property cache, server is determined using pre-stored property cache as the property cache inquiring.
If 206 do not store the property cache of data in advance, or exceed first threshold the settling time of pre-stored property cache, server obtains the property cache of data from the second client computer, determines using the property cache of the data that get as the property cache inquiring.
If exceed first threshold the settling time of pre-stored property cache, illustrate that this pre-stored property cache lost efficacy, abandon this pre-stored property cache in buffer memory, from the second client computer, again obtain the property cache of data, determine using the property cache of the data that get as the property cache inquiring.If do not store in advance the property cache of data, from the second client computer, obtain equally the property cache of data, determine using the property cache of the data that get as the property cache inquiring.Because property cache is used to indicate metadata, and the data volume of metadata is very little, and therefore, the impact that is subject to network state is very little, has avoided because good the do not brought server performance of network state declines.
207, server judges that whether the property cache of data corresponding data copy is identical with the property cache inquiring, if identical, carries out 208, if different, carries out 209-211.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition.
The property cache of the property cache inquiring and data corresponding data copy is judged, according to the metadata of property cache indication, know that whether copy in advance the data trnascription of acquisition identical with the data of current storage in the second client computer, whether the data trnascription that copies in advance acquisition is effective.
It should be noted that, server in the present embodiment need to utilize disk or buffer memory to carry out pre-stored to data trnascription, if while there is idle storage space in disk or buffer memory, can directly store, if while there is not idle storage space in disk or buffer memory, need to use, recent minimum use (Least Recently Used, LRU) algorithm is stored after the data trnascription of having stored in disk or buffer memory is replaced.
If 208 is identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, and send the first operating result to the second client computer, so that the second client computer is upgraded the data of storing in the second client computer according to the first operating result.
If 209 is different, server is to the second client computer Transmit message processing instruction of file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result.
210, server obtains the second operating result from the second client computer, and the second operating result getting is joined in message queue.
Server obtains the second operating result from the second client computer, the second operating result getting is joined in message queue, server is processed the second operating result in described message queue according to first in first out (First Input First Output, FIFO) principle.
211,, during second operating result of server in carrying out message queue, to the first client computer, send the second operating result in message queue.
During the second operating result in carrying out described message queue, described server judges that the second operating result in described message queue joins the time in described message queue; If the time that described the second operating result joins in described message queue exceeds Second Threshold, described server abandons described the second operating result in described message queue, thereby has reduced the propagation delay time of data; If the time that described the second operating result joins in described message queue does not exceed described Second Threshold, described server sends described the second operating result to described the first client computer.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The structural representation of the server that Fig. 3 provides for one embodiment of the invention, as shown in Figure 3, comprising:receiver module 31,enquiry module 32,judge module 33 and thefirst sending module 34.
Receiver module 31, the file manipulation command sending for receiving the first client computer.
Wherein, file manipulation command be used to indicate the data of required processing, to the second client computer under the operation of the required execution of data and data; Data comprise file and file directory; File directory is used to indicate the memory location of file in the second client computer.
Enquiry module 32, is connected withreceiver module 31, the property cache of the data of indicating for inquiry file processing command.
Wherein, property cache is used to indicate the metadata of data.Metadata, comprising: at least one in data volume, last modification time, last access time and creation-time.
Whetherjudge module 33 is identical with the property cache inquiring for judging the property cache of data corresponding data copy.
Wherein, data trnascription is in advance data in the second client computer to be copied to acquisition;
Thefirst sending module 34, if be identical for the result of judge module judgement, the operation to the indication of data trnascription execute file processing command, generates the first operating result, and sends the first operating result to the first client computer.
Wherein, operation, comprising: open, close, at least one in read operation and write operation.
In the present embodiment, by server, receive the file manipulation command of the data that are used to indicate required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
The structural representation of the server that Fig. 4 provides for another embodiment of the present invention, as shown in Figure 4, on the basis of a upper embodiment, theenquiry module 32 in the present embodiment further comprises:
Judgingunit 321, for judging whether the pre-stored property cache that has data.
Wherein, pre-stored property cache obtains in advance from the second client computer.
The first determiningunit 322, be connected with judgingunit 321, if be used for the pre-stored property cache that has, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed first threshold the settling time of pre-stored property cache, using pre-stored property cache as the property cache inquiring.
Wherein, the settling time of property cache be from get in advance property cache time be carved into the time between current time.
The second determiningunit 323, be connected with the first determiningunit 322 with judgingunit 321 respectively, if for not storing in advance the property cache of data, or exceed first threshold the settling time of determining pre-stored property cache, from the second client computer, obtain the property cache of data, using the property cache of the data that get as the property cache inquiring.
Further, the server that the present embodiment provides, also comprises: thesecond sending module 35, identity andAuthority Verification module 36 and set upmodule 37.
Thesecond sending module 35, be connected withjudge module 33, if the result forjudge module 33 judgements is different, the second client computer Transmit message processing instruction to file manipulation command indication, so that the operation of the second client computer to the data execution file processing instruction indication of file process instruction indication generates the second operating result; From the second client computer, obtain the second operating result, the second operating result getting is joined in message queue; Real-Time Monitoring the second operating result joins the time in message queue, when the second operating result joins time in message queue while exceeding Second Threshold, abandons the second operating result in message queue; During the second operating result in carrying out message queue, to the first client computer, send the second operating result in message queue.
Identity andAuthority Verification module 36, the connection request for sending according to the first client computer, carries out identity and Authority Verification to the first client computer; Connection request comprises username and password;
Set upmodule 37, be connected withreceiver module 31 withAuthority Verification module 36 with identity respectively, if for being verified, set up the service processes corresponding with the first client computer, the file manipulation command that service processes sends for receiving the first client computer.
Based on this, receiver module is specifically for utilizing described service processes to receive the described file manipulation command that described the first client computer sends.
Server in the present embodiment can be the server in distributed file system, and this distributed file system comprises at least two client computer and the server that is positioned at high in the clouds.Server, the first client computer and the second client computer can be used for providing software to serve (Software as a Service), SaaS is a kind of Software Delivery pattern, software supplier's unified management is deployed in the application software in server, user is by above-mentioned client computer access network, thereby the application software in RAS, that is to say, the execution of application software is being positioned on the server in high in the clouds, utilizes the application software in high in the clouds to process user's file.User's file can be stored in above-mentioned client computer, also can be stored in server.When the file of the required processing of user is stored in the second client computer, and it is while utilizing the first client computer RAS, user need to send connection request by the first client computer, to open the file directory in the second client computer by server, know the file under this document catalogue, and then therefrom obtain the file of required processing.
In the present embodiment, by reception, be used to indicate the file manipulation command of the data of required processing, the property cache of the data of inquiry file processing command indication, whether the property cache that judges data corresponding data copy is identical with the property cache inquiring, if identical, operation to the indication of data trnascription execute file processing command, generate the first operating result, and send the first operating result to the first client computer, property cache due to pre-stored data, and data trnascription corresponding to pre-stored data, having reduced by the first client computer utilizes network to conduct interviews to server, the number of times of by the network between server and the second client computer, the data that comprise file or catalogue in the second client computer being processed again, thereby solve in distributed file system, the performance of data processing is subject to the larger technical problem of impact of network condition.
One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each embodiment of the method can complete by the relevant hardware of program command.Aforesaid program can be stored in a computer read/write memory medium.This program, when carrying out, is carried out the step that comprises above-mentioned each embodiment of the method; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CDs.
Finally it should be noted that: each embodiment, only in order to technical scheme of the present invention to be described, is not intended to limit above; Although the present invention is had been described in detail with reference to aforementioned each embodiment, those of ordinary skill in the art is to be understood that: its technical scheme that still can record aforementioned each embodiment is modified, or some or all of technical characterictic is wherein equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.

Claims (10)

1. the data processing method based on distributed file system, is characterized in that, comprising:
Server receives the file manipulation command that the first client computer sends; Described file manipulation command is used to indicate the data of required processing, to the second client computer under the operation of the required execution of described data and described data; Described data comprise at least one in file and file directory; Described file directory is used to indicate the memory location of described file in described the second client computer;
The property cache of the described data of file manipulation command indication described in described server lookup; Described property cache is used to indicate the metadata of described data;
Described server judges that whether the property cache of described data corresponding data copy is identical with the property cache inquiring; Described data trnascription is in advance data described in described the second client computer to be copied to acquisition;
If the property cache of described data corresponding data copy with described in the property cache that inquires identical, described server is carried out the described operation of described file manipulation command indication to described data trnascription, generate the first operating result, to described the first client computer, send described the first operating result.
2. the data processing method based on distributed file system according to claim 1, is characterized in that, the property cache of the described data of file manipulation command indication described in described server lookup, comprising:
Described server judges whether the pre-stored property cache that has described data; Pre-stored described property cache obtains in advance from described the second client computer;
If described server is pre-stored, there is described property cache, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed described first threshold the settling time of pre-stored property cache, described server is using described pre-stored property cache as the described property cache inquiring; The settling time of described property cache be from get in advance described property cache time be carved into the time between current time;
If described server does not store the property cache of described data in advance, or exceed described first threshold the settling time of determining described pre-stored property cache, described server obtains the property cache of described data from described the second client computer, using the property cache of the described data that get as the described property cache inquiring.
3. the data processing method based on distributed file system according to claim 1, is characterized in that, the described data processing method based on distributed file system also comprises:
If the property cache of described data corresponding data copy from described in the property cache that inquires different, described server sends described file process instruction to described second client computer of described file manipulation command indication, so that described the second client computer is carried out the described operation of described file process instruction indication to the described data of described file process instruction indication, generate the second operating result; Described server obtains described the second operating result from described the second client computer; Described server sends described the second operating result to described the first client computer.
4. the data processing method based on distributed file system according to claim 3, is characterized in that, described server sends described the second operating result to described the first client computer, comprising:
Described server joins described the second operating result getting in message queue;
Described server is processed described the second operating result in described message queue according to first in first out;
During the second operating result in carrying out described message queue, described server judges that the second operating result in described message queue joins the time in described message queue;
If the time that described the second operating result joins in described message queue exceeds Second Threshold, described server abandons described the second operating result in described message queue;
If the time that described the second operating result joins in described message queue does not exceed described Second Threshold, described server sends described the second operating result to described the first client computer.
5. the data processing method based on distributed file system according to claim 1, is characterized in that, before the file manipulation command that described reception the first client computer sends, comprising:
The connection request sending according to described the first client computer, carries out identity and Authority Verification to described the first client computer; Described connection request comprises username and password;
If be verified, set up the service processes corresponding with described the first client computer;
Based on this, described server receives the file manipulation command that the first client computer sends, and comprising:
Utilize described service processes to receive the described file manipulation command that described the first client computer sends.
6. according to the data processing method based on distributed file system described in claim 1-5 any one, it is characterized in that, described metadata, comprising: at least one in data volume, last modification time, last access time and creation-time;
Described operation, comprising: open, close, at least one in read operation and write operation.
7. a server, is arranged in distributed file system, it is characterized in that, comprising:
Receiver module, the file manipulation command sending for receiving the first client computer; Described file manipulation command is used to indicate the data of required processing, to the second client computer under the operation of the required execution of described data and described data; Described data comprise at least one in file and file directory; Described file directory is used to indicate the memory location of described file in described the second client computer;
Enquiry module, for inquiring about the property cache of the described data of described file manipulation command indication; Described property cache is used to indicate the metadata of described data;
Whether judge module is identical with the property cache inquiring for judging the property cache of described data corresponding data copy; Described data trnascription is in advance data described in described the second client computer to be copied to acquisition;
The first sending module, if be identical for the result of described judge module judgement, carries out the described operation of described file manipulation command indication to described data trnascription, generate the first operating result, to described the first client computer, sends described the first operating result.
8. server according to claim 7, is characterized in that, described enquiry module, comprising:
Judging unit, for judging whether the pre-stored property cache that has described data; Pre-stored described property cache obtains in advance from described the second client computer;
The first determining unit, if be used for the pre-stored described property cache that has, whether exceed first threshold the settling time of determining pre-stored property cache, if determine and do not exceed described first threshold the settling time of pre-stored property cache, using described pre-stored property cache as the described property cache inquiring; The settling time of described property cache be from get in advance described property cache time be carved into the time between current time;
The second determining unit, if for not storing in advance the property cache of described data, or exceed described first threshold the settling time of determining described pre-stored property cache, from described the second client computer, obtain the property cache of described data, using the property cache of the described data that get as the described property cache inquiring.
9. server according to claim 7, is characterized in that, described server also comprises:
The second sending module, if the result for described judge module judgement is different, described the second client computer to described file manipulation command indication sends described file process instruction, so that described the second client computer is carried out the described operation of described file process instruction indication to the described data of described file process instruction indication, generate the second operating result; From described the second client computer, obtain described the second operating result, to described the first client computer, send described the second operating result.
10. server according to claim 7, is characterized in that, described server also comprises:
Identity and Authority Verification module, the connection request for sending according to described the first client computer, carries out identity and Authority Verification to described the first client computer; Described connection request comprises username and password;
Set up module, if for being verified, set up the service processes corresponding with described the first client computer;
Based on this, described receiver module is specifically for utilizing described service processes to receive the described file manipulation command that described the first client computer sends.
CN201310482041.4A2013-10-152013-10-15Data processing method based on distributed file system and serverActiveCN103595761B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN201310482041.4ACN103595761B (en)2013-10-152013-10-15Data processing method based on distributed file system and server

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN201310482041.4ACN103595761B (en)2013-10-152013-10-15Data processing method based on distributed file system and server

Publications (2)

Publication NumberPublication Date
CN103595761Atrue CN103595761A (en)2014-02-19
CN103595761B CN103595761B (en)2017-01-04

Family

ID=50085741

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN201310482041.4AActiveCN103595761B (en)2013-10-152013-10-15Data processing method based on distributed file system and server

Country Status (1)

CountryLink
CN (1)CN103595761B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN105760395A (en)*2014-12-182016-07-13华为技术有限公司Data processing method, device and system
CN106790666A (en)*2017-01-202017-05-31泰华智慧产业集团股份有限公司Load balancing mesh architecture and its build operation method
CN106850827A (en)*2017-02-242017-06-13深圳市中博睿存信息技术有限公司A kind of metadata of distributed type file system treating method and apparatus
CN110888840A (en)*2019-11-072020-03-17浪潮电子信息产业股份有限公司 File query method, device, device and medium in distributed file system
CN113419879A (en)*2021-07-092021-09-21中国电信股份有限公司Message processing method, device, equipment and storage medium
WO2024233212A1 (en)*2023-05-102024-11-14Google LlcQuery acceleration and metadata caching

Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101741986A (en)*2009-12-312010-06-16优视科技有限公司Page cache method for mobile communication equipment terminal
US20100198888A1 (en)*2009-01-302010-08-05Blomstedt Linda CSystem for managing distributed assets and medadata
CN102299960A (en)*2011-08-222011-12-28盛乐信息技术(上海)有限公司Peer-to-peer (P2P) technology-based network file system and method
CN102523279A (en)*2011-12-122012-06-27云海创想信息技术(无锡)有限公司Distributed file system and hot file access method thereof
CN102523285A (en)*2011-12-152012-06-27杭州电子科技大学Storage caching method of object-based distributed file system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US20100198888A1 (en)*2009-01-302010-08-05Blomstedt Linda CSystem for managing distributed assets and medadata
CN101741986A (en)*2009-12-312010-06-16优视科技有限公司Page cache method for mobile communication equipment terminal
CN102299960A (en)*2011-08-222011-12-28盛乐信息技术(上海)有限公司Peer-to-peer (P2P) technology-based network file system and method
CN102523279A (en)*2011-12-122012-06-27云海创想信息技术(无锡)有限公司Distributed file system and hot file access method thereof
CN102523285A (en)*2011-12-152012-06-27杭州电子科技大学Storage caching method of object-based distributed file system

Cited By (8)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN105760395A (en)*2014-12-182016-07-13华为技术有限公司Data processing method, device and system
CN106790666A (en)*2017-01-202017-05-31泰华智慧产业集团股份有限公司Load balancing mesh architecture and its build operation method
CN106850827A (en)*2017-02-242017-06-13深圳市中博睿存信息技术有限公司A kind of metadata of distributed type file system treating method and apparatus
CN110888840A (en)*2019-11-072020-03-17浪潮电子信息产业股份有限公司 File query method, device, device and medium in distributed file system
CN113419879A (en)*2021-07-092021-09-21中国电信股份有限公司Message processing method, device, equipment and storage medium
CN113419879B (en)*2021-07-092023-08-04天翼云科技有限公司 Message processing method, device, equipment and storage medium
WO2024233212A1 (en)*2023-05-102024-11-14Google LlcQuery acceleration and metadata caching
US12339850B2 (en)2023-05-102025-06-24Google LlcQuery acceleration and metadata caching

Also Published As

Publication numberPublication date
CN103595761B (en)2017-01-04

Similar Documents

PublicationPublication DateTitle
JP6139773B2 (en) Token-based authorization control for duplicate writes
CN103150263B (en)Classification storage means
US12229094B2 (en)Proxy-based database scaling
CN103595761A (en)Data processing method based on distributed file system and server
US9883006B2 (en)Techniques to transfer large collection containers
US8955087B2 (en)Method and system for transferring replicated information from source storage to destination storage
US20140081911A1 (en)Optimizing automatic deletion of backup files
CN102780603B (en)Web traffic control method and device
US10521401B2 (en)Data object lockdown
CN103559231B (en)A kind of file system quota management method, apparatus and system
CN109120709A (en)A kind of caching method, device, equipment and medium
US10204021B2 (en)Recovery of an infected and quarantined file in a primary storage controller from a secondary storage controller
KR102567900B1 (en) Method and Apparatus for Ensuring Continuous Device Operational Stability in Cloud Degraded Mode
WO2014183514A1 (en)Method, device, and computer storage medium for hierarchical storage
US20220164228A1 (en)Fine-grained virtualization resource provisioning for in-place database scaling
US20200341674A1 (en)Method, device and computer program product for restoring data
CN105760271A (en)Method for monitoring credible state of computing node in cluster mode
CN111752861A (en) Application multi-level caching method, device, client and storage medium
EP3180697A1 (en)Coalescing storage operations
CN105574008B (en) Task scheduling method and device applied to distributed file system
US10949102B2 (en)Leveraging snapshot for time series pattern analysis prediction
US11200254B2 (en)Efficient configuration replication using a configuration change log
US20200028769A1 (en)Efficient heartbeat with remote servers by nas cluster nodes
US9787604B2 (en)Cloud computing infrastructure, method and application
US11321495B2 (en)Anomalous cache coherence transaction detection in a heterogeneous system

Legal Events

DateCodeTitleDescription
C06Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
C14Grant of patent or utility model
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp