Disclosure of Invention
The embodiment of the invention provides a data access method, a device, equipment and a storage medium, which can improve the hit rate of cache file access on a gateway node, thereby improving the performance of a distributed file system.
In a first aspect, an embodiment of the present invention provides a data access method, including:
receiving a data access request sent by a client; the data access request comprises file attribute information and data block information to be accessed;
acquiring first lease information and first version information of a file cached in a gateway node according to the file attribute information;
acquiring second version information and second lease information of a cache data block according to the data block information to be accessed;
comparing the first version information with the second version information, and if the first version information and the second version information are different, comparing the first lease information with the second lease information;
and if the first lease information and the second lease information are the same, the cache data block is the target access data.
Further, after receiving the data access request sent by the client, the method further comprises:
sending an authorization request to a file transaction management server according to the data access request; and enabling the transaction management server to authorize the gateway node and judging whether the first lease information needs to be updated according to the authorization request.
Further, the authorization request includes a first gateway node ID, and the first lease information includes a second gateway node ID and a timestamp; judging whether the first lease information needs to be updated according to the authorization request, including:
if the first gateway node ID is the same as the second gateway node ID, the first lease information does not need to be updated;
and if the first gateway node ID is different from the second gateway node ID, updating the second gateway node of the first lease information to the first gateway node, and updating the time stamp.
Further, after comparing the first version information with the second version information, the method further comprises:
and if the first version information is the same as the second version information, the cached data block is the target access data.
Further, after comparing the first lease information and the second lease information, further comprising:
if the first lease information and the second lease information are different, acquiring target access data from a shared memory;
and replacing the cache data block corresponding to the data block information to be accessed with the target access data, and replacing the second lease information with the first lease information.
In a second aspect, an embodiment of the present invention further provides a data access apparatus, including:
the data access request receiving module is used for receiving a data access request sent by the client; the data access request comprises file attribute information and data block information to be accessed;
a first version information obtaining module, configured to obtain first lease information and first version information of a file cached in a gateway node according to the file attribute information;
the second version information acquisition module is used for acquiring second version information and second lease information of the cache data block according to the data block information to be accessed;
the comparison module is used for comparing the first version information with the second version information, and if the first version information and the second version information are different, the first lease information and the second lease information are compared;
and the target access data determining module is used for caching the data block as target access data when the first lease information and the second lease information are the same.
Further, the method further comprises the following steps: an authorization request sending module, configured to:
sending an authorization request to a file transaction management server according to the data access request; and enabling the transaction management server to authorize the gateway node and judging whether the first lease information needs to be updated according to the authorization request.
Further, the authorization request includes a first gateway node ID, and the first lease information includes a second gateway node ID and a timestamp; lease information updating module for:
if the first gateway node ID is the same as the second gateway node ID, the first lease information does not need to be updated;
and if the first gateway node ID is different from the second gateway node ID, updating the second gateway node of the first lease information to the first gateway node, and updating the time stamp.
In a third aspect, an embodiment of the present invention further provides a computer apparatus, the apparatus including: comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which processor implements the data access method according to the embodiments of the invention when executing the program.
In a third aspect, embodiments of the present invention further provide a computer readable storage medium having stored thereon a computer program which, when executed by a processing device, implements a data access method according to embodiments of the present invention.
According to the embodiment of the invention, a data access request sent by a client is received firstly, then, first lease information and first version information of a file cached in a gateway node are acquired according to file attribute information, then, second version information and second lease information of a cached data block are acquired according to data block information to be accessed, then, the first version information and the second version information are compared, and if the first lease information and the second lease information are different, the first lease information and the second lease information are compared; and if the first lease information and the second lease information are the same, the cache data block is the target access data. According to the data access method provided by the embodiment of the invention, when the first version information of the file is different from the second version information of the cached data block, the first lease information of the file is compared with the second lease information of the cached data block, if the first lease information is the same as the second lease information, the cached data block is the target access data, and the hit rate of the cached file access on the gateway node can be improved, so that the performance of the distributed file system is improved.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
The present embodiment is implemented based on a data access system. Fig. 1 is a schematic structural diagram of a data access system according to an embodiment of the present invention, as shown in fig. 1, the system includes: a plurality of gateway nodes, a transaction management server and a shared memory.
Wherein, the gateway node supports protocols such as NFS, SMB and the like. Data access requests (including read requests and write requests) sent by clients are received, and then the shared memory is accessed for reading and writing (I/O) of data according to the data access requests. The transaction management server is used for controlling concurrent access data transactions of a plurality of gateway nodes. And authenticating and authorizing the data access transaction of the gateway node to ensure the security and the correctness of the data operation of each gateway node. The shared memory is used for storing data and receiving data access of the gateway node.
Example 1
Fig. 2 is a flowchart of a data access method according to a first embodiment of the present invention, where the method may be applied to a data access situation, and the method may be performed by a data access device, where the device may be composed of hardware and/or software, and may generally be integrated into a device having a data access function, where the device may be an electronic device such as a server or a server cluster. As shown in fig. 1, the method specifically includes the following steps:
step 110, a data access request sent by a client is received.
Wherein the data access request includes file attribute information and data block information to be accessed. The file attribute information may include information such as a name of the file, a modification time of the file, and a size of the file. The data block information may include address information of the data block in the file. The data access request may include a data read request and a data write request.
Optionally, after receiving the data access request sent by the client, the method further includes the following steps: sending an authorization request to a file transaction management server according to the data access request; the transaction management server authorizes the gateway node and judges whether the first lease information needs to be updated according to the authorization request.
Wherein the authorization request includes a first gateway node ID and the first lease information includes a second gateway node ID and a timestamp.
Specifically, the method for judging whether the first lease information needs to be updated according to the authorization request may be: if the first gateway node ID is the same as the second gateway node ID, the first lease information does not need to be updated; if the first gateway node ID is different from the second gateway node ID, updating the second gateway node of the first lease information to the first gateway node, and updating the time stamp.
In this embodiment, each file corresponds to a piece of first lease information. If the ID of the first gateway node accessing the current file is the same as the ID of the second gateway node in the lease information, the fact that the same gateway node continuously accesses the file is indicated, and the first lease information of the file does not need to be updated; if the ID of the first gateway node accessing the current file is different from the ID of the second gateway node in the lease information, the first lease information of the file needs to be updated if the ID of the first gateway node accessing the current file is different from the ID of the second gateway node in the lease information.
The first lease information may be stored in a shared buffer, and each gateway node may obtain the first lease information in real time.
Step 120, obtaining the first lease information and the first version information of the file cached in the gateway node according to the file attribute information.
Wherein the first version information may characterize the number of times the file is accessed.
Specifically, after receiving a data access request sent by a client, the gateway node judges whether the file is cached in the gateway node according to the file attribute information, and if so, acquires first lease information and first version information of the file cached in the gateway node. If the file is not cached in the gateway node, the data needs to be accessed into the shared memory.
And 130, obtaining second version information and second lease information of the cached data block according to the data block information to be accessed.
The cached data block may be understood as a data block cached in a file of the gateway node. The second version information is used to characterize the number of times the data block is accessed.
Specifically, second version information and second lease information of the cached data block are obtained according to the address information of the data block in the data block information.
Step 140, comparing the first version information with the second version information, and if not, comparing the first lease information with the second lease information.
And comparing the first version information with the second version information, and if the first version information and the second version information are the same, taking the cached data block as target access data. If the first lease information and the second lease information are different, the first lease information and the second lease information are continuously compared.
Step 150, if the first lease information and the second lease information are the same, the cache data block is the target access data.
Optionally, if the first lease information and the second lease information are different, acquiring target access data from the shared memory; and replacing the cache data block corresponding to the data block information to be accessed with target access data, and replacing the second lease information with the first lease information.
According to the technical scheme, a data access request sent by a client is received, first lease information and first version information of a file cached in a gateway node are obtained according to file attribute information, second version information and second lease information of a cached data block are obtained according to data block information to be accessed, the first version information and the second version information are compared, and if the first lease information and the second lease information are different, the first lease information and the second lease information are compared; and if the first lease information and the second lease information are the same, the cache data block is the target access data. According to the data access method provided by the embodiment of the invention, when the first version information of the file is different from the second version information of the cached data block, the first lease information of the file is compared with the second lease information of the cached data block, if the first lease information is the same as the second lease information, the cached data block is the target access data, and the hit rate of the cached file access on the gateway node can be improved, so that the performance of the distributed file system is improved.
Example two
Fig. 3 is a schematic structural diagram of a data access device according to a second embodiment of the present invention. As shown in fig. 3, the apparatus includes: a data access request receiving module 210, a first version information acquiring module 220, a second version information acquiring module 230, a comparing module 240 and a target access data determining module 250.
A data access request receiving module 210, configured to receive a data access request sent by a client; the data access request comprises file attribute information and data block information to be accessed;
a first version information obtaining module 220, configured to obtain, according to the file attribute information, first lease information and first version information of a file cached in a gateway node;
a second version information obtaining module 230, configured to obtain second version information and second lease information of the cached data block according to the data block information to be accessed;
a comparing module 240, configured to compare the first version information with the second version information, and if the first version information and the second version information are different, compare the first lease information with the second lease information;
the target access data determining module 250 is configured to cache a data block as target access data when the first lease information and the second lease information are the same.
Optionally, the method further comprises: lease update module for:
sending an authorization request to a file transaction management server according to the data access request; and enabling the transaction management server to authorize the gateway node and judging whether the first lease information needs to be updated according to the authorization request.
Optionally, the authorization request includes a first gateway node ID, and the first lease information includes a second gateway node ID and a time stamp; lease update module, further for:
if the first gateway node ID is the same as the second gateway node ID, the first lease information does not need to be updated;
and if the first gateway node ID is different from the second gateway node ID, updating the second gateway node of the first lease information to the first gateway node, and updating the time stamp.
Optionally, the target access data determining module 250 is further configured to:
and if the first version information is the same as the second version information, the cached data block is the target access data.
Optionally, the target access data determining module 250 is further configured to:
if the first lease information and the second lease information are different, acquiring target access data from a shared memory;
and replacing the cache data block corresponding to the data block information to be accessed with the target access data, and replacing the second lease information with the first lease information.
The device can execute the method provided by all the embodiments of the invention, and has the corresponding functional modules and beneficial effects of executing the method. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the invention.
Example III
Fig. 4 is a schematic structural diagram of a computer device according to a third embodiment of the present invention. FIG. 4 illustrates a block diagram of a computer device 312 suitable for use in implementing embodiments of the present invention. The computer device 312 shown in fig. 4 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention. Device 312 is a typical data access function computing device.
As shown in FIG. 4, computer device 312 is in the form of a general purpose computing device. Components of computer device 312 may include, but are not limited to: one or more processors 316, a storage device 328, and a bus 318 that connects the different system components (including the storage device 328 and the processor 316).
Bus 318 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include industry standard architecture (Industry Standard Architecture, ISA) bus, micro channel architecture (Micro Channel Architecture, MCA) bus, enhanced ISA bus, video electronics standards association (Video Electronics Standards Association, VESA) local bus, and peripheral component interconnect (Peripheral Component Interconnect, PCI) bus.
Computer device 312 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by computer device 312 and includes both volatile and nonvolatile media, removable and non-removable media.
The storage 328 may include computer system-readable media in the form of volatile memory, such as random access memory (Random Access Memory, RAM) 330 and/or cache memory 332. The computer device 312 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 334 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, commonly referred to as a "hard disk drive"). Although not shown in fig. 4, a disk drive for reading from and writing to a removable nonvolatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from and writing to a removable nonvolatile optical disk (e.g., a Compact Disc-Read Only Memory (CD-ROM), digital versatile Disc (Digital Video Disc-Read Only Memory, DVD-ROM), or other optical media) may be provided. In such cases, each drive may be coupled to bus 318 through one or more data medium interfaces. Storage 328 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the invention.
Programs 336 having a set (at least one) of program modules 326 may be stored, for example, in storage 328, such program modules 326 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 326 generally perform the functions and/or methods in the described embodiments of the invention.
The computer device 312 may also communicate with one or more external devices 314 (e.g., keyboard, pointing device, camera, display 324, etc.), one or more devices that enable a user to interact with the computer device 312, and/or any devices (e.g., network card, modem, etc.) that enable the computer device 312 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 322. Moreover, the computer device 312 may also communicate with one or more networks such as a local area network (Local Area Network, LAN), a wide area network Wide Area Network, a WAN) and/or a public network such as the internet via the network adapter 320. As shown, network adapter 320 communicates with other modules of computer device 312 via bus 318. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with computer device 312, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk array (Redundant Arrays of Independent Disks, RAID) systems, tape drives, data backup storage systems, and the like.
The processor 316 executes various functional applications and data processing by running programs stored in the storage 328, for example, to implement the data access method provided by the above-described embodiments of the present invention.
Example IV
The embodiment of the invention provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the program is executed by a processing device, the program realizes the data access method as in the embodiment of the invention. The computer readable medium of the present invention described above may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a source text input by a user, and translating the source text into a target text corresponding to a target language; acquiring the historical correction behavior of the user; correcting the target text according to the history correction behavior to obtain a translation result, and pushing the translation result to a client where the user is located.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.