CN111026681A - Caching method, caching system and caching medium based on Ceph - Google Patents

Abstract

The invention discloses a caching method, a caching system and a caching medium based on Ceph. The method comprises the following steps: writing data generated by a client into a solid state disk; the data stored on the solid state disk is printed back into the Ceph according to a preset rule; and judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, accessing the Ceph to read the data. The invention has high data reliability and small risk of data loss.

Description

Caching method, caching system and caching medium based on Ceph

Technical Field

The invention belongs to the technical field of cloud computing, and particularly relates to a caching method, a caching system and a caching medium based on Ceph.

Background

Ceph is a storage system platform that focuses on distributed, elastically scalable, highly reliable, and excellent performance, and can be used to provide a block storage scheme for virtual machines or a conventional file system through FUSEs. Ceph is a highly configurable system that an administrator can control various aspects of the system. It provides a command line interface for monitoring and controlling its storage clusters. In the prior art, as shown in fig. 1, a client accesses a Ceph, and generally uses a memory implemented by the Ceph to cache data, and a client reads and writes data to the memory first, so that the memory temporarily stores data of a certain volume, and after the data in the memory meets a condition of writing in the Ceph, the data in the memory is written in the Ceph. However, caching data with memory poses a serious risk: 1) the memory is a volatile memory, and when the client is abnormal, the data stored in the memory can be lost, for example, the data in the memory cached by Ceph is lost due to sudden power failure of a machine, program abnormality and the like; 2) the memory resource is limited, and the requirement of concurrent client access cannot be met; 3) the machine restarts and cannot access the previously cached data.

Disclosure of Invention

In view of at least one of the defects or improvement requirements of the prior art, the present invention provides a caching method, system and storage medium based on Ceph, and aims to solve the problem in the prior art that data is easily lost.

To achieve the above object, according to a first aspect of the present invention, there is provided a Ceph-based caching method, including the steps of:

writing data generated by a client into a solid state disk;

the data stored on the solid state disk is printed back into the Ceph according to a preset rule;

and judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, accessing the Ceph to read the data.

Preferably, the rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison between the time when the data on the solid state disk is not accessed and the preset time.

Preferably, the rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison of the ratio of the cache data amount on the solid state disk to the total capacity of the solid state disk and a preset ratio.

Preferably, the data on the solid state disk is directly flushed back to the Ceph in an FIFO manner.

Preferably, the data which is not accessed in the solid state disk hot chain for more than the preset time is moved to the solid state disk cold chain by adopting an LRU mode, and the data which is not accessed in the solid state disk cold chain for more than the preset time is printed back to the Ceph.

According to a second aspect of the present invention, there is provided a Ceph-based cache system, comprising:

the solid state disk is used for writing data generated by the client into the solid state disk;

the Ceph is used for refreshing the data stored on the solid state disk to the Ceph according to a preset rule;

and the cache management unit is used for judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, accessing the Ceph to read the data.

According to a third aspect of the invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the above.

In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects: 1) when the client is abnormal, the data stored in the memory cannot be lost; 2) the machine is restarted, and the previously cached data can be accessed; 3) the client can be highly concurrent under the condition of starting the cache; 4) the capacity of the solid state disk is larger than that of the memory, and the solid state disk has price advantage and can be used in a large scale.

Drawings

FIG. 1 is a diagram of a prior art Ceph cache system;

fig. 2 is a schematic diagram of a Ceph cache system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

According to the caching method, the caching system and the caching medium based on the Ceph, cached data are stored on the solid state disk which is the nonvolatile memory, so that the data cached on the nonvolatile memory cannot be lost even if the power is off, and the storage speed of the solid state disks such as the SSD and the NMVE is high, so that the data accessing speed of a user can be improved.

As shown in fig. 2, a cache system based on Ceph according to an embodiment of the present invention includes:

the solid state disk is used for caching the data generated by the client to the solid state disk;

the Ceph is used for refreshing the data stored on the solid state disk to the Ceph according to a preset rule;

and the cache management unit is responsible for managing cache data and judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, directly accessing the Ceph to read the data.

The caching method based on the Ceph comprises the following steps:

s1, when caching data, the client writes the data generated by the client into the solid state disk;

and S2, and then, the data stored on the solid state disk is flushed back to Ceph according to a preset rule.

There are two kinds of back-brushing strategies:

(1) time strategy

The time for which the swap-out operation is required to be performed when the data is not accessed in the solid state disk cache for a long time can be dynamically set by configuration parameters, and the default time is 10 minutes.

After the back-flushing interface is called, the code prepares to flush back to a Ceph data mode by judging whether each cache data block has data which is not accessed for more than 10 minutes, and the following two modes exist:

in FIFO mode, data is flushed back directly to Ceph. FIFO is to brush back data according to the rule of first-in first-out.

In the LRU approach, data in the solid state disk hot chain that was not accessed for more than 10 minutes will be moved to the solid state disk cold chain, and data in the solid state disk cold chain that was not accessed for more than 10 minutes will be flushed back to Ceph. LRU flushes data back according to the least recently used rule.

(2) Spatial strategy

The threshold value of the ratio of the cache data volume to the total capacity of the solid state disk can be dynamically set by parameters, and the default value is 20%.

When the refresh interface is called, whether the ratio is reached is judged to determine whether the data block needs to be refreshed to Ceph.

And S3, when the client reads the data, judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, directly accessing the Ceph to read the data.

Compared with the prior art in which a memory is used for storing cache data, the cache method, the cache system and the storage medium based on the Ceph provided by the invention have the following advantages: 1) when the client is abnormal, the data stored in the memory cannot be lost; 2) the machine is restarted, and the previously cached data can be accessed; 3) the client can be highly concurrent under the condition of starting the cache; 4) the capacity of the solid state disk is larger than that of the memory, and the solid state disk has price advantage and can be used in a large scale.

The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement any of the technical solutions of the above embodiments of the Ceph-based caching method. The implementation principle and the technical effect are similar, and the detailed description is omitted here.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. A cache method based on Ceph is characterized by comprising the following steps:

writing data generated by a client into a solid state disk;

the data stored on the solid state disk is printed back into the Ceph according to a preset rule;

and judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, accessing the Ceph to read the data.

2. The Ceph-based caching method of claim 1, wherein the rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison between the time when the data on the solid state disk is not accessed and the preset time.

3. The Ceph-based caching method of claim 1, wherein the rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison of the ratio of the cache data amount on the solid state disk to the total capacity of the solid state disk and a preset ratio.

4. The Ceph-based caching method according to claim 2, wherein the data on the solid state disk is directly flushed back to the Ceph in a FIFO manner.

5. The Ceph-based caching method according to claim 2, wherein data in the solid state disk hot chain that has not been accessed for more than the preset time is moved to the solid state disk cold chain by an LRU method, and the data in the solid state disk cold chain that has not been accessed for more than the preset time is copied back to the Ceph.

6. A Ceph-based cache system, comprising:

the solid state disk is used for writing data generated by the client into the solid state disk;

the Ceph is used for refreshing the data stored on the solid state disk to the Ceph according to a preset rule;

and the cache management unit is used for judging whether the data to be read by the client is stored on the solid state disk, if so, accessing the solid state disk to read the data, and if not, accessing the Ceph to read the data.

7. A Ceph-based caching system as claimed in claim 6, wherein said rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison between the time when the data on the solid state disk is not accessed and the preset time.

8. A Ceph-based caching system as claimed in claim 6, wherein said rule is:

and determining whether to refresh the data stored on the solid state disk into the Ceph according to the comparison of the ratio of the cache data amount on the solid state disk to the total capacity of the solid state disk and a preset ratio.

9. The Ceph-based cache system according to claim 7, wherein the data on the solid state disk is directly flushed back to the Ceph in a FIFO manner.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 5.

Images