Disclosure of Invention
Embodiments of the present invention provide a method and an apparatus for managing a storage array, which can more reasonably use each storage module in the storage array according to a performance test parameter of each storage module.
In a first aspect, a storage array management method is provided, including:
acquiring performance test data of each storage module in the storage array; the storage array at least comprises one storage module;
and determining the partition type of each storage module according to the performance test data of each storage module in the storage array, wherein the partitions of different types are used for storing data with different activity degrees.
With reference to the first aspect, in a first implementation manner of the first aspect, the determining, according to the performance test data of each memory module in the memory array, a partition type to which each memory module belongs includes:
determining the performance score of each storage module according to the performance test data of each storage module;
sorting the storage modules according to the scores;
and determining the partition type of each storage module according to the sorting result.
With reference to the first aspect, in a second implementation manner of the first aspect, after determining the partition type to which each storage module belongs, the method further includes:
and adding the identification information of the storage module into the module identification list of the corresponding partition category.
With reference to the first aspect, in a third implementation manner of the first aspect, the method further includes:
performing performance test on the storage module in advance to obtain performance test data, and storing the performance test data at a specific position of the storage module;
the acquiring performance test data of the memory modules in the memory array comprises:
performance test data stored at a particular location of the memory module is obtained.
With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the method further includes:
judging the accuracy of the performance test data information stored at the specific position;
and if the data is inaccurate, respectively carrying out performance test on each storage module according to a preset scene to respectively obtain the performance test data of each storage module.
With reference to the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the method further includes:
and updating the performance test data stored at the specific position on the storage module according to the performance test data obtained by the test.
With reference to the first aspect, in a sixth implementation manner of the first aspect, the acquiring performance test data of each memory module in the memory array includes:
performing read-write test on each storage module according to a prefabricated scene at a set time interval and/or when the system is idle; the prefabricated scene comprises sequential reading and writing, random reading and writing or reading and writing mixing;
respectively determining performance test data of each storage module according to the test result; the performance test data includes at least one of a number of read and write (I/O) operations per second, read and write bandwidths, and read and write delays.
With reference to the first aspect or any one of the first to the sixth implementation manners of the first aspect, in a seventh implementation manner of the first aspect, the storage array is a full flash memory array, the media types of the storage module include SLC media and MLC media, and the different classes of partitions are used to store data with different liveness degrees and include:
judging the media types of the storage modules belonging to the partition categories;
if the partition formed by the storage modules with better performance test data comprises the storage module of the MLC medium, the storage module of the MLC medium is used for storing the data with higher activity;
and if the partition formed by the storage modules with poor performance test data comprises the storage module of the SLC medium, using the storage module of the SLC medium for storing the data with low activity.
In a second aspect, a storage array management apparatus is provided, including:
the device comprises an acquisition unit, a storage unit and a control unit, wherein the acquisition unit is used for acquiring performance test data of each storage module in a storage array; the storage array at least comprises one storage module;
and the partition unit is used for determining the partition type of each storage module according to the performance test data of each storage module in the storage array, wherein the partitions of different types are used for storing data with different liveness.
With reference to the second aspect, in a first implementation manner of the second aspect, the partition unit includes:
the score determining subunit is used for determining the performance score of each storage module according to the performance test data of each storage module;
the sorting subunit is used for sorting the storage modules according to the scores;
and the partition subunit is used for determining the partition type of each storage module according to the sorting result.
With reference to the second aspect, in a second implementation manner of the second aspect, the method further includes:
and the identification unit is used for adding the identification information of the storage module into the module identification list of the corresponding partition category.
With reference to the second aspect, in a third implementation manner of the second aspect, the apparatus further includes:
the test unit is used for carrying out performance test on the storage module in advance to obtain performance test data, and storing the performance test data at a specific position of the storage module;
the obtaining unit is specifically configured to obtain the performance test data stored at the specific location of the storage module.
With reference to the second aspect, in a fourth implementation manner of the second aspect, the method further includes:
the judging unit is used for judging the accuracy of the performance test data information stored at the specific position;
and the test unit is also used for respectively carrying out performance test on each storage module according to a preset scene if the performance test data of each storage module is inaccurate.
With reference to the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the method further includes:
and the updating unit is used for updating the performance test data stored at the specific position on the storage module according to the performance test data obtained by the test.
With reference to the second aspect, in a sixth implementation manner of the second aspect, the obtaining unit includes:
the read-write test subunit is used for performing read-write test on each storage module according to the preset scene at a set time interval and/or when the system is idle; the prefabricated scene comprises sequential reading and writing, random reading and writing or reading and writing mixing;
the data determining subunit is used for respectively determining the performance test data of each storage module according to the test result; the performance test data includes at least one of a number of read and write (I/O) operations per second, read and write bandwidths, and read and write delays.
With reference to the second aspect or any one of the first to the sixth implementation manners of the first aspect, in a seventh implementation manner of the second aspect, the memory array is a full flash memory array, the media types of the memory modules include SLC media and MLC media, and the partition unit is specifically configured to,
judging the media types of the storage modules belonging to the partition categories;
if the partition formed by the storage modules with better performance test data comprises the storage module of the MLC medium, the storage module of the MLC medium is used for storing the data with higher activity;
and if the partition formed by the storage modules with poor performance test data comprises the storage module of the SLC medium, using the storage module of the SLC medium for storing the data with low activity.
According to the storage array management method and device provided by the embodiment of the invention, the partition type of each storage module is determined according to the performance of each storage module, the partitions of different types are used for storing data with different liveness degrees, the partition with better performance can be used for storing the data with higher liveness degree, the corresponding partition with lower performance is used for storing the data with lower liveness degree, the partition of the storage array is not limited to the medium, the possibility of being full is reduced, and each storage module in the storage array can be reasonably used.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For the convenience of understanding of the embodiments of the present invention, the following description will be further explained with reference to specific embodiments, which are not to be construed as limiting the embodiments of the present invention.
In the embodiment of the present invention, considering that standards of each storage module in the same storage array may be different, an actual read-write speed of each storage module may be greatly different from an expected read-write speed provided by a storage module manufacturer, or the read-write speed of the SLC is more rapidly attenuated after a period of use, and the hard disks are classified by using the storage medium of the hard disks, which may cause that the storage modules cannot be reasonably utilized, thereby affecting the storage speed of the storage array. By partitioning the storage array based on performance information of individual storage modules, rather than just the media of the storage modules, individual modules in the storage array are more reasonably used.
FIG. 2 is a flowchart of a storage array management method according to an embodiment of the present invention. As shown in fig. 2, the method specifically includes:
s210, acquiring performance test data of each storage module in the storage array.
It should be noted that, in general, the memory array and the memory module are produced by different manufacturers, and even different memory modules of the same memory array are from different manufacturers or different products of the same manufacturer. Fig. 3 is a schematic diagram of a memory array structure, and as shown in fig. 3, a memory array manufacturer combines all memory modules together to form a memory array, where the memory array generally controls all modules in the memory array through a memory controller to perform read and write operations on all memory modules. The storage module provider may store the original performance test data of the storage module at a specific location, for example, the storage module provider may use a fixed-byte storage module starting at a specified LBA (Logical Block Address) to store the performance test data of the storage module, and for example, the storage module provider may map a Register memory in a BAR (Base Address Register) space as a space of the performance test data (the Register is recommended to be readable and writable), where specific implementation forms of the specific location are various, and this is merely an example and is not limited specifically. That is, the initial performance test data may be obtained from tests by the memory module provider and stored at a particular location known to the storage array manufacturer. When the memory controller determines the performance test data of the memory modules in the memory array, the parameter data at a specific position of the memory module can be directly read.
There may be multiple implementation manners for determining the performance test data of the memory modules in the memory array, which are not described in detail herein.
It should be further noted that the performance test data of the memory module may include at least one of TBW (Total Band Width ), write IOPS (write Input/Output Operations Per Second, number of write (I/O) Operations Per Second), write bandwidth (write bandwidth), write latency (write latency), read IOPS (read Input/Output Operations Per Second, number of read (I/O) Operations Per Second), read bandwidth (read bandwidth), read latency (read latency), and the like. And may also include the constituting medium of the storage module. The form of the performance test data for a particular memory module is shown in table 1.
TABLE 1
In addition, in a specific implementation process of the embodiment of the present invention, one storage array generally includes one or more storage modules, and storage media of the respective storage modules may be different, and the embodiment of the present invention is not limited in particular, for example, the storage module may be an SSD (Solid State Drive) with the storage medium SLC (SLC flash memory), an SSD (MLC flash memory) with the storage medium MLC (MLC flash memory), or an HDD (Hard Disk Drive), and the like.
S220, determining the partition type of each storage module according to the performance test data of each storage module in the storage array.
It is illustrated that different classes of partitions are used to store data of different liveness. Specifically, the number of partitions depends on the specific needs of the stored data, and generally, the partitions can be divided into at least a cold data storage partition for storing cold data and a hot data storage partition for storing hot data, and certainly, some storage partitions for storing data with moderate cold and hot data can be included. The cold data and the hot data are determined according to the activity of the data, the data with high activity can be called hot data, and the data with low activity can be called cold data.
It should be noted that the threshold interval of the performance test data of each type of partition may be determined according to the requirements of the data with different liveness on the performance of the storage module during reading and writing; the storage modules belonging to each class are determined according to the threshold interval of the performance test data of the partitions of the class.
In addition, the performance score of each storage module can be determined according to the performance test data of each storage module;
sorting the storage modules according to the scores;
and determining the partition type of each storage module according to the sorting result.
The performance test data to be calculated can be multiplied by respective weights, then the sum is obtained to obtain a comprehensive score, and the storage modules are ranked according to the comprehensive score; it is also possible to score only data requiring higher performance tests. And are not described in detail herein.
There may be multiple implementation manners for determining the partition type to which each storage module belongs, and details are not repeated here.
In one example, as shown in FIG. 3, assume that the performance test data in each of the memory modules in FIG. 3 is as shown in Table 2.
TABLE 2
| Name (R) | Medium | write IOPS | Write latency | read IOPS | Read delay | |
| Memory module 1 | SLC | 500MB/Sec | | 510MB/Sec | | |
| Memory module 2 | SLC | 250MB/Sec | | 310MB/Sec | | |
| Memory module 3 | MLC | 350MB/Sec | | 400MB/Sec | | |
| Memory module 4 | MLC | 110MB/Sec | | 210MB/Sec | | |
In this example, it is assumed that the performance interval of the partition for storing data with a high activity level is: write IOPS, not less than 300 MB/Sec; the read IOPS is more than or equal to 350 MB/Sec. The performance intervals for the partitions storing data with lower liveness are: write IOPS, 100MB/Sec-300 MB/Sec; read IOPS, 150MB/Sec-350 MB/Sec.
Firstly, determining performance test data of each storage module to obtain parameter data shown in table 2; then, determining the partition type to which each storage module belongs according to the specification in this example can obtain: the partition used for storing the data with higher activity comprises a storage module 1 and a storage module 3; the partition for storing the data with lower activity comprises a storage module 2 and a storage module 4.
Another assumption is that, in this example, the performance score of the memory module is (write IOPS × 60+ read IOPS × 40) ÷ 100.
Then, the performance score for memory module 1 is 504; the performance score for memory module 2 is 274; the performance score of the memory module 3 is 370; the performance score of the memory module 4 is 150; sorting the storage modules according to the performance scores: the device comprises a storage module 1, a storage module 3, a storage module 2 and a storage module 4; if the data to be stored is two, and each storage module needs two storage modules, then the partition type of each storage module is determined according to the sorting, and the following results can be obtained: the partition used for storing the data with higher activity comprises a storage module 1 and a storage module 3; the partition for storing the data with lower activity comprises a storage module 2 and a storage module 4.
It can be seen that in this example, the storage module for storing data with a higher liveness includes both SLC media and MLC media.
In a word, the partition type of each storage module is determined according to the performance of each storage module, the partitions of different types are used for storing data with different liveness degrees, the partition with better performance can be used for storing the data with higher liveness degree, the corresponding partition with second performance is used for storing the data with lower liveness degree, and the partition of the storage array is not limited to the medium, so that the possibility of being filled with the data is reduced, and each storage module in the storage array can be reasonably used.
In one embodiment, after S220, the method may further include: and adding the identification information of the storage module into the module identification list of the corresponding partition category.
It should be noted that the identification information of the storage module may be a LUN (Logical Unit Number) or a Namespace (Namespace), and can be used to distinguish the storage modules or the identification information for addressing.
It should be noted that, when read/write operations are required to be performed on the storage queue, the module identifier list may be queried according to the liveness of information to be read or written, so as to determine the storage module to which the information needs to be written, or the storage module in which the information is stored.
Through tabulation management of the storage modules and the corresponding partition types, the storage controller can read and write data of each partition more conveniently.
In one embodiment, the step S210 of determining the performance parameters of each storage module in the storage array may include: and respectively carrying out performance test on each storage module according to a preset scene to respectively obtain the performance parameters of each storage module.
It should be noted that, in the case that the performance test data is not provided or provided to the memory module is inaccurate, the specific performance test data may be determined by performing the performance test, and S220 is directly executed according to the performance test data. The specific prefabricated scenes may be various, for example, according to the application scene, the scenes of sequential reading, writing, random reading, writing, or mixed reading and writing can be selected, and according to the size of the data block possibly issued by the service, the IO read-write test is started by performing 4K, 16K, 32K or large packets.
In addition, the accuracy of the performance test data information written in the specific position can be determined according to the performance test data obtained through the performance test, and if the accuracy is not correct, the performance test data stored in the specific position on the storage module is updated according to the performance test data obtained through the test. In another embodiment, the controller reads the performance test data at the specific location and then performs step S220. In another embodiment, performance test data stored at a specific position on each memory module before updating may be counted and stored, so that the corresponding memory module may be evaluated according to the counted performance test data change of each memory module, and further the performance-to-price ratio, the endurance, and the like of the performance module may be determined.
By carrying out performance test, the performance test data of the storage module can be more accurately determined, the storage module can be tested regularly or irregularly, the performance test data stored at a specific position can be updated, and the performance of the storage module can be captured dynamically and in real time.
In one embodiment, before obtaining the performance test data of each memory module in the memory array at S210, the method may further include: performing performance test on the storage module in advance to obtain performance test data, and storing the performance test data at a specific position of the storage module; s210 may specifically include: performance test data stored at a particular location of the memory module is obtained.
Here, it is noted that the performance test data determined by performing the performance test in advance may be stored at a specific location of the memory module, and when it is necessary to partition each memory module, S220 may be performed by acquiring the performance parameter stored at the specific location. That is, after the performance test is performed, S220 is not necessarily performed immediately, and S220 may be performed after a command is needed or received, where the command may be an operation command of the memory array.
It should be noted that, since the performance test data stored at a specific location may be different from the actual performance test data of the memory module, in another embodiment, fig. 4 is a flowchart of a storage array management method according to another embodiment of the present invention. As shown in fig. 4, before the foregoing steps, the method may further include: s211, judging the accuracy of the performance test data information written in the specific position;
in this description, in S211, there are various specific ways to determine the accuracy of the performance test data information written in the specific location.
Sampling test can be carried out on certain performance test data of the storage module; the method may further include monitoring the read-write operation, and determining whether the read-write speed is too slow, or the delay time is too long, or the like, according to whether the expected effect is achieved when the memory module executes the read-write operation. And are not described in detail herein.
And S212, if the data is inaccurate, respectively carrying out performance test on each storage module according to a preset scene to respectively obtain performance test data of each storage module.
It should be noted that the performance test data obtained through the test of S212 can be directly used to execute step S220.
In addition, after S212, the method may further include: and updating the performance test data stored at the specific position on the storage module according to the performance test data obtained by the test.
It should be noted that the performance test data obtained through the test in S212 may be used to update the performance test data stored in the specific location of the memory module, and in the implementation process, the memory controller may execute step S220 according to the updated performance test data stored in the specific location by reading the updated performance test data.
In one embodiment, S210 may specifically include: performing read-write test on each storage module according to a prefabricated scene at a set time interval and/or when the system is idle; the prefabricated scene comprises sequential reading and writing, random reading and writing or reading and writing mixing; and respectively determining the performance test data of each storage module according to the test result.
In another embodiment, the memory array may be a full flash memory array, with the media of the memory module (flash module) including SLC media and MLC media; the specific different types of partitions are used for storing data with different liveness degrees, and the specific different types of partitions comprise: judging the media types of the storage modules belonging to the partition categories; if the partition formed by the storage modules with better performance test data comprises the storage module of the MLC medium, the storage module of the MLC medium is used for storing the data with higher activity; and if the partition formed by the storage modules with poor performance test data comprises the modules of the SLC medium, using the modules of the SLC medium to store the data with low activity.
In summary, according to the embodiments of the present invention, the partition types of each storage module are determined according to the performance of each storage module, and the partitions of different types are used to store data with different liveness degrees, so that the partition with better performance can be used to store the data with higher liveness degree, and the partition with the second performance can be used to store the data with lower liveness degree.
In the specific implementation process of the embodiment of the invention, the storage controller can read and write data of each partition more conveniently by performing tabulation management on the storage module and the corresponding partition type; by carrying out performance test, the performance test data of the storage module can be more accurately determined, the storage module can be tested regularly or irregularly, the performance test data stored at a specific position can be updated, and the performance of the storage module can be captured dynamically and in real time. When the performance test data at a specific position is particularly needed, the accuracy of the parameter can be judged firstly, and the quasi-determination of the performance test data used for determining each module partition type is ensured, so that each storage module in the storage array can be used more reasonably.
FIG. 5 is a diagram illustrating a storage array management apparatus according to an embodiment of the present invention. As shown in fig. 5, the apparatus may include:
an obtainingunit 510, configured to obtain performance test data of each memory module in the memory array; the storage array at least comprises one storage module;
and apartition unit 520, configured to determine a partition type to which each memory module belongs according to the performance test data of each memory module in the memory array, where different types of partitions are used to store data with different liveness.
Specifically, thepartition unit 520 may include:
the score determining subunit is used for determining the performance score of each storage module according to the performance test data of each storage module;
the sorting subunit is used for sorting the storage modules according to the scores;
and the partition subunit is used for determining the partition type of each storage module according to the sorting result.
In another implementation, the embodiment of the present invention may further include:
and the identification unit is used for adding the identification information of the storage module into the module identification list of the corresponding partition category.
The embodiment of the present invention may further include:
and the test unit is used for carrying out performance test on the storage module in advance to obtain performance test data, and storing the performance test data at a specific position of the storage module.
At this time, the obtainingunit 510 is specifically configured to obtain the performance test data stored in the specific location of the storage module.
Further, the embodiment of the present invention may further include:
the judging unit is used for judging the accuracy of the performance test data information stored at the specific position;
at this time, the test unit is further configured to, if the data is inaccurate, perform performance tests on the storage modules according to a preset scene, and obtain performance test data of the storage modules respectively.
Further, the embodiment of the present invention may further include:
and the updating unit is used for updating the performance test data stored at the specific position on the storage module according to the performance test data obtained by the test.
In another implementation manner, in this embodiment of the present invention, the obtainingunit 510 may include:
the read-write test subunit is used for performing read-write test on each storage module according to the preset scene at a set time interval and/or when the system is idle; the prefabricated scene comprises sequential reading and writing, random reading and writing or reading and writing mixing;
the data determining subunit is used for respectively determining the performance test data of each storage module according to the test result; the performance test data includes at least one of a number of read and write (I/O) operations per second, read and write bandwidths, and read and write delays.
Further, the memory array may be a full flash memory array, the media types of the memory modules may include SLC media as well as MLC media, thepartition unit 520 is specifically configured to,
judging the media types of the storage modules belonging to the partition categories;
if the partition formed by the storage modules with better performance test data comprises the storage module of the MLC medium, the storage module of the MLC medium is used for storing the data with higher activity;
and if the partition formed by the storage modules with poor performance test data comprises the modules of the SLC medium, using the modules of the SLC medium to store the data with low activity.
FIG. 6 is a schematic diagram of a memory array device according to an embodiment of the invention. As shown in fig. 6, the apparatus may include: memory controller 610,memory module 620. Thesystem bus 630 is used for communication between devices, such as a processor, operating the memory array and the memory controller 610.
The memory module may be a mechanical hard disk, SLC flash memory or MLC flash memory, etc.
At startup, the memory controller 610 loads the memory module driver, processes it for eachmemory module 620, and executes the following commands:
acquiring performance test data of eachstorage module 620 in the storage array;
according to the performance test data of eachstorage module 620 in the storage array, the partition type to which each storage module belongs is determined, wherein the partitions of different types are used for storing data with different activity degrees.
Specifically, it is possible to perform:
determining the performance score of eachstorage module 620 according to the performance test data of eachstorage module 620;
sorting thestorage modules 620 according to the scores;
and determining the partition type of eachstorage module 620 according to the sorting result.
In another implementation, the embodiment of the present invention may further perform:
the identification information of thestorage module 620 is added to the module identification list of the corresponding partition class.
The embodiment of the invention can also execute the following steps in concrete implementation:
the performance test data is obtained by performing a performance test on thestorage module 620 in advance, and the performance test data is stored in a specific position of thestorage module 620.
The performance test data stored at a specific location of thestorage module 620 is retrieved.
Further, the embodiment of the present invention may further perform:
determining an accuracy of the performance test data information stored 620 at the particular location;
if the data is inaccurate, the performance test is performed on eachstorage module 620 according to the preset scene, and the performance test data of eachstorage module 620 is obtained.
Further, the embodiment of the present invention may further perform:
and updating the performance test data stored at the specific position on thestorage module 620 according to the performance test data obtained by the test.
In another implementation manner, in the embodiment of the present invention, it may be performed that:
performing read-write test on eachstorage module 620 according to the prefabricated scene at a set time interval and/or when the system is idle; the prefabricated scene comprises sequential reading and writing, random reading and writing or reading and writing mixing;
respectively determining performance test data of eachstorage module 620 according to the test result; the performance test data includes at least one of a number of read and write (I/O) operations per second, read and write bandwidths, and read and write delays.
Further, the memory array may be a full flash memory array, the media types of thememory module 620 may include SLC media and MLC media, and embodiments of the present invention may also be implemented,
determining the media type of thestorage module 620 belonging to each partition class;
if the partition formed by thestorage modules 620 with better performance test data comprises the storage modules of the MLC medium, thestorage modules 620 of the MLC medium are used for storing the data with higher activity;
if the partition composed of the memory modules with poor performance test data includes the memory module of SLC medium, thememory module 620 of SLC medium is used for storing the data with low activity.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, storage array, removable storage array, CD-ROM, or any other form of storage medium known in the art.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.