Movatterモバイル変換


[0]ホーム

URL:


CN102123171A - Storage virtualization method for mass storage management - Google Patents

Storage virtualization method for mass storage management
Download PDF

Info

Publication number
CN102123171A
CN102123171ACN2011100253618ACN201110025361ACN102123171ACN 102123171 ACN102123171 ACN 102123171ACN 2011100253618 ACN2011100253618 ACN 2011100253618ACN 201110025361 ACN201110025361 ACN 201110025361ACN 102123171 ACN102123171 ACN 102123171A
Authority
CN
China
Prior art keywords
storage
file
resource
node
bandwidth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011100253618A
Other languages
Chinese (zh)
Other versions
CN102123171B (en
Inventor
冯丹
施展
李勇
叶为民
赵恒�
周峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and TechnologyfiledCriticalHuazhong University of Science and Technology
Priority to CN 201110025361priorityCriticalpatent/CN102123171B/en
Publication of CN102123171ApublicationCriticalpatent/CN102123171A/en
Application grantedgrantedCritical
Publication of CN102123171BpublicationCriticalpatent/CN102123171B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Images

Landscapes

Abstract

Translated fromChinese

一种面向海量存储管理的存储虚拟化方法,属于计算机存储技术领域,解决现有海量异构存储中存储资源高度复杂、规模庞大的问题,以提高存储资源利用率和降低存储系统的管理成本。本发明包括:(1)存储资源发现步骤;(2)存储资源代理步骤;(3)存储资源分配步骤。本发明通过整个存储空间里的存储容量和存储性能的合理分配,提高了存储资源利用率;通过独立的控制和数据传输路径,提高了数据传输的性能;通过为分散的企业存储提供集中的管理,降低了存储系统的管理成本,适用于面向海量异构存储系统的虚拟化管理。

Figure 201110025361

A storage virtualization method oriented to mass storage management, belonging to the field of computer storage technology, solves the problem of highly complex and large-scale storage resources in existing mass heterogeneous storage, so as to improve the utilization rate of storage resources and reduce the management cost of storage systems. The invention includes: (1) a storage resource discovery step; (2) a storage resource agent step; (3) a storage resource allocation step. The present invention improves the utilization rate of storage resources through reasonable allocation of storage capacity and storage performance in the entire storage space; improves the performance of data transmission through independent control and data transmission paths; provides centralized management for decentralized enterprise storage , which reduces the management cost of the storage system and is suitable for virtualization management of massive heterogeneous storage systems.

Figure 201110025361

Description

Translated fromChinese
一种面向海量存储管理的存储虚拟化方法A Storage Virtualization Method Oriented to Mass Storage Management

技术领域technical field

本发明属于计算机存储技术领域,具体涉及一种面向海量存储管理的存储虚拟化方法。The invention belongs to the technical field of computer storage, and in particular relates to a storage virtualization method for massive storage management.

背景技术Background technique

当前应用的海量存储系统是由存储节点、管理节点、用户节点构成的海量存储网络。各存储节点由不同厂商的存储设备和存储软件构成,互相之间存在不兼容的现象,功能区域划分复杂,设备、协议种类繁多,存储节点数量庞大,各种应用负载高度混合,导致存储系统管理复杂、资源浪费。虽然业内早已提出产品标准化的需求,但是从现实状况来看,对于异构存储环境的整合、管理依然很不理想,从而造成存储资源的大量浪费和重复投资,而且还影响了数据的一致性访问以及系统整体的备份容灾等诸多方面。The currently applied mass storage system is a mass storage network composed of storage nodes, management nodes, and user nodes. Each storage node is composed of storage devices and storage software from different manufacturers, and there are incompatibilities among them. The division of functional areas is complex, there are many types of equipment and protocols, the number of storage nodes is huge, and various application loads are highly mixed, resulting in storage system management. Complicated and wasteful of resources. Although the industry has long put forward the demand for product standardization, from the reality, the integration and management of heterogeneous storage environments are still not ideal, resulting in a large waste of storage resources and repeated investment, and also affects the consistent access of data And many aspects such as backup and disaster recovery of the system as a whole.

同时,海量异构存储系统具有以下特点:(1)异构性高,存储资源信息高度复杂,访问开销大;(2)规模庞大,超过单一的卷控制器处理能力,需要配置多台高性能的卷控制器设备。At the same time, massive heterogeneous storage systems have the following characteristics: (1) High heterogeneity, highly complex storage resource information, and high access overhead; (2) Huge scale, exceeding the processing capacity of a single volume controller, requiring multiple high-performance volume controller device.

存储虚拟化技术通常有设备级虚拟化、子系统级虚拟化、磁带虚拟化、块级虚拟化和文件级虚拟化几种类型,实现方法则包含带内和带外两种,带内是指数据传输与控制传输同路径,带外指数据传输与控制传输不同路径。其中设备级和子系统级虚拟化规模较小,多由固件或硬件方式实现,磁带虚拟化多见于备份系统,在海量存储系统中一般采用块级虚拟化和文件级虚拟化两种类型。传统的存储虚拟化方法多采用带内形式,便于集中控制和管理,以IBM的存储卷控制器为代表,其块级存储虚拟化解决方案多采用带内方式,在一定规模的同构存储网络环境中可以较好的组织存储资源,简化对这些资源的管理,向应用提供统一的虚拟存储设备,但是由于其所有虚拟化数据都要通过虚拟化节点进行处理,因此虚拟化节点很容易成为系统瓶颈,另一方面系统的扩展也受到虚拟化节点处理能力的限制,结合实现成本、扩展性、存储设备性能等多方面因素考虑,现有方案将难以有效实现对海量异构存储资源的管理。Storage virtualization technologies usually include device-level virtualization, subsystem-level virtualization, tape virtualization, block-level virtualization, and file-level virtualization. The implementation methods include in-band and out-of-band. In-band refers to Data transmission and control transmission are on the same path, and out-of-band refers to data transmission and control transmission on different paths. Among them, device-level and subsystem-level virtualization are small in scale and are mostly implemented by firmware or hardware. Tape virtualization is mostly seen in backup systems. In mass storage systems, two types of block-level virtualization and file-level virtualization are generally used. Traditional storage virtualization methods mostly use in-band, which is convenient for centralized control and management. Represented by IBM's storage volume controller, its block-level storage virtualization solutions mostly use in-band. Storage resources can be better organized in the environment, the management of these resources can be simplified, and unified virtual storage devices can be provided to applications. However, since all virtualized data must be processed by virtualized nodes, virtualized nodes can easily become system On the other hand, the expansion of the system is also limited by the processing capacity of virtualized nodes. Considering various factors such as implementation cost, scalability, and storage device performance, it will be difficult for existing solutions to effectively manage massive heterogeneous storage resources.

针对以上问题,用户亟需更加有效的面向海量存储管理的存储虚拟化方法,对海量异构存储系统进行动态管理,实现智能的带外方式块级虚拟化,为存储应用提供灵活、统一的平台。In response to the above problems, users urgently need a more effective storage virtualization method for mass storage management, dynamically manage massive heterogeneous storage systems, realize intelligent out-of-band block-level virtualization, and provide a flexible and unified platform for storage applications .

发明内容Contents of the invention

本发明提出一种面向海量存储管理的存储虚拟化方法,解决现有海量异构存储中存储资源高度复杂、规模庞大的问题,以提高存储资源利用率和降低存储系统的管理成本。The invention proposes a storage virtualization method oriented to mass storage management, which solves the problem of highly complex and large-scale storage resources in the existing mass heterogeneous storage, so as to improve the utilization rate of storage resources and reduce the management cost of the storage system.

本发明的一种面向海量存储管理的存储虚拟化方法,包括下列步骤:A storage virtualization method oriented to mass storage management of the present invention comprises the following steps:

(1)存储资源发现步骤,在各个存储节点上运行,顺序进行下述子步骤:(1) The storage resource discovery step runs on each storage node, and performs the following sub-steps in sequence:

(1.1)用XML文件描述本存储节点的属性,形成存储资源文件;存储资源文件包括存储节点名称、存储容量、带宽、网络地址和ID号;(1.1) Describe the attribute of this storage node with XML file, form storage resource file; Storage resource file comprises storage node name, storage capacity, bandwidth, network address and ID number;

(1.2)每隔T秒判断本存储节点是否正常工作,是则将存储资源文件发送到管理节点;否则不发送存储资源文件;(1.2) Determine whether the storage node is working normally every T seconds, and if so, send the storage resource file to the management node; otherwise, do not send the storage resource file;

(2)存储资源代理步骤,在有存储资源需求的用户节点上运行,顺序进行下述子步骤:(2) The storage resource agent step is run on the user node with storage resource requirements, and the following sub-steps are performed in sequence:

(2.1)用XML文件描述用户的存储资源需求,形成资源需求文件;资源需求文件包括用户节点名称、网络地址、使用的操作系统和ID号,所需求的虚拟存储设备的名称、容量、带宽和磁盘阵列等级;(2.1) Describe the user's storage resource requirements with an XML file to form a resource requirement file; the resource requirement file includes the user's node name, network address, operating system and ID number used, the name, capacity, bandwidth, and name of the required virtual storage device disk array class;

(2.2)将资源需求文件发送到管理节点,申请用户所需要的存储资源;(2.2) Send the resource requirement file to the management node, and apply for the storage resources required by the user;

(2.3)等待管理节点的资源分配文件,收到资源分配文件后,根据资源分配文件为用户创建虚拟存储设备;(2.3) wait for the resource allocation file of the management node, after receiving the resource allocation file, create a virtual storage device for the user according to the resource allocation file;

(2.4)检查虚拟存储设备是否创建成功,是则用XML文件描述存储资源使用情况,形成资源使用文件,发送给管理节点;否则用XML文件描述错误信息,发送给管理节点;资源使用文件包括已经分配虚拟存储设备的用户节点名称、网络地址、使用的操作系统和ID号,分配的存储虚拟设备名称、存储容量、带宽和ID号,所有存储节点名称、存储容量、带宽、网络地址和ID号,虚拟存储设备使用的存储节点存储容量、网络地址、带宽、剩余容量和ID号;(2.4) Check whether the virtual storage device is created successfully, if it is, describe the usage of storage resources with an XML file, form a resource usage file, and send it to the management node; otherwise describe the error message with an XML file, and send it to the management node; the resource usage file includes the The user node name, network address, operating system and ID number of the allocated virtual storage device, the allocated storage virtual device name, storage capacity, bandwidth and ID number, and all storage node names, storage capacity, bandwidth, network address and ID number , the storage capacity, network address, bandwidth, remaining capacity and ID number of the storage node used by the virtual storage device;

(3)存储资源分配步骤,在管理节点上运行,包括下述子步骤:(3) The storage resource allocation step is run on the management node, including the following sub-steps:

(3.1)每隔T秒判断是否接收到来自各存储节点发送的存储资源文件,是则将所接收的存储资源文件更新到数据库中;否则认为该存储节点失效,删除对应的存储资源文件,T=3、4、5、6、7或8;(3.1) Every T seconds, it is judged whether the storage resource file sent from each storage node is received, and if so, the received storage resource file is updated to the database; otherwise, the storage node is considered invalid, and the corresponding storage resource file is deleted, T = 3, 4, 5, 6, 7 or 8;

所述更新是指如果数据库中存在旧存储资源文件,则替换该旧存储资源文件,数据库中不存在旧存储资源文件,则保存该存储资源文件;The update means that if there is an old storage resource file in the database, then replace the old storage resource file, and if there is no old storage resource file in the database, then save the storage resource file;

(3.2)接收来自各用户节点发送的资源需求文件,检索数据库中的存储资源文件,进行子步骤(3.3);(3.2) Receive the resource requirement file sent from each user node, retrieve the stored resource file in the database, and perform substep (3.3);

(3.3)寻找存储资源,用XML文件描述,形成资源分配文件,进行子步骤(3.4);资源分配文件包括分配的虚拟存储设备名称、存储容量、使用存储节点数量、磁盘阵列等级和用户节点ID号,虚拟存储设备使用的存储节点ID号、存储容量、网络地址、带宽和使用容量;(3.3) Look for storage resources, describe with XML file, form resource allocation file, carry out substep (3.4); Resource allocation file includes the virtual storage device name of allocation, storage capacity, use storage node quantity, disk array level and user node ID ID number, storage node ID number, storage capacity, network address, bandwidth and capacity used by the virtual storage device;

(3.4)将资源分配文件发送给用户节点,进行子步骤(3.5);(3.4) send the resource allocation file to the user node, and carry out sub-step (3.5);

(3.5)接收来自各用户节点发送的资源使用文件,更新到数据库中。(3.5) Receive resource usage files sent from each user node and update them into the database.

所述存储资源分配步骤中,子步骤(3.3)顺序进行下述过程:In the step of allocating storage resources, the sub-step (3.3) performs the following process in sequence:

A.扫描数据库中的存储资源文件,判断存储资源文件中各存储资源的带宽是否大于或等于资源需求文件要求的带宽,是则将符合要求的存储资源,按带宽从小到大排列生成链表L,转过程B;否则选择最接近资源需求文件要求的带宽的存储资源,生成链表L,转过程B;A. Scan the storage resource files in the database to determine whether the bandwidth of each storage resource in the storage resource file is greater than or equal to the bandwidth required by the resource requirement file. If so, arrange the storage resources that meet the requirements in ascending order of bandwidth to generate a linked list L, Transfer to process B; otherwise, select the storage resource closest to the bandwidth required by the resource requirement file, generate a linked list L, and transfer to process B;

B.从链表L的表头开始,判断链表L中节点的存储容量是否大于或等于资源需求文件要求的存储容量,是则选取该节点,转过程C;否则将原资源需求文件要求的存储容量减去该节点的存储容量作为资源需求文件要求的存储容量,将原链表L表头节点的下一节点作为链表L的表头,转过程B;B. Starting from the head of the linked list L, judge whether the storage capacity of the node in the linked list L is greater than or equal to the storage capacity required by the resource requirement file, if so, select the node and go to process C; otherwise, the storage capacity required by the original resource requirement file Subtract the storage capacity of this node as the storage capacity required by the resource requirements file, and use the next node of the head node of the original linked list L as the header of the linked list L, and turn to process B;

C.将链表L中最后选取的节点的存储容量、带宽及网络地址,用XML文件描述,形成资源分配文件。C. Describe the storage capacity, bandwidth and network address of the last selected node in the linked list L with an XML file to form a resource allocation file.

存储资源发现步骤:在系统运行期间,系统管理员采集需要新加入系统的存储资源的属性信息,周期性地向管理节点报告存储节点存活信息,用于将存储资源加入到系统;管理节点通过存活信息检测存储资源是否因故障等原因停止使用,避免错误使用存储资源。Storage resource discovery step: During system operation, the system administrator collects attribute information of storage resources that need to be added to the system, and periodically reports storage node survival information to the management node for adding storage resources to the system; Information checks whether the storage resources are out of use due to failure or other reasons, so as to avoid wrong use of storage resources.

存储资源代理步骤:用户将自己的存储需求发送给管理节点,申请可以满足用户需要的存储资源;当接收到管理节点的反馈信息后,读取反馈信息中资源分配文件,通过资源分配文件为用户创建虚拟存储设备。Storage resource agent step: the user sends his storage requirements to the management node, and applies for storage resources that can meet the user's needs; after receiving the feedback information from the management node, read the resource allocation file in the feedback information, and use the resource allocation file for the user Create a virtual storage device.

存储资源分配步骤:用于集中保存和维护海量存储系统中的存储资源信息以及虚拟设备信息,并进行存储资源分配。管理节点和用户节点的传输路径是控制路径,和数据路径独立开来,这样能够减轻虚拟存储设备的数据传输压力,提高虚拟存储设备的性能。Storage resource allocation step: used to centrally save and maintain storage resource information and virtual device information in the mass storage system, and perform storage resource allocation. The transmission path of the management node and the user node is a control path, which is independent from the data path, which can reduce the data transmission pressure of the virtual storage device and improve the performance of the virtual storage device.

本发明通过整个存储空间里的存储容量和存储性能的合理分配,提高了存储资源利用率;通过独立的控制和数据传输路径,提高了数据传输的性能;通过为分散的企业存储提供集中的管理,降低了存储系统的管理成本,适用于面向海量异构存储系统的虚拟化管理。The present invention improves the utilization rate of storage resources through reasonable allocation of storage capacity and storage performance in the entire storage space; improves the performance of data transmission through independent control and data transmission paths; provides centralized management for decentralized enterprise storage , which reduces the management cost of the storage system and is suitable for virtualization management of massive heterogeneous storage systems.

附图说明Description of drawings

图1为本发明实施例的示意图;Fig. 1 is the schematic diagram of the embodiment of the present invention;

图2为本发明实施例使用的存储系统示意图;FIG. 2 is a schematic diagram of a storage system used in an embodiment of the present invention;

图3为存储资源发现步骤示意图;FIG. 3 is a schematic diagram of storage resource discovery steps;

图4为存储资源文件XML格式示意图;Fig. 4 is a schematic diagram of XML format for storing resource files;

图5为存储资源代理步骤示意图;Fig. 5 is a schematic diagram of storage resource agent steps;

图6为资源需求文件XML格式示意图;Fig. 6 is a schematic diagram of the XML format of the resource requirement file;

图7为资源使用文件XML格式示意图;FIG. 7 is a schematic diagram of the resource usage file XML format;

图8为存储资源分配步骤示意图;FIG. 8 is a schematic diagram of storage resource allocation steps;

图9为资源分配文件XML格式示意图;Fig. 9 is a schematic diagram of the resource allocation file XML format;

图10为存储资源分配步骤的子步骤(3.3)示意图。Fig. 10 is a schematic diagram of the sub-step (3.3) of the storage resource allocation step.

具体实施方式Detailed ways

如图1所示,本发明包括:(1)存储资源发现步骤;(2)存储资源代理步骤;(3)存储资源分配步骤。As shown in Fig. 1, the present invention includes: (1) a storage resource discovery step; (2) a storage resource agent step; (3) a storage resource allocation step.

如图2所示,假设本发明实施例使用的存储系统包括存储节点、用户节点和管理节点,存储节点为光纤通道存储设备、因特网小型计算机系统接口存储设备、无限带宽存储设备,分别用存储节点D1、D2和D3表示,用户节点包括用户节点H1、H2和H3,管理节点为M1,用户节点H1需要创建虚拟存储设备LV1跨越存储节点D1和D2。As shown in Figure 2, it is assumed that the storage system used in the embodiment of the present invention includes a storage node, a user node and a management node, and the storage node is a fiber channel storage device, an Internet small computer system interface storage device, and an InfiniBand storage device, and the storage node D1, D2, and D3 indicate that the user nodes include user nodes H1, H2, and H3, the management node is M1, and user node H1 needs to create a virtual storage device LV1 spanning storage nodes D1 and D2.

本发明实施例包括:Embodiments of the invention include:

(1)存储资源发现步骤,在各个存储节点上运行,如图3所示,顺序进行下述子步骤:(1) The storage resource discovery step is run on each storage node, as shown in Figure 3, the following sub-steps are performed in sequence:

(1.1)用XML文件描述存储节点D1、D2和D3的属性,包括存储容量、带宽、网络地址,形成存储资源文件,如图4所示;(1.1) describe the attribute of storage node D1, D2 and D3 with XML file, comprise storage capacity, bandwidth, network address, form storage resource file, as shown in Figure 4;

(1.2)每隔5秒存储节点D1、D2和D3判断自身是否正常工作,是则将存储资源文件发送到管理节点;否则不发送存储资源文件;(1.2) Storage nodes D1, D2 and D3 judge whether they are working normally every 5 seconds, and if so, send the storage resource file to the management node; otherwise, do not send the storage resource file;

(2)存储资源代理步骤,在用户节点H1上运行,如图5所示,顺序进行下述子步骤:(2) The storage resource agent step is run on the user node H1, as shown in Figure 5, the following sub-steps are performed in sequence:

(2.1)用XML文件描述用户节点H1的存储资源需求,形成资源需求文件,如图6所示;(2.1) Describe the storage resource requirements of the user node H1 with an XML file to form a resource requirement file, as shown in Figure 6;

(2.2)将资源需求文件发送到管理节点M1,申请用户节点H1所需要的存储资源;(2.2) Send the resource requirement file to the management node M1, and apply for the storage resources required by the user node H1;

(2.3)等待管理节点M1的资源分配文件,收到资源分配文件后,根据资源分配文件为用户节点H1创建虚拟存储设备LV1;(2.3) Waiting for the resource allocation file of the management node M1, after receiving the resource allocation file, creating a virtual storage device LV1 for the user node H1 according to the resource allocation file;

(2.4)检查虚拟存储设备LV1是否创建成功,是则用XML文件描述存储资源使用情况,形成资源使用文件,发送给管理节点M1;否则用XML文件描述错误信息,发送给管理节点M1;资源使用文件如图7所示;(2.4) Check whether the virtual storage device LV1 is created successfully. If yes, use an XML file to describe the storage resource usage, form a resource usage file, and send it to the management node M1; otherwise, use an XML file to describe the error message and send it to the management node M1; resource usage The file is shown in Figure 7;

(3)存储资源分配步骤,在管理节点M1上运行,如图8所示,顺序进行下述子步骤:(3) The storage resource allocation step is run on the management node M1, as shown in Figure 8, the following sub-steps are performed in sequence:

(3.1)判断每隔5秒是否接收到来自存储节点D1、D2、D3发送的存储资源文件,是则将所接收的存储资源文件更新到数据库中;否则认为该存储节点失效,删除对应的存储资源文件;(3.1) Determine whether the storage resource files sent from the storage nodes D1, D2, and D3 are received every 5 seconds, and if so, update the received storage resource files to the database; otherwise, the storage node is considered invalid and the corresponding storage node is deleted resource;

所述更新是指如果数据库中存在旧存储资源文件,则替换该旧存储资源文件,数据库中不存在旧存储资源文件,则保存该存储资源文件;The update means that if there is an old storage resource file in the database, then replace the old storage resource file, and if there is no old storage resource file in the database, then save the storage resource file;

(3.2)接收来自用户节点H1发送的资源需求文件,检索数据库中的存储资源文件,(3.2) Receive the resource requirement file sent from user node H1, retrieve the storage resource file in the database,

(3.3)寻找存储资源,用XML文件描述,形成资源分配文件,如图9所示;(3.3) Look for storage resources, describe them with XML files, and form resource allocation files, as shown in Figure 9;

(3.4)将资源分配文件发送给用户节点H1;(3.4) Send the resource allocation file to the user node H1;

(3.5)接收来自用户节点H1、H2、H3发送的资源使用文件,更新到数据库中。(3.5) Receive resource usage files sent from user nodes H1, H2, and H3, and update them into the database.

如图10所示,存储资源分配步骤的子步骤(3.3)顺序进行下述过程:As shown in Figure 10, the sub-step (3.3) of the storage resource allocation step performs the following process in sequence:

A.扫描数据库中的存储资源文件,判断存储资源文件中各存储资源的带宽是否大于或等于资源需求文件要求的带宽,是则将符合要求的存储资源,按带宽从小到大排列生成链表L,转过程B;否则选择最接近资源需求文件要求的带宽的存储资源,生成链表L,转过程B;A. Scan the storage resource files in the database to determine whether the bandwidth of each storage resource in the storage resource file is greater than or equal to the bandwidth required by the resource requirement file. If so, arrange the storage resources that meet the requirements in ascending order of bandwidth to generate a linked list L, Transfer to process B; otherwise, select the storage resource closest to the bandwidth required by the resource requirement file, generate a linked list L, and transfer to process B;

B.从链表L的表头开始,判断链表L中节点的存储容量是否大于或等于资源需求文件要求的存储容量,是则选取该节点,转过程C;否则将原资源需求文件要求的存储容量减去该节点的存储容量作为资源需求文件要求的存储容量,将原链表L表头节点的下一节点作为链表L的表头,转过程B;B. Starting from the head of the linked list L, judge whether the storage capacity of the node in the linked list L is greater than or equal to the storage capacity required by the resource requirement file, if so, select the node and go to process C; otherwise, the storage capacity required by the original resource requirement file Subtract the storage capacity of this node as the storage capacity required by the resource requirements file, and use the next node of the head node of the original linked list L as the header of the linked list L, and turn to process B;

C.将链表L中最后选取的节点的存储容量、带宽及网络地址,用XML文件描述,形成资源分配文件。C. Describe the storage capacity, bandwidth and network address of the last selected node in the linked list L with an XML file to form a resource allocation file.

图4为存储资源文件XML格式示意图:元素LVML定义文件为一种面向海量存储管理的存储虚拟化方法的XML格式文件,其中属性UUID定义该XML格式文件的ID号,属性description定义该XML格式文件的描述信息,属性version定义该XML格式文件的版本号;元素DiskInfo定义该XML格式文件为存储资源文件;元素Disk定义一个存储节点,其中,属性DiskName定义存储节点的名称,属性DiskSize定义存储节点的容量,属性DiskIP定义存储节点的网络地址,属性Bandwidth定义带宽信息,属性UUID定义存储节点的ID号。Figure 4 is a schematic diagram of the storage resource file XML format: the element LVML definition file is an XML format file for a storage virtualization method oriented to mass storage management, wherein the attribute UUID defines the ID number of the XML format file, and the attribute description defines the XML format file The attribute version defines the version number of the XML format file; the element DiskInfo defines the XML format file as a storage resource file; the element Disk defines a storage node, where the attribute DiskName defines the name of the storage node, and the attribute DiskSize defines the size of the storage node Capacity, the attribute DiskIP defines the network address of the storage node, the attribute Bandwidth defines the bandwidth information, and the attribute UUID defines the ID number of the storage node.

图6为资源需求文件XML格式示意图:元素LvmReq定义该XML格式文件为资源需求文件;元素Host定义一个用户节点,其中属性Format定义用户节点使用的操作系统,值为1是Linux操作系统,值为2是Windows操作系统,属性HostIP定义用户节点的网络地址,属性HostName定义用户节点的名称,属性UUID定义用户节点的ID号;元素LogicVolume定义用户节点需求的虚拟存储设备,其中属性LVName定义虚拟存储设备的名称,属性LVSize定义虚拟存储设备的容量,属性Raid定义虚拟存储设备的磁盘阵列等级,值为-1是非磁盘阵列,值为0为RAID0,值为1是RAID1,值为5是RAID5。Figure 6 is a schematic diagram of the resource requirement file XML format: the element LvmReq defines the XML format file as a resource requirement file; the element Host defines a user node, and the attribute Format defines the operating system used by the user node, the value is 1 for the Linux operating system, and the value is 2 is the Windows operating system, the attribute HostIP defines the network address of the user node, the attribute HostName defines the name of the user node, the attribute UUID defines the ID number of the user node; the element LogicVolume defines the virtual storage device required by the user node, and the attribute LVName defines the virtual storage device The attribute LVSize defines the capacity of the virtual storage device, and the attribute Raid defines the disk array level of the virtual storage device. A value of -1 means non-disk array, a value of 0 means RAID0, a value of 1 means RAID1, and a value of 5 means RAID5.

图7为资源使用文件XML格式示意图:元素LvmInfo定义该XML格式文件为资源使用文件;元素LogicVolume定义创建的虚拟存储设备,其中属性HostUUID定义使用该虚拟存储设备的用户节点ID号,属性UUID定义虚拟存储设备的ID号;元素Disk一个存储节点,其中属性FreeSize定义存储节点未使用的容量Figure 7 is a schematic diagram of the resource usage file XML format: the element LvmInfo defines the XML format file as a resource usage file; the element LogicVolume defines the created virtual storage device, in which the attribute HostUUID defines the user node ID number that uses the virtual storage device, and the attribute UUID defines the virtual storage device. The ID number of the storage device; the element Disk is a storage node, and the attribute FreeSize defines the unused capacity of the storage node

图9为资源分配文件XML格式示意图:元素DiskAllo定义该XML格式文件为资源分配文件;元素LogicVolume定义分配的虚拟存储设备,其中属性DiskNum定义虚拟存储设备使用的存储节点个数;元素Disk定义分配的存储节点,其中属性UsedSize定义分配给虚拟存储设备使用的存储节点容量。Figure 9 is a schematic diagram of the resource allocation file XML format: the element DiskAllo defines the XML format file as a resource allocation file; the element LogicVolume defines the allocated virtual storage device, and the attribute DiskNum defines the number of storage nodes used by the virtual storage device; the element Disk defines the allocated A storage node, where the attribute UsedSize defines the capacity of the storage node allocated to the virtual storage device.

Claims (2)

Translated fromChinese
1.一种面向海量存储管理的存储虚拟化方法,包括下列步骤:1. A storage virtualization method for mass storage management, comprising the following steps:(1)存储资源发现步骤,在各个存储节点上运行,顺序进行下述子步骤:(1) The storage resource discovery step is run on each storage node, and the following sub-steps are performed in sequence:(1.1)用XML文件描述本存储节点的属性,形成存储资源文件;存储资源文件包括存储节点名称、存储容量、带宽、网络地址和ID号;(1.1) Use an XML file to describe the attributes of the storage node to form a storage resource file; the storage resource file includes the name of the storage node, storage capacity, bandwidth, network address and ID number;(1.2)每隔T秒判断本存储节点是否正常工作,是则将存储资源文件发送到管理节点;否则不发送存储资源文件;(1.2) Determine whether the storage node is working normally every T seconds, and if so, send the storage resource file to the management node; otherwise, do not send the storage resource file;(2)存储资源代理步骤,在有存储资源需求的用户节点上运行,顺序进行下述子步骤:(2) The storage resource agent step is run on the user node with storage resource requirements, and the following sub-steps are performed in sequence:(2.1)用XML文件描述用户的存储资源需求,形成资源需求文件;资源需求文件包括用户节点名称、网络地址、使用的操作系统和ID号,所需求的虚拟存储设备的名称、容量、带宽和磁盘阵列等级;(2.1) Use an XML file to describe the user's storage resource requirements to form a resource requirement file; the resource requirement file includes the user node name, network address, operating system and ID number used, the name, capacity, bandwidth and disk array class;(2.2)将资源需求文件发送到管理节点,申请用户所需要的存储资源;(2.2) Send the resource requirement file to the management node, and apply for the storage resources required by the user;(2.3)等待管理节点的资源分配文件,收到资源分配文件后,根据资源分配文件为用户创建虚拟存储设备;(2.3) Waiting for the resource allocation file of the management node, after receiving the resource allocation file, create a virtual storage device for the user according to the resource allocation file;(2.4)检查虚拟存储设备是否创建成功,是则用XML文件描述存储资源使用情况,形成资源使用文件,发送给管理节点;否则用XML文件描述错误信息,发送给管理节点;资源使用文件包括已经分配虚拟存储设备的用户节点名称、网络地址、使用的操作系统和ID号,分配的存储虚拟设备名称、存储容量、带宽和ID号,所有存储节点名称、存储容量、带宽、网络地址和ID号,虚拟存储设备使用的存储节点存储容量、网络地址、带宽、剩余容量和ID号;(2.4) Check whether the virtual storage device is successfully created. If yes, describe the usage of storage resources in an XML file, form a resource usage file, and send it to the management node; otherwise, describe the error information in an XML file and send it to the management node; the resource usage file includes The user node name, network address, operating system and ID number of the allocated virtual storage device, the allocated storage virtual device name, storage capacity, bandwidth and ID number, and all storage node names, storage capacity, bandwidth, network address and ID number , the storage capacity, network address, bandwidth, remaining capacity and ID number of the storage node used by the virtual storage device;(3)存储资源分配步骤,在管理节点上运行,包括下述子步骤:(3) The storage resource allocation step is run on the management node, including the following sub-steps:(3.1)每隔T秒判断是否接收到来自各存储节点发送的存储资源文件,是则将所接收的存储资源文件更新到数据库中;否则认为该存储节点失效,删除对应的存储资源文件,T=3、4、5、6、7或8;(3.1) Every T seconds, it is judged whether the storage resource file sent by each storage node is received, and if so, the received storage resource file is updated to the database; otherwise, the storage node is considered invalid, and the corresponding storage resource file is deleted, T = 3, 4, 5, 6, 7 or 8;所述更新是指如果数据库中存在旧存储资源文件,则替换该旧存储资源文件,数据库中不存在旧存储资源文件,则保存该存储资源文件;The update means that if there is an old storage resource file in the database, then replace the old storage resource file, and if there is no old storage resource file in the database, then save the storage resource file;(3.2)接收来自各用户节点发送的资源需求文件,检索数据库中的存储资源文件,进行子步骤(3.3);(3.2) Receive the resource requirement files sent by each user node, retrieve the stored resource files in the database, and perform sub-step (3.3);(3.3)寻找存储资源,用XML文件描述,形成资源分配文件,进行子步骤(3.4);资源分配文件包括分配的虚拟存储设备名称、存储容量、使用存储节点数量、磁盘阵列等级和用户节点ID号,虚拟存储设备使用的存储节点ID号、存储容量、网络地址、带宽和使用容量;(3.3) Look for storage resources, describe them with XML files, form a resource allocation file, and proceed to sub-step (3.4); the resource allocation file includes the name of the allocated virtual storage device, storage capacity, number of storage nodes used, disk array level and user node ID ID number, storage node ID number, storage capacity, network address, bandwidth and capacity used by the virtual storage device;(3.4)将资源分配文件发送给用户节点,进行子步骤(3.5);(3.4) Send the resource allocation file to the user node and proceed to sub-step (3.5);(3.5)接收来自各用户节点发送的资源使用文件,更新到数据库中。(3.5) Receive resource usage files sent from each user node and update them into the database.2.如权利要求1所述的面向海量存储管理的存储虚拟化方法,其特征在于:2. The storage virtualization method for massive storage management as claimed in claim 1, characterized in that:所述存储资源分配步骤中,子步骤(3.3)顺序进行下述过程:In the step of allocating storage resources, the sub-step (3.3) sequentially performs the following processes:A.扫描数据库中的存储资源文件,判断存储资源文件中各存储资源的带宽是否大于或等于资源需求文件要求的带宽,是则将符合要求的存储资源,按带宽从小到大排列生成链表L,转过程B;否则选择最接近资源需求文件要求的带宽的存储资源,生成链表L,转过程B;A. Scan the storage resource files in the database to determine whether the bandwidth of each storage resource in the storage resource file is greater than or equal to the bandwidth required by the resource requirement file. If so, arrange the storage resources that meet the requirements in ascending order of bandwidth to generate a linked list L, Transfer to process B; otherwise, select the storage resource closest to the bandwidth required by the resource requirement file, generate a linked list L, and transfer to process B;B.从链表L的表头开始,判断链表L中节点的存储容量是否大于或等于资源需求文件要求的存储容量,是则选取该节点,转过程C;否则将原资源需求文件要求的存储容量减去该节点的存储容量作为资源需求文件要求的存储容量,将原链表L表头节点的下一节点作为链表L的表头,转过程B; B. Starting from the head of the linked list L, judge whether the storage capacity of the node in the linked list L is greater than or equal to the storage capacity required by the resource requirement file, if so, select the node and go to process C; otherwise, the storage capacity required by the original resource requirement file Subtract the storage capacity of this node as the storage capacity required by the resource requirements file, and use the next node of the head node of the original linked list L as the header of the linked list L, and turn to process B;C.将链表L中最后选取的节点的存储容量、带宽及网络地址,用XML文件描述,形成资源分配文件。C. Describe the storage capacity, bandwidth and network address of the last selected node in the linked list L with an XML file to form a resource allocation file.
CN 2011100253612011-01-242011-01-24Storage virtualization method for mass storage managementActiveCN102123171B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN 201110025361CN102123171B (en)2011-01-242011-01-24Storage virtualization method for mass storage management

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN 201110025361CN102123171B (en)2011-01-242011-01-24Storage virtualization method for mass storage management

Publications (2)

Publication NumberPublication Date
CN102123171Atrue CN102123171A (en)2011-07-13
CN102123171B CN102123171B (en)2013-06-05

Family

ID=44251622

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN 201110025361ActiveCN102123171B (en)2011-01-242011-01-24Storage virtualization method for mass storage management

Country Status (1)

CountryLink
CN (1)CN102123171B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN103036952A (en)*2012-11-302013-04-10中国电力科学研究院Enterprise-level heterogeneous fusion memory management system
CN107636598A (en)*2015-07-152018-01-26思科技术公司 Bid/Ask Protocol in Scale-Out NVMe Storage
CN111399761A (en)*2019-11-192020-07-10杭州海康威视系统技术有限公司Storage resource allocation method, device and equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN1955912A (en)*2006-10-132007-05-02清华大学 Resource Allocation Method in Mass Storage System
CN101496005A (en)*2005-12-292009-07-29亚马逊科技公司Distributed storage system with web services client interface

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101496005A (en)*2005-12-292009-07-29亚马逊科技公司Distributed storage system with web services client interface
CN1955912A (en)*2006-10-132007-05-02清华大学 Resource Allocation Method in Mass Storage System

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
《Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2006. SNPD 2006. Seventh ACIS International Conference on》 20060620 Jie Xiao,et al Flexible Metadata Management for Block-level Storage System 第127-132页 ,*
JIE XIAO,ET AL: "Flexible Metadata Management for Block–level Storage System", 《SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE, NETWORKING, AND PARALLEL/DISTRIBUTED COMPUTING, 2006. SNPD 2006. SEVENTH ACIS INTERNATIONAL CONFERENCE ON》*
冯 丹,叶 飞,施 展: "VISA 存储网络系统元数据管理的研究", 《华 中 科 技 大 学 学 报(自然科学版)》*

Cited By (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN103036952A (en)*2012-11-302013-04-10中国电力科学研究院Enterprise-level heterogeneous fusion memory management system
CN103036952B (en)*2012-11-302016-04-20国家电网公司A kind of enterprise-level isomery merges storage management system
CN107636598A (en)*2015-07-152018-01-26思科技术公司 Bid/Ask Protocol in Scale-Out NVMe Storage
CN107636598B (en)*2015-07-152021-06-01思科技术公司 Bid/Ask Agreement in Scale-Out NVMe Storage
CN111399761A (en)*2019-11-192020-07-10杭州海康威视系统技术有限公司Storage resource allocation method, device and equipment and storage medium

Also Published As

Publication numberPublication date
CN102123171B (en)2013-06-05

Similar Documents

PublicationPublication DateTitle
US11226777B2 (en)Cluster configuration information replication
US11016864B2 (en)Cluster-wide service agents
US11182096B1 (en)Data storage system with configurable durability
CN108139941B (en) Dynamic Resource Allocation Based on Network Flow Control
CN104156216B (en)A kind of memory management system and method towards cloud computing
US20150381426A1 (en)Dynamically composed compute nodes comprising disaggregated components
US10826812B2 (en)Multiple quorum witness
US20160085606A1 (en)Cluster-wide outage detection
CN102123176B (en)Space distribution and management method and device for network storage system
CN103118073B (en)Virtual machine data persistence storage system and method in cloud environment
CN105278874A (en)Big data platform system and running method therefor
CN109547537A (en)The method for realizing openstack High Availabitity based on SAN storage shared volume
CN103036952B (en)A kind of enterprise-level isomery merges storage management system
CN111666034B (en) A container cluster disk management method and device
CN102123171A (en)Storage virtualization method for mass storage management
CN105005515B (en)A kind of LAN free standby systems
US20100057989A1 (en)Method of moving data in logical volume, storage system, and administrative computer
WO2016109743A1 (en)Systems and methods for implementing stretch clusters in a virtualization environment
KR20120063946A (en)Memory apparatus for collective volume memory and metadate managing method thereof
CN112398668B (en)IaaS cluster-based cloud platform and node switching method
US10768834B2 (en)Methods for managing group objects with different service level objectives for an application and devices thereof
CN114546580A (en) Cache deployment system, cache deployment method, electronic device and storage medium
CN107391236B (en) A method and device for realizing cluster block storage
Lee et al.ECSim-2: A Performance Evaluator for Erasure Code based Storage Systems
KR20140060962A (en)System and method for providing multiple service using idle resource of network distributed file system

Legal Events

DateCodeTitleDescription
C06Publication
PB01Publication
C10Entry into substantive examination
SE01Entry into force of request for substantive examination
C14Grant of patent or utility model
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp