Movatterモバイル変換


[0]ホーム

URL:


CN115543961A - A DAOS-based data cross-protection domain migration method and device - Google Patents

A DAOS-based data cross-protection domain migration method and device
Download PDF

Info

Publication number
CN115543961A
CN115543961ACN202211137909.2ACN202211137909ACN115543961ACN 115543961 ACN115543961 ACN 115543961ACN 202211137909 ACN202211137909 ACN 202211137909ACN 115543961 ACN115543961 ACN 115543961A
Authority
CN
China
Prior art keywords
migration
protection domain
daos
time point
pds
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211137909.2A
Other languages
Chinese (zh)
Inventor
孔令杰
余攀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CLP Cloud Digital Intelligence Technology Co Ltd
Original Assignee
CLP Cloud Digital Intelligence Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CLP Cloud Digital Intelligence Technology Co LtdfiledCriticalCLP Cloud Digital Intelligence Technology Co Ltd
Priority to CN202211137909.2ApriorityCriticalpatent/CN115543961A/en
Publication of CN115543961ApublicationCriticalpatent/CN115543961A/en
Pendinglegal-statusCriticalCurrent

Links

Images

Classifications

Landscapes

Abstract

Translated fromChinese

本申请公开了一种基于DAOS的数据跨保护域迁移方法及设备,包括:在创建DAOS池的情况下,创建数个保护域分片(PDS),其中任一PDS对应于一个保护域,且多个PDS可映射到同一个保护域上;将PDS跨保护域的迁移流程分为单写阶段和双写阶段:在所述单写阶段,对任一PDS的对象写请求,仅落到对应的保护域上,以进行多轮数据迁移;在所述双写阶段,对任一PDS的对象写请求,同时落到所述任一PDS以及对应的保护域上,以仅进行一轮数据迁移。本申请实施例对象跨保护域迁移以PDS为粒度,方便减少计算量以及对象迁移的管理,实现了兼顾数据的完整性和准确性、业务的连续性以及兼顾性能。

Figure 202211137909

The present application discloses a DAOS-based data cross-protection domain migration method and device, including: in the case of creating a DAOS pool, creating several protection domain fragments (PDS), wherein any PDS corresponds to a protection domain, and Multiple PDSs can be mapped to the same protection domain; the migration process of PDSs across protection domains is divided into single-write phase and double-write phase: in the single-write phase, the object write request for any PDS only falls to the corresponding In the protection domain, multiple rounds of data migration are performed; in the double-write phase, the object write request for any PDS falls on the any PDS and the corresponding protection domain at the same time, so that only one round of data migration is performed . The object migration across protection domains in the embodiment of the present application takes PDS as the granularity, which facilitates the reduction of calculation amount and the management of object migration, and achieves both data integrity and accuracy, business continuity, and performance.

Figure 202211137909

Description

Translated fromChinese
一种基于DAOS的数据跨保护域迁移方法及设备A DAOS-based data cross-protection domain migration method and device

技术领域technical field

本申请涉及分布式存储技术领域,尤其涉及一种基于DAOS的数据跨保护域迁移方法及设备。The present application relates to the technical field of distributed storage, and in particular to a DAOS-based data cross-protection domain migration method and device.

背景技术Background technique

DAOS是一款轻量级的分布式对象存储软件,DAOS为了提高可靠性以及提高重构的性能,将对象的复制组(copy set)限制在保护域内,当出现扩容或者缩容的场景的时候,为了让集群能达到容量的动态均衡,所以数据需要进行跨保护域的迁移,DAOS本身有数据重构的功能,在数据重构的过程中,为了保证业务不中断,以及数据的完整性和正确性,重构的过程中会扩大数据的复制组,也就是原复制组和新映射的复制组。在跨保护域的场景下,数据的复制组会翻倍,由于DAOS是强一致性的事务模型,翻倍的复制组对性能非常不友好。DAOS is a lightweight distributed object storage software. In order to improve the reliability and improve the performance of reconstruction, DAOS limits the copy set of the object to the protection domain. When there is a scenario of expansion or contraction , in order for the cluster to achieve dynamic balance of capacity, data needs to be migrated across protection domains. DAOS itself has the function of data reconstruction. In the process of data reconstruction, in order to ensure uninterrupted business, data integrity and Correctness, the data replication group will be expanded during the reconstruction process, that is, the original replication group and the newly mapped replication group. In the cross-protection domain scenario, the data replication group will be doubled. Since DAOS is a strongly consistent transaction model, the doubled replication group is very unfriendly to performance.

发明内容Contents of the invention

本申请实施例提供一种基于DAOS的数据跨保护域迁移方法及设备,用以在DAOS跨保护域场景下,兼顾数据的完整性和准确性、业务的连续性以及兼顾性能。The embodiment of the present application provides a DAOS-based data cross-protection domain migration method and device, which are used to take into account data integrity and accuracy, service continuity, and performance in a DAOS cross-protection domain scenario.

本申请实施例提供一种基于DAOS的数据跨保护域迁移方法,包括:The embodiment of this application provides a DAOS-based data migration method across protection domains, including:

在创建DAOS池的情况下,创建数个保护域分片(protection domain shard,PDS),其中任一PDS对应于一个保护域,且多个PDS可映射到同一个保护域上;In the case of creating a DAOS pool, create several protection domain shards (PDS), where any PDS corresponds to a protection domain, and multiple PDSs can be mapped to the same protection domain;

将PDS跨保护域的迁移流程分为单写阶段和双写阶段:The migration process of PDS across protection domains is divided into single-write phase and double-write phase:

在所述单写阶段,对任一PDS的对象写请求,仅落到对应的保护域上,进行多轮数据迁移;In the single write phase, the object write request for any PDS only falls on the corresponding protection domain, and multiple rounds of data migration are performed;

在所述双写阶段,对任一PDS的对象写请求,同时落到所述任一PDS以及对应的保护域上,仅进行一轮数据迁移。In the double-write phase, the object write request to any PDS falls on the any PDS and the corresponding protection domain at the same time, and only one round of data migration is performed.

可选的,在所述单写阶段具体执行:Optionally, specifically execute in the single write phase:

以发生跨保护域迁移请求的时间点作为右区间,以在先的迁移请求的时间点作为左区间,将所述左区间到所述右区间写入的所有的数据迁移到所述任一PDS上,以执行一轮数据迁移;Take the time point when the cross-protection domain migration request occurs as the right interval, and use the time point of the previous migration request as the left interval, and migrate all the data written from the left interval to the right interval to any PDS to perform a round of data migration;

在任一轮数据迁移完成后,记录迁移完成的时间点,并根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段;After any round of data migration is completed, record the time point when the migration is completed, and judge whether the migration process has entered the double-write phase according to the difference between the time point when the migration is completed and the time point of the previous migration request;

在判断迁移流程仍处于单写阶段的情况下,将迁移完成的时间点作为右区间,以相邻在先的迁移请求的时间点作为左区间,执行下一轮数据迁移。When it is judged that the migration process is still in the single-write stage, the time point when the migration is completed is used as the right interval, and the time point of the adjacent previous migration request is used as the left interval to execute the next round of data migration.

可选的,所述迁移请求的时间点是根据DAOS中的任一对象的每次写入记录对应的版本号来确定的,所述版本号是任一对象的对象写请求写入时的时钟。Optionally, the time point of the migration request is determined according to the version number corresponding to each write record of any object in DAOS, and the version number is the clock when the object write request of any object is written .

可选的,根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段是根据所述差值与预设阈值来实现的。Optionally, judging whether the migration process enters the double-write phase according to the difference between the time point of migration completion and the time point of the previous adjacent migration request is realized according to the difference value and a preset threshold.

可选的,在所述单写阶段,还包括在各右区间对应的迁移请求的时间点,为DAOS池中的容器打上快照。Optionally, in the single write phase, it also includes taking a snapshot of the container in the DAOS pool at the time point of the migration request corresponding to each right section.

可选的,在所述双写阶段具体执行:Optionally, specifically execute in the double-write phase:

以在单写阶段最后迁移完成的时间点为左区间,以最大迁移时间点为右区间,执行一轮数据迁移。Take the time point when the last migration is completed in the single write phase as the left interval, and use the maximum migration time point as the right interval to perform a round of data migration.

可选的,在所述双写阶段还包括在单写阶段最后迁移完成的时间点,为DAOS池中的容器打上快照。Optionally, the double-write phase also includes taking a snapshot of the containers in the DAOS pool at the point in time when the final migration of the single-write phase is completed.

可选的,还包括在迁移完成后,删除相应保护域上的数据,且数据读写只允许发生在PDS上。Optionally, it also includes deleting the data on the corresponding protection domain after the migration is completed, and the reading and writing of data is only allowed to occur on the PDS.

本申请实施例还提供一种计算机设备,包括处理器和存储器,所述存储器上存储有计算机程序,所述计算机程序被处理器执行时实现如前述的基于DAOS的数据跨保护域迁移方法的步骤。The embodiment of the present application also provides a computer device, including a processor and a memory, and a computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the aforementioned DAOS-based data cross-protection domain migration method are implemented .

本申请实施例还提供一种计算机可读存储介质,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如前述的基于DAOS的数据跨保护域迁移方法的步骤。The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the aforementioned DAOS-based data cross-protection domain migration method is implemented. step.

本申请实施例对象跨保护域迁移以PDS为粒度,方便减少计算量以及对象迁移的管理,实现了兼顾数据的完整性和准确性、业务的连续性以及兼顾性能。The object migration across protection domains in the embodiment of the present application takes PDS as the granularity, which facilitates the reduction of calculation amount and the management of object migration, and achieves both data integrity and accuracy, business continuity, and performance.

上述说明仅是本申请技术方案的概述,为了能够更清楚了解本申请的技术手段,而可依照说明书的内容予以实施,并且为了让本申请的上述和其它目的、特征和优点能够更明显易懂,以下特举本申请的具体实施方式。The above description is only an overview of the technical solution of the present application. In order to better understand the technical means of the present application, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and understandable , the following specifically cites the specific implementation manner of the present application.

附图说明Description of drawings

通过阅读下文优选实施方式的详细描述,各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的,而并不认为是对本申请的限制。而且在整个附图中,用相同的参考符号表示相同的部件。在附图中:Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the application. Also throughout the drawings, the same reference numerals are used to designate the same parts. In the attached picture:

图1为本申请实施例的数据跨保护域迁移方法的迁移流程示例;Figure 1 is an example of the migration process of the data cross-protection domain migration method of the embodiment of the present application;

图2为本申请实施例的数据跨保护域迁移方法的迁移阶段示例;Figure 2 is an example of the migration phase of the data cross-protection domain migration method of the embodiment of the present application;

图3为本申请实施例的多阶段数据迁移过程示例。FIG. 3 is an example of a multi-stage data migration process in an embodiment of the present application.

具体实施方式detailed description

下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例,然而应当理解,可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反,提供这些实施例是为了能够更透彻地理解本公开,并且能够将本公开的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

本申请实施例提供一种基于DAOS的数据跨保护域迁移方法,如图1所示,包括:The embodiment of this application provides a DAOS-based data migration method across protection domains, as shown in Figure 1, including:

在创建DAOS池的情况下,创建数个保护域分片(protection domain shard,PDS),其中任一PDS对应于一个保护域,且多个PDS可映射到同一个保护域上。具体的,本实施例在保护域的管理方法中,在保护域的硬盘池中新增PDS(protection domain shard),具体的创建DAOS池时,创建多个PDS,每个PDS对应到一个保护域,多个PDS可以映射到一个保护域上。对象首先计算映射到哪个PDS,即可知道该对象所属的保护域,本申请中对象跨保护域迁移以PDS为粒度,从而方便减少计算量以及对象迁移的管理。In the case of creating a DAOS pool, several protection domain shards (protection domain shards, PDSs) are created, wherein any PDS corresponds to one protection domain, and multiple PDSs can be mapped to the same protection domain. Specifically, in this embodiment, in the protection domain management method, a new PDS (protection domain shard) is added in the hard disk pool of the protection domain. Specifically, when creating a DAOS pool, multiple PDSs are created, and each PDS corresponds to a protection domain , multiple PDSs can be mapped to one protection domain. The protection domain to which the object belongs can be known by first calculating which PDS the object is mapped to. In this application, the object migration across protection domains takes PDS as the granularity, so as to facilitate the reduction of calculation and the management of object migration.

如图1所示,将PDS跨保护域的迁移流程分为单写阶段和双写阶段:As shown in Figure 1, the PDS migration process across protection domains is divided into a single-write phase and a double-write phase:

在所述单写阶段,对任一PDS(图1中新保护域)的对象写请求,仅落到对应的保护域(图1中旧保护域)上,进行多轮数据迁移。In the single write phase, the object write request for any PDS (new protection domain in FIG. 1 ) only falls on the corresponding protection domain (old protection domain in FIG. 1 ), and multiple rounds of data migration are performed.

在所述双写阶段,对任一PDS(图1中新保护域)的对象写请求,同时落到所述任一PDS以及对应的保护域(图1中旧保护域)上,仅进行一轮数据迁移。In the double-write phase, the object write request to any PDS (new protection domain in FIG. 1 ) falls on the any PDS and corresponding protection domain (old protection domain in FIG. 1 ) at the same time, and only one round of data migration.

本申请实施例对象跨保护域迁移以PDS为粒度,方便减少计算量以及对象迁移的管理,实现了兼顾数据的完整性和准确性、业务的连续性以及兼顾性能。The object migration across protection domains in the embodiment of the present application takes PDS as the granularity, which facilitates the reduction of calculation amount and the management of object migration, and achieves both data integrity and accuracy, business continuity, and performance.

在一些实施例中,在所述单写阶段具体执行:In some embodiments, in the single write stage, specifically execute:

以发生跨保护域迁移请求的时间点作为右区间,以在先的迁移请求的时间点作为左区间,将所述左区间到所述右区间写入的所有的数据迁移到所述任一PDS上,以执行一轮数据迁移。在具体应用中,DAOS使用了MVCC的机制保证读写互斥,对于DAOS中的每个对象的每一次写入记录都有一个版本号epoch来唯一标识,且该epoch是对象写请求写入时候的时钟。在一些实施例中,所述迁移请求的时间点是根据DAOS中的任一对象的每次写入记录对应的版本号来确定的,所述版本号是任一对象的对象写请求写入时的时钟。Take the time point when the cross-protection domain migration request occurs as the right interval, and use the time point of the previous migration request as the left interval, and migrate all the data written from the left interval to the right interval to any PDS to perform a round of data migration. In specific applications, DAOS uses the MVCC mechanism to ensure read-write mutual exclusion. For each write record of each object in DAOS, there is a version number epoch to uniquely identify it, and the epoch is when the object write request is written. clock. In some embodiments, the time point of the migration request is determined according to the version number corresponding to each write record of any object in DAOS, and the version number is when the object write request of any object is written clock.

具体的,如图2、图3所示,最开始PDS先进入单写阶段,对于单写阶段的第一轮迁移,以发生跨保护域迁移请求时间点epoch1作为右区间,以0值作为左区间,第一轮先迁移[0,epoch1]这个区间写入的所有的数据到新保护域(PDS)上。Specifically, as shown in Figure 2 and Figure 3, the PDS first enters the single-write stage. For the first round of migration in the single-write stage, the time point epoch1 when the cross-protection domain migration request occurs is used as the right interval, and the value 0 is used as the left interval. In the first round, all the data written in the interval [0, epoch1] will be migrated to the new protection domain (PDS).

在任一轮数据迁移完成后,记录迁移完成的时间点,并根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段。具体的,当第一轮数据迁移完成后,记录完成后的时间epoch2,判断epoch1和epoch2的差值Δepoch,以Δepoch的大小来判断是否进入第二阶段还是基于第一阶段再开启一轮迁移。在一些实施例中,根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段是根据所述差值与预设阈值来实现的。After any round of data migration is completed, record the time point when the migration is completed, and judge whether the migration process has entered the double-write phase based on the difference between the time point when the migration is completed and the time point of the previous migration request. Specifically, when the first round of data migration is completed, record the time epoch2 after completion, judge the difference Δepoch between epoch1 and epoch2, and use the size of Δepoch to judge whether to enter the second stage or start another round of migration based on the first stage. In some embodiments, judging whether the migration process enters the double-write phase according to the difference between the time point of migration completion and the time point of the previous adjacent migration request is realized according to the difference value and a preset threshold of.

在判断迁移流程仍处于单写阶段的情况下,将迁移完成的时间点作为右区间,以相邻在先的迁移请求的时间点作为左区间,执行下一轮数据迁移。When it is judged that the migration process is still in the single-write stage, the time point when the migration is completed is used as the right interval, and the time point of the adjacent previous migration request is used as the left interval to execute the next round of data migration.

若判断出仍处于第一阶段(单写阶段),则执行第二轮迁移,第二轮迁移的时间戳范围是[epoch1,epoch2],第二轮迁移完成后,此时时钟时间是epoch3,继续判断epoch3和epoch2的差值Δepoch。经过N轮迁移后,如果epochN和epoch(N-1)的差值Δepoch符合阈值,则进入第二阶段(双写阶段)。If it is judged that it is still in the first stage (single write stage), the second round of migration will be executed. The timestamp range of the second round of migration is [epoch1, epoch2]. After the second round of migration is completed, the clock time is epoch3. Continue to judge the difference Δepoch between epoch3 and epoch2. After N rounds of migration, if the difference Δepoch between epochN and epoch(N-1) meets the threshold, then enter the second stage (double write stage).

在一些实施例中,在所述单写阶段,还包括在各右区间对应的迁移请求的时间点,为DAOS池中的容器打上快照。例如对应于第一轮迁移,在epoch1对DAOS池中的容器打上一个快照,保证epoch1之前的数据不会被修改。对应于第二轮迁移,在epoch2对DAOS池中的容器打上一个快照,以此类推。In some embodiments, in the single write phase, it also includes taking a snapshot for the container in the DAOS pool at the time point of the migration request corresponding to each right section. For example, corresponding to the first round of migration, take a snapshot of the containers in the DAOS pool in epoch1 to ensure that the data before epoch1 will not be modified. Corresponding to the second round of migration, take a snapshot of the containers in the DAOS pool in epoch2, and so on.

在一些实施例中,如图3所示,在所述双写阶段具体执行:以在单写阶段最后迁移完成的时间点为左区间,以最大迁移时间点为右区间,执行一轮数据迁移。In some embodiments, as shown in FIG. 3 , it is specifically executed in the double-write phase: the time point at which the last migration is completed in the single-write phase is used as the left interval, and the maximum migration time point is used as the right interval to perform a round of data migration .

对于双写阶段(第二阶段),将写入请求同时写入到新旧两个保护域上,对[epochN,MAX]范围内的数据进行迁移。在一些实施例中,在所述双写阶段还包括在单写阶段最后迁移完成的时间点,为DAOS池中的容器打上快照,例如可以将epochN对DAOS池中的容器打上打上快照。由于前一轮的Δepoch时间特别小,符合预期,因此可以确定这个区间内写入的数据量不会太大,因此双写持续时间会比较短,减少对性能的损耗,当[epochN,MAX]范围内的数据迁移完成后,迁移结束。迁移结束后,数据读写只发生在新保护域上。For the double-write phase (second phase), the write request is written to the old and new protection domains at the same time, and the data within the [epochN, MAX] range is migrated. In some embodiments, the double-write phase also includes taking a snapshot of the container in the DAOS pool at the point in time when the last migration of the single-write phase is completed, for example, the epochN can be used to make a snapshot of the container in the DAOS pool. Since the Δepoch time of the previous round is very small, which is in line with expectations, it can be determined that the amount of data written in this interval will not be too large, so the duration of double writing will be relatively short, reducing the loss of performance. When [epochN, MAX] The migration ends when the in-scope data migration is complete. After the migration is complete, data reads and writes only occur on the new protection domain.

在一些实施例中,还包括在迁移完成后,删除相应保护域上的数据,且数据读写只允许发生在PDS上。In some embodiments, after the migration is completed, deleting the data on the corresponding protection domain, and reading and writing data are only allowed to occur on the PDS.

本申请的数据跨保护域迁移方法将迁移任务划分为多轮多个小区间内的迁移,所有区间内数据迁移完成后,新旧保护域数据达成一致,本申请的方法简化了迁移的流程,便于管理,能有效减少迁移过程中的计算量,减少对硬件资源的损耗,并且有效降低了数据迁移对业务的影响。The data cross-protection domain migration method of this application divides the migration task into multiple rounds of migration between multiple cells. After the data migration in all intervals is completed, the old and new protection domain data are consistent. The method of this application simplifies the migration process and facilitates Management can effectively reduce the amount of calculation during the migration process, reduce the loss of hardware resources, and effectively reduce the impact of data migration on business.

本申请实施例还提供一种计算机设备,包括处理器和存储器,所述存储器上存储有计算机程序,所述计算机程序被处理器执行时实现如前述的基于DAOS的数据跨保护域迁移方法的步骤。The embodiment of the present application also provides a computer device, including a processor and a memory, and a computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the aforementioned DAOS-based data cross-protection domain migration method are implemented .

本申请实施例还提供一种计算机可读存储介质,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如前述的基于DAOS的数据跨保护域迁移方法的步骤。The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the aforementioned DAOS-based data cross-protection domain migration method is implemented. step.

需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。It should be noted that, in this document, the terms "comprising", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

上述本申请实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端(可以是手机,计算机,服务器或者网络设备等)执行本申请各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to enable a terminal (which may be a mobile phone, a computer, a server or a network device, etc.) to execute the methods described in the various embodiments of the present application.

上面结合附图对本申请的实施例进行了描述,但是本申请并不局限于上述的具体实施方式,上述的具体实施方式仅仅是示意性的,而不是限制性的,本领域的普通技术人员在本申请的启示下,在不脱离本申请宗旨和权利要求所保护的范围情况下,还可做出很多形式,这些均属于本申请的保护之内。The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the enlightenment of this application, many forms can also be made without departing from the purpose of this application and the scope of protection of the claims, and these all belong to the protection of this application.

Claims (10)

Translated fromChinese
1.一种基于DAOS的数据跨保护域迁移方法,其特征在于,包括:1. A DAOS-based data cross-protection domain migration method, characterized in that it comprises:在创建DAOS池的情况下,创建数个保护域分片(protection domain shard,PDS),其中任一PDS对应于一个保护域,且多个PDS可映射到同一个保护域上;In the case of creating a DAOS pool, create several protection domain shards (PDS), where any PDS corresponds to a protection domain, and multiple PDSs can be mapped to the same protection domain;将PDS跨保护域的迁移流程分为单写阶段和双写阶段:The migration process of PDS across protection domains is divided into single-write phase and double-write phase:在所述单写阶段,对任一PDS的对象写请求,仅落到对应的保护域上,以进行多轮数据迁移;In the single write phase, the object write request for any PDS only falls on the corresponding protection domain to perform multiple rounds of data migration;在所述双写阶段,对任一PDS的对象写请求,同时落到所述任一PDS以及对应的保护域上,以仅进行一轮数据迁移。In the double-write phase, the object write request to any PDS falls on the any PDS and the corresponding protection domain at the same time, so that only one round of data migration is performed.2.如权利要求1所述的基于DAOS的数据跨保护域迁移方法,其特征在于,在所述单写阶段具体执行:2. The DAOS-based data cross-protection domain migration method according to claim 1, characterized in that, in the single write stage, it is specifically executed:以发生跨保护域迁移请求的时间点作为右区间,以在先的迁移请求的时间点作为左区间;The time point when a cross-protection domain migration request occurs is taken as the right interval, and the time point of the previous migration request is taken as the left interval;将所述左区间到所述右区间写入的所有的数据迁移到所述任一PDS上,以执行一轮数据迁移;Migrating all the data written from the left interval to the right interval to any of the PDSs to perform a round of data migration;在任一轮数据迁移完成后,记录迁移完成的时间点,并根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段;After any round of data migration is completed, record the time point when the migration is completed, and judge whether the migration process has entered the double-write phase according to the difference between the time point when the migration is completed and the time point of the previous migration request;在判断迁移流程仍处于单写阶段的情况下,将迁移完成的时间点作为右区间,以相邻在先的迁移请求的时间点作为左区间,执行下一轮数据迁移。When it is judged that the migration process is still in the single-write stage, the time point when the migration is completed is used as the right interval, and the time point of the adjacent previous migration request is used as the left interval to execute the next round of data migration.3.如权利要求2所述的基于DAOS的数据跨保护域迁移方法,其特征在于,所述迁移请求的时间点是根据DAOS中的任一对象的每次写入记录对应的版本号来确定的,所述版本号是任一对象的对象写请求写入时的时钟。3. The DAOS-based data cross-protection domain migration method according to claim 2, wherein the time point of the migration request is determined according to the version number corresponding to each write record of any object in DAOS , the version number is the clock when the object write request for any object is written.4.如权利要求2所述的基于DAOS的数据跨保护域迁移方法,其特征在于,根据迁移完成的时间点与相邻在先的迁移请求的时间点之间的差值,来判断迁移流程是否进入双写阶段是根据所述差值与预设阈值来实现的。4. The DAOS-based data cross-protection domain migration method according to claim 2, wherein the migration process is judged according to the difference between the time point when the migration is completed and the time point of the previous adjacent migration request Whether to enter the double writing phase is realized according to the difference and a preset threshold.5.如权利要求2所述的基于DAOS的数据跨保护域迁移方法,其特征在于,在所述单写阶段,还包括在各右区间对应的迁移请求的时间点,为DAOS池中的容器打上快照。5. The DAOS-based data cross-protection domain migration method according to claim 2, characterized in that, in the single write stage, it also includes the time point of the migration request corresponding to each right interval, which is the container in the DAOS pool Take a snapshot.6.如权利要求5所述的基于DAOS的数据跨保护域迁移方法,其特征在于,在所述双写阶段具体执行:6. The DAOS-based data cross-protection domain migration method according to claim 5, characterized in that, in the double-write phase, it is specifically executed:以在单写阶段最后迁移完成的时间点为左区间,以最大迁移时间点为右区间,执行一轮数据迁移。Take the time point when the last migration is completed in the single write phase as the left interval, and use the maximum migration time point as the right interval to perform a round of data migration.7.如权利要求6所述的基于DAOS的数据跨保护域迁移方法,其特征在于,在所述双写阶段还包括在单写阶段最后迁移完成的时间点,为DAOS池中的容器打上快照。7. The DAOS-based data cross-protection domain migration method according to claim 6, characterized in that, in the double-write phase, it also includes a time point when the final migration in the single-write phase is completed, and a snapshot is made for the container in the DAOS pool .8.如权利要求6所述的基于DAOS的数据跨保护域迁移方法,其特征在于,还包括在迁移完成后,删除相应保护域上的数据,且数据读写只允许发生在PDS上。8. The DAOS-based data migration method across protection domains according to claim 6, further comprising deleting the data on the corresponding protection domain after the migration is completed, and data reading and writing are only allowed to occur on the PDS.9.一种计算机设备,其特征在于,包括处理器和存储器,所述存储器上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1至8中任一项所述的基于DAOS的数据跨保护域迁移方法的步骤。9. A computer device, characterized in that it comprises a processor and a memory, and a computer program is stored on the memory, and when the computer program is executed by the processor, the method based on any one of claims 1 to 8 is realized. The steps of the DAOS data migration method across protection domains.10.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1至8中任一项所述的基于DAOS的数据跨保护域迁移方法的步骤。10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is realized. The steps of the DAOS-based data migration method across protection domains.
CN202211137909.2A2022-09-192022-09-19 A DAOS-based data cross-protection domain migration method and devicePendingCN115543961A (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202211137909.2ACN115543961A (en)2022-09-192022-09-19 A DAOS-based data cross-protection domain migration method and device

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202211137909.2ACN115543961A (en)2022-09-192022-09-19 A DAOS-based data cross-protection domain migration method and device

Publications (1)

Publication NumberPublication Date
CN115543961Atrue CN115543961A (en)2022-12-30

Family

ID=84727139

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202211137909.2APendingCN115543961A (en)2022-09-192022-09-19 A DAOS-based data cross-protection domain migration method and device

Country Status (1)

CountryLink
CN (1)CN115543961A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN116627934A (en)*2023-05-112023-08-22中国建设银行股份有限公司Data migration method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN106383667A (en)*2016-09-182017-02-08华为数字技术(成都)有限公司Method and apparatus for LUN migration in business operation
US20190179808A1 (en)*2016-12-012019-06-13Tencent Technology (Shenzhen) Company LimitedMethod and apparatus for data migration in database cluster, and storage medium
CN111291023A (en)*2020-02-092020-06-16苏州浪潮智能科技有限公司 A method, system, device and medium for data migration
CN111913663A (en)*2020-07-292020-11-10星辰天合(北京)数据科技有限公司 Method and device for online storage volume migration, and system for online storage volume migration
CN113254421A (en)*2021-05-312021-08-13重庆富民银行股份有限公司Database double-writing based cross-library account migration method
CN114237519A (en)*2022-02-232022-03-25苏州浪潮智能科技有限公司 A method, device, device and medium for object storage data migration

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN106383667A (en)*2016-09-182017-02-08华为数字技术(成都)有限公司Method and apparatus for LUN migration in business operation
US20190179808A1 (en)*2016-12-012019-06-13Tencent Technology (Shenzhen) Company LimitedMethod and apparatus for data migration in database cluster, and storage medium
CN111291023A (en)*2020-02-092020-06-16苏州浪潮智能科技有限公司 A method, system, device and medium for data migration
CN111913663A (en)*2020-07-292020-11-10星辰天合(北京)数据科技有限公司 Method and device for online storage volume migration, and system for online storage volume migration
CN113254421A (en)*2021-05-312021-08-13重庆富民银行股份有限公司Database double-writing based cross-library account migration method
CN114237519A (en)*2022-02-232022-03-25苏州浪潮智能科技有限公司 A method, device, device and medium for object storage data migration

Cited By (1)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN116627934A (en)*2023-05-112023-08-22中国建设银行股份有限公司Data migration method and device

Similar Documents

PublicationPublication DateTitle
CN110531940B (en) Video file processing method and device
US11853549B2 (en)Index storage in shingled magnetic recording (SMR) storage system with non-shingled region
US11256715B2 (en)Data backup method and apparatus
US9645892B1 (en)Recording file events in change logs while incrementally backing up file systems
CN111723056B (en) Small file processing method, device, equipment and storage medium
US11093387B1 (en)Garbage collection based on transmission object models
CN107357920B (en)Incremental multi-copy data synchronization method and system
CN103197988A (en)Data backup and recovery method, device and database system
CN109144416B (en)Method and device for querying data
WO2017041654A1 (en)Method and apparatus for writing and acquiring data in distributed storage system
CN106357703B (en)Cluster switching method and device
CN106537353B (en) A data replication method, device and system
CN110018989A (en)A kind of method and apparatus that snapshot compares
CN106446044A (en)Storage space reclaiming method and device
CN115357429B (en)Method, device and client for recovering data file
CN111158858A (en)Cloning method and device of virtual machine and computer readable storage medium
CN114741449A (en)Object storage method and device based on distributed database
CN108984343A (en)A kind of virtual machine backup and memory management method based on content analysis
CN115543961A (en) A DAOS-based data cross-protection domain migration method and device
CN112699129A (en)Data processing system, method and device
CN107357928A (en)A kind of method and system for realizing storage snapshot
CN118568054B (en) Method, device, equipment, medium and product for data migration
CN111858159A (en) A backup method and system based on chain clone feature
CN116186033B (en) Data archiving method, system, device and storage medium
CN111221801A (en) A database migration method, system and related device

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination

[8]ページ先頭

©2009-2025 Movatter.jp