CN114385309A - Data migration method, data snapshot method and device - Google Patents
Data migration method, data snapshot method and deviceDownload PDFInfo
- Publication number
- CN114385309A CN114385309ACN202111462665.0ACN202111462665ACN114385309ACN 114385309 ACN114385309 ACN 114385309ACN 202111462665 ACN202111462665 ACN 202111462665ACN 114385309 ACN114385309 ACN 114385309A
- Authority
- CN
- China
- Prior art keywords
- data
- storage device
- snapshot
- virtual server
- server instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本公开涉及虚拟机热迁移领域，特别涉及一种数据迁移方法、数据快照方法及装置。The present disclosure relates to the field of virtual machine hot migration, and in particular, to a data migration method, a data snapshot method and an apparatus.
背景技术Background technique
虚拟机(Virtual Machine，简称VM)热迁移技术主要包括PreCopy和PostCopy两种。Virtual Machine (Virtual Machine, VM for short) live migration technologies mainly include PreCopy and PostCopy.
PreCopy是一种先进行数据迁移后切换IO的方式。在使用PreCopy方式对虚拟机磁盘(云盘)进行数据迁移过程中，如果用户对云盘执行生成快照操作，由于PreCopy迁移属于后台运维行为，PreCopy迁移流程终止，优先响应用户的快照请求。而PreCopy在迁移完成后再切换IO，带来的问题是云盘的写入持续在源集群上，无法做到立即数据分流。并且，PreCopy往往需要多轮拷贝，使得整个迁移过程耗费时间较长。PreCopy is a way to switch IO after data migration. During the data migration of virtual machine disks (cloud disks) using PreCopy, if the user performs a snapshot operation on the cloud disk, the PreCopy migration is a background operation and maintenance behavior, and the PreCopy migration process is terminated, and the user's snapshot request is given priority. However, PreCopy switches IO after the migration is completed. The problem is that the writes to the cloud disk continue to be on the source cluster, and immediate data distribution cannot be achieved. In addition, PreCopy often requires multiple rounds of copying, making the entire migration process time-consuming.
所以优选使用PostCopy迁移方式。PostCopy是一种先进行IO流切换后进行数据迁移的方式。在PostCopy迁移过程中，由于源集群上面只有切换IO流之前的那部分数据，切换IO流之后的数据都在目标集群，即任意集群都没有虚拟机的全量数据，使得在PostCopy迁移过程中无法把迁移停下来而优先响应快照请求。Therefore, it is preferable to use the PostCopy migration method. PostCopy is a way to perform data migration after IO stream switching. During the PostCopy migration process, since there is only the part of the data before switching the IO stream on the source cluster, the data after switching the IO stream is all in the target cluster, that is, no cluster has the full amount of data of the virtual machine, which makes it impossible to transfer the data during the PostCopy migration process. The migration stops and responds to snapshot requests first.
因此，需要一种支持在PostCopy迁移过程中生成快照的方案。Therefore, there is a need for a scheme that supports generating snapshots during PostCopy migrations.
发明内容SUMMARY OF THE INVENTION
本公开要解决的一个技术问题是提供一种支持PostCopy迁移过程中生成快照的方案。A technical problem to be solved by the present disclosure is to provide a solution for supporting snapshot generation during PostCopy migration.
根据本公开的第一个方面，提供了一种数据快照方法，包括：将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置；响应于快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照；在迁移完成后对第三存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第二快照。According to a first aspect of the present disclosure, a data snapshot method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that the computing device can The data stored in the storage device starts the virtual server instance, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, the data read and write request related to the virtual server instance is executed with respect to the second storage device; The virtual server instance-related data stored in a storage device and associated with the operation of the virtual server instance is migrated to a third storage device; in response to the snapshot request, snapshot processing is performed on the virtual server instance-related data stored in the second storage device to obtain a third storage device. A snapshot; after the migration is completed, snapshot processing is performed on the data related to the virtual server instance stored in the third storage device to obtain a second snapshot.
可选地，基于第一快照和第二快照得到虚拟服务器实例在快照请求对应的时刻的全量数据。Optionally, the full amount of data of the virtual server instance at the moment corresponding to the snapshot request is obtained based on the first snapshot and the second snapshot.
可选地，第一快照包括第二存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第一数据位图，第二快照包括第三存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第二数据位图，基于第一快照和第二快照得到虚拟服务器实例在快照请求对应的时刻的全量数据的步骤包括：合并第一数据位图和第二数据位图，以得到能够表征虚拟服务器实例在快照请求对应的时刻的全量数据的逻辑地址的第三数据位图；针对第三数据位图中的每个逻辑地址，判断第一快照中是否存在与该逻辑地址对应的数据，若存在则从第一快照中选取与该逻辑地址对应的数据，若不存在则从第二快照中选取与该逻辑地址对应的数据。Optionally, the first snapshot includes virtual server instance-related data stored in the second storage device and a first data bitmap used to represent a logical address of the data, and the second snapshot includes virtual server instance-related data stored in the third storage device. data and a second data bitmap for representing the logical address of the data, the step of obtaining the full amount of data of the virtual server instance at the moment corresponding to the snapshot request based on the first snapshot and the second snapshot includes: merging the first data bitmap and the second A data bitmap to obtain a third data bitmap that can represent the logical addresses of the full amount of data of the virtual server instance at the moment corresponding to the snapshot request; for each logical address in the third data bitmap, determine whether there is any in the first snapshot If the data corresponding to the logical address exists, the data corresponding to the logical address is selected from the first snapshot, and if it does not exist, the data corresponding to the logical address is selected from the second snapshot.
可选地，该方法还可以包括：将虚拟服务器实例在快照请求对应的时刻的全量数据上传到存储服务器进行备份。Optionally, the method may further include: uploading the full amount of data of the virtual server instance at the moment corresponding to the snapshot request to the storage server for backup.
可选地，该方法还可以包括：将第二快照的数据导入第二存储装置。Optionally, the method may further include: importing the data of the second snapshot into the second storage device.
根据本公开的第二个方面，提供了一种数据迁移方法，包括：将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置；在迁移完成后，将第三存储装置中存储的虚拟服务器实例相关数据导入第二存储装置。According to a second aspect of the present disclosure, a data migration method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that the computing device can The data stored in the storage device starts the virtual server instance, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, the data read and write request related to the virtual server instance is executed with respect to the second storage device; The virtual server instance related data stored in a storage device and associated with the running of the virtual server instance is migrated to the third storage device; after the migration is completed, the virtual server instance related data stored in the third storage device is imported into the second storage device.
根据本公开的第三个方面，提供了一种数据快照方法，包括：将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；对第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据进行快照处理，得到第三快照；响应于快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照。According to a third aspect of the present disclosure, a data snapshot method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that the computing device can The data stored in the storage device starts the virtual server instance, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, a data read and write request related to the virtual server instance is executed for the second storage device; Perform snapshot processing on the virtual server instance-related data stored in a storage device and associated with the operation of the virtual server instance to obtain a third snapshot; in response to the snapshot request, perform snapshot processing on the virtual server instance-related data stored in the second storage device, Get the first snapshot.
可选地，该方法还可以包括：基于第一快照和第三快照得到虚拟服务器实例在快照请求对应的时刻的全量数据。Optionally, the method may further include: obtaining, based on the first snapshot and the third snapshot, the full amount of data of the virtual server instance at the moment corresponding to the snapshot request.
根据本公开的第四个方面，提供了一种数据快照装置，包括：第一迁移模块，用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；第二迁移模块，用于将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置；第一快照模块，用于响应于快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照；第二快照模块，用于在迁移完成后对第三存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第二快照。According to a fourth aspect of the present disclosure, a data snapshot device is provided, comprising: a first migration module configured to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device to The computing device is enabled to start the virtual server instance according to the data stored in the second storage device, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, executes a virtual server instance-related operation on the second storage device. a data read and write request; a second migration module for migrating virtual server instance-related data stored in the first storage device and associated with the operation of the virtual server instance to a third storage device; a first snapshot module for responding to the snapshot request, perform snapshot processing on the data related to the virtual server instance stored in the second storage device to obtain a first snapshot; the second snapshot module is used to perform a snapshot of the related data of the virtual server instance stored in the third storage device after the migration is completed Process to get a second snapshot.
根据本公开的第五个方面，提供了一种数据迁移装置，包括：第一迁移模块，用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；第二迁移模块，用于将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置；导入模块，用于在迁移完成后，将第三存储装置中存储的虚拟服务器实例相关数据导入第二存储装置。According to a fifth aspect of the present disclosure, a data migration device is provided, comprising: a first migration module configured to migrate data required for starting a virtual server instance stored in the first storage device to the second storage device to The computing device is enabled to start the virtual server instance according to the data stored in the second storage device, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, executes a virtual server instance-related operation on the second storage device. a data read and write request; a second migration module for migrating virtual server instance-related data stored in the first storage device and associated with the operation of the virtual server instance to a third storage device; an import module for, after the migration is completed, Import the virtual server instance related data stored in the third storage device into the second storage device.
根据本公开的第六个方面，提供了一种数据快照装置，包括：迁移模块，用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；第三快照模块，用于对第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据进行快照处理，得到第三快照；第一快照模块，用于响应于迁移过程中的快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照。According to a sixth aspect of the present disclosure, there is provided a data snapshot device, comprising: a migration module for migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that computing The device is capable of starting the virtual server instance according to the data stored in the second storage device, wherein after the computing device starts the virtual server instance according to the data stored in the second storage device, a data read related to the virtual server instance is performed with respect to the second storage device a write request; a third snapshot module for performing snapshot processing on the virtual server instance-related data stored in the first storage device and associated with the operation of the virtual server instance to obtain a third snapshot; the first snapshot module for responding to the migration For the snapshot request in the process, snapshot processing is performed on the data related to the virtual server instance stored in the second storage device to obtain the first snapshot.
根据本公开的第七个方面，提供了一种计算设备，包括：处理器；以及存储器，其上存储有可执行代码，当可执行代码被处理器执行时，使处理器执行如上述第一方面至第三方面中任一方面所述的方法。According to a seventh aspect of the present disclosure, there is provided a computing device, comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, causes the processor to execute the above-mentioned first The method of any of the aspects to the third aspect.
根据本公开的第八个方面，提供了一种计算机程序产品，包括可执行代码，当所述可执行代码被电子设备的处理器执行时，使所述处理器执行如上述第一方面至第三方面中任一方面所述的方法。According to an eighth aspect of the present disclosure, there is provided a computer program product, comprising executable code, when the executable code is executed by a processor of an electronic device, the processor is caused to execute the above-mentioned first to sixth aspects. The method of any one of the three aspects.
根据本公开的第九个方面，提供了一种非暂时性机器可读存储介质，其上存储有可执行代码，当可执行代码被电子设备的处理器执行时，使处理器执行如上述第一方面至第三方面中任一方面所述的方法。According to a ninth aspect of the present disclosure, there is provided a non-transitory machine-readable storage medium, on which executable codes are stored, and when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the above-mentioned first step. The method of any one of the one to third aspects.
由此，本公开通过设置用于作为中间迁移目标的第三存储装置，将第一存储装置中的数据迁移到第三存储装置，可以在一定程度上减少上述互斥情况的发生；在此基础上，响应于快照请求对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理得到第一快照，并在迁移完成后将对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照，使得基于第一快照和第二快照可以得到虚拟服务器实例在快照请求对应时刻的全量数据。Therefore, the present disclosure can reduce the occurrence of the above-mentioned mutually exclusive situation to a certain extent by setting the third storage device as an intermediate migration target and migrating the data in the first storage device to the third storage device; above, in response to the snapshot request, performing snapshot processing on the virtual server instance-related data stored in the second storage device to obtain a first snapshot, and after the migration is completed, performing snapshot processing on the virtual server instance-related data stored in the second storage device, The first snapshot is obtained, so that the full amount of data of the virtual server instance at the moment corresponding to the snapshot request can be obtained based on the first snapshot and the second snapshot.
附图说明Description of drawings
通过结合附图对本公开示例性实施方式进行更详细的描述，本公开的上述以及其它目的、特征和优势将变得更加明显，其中，在本公开示例性实施方式中，相同的参考标号通常代表相同部件。The above and other objects, features and advantages of the present disclosure will become more apparent from the more detailed description of the exemplary embodiments of the present disclosure taken in conjunction with the accompanying drawings, wherein the same reference numerals generally refer to the exemplary embodiments of the present disclosure. same parts.
图1示出了基于PostCopy的数据迁移流程示意图。Figure 1 shows a schematic diagram of the data migration process based on PostCopy.
图2示出了根据本公开一个实施例的基于改进的PostCopy的数据迁移流程示意图。FIG. 2 shows a schematic diagram of a data migration process based on an improved PostCopy according to an embodiment of the present disclosure.
图3示出了根据本公开一个实施例的数据迁移过程中的快照方法的示意性流程图。FIG. 3 shows a schematic flowchart of a snapshot method in a data migration process according to an embodiment of the present disclosure.
图4示出了根据本公开一个实施例的快照装置的结构示意图。FIG. 4 shows a schematic structural diagram of a snapshot device according to an embodiment of the present disclosure.
图5示出了根据本公开一个实施例的数据迁移装置的结构示意图。FIG. 5 shows a schematic structural diagram of a data migration apparatus according to an embodiment of the present disclosure.
图6示出了根据本公开另一个实施例的数据迁移过程中的快照装置的结构示意图。FIG. 6 shows a schematic structural diagram of a snapshot device in a data migration process according to another embodiment of the present disclosure.
图7示出了根据本公开一个实施例的计算设备的结构示意图。FIG. 7 shows a schematic structural diagram of a computing device according to an embodiment of the present disclosure.
图8示出了PostCopy迁移过程示意图。Figure 8 shows a schematic diagram of the PostCopy migration process.
图9示出了PostCopy迁移过程中获取完整快照的流程示意图。FIG. 9 shows a schematic flowchart of obtaining a complete snapshot during PostCopy migration.
具体实施方式Detailed ways
下面将参照附图更详细地描述本公开的优选实施方式。虽然附图中显示了本公开的优选实施方式，然而应该理解，可以以各种形式实现本公开而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了使本公开更加透彻和完整，并且能够将本公开的范围完整地传达给本领域的技术人员。Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
图1示出了基于PostCopy的数据迁移流程示意图。Figure 1 shows a schematic diagram of the data migration process based on PostCopy.
第一存储装置用于作为虚拟服务器实例(如虚拟机)的存储设备，存储与虚拟服务器实例的运行关联的虚拟服务器实例相关数据。例如，第一存储装置可以是指虚拟机磁盘。虚拟机磁盘，也即虚拟机块设备，简称云盘。The first storage device is used as a storage device of the virtual server instance (such as a virtual machine), and stores data related to the virtual server instance associated with the running of the virtual server instance. For example, the first storage device may refer to a virtual machine disk. Virtual machine disk, also known as virtual machine block device, referred to as cloud disk.
虚拟服务器实例相关数据可以包括虚拟服务器实例启动所需的数据，即虚拟服务器实例启动时必须依赖的数据，如虚拟机的系统盘数据。The data related to the virtual server instance may include data required to start the virtual server instance, that is, data that must be relied upon when the virtual server instance is started, such as system disk data of the virtual machine.
虚拟服务器实例相关数据还可以包括虚拟服务器实例运行过程中写入第一存储装置的数据，该数据在虚拟服务器实例的运行过程中可能会被调用。The data related to the virtual server instance may further include data written to the first storage device during the running process of the virtual server instance, and the data may be called during the running process of the virtual server instance.
第一存储装置可以视为数据迁移的源端，第二存储装置可以视为数据迁移的目的端。在执行数据迁移操作(即将第一存储装置中的虚拟服务器实例相关数据迁移到第二存储装置)时，可以首先将虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例。The first storage device may be regarded as the source end of data migration, and the second storage device may be regarded as the destination end of data migration. When performing a data migration operation (that is, migrating the data related to the virtual server instance in the first storage device to the second storage device), the data required for starting the virtual server instance may be first migrated to the second storage device, so that the computing device can The virtual server instance is started according to the data stored in the second storage device.
计算装置用于提供虚拟服务器实例的运行环境。第二存储装置可以是指在物理上独立于计算装置的存储设备。计算装置与第二存储装置可以构成计算、存储分离的架构。即，计算装置可以是指该架构中用于提供计算资源的计算服务器，第二存储装置可以是指该架构中用于提供存储资源的存储服务器。在数据迁移操作开始之前，计算装置可以根据第一存储装置中存储的数据启动并运行虚拟服务器实例。The computing device is used to provide an operating environment for the virtual server instance. The second storage device may refer to a storage device that is physically separate from the computing device. The computing device and the second storage device may form an architecture in which computing and storage are separated. That is, the computing device may refer to a computing server for providing computing resources in the architecture, and the second storage device may refer to a storage server for providing storage resources in the architecture. Before the data migration operation begins, the computing device may start and run the virtual server instance according to the data stored in the first storage device.
在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求。即，在根据第二存储装置中存储的数据启动虚拟服务器实例后，与虚拟服务器实例相关的IO请求(即IO流)将会从第一存储装置切换到第二存储装置。由此，虚拟服务器实例运行过程中的新数据将会写入第二存储装置。After the computing device starts the virtual server instance according to the data stored in the second storage device, a data read/write request related to the virtual server instance is executed with respect to the second storage device. That is, after starting the virtual server instance according to the data stored in the second storage device, IO requests (ie, IO streams) related to the virtual server instance will be switched from the first storage device to the second storage device. Thus, new data during the running of the virtual server instance will be written to the second storage device.
在根据第二存储装置中存储的数据启动虚拟服务器实例，将IO流从第一存储装置切换到第二存储装置后，可以继续执行数据迁移流程，将第一存储装置中存储的其他虚拟服务器实例相关数据持续迁移到第二存储装置。After starting the virtual server instance according to the data stored in the second storage device, and switching the IO flow from the first storage device to the second storage device, the data migration process can be continued, and other virtual server instances stored in the first storage device can be Relevant data is continuously migrated to the second storage device.
在IO流从第一存储装置切换到第二存储装置后，对于IO请求中的数据写请求，可以直接将新数据写入第二存储装置；对于IO请求中的数据读请求，可以在第二存储装置中查找是否存在与数据读请求对应的数据，如果存在则直接读取，如果不存在，则可以从第一存储装置获取该数据。After the IO stream is switched from the first storage device to the second storage device, for the data write request in the IO request, new data can be directly written into the second storage device; for the data read request in the IO request, it can be written in the second storage device. The storage device is searched for whether there is data corresponding to the data read request, and if it exists, it is directly read, and if it does not exist, the data can be obtained from the first storage device.
在图1所示的基于PostCopy的数据迁移过程中，第二存储装置中存在两种数据写入操作：将第一存储装置中的数据迁移到第二存储装置的过程中存在的数据写入操作；以及虚拟服务器实例运行过程中响应于数据写请求而针对第二存储装置执行的数据写入操作。当上述两种写入操作针对的是同一数据对象时，将会出现“写”互斥。In the data migration process based on PostCopy shown in FIG. 1 , there are two data writing operations in the second storage device: the data writing operation existing in the process of migrating data from the first storage device to the second storage device ; and a data write operation performed on the second storage device in response to a data write request during the running of the virtual server instance. "Write" mutual exclusion occurs when both of the above write operations are directed to the same data object.
为了避免上述互斥情况的发生，本公开提出了改进的PostCopy数据迁移流程。In order to avoid the occurrence of the above-mentioned mutually exclusive situation, the present disclosure proposes an improved PostCopy data migration process.
图2示出了根据本公开一个实施例的基于改进的PostCopy的数据迁移流程示意图。FIG. 2 shows a schematic diagram of a data migration process based on an improved PostCopy according to an embodiment of the present disclosure.
与图1所示的数据迁移流程不同之处在于，本实施例设置了一个用于作为中间迁移目标的第三存储装置。在根据第二存储装置中存储的数据启动虚拟服务器实例，将IO流从第一存储装置切换到第二存储装置后，第一存储装置中的其他虚拟服务器实例相关数据不直接迁移到第二存储装置，而是将第一存储装置中存储的虚拟服务器实例相关数据迁移到第三存储装置，并在迁移完成后再将第三存储装置中存储的虚拟服务器实例相关数据导入第二存储装置，以使得第二存储装置拥有虚拟服务器实例的全量数据，完成数据迁移。The difference from the data migration process shown in FIG. 1 is that this embodiment sets a third storage device used as an intermediate migration target. After starting the virtual server instance according to the data stored in the second storage device and switching the IO stream from the first storage device to the second storage device, the data related to other virtual server instances in the first storage device are not directly migrated to the second storage device Instead, the virtual server instance-related data stored in the first storage device is migrated to the third storage device, and after the migration is completed, the virtual server instance-related data stored in the third storage device is imported into the second storage device, so as to The second storage device is made to possess the full amount of data of the virtual server instance, and the data migration is completed.
迁移到第三存储装置中的虚拟服务器实例相关数据，可以是指第一存储装置中存储的所有与虚拟服务器实例的运行关联的虚拟服务器实例相关数据，也可以是指所有虚拟服务器实例相关数据中去除了虚拟机启动所需数据后的数据。The virtual server instance-related data migrated to the third storage device may refer to all virtual server instance-related data stored in the first storage device and associated with the operation of the virtual server instance, or may refer to all virtual server instance-related data. The data after removing the data required for the virtual machine to start.
在实际应用场景中，第一存储装置、第二存储装置可以属于不同的块设备存储集群(Elastic Block Service，简称EBS)，第三存储装置与第二存储装置可以属于同一块设备存储集群。跨集群的数据迁移比较费时，更容易导致“写”互斥情况的发生。In practical application scenarios, the first storage device and the second storage device may belong to different block device storage clusters (Elastic Block Service, EBS for short), and the third storage device and the second storage device may belong to the same block device storage cluster. Cross-cluster data migration is time-consuming and more likely to cause "write" mutual exclusion.
本公开通过设置用于作为中间迁移目标的、与第二存储装置属于同一块设备存储集群的第三存储装置，并将第一存储装置中的数据迁移到第三存储装置，再从第三存储装置导入第二存储装置，则可以在一定程度上减少上述互斥情况的发生。In the present disclosure, a third storage device that is used as an intermediate migration target and belongs to the same piece of equipment storage cluster as the second storage device is set, and the data in the first storage device is migrated to the third storage device, and then the data from the third storage device is migrated from the third storage device. If the device is imported into the second storage device, the occurrence of the above-mentioned mutually exclusive situation can be reduced to a certain extent.
在图2所示的基于改进的PostCopy的数据迁移流程的基础上，本公开提出了一种数据迁移过程中的快照方案。Based on the improved PostCopy-based data migration process shown in FIG. 2 , the present disclosure proposes a snapshot solution in the data migration process.
图3示出了根据本公开一个实施例的数据迁移过程中的快照方法的示意性流程图。其中，数据迁移过程可以是指图2所示的基于改进的PostCopy的数据迁移流程。FIG. 3 shows a schematic flowchart of a snapshot method in a data migration process according to an embodiment of the present disclosure. The data migration process may refer to the improved PostCopy-based data migration process shown in FIG. 2 .
如图3所示，在步骤S310，响应于快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照。As shown in FIG. 3 , in step S310, in response to the snapshot request, snapshot processing is performed on the data related to the virtual server instance stored in the second storage device to obtain a first snapshot.
在步骤S320，在迁移完成后对第三存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第二快照。In step S320, after the migration is completed, snapshot processing is performed on the data related to the virtual server instance stored in the third storage device to obtain a second snapshot.
快照请求可以是指将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置这一过程中的快照请求。The snapshot request may refer to a snapshot request in the process of migrating the virtual server instance related data stored in the first storage device and associated with the operation of the virtual server instance to the third storage device.
对第二存储装置进行快照处理得到的第一快照主要包括IO流切换后虚拟服务器实例的数据部分；迁移完成后对第三存储装置进行快照处理得到的第二快照主要包括IO流切换前虚拟服务器实例的数据部分。由此，第一快照和第二快照涵盖了虚拟服务器实例在快照请求对应的时刻的全量数据(即完整数据)，使得可以基于第一快照和第二快照得到虚拟服务器实例在快照请求对应的时刻的全量数据。The first snapshot obtained by performing snapshot processing on the second storage device mainly includes the data part of the virtual server instance after the IO stream switching; the second snapshot obtained by performing snapshot processing on the third storage device after the migration is completed mainly includes the virtual server before the IO stream switching. The data part of the instance. Therefore, the first snapshot and the second snapshot cover the full amount of data (ie, complete data) of the virtual server instance at the moment corresponding to the snapshot request, so that the virtual server instance at the moment corresponding to the snapshot request can be obtained based on the first snapshot and the second snapshot of full data.
本公开可以将虚拟服务器实例在快照请求对应的时刻的全量数据上传到存储服务器进行备份。与第二快照中的数据相比，第一快照中的数据可以视为虚拟服务器实例的新数据，考虑存在覆盖写数据的场景，在获取全量数据进行上传时，可以首先从第一快照中读取数据进行上传，在第一快照中缺失数据(即某个逻辑地址没有数据)的情况下再从第二快照中读取数据(读取该逻辑地址的数据)进行上传。The present disclosure can upload the full amount of data of the virtual server instance at the moment corresponding to the snapshot request to the storage server for backup. Compared with the data in the second snapshot, the data in the first snapshot can be regarded as the new data of the virtual server instance. Considering the scenario of overwriting data, when acquiring the full amount of data for uploading, you can first read the data from the first snapshot. Take the data for uploading, and then read the data from the second snapshot (read the data of the logical address) for uploading when the data is missing in the first snapshot (that is, there is no data in a certain logical address).
具体而言，第一快照能够表征第二存储装置中存储的虚拟服务器实例相关数据以及数据的逻辑地址，第二快照能够表征第三存储装置中存储的虚拟服务器实例相关数据以及数据的逻辑地址。第一快照所表征的第二存储装置中存储的虚拟服务器实例相关数据，可以包括虚拟服务器实例启动所需的数据，以及切换IO流后写入第二存储装置的数据。第二快照所表征的第三存储装置中存储的虚拟服务器实例相关数据，是指迁移到第三存储装置中的所有与虚拟服务器实例相关的数据，如可以是指切换IO流前虚拟服务器实例的全量数据。其中，逻辑地址可以是指逻辑磁盘地址(Logical Block Address，简称LBA)。Specifically, the first snapshot can represent the virtual server instance related data and the logical address of the data stored in the second storage device, and the second snapshot can represent the virtual server instance related data and the logical address of the data stored in the third storage device. The data related to the virtual server instance stored in the second storage device represented by the first snapshot may include data required to start the virtual server instance and data written to the second storage device after switching the IO stream. The data related to the virtual server instance stored in the third storage device represented by the second snapshot refers to all data related to the virtual server instance migrated to the third storage device. For example, it may refer to the data of the virtual server instance before switching the IO stream. Full data. The logical address may refer to a logical disk address (Logical Block Address, LBA for short).
可以首先将第一快照和第二快照的逻辑地址进行合并，得到一个总的逻辑地址集合，对于集合中的每个逻辑地址，若第一快照中存在对应该逻辑地址的数据，则从第一快照中取数据进行上传，若第一快照中没有对应该逻辑地址的数据，再从第二快照中取于该逻辑地址对应的数据进行上传。The logical addresses of the first snapshot and the second snapshot can be combined to obtain a total logical address set. For each logical address in the set, if there is data corresponding to the logical address in the first The data is taken from the snapshot for uploading. If there is no data corresponding to the logical address in the first snapshot, the data corresponding to the logical address is taken from the second snapshot for upload.
作为示例，第一快照可以包括第二存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第一数据位图，第二快照可以包括所述第三存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第二数据位图。As an example, the first snapshot may include virtual server instance related data stored in the second storage device and a first data bitmap used to represent logical addresses of the data, and the second snapshot may include virtual server instance stored in the third storage device. Server instance related data and a second data bitmap for representing the logical address of the data.
可以合并第一数据位图和第二数据位图，以得到能够表征虚拟服务器实例在快照请求对应的时刻的全量数据的逻辑地址的第三数据位图；然后针对第三数据位图中的每个逻辑地址，判断第一快照中是否存在与该逻辑地址对应的数据，若存在则从第一快照中选取与该逻辑地址对应的数据，若不存在则从第二快照中选取与该逻辑地址对应的数据。由此，可以得到第三数据位图中各个逻辑地址对应的数据，即虚拟服务器实例在快照请求对应的时刻的全量数据。The first data bitmap and the second data bitmap can be combined to obtain a third data bitmap that can represent the logical address of the full amount of data of the virtual server instance at the time corresponding to the snapshot request; then for each data bitmap in the third data bitmap a logical address, determine whether there is data corresponding to the logical address in the first snapshot, if so, select the data corresponding to the logical address from the first snapshot, if not, select the data corresponding to the logical address from the second snapshot corresponding data. Thus, the data corresponding to each logical address in the third data bitmap can be obtained, that is, the full amount of data of the virtual server instance at the moment corresponding to the snapshot request.
在对第三存储装置进行快照处理得到第二快照后，还可以将第二快照的数据导入第二存储装置，以使得第二存储装置拥有虚拟服务器实例当前的全量数据。即，可以通过将第二快照的数据导入第二存储装置，实现迁移。After the second snapshot is obtained by performing snapshot processing on the third storage device, the data of the second snapshot may also be imported into the second storage device, so that the second storage device has the current full data of the virtual server instance. That is, the migration can be implemented by importing the data of the second snapshot into the second storage device.
在迁移完成后生成虚拟服务器实例的快照时，由于第二存储装置已经拥有了虚拟服务器实例的全量数据，因此只需对第二存储装置进行快照处理即可。When a snapshot of the virtual server instance is generated after the migration is completed, since the second storage device already has the full amount of data of the virtual server instance, it is only necessary to perform snapshot processing on the second storage device.
综上，本公开设计了一种在云盘PostCopy迁移期间从多个快照(第一快照和第二快照)拼出虚拟服务器实例的全量数据的方法，解决了PreCopy场景下迁移遇到快照生成操作需要回退的缺点，实现了PostCopy迁移期间对快照生成操作的支持。本公开的快照方法无需在响应于快照请求生成快照时立即进行数据补齐，也无需单独增加流程在快照生成操作完成后从源端补充数据从而增加系统复杂度。In summary, the present disclosure devises a method for piecing out the full amount of data of a virtual server instance from multiple snapshots (the first snapshot and the second snapshot) during PostCopy migration of a cloud disk, which solves the problem of the snapshot generation operation encountered during migration in the PreCopy scenario. The disadvantage of requiring a fallback, implements support for snapshot generation operations during PostCopy migrations. The snapshot method of the present disclosure does not need to perform data completion immediately when a snapshot is generated in response to a snapshot request, nor does it require a separate process to supplement data from the source end after the snapshot generation operation is completed, thereby increasing the system complexity.
作为可选实施例，针对图1或图2所示的数据迁移流程，本公开还提出了另一种数据迁移过程中的快照方法。与图3所示的快照方法的不同之处在于，在本实施例中，可以不对第三存储装置生成快照，而是可以对第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据进行快照处理，得到第三快照；基于第三快照和第一快照得到虚拟服务器实例在快照请求对应的时刻的全量数据。As an optional embodiment, for the data migration process shown in FIG. 1 or FIG. 2 , the present disclosure also proposes another snapshot method in the data migration process. The difference from the snapshot method shown in FIG. 3 is that in this embodiment, a snapshot may not be generated for the third storage device, but a virtual server stored in the first storage device and associated with the operation of the virtual server instance may be generated. Snapshot processing is performed on instance-related data to obtain a third snapshot; based on the third snapshot and the first snapshot, full data of the virtual server instance at the moment corresponding to the snapshot request is obtained.
具体而言，第一快照能够表征第二存储装置中存储的虚拟服务器实例相关数据以及数据的逻辑地址，第三快照能够表征第一存储装置中存储的虚拟服务器实例相关数据以及数据的逻辑地址。可以首先将第一快照和第三快照的逻辑地址进行合并，得到一个总的逻辑地址集合，对于集合中的每个逻辑地址，若第一快照中存在对应该逻辑地址的数据，则从第一快照中取数据进行上传，若第一快照中没有对应该逻辑地址的数据，再从第三快照中取于该逻辑地址对应的数据进行上传。Specifically, the first snapshot can represent the virtual server instance related data and the logical address of the data stored in the second storage device, and the third snapshot can represent the virtual server instance related data and the logical address of the data stored in the first storage device. The logical addresses of the first snapshot and the third snapshot can be combined to obtain a total logical address set. For each logical address in the set, if there is data corresponding to the logical address in the first The data is taken from the snapshot for uploading. If there is no data corresponding to the logical address in the first snapshot, the data corresponding to the logical address is taken from the third snapshot for upload.
作为示例，第一快照可以包括第二存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第一数据位图，第三快照可以包括第一存储装置中存储的虚拟服务器实例相关数据以及用于表征数据的逻辑地址的第四数据位图。As an example, the first snapshot may include data related to the virtual server instance stored in the second storage device and a first data bitmap used to represent logical addresses of the data, and the third snapshot may include the virtual server instance stored in the first storage device. associated data and a fourth data bitmap for characterizing the logical address of the data.
可以合并第一数据位图和第四数据位图，以得到能够表征虚拟服务器实例在快照请求对应的时刻的全量数据的逻辑地址的第五数据位图；然后针对第五数据位图中的每个逻辑地址，判断第一快照中是否存在与该逻辑地址对应的数据，若存在则从第一快照中选取与该逻辑地址对应的数据，若不存在则从第三快照中选取与该逻辑地址对应的数据。由此，可以得到第五数据位图中各个逻辑地址对应的数据，即虚拟服务器实例在快照请求对应的时刻的全量数据。The first data bitmap and the fourth data bitmap can be combined to obtain a fifth data bitmap that can represent the logical address of the full amount of data of the virtual server instance at the time corresponding to the snapshot request; then, for each data bitmap in the fifth data bitmap a logical address, determine whether there is data corresponding to the logical address in the first snapshot, if so, select the data corresponding to the logical address from the first snapshot, if not, select the data corresponding to the logical address from the third snapshot corresponding data. Thus, the data corresponding to each logical address in the fifth data bitmap can be obtained, that is, the full amount of data of the virtual server instance at the moment corresponding to the snapshot request.
本公开的数据迁移过程中的快照方法还可以实现为一种数据迁移过程中的快照装置。图4示出了根据本公开一个实施例的快照装置的结构示意图。其中，快照装置的功能单元可以由实现本公开原理的硬件、软件或硬件和软件的结合来实现。本领域技术人员可以理解的是，图4所描述的功能单元可以组合起来或者划分成子单元，从而实现上述发明的原理。因此，本文的描述可以支持对本文描述的功能单元的任何可能的组合、或者划分、或者更进一步的限定。The snapshot method in the data migration process of the present disclosure can also be implemented as a snapshot device in the data migration process. FIG. 4 shows a schematic structural diagram of a snapshot device according to an embodiment of the present disclosure. The functional units of the snapshot device may be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. Those skilled in the art can understand that the functional units described in FIG. 4 can be combined or divided into sub-units, so as to realize the principle of the above invention. Accordingly, the description herein may support any possible combination, or division, or further definition of the functional units described herein.
下面就快照装置可以具有的功能单元以及各功能单元可以执行的操作做简要说明，对于其中涉及的细节部分可以参见上文相关描述，这里不再赘述。The following briefly describes the functional units that the snapshot device can have and the operations that each functional unit can perform. For the details involved, reference may be made to the above related descriptions, which will not be repeated here.
参见图4，快照装置400包括第一迁移模块410、第二迁移模块420、第一快照模块430以及第二快照模块440。Referring to FIG. 4 , the snapshot device 400 includes a first migration module 410 , a second migration module 420 , a first snapshot module 430 and a second snapshot module 440 .
第一迁移模块410、第二迁移模块420可以用于执行上文结合图2所示的基于改进的PostCopy的数据迁移流程。The first migration module 410 and the second migration module 420 may be used to perform the data migration process based on the improved PostCopy shown above in conjunction with FIG. 2 .
具体而言，第一迁移模块410用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求。第二迁移模块420用于将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置。Specifically, the first migration module 410 is configured to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server according to the data stored in the second storage device The server instance, wherein, after the computing device starts the virtual server instance according to the data stored in the second storage device, a data read and write request related to the virtual server instance is executed with respect to the second storage device. The second migration module 420 is configured to migrate the virtual server instance related data stored in the first storage device and associated with the running of the virtual server instance to the third storage device.
第一快照模块430用于响应于快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照。第二快照模块440用于在迁移完成后对第三存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第二快照。The first snapshot module 430 is configured to, in response to the snapshot request, perform snapshot processing on the data related to the virtual server instance stored in the second storage device to obtain a first snapshot. The second snapshot module 440 is configured to perform snapshot processing on the data related to the virtual server instance stored in the third storage device after the migration is completed to obtain a second snapshot.
快照装置400还可以包括全量数据获取模块，用于基于第一快照和第二快照得到所述虚拟服务器实例在所述快照请求对应的时刻的全量数据。关于全量数据的具体获取过程可以参见上文相关描述。The snapshot apparatus 400 may further include a full data acquisition module, configured to obtain full data of the virtual server instance at the moment corresponding to the snapshot request based on the first snapshot and the second snapshot. For the specific acquisition process of the full amount of data, please refer to the above related description.
快照装置400还可以包括上传模块，用于将虚拟服务器实例在快照请求对应的时刻的全量数据上传到存储服务器进行备份。The snapshot apparatus 400 may further include an uploading module configured to upload the full amount of data of the virtual server instance at the moment corresponding to the snapshot request to the storage server for backup.
快照装置400还可以包括导入模块，用于将第二快照的数据导入第二存储装置。The snapshot device 400 may further include an import module for importing the data of the second snapshot into the second storage device.
图5示出了根据本公开一个实施例的数据迁移装置的结构示意图。其中，数据迁移装置的功能单元可以由实现本公开原理的硬件、软件或硬件和软件的结合来实现。本领域技术人员可以理解的是，图5所描述的功能单元可以组合起来或者划分成子单元，从而实现上述发明的原理。因此，本文的描述可以支持对本文描述的功能单元的任何可能的组合、或者划分、或者更进一步的限定。FIG. 5 shows a schematic structural diagram of a data migration apparatus according to an embodiment of the present disclosure. The functional units of the data migration apparatus may be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. Those skilled in the art can understand that the functional units described in FIG. 5 can be combined or divided into sub-units, so as to realize the principle of the above invention. Accordingly, the description herein may support any possible combination, or division, or further definition of the functional units described herein.
数据迁移装置500可以用于执行图2所示的基于改进的PostCopy的数据迁移流程。下面就数据迁移装置500可以具有的功能单元以及各功能单元可以执行的操作做简要说明，对于其中涉及的细节部分可以参见上文相关描述，这里不再赘述。The data migration apparatus 500 can be used to execute the data migration process based on the improved PostCopy shown in FIG. 2 . The following briefly describes the functional units that the data migration apparatus 500 may have and the operations that each functional unit may perform. For the details involved, reference may be made to the above related descriptions, which will not be repeated here.
参见图5，数据迁移装置500包括第一迁移模块510、第二迁移模块520以及导入模块530。Referring to FIG. 5 , the data migration apparatus 500 includes a first migration module 510 , a second migration module 520 and an import module 530 .
第一迁移模块510用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据第二存储装置中存储的数据启动虚拟服务器实例，其中，在计算装置根据第二存储装置中存储的数据启动虚拟服务器实例后，针对第二存储装置执行与虚拟服务器实例相关的数据读写请求；The first migration module 510 is configured to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server instance according to the data stored in the second storage device, wherein , after the computing device starts the virtual server instance according to the data stored in the second storage device, executes data read and write requests related to the virtual server instance for the second storage device;
第二迁移模块520用于将第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据迁移到第三存储装置；The second migration module 520 is configured to migrate the virtual server instance related data stored in the first storage device and associated with the running of the virtual server instance to the third storage device;
导入模块530用于在迁移完成后，将第三存储装置中存储的虚拟服务器实例相关数据导入第二存储装置。作为示例，数据迁移装置500还可以包括快照模块，用于在迁移完成后对第三存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第二快照。导入模块530可以将第二快照的数据导入第二存储装置。The import module 530 is configured to import the virtual server instance-related data stored in the third storage device into the second storage device after the migration is completed. As an example, the data migration apparatus 500 may further include a snapshot module, configured to perform snapshot processing on the data related to the virtual server instance stored in the third storage apparatus after the migration is completed to obtain a second snapshot. The import module 530 may import the data of the second snapshot into the second storage device.
图6示出了根据本公开另一个实施例的数据迁移过程中的快照装置的结构示意图。在本实施例中，数据迁移过程既可以是指图1所示的基于PostCopy的数据迁移流程，也可以是指图2所示的基于改进的PostCopy的数据迁移流程。FIG. 6 shows a schematic structural diagram of a snapshot device in a data migration process according to another embodiment of the present disclosure. In this embodiment, the data migration process may refer to either the PostCopy-based data migration process shown in FIG. 1 or the improved PostCopy-based data migration process shown in FIG. 2 .
参见图6，快照装置600包括迁移模块610、第三快照模块620以及第一快照模块630。Referring to FIG. 6 , the snapshot device 600 includes a migration module 610 , a third snapshot module 620 and a first snapshot module 630 .
迁移模块610用于将第一存储装置中存储的虚拟服务器实例启动所需的数据迁移到第二存储装置，以使得计算装置能够根据所述第二存储装置中存储的数据启动所述虚拟服务器实例，其中，在所述计算装置根据所述第二存储装置中存储的数据启动所述虚拟服务器实例后，针对所述第二存储装置执行与所述虚拟服务器实例相关的数据读写请求。迁移模块610具体可以执行图1所示的基于PostCopy的数据迁移流程，也可以执行图2所示的基于改进的PostCopy的数据迁移流程。The migration module 610 is configured to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server instance according to the data stored in the second storage device , wherein, after the computing device starts the virtual server instance according to the data stored in the second storage device, a data read/write request related to the virtual server instance is executed for the second storage device. The migration module 610 may specifically execute the data migration process based on PostCopy shown in FIG. 1 , or may execute the data migration process based on the improved PostCopy shown in FIG. 2 .
第三快照模块620用于对第一存储装置中存储的与虚拟服务器实例的运行关联的虚拟服务器实例相关数据进行快照处理，得到第三快照；The third snapshot module 620 is configured to perform snapshot processing on the virtual server instance-related data stored in the first storage device and associated with the operation of the virtual server instance to obtain a third snapshot;
第一快照模块630用于响应于迁移过程中的快照请求，对第二存储装置中存储的虚拟服务器实例相关数据进行快照处理，得到第一快照。The first snapshot module 630 is configured to perform snapshot processing on the data related to the virtual server instance stored in the second storage device in response to the snapshot request in the migration process to obtain the first snapshot.
快照装置600还可以包括全量数据获取模块，用于基于第一快照和第三快照得到所述虚拟服务器实例在所述快照请求对应的时刻的全量数据。关于全量数据的具体获取过程可以参见上文相关描述。The snapshot apparatus 600 may further include a full data acquisition module, configured to obtain full data of the virtual server instance at the moment corresponding to the snapshot request based on the first snapshot and the third snapshot. For the specific acquisition process of the full amount of data, please refer to the above related description.
快照装置600还可以包括上传模块，用于将虚拟服务器实例在快照请求对应的时刻的全量数据上传到存储服务器进行备份。The snapshot apparatus 600 may further include an uploading module configured to upload the full amount of data of the virtual server instance at the moment corresponding to the snapshot request to the storage server for backup.
图7示出了根据本公开一实施例可用于实现上述快照方法或数据迁移方法的计算设备的结构示意图。FIG. 7 shows a schematic structural diagram of a computing device that can be used to implement the above snapshot method or data migration method according to an embodiment of the present disclosure.
参见图7，计算设备700包括存储器710和处理器720。Referring to FIG. 7 , computing device 700 includes memory 710 and processor 720 .
处理器720可以是一个多核的处理器，也可以包含多个处理器。在一些实施例中，处理器720可以包含一个通用的主处理器以及一个或多个特殊的协处理器，例如图形处理器(GPU)、数字信号处理器(DSP)等等。在一些实施例中，处理器720可以使用定制的电路实现，例如特定用途集成电路(ASIC，Application Specific Integrated Circuit)或者现场可编程逻辑门阵列(FPGA，Field Programmable Gate Arrays)。The processor 720 may be a multi-core processor, or may include multiple processors. In some embodiments, processor 720 may comprise a general-purpose main processor and one or more special-purpose co-processors, such as a graphics processing unit (GPU), a digital signal processor (DSP), and the like. In some embodiments, the processor 720 may be implemented using customized circuits, such as Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs).
存储器710可以包括各种类型的存储单元，例如系统内存、只读存储器(ROM)，和永久存储装置。其中，ROM可以存储处理器720或者计算机的其他模块需要的静态数据或者指令。永久存储装置可以是可读写的存储装置。永久存储装置可以是即使计算机断电后也不会失去存储的指令和数据的非易失性存储设备。在一些实施方式中，永久性存储装置采用大容量存储装置(例如磁或光盘、闪存)作为永久存储装置。另外一些实施方式中，永久性存储装置可以是可移除的存储设备(例如软盘、光驱)。系统内存可以是可读写存储设备或者易失性可读写存储设备，例如动态随机访问内存。系统内存可以存储一些或者所有处理器在运行时需要的指令和数据。此外，存储器710可以包括任意计算机可读存储媒介的组合，包括各种类型的半导体存储芯片(DRAM，SRAM，SDRAM，闪存，可编程只读存储器)，磁盘和/或光盘也可以采用。在一些实施方式中，存储器710可以包括可读和/或写的可移除的存储设备，例如激光唱片(CD)、只读数字多功能光盘(例如DVD-ROM，双层DVD-ROM)、只读蓝光光盘、超密度光盘、闪存卡(例如SD卡、min SD卡、Micro-SD卡等等)、磁性软盘等等。计算机可读存储媒介不包含载波和通过无线或有线传输的瞬间电子信号。Memory 710 may include various types of storage units, such as system memory, read only memory (ROM), and persistent storage. The ROM may store static data or instructions required by the processor 720 or other modules of the computer. Persistent storage devices may be readable and writable storage devices. Permanent storage may be a non-volatile storage device that does not lose stored instructions and data even if the computer is powered off. In some embodiments, persistent storage devices employ mass storage devices (eg, magnetic or optical disks, flash memory) as persistent storage devices. In other embodiments, persistent storage may be a removable storage device (eg, a floppy disk, an optical drive). System memory can be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. System memory can store some or all of the instructions and data that the processor needs at runtime. Additionally, memory 710 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read only memory), magnetic and/or optical disks may also be employed. In some embodiments, memory 710 may include a removable storage device that is readable and/or writable, such as a compact disc (CD), a read-only digital versatile disc (eg, DVD-ROM, dual-layer DVD-ROM), Read-only Blu-ray Discs, Ultra-Density Discs, Flash Cards (eg SD Card, Min SD Card, Micro-SD Card, etc.), Magnetic Floppy Disks, etc. Computer-readable storage media do not contain carrier waves and transient electronic signals transmitted over wireless or wireline.
存储器710上存储有可执行代码，当可执行代码被处理器720处理时，可以使处理器720执行上文述及的快照方法或数据迁移方法。Executable codes are stored on the memory 710, and when the executable codes are processed by the processor 720, the processor 720 can be caused to execute the above-mentioned snapshot method or data migration method.
应用例Application example
在将一种版本(如版本1.0)的EBS架构集群的云盘通过PostCopy方式迁移到另一种版本(如版本2.0)的EBS架构集群的过程中，可以执行本公开上述快照方案，以支持在PostCopy迁移云盘期间对响应用户对云盘进行打快照操作。During the process of migrating a cloud disk of an EBS architecture cluster of one version (eg, version 1.0) to an EBS architecture cluster of another version (eg, version 2.0) through PostCopy, the above snapshot solution of the present disclosure can be implemented to support the During the cloud disk migration, PostCopy performs a snapshot operation on the cloud disk for the responding user.
云盘PostCopy迁移启动后，新的写IO流会从源集群(如EBS 1.0)切换到目标集群(如EBS 2.0)，导致源集群和目标集群各有一部分该云盘的数据并且都不完整。那么在响应用户打快照请求时，最重要的事情是把打快照那个时刻的数据拼出一份完整数据。After the cloud disk PostCopy migration is started, the new write IO flow will be switched from the source cluster (such as EBS 1.0) to the target cluster (such as EBS 2.0), resulting in the source cluster and the target cluster each having a part of the cloud disk data and incomplete. Then, when responding to the user's snapshot request, the most important thing is to spell out a complete data of the data at the moment of the snapshot.
图8示出了PostCopy迁移过程示意图。Figure 8 shows a schematic diagram of the PostCopy migration process.
clustrer1为源集群，clustrer2为目标集群。device对应于上文述及的第一存储装置，为源集群中要迁移的云盘，device2对应于上文述及的第二存储装置，为目标集群中的目标盘，device1对应于上文述及的第三存储装置，为目标集群中的中间盘(tempsevice)。clusterer1 is the source cluster and clusterer2 is the target cluster. device corresponds to the first storage device mentioned above, which is the cloud disk to be migrated in the source cluster, device2 corresponds to the second storage device mentioned above, and is the target disk in the target cluster, and device1 corresponds to the aforementioned and the third storage device, which is a tempsevice in the target cluster.
device1作为临时盘，可以在后台接收device的迁移数据。并且，在device1接收device的迁移数据完成后，对device1执行快照处理得到snapshot1，使用snapshot1将数据导入(import)到device2中。snapshot1对应于上文述及的第二快照。As a temporary disk, device1 can receive the migration data of the device in the background. And, after the device1 receives the migration data of the device, it performs a snapshot process on the device1 to obtain the snapshot1, and uses the snapshot1 to import the data into the device2. snapshot1 corresponds to the second snapshot mentioned above.
云盘PostCopy迁移启动前，来自数据中心(TDC)的IO流(数据读写)均针对device执行，云盘PostCopy迁移启动后，新的IO流切换到device2。对于切换IO流切换后读未在device2中命中的数据则可以从device读取。Before the cloud disk PostCopy migration is started, the IO streams (data read and write) from the data center (TDC) are all performed on the device. After the cloud disk PostCopy migration is started, the new IO stream is switched to device2. For data that is not hit in device2 after switching IO stream switching, it can be read from device.
由此，在云盘PostCopy迁移过程中，在前台IO流切换到device2，在后台将device的数据迁移到device1，迁移过程中前台和后台互不干扰。Therefore, during the cloud disk PostCopy migration process, the IO stream in the foreground is switched to device2, and the data of device is migrated to device1 in the background, and the foreground and background do not interfere with each other during the migration process.
图9示出了PostCopy迁移过程中获取完整快照的流程示意图。FIG. 9 shows a schematic flowchart of obtaining a complete snapshot during PostCopy migration.
1、用户打快照1. The user takes a snapshot
在PostCopy迁移过程中(可以在由目标集群管控流程)接收到用户下发的打快照请求时，可以对目标集群已有的数据进行打快照处理，这时候得到用户快照snapshot。snapshot对应于上文述及的第一快照。snapshot拥有自己的数据和表示写入位置的数据位图verMap。(注意snapshot只有一部分数据，另外一部分数据在迁移源集群上)。During the PostCopy migration process (which can be managed and controlled by the target cluster), when a snapshot request issued by the user is received, the existing data in the target cluster can be snapshotted, and the user snapshot snapshot can be obtained at this time. The snapshot corresponds to the first snapshot mentioned above. A snapshot has its own data and a data bitmap verMap representing the write location. (Note that the snapshot only has a part of the data, and the other part of the data is on the migration source cluster).
2、中间盘打快照2. Take a snapshot of the intermediate disk
在目标集群中间盘device1迁移完成后打一个内部快照并转换为外部(external)快照snapshot1，snapshot1包含自己的数据和表示写入位置的数据位图verMap1。其中，在一些情况(如EBS 2.0架构)下，只有external快照才能用于导入数据。After the migration of the intermediate disk device1 of the target cluster is completed, an internal snapshot is taken and converted to an external (external) snapshot snapshot1. The snapshot1 contains its own data and the data bitmap verMap1 representing the write location. Among them, in some cases (such as EBS 2.0 architecture), only external snapshots can be used to import data.
3、snapshot1导入3, snapshot1 import
可以将snapshot1的数据导入到目标盘device2。此时device2用有全量数据，可以为后续打快照提供全量数据。The data of snapshot1 can be imported to the target disk device2. At this point, device2 has full data and can provide full data for subsequent snapshots.
4、snapshot上传OSS完成备份4. Snapshot upload to OSS to complete backup
此时通过snapshot和snapshot1可以拼出完整数据。具体拼接流程如下：At this point, the complete data can be spelled out through snapshot and snapshot1. The specific splicing process is as follows:
1)通过snapshot与snapshot1各自的verMap合并，得到完整的数据位图；1) The complete data bitmap is obtained by merging the respective verMaps of snapshot and snapshot1;
2)通过上一步得到的完整数据位图，获取对应的数据块进行上传OSS(ObjectStorage Service，对象存储)处理。这里数据块有两个来源：PostCopy切IO流后写入的数据在snapshot上，PostCopy切IO流前写入的数据在snapshot1上。考虑存在覆盖写数据块的场景，在上传时取数据仅需确保取数据先取snapshot，如果没有再从snapshot1上取。2) Through the complete data bitmap obtained in the previous step, the corresponding data block is obtained and uploaded to OSS (Object Storage Service, object storage) for processing. There are two sources of data blocks here: the data written after PostCopy cuts the IO stream is on snapshot, and the data written before PostCopy cuts the IO stream is on snapshot1. Considering the scenario of overwriting data blocks, when fetching data during uploading, you only need to ensure that the snapshot is fetched first, and if not, it is fetched from snapshot1.
综上，本实施例为云盘PostCopy迁移期间提供了一种云盘打快照的方法，该方法无需在打快照时立即进行快照数据补齐，也无需单独增加流程在打快照完成后从源端补充数据从而增加系统复杂度。To sum up, this embodiment provides a method for taking snapshots of cloud disks during PostCopy migration of cloud disks. This method does not need to perform snapshot data completion immediately when taking snapshots, nor does it need to add a separate process to send snapshots from the source end after taking snapshots. Supplementary data increases system complexity.
作为可选实施例，在打完快照后，快照拥有一部分数据，剩余的数据可以从源盘device获取，但相比本实施例实现更复杂。As an optional embodiment, after the snapshot is taken, the snapshot has some data, and the remaining data can be obtained from the source disk device, but the implementation is more complicated than this embodiment.
上文中已经参考附图详细描述了根据本公开的快照方法、数据迁移方法、装置以及计算设备。The snapshot method, data migration method, apparatus, and computing device according to the present disclosure have been described in detail above with reference to the accompanying drawings.
此外，根据本公开的方法还可以实现为一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括用于执行本公开的上述方法中限定的上述各步骤的计算机程序代码指令。Furthermore, the method according to the present disclosure can also be implemented as a computer program or computer program product comprising computer program code instructions for performing the above-mentioned steps defined in the above-mentioned method of the present disclosure.
或者，本公开还可以实施为一种非暂时性机器可读存储介质(或计算机可读存储介质、或机器可读存储介质)，其上存储有可执行代码(或计算机程序、或计算机指令代码)，当所述可执行代码(或计算机程序、或计算机指令代码)被电子设备(或计算设备、服务器等)的处理器执行时，使所述处理器执行根据本公开的上述方法的各个步骤。Alternatively, the present disclosure can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having executable code (or computer program, or computer instruction code stored thereon) ), when the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or a computing device, a server, etc.), the processor is caused to perform each step of the above-mentioned method according to the present disclosure .
本领域技术人员还将明白的是，结合这里的公开所描述的各种示例性逻辑块、模块、电路和算法步骤可以被实现为电子硬件、计算机软件或两者的组合。Those skilled in the art will also appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.
附图中的流程图和框图显示了根据本公开的多个实施例的系统和方法的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标记的功能也可以以不同于附图中所标记的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods in accordance with various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.
Claims (14)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111462665.0ACN114385309A (en) | 2021-12-02 | 2021-12-02 | Data migration method, data snapshot method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111462665.0ACN114385309A (en) | 2021-12-02 | 2021-12-02 | Data migration method, data snapshot method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114385309Atrue CN114385309A (en) | 2022-04-22 |
Family
ID=81195080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111462665.0APendingCN114385309A (en) | 2021-12-02 | 2021-12-02 | Data migration method, data snapshot method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114385309A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450605A (en)* | 2023-03-17 | 2023-07-18 | 深圳华为云计算技术有限公司 | A data migration method, device, storage medium and chip system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102741820A (en)* | 2010-02-08 | 2012-10-17 | 微软公司 | Background migration of virtual storage |
| CN107247619A (en)* | 2017-06-12 | 2017-10-13 | 上海优刻得信息科技有限公司 | Live migration of virtual machine method, device, system, storage medium and equipment |
| CN107885580A (en)* | 2017-11-22 | 2018-04-06 | 深信服科技股份有限公司 | A kind of cloud moving method, device, equipment and storage medium |
| CN109842636A (en)* | 2017-11-24 | 2019-06-04 | 阿里巴巴集团控股有限公司 | Cloud service moving method, device and electronic equipment |
- 2021
- 2021-12-02CNCN202111462665.0Apatent/CN114385309A/enactivePending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102741820A (en)* | 2010-02-08 | 2012-10-17 | 微软公司 | Background migration of virtual storage |
| CN107247619A (en)* | 2017-06-12 | 2017-10-13 | 上海优刻得信息科技有限公司 | Live migration of virtual machine method, device, system, storage medium and equipment |
| CN107885580A (en)* | 2017-11-22 | 2018-04-06 | 深信服科技股份有限公司 | A kind of cloud moving method, device, equipment and storage medium |
| CN109842636A (en)* | 2017-11-24 | 2019-06-04 | 阿里巴巴集团控股有限公司 | Cloud service moving method, device and electronic equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450605A (en)* | 2023-03-17 | 2023-07-18 | 深圳华为云计算技术有限公司 | A data migration method, device, storage medium and chip system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20250028472A1 (en) | Volume Duplication Using Logical Identifiers | |
| US11604708B2 (en) | Memory first live snapshot | |
| US10198377B2 (en) | Virtual machine state replication using DMA write records | |
| US10379967B2 (en) | Live rollback for a computing environment | |
| US8689211B2 (en) | Live migration of virtual machines in a computing environment | |
| CN106569738B (en) | A kind of method and device for deleting cascade snapshot | |
| CN103034566B (en) | Method and device for restoring virtual machine | |
| US11182084B2 (en) | Restorable memory allocator | |
| CN106528327A (en) | Data processing method and backup server | |
| CN111158858B (en) | Cloning method and device of virtual machine and computer readable storage medium | |
| US10496492B2 (en) | Virtual machine backup with efficient checkpoint handling based on a consistent state of the virtual machine of history data and a backup type of a current consistent state of the virtual machine | |
| US10853185B1 (en) | Virtual machine image backup to exclude system page file | |
| US11231953B2 (en) | Minimizing downtime when importing virtual machines from other platforms | |
| CN109416617B (en) | Replication between heterogeneous storage systems | |
| CN114385309A (en) | Data migration method, data snapshot method and device | |
| JP2017531892A (en) | Improved apparatus and method for performing a snapshot of a block level storage device | |
| US12124866B2 (en) | Fast virtual machine resume at host upgrade | |
| US11514002B2 (en) | Indexing splitter for any pit replication | |
| CN106775846B (en) | Method and apparatus for online migration of physical servers | |
| CN116700904B (en) | Memory snapshot generation method and device, computer equipment and storage medium | |
| EP3264254A1 (en) | System and method for a simulation of a block storage system on an object storage system | |
| CN116955004A (en) | Incremental data backup methods, devices, computer equipment and storage media | |
| CN107483590A (en) | Cloud management system and method in cloud data system | |
| CN105389185A (en) | Method and apparatus for processing boot storm of virtual cloud desktops | |
| US9880904B2 (en) | Supporting multiple backup applications using a single change tracker |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |