CN109165095B - The method of rapid opening of information system based on task package - Google Patents
The method of rapid opening of information system based on task packageDownload PDFInfo
- Publication number
- CN109165095B CN109165095BCN201810933672.6ACN201810933672ACN109165095BCN 109165095 BCN109165095 BCN 109165095BCN 201810933672 ACN201810933672 ACN 201810933672ACN 109165095 BCN109165095 BCN 109165095B
- Authority
- CN
- China
- Prior art keywords
- service
- unit
- data
- migration
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及计算机领域，具体来讲涉及一种基于任务套餐的信息系统快速开设方法。The invention relates to the field of computers, in particular to a method for quickly opening an information system based on a task package.
背景技术Background technique
云平台可以将物理资源虚拟化为容器(虚拟机)资源池，灵活调用软硬件资源，实现对用户的按需访问。而且在运行过程中根据用户并发量不同，能够实时迁移容器(虚拟机)资源，一方面保证提供高质量服务，另一方面最小化资源成本，提高CPU、内存等的利用率。所述迁移包括服务迁移和数据迁移，服务迁移是将应用及服务从云平台或者服务框架中平滑转移到另一云平台或服务框架中，数据迁移是将某一存储中心的数据拷贝至另一存储中心。现有技术中服务迁移和数据迁移通常是分开进行的，由于服务和服务间的依赖关系不清晰、服务和数据间的依赖关系不清晰，在迁移过程中会造成冗余迁移并增大了时间成本，这样不仅造成资源浪费还会影响迁移效率，更不利于迁移后的服务及数据的组织。The cloud platform can virtualize physical resources into container (virtual machine) resource pools, flexibly call software and hardware resources, and realize on-demand access to users. Moreover, according to the different concurrency of users, container (virtual machine) resources can be migrated in real time during operation, which ensures the provision of high-quality services, minimizes resource costs, and improves the utilization of CPU and memory. The migration includes service migration and data migration. Service migration is to smoothly transfer applications and services from a cloud platform or service framework to another cloud platform or service framework. Data migration is to copy data from one storage center to another. storage center. In the prior art, service migration and data migration are usually carried out separately. Due to unclear dependencies between services and services, and unclear dependencies between services and data, redundant migration will be caused during the migration process and the time will be increased. This not only wastes resources, but also affects migration efficiency, and is not conducive to the organization of services and data after migration.
为此，本发明提出了一种基于任务套餐的信息系统快速开设方法，任务套餐即为应用方可以选择需要开设的服务，基于这些服务以及所依赖的服务和数据形成的新的信息系统会被开设到指定的平台上。本方案中解决了服务对服务依赖、服务对数据依赖的问题，这样在开设过程中，应用方只需要选择其关注的服务即可，而这些服务所依赖的服务和数据会随同任务套餐一同被开设到新的平台上。由于所有的服务和数据都是清晰的，所以在新的平台上重新复现整套信息系统的过程是快捷的。此外在开设时，任务套餐中所有的服务和数据可以并行的迁移，从而达到了快速迁移的效果。To this end, the present invention proposes a method for quickly opening an information system based on a task package. The task package is that the application side can select the services to be opened, and the new information system formed based on these services and the services and data they depend on will be Open to the designated platform. This solution solves the problems of service dependence on services and data dependence on services, so that in the process of opening, the application side only needs to select the services it cares about, and the services and data that these services depend on will be included with the task package. Open to a new platform. Since all services and data are clear, the process of re-reproducing the entire information system on the new platform is fast. In addition, when opening, all services and data in the task package can be migrated in parallel, thus achieving the effect of rapid migration.
发明内容SUMMARY OF THE INVENTION
为实现本发明之目的，采用以下技术方案予以实现：For realizing the purpose of the present invention, adopt following technical scheme to realize:
一种信息系统的快速开设方法，包括以下步骤：A method for quickly opening an information system, comprising the following steps:
(1)新建一个开设任务，选定开设的源端及目的端；(1) Create a new opening task, and select the source and destination to be opened;
(2)从源端选择开设任务所需的服务单元；(2) Select the service unit required to open the task from the source end;
(3)根据步骤(2)中选择的服务单元自动搜索所述服务单元所依赖的服务单元和数据单元，生成任务套餐；(3) according to the service unit selected in step (2), the service unit and the data unit that the described service unit depends on are automatically searched, and the task package is generated;
(4)将任务套餐中的服务单元和数据单元由源端并行迁移到目的端。(4) Migrate the service units and data units in the task package from the source end to the destination end in parallel.
所述的方法，其中：步骤(2)包括提供源端的服务列表，选择开设任务所需的服务单元。The method, wherein: step (2) includes providing a service list of the source end, and selecting a service unit required for opening a task.
所述的方法，其中：所述步骤(3)具体为：根据步骤(2)中选择的服务单元，从源端的元数据存储组件中的信息中自动搜索这些服务单元所依赖的服务单元和数据单元，生成任务套餐。The method, wherein: the step (3) is specifically: according to the service unit selected in the step (2), from the information in the metadata storage component of the source end, the service unit and data that these service units depend on are automatically searched. unit to generate task packages.
所述的方法，其中：所述步骤(4)中的并行迁移是按如下方式进行的：为每个服务单元和数据单元启动一个或多个线程，以完成该服务单元或数据单元的迁移。The method, wherein: the parallel migration in the step (4) is performed as follows: for each service unit and data unit, one or more threads are started to complete the migration of the service unit or data unit.
所述的方法，其中：在步骤(1)之前，还包括创建服务单元和数据单元的步骤。The method, wherein: before step (1), it further comprises the steps of creating a service unit and a data unit.
所述的方法，其中：创建服务单元和数据单元包括：为每个服务单元定义一个对象，这个对象中包括多个成员变量，包括服务名、服务类别、服务实例存储位置、服务实例镜像名、服务所属的分系统，从而将一个服务实例抽象成一个服务单元；为每个数据单元定义一个对象，这个对象中包括多个成员变量，包括数据名、数据类别、数据实例存储位置、数据所属的分系统，从而将一个数据实例抽象成一个数据单元。The method, wherein: creating a service unit and a data unit includes: defining an object for each service unit, and this object includes a plurality of member variables, including service name, service category, service instance storage location, service instance image name, The subsystem to which the service belongs, thereby abstracting a service instance into a service unit; an object is defined for each data unit, and this object includes multiple member variables, including data name, data category, data instance storage location, and data to which the data belongs. Subsystem, thereby abstracting a data instance into a data unit.
所述的方法，其中：服务单元的对象中还包括服务的版本信息。The method, wherein: the object of the service unit further includes version information of the service.
所述的方法，其中：元数据存储组件中的信息包括：(1)待迁移的服务所依赖的服务的信息，包括所依赖的服务的服务名、服务类型、服务存储位置；(2)待迁移的服务所依赖的数据单元信息，包括所依赖的数据单元的数据单元名、数据单元类型、数据单元存储位置。The method, wherein: the information in the metadata storage component includes: (1) the information of the service that the service to be migrated depends on, including the service name, service type, and service storage location of the dependent service; (2) the service to be migrated The data unit information that the migrated service depends on, including the data unit name, data unit type, and data unit storage location of the dependent data unit.
所述的方法，其中：元数据存储组件中的信息还包括服务的版本信息。The method, wherein: the information in the metadata storage component further includes version information of the service.
所述的方法，其中：根据步骤(2)中选择的服务单元，从源端的元数据存储组件中的信息中自动搜索这些服务单元所依赖的服务单元和数据单元，形成服务单元和数据单元之间的依赖森林图，若森林图中存在重复的服务单元或者数据单元，去重并生成任务队列，队列中存储着所有需要迁移的服务单元和数据单元，即完成任务套餐的生成。The described method, wherein: according to the service unit selected in step (2), from the information in the metadata storage component of the source end, the service unit and the data unit that these service units depend on are automatically searched to form a relationship between the service unit and the data unit. The dependency forest diagram between the two, if there are duplicate service units or data units in the forest diagram, remove the duplication and generate a task queue. The queue stores all the service units and data units that need to be migrated, that is, the generation of the task package is completed.
所述的方法，其中：步骤(4)中将服务单元和数据单元由源端并行迁移到目的端采用断点续传的方式进行。The method, wherein: in step (4), the parallel migration of the service unit and the data unit from the source end to the destination end is performed in a way of continuous transmission at a break point.
所述的方法，其中：所述断点续传的方式包括：The method, wherein: the method of resuming the transmission from a breakpoint includes:
迁移单元在传输过程中记录每条线程的传输进度，将传输记录存储在元数据存储组件中；每次传输开始之前迁移单元先读取数据存储组件，查询是否有未完成的记录，如果有就继续传输，如果没有则创建新记录并存储在元数据存储组件中；在每次传输完预定数量的数据后，迁移单元在数据存储组件中更新传输进度；每条线程传输完成之后迁移单元删除元数据存储组件中相关的传输记录。The migration unit records the transmission progress of each thread during the transmission process, and stores the transmission record in the metadata storage component; before each transmission starts, the migration unit first reads the data storage component, inquires whether there are unfinished records, and if so, Continue the transmission, if not, create a new record and store it in the metadata storage component; after each transmission of a predetermined amount of data, the migration unit updates the transmission progress in the data storage component; after each thread transmission is completed, the migration unit deletes the metadata The associated transfer record in the data store component.
所述的方法，其中：步骤(2)中从源端选择开设任务所需的服务单元时，如果从源端所选择的服务单元在目的端存在，则从目的端元数据存储组件获得该服务的版本信息，并与目的端的服务的版本信息比对，在开设界面上提示用户版本信息比对结果，并提示用户是否还要选取该服务单元。The described method, wherein: in step (2), when the service unit required for opening the task is selected from the source end, if the service unit selected from the source end exists at the destination end, then the service is obtained from the destination end metadata storage component The version information is compared with the version information of the service at the destination end, and the user is prompted on the opening interface of the version information comparison result, and the user is prompted whether to select the service unit.
所述的方法，其中：步骤(3)在搜索到所述选择的服务单元所依赖的服务单元时，如果该依赖的服务单元在目的端存在，则从目的端元数据存储组件获得该服务的版本信息，并与目的端的服务的版本信息比对，且不向用户发出提示，而是将比对结果存入源端元数据存储组件中。The method, wherein: in step (3), when the service unit that the selected service unit depends on is searched, if the dependent service unit exists at the destination end, the service unit is obtained from the destination end metadata storage component. The version information is compared with the version information of the destination service, and the user is not prompted, but the comparison result is stored in the source metadata storage component.
所述的方法，其中：将任务套餐中的服务单元和数据单元由源端并行迁移到目的端前，先对目的端已存在的服务单元进行备份，备份的流程为先将服务单元的实例停止服务，将实例对应的镜像或者安装目录另存，备份成功后才会开始源端服务单元迁移的过程；对于数据单元，同样先将目的端的数据单元备份，备份的流程为将数据单元对应的数据目录或表另存，备份完成后才会开始源端数据单元迁移的过程。The method, wherein: before migrating the service unit and the data unit in the task package from the source end to the destination end in parallel, the existing service unit of the destination end is backed up, and the backup process is to stop the instance of the service unit first. For services, save the image or installation directory corresponding to the instance as a file, and the process of migrating the source-side service unit will start after the backup is successful; for the data unit, the data unit of the destination end is also backed up first, and the backup process is to copy the data unit corresponding to the data directory. Or the table is saved, and the source data unit migration process will not start until the backup is completed.
所述的方法，如果目的端存在被迁移的服务单元，则向用户发出是否迁移该服务单元的信息，如果用户选择继续迁移，则迁移单元将该服务迁移至目的端，如果用户选择放弃该服务单元的迁移，则不再迁移该服务单元。In the described method, if there is a service unit to be migrated at the destination end, the information on whether to migrate the service unit is sent to the user, if the user chooses to continue the migration, the migration unit migrates the service to the destination end, and if the user chooses to abandon the service If the unit is migrated, the service unit will no longer be migrated.
一种信息系统的快速开设系统，包括开设启动单元、服务选择单元、任务生成单元和迁移单元。A rapid opening system of an information system includes an opening start unit, a service selection unit, a task generation unit and a migration unit.
所述的系统，其中：开设启动单元用于新建一个开设任务，选定开设的源端及目的端，并调用服务选择单元；服务选择单元用于根据源端的元数据信息提供服务列表，服务列表包括源端所有的服务名称，供应用方选取需要开设的服务单元；The described system, wherein: opening a start-up unit is used to create a new opening task, select a source end and a destination end to be opened, and call a service selection unit; the service selection unit is used to provide a service list according to the metadata information of the source end, and the service list Including all the service names of the source, for the supplier to select the service unit to be opened;
任务生成单元用于根据应用方选取的服务单元，搜索该服务单元所依赖的服务单元和数据单元，最终生成一个任务套餐，在应用方启动开启任务后调用迁移单元；迁移单元用于对服务和数据进行开设。The task generation unit is used to search for the service unit and data unit that the service unit depends on according to the service unit selected by the application side, finally generate a task package, and call the migration unit after the application side starts the task; the migration unit is used for service and data is opened.
所述的系统，其中：迁移单元对服务和数据进行开设时，首先读取包含了服务单元和数据单元的任务队列，为每个服务单元和数据单元启动一个甚至多个线程完成迁移过程，进行并行迁移。The system, wherein: when the migration unit opens the service and data, it first reads the task queue that includes the service unit and the data unit, and starts one or more threads for each service unit and data unit to complete the migration process, and then performs the migration process. Parallel migration.
所述的系统，其中：服务选择单元还用于创建服务单元和数据单元，包括：(1)为每个服务单元定义一个对象，这个对象中包括多个成员变量，包括服务名、服务类别、服务实例存储位置、服务实例镜像名、服务所属的分系统，从而将一个服务实例抽象成一个服务单元；(2)为每个数据单元定义一个对象，这个对象中包括多个成员变量，包括数据名、数据类别、数据实例存储位置、数据所属的分系统，从而将一个数据实例抽象成一个数据单元。The system, wherein: the service selection unit is also used to create a service unit and a data unit, including: (1) define an object for each service unit, and this object includes a plurality of member variables, including service name, service category, The storage location of the service instance, the image name of the service instance, and the subsystem to which the service belongs, thereby abstracting a service instance into a service unit; (2) Define an object for each data unit, which includes multiple member variables, including data Name, data category, data instance storage location, and the subsystem to which the data belongs, thereby abstracting a data instance into a data unit.
所述的系统，其中：服务单元的对象中还包括服务的版本信息。The system, wherein: the object of the service unit further includes version information of the service.
所述的系统，其中：元数据存储组件中的信息包括：(1)待迁移的服务所依赖的服务的信息，包括所依赖的服务的服务名、服务类型、服务存储位置；(2)待迁移的服务所依赖的数据单元信息，包括所依赖的数据单元的数据单元名、数据单元类型、数据单元存储位置。The system, wherein: the information in the metadata storage component includes: (1) the information of the service that the service to be migrated depends on, including the service name, service type, and service storage location of the service to be migrated; (2) the service to be migrated The data unit information that the migrated service depends on, including the data unit name, data unit type, and data unit storage location of the dependent data unit.
所述的系统，其中：元数据存储组件中的信息还包括服务的版本信息。The system, wherein: the information in the metadata storage component further includes version information of the service.
所述的系统，其中：迁移单元对服务和数据进行开设采用断点续传的方式，所述断点续传的方式包括：The described system, wherein: the migrating unit opens the service and data by adopting the mode of resuming transmission from a breakpoint, and the method of resuming transmission from a breakpoint includes:
迁移单元在传输过程中记录每条线程的传输进度，将传输记录存储在元数据存储组件中；每次传输开始之前迁移单元先读取数据存储组件，查询是否有未完成的记录，如果有就继续传输，如果没有则创建新记录并存储在元数据存储组件中；在每次传输完预定数量的数据后，迁移单元在数据存储组件中更新传输进度；每条线程传输完成之后迁移单元删除元数据存储组件中相关的传输记录。The migration unit records the transmission progress of each thread during the transmission process, and stores the transmission record in the metadata storage component; before each transmission starts, the migration unit first reads the data storage component, inquires whether there are unfinished records, and if so, Continue the transmission, if not, create a new record and store it in the metadata storage component; after each transmission of a predetermined amount of data, the migration unit updates the transmission progress in the data storage component; after each thread transmission is completed, the migration unit deletes the metadata The associated transfer record in the data store component.
所述的系统，其中：从源端选择开设任务所需的服务单元时，如果从源端所选择的服务单元在目的端存在，则服务选择单元从目的端元数据存储组件获得该服务的版本信息，并与目的端的服务的版本信息比对，在开设界面上提示用户版本信息比对结果，并提示用户是否还要选取该服务单元。The described system, wherein: when the service unit required for opening a task is selected from the source end, if the service unit selected from the source end exists at the destination end, the service selection unit obtains the version of the service from the destination end metadata storage component The information is compared with the version information of the service at the destination end, and the user is prompted for the version information comparison result on the opening interface, and the user is prompted whether to select the service unit.
所述的系统，其中：任务生成单元在搜索到所述选择的服务单元所依赖的服务单元时，如果该依赖的服务单元在目的端存在，则从目的端元数据存储组件获得该服务的版本信息，并与目的端的服务的版本信息比对，且不向用户发出提示，而是将比对结果存入源端元数据存储组件中。The system, wherein: when the task generation unit searches for the service unit that the selected service unit depends on, if the dependent service unit exists at the destination end, then obtains the version of the service from the destination end metadata storage component information, and compare it with the version information of the service at the destination end, without prompting the user, but storing the comparison result in the metadata storage component at the source end.
所述的系统，其中：迁移单元将任务套餐中的服务单元和数据单元由源端并行迁移到目的端前，先对目的端已存在的服务单元进行备份，备份的流程为先将服务单元的实例停止服务，将实例对应的镜像或者安装目录另存，备份成功后才会开始源端服务单元迁移的过程；对于数据单元，同样先将目的端的数据单元备份，备份的流程为将数据单元对应的数据目录或表另存，备份完成后才会开始源端数据单元迁移的过程。The system, wherein: before the migration unit migrates the service units and data units in the task package from the source end to the destination end in parallel, it first backs up the existing service units at the destination end, and the backup process is to Stop the instance from service, save the corresponding image or installation directory of the instance, and start the process of migrating the source service unit after the backup is successful; for the data unit, also back up the data unit of the destination end first, and the backup process is to copy the data unit corresponding to the data unit The data directory or table is saved, and the process of migrating the source data unit will not start until the backup is completed.
所述的系统，如果目的端存在被迁移的服务单元，则迁移单元向用户发出是否迁移该服务单元的信息，如果用户选择继续迁移，则迁移单元将该服务迁移至目的端，如果用户选择放弃该服务单元的迁移，则不再迁移该服务单元。In the described system, if there is a migrated service unit at the destination, the migration unit sends information to the user whether to migrate the service unit, if the user chooses to continue the migration, the migration unit migrates the service to the destination, and if the user chooses to give up If the service unit is migrated, the service unit is no longer migrated.
附图说明Description of drawings
图1为本发明的任务套餐服务单元及数据单元依赖森林图；Fig. 1 is a task package service unit and a data unit dependency forest diagram of the present invention;
图2为本发明信息系统的快速开设方法的流程图；Fig. 2 is the flow chart of the quick opening method of the information system of the present invention;
图3为本发明信息系统的快速开设系统示意图。FIG. 3 is a schematic diagram of a quick opening system of the information system of the present invention.
具体实施方式Detailed ways
以下结合附图1-3通过实施例对本发明作进一步详细说明：Below in conjunction with accompanying drawing 1-3, the present invention will be described in further detail by embodiment:
图1中，服务单元是指：在信息系统中，能够独立对外提供信息交互、数据处理、流量转发的应用服务被称为服务单元，其实体可以是云平台中完成封装的镜像文件,也可以是服务的安装目录；数据单元是指：在信息系统中，被某一服务所依赖的数据集合被称为数据单元，一个数据单元可以包含其他数据单元，但不可以依赖其他数据单元。其实体可以是文件系统上的一个目录，包括目录下的文件和目录，也可以是数据库中的库表所对应的目录及文件。本发明通过将服务和数据抽象化以构建服务单元和数据单元，即为每个服务单元定义一个对象，这个对象中包括多个成员变量，包括服务名、服务类别、服务实例存储位置、服务实例镜像名、服务所属的分系统等，优选的还包括服务的版本信息，通过这样的描述方式将一个服务实例抽象成一个服务单元。为每个数据单元定义一个对象，这个对象中包括多个成员变量，包括数据名、数据类别、数据实例存储位置、数据所属的分系统等，通过这样的描述方式将一个数据实例抽象成一个数据单元。In Figure 1, a service unit refers to: in an information system, an application service that can independently provide information exchange, data processing, and traffic forwarding is called a service unit, and its entity can be an encapsulated image file in a cloud platform, or It is the installation directory of the service; the data unit refers to: in the information system, the data set that a service depends on is called a data unit. A data unit can contain other data units, but cannot depend on other data units. Its entity can be a directory on the file system, including the files and directories under the directory, or the directories and files corresponding to the library table in the database. The present invention constructs service units and data units by abstracting services and data, that is, an object is defined for each service unit, and this object includes a plurality of member variables, including service name, service category, service instance storage location, service instance The image name, the subsystem to which the service belongs, etc., preferably also includes the version information of the service, and a service instance is abstracted into a service unit through such a description method. Define an object for each data unit, which includes multiple member variables, including data name, data category, data instance storage location, subsystem to which the data belongs, etc. A data instance is abstracted into a data through this description unit.
在信息系统中，服务A若要依赖服务B，则在服务A的元数据存储组件中，服务A会把其依赖服务B的信息注册进去，主要包括服务B的服务名、服务类型、服务存储位置、版本信息等，这一过程称为注册服务单元；在信息系统中，服务A若要依赖数据单元A，则在服务A的元数据存储组件中，服务A会把其依赖数据单元A的信息注册进去，主要包括数据单元A的数据单元名、数据单元类型、数据单元存储位置等，这一过程称作注册数据单元；上述元数据存储组件是指可持久化的高性能存储软件(模块)，该类存储组件通常将数据存储在内存中以达到读写的高性能，同时其还可以根据一定规则，比如定期、定量、异常情况下强制落盘等将内存中的数据存储到磁盘中，完成数据持久化，保证数据不丢失，该组件用于存储服务单元和数据单元的所有元数据信息。In the information system, if service A wants to depend on service B, in the metadata storage component of service A, service A will register the information of its dependent service B, mainly including the service name, service type, service storage of service B Location, version information, etc., this process is called registration service unit; in the information system, if service A depends on data unit A, then in the metadata storage component of service A, service A will depend on data unit A. The information is registered, mainly including the data unit name, data unit type, data unit storage location, etc. of data unit A. This process is called registering data units; the above metadata storage components refer to persistent high-performance storage software (modules). ), this type of storage component usually stores data in memory to achieve high performance of reading and writing, and it can also store data in memory to disk according to certain rules, such as regular, quantitative, and forced disk placement under abnormal conditions. , completes data persistence and ensures that data is not lost. This component is used to store all metadata information of service units and data units.
开设是针对信息系统的，对于开设会存在源端和目的端，开设的方向是从源端到目的端。源端和目的端预先安装有基础平台，所谓基础平台就是指承载各种信息系统的平台，现多为云平台；信息系统是指运行在云平台上的各个应用服务组成的系统。所以开设就是将源端云平台上的信息系统在目的端云平台上重现的过程。The opening is for the information system. For the opening, there will be a source end and a destination end, and the opening direction is from the source end to the destination end. The source end and the destination end are pre-installed with a basic platform. The so-called basic platform refers to the platform that carries various information systems, most of which are now cloud platforms; the information system refers to the system composed of various application services running on the cloud platform. Therefore, opening is the process of reproducing the information system on the source cloud platform on the destination cloud platform.
当有开设需求时，应用方根据需求在源端快速开设系统页面上选择相应的服务单元，系统会根据元数据存储组件中的信息自动搜索这些服务单元所依赖的服务单元和数据单元。如图1所示，最终会形成服务单元和数据单元之间的依赖森林图，若森林图中存在重复的服务单元或者数据单元，系统会自动去重并生成任务队列，队列中存储着所有需要迁移的服务单元和数据单元，即完成任务套餐的生成。所述任务套餐是指：一个大的信息系统会由很多的服务和支撑服务的数据组成，将服务和数据抽象为服务单元和数据单元后，应用方可以根据需要自由的组合必要的服务单元和数据单元，组合的结果被称为任务套餐。当应用方确认后，系统启动开设进程，开设进程会读取包含了服务单元和数据单元的任务队列，由于所有需要迁移的服务单元和数据单元之间的依赖关系都是清晰的，所以可以为每个服务单元和数据单元启动一个甚至多个线程完成迁移过程，这样可以使得迁移的过程并行化。对于服务单元，需要根据元数据存储组件中存储的信息将其镜像文件拷贝到开设后平台(目的端)的指定存储位置，并加载到指定的镜像仓库中，或将服务目录拷贝到开设后平台的对应原平台的存储位置(例如服务目录在源端的存储位置是C:\TRUS，则在目的端的存储位置也是C:\TRUS)；对于数据单元，需要根据元数据存储组件中存储的信息将其数据目录拷贝至指定平台(目的端)的指定存储位置。由于是并行迁移，所以在网络带宽允许的条件下，整个开设过程是较快的。但若遇到网络带宽较低甚至网络环境恶化到无法传输数据的情况，本系统仍支持断点续传。断点续传的理解可以分为两部分：一部分是断点，一部分是续传。断点的由来是在传输过程中，将一个文件分成了多个部分，同时进行多个部分一起的传输，当某个时间点，任务被暂停了，此时传输暂停的位置就是断点。续传就是当一个未完成的传输任务再次开始时，会从上次的断点继续传送。本发明断点续传的过程如下：①迁移单元在传输过程中记录每条线程的传输进度，传输记录存储在元数据存储组件中；②每次传输开始之前迁移单元先读取数据存储组件，查询是否有未完成的记录，有就继续传输，没有则创建新记录插入数据库；③在每次传输完预定数量的数据后，迁移单元在数据存储组件中更新传输进度；④传输完成之后迁移单元删除元数据存储组件中的传输记录。最终，在网络稳定的情况下，新的信息系统会被开设到指定的平台上，并能够保证其中的服务和数据会被完整的迁移过去，这种迁移称为开设，即将信息系统从一个平台成功复制到另一个平台上的过程被称为开设。而由于本发明提供了断点续传的功能，因此即使在迁移过程中发生网络中断，亦可在网络恢复后完成全部服务和数据的迁移。When there is a demand for opening, the application side selects the corresponding service unit on the page of the system to quickly open the system according to the demand, and the system will automatically search for the service unit and data unit that these service units depend on according to the information in the metadata storage component. As shown in Figure 1, a dependency forest graph between service units and data units will eventually be formed. If there are duplicate service units or data units in the forest graph, the system will automatically deduplicate and generate a task queue. The queue stores all the required data units. The migrated service units and data units, that is, to complete the generation of task packages. The task package means that a large information system will be composed of many services and data supporting the services. After the services and data are abstracted into service units and data units, the application side can freely combine the necessary service units and data units as needed. Units of data, the result of the combination is called a task package. After the application side confirms, the system starts the setup process, and the setup process will read the task queue containing the service unit and the data unit. Since the dependencies between all the service units and data units that need to be migrated are clear, it can be Each service unit and data unit starts one or more threads to complete the migration process, so that the migration process can be parallelized. For the service unit, according to the information stored in the metadata storage component, its image file needs to be copied to the designated storage location of the platform (destination end) after the opening, and loaded into the designated mirror warehouse, or the service directory needs to be copied to the platform after opening. The storage location of the corresponding original platform (for example, the storage location of the service directory at the source end is C:\TRUS, and the storage location at the destination end is also C:\TRUS); Its data directory is copied to the specified storage location of the specified platform (destination). Due to the parallel migration, the entire opening process is faster when the network bandwidth allows. However, if the network bandwidth is low or even the network environment deteriorates to the point where data cannot be transmitted, the system still supports resumed transmission from a breakpoint. The understanding of breakpoint continuation can be divided into two parts: one is the breakpoint, and the other is the continuation. The origin of the breakpoint is that during the transmission process, a file is divided into multiple parts, and multiple parts are transmitted together at the same time. When the task is suspended at a certain point in time, the position where the transmission is suspended is the breakpoint. Resume transfer is when an unfinished transfer task starts again, it will continue the transfer from the previous breakpoint. The process of resuming transmission from a breakpoint of the present invention is as follows: 1. the migration unit records the transmission progress of each thread during the transmission process, and the transmission record is stored in the metadata storage component; 2. the migration unit first reads the data storage component before each transmission starts, Query whether there are unfinished records, continue to transmit if there is, create new records and insert them into the database if not; ③ After each transmission of a predetermined amount of data, the migration unit updates the transmission progress in the data storage component; ④ After the transmission is completed, the migration unit Delete the transfer record in the metadata store component. Finally, when the network is stable, the new information system will be opened on the designated platform, and the services and data in it will be completely migrated. The process of successfully replicating on another platform is called opening. Since the present invention provides the function of resuming transmission from a breakpoint, even if the network is interrupted during the migration process, the migration of all services and data can be completed after the network is restored.
图2是本发明开设方法的流程图，结合附图3具体如下：Fig. 2 is the flow chart of the opening method of the present invention, and is as follows in conjunction with accompanying drawing 3:
本发明信息系统的快速开设系统包括开设启动单元、服务选择单元、任务生成单元、迁移单元。The rapid opening system of the information system of the present invention includes an opening start unit, a service selection unit, a task generation unit, and a migration unit.
a)应用方通过开设启动单元新建一个开设任务，选定开设的源端及目的端，开设启动单元调用服务选择单元；A) the application side creates a new opening task by opening the startup unit, selects the source end and the destination end to be opened, and sets up the startup unit to call the service selection unit;
b)服务选择单元根据源端的元数据信息提供服务列表，服务列表包括源端所有的服务名称，应用方选取需要开设的服务单元；b) the service selection unit provides a service list according to the metadata information of the source end, the service list includes all the service names of the source end, and the application side selects the service unit that needs to be opened;
c)任务生成单元根据应用方选取的服务单元，搜索该服务单元所依赖的服务单元和数据单元，最终生成一个任务套餐，优选的，所述套餐中包括待迁移的服务单元和数据单元的任务队列；c) The task generation unit searches for the service unit and data unit that the service unit depends on according to the service unit selected by the application side, and finally generates a task package, preferably, the package includes the tasks of the service unit and the data unit to be migrated queue;
d)应用方确认该任务套餐无误；d) The application party confirms that the task package is correct;
e)应用方启动开设任务，任务生成单元调用迁移单元，迁移单元启动并完成服务和数据的并行开设，将服务和数据拷贝到目的端预定存储位置；e) the application side starts the opening task, the task generation unit calls the migration unit, the migration unit starts and completes the parallel opening of the service and the data, and copies the service and the data to the destination predetermined storage location;
f)基于任务套餐的信息系统开设成功。f) The information system based on the task package was successfully established.
上述的开设过程是假设目的端不存在被迁移服务的情况，若目标端已经存在被迁移的服务，则需要在开设任务之初先进行目的端服务单元的版本检查。在快速开设系统的开设任务界面，用户选取好所需的服务单元后，快速开设系统会自动读取目的端元数据存储组件，判断这些从源端所选择的服务单元在目的端是否存在，如果存在，则从目的端元数据存储组件获得该服务的版本信息，并与目的端的服务单元的版本信息比对，在开设界面上提示用户版本信息比对结果，并提示用户是否还要选取该服务单元。此外针对用户所选的服务单元所依赖的服务单元，快速开设系统同样会完成上述比对过程，但不会向用户发出提示，而是将比对结果存入本端元数据存储组件中。当用户确认后，启动开设进程。迁移单元在开设时，会先对目的端已存在的服务单元进行备份，备份的流程为先将服务单元的实例停止服务，将实例对应的镜像或者安装目录另存，备份成功后才会开始源端服务单元迁移的过程。对于数据单元，同样需要先将目的端的数据单元备份，备份的流程为将数据单元对应的数据目录或表另存，备份完成后才会开始源端数据单元迁移的过程。之后具体的迁移流程与上述的迁移流程相同。如果目的端不存在被迁移的服务单元，则直接进行后续的流程，不再进行版本信息的获取比对等步骤。如果目的端存在被迁移的服务单元，则迁移单元向用户发出是否迁移该服务单元的信息，如果用户选择继续迁移，则迁移单元将该服务迁移至目的端，如果用户选择放弃该服务单元的迁移，则迁移单元不再迁移该服务单元，并继续其他服务单元的迁移。The above creation process assumes that there is no migrated service at the destination. If the migrated service already exists at the destination, it is necessary to check the version of the service unit of the destination at the beginning of the task creation. In the setup task interface of the quick setup system, after the user selects the required service units, the quick setup system will automatically read the destination metadata storage component, and determine whether these service units selected from the source end exist at the destination end. Exist, then obtain the version information of the service from the destination metadata storage component, and compare with the version information of the service unit of the destination end, prompt the user version information comparison result on the opening interface, and prompt the user whether to select the service unit. In addition, for the service unit on which the service unit selected by the user depends, the quick opening system will also complete the above comparison process, but will not issue a prompt to the user, but store the comparison result in the local metadata storage component. When the user confirms, the opening process is started. When the migration unit is opened, the existing service unit on the destination end will be backed up first. The backup process is to first stop the service of the instance of the service unit, save the corresponding image or installation directory of the instance, and start the source end after the backup is successful. The process of service unit migration. For data units, it is also necessary to back up the data units at the destination end first. The backup process is to save the data directory or table corresponding to the data units as a separate file. After the backup is completed, the process of migrating the source end data units starts. The specific migration process after that is the same as the above-mentioned migration process. If there is no migrated service unit at the destination end, the subsequent process is directly performed, and the steps of obtaining and comparing version information are no longer performed. If there is a migrated service unit at the destination, the migration unit sends information to the user whether to migrate the service unit. If the user chooses to continue the migration, the migration unit migrates the service to the destination. If the user chooses to abandon the migration of the service unit , the migration unit will no longer migrate the service unit and continue the migration of other service units.
本发明可以使信息系统开设变得十分简便、高效、可靠，应用方只需要在可视化页面上完成点击操作即可完成一个信息系统的迁移，而不需要再费神考虑都需要迁移哪些服务和数据，也无需担心迁移后系统是否可用，更不用担心对迁移的时间预估不准。在特殊的应用场景下(例如军事方面)，提高了效率，提高了可靠性，赢得了时间。The invention can make the establishment of an information system very simple, efficient and reliable, and the application side only needs to complete the click operation on the visualized page to complete the migration of an information system, and does not need to worry about which services and data need to be migrated. There is also no need to worry about whether the system will be available after the migration, let alone the inaccurate estimation of the migration time. In special application scenarios (such as military), efficiency is improved, reliability is improved, and time is gained.
Claims (2)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933672.6ACN109165095B (en) | 2018-08-16 | 2018-08-16 | The method of rapid opening of information system based on task package |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933672.6ACN109165095B (en) | 2018-08-16 | 2018-08-16 | The method of rapid opening of information system based on task package |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109165095A CN109165095A (en) | 2019-01-08 |
| CN109165095Btrue CN109165095B (en) | 2022-04-15 |
Family
ID=64895881
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810933672.6AActiveCN109165095B (en) | 2018-08-16 | 2018-08-16 | The method of rapid opening of information system based on task package |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109165095B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101281461A (en)* | 2007-04-04 | 2008-10-08 | 国际商业机器公司 | Method and device for transfer applying dependent system environment |
| CN102196503A (en)* | 2011-06-28 | 2011-09-21 | 哈尔滨工程大学 | Service quality assurance oriented cognitive network service migration method |
| US20120089711A1 (en)* | 2010-10-12 | 2012-04-12 | Microsoft Corporation | Live migration method for large-scale it management systems |
| US20130339947A1 (en)* | 2012-06-15 | 2013-12-19 | Oracle International Corporation | Method and system for hypervisor-based service management |
| CN106603592A (en)* | 2015-10-15 | 2017-04-26 | 中国电信股份有限公司 | Application cluster migrating method and migrating device based on service model |
- 2018
- 2018-08-16CNCN201810933672.6Apatent/CN109165095B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101281461A (en)* | 2007-04-04 | 2008-10-08 | 国际商业机器公司 | Method and device for transfer applying dependent system environment |
| US20120089711A1 (en)* | 2010-10-12 | 2012-04-12 | Microsoft Corporation | Live migration method for large-scale it management systems |
| CN102196503A (en)* | 2011-06-28 | 2011-09-21 | 哈尔滨工程大学 | Service quality assurance oriented cognitive network service migration method |
| US20130339947A1 (en)* | 2012-06-15 | 2013-12-19 | Oracle International Corporation | Method and system for hypervisor-based service management |
| CN106603592A (en)* | 2015-10-15 | 2017-04-26 | 中国电信股份有限公司 | Application cluster migrating method and migrating device based on service model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109165095A (en) | 2019-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12197758B2 (en) | Distributed object replication architecture | |
| US11327799B2 (en) | Dynamic allocation of worker nodes for distributed replication | |
| US11010240B2 (en) | Tracking status and restarting distributed replication | |
| US11349915B2 (en) | Distributed replication and deduplication of an object from a source site to a destination site | |
| US11741046B2 (en) | Method and apparatus for creating system disk snapshot of virtual machine | |
| US11232074B2 (en) | Systems and methods for searching deduplicated data | |
| US20190265891A1 (en) | Transactional operations in multi-master distributed data management systems | |
| US11461140B2 (en) | Systems and methods for controller-worker architecture for searching a storage system | |
| US8977662B1 (en) | Storing data objects from a flat namespace in a hierarchical directory structured file system | |
| CN105824723B (en) | The method and system that a kind of data to publicly-owned cloud storage account are backed up | |
| CN113296891B (en) | Platform-based multi-scenario knowledge graph processing method and device | |
| CN107181773A (en) | Data storage and data managing method, the equipment of distributed memory system | |
| WO2024078211A1 (en) | Backup method for service cluster instance, recovery method for service cluster instance, and related device | |
| JP2023515354A (en) | System and method for agentless accelerated database backup | |
| US10592530B2 (en) | System and method for managing transactions for multiple data store nodes without a central log | |
| WO2024260453A1 (en) | Data backup method, apparatus and system, and nonvolatile readable storage medium | |
| CN109165095B (en) | The method of rapid opening of information system based on task package | |
| CN115297107A (en) | Data transmission control method and related equipment | |
| US10713121B1 (en) | Dynamic migration of a cloud based distributed file system metadata server | |
| CN113282551B (en) | Data processing method, system and equipment | |
| US11675668B2 (en) | Leveraging a cloud-based object storage to efficiently manage data from a failed backup operation | |
| CN110134662A (en) | SDN distributed storage system, data processing method and storage medium | |
| CN109542588B (en) | Method and device for managing virtual equipment in cloud environment | |
| CN118796773A (en) | A storage management method, device, storage management equipment and readable storage medium | |
| WO2024255085A1 (en) | Business service establishment method and apparatus, computer device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right | Effective date of registration:20241206 Address after:B-1608, 16th Floor, Building 1, No. 18 Zhongguancun East Road, Haidian District, Beijing 100190 Patentee after:Beijing Zhongke Frontier Technology Co.,Ltd. Country or region after:China Address before:215121 Room 601, building nw-01, Northwest District, Suzhou nano City, No. 99, Jinjihu Avenue, Suzhou Industrial Park, Jiangsu Province Patentee before:EDGE INTELLIGENCE OF CAS Co.,Ltd. Country or region before:China |