A kind of online moving method of cloud container pre-copy based on P.haul frameTechnical field
It is the present invention relates to the online migrating technology field of cloud computing container, in particular to a kind of based on P.haul frameThe online moving method of Docker container pre-copy.
Background technique
Container technique provides increasingly wide application environment for cloud computing, in order to provide more for container life cycleAdd flexible application, adapt to richer usage scenario, it is necessary to use the online migrating technology of container.Container moves onlineThermophoresis is also named in shifting, and referring to is ensuring in the case where accessing unbroken situation, to the appearance between different physical machines or cloud platformDevice carries out mobile process.In transition process, need to connect on source host upper container run-time memory, file system and networkIt the status informations such as connects to migrate to destination host, while ensuring container after migration to continue the working condition of source host.
The ongoing work of source host in transition process can be transferred to purpose master by this online migrating technology in real timeOn machine, migrated so that user is even imperceptible.Nowadays the scheme much migrated online all has on a virtual machineVery mature application, and the online migration field of container are based especially on the online migration of Docker, and there is presently no a set of non-Often mature solution.
In recent years, academia copies the thinking of virtual machine (vm) migration, proposes " stopping-copy " and " pre-copy " of containerMigration strategy is realized and is migrated online to the container of Docker.Also research attempts to have used logging-and-replay'sMigration strategy, but dependent on the record to container event to be migrated, and have many restrictions, so logging-and-replayStrategy is also immature in container migration.
Summary of the invention
The purpose of the present invention is to overcome the shortcomings of the existing technology and deficiency, provides a kind of cloud appearance based on P.haul frameThe online moving method of device pre-copy may be implemented online migration of the Docker container between source host and destination host, and lead toThe volume of transmitted data in transition process can be effectively reduced and migrate total by crossing pre-copy and the transmission delay mechanism based on historyTime.
The purpose of the present invention is realized by the following technical solution: a kind of cloud container pre-copy based on P.haul frame existsLine moving method, comprising:
T1, container configuration are obtained and migration network establishment step: obtaining the configuration file of container to be migrated, it is established that source masterGenerator terminal to destination host end network connection, in order in container migration step carry out associated migratory work；
T2, container migration step, comprising:
Container assemblies migrate sub-step: for sending container assemblies associated documents and job status document to destination host；
Internal memory migration sub-step: for sending container current working status file and memory to be migrated to destination host iterationInitial data；
T3, container is rebuild and cleanup step: status file for being transmitted in destination host according to step T2 andInternal storage data rebuilds new container, and clears up the container in source host and configuration file.
Preferably, the step T1 obtains the configuration information of container to be migrated, and generates checkpoint catalogue, toolBody includes:
Step T1.1, according to the id of container to be migrated, current container Tty feelings are read using docker inspect orderCondition, if it find that the container opens Tty, then the container can not migrate, and return to error message；
Step T1.2, the network linking between source host and destination host is established；Network linking needs to set up two netsNetwork link: order transmissions links and data transmission link；Order transmissions links use RPC far call mode, i.e., in source hostMiddle transmission order is run in destination host；Data link uses Rsync data synch command, and related data is passed from source hostTransport to destination host；
Step T1.3, using docker checkpoint create--checkpoint-dir={ checkpoint mesh{ container id } { checkpoint serial number } order, the status file of container to be migrated is obtained, and be stored in checkpointIn catalogue, for being migrated to destination host.
Further, the status file for the container to be migrated obtained being needed described in step T1.3 includes: that container is retouchedMessage file, the filec descriptor message file of opening, CPU information file and memory mapping table are stated, these information are with JSONData format is stored in respective file；Page initial data when container operation should be also obtained, simultaneously with 4KByte sizeIt is stored in binary file for unit.
Further, two network linkings described in step T1.2 are constructed in P.haul frame based on C/S structure,It is logical by establishing two between the two to guarantee the transmission that can carry out data between source host and destination host and control operationLetter socket is transmitted respectively for control command and data；Control command socket uses RPC remote process control mode, by sourceHost sends relevant control command to destination host；Data socket then uses Transmission Control Protocol, for that will obtain from source hostTo operating status file be sent in destination host.
Preferably, container assemblies described in T2 migrate sub-step, for transmitting the associated component of container in source hostTo destination host, need to migrate 3 catalogues: root file system, container config directory and container in the transition process of DockerMetadata catalog when operation, specifically includes:
Step 1, the image file using in Rsync order respectively general/var/lib/docker/image catalogue ,/var/Container configuration file and/var/run/docke-r/execdriver/native in lib/docker/containers catalogueContainer operation when metadata be transferred in destination host；
Step 2 sends RPC order from source host to destination host, carries out three kinds of corresponding mesh in step 1 at destination host endThe reconstruction of record.
Preferably, internal memory migration sub-step described in T2, for by the memory initial data of container to be migrated with iterationMode is transmitted in destination host, to reduce downtime when container migration, is specifically included:
Step 1, setting iteration pre-copy stop condition determine to need according to internal storage data size derived from container to be migratedTo use different one of stop conditions；
-- fdrpc<fd>-- fdmem<fd>-- pre-dump is used for step 2, input order p.haul docker<id>P.haul frame, the pre-copy for opening Docker container migrate online；
Newly generated memory containing dirty pages can be transmitted to destination host in each iteration with incremental form by step 3, p.haul.
Specifically, setting iteration pre-copy stop condition determines to need according to internal storage data size derived from container to be migratedIt will be using one of three kinds of stop conditions: when migrating internal storage data size less than 128M, using condition (1) in the pre- of last timeIn copy, generates and be no more than 64 memory containing dirty pages；When migration internal storage data size is greater than 128M and is less than 512M, using condition(2) the page copy quantity of last time preexamination shellfish increases less than 10%；The 512M when migrating internal storage data size and being greater than, using itemPart (3) iteration number is no more than 8 times.
Further, the memory containing dirty pages incremental transmission uses the transmission delay mechanism based on history, specifically includes:
Memory containing dirty pages bitmap values dirtymap when (n-1)th step 1, preservation iterationn-1；
Step 2, the bitmap values proceeded in nth iteration be dirtymapn；
Step 3, the occurrence that position is corresponded to by comparing the two bitmaps, to determine whether this is exported.
Specifically, the memory containing dirty pages incremental transmission uses the transmission delay mechanism based on history:
(1) work as dirtymapnAnd dirtymapn-1It when being all 0, indicates that the page is never modified, does not have to copy；
(2) work as dirtymapnIt is 1, dirtymapn-1Indicated when being 0 the page be it is newly generated, need to be copied；
(3) work as dirtymapnIt is 0, dirtymapn-1Indicate that the page was once modified when being 1, it may be possible to continuousThe page modified and do not copied, and will not may still modify next time, so needing exist for being copied；
(4) work as dirtymapnAnd dirtymapn-1When being all 1, indicate that the page was modified continuously, this indicates that the page has very muchIt may can also be modified next time, therefore this does not copy the page.
Preferably, container reconstruction and cleanup step T3, for container-like according to getting on destination hostState file and memory initial data rebuild new container, and delete the container of having moved in source host, while deleting correspondingCheckpoint catalogue, specifically includes:
Step T3.1, the container of having moved deleted in source host is ordered using docker rm<id>；
Step T3.2, the docker system of destination host is restarted, so that newly migrating past container configuration information and root textPart system comes into force；
Step T3.3, using docker start--checkpoint-dir=<checkpoint catalogue>--Checkpoint=<checkpoint serial number><container id>rebuilds new container in destination host.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) online migration of the Docker container between source host and destination host may be implemented in the present invention.
(2) the present invention is based on more mature P.haul frame, there are good stability and scalability.
(3) present invention can be effectively reduced in transition process by pre-copy and the transmission delay mechanism based on historyVolume of transmitted data and migration total time.
Detailed description of the invention
Fig. 1 is the online migration system architecture figure of embodiment container.
Fig. 2 is embodiment migratory system network linking topological diagram.
Fig. 3 is embodiment container migration flow chart.
Fig. 4 is embodiment iteration pre-copy schematic diagram.
Specific embodiment
Present invention will now be described in further detail with reference to the embodiments and the accompanying drawings, but embodiments of the present invention are unlimitedIn this.
Embodiment 1
The present invention is a kind of online moving method of cloud container pre-copy based on P.haul frame, is mainly used in DockerOnline migration of the container between different hosts is the basic work that container payload equilibrium and container scheduling of resource are realized in cloud environmentMake.
Below in conjunction with the online migration system architecture figure (Fig. 1) of container, migratory system network linking topological diagram (Fig. 2), containerMigration flow chart (Fig. 3) and the transmission delay mechanism description table (table 1) based on history illustrate.
A kind of online moving method of cloud container pre-copy based on P.haul frame, using system structure as shown in Figure 1Figure, its implementation specifically include following step, and flow chart is as shown in Figure 3:
T1, container configuration are obtained and migration network establishment step:
Obtain the configuration file of container to be migrated, it is established that the network connection at source host end to destination host end, in order toAssociated migratory work is carried out in container migration step, specific steps include:
Step T1.1, according to the id of container to be migrated, current container Tty feelings are read using docker inspect orderCondition, if it find that the container opens Tty, then the container can not migrate, and return to error message.
Step T1.2, the network linking between source host and destination host is established.In the present system, it needs to set up twoNetwork linking: order transmissions links and data transmission link.Order transmissions links use RPC far call mode, i.e., in source masterOrder is sent in machine to run in destination host；Data link uses Rsync data synch command, by related data from source hostIt is transmitted to destination host.
Step T1.3, using docker checkpoint create--checkpoint-dir=< checkp-oint mesh><container id><checkpoint serial number>order, the status file of container to be migrated is obtained, and be stored in checkpointIn catalogue, for being migrated to destination host.
T2, container migration step, for moving associated component, status file and memory initial data needed for migration containerShifting is migrated by source host to destination host, and specific steps include:
Step T2.1, using in Rsync order respectively general/var/lib/docker/image catalogue image file ,/Container configuration file and/var/run/doc-ker/execdriver/ in var/lib/docker/containers catalogueMetadata is transferred in destination host when the container operation of native.
Step T2.2, using docker checkpoint create--checkpoint-dir={ check-point mesh{ container id } { checkpoint serial number } order, the status file of container to be migrated is obtained, and be stored in checkpointIn catalogue, for being migrated to destination host.And RPC order is sent from source host to destination host, three are carried out at destination host endThe reconstruction of the corresponding catalogue of kind.
Step T2.3, -- pre-dump makes fdrpc<fd>-- fdmem<fd>-- for input order p.haul docker<id>With p.haul frame, the pre-copy for opening Docker container migrates online, according to pre-copy stop condition and based on the biography of historyDefeated delay mechanism is migrated.
T3, container is rebuild and cleanup step, for the container state file and memory that basis is got on destination hostInitial data rebuilds new container, and deletes the container of having moved in source host, while deleting corresponding checkpoint catalogue, hasBody includes:
Step T3.1, the container of having moved deleted in source host is ordered using docker rm<id>.
Step T3.2, the docker system of destination host is restarted, so that newly migrating past container configuration information and root textPart system comes into force.
Step T3.3, using docker start--checkpoint-dir=<checkpoint catalogue>--Checkpoint=<checkpoint serial number><container id>rebuilds new container in destination host.
Tty terminal refers to the terminal interacted with container, and P.haul can not be migrated with interactive terminal at presentContainer then returns to error message, while exiting entire transition process once detecting that container opens Tty in step T1.
Checkpoint status file catalogue, the filec descriptor of these status file container descriptor files, openingMessage file, CPU information file and memory mapping table, these information are stored in respective file with JSON data format.TogetherWhen should also obtain container operation when page initial data, be stored in binary file as unit of 4KByte size.
Source host and destination host network linking, figure it is seen that P.haul frame is based on C/S structure, in order to guaranteeThe transmission and control operation that data can be carried out between source host and destination host, between the two by establishing two communicationsSocket is transmitted respectively for control command and data.Control command socket mainly uses RPC remote process control mode, bySource host sends relevant control command to destination host.Data socket then uses Transmission Control Protocol, and being mainly used for will be from source masterThe operating status file got on machine is sent in destination host.
Pre-copy iteration stopping condition, because the more more then last time transmission downtimes of the number of iterations can be smaller, but mistakeIt is excessively high that more the number of iterations will cause migration total time, it is therefore desirable to be set according to the memory size occupied to container different pre-Copy the iteration stopping time: migrate internal storage data size be less than 128M when, using condition (1) last time pre-copyIn, it generates and is no more than 64 memory containing dirty pages (256K)；When migration internal storage data size is greater than 128M and is less than 512M, using itemThe page copy quantity of part (2) last time preexamination shellfish increases less than 10%；The 512M when migrating internal storage data size and being greater than, usesCondition (3) iteration number is no more than 8 times.
Transmission delay mechanism based on history is by comparing memory containing dirty pages bitmap twice in succession, to determine that this changesWhether generation transmits corresponding page.As can be seen from Table 1:
(1) work as dirtymapnAnd dirtymapn-1It when being all 0, indicates that the page is never modified, does not have to copy；
(2) work as dirtymapnIt is 1, dirtymapn-1Indicated when being 0 the page be it is newly generated, need to be copied；
(3) work as dirtymapnIt is 0, dirtymapn-1Indicate that the page was once modified when being 1, it may be possible to continuousThe page modified and do not copied, and will not may still modify next time, so needing exist for being copied；
(4) work as dirtymapnAnd dirtymapn-1When being all 1, indicate that the page was modified continuously, this indicates that the page has very muchIt may can also be modified next time, therefore this does not copy the page.
Transmission delay mechanism description table of the table 1 based on history
| dirtymapn-1 | 0 | 0 | 1 | 1 |
| dirtymapn | 0 | 1 | 0 | 1 |
| Whether copy | It is no | It is | It is | It is no |
The above embodiment is a preferred embodiment of the present invention, but embodiments of the present invention are not by above-described embodimentLimitation, other any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present invention,It should be equivalent substitute mode, be included within the scope of the present invention.