CN112714163A - Data transmission method, data transmission device, electronic equipment, medium and computer program product - Google Patents

Abstract

Translated fromChinese

本公开公开了一种数据传输方法、装置、电子设备、介质和计算机程序产品，涉及人工智能技术领域，尤其涉及容器技术、云计算及云服务技术领域。具体实现方案为：从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据；将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中；从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。本公开实现了在不影响镜像分发的前提下，减少了镜像仓库的元数据存储压力，提高镜像仓库的镜像分发效率的效果。

The present disclosure discloses a data transmission method, device, electronic device, medium and computer program product, and relates to the technical field of artificial intelligence, in particular to the technical field of container technology, cloud computing and cloud services. The specific implementation scheme is: obtaining a data upload instruction from an image device; wherein, the data upload instruction includes the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded in the image file to be uploaded; the image layer identifier of the image layer to be uploaded is The image layer data to be uploaded is stored in the image storage system in association with the image layer data to be uploaded; the data download instruction is obtained from the image device, and the image layer data to be downloaded is determined in the image storage system according to the data download instruction, and all The mirror layer data to be downloaded is sent to the mirror device. The present disclosure achieves the effect of reducing the metadata storage pressure of the mirror warehouse and improving the image distribution efficiency of the mirror warehouse without affecting the mirror image distribution.

Description

Translated fromChinese

数据传输方法、装置、电子设备、介质和计算机程序产品Data transmission method, apparatus, electronic device, medium and computer program product

技术领域technical field

本公开涉及人工智能技术领域，尤其涉及容器技术、云计算及云服务技术领域，特别涉及一种数据传输方法、装置、电子设备、介质和计算机程序产品。The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of container technology, cloud computing and cloud services, and in particular to a data transmission method, apparatus, electronic device, medium and computer program product.

背景技术Background technique

Docker容器技术与虚拟机相比，大大降低了资源占用率，并方便了虚拟化的运维。但是面成千上万台Docker主机，如何高效的管理和分发Docker镜像是决定是否能快速部署Docker容器的关键因素。Compared with virtual machines, Docker container technology greatly reduces resource occupancy and facilitates the operation and maintenance of virtualization. But with thousands of Docker hosts, how to efficiently manage and distribute Docker images is a key factor in determining whether Docker containers can be quickly deployed.

现有技术是在后端存储系统对Docker镜像进行文件式存储，主机向镜像仓库发送请求指令，镜像仓库在后端存储系统中获取目标Docker镜像并发送给主机。In the prior art, the Docker image is stored in a file format in the back-end storage system, the host sends a request instruction to the mirror warehouse, and the mirror warehouse obtains the target Docker image in the back-end storage system and sends it to the host.

发明内容SUMMARY OF THE INVENTION

本公开提供了一种用于减少镜像仓库存储压力的方法、装置、电子设备、介质和计算机程序产品。The present disclosure provides a method, apparatus, electronic device, medium and computer program product for reducing the storage pressure of an image warehouse.

根据本公开的一方面，提供了一种数据传输方法，包括：According to an aspect of the present disclosure, a data transmission method is provided, comprising:

从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据；Obtain a data upload instruction from the mirror device; wherein, the data upload instruction includes the image layer identifier of the image layer to be uploaded in the image file to be uploaded and the image layer data to be uploaded;

将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中；The image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored in the image storage system in the form of object storage;

从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。Obtain a data download instruction from a mirror device, determine in the mirror storage system the mirror layer data to be downloaded according to the data download command, and send the to-be-downloaded mirror layer data to the mirror device.

根据本公开的另一方面，提供了一种数据传输装置，包括：According to another aspect of the present disclosure, a data transmission device is provided, comprising:

数据上传指令获取模块，用于从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据；a data upload instruction acquisition module, configured to acquire a data upload instruction from an image device; wherein, the data upload instruction includes the image layer identifier of the image layer to be uploaded in the image file to be uploaded and the image layer data to be uploaded;

对象存储模块，用于将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中；The object storage module is used to associate and store the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded into the image storage system in the form of object storage;

数据分发模块，用于从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。A data distribution module, configured to obtain a data download instruction from a mirroring device, determine in the mirroring storage system according to the data downloading instruction to-be-downloaded mirror layer data, and send the to-be-downloaded mirror layer data to the mirroring device .

根据本公开的另一方面，提供了一种电子设备，包括：According to another aspect of the present disclosure, there is provided an electronic device, comprising:

至少一个处理器；以及at least one processor; and

与所述至少一个处理器通信连接的存储器；其中，a memory communicatively coupled to the at least one processor; wherein,

所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本公开中任一项所述的方法。The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any of the present disclosure.

根据本公开的另一方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行本公开中任一项所述的方法。According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of the present disclosures.

根据本公开的另一方面，提供了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据本公开中任一项所述的方法。According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of the present disclosures.

应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

附图说明Description of drawings

附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

图1是根据本公开实施例公开的一种数据传输方法的流程图；1 is a flowchart of a data transmission method disclosed according to an embodiment of the present disclosure;

图2是根据本公开实施例公开的一种数据传输方法的流程图；2 is a flowchart of a data transmission method disclosed according to an embodiment of the present disclosure;

图3是根据本公开实施例公开的一种数据传输装置的结构示意图；3 is a schematic structural diagram of a data transmission apparatus disclosed according to an embodiment of the present disclosure;

图4是用来实现本公开实施例公开的数据传输方法的电子设备的框图。FIG. 4 is a block diagram of an electronic device used to implement the data transmission method disclosed in the embodiment of the present disclosure.

具体实施方式Detailed ways

以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

申请人在研发过程中发现，现有的镜像分发方法通常是将镜像文件中各镜像层的镜像层数据，以文件存储形式存储到镜像存储系统中，并在镜像仓库中维护各镜像文件对应的元数据，基于维护的元数据才能保证镜像分发操作的正常进行。然而随着镜像文件数量的增加，镜像仓库中的元数据的数据量越来越大，这无疑增大了镜像仓库的存储压力，相应的降低了镜像仓库的镜像分发效率。The applicant found in the research and development process that the existing image distribution method usually stores the image layer data of each image layer in the image file in the image storage system in the form of file storage, and maintains the corresponding image files in the image warehouse. Metadata, based on the maintained metadata, can ensure the normal operation of the image distribution operation. However, as the number of mirror files increases, the amount of metadata in the mirror warehouse becomes larger and larger, which undoubtedly increases the storage pressure of the mirror warehouse and correspondingly reduces the image distribution efficiency of the mirror warehouse.

图1是根据本公开实施例公开的一种数据传输方法的流程图，本实施例可以适用于镜像仓库对镜像设备上传的镜像层数据进行存储的情况。本实施例方法可以由数据传输方法装置来执行，所述装置可采用软件和/或硬件实现，并可集成在任意的具有计算能力的电子设备上。FIG. 1 is a flowchart of a data transmission method disclosed according to an embodiment of the present disclosure. This embodiment can be applied to a situation where an image warehouse stores image layer data uploaded by an image device. The method of this embodiment may be executed by a data transmission method apparatus, and the apparatus may be implemented by software and/or hardware, and may be integrated on any electronic device with computing capability.

如图1所示，本实施例公开的数据传输方法可以包括：As shown in FIG. 1 , the data transmission method disclosed in this embodiment may include:

S101、从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据。S101. Acquire a data upload instruction from an image device; wherein, the data upload instruction includes an image layer identifier of an image layer to be uploaded in an image file to be uploaded and image layer data to be uploaded.

其中，镜像设备指的是可运行镜像文件的电子设备，例如Docker镜像设备即是一种镜像设备，其可运行Docker镜像文件以启动Docker容器工作。镜像文件包含了用于容器启动工作的基础信息，例如程序、库、资源、配置文件以及配置参数等，即在镜像设备本地端基于镜像文件可启动对应容器开启工作。镜像文件由至少一层松耦合的镜像层构成，每一层镜像层都包含有各自对应的数据内容，在一个镜像文件建立的初始阶段只包含一层基础镜像层，随着该镜像文件内容的修改或增加，则会在基础镜像层之上，创建新的镜像层。镜像文件始终保持是当前所有镜像层的组合，即所有镜像层数据之和就是镜像文件的镜像数据。The image device refers to an electronic device that can run an image file. For example, a Docker image device is an image device that can run a Docker image file to start a Docker container. The image file contains basic information for container startup, such as programs, libraries, resources, configuration files, and configuration parameters. That is, the corresponding container startup can be started based on the image file on the local side of the mirror device. The image file is composed of at least one layer of loosely coupled image layers, each layer of image layer contains its own corresponding data content, and only one basic image layer is included in the initial stage of the establishment of an image file. Modifications or additions will create a new image layer on top of the base image layer. The image file always remains the combination of all the current image layers, that is, the sum of the data of all the image layers is the image data of the image file.

在一种实施方式中，任一镜像设备根据待上传镜像文件中，待上传镜像层的镜像层标识和待上传镜像层数据，生成数据上传指令，镜像仓库从该镜像设备中获取数据上传指令。其中，镜像仓库是一种用于存储镜像文件的电子设备，镜像设备可向镜像仓库发起下载请求，以从镜像仓库中下载所需的镜像文件；所述数据上传指令包括push指令。In one embodiment, any mirror device generates a data upload instruction according to the mirror layer identifier of the to-be-uploaded mirror layer and the to-be-uploaded mirror layer data in the to-be-uploaded mirror file, and the mirror warehouse obtains the data upload command from the mirror device. The mirror warehouse is an electronic device for storing mirror files, and the mirror device can initiate a download request to the mirror warehouse to download the required mirror files from the mirror warehouse; the data upload instruction includes a push instruction.

可选的，待上传镜像层的镜像层标识是通过对所述待上传镜像层的待上传镜像层数据进行哈希计算得到的。Optionally, the image layer identifier of the image layer to be uploaded is obtained by performing hash calculation on the image layer data to be uploaded of the image layer to be uploaded.

其中，镜像设备对任一待上传镜像层的待上传镜像层数据，进行哈希计算，将得到的哈希值作为该待上传镜像层的镜像层标识。由于哈希值的唯一性，使得不同待上传镜像层对应不同的镜像层标识。Wherein, the mirroring device performs hash calculation on the to-be-uploaded image layer data of any to-be-uploaded image layer, and uses the obtained hash value as the image layer identifier of the to-be-uploaded image layer. Due to the uniqueness of the hash value, different image layers to be uploaded correspond to different image layer identifiers.

通过对待上传镜像层的待上传镜像层数据进行哈希计算，得到待上传镜像层的镜像层标识，实现了准确区分各待上传镜像层的效果，并为后续将待上传镜像层的镜像层标识和待上传镜像层数据进行对象存储，奠定了基础。The image layer identifier of the image layer to be uploaded is obtained by performing hash calculation on the image layer data to be uploaded to be uploaded, which achieves the effect of accurately distinguishing the image layers to be uploaded, and provides the image layer identifier for the image layer to be uploaded subsequently. It lays a foundation for object storage with the image layer data to be uploaded.

S102、将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中。S102: Associate and store the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded into the image storage system in the form of object storage.

其中，镜像存储系统为具有数据存储能力的电子设备，其与镜像仓库通过接口通信连接。对象存储是一种基于对象的存储方式，在对象存储中，数据会被分解为称为“对象”的单元，并保存在单个存储库中，在存储库中，存储有数据，即“对象”、描述数据的元数据、以及允许在镜像存储系统上找到“对象”的唯一标识符，即“对象名称”。对象存储方式允许镜像存储系统同时存储数据、标识符以及元数据，很好的解决的了镜像仓库存储压力过大的问题。Wherein, the mirror storage system is an electronic device with data storage capability, which is connected to the mirror warehouse through interface communication. Object storage is an object-based storage method in which data is decomposed into units called "objects" and stored in a single repository where data, or "objects", are stored , metadata describing the data, and a unique identifier that allows the "object" to be found on the mirrored storage system, the "object name". The object storage method allows the mirror storage system to store data, identifiers and metadata at the same time, which solves the problem of excessive storage pressure in the mirror warehouse.

在一种实施方式中，镜像仓库除了包含有文件存储接口外，例如posix接口，还包含有对象存储接口。镜像仓库将待上传镜像层的镜像层标识作为对象存储中的“对象名称”，将待上传镜像层的待上传镜像层数据作为对象存储中的“对象”，并基于预设的对象存储接口，将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中。由于对象存储方式的特性，元数据可存储于镜像存储系统中，而在镜像仓库本地端，只需要存储可找到“对象”的标识符，即镜像层的镜像层标识，便可根据镜像层标识，在镜像存储系统中快速定位对应的镜像层数据，从而实现镜像分发。In one embodiment, the image repository includes an object storage interface in addition to a file storage interface, such as a posix interface. The image warehouse uses the image layer identifier of the image layer to be uploaded as the "object name" in the object storage, and the image layer data to be uploaded of the image layer to be uploaded as the "object" in the object storage, and based on the preset object storage interface, The image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored in the image storage system in the form of object storage. Due to the characteristics of the object storage method, the metadata can be stored in the mirror storage system. On the local end of the mirror warehouse, only the identifier of the "object" that can be found, that is, the mirror layer ID of the mirror layer, can be stored according to the mirror layer ID. , to quickly locate the corresponding image layer data in the image storage system, so as to realize image distribution.

通过将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中，减少了镜像仓库的元数据存储压力。The image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored in the image storage system in the form of object storage, thereby reducing the metadata storage pressure of the image warehouse.

S103、从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。S103. Acquire a data download instruction from a mirror device, determine in the mirror storage system the mirror layer data to be downloaded according to the data download command, and send the to-be-downloaded mirror layer data to the mirror device.

在一种实施方式中，镜像仓库接收到任一镜像设备发送的数据下载指令，并根据该数据下载指令以及镜像仓库存储的镜像层标识，确定待下载镜像层的镜像层标识，进而根据在镜像存储系统中建立的镜像层标识和镜像层数据的关联关系，从镜像存储系统中获取待下载镜像层的待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。In one embodiment, the mirror repository receives a data download instruction sent by any mirror device, and determines the image layer identifier of the image layer to be downloaded according to the data download instruction and the image layer identifier stored in the mirror repository, and then determines the image layer identifier of the image layer to be downloaded according to the data download instruction and the image layer identifier stored in the image repository. The association relationship between the image layer identifier and the image layer data established in the storage system, obtains the image layer data to be downloaded of the image layer to be downloaded from the image storage system, and sends the image layer data to be downloaded to the image device.

根据本实施例的技术方案，通过从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据，并将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中，由于对象存储的特性，无需在镜像仓库本地端维护大量的元数据，实现了在不影响镜像分发的前提下，减少了镜像仓库的元数据存储压力，提高镜像仓库的镜像分发效率的效果。According to the technical solution of this embodiment, the data upload instruction is obtained by obtaining the data upload instruction from the mirror device; wherein, the data upload instruction includes the image layer identifier of the image layer to be uploaded in the image file to be uploaded and the image layer data to be uploaded, and the image to be uploaded is uploaded. The image layer identifier of the layer and the image layer data to be uploaded are stored in the image storage system in the form of object storage. Due to the characteristics of object storage, there is no need to maintain a large amount of metadata at the local end of the image warehouse, which realizes the premise of not affecting the image distribution. It reduces the metadata storage pressure of the mirror warehouse and improves the image distribution efficiency of the mirror warehouse.

在上述实施例的基础上，S101之后，包括：On the basis of the above embodiment, after S101, including:

在镜像仓库中已存储有所述待上传镜像层的镜像层标识的情况下，拒绝所述数据上传指令。In the case that the image layer identifier of the image layer to be uploaded is already stored in the image warehouse, the data upload instruction is rejected.

在一种实施方式中，镜像仓库获取数据上传指令后，解析得到待上传镜像层的镜像层标识，并将待上传镜像层的镜像层标识，与镜像仓库本地端已存储的镜像层标识进行比对。若待上传镜像层的镜像层标识，与任一镜像仓库已存储有的镜像层标识相同，表示待上传镜像层的镜像层数据已上传到镜像仓库，则拒绝待上传镜像层关联的数据上传指令。In one embodiment, after the mirror repository obtains the data upload instruction, it parses to obtain the image layer identifier of the image layer to be uploaded, and compares the image layer identifier of the image layer to be uploaded with the image layer identifier stored at the local end of the image repository right. If the image layer identifier of the image layer to be uploaded is the same as the image layer identifier stored in any image repository, it means that the image layer data of the image layer to be uploaded has been uploaded to the image repository, and the data upload instruction associated with the image layer to be uploaded is rejected. .

通过在镜像仓库中已存储有待上传镜像层的镜像层标识的情况下，拒绝数据上传指令，实现了限制同名对象重复上传的效果，且实现了镜像层数据复用，减少了镜像仓库的存储压力的效果。By rejecting the data upload instruction when the image layer identifier of the image layer to be uploaded is stored in the image warehouse, the effect of restricting the repeated upload of objects with the same name is realized, and the data reuse of the image layer is realized, which reduces the storage pressure of the image warehouse. Effect.

图2是根据本公开实施例公开的一种数据传输方法的流程图，基于上述技术方案进一步优化与扩展，并可以与上述各个可选实施方式进行结合。FIG. 2 is a flowchart of a data transmission method disclosed according to an embodiment of the present disclosure, which is further optimized and expanded based on the above-mentioned technical solution, and can be combined with each of the above-mentioned optional embodiments.

如图2所示，该方法可以包括：As shown in Figure 2, the method may include:

S201、从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件的文件标识、待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据。S201. Acquire a data upload instruction from an image device; wherein, the data upload instruction includes the file identifier of the image file to be uploaded, the image layer identifier of the image layer to be uploaded in the image file to be uploaded, and the image layer data to be uploaded.

可选的，所述待上传镜像文件的文件标识是通过对所述待上传镜像文件中待上传镜像层总数据进行哈希计算得到的。Optionally, the file identifier of the to-be-uploaded image file is obtained by hashing the total data of the to-be-uploaded image layer in the to-be-uploaded image file.

其中，镜像设备对待上传镜像文件中所有镜像层数据之和，进行哈希计算，将得到的哈希值作为该待上传镜像文件的文件标识。由于哈希值的唯一性，使得不同待上传镜像文件对应不同的文件标识。The image device performs hash calculation on the sum of all image layer data in the image file to be uploaded, and uses the obtained hash value as the file identifier of the image file to be uploaded. Due to the uniqueness of the hash value, different image files to be uploaded correspond to different file identifiers.

通过对待上传镜像文件中待上传镜像层总数据进行哈希计算，得到待上传镜像文件的文件标识，为后续区分镜像仓库中记录的不同镜像文件的镜像层数据，奠定了基础。By hashing the total data of the image layer to be uploaded in the image file to be uploaded, the file identifier of the image file to be uploaded is obtained, which lays a foundation for the subsequent distinction of image layer data of different image files recorded in the image warehouse.

S202、在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识。S202: Record the image layer identifier of the image layer to be uploaded in the image warehouse in the first manner.

在一种实施方式中，第一方式记录为永久记录，即镜像仓库将待上传镜像层的镜像层标识永久记录在镜像仓库本地端。记录的手段可选的包括将待上传镜像层的镜像层标识存储于镜像仓库开发文件中，例如镜像仓库的manifest文件等。In one embodiment, the first method is recorded as a permanent record, that is, the mirror repository permanently records the image layer identifier of the image layer to be uploaded on the local end of the mirror repository. Optionally, the recording method includes storing the image layer identifier of the image layer to be uploaded in the image repository development file, such as the manifest file of the image repository.

可选的，S202包括：Optionally, S202 includes:

在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识，并将所述待上传镜像文件的文件标识作为记录标签。The image layer identifier of the image layer to be uploaded is recorded in the image warehouse in the first manner, and the file identifier of the image file to be uploaded is used as a record label.

在一种实施方式中，镜像仓库将待上传镜像层的镜像层标识永久记录在镜像仓库本地端，并将待上传镜像文件的文件标识作为记录标签，从而区分不同镜像文件的镜像层标识。可选的，镜像仓库新建一个开发文件，将待上传镜像层的镜像层标识存储于该开发文件中，并将待上传镜像文件的文件标识作为该开发文件的标签。即一个待上传镜像文件对应一个开发文件，且该开发文件中仅存储有该待上传镜像文件中待上传镜像层的镜像层标识。In one embodiment, the mirror repository permanently records the image layer identifier of the image layer to be uploaded on the local end of the image repository, and uses the file identifier of the image file to be uploaded as a record label, thereby distinguishing image layer identifiers of different image files. Optionally, create a new development file in the mirror repository, store the image layer identifier of the image layer to be uploaded in the development file, and use the file identifier of the image file to be uploaded as the label of the development file. That is, one image file to be uploaded corresponds to one development file, and only the image layer identifier of the image layer to be uploaded in the image file to be uploaded is stored in the development file.

通过在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识，并将待上传镜像文件的文件标识作为记录标签，实现了区分不同待上传镜像文件的镜像层标识的效果，方便镜像仓库管理，且为后续镜像仓库根据获取的文件标识，确定对应的镜像层标识，奠定了基础。By recording the image layer identifier of the image layer to be uploaded in the image repository in the first way, and using the file identifier of the image file to be uploaded as the record label, the effect of distinguishing image layer identifiers of different image files to be uploaded is realized, which is convenient for the image repository It also lays a foundation for the subsequent image warehouse to determine the corresponding image layer identifier according to the obtained file identifier.

S203、将所述待上传镜像层的镜像层标识和所述待上传镜像层数据，通过键值对形式关联存储到镜像存储系统中；其中，在所述键值对中，所述待上传镜像层的镜像层标识为关键字，所述待上传镜像层数据为数据值。S203. Associate and store the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded in the image storage system in the form of a key-value pair; wherein, in the key-value pair, the image to be uploaded The image layer identifier of the layer is a keyword, and the image layer data to be uploaded is a data value.

在一种实施方式中，镜像仓库根据待上传镜像层的镜像层标识，和待上传镜像层数据，建立键值对，在键值对中待上传镜像层的镜像层标识为关键字，即对应对象存储中的“对象名称”，待上传镜像层数据为数据值，即对应对象存储中的“对象”。进而将待上传镜像层的镜像层标识，和待上传镜像层数据，通过键值对形式关联存储到镜像存储系统中。In one embodiment, the image repository establishes a key-value pair according to the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded. In the key-value pair, the image layer identifier of the image layer to be uploaded is a keyword, which corresponds to The "object name" in the object storage, the image layer data to be uploaded is the data value, that is, the "object" in the corresponding object storage. Further, the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored in the image storage system in the form of a key-value pair.

示例性的，假设待上传镜像层的镜像层标识为“00001”，待上传镜像层数据为“QWERSDFZXCV”，则将“00001”作为关键字Key，“QWERSDFZXCV”作为数据值Value，建立Key-Value键值对。Exemplarily, assuming that the image layer identifier of the image layer to be uploaded is "00001", and the image layer data to be uploaded is "QWERSDFZXCV", then "00001" is used as the keyword Key, "QWERSDFZXCV" is used as the data value Value, and a Key-Value is established. key-value pair.

S204、在任一待上传镜像层的待上传镜像层数据存储成功的情况下，在镜像仓库中通过第二方式记录该待上传镜像层的镜像层标识。S204: In the case that the data of the to-be-uploaded image layer of any of the to-be-uploaded image layers is successfully stored, record the image layer identifier of the to-be-uploaded image layer in the image repository in a second manner.

在一种实施方式中，第二方式记录为临时记录，当任一待上传镜像层的待上传镜像层数据，在镜像存储系统中存储成功后，镜像存储系统会向镜像仓库反馈存储成功信息，镜像仓库响应该存储成功信息，将存储成功的待上传镜像层的镜像层标识临时记录在镜像仓库本地端。记录的手段可选的包括将存储成功的待上传镜像层的镜像层标识，存储于本地临时文件中，例如manifest.tmp文件。In one embodiment, the second method is recorded as a temporary record. When any image layer data to be uploaded of the image layer to be uploaded is successfully stored in the image storage system, the image storage system will feedback the storage success information to the image warehouse, The mirror warehouse responds to the storage success information, and temporarily records the mirror layer identifier of the successfully stored mirror layer to be uploaded on the local end of the mirror warehouse. The recording method can optionally include storing the image layer identifier of the successfully uploaded image layer in a local temporary file, such as the manifest.tmp file.

S205、比较所述第一方式记录的镜像层标识和所述第二方式记录的镜像层标识，并根据比较结果确定所述待上传镜像层的待上传镜像层数据是否上传成功。S205. Compare the image layer identifier recorded in the first method with the image layer identifier recorded in the second method, and determine whether the image layer data to be uploaded of the image layer to be uploaded is successfully uploaded according to the comparison result.

在一种实施方式中，镜像仓库将第一方式记录的镜像层标识，和第二方式记录的镜像层标识进行比较，若第一方式记录的镜像层标识和第二方式记录的镜像层标识相同，则确定待上传镜像层的待上传镜像层数据已上传成功，镜像仓库则与镜像存储系统断开通信连接，以降低系统资源损耗；若第一方式记录的镜像层标识中，还包括除第二方式记录的镜像层标识以外的镜像层标识，则确定待上传镜像层的待上传镜像层数据未上传成功，则继续保持与镜像存储系统的通信连接，以继续上传镜像层数据。In one embodiment, the mirror repository compares the mirror layer ID recorded in the first mode with the mirror layer ID recorded in the second mode, if the mirror layer ID recorded in the first mode is the same as the mirror layer ID recorded in the second mode , then it is determined that the image layer data to be uploaded of the image layer to be uploaded has been uploaded successfully, and the image warehouse disconnects the communication connection with the image storage system to reduce the loss of system resources; If the image layer identifier other than the image layer identifier recorded in the second method is determined, it is determined that the image layer data to be uploaded of the image layer to be uploaded has not been uploaded successfully, and the communication connection with the image storage system is continued to continue uploading the image layer data.

在一种可选实施方式中，镜像仓库将开发文件中永久记录的镜像层标识，和本地临时文件中临时记录的镜像层标识进行比较，若相同，则确定待上传镜像层的待上传镜像层数据已上传成功，镜像仓库则与镜像存储系统断开通信连接，并删除该本地临时文件，以避免占用镜像仓库本地资源。In an optional embodiment, the mirror warehouse compares the mirror layer identifier permanently recorded in the development file with the mirror layer identifier temporarily recorded in the local temporary file, and if they are the same, determines the mirror layer to be uploaded of the mirror layer to be uploaded After the data has been uploaded successfully, the mirror warehouse disconnects the communication connection with the mirror storage system and deletes the local temporary file to avoid occupying the local resources of the mirror warehouse.

S206、从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。S206: Acquire a data download instruction from the mirror device, determine in the mirror storage system the mirror layer data to be downloaded according to the data download command, and send the to-be-downloaded mirror layer data to the mirror device.

根据本实施例的技术方案，通过在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识，实现了记录待上传镜像文件中所有待上传镜像层的镜像层标识的效果，保证后续镜像仓库镜像分发的顺利进行，并且为后续确定所有待上传镜像层数据是否上传成功奠定了基础；通过将待上传镜像层的镜像层标识和待上传镜像层数据，通过键值对形式关联存储到镜像存储系统中，实现了将待上传镜像层的镜像层标识和待上传镜像层数据，进行对象存储的效果，实现了在不影响镜像分发的前提下，减少了镜像仓库的元数据存储压力，提高镜像仓库的镜像分发效率的效果；通过在任一待上传镜像层的待上传镜像层数据存储成功的情况下，在镜像仓库中通过第二方式记录该待上传镜像层的镜像层标识，实现了记录存储成功的待上传镜像层的镜像层标识的效果，为后续确定所有待上传镜像层数据是否上传成功奠定了基础；比较第一方式记录的镜像层标识和第二方式记录的镜像层标识，并根据比较结果确定待上传镜像层的待上传镜像层数据是否上传成功，实现了对待上传镜像层数据是否全部存储成功进行了校验，保证了待上传镜像文件中所有待上传镜像层的待上传镜像层数据，都成功存储在镜像存储系统中。According to the technical solution of the present embodiment, by recording the image layer identifiers of the image layers to be uploaded in the image warehouse in the first manner, the effect of recording the image layer identifiers of all image layers to be uploaded in the image files to be uploaded is realized, and subsequent images are guaranteed. Warehouse image distribution goes smoothly, and lays the foundation for subsequent determination of whether all the image layer data to be uploaded is successfully uploaded; the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored to the image in the form of key-value pairs In the storage system, the effect of storing the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded is realized, which reduces the metadata storage pressure of the image warehouse without affecting the image distribution, and improves the The effect of the image distribution efficiency of the mirror warehouse; when the data of the to-be-uploaded mirror layer of any to-be-uploaded mirror layer is successfully stored, the mirror layer identifier of the to-be-uploaded mirror layer is recorded in the mirror warehouse by the second method, and the record is realized. The effect of storing the image layer identifiers of the image layers to be uploaded successfully lays the foundation for subsequent determination of whether all the image layer data to be uploaded are uploaded successfully; compare the image layer identifiers recorded in the first method and the image layer identifiers recorded in the second method, and According to the comparison result, it is determined whether the data of the image layer to be uploaded of the image layer to be uploaded is uploaded successfully, which realizes the verification of whether all the data of the image layer to be uploaded is successfully stored, and ensures that the images to be uploaded of all the image layers to be uploaded in the image file to be uploaded are successfully uploaded. Layer data is successfully stored in the mirror storage system.

在上述实施例的基础上，S206可选的包括以下A、B、C和D四个步骤：On the basis of the above embodiment, S206 optionally includes the following four steps A, B, C and D:

A、从镜像设备获取数据下载指令；其中，所述数据下载指令包括待下载镜像文件的文件标识。A. Obtain a data download instruction from a mirror device; wherein, the data download instruction includes the file identifier of the image file to be downloaded.

在一种实施方式中，任一镜像设备根据待下载镜像文件的文件标识，生成数据下载指令，镜像仓库从该镜像设备中获取数据下载指令。其中，数据下载指令包括pull指令。In one embodiment, any mirror device generates a data download instruction according to the file identifier of the mirror file to be downloaded, and the mirror repository obtains the data download instruction from the mirror device. The data download instruction includes a pull instruction.

B、根据所述待下载镜像文件的文件标识，确定所述待下载镜像文件中待下载镜像层的镜像层标识。B. Determine the image layer identifier of the image layer to be downloaded in the image file to be downloaded according to the file identifier of the image file to be downloaded.

在一种实施方式中，镜像仓库根据待下载镜像文件的文件标识进行条件搜索，在镜像仓库本地存储中，确定待下载镜像文件中待下载镜像层的镜像层标识。In one embodiment, the mirror repository performs a conditional search according to the file identifier of the image file to be downloaded, and determines the image layer identifier of the image layer to be downloaded in the image file to be downloaded in the local storage of the image repository.

可选的，步骤B包括：Optionally, step B includes:

在所述镜像仓库中，确定与所述待下载镜像文件的文件标识相匹配的记录标签；根据所述记录标签，确定所述待下载镜像文件中待下载镜像层的镜像层标识。In the mirror repository, a record label matching the file identifier of the to-be-downloaded mirror file is determined; and according to the record label, the mirror layer identifier of the to-be-downloaded mirror layer in the to-be-downloaded mirror file is determined.

在一种实施方式中，将待下载镜像文件的文件标识作为搜索条件，在各记录标签中进行遍历搜索，确定与待下载镜像文件的文件标识相同的记录标签，并将该记录标签关联记录内容中的镜像层标识，作为待下载镜像文件中待下载镜像层的镜像层标识。In one embodiment, the file identifier of the image file to be downloaded is used as a search condition, a traversal search is performed in each record label, the record label that is the same as the file identifier of the image file to be downloaded is determined, and the record label is associated with the record content The image layer identifier in the image file to be downloaded is used as the image layer identifier of the image layer to be downloaded in the image file to be downloaded.

在一种可选的实施方式中，将待下载镜像文件的文件标识作为搜索条件，在开发文件标签中进行遍历搜索，确定与待下载镜像文件的文件标识相同的开发文件的标签，并对该开发文件进行解析，将解析得到的镜像层标识，作为待下载镜像文件中待下载镜像层的镜像层标识。In an optional implementation manner, the file identifier of the image file to be downloaded is used as a search condition, and a traversal search is performed in the development file tag to determine the tag of the development file that is the same as the file identifier of the image file to be downloaded, and the tag of the development file is determined. The development file is parsed, and the parsed image layer identifier is used as the image layer identifier of the to-be-downloaded image layer in the to-be-downloaded image file.

通过在镜像仓库中，确定与待下载镜像文件的文件标识相匹配的记录标签；根据记录标签，确定待下载镜像文件中待下载镜像层的镜像层标识，实现了确定待下载镜像文件中待下载镜像层的镜像层标识的效果，为后续确定待下载镜像层数据，奠定了基础。By determining the record label matching the file identifier of the image file to be downloaded in the mirror repository; according to the record label, determining the image layer identifier of the image layer to be downloaded in the image file to be downloaded, so as to realize the determination of the image layer to be downloaded in the image file to be downloaded The effect of the image layer identification of the image layer lays the foundation for the subsequent determination of the image layer data to be downloaded.

C、将所述待下载镜像层的镜像层标识作为搜索关键字，在所述镜像存储系统中进行搜索，确定与所述搜索关键字匹配的目标关键字。C. Use the image layer identifier of the image layer to be downloaded as a search key, perform a search in the image storage system, and determine a target key matching the search key.

在一种实施方式中，镜像仓库将待下载镜像层的镜像层标识作为搜索关键字，在镜像存储系统中存储的各键值对中进行搜索，确定与搜索关键字匹配的目标关键字。In one embodiment, the image repository uses the image layer identifier of the image layer to be downloaded as a search key, searches each key-value pair stored in the image storage system, and determines a target key matching the search key.

D、将所述目标关键字对应的数据值，作为所述待下载镜像层的待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。D. Use the data value corresponding to the target keyword as the mirror layer data to be downloaded of the mirror layer to be downloaded, and send the mirror layer data to be downloaded to the mirror device.

在一种实施方式中，镜像仓库根据目标关键字与其对应数据值的关联关系，将目标关键字对应的数据值，作为待下载镜像层的待下载镜像层数据，并从镜像存储系统中获取该待下载镜像层数据，进而将获取的待下载镜像层数据，发送给该镜像设备，完成镜像层数据的下载。In one embodiment, the mirror warehouse takes the data value corresponding to the target keyword as the data value of the mirror layer to be downloaded of the mirror layer to be downloaded according to the relationship between the target keyword and its corresponding data value, and obtains the data value from the mirror storage system. The mirror layer data to be downloaded, and then the obtained mirror layer data to be downloaded is sent to the mirror device to complete the download of the mirror layer data.

通过从镜像设备获取数据下载指令，并根据待下载镜像文件的文件标识，确定待下载镜像文件中待下载镜像层的镜像层标识，进而将待下载镜像层的镜像层标识作为搜索关键字，在镜像存储系统中进行搜索，确定与搜索关键字匹配的目标关键字，最终将目标关键字对应的数据值，作为待下载镜像层的待下载镜像层数据，并将待下载镜像层数据发送给镜像设备，实现了基于镜像存储系统的对象存储方式，仅根据待下载镜像文件的文件标识，即可获取待下载镜像文件中，待下载镜像层的待下载镜像层数据的效果，提高了镜像分发效率。By obtaining the data download instruction from the mirror device, and according to the file identifier of the image file to be downloaded, the image layer identifier of the image layer to be downloaded in the image file to be downloaded is determined, and then the image layer identifier of the image layer to be downloaded is used as the search key, and then the image layer identifier of the image layer to be downloaded is determined. Search in the image storage system to determine the target keyword matching the search keyword, and finally use the data value corresponding to the target keyword as the image layer data to be downloaded of the image layer to be downloaded, and send the image layer data to be downloaded to the image. The device implements the object storage method based on the image storage system. Only according to the file identifier of the image file to be downloaded, the effect of the image layer data to be downloaded of the image layer to be downloaded in the image file to be downloaded can be obtained, which improves the image distribution efficiency. .

图3是根据本公开实施例公开的一种数据传输装置的结构示意图，可以适用于镜像仓库对镜像设备上传的镜像层数据进行存储的情况。本实施例装置可采用软件和/或硬件实现，并可集成在任意的具有计算能力的电子设备上。FIG. 3 is a schematic structural diagram of a data transmission apparatus disclosed according to an embodiment of the present disclosure, which can be applied to a situation where a mirror warehouse stores mirror layer data uploaded by a mirror device. The apparatus of this embodiment may be implemented by software and/or hardware, and may be integrated on any electronic device with computing capability.

如图3所示，本实施例公开的数据传输装置30可以包括数据上传指令获取模块31、对象存储模块32和数据分发模块33，其中：As shown in FIG. 3 , thedata transmission apparatus 30 disclosed in this embodiment may include a data uploadinstruction acquisition module 31, anobject storage module 32, and adata distribution module 33, wherein:

数据上传指令获取模块31，用于从镜像设备获取数据上传指令；其中，所述数据上传指令包括待上传镜像文件中待上传镜像层的镜像层标识和待上传镜像层数据；The data uploadinstruction acquisition module 31 is used for acquiring a data upload instruction from the mirror device; wherein, the data upload instruction includes the image layer identifier of the image layer to be uploaded in the image file to be uploaded and the image layer data to be uploaded;

对象存储模块32，用于将待上传镜像层的镜像层标识和待上传镜像层数据通过对象存储形式关联存储到镜像存储系统中；Theobject storage module 32 is configured to associate and store the image layer identifier of the image layer to be uploaded and the image layer data to be uploaded into the image storage system in the form of object storage;

数据分发模块33，用于从镜像设备获取数据下载指令，并根据所述数据下载指令在所述镜像存储系统中确定待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。Thedata distribution module 33 is configured to obtain a data download instruction from the mirror device, and determine the mirror layer data to be downloaded in the mirror storage system according to the data download command, and send the to-be-downloaded mirror layer data to the mirror equipment.

可选的，所述对象存储模块32，具体用于：Optionally, theobject storage module 32 is specifically used for:

将所述待上传镜像层的镜像层标识和所述待上传镜像层数据，通过键值对形式关联存储到镜像存储系统中；其中，在所述键值对中，所述待上传镜像层的镜像层标识为关键字，所述待上传镜像层数据为数据值。The image layer identifier of the image layer to be uploaded and the image layer data to be uploaded are associated and stored in the image storage system in the form of a key-value pair; wherein, in the key-value pair, the image layer of the to-be-uploaded image layer is stored. The image layer identifier is a keyword, and the image layer data to be uploaded is a data value.

可选的，所述装置还包括第一方式记录模块，具体用于：Optionally, the device further includes a first mode recording module, which is specifically used for:

在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识；Record the image layer identifier of the image layer to be uploaded in the image warehouse by the first method;

所述装置还包括数据上传校验模块，具体用于：The device also includes a data upload verification module, which is specifically used for:

在任一待上传镜像层的待上传镜像层数据存储成功的情况下，在镜像仓库中通过第二方式记录该待上传镜像层的镜像层标识；In the case that the data storage of the to-be-uploaded image layer of any of the to-be-uploaded image layers is successful, the image layer identifier of the to-be-uploaded image layer is recorded in the image warehouse by a second method;

比较所述第一方式记录的镜像层标识和所述第二方式记录的镜像层标识，并根据比较结果确定所述待上传镜像层的待上传镜像层数据是否上传成功。Compare the image layer identifier recorded in the first manner with the image layer identifier recorded in the second manner, and determine whether the image layer data to be uploaded of the image layer to be uploaded is successfully uploaded according to the comparison result.

可选的，所述数据上传指令还包括待上传镜像文件的文件标识；Optionally, the data upload instruction further includes the file identifier of the image file to be uploaded;

相应的，所述第一方式记录模块，具体还用于：Correspondingly, the recording module in the first manner is further used for:

在镜像仓库中通过第一方式记录待上传镜像层的镜像层标识，并将所述待上传镜像文件的文件标识作为记录标签。The image layer identifier of the image layer to be uploaded is recorded in the image warehouse in the first manner, and the file identifier of the image file to be uploaded is used as a record label.

可选的，所述装置还包括镜像层标识校验模块，具体用于：Optionally, the device further includes an image layer identification verification module, which is specifically used for:

在镜像仓库中已存储有所述待上传镜像层的镜像层标识的情况下，拒绝所述数据上传指令。In the case that the image layer identifier of the image layer to be uploaded is already stored in the image warehouse, the data upload instruction is rejected.

可选的，所述待上传镜像文件的文件标识是通过对所述待上传镜像文件中待上传镜像层总数据进行哈希计算得到的；所述待上传镜像层的镜像层标识是通过对所述待上传镜像层的待上传镜像层数据进行哈希计算得到的。Optionally, the file identifier of the image file to be uploaded is obtained by hashing the total data of the image layer to be uploaded in the image file to be uploaded; the image layer identifier of the image layer to be uploaded is obtained by hashing all the image layers to be uploaded. The to-be-uploaded image layer data of the to-be-uploaded image layer is obtained by hash calculation.

可选的，所述数据分发模块33，具体用于：Optionally, thedata distribution module 33 is specifically used for:

从镜像设备获取数据下载指令；其中，所述数据下载指令包括待下载镜像文件的文件标识；Obtain a data download instruction from the mirror device; wherein, the data download instruction includes the file identifier of the image file to be downloaded;

根据所述待下载镜像文件的文件标识，确定所述待下载镜像文件中待下载镜像层的镜像层标识；According to the file identifier of the image file to be downloaded, determine the image layer identifier of the image layer to be downloaded in the image file to be downloaded;

将所述待下载镜像层的镜像层标识作为搜索关键字，在所述镜像存储系统中进行搜索，确定与所述搜索关键字匹配的目标关键字；Using the image layer identifier of the image layer to be downloaded as a search keyword, perform a search in the image storage system, and determine a target keyword matching the search keyword;

将所述目标关键字对应的数据值，作为所述待下载镜像层的待下载镜像层数据，并将所述待下载镜像层数据发送给所述镜像设备。The data value corresponding to the target keyword is used as the to-be-downloaded image layer data of the to-be-downloaded image layer, and the to-be-downloaded image layer data is sent to the mirror device.

可选的，所述数据分发模块33，具体还用于：Optionally, thedata distribution module 33 is also specifically used for:

在所述镜像仓库中，确定与所述待下载镜像文件的文件标识相匹配的记录标签；In the mirror warehouse, determine a record label that matches the file identifier of the mirror file to be downloaded;

根据所述记录标签，确定所述待下载镜像文件中待下载镜像层的镜像层标识。According to the record label, the image layer identifier of the image layer to be downloaded in the image file to be downloaded is determined.

本公开实施例所公开的数据传输装置30可执行本公开实施例所公开的数据传输方法，具备执行方法相应的功能模块和有益效果。本实施例中未详尽描述的内容可以参考本公开任意方法实施例中的描述。Thedata transmission apparatus 30 disclosed in the embodiments of the present disclosure can execute the data transmission methods disclosed in the embodiments of the present disclosure, and has functional modules and beneficial effects corresponding to the execution methods. For the content not described in detail in this embodiment, reference may be made to the description in any method embodiment of the present disclosure.

根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

图4示出了可以用来实施本公开的实施例的示例电子设备400的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。FIG. 4 shows a schematic block diagram of an exampleelectronic device 400 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

如图4所示，设备400包括计算单元401，其可以根据存储在只读存储器(ROM)402中的计算机程序或者从存储单元408加载到随机访问存储器(RAM)403中的计算机程序，来执行各种适当的动作和处理。在RAM 403中，还可存储设备400操作所需的各种程序和数据。计算单元401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(I/O)接口405也连接至总线404。As shown in FIG. 4 , thedevice 400 includes acomputing unit 401 that can be executed according to a computer program stored in a read only memory (ROM) 402 or loaded from astorage unit 408 into a random access memory (RAM) 403 Various appropriate actions and handling. In theRAM 403, various programs and data necessary for the operation of thedevice 400 can also be stored. Thecomputing unit 401 , theROM 402 , and theRAM 403 are connected to each other through abus 404 . An input/output (I/O)interface 405 is also connected tobus 404 .

设备400中的多个部件连接至I/O接口405，包括：输入单元406，例如键盘、鼠标等；输出单元407，例如各种类型的显示器、扬声器等；存储单元408，例如磁盘、光盘等；以及通信单元409，例如网卡、调制解调器、无线通信收发机等。通信单元409允许设备400通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Various components in thedevice 400 are connected to the I/O interface 405, including: aninput unit 406, such as a keyboard, mouse, etc.; anoutput unit 407, such as various types of displays, speakers, etc.; astorage unit 408, such as a magnetic disk, an optical disk, etc. ; and acommunication unit 409, such as a network card, a modem, a wireless communication transceiver, and the like. Thecommunication unit 409 allows thedevice 400 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

计算单元401可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元401的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元401执行上文所描述的各个方法和处理，例如数据传输方法。例如，在一些实施例中，数据传输方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元408。在一些实施例中，计算机程序的部分或者全部可以经由ROM 402和/或通信单元409而被载入和/或安装到设备400上。当计算机程序加载到RAM 403并由计算单元401执行时，可以执行上文描述的数据传输方法的一个或多个步骤。备选地，在其他实施例中，计算单元401可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行数据传输方法。Computing unit 401 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computingunits 401 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. Thecomputing unit 401 performs the various methods and processes described above, such as the data transmission method. For example, in some embodiments, a data transfer method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such asstorage unit 408 . In some embodiments, part or all of the computer program may be loaded and/or installed ondevice 400 viaROM 402 and/orcommunication unit 409 . When a computer program is loaded intoRAM 403 and executed by computingunit 401, one or more steps of the data transfer method described above may be performed. Alternatively, in other embodiments, thecomputing unit 401 may be configured to perform the data transfer method by any other suitable means (eg, by means of firmware).

本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)、区块链网络和互联网。The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the Internet.

计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务中，存在的管理难度大，业务扩展性弱的缺陷。A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS services, which are difficult to manage and weak in business scalability. defect.

应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.

上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (15)

1. A method of data transmission, comprising:

acquiring a data uploading instruction from mirror image equipment; the data uploading instruction comprises a mirror image layer identifier of a mirror image layer to be uploaded in a mirror image file to be uploaded and mirror image layer data to be uploaded;

storing the mirror layer identification of the mirror layer to be uploaded and the mirror layer data to be uploaded into a mirror storage system in an associated manner through an object storage form;

acquiring a data downloading instruction from mirror equipment, determining mirror layer data to be downloaded in the mirror storage system according to the data downloading instruction, and sending the mirror layer data to be downloaded to the mirror equipment.

2. The method of claim 1, wherein the step of storing the mirror layer identifier of the mirror layer to be uploaded and the mirror layer data to be uploaded in a mirror storage system in an associated manner through an object storage form comprises the following steps:

storing the mirror layer identification of the mirror layer to be uploaded and the mirror layer data to be uploaded into a mirror storage system in a key value pair form in an associated manner; in the key value pair, the mirror layer identifier of the mirror layer to be uploaded is a keyword, and the data of the mirror layer to be uploaded is a data value.

3. The method according to claim 2, wherein before storing the mirror layer identifier of the mirror layer to be uploaded and the mirror layer data to be uploaded in the mirror storage system in an associated manner through an object storage form, the method further comprises:

recording a mirror image layer identifier of a mirror image layer to be uploaded in a mirror image warehouse through a first mode;

after the mirror image layer identifier of the mirror image layer to be uploaded and the mirror image layer data to be uploaded are stored in the mirror image storage system in an associated manner in an object storage form, the method further comprises the following steps:

under the condition that the data of the mirror layer to be uploaded of any mirror layer to be uploaded is successfully stored, recording the mirror layer identification of the mirror layer to be uploaded in a mirror warehouse through a second mode;

and comparing the mirror layer identifier recorded in the first mode with the mirror layer identifier recorded in the second mode, and determining whether the mirror layer data to be uploaded of the mirror layer to be uploaded is uploaded successfully according to the comparison result.

4. The method of claim 3, wherein the data uploading instruction further comprises a file identifier of an image file to be uploaded;

correspondingly, the mirror image layer identifier of the mirror image layer to be uploaded is recorded in the mirror image warehouse in a first mode, and the method comprises the following steps:

and recording the mirror image layer identification of the mirror image layer to be uploaded in a mirror image warehouse in a first mode, and taking the file identification of the mirror image file to be uploaded as a recording label.

5. The method of claim 1, wherein after the obtaining the data upload instruction from the mirroring device, further comprising:

and rejecting the data uploading instruction under the condition that the mirror image layer identification of the mirror image layer to be uploaded is stored in the mirror image warehouse.

6. The method according to claim 4, wherein the file identifier of the image file to be uploaded is obtained by performing hash calculation on total data of an image layer to be uploaded in the image file to be uploaded; and the mirror layer identification of the mirror layer to be uploaded is obtained by performing hash calculation on the mirror layer data of the mirror layer to be uploaded.

7. The method of claim 4, wherein obtaining a data download instruction from a mirroring device, determining mirroring layer data to be downloaded in the mirroring storage system according to the data download instruction, and sending the mirroring layer data to be downloaded to the mirroring device comprises:

acquiring a data downloading instruction from mirror equipment; the data downloading instruction comprises a file identifier of a mirror image file to be downloaded;

determining the mirror image layer identification of the mirror image layer to be downloaded in the mirror image file to be downloaded according to the file identification of the mirror image file to be downloaded;

taking the image layer identification of the image layer to be downloaded as a search keyword, searching in the image storage system, and determining a target keyword matched with the search keyword;

and taking the data value corresponding to the target keyword as the image layer data to be downloaded of the image layer to be downloaded, and sending the image layer data to be downloaded to the image device.

8. The method of claim 7, wherein determining the image layer identifier of the image layer to be downloaded in the image file to be downloaded according to the file identifier of the image file to be downloaded comprises:

determining a record label matched with the file identification of the mirror image file to be downloaded in the mirror image warehouse;

and determining the mirror image layer identification of the mirror image layer to be downloaded in the mirror image file to be downloaded according to the record label.

9. A data transmission apparatus comprising:

the data uploading instruction acquisition module is used for acquiring a data uploading instruction from the mirror image equipment; the data uploading instruction comprises a mirror image layer identifier of a mirror image layer to be uploaded in a mirror image file to be uploaded and mirror image layer data to be uploaded;

the object storage module is used for storing the mirror image layer identification of the mirror image layer to be uploaded and the mirror image layer data to be uploaded into the mirror image storage system in an associated manner through an object storage form;

and the data distribution module is used for acquiring a data downloading instruction from the mirror image equipment, determining mirror image layer data to be downloaded in the mirror image storage system according to the data downloading instruction, and sending the mirror image layer data to be downloaded to the mirror image equipment.

10. The apparatus of claim 9, wherein the object storage module is specifically configured to:

storing the mirror layer identification of the mirror layer to be uploaded and the mirror layer data to be uploaded into a mirror storage system in a key value pair form in an associated manner; in the key value pair, the mirror layer identifier of the mirror layer to be uploaded is a keyword, and the data of the mirror layer to be uploaded is a data value.

11. The apparatus according to claim 10, wherein the apparatus further comprises a first mode recording module, specifically configured to:

recording a mirror image layer identifier of a mirror image layer to be uploaded in a mirror image warehouse through a first mode;

the device also comprises a data uploading verification module, which is specifically used for:

under the condition that the data of the mirror layer to be uploaded of any mirror layer to be uploaded is successfully stored, recording the mirror layer identification of the mirror layer to be uploaded in a mirror warehouse through a second mode;

and comparing the mirror layer identifier recorded in the first mode with the mirror layer identifier recorded in the second mode, and determining whether the mirror layer data to be uploaded of the mirror layer to be uploaded is uploaded successfully according to the comparison result.

12. The apparatus of claim 11, wherein the data upload instruction further comprises a file identifier of an image file to be uploaded;

correspondingly, the first mode recording module is further specifically configured to:

and recording the mirror image layer identification of the mirror image layer to be uploaded in a mirror image warehouse in a first mode, and taking the file identification of the mirror image file to be uploaded as a recording label.

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8.

Images