Movatterモバイル変換


[0]ホーム

URL:


CN119814770A - A data transmission method, cloud platform, computer equipment and storage medium - Google Patents

A data transmission method, cloud platform, computer equipment and storage medium
Download PDF

Info

Publication number
CN119814770A
CN119814770ACN202412000297.8ACN202412000297ACN119814770ACN 119814770 ACN119814770 ACN 119814770ACN 202412000297 ACN202412000297 ACN 202412000297ACN 119814770 ACN119814770 ACN 119814770A
Authority
CN
China
Prior art keywords
transmission
file
storage device
data block
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202412000297.8A
Other languages
Chinese (zh)
Inventor
冯瑞鹏
赵鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co LtdfiledCriticalSuzhou Metabrain Intelligent Technology Co Ltd
Priority to CN202412000297.8ApriorityCriticalpatent/CN119814770A/en
Publication of CN119814770ApublicationCriticalpatent/CN119814770A/en
Pendinglegal-statusCriticalCurrent

Links

Landscapes

Abstract

Translated fromChinese

本发明涉及存储技术领域,公开了一种数据传输方法、云平台、计算机设备及存储介质,该方法包括:接收数据传输请求,数据传输请求用于请求将存储设备中第一逻辑卷中目标文件传输到第二逻辑卷;获取云平台与存储设备之间的传输通道,并获取以目标数据块大小为单位的组成目标文件的至少一个传输文件;当第一逻辑卷和第二逻辑卷均支持逻辑卷之间进行数据传输的数据传输特性时,基于每个传输文件生成至少一个传输请求,并将每个传输请求通过传输通道依次发送给存储设备,以使存储设备传输每个传输文件到第二逻辑卷,直到至少一个传输文件传输完成。本发明可以解决数据传输效率低,且需要大量占用虚拟机的计算资源、内存资源和设备之间网络资源的问题。

The present invention relates to the field of storage technology, and discloses a data transmission method, a cloud platform, a computer device and a storage medium, the method comprising: receiving a data transmission request, the data transmission request is used to request to transfer a target file in a first logical volume in a storage device to a second logical volume; obtaining a transmission channel between the cloud platform and the storage device, and obtaining at least one transmission file constituting the target file in units of a target data block size; when both the first logical volume and the second logical volume support data transmission characteristics for data transmission between logical volumes, generating at least one transmission request based on each transmission file, and sending each transmission request to the storage device in sequence through the transmission channel, so that the storage device transmits each transmission file to the second logical volume until at least one transmission file is transmitted. The present invention can solve the problem of low data transmission efficiency and the need to occupy a large amount of computing resources, memory resources and network resources between devices of a virtual machine.

Description

Data transmission method, cloud platform, computer equipment and storage medium
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data transmission method, a cloud platform, a computer device, and a storage medium.
Background
The OpenStack cloud platform is an open-source cloud computing management platform project and is formed by combining a series of software open-source projects. The goal of the OpenStack cloud platform is to provide extensible and resilient cloud computing services for private and public clouds. It supports almost all types of cloud environments and aims to provide a cloud computing management platform that is simple to implement, scalable on a large scale, rich, and standard-uniform. The Cinder service is a component in the OpenStack cloud platform for providing a block storage service. The user may use the Cinder service to provide virtual disk management services for the virtual machine instance. Data transmission among a plurality of logical volumes in the same storage device can be performed for the same storage device between the same or different virtual machines on the OpenStack cloud platform.
In the related art, taking a logical volume a of a storage device used by a virtual machine a and a logical volume B of the storage device used by a virtual machine B as examples, in a scenario of copying files from the logical volume a of the virtual machine a to the logical volume B of the virtual machine B, the virtual machine a needs to obtain file data from the logical volume a of the storage device through a switch, the virtual machine a copies the file data to the virtual machine B through a network between the virtual machine a and the virtual machine B, and the virtual machine B writes the file data into the logical volume B through the switch to complete copying the files. Such a data transmission manner has low data transmission efficiency, and requires a large amount of occupied computing resources and memory resources of the virtual machines, and network resources between the virtual machines and between the virtual machines-switch-storage devices.
Disclosure of Invention
In view of the above, the present invention provides a data transmission method, a cloud platform, a computer device and a storage medium, so as to solve the problems of low data transmission efficiency and need to occupy a large amount of computing resources and memory resources of virtual machines, and network resources between virtual machines and between virtual machines-switch-storage devices.
The data transmission method comprises the steps of receiving a data transmission request sent by a virtual machine, wherein the data transmission request is used for requesting to transmit a target file in a first logic volume in storage equipment to a second logic volume, acquiring a transmission channel between the cloud platform and the storage equipment, acquiring at least one transmission file with the size of a target data block as a unit, forming the target file by the at least one transmission file, generating at least one transmission request based on each transmission file when the first logic volume and the second logic volume support data transmission characteristics of data transmission between the logic volumes, and sequentially sending each transmission request to the storage equipment through the transmission channel, so that the storage equipment can transmit each transmission file to the second logic volume until at least one transmission file is transmitted.
According to the method provided by the aspect, the cloud platform can receive the data transmission requests sent by the virtual machine, acquire the transmission channel between the cloud platform and the storage device and at least one transmission file taking the size of the target data block as a unit, generate at least one transmission request based on each transmission file when the first logical volume and the second logical volume support the data transmission characteristics of data transmission between the logical volumes, and sequentially send each transmission request to the storage device through the transmission channel so as to transmit the target file.
In addition, the first logical volume and the second logical volume support the data transmission characteristics of data transmission between the logical volumes, so that the storage device can directly transmit the target file in the first logical volume to the second logical volume in the storage device. When the data transmission is carried out between the cloud platform and the storage device, the switch is not needed, and the file transmission can be directly carried out in the storage device, so that the data transmission efficiency can be improved, the computing resources and the memory resources of the virtual machine are saved, the network resources among the virtual machines and among the virtual machine, the switch and the storage device are saved, the performances of the storage device and the cloud platform are improved, and the user experience is improved.
With reference to the first aspect, in an optional implementation manner, acquiring the transmission channel between the cloud platform and the storage device includes acquiring the transmission channel between the cloud platform and the storage device in response to a selection operation of a user on a display interface of the cloud platform.
In the embodiment of the invention, a user can select whether a transmission channel is established between the cloud platform and the storage device through the graphical interface, namely, the transmission channel between the cloud platform and the storage device is acquired, so that at least one transmission request generated by the target file can be conveniently sent based on the transmission channel, and the storage device can transmit the target file based on the at least one transmission request.
With reference to the first aspect, in an optional implementation manner, before receiving the data transmission request sent by the virtual machine, the method further includes sending a query instruction of a preset protocol to each logical volume in the storage device according to the granularity of the logical volume when the cloud platform is started, and receiving response information of each logical volume returned by the storage device, where the response information is used to indicate whether each logical volume supports the data transmission characteristic and the transmission data block size information set by each logical volume.
In the embodiment of the invention, when the cloud platform is started, whether each logic volume in the storage device supports the data transmission characteristic and the set transmission data block size information can be inquired, so that the storage device can conveniently judge whether data transmission can be carried out between the logic volumes when a data transmission request is received later, and the transmission unit for dividing the target file is conveniently determined based on the transmission data block size information later.
With reference to the first aspect, in an optional implementation manner, the obtaining at least one transmission file with the target data block size as a unit includes obtaining transmission data block size information set by the second logical volume, determining that the unit of the storage device for transmitting the target file is the target data block size according to the transmission data block size information and a preset policy, obtaining the target file, and dividing the target file into at least one transmission file with the target data block size.
In the embodiment of the invention, the unit of the transmission target file can be dynamically adjusted and determined according to the size of the transmission data block set by the second logical volume, and then the target file is divided into at least one transmission file according to the transmission unit, so that the cloud platform can send a plurality of transmission requests in parallel, and further the plurality of transmission files are transmitted in parallel, and the transmission speed is obviously improved.
With reference to the first aspect, in an optional implementation manner, determining, according to the transmission data block size information and the preset policy, that the unit of the storage device for transmitting the target file is the target data block size includes determining that the target data block size is the first value when the transmission data block size information indicates that the transmission data block size set by the second logical volume is greater than or equal to the first value, and determining that the target data block size is the set transmission data block size when the transmission data block size information indicates that the transmission data block size set by the second logical volume is greater than or equal to the second value and less than the first value.
In the embodiment of the invention, the size information of the transmission data block indicates that the size of the transmission data block set by the second logic volume is larger than or equal to the first value, namely the size of the transmission data block set by the second logic volume is larger than or equal to the second value and smaller than the first value, and the size information of the transmission data block indicates that the size of the transmission data block set by the second logic volume is larger than or equal to the second value and smaller than the first value, so that the size of the transmission data block set by the second logic volume is moderate, and therefore, the size of the target data block is determined to be the size of the transmission data block set, thereby avoiding overlarge transmission delay caused by overlarge transmission files or avoiding frequent read-write operation of storage equipment caused by overlarge transmission files.
The method comprises the steps of sequentially receiving at least one transmission request sent by a cloud platform through a transmission channel under the condition that the transmission channel between the cloud platform and a storage device is acquired, wherein the at least one transmission request is generated by the cloud platform based on the received data transmission request sent by a virtual machine and at least one transmission file with a target data block size as a unit, the data transmission request is used for requesting to transmit a target file in a first logic volume in the storage device to a second logic volume, the at least one transmission file forms the target file, and when the first logic volume and the second logic volume both support data transmission characteristics of data transmission between the logic volumes, each transmission file is transmitted to the second logic volume according to each transmission request.
According to the method provided by the aspect, the storage device can sequentially receive at least one transmission request sent by the cloud platform through the transmission channel under the condition that the transmission channel between the cloud platform and the storage device is acquired, and when the first logical volume and the second logical volume both support the data transmission characteristics of data transmission between the logical volumes, each transmission file is transmitted to the second logical volume according to each transmission request so as to complete the transmission of the target file.
In addition, the first logical volume and the second logical volume support the data transmission characteristics of data transmission between the logical volumes, so that the storage device can directly transmit the target file in the first logical volume to the second logical volume in the storage device. When the data transmission is carried out between the cloud platform and the storage device, the switch is not needed, and the file transmission can be directly carried out in the storage device, so that the data transmission efficiency can be improved, the computing resources and the memory resources of the virtual machine are saved, the network resources among the virtual machines and among the virtual machine, the switch and the storage device are saved, the performances of the storage device and the cloud platform are improved, and the user experience is improved.
In combination with the second aspect, in an alternative implementation manner, the at least one transmission request includes a first transmission request, the first transmission request is used for transmitting the first transmission file, transmitting each transmission file to the second logical volume according to each transmission request, and the method includes the steps of obtaining a target address, carried in the first transmission request, of the first transmission file to be stored in the second logical volume, determining a target data block, corresponding to the target address, of the first transmission file to be stored according to the target address, detecting whether the target data block is in a locking state, locking the target data block if not, and writing the first transmission file into the target data block.
In the embodiment of the invention, when the first transmission file is written into the target data block, the target data block is locked when the target data block is detected to be in an unlocked state, so that the operation of other read-write requests on the target data block can be avoided, and the condition of data confusion caused by concurrent writing of the target data block can be avoided.
In a third aspect, the invention provides a cloud platform, which comprises a transceiver module, a first storage device and a second storage device, wherein the transceiver module is used for receiving a data transmission request sent by a virtual machine, and the data transmission request is used for requesting to transmit a target file in a first logic volume in the storage device to a second logic volume. The receiving and transmitting module is also used for acquiring a transmission channel between the cloud platform and the storage device, and acquiring at least one transmission file taking the size of the target data block as a unit, wherein the at least one transmission file forms the target file. And the processing module is used for generating at least one transmission request based on each transmission file when the first logic volume and the second logic volume support the data transmission characteristics of data transmission between the logic volumes, and sequentially sending each transmission request to the storage device through the transmission channel so that the storage device transmits each transmission file to the second logic volume until the transmission of at least one transmission file is completed.
In a third aspect, the invention provides a storage device, which comprises a receiving module, a transmission module and a transmission module, wherein the receiving module is used for sequentially receiving at least one transmission request sent by a cloud platform through a transmission channel under the condition that the transmission channel between the cloud platform and the storage device is acquired, the at least one transmission request is generated by the cloud platform based on the received data transmission request sent by a virtual machine and at least one transmission file with a target data block size as a unit, the data transmission request is used for requesting to transmit a target file in a first logic volume in the storage device to a second logic volume, the at least one transmission file forms the target file, and the transmission module is used for transmitting each transmission file into the second logic volume according to each transmission request when the first logic volume and the second logic volume support the data transmission characteristics of data transmission between the logic volumes.
In a third aspect, the present invention provides a computer device comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to thereby perform the data transmission method of the first aspect or any of its corresponding embodiments.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the data transmission method of the first aspect or any of its corresponding embodiments.
In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to perform the data transmission method of the first aspect or any of its corresponding embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a scenario in which data transfer between multiple logical volumes on the same storage device occurs between different virtual machines according to an embodiment of the present invention;
fig. 2 is a topology of a data transmission system according to an embodiment of the present invention;
Fig. 3 is a flowchart of a data transmission method according to an embodiment of the present invention;
fig. 4 is a flowchart of yet another data transmission method according to an embodiment of the present invention;
FIG. 5 is a block diagram of a cloud platform according to an embodiment of the invention;
FIG. 6 is a block diagram of a memory device according to an embodiment of the invention;
Fig. 7 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention is applied to the scenes of data transmission among a plurality of logical volumes in the same storage device aiming at the same storage device between the same or different virtual machines on the OpenStack cloud platform.
In the related art, as shown in fig. 1, fig. 1 is a schematic view of a scenario in which data transmission between multiple logical volumes on the same storage device is performed between different virtual machines according to an embodiment of the present invention. In fig. 1, in the scenario of copying files from a logical volume a of a virtual machine a to a logical volume B of a virtual machine B, the virtual machine a needs to obtain file data from the logical volume a of a storage device through a switch, the virtual machine a copies the file data to the virtual machine B through a network between the virtual machine a and the virtual machine B, and the virtual machine B writes the file data into the logical volume B through the switch to complete copying the files. The data transmission mode has low data transmission efficiency and occupies a large amount of computing resources, memory resources and network resources.
In order to solve the technical problems described above, an embodiment of the present invention provides a data transmission method, which requests a storage device to transmit a target file in a first logical volume to a second logical volume when the first logical volume and the second logical volume both support data transmission characteristics of data transmission between the logical volumes, so as to achieve the effects of improving data transmission efficiency and saving computing resources, memory resources and network resources.
The method provided by the embodiment of the present application will be described below by taking the data transmission system 200 shown in fig. 2 as an example. Fig. 2 is only a schematic diagram, and does not limit the applicable scenario of the technical solution provided by the present application.
As shown in fig. 2, fig. 2 is a topology diagram of a data transmission system according to an embodiment of the present invention. In fig. 2, a data transmission system 200 may include a cloud platform 201, a storage device 202, and a virtual machine 203.
The cloud platform 201 in the embodiment of the present invention may be any device having a computing function and a communication function. The device has a display interface. OpenStack cloud platform. The cloud platform is an open-source cloud computing management platform project. The cloud platform is used for providing extensible and elastic cloud computing services for private clouds and public clouds. The cloud platform supports various types of cloud environments. In architecture, the cloud platform is composed of computing (Nova, zun), storage (Cinder, manila, swift), network (Neutron, octavia, designate), hardware (Ironic, cyborg), workflow (Magnum, trove, sahara), application (Murano, freezer, solum, masakari), orchestration (Heat, mistral, aodh, senlin, zaqar, blazar), public services (Keystone, placement, glance, barbican) and other types of components and corresponding services. These components work cooperatively to manage resources (e.g., storage resources, virtual machine resources, network resources) in the cloud platform through a unified management interface.
The OpenStack cloud platform comprises Nova computing service and Cinder, manila, swift storage services, wherein the Cinder service is a block storage service, the Manila service is a file storage service, and the Swift is an object storage service. The Cinder service can be driven by various manufacturers, and can use basic functions such as volume creation/deletion/capacity expansion/mounting/unloading and the like provided by the back-end storage array, and can also use advanced characteristics such as cloning, copying, snapshot and the like of the back-end storage array to perform management type operation.
The memory device 202 in the embodiment of the present invention may be a memory array. The storage device 202 may include a controller and a storage medium. The storage device 202 includes a plurality of logical volumes. The plurality of logical volumes includes a first logical volume and a second logical volume.
The virtual machine 203 in the embodiment of the present invention may be any virtual machine created by the cloud platform 201.
The data transmission system 200 shown in fig. 2 is only for example and is not intended to limit the technical solution of the present application. Those skilled in the art will appreciate that in a specific implementation, the data transmission system 200 may include further storage devices and virtual machines, without limitation.
According to an embodiment of the present invention, there is provided a data transmission method embodiment, it being noted that the steps shown in the flowcharts of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.
In this embodiment, a data transmission method is provided, which may be used in the cloud platform described above, and fig. 3 is a flowchart of the data transmission method according to an embodiment of the present invention, as shown in fig. 3, where the flowchart includes the following steps:
s301, receiving a data transmission request sent by a virtual machine.
The data transmission request is used for requesting to transmit the target file in the first logic volume to the second logic volume in the storage device. The data transfer request includes a device identifier of the storage device, a volume identifier of the first logical volume, a volume identifier of the second logical volume, a source start address and a source end address of the target file stored in the first logical volume, and a target start address and a target end address of the target file to be stored in the second logical volume.
S302, acquiring a transmission channel between the cloud platform and the storage device, and acquiring at least one transmission file taking the size of a target data block as a unit.
Wherein at least one transport file constitutes the target file.
In one example, a cloud platform obtains a transmission channel between the cloud platform and a storage device in response to a selection operation of a user on a display interface of the cloud platform.
Illustratively, a user selects to turn on the "drop data transfer feature" on the display interface of the cloud platform. The cloud platform responds to the selection operation of a user on a display interface of the cloud platform, and a transmission channel between the cloud platform and the storage device is opened, namely, the transmission channel between the cloud platform and the storage device is obtained. The 'drop data transmission characteristic' is used for indicating that a transmission channel is supported to be opened between the cloud platform and the storage device.
It can be understood that if the user selects to close the "drop data transmission characteristic" on the display interface of the cloud platform, the cloud platform cannot acquire the transmission channel between the cloud platform and the storage device, and the switch is still required to be used as the transmission device between the cloud platform and the storage device for data transmission.
In some optional embodiments, before receiving a data transmission request sent by a virtual machine, when the cloud platform is started, the cloud platform sends a query instruction of a preset protocol to each logical volume in the storage device according to the granularity of the logical volumes, and receives response information of each logical volume returned by the storage device.
The format of the query instruction is a command description block (Command Descriptor Block, CDB) structure format. The length of the query instruction is byte-12. As shown in table 1, table 1 is a schematic diagram of a query instruction.
TABLE 1
In Table 1, the query instruction includes an OPERATION CODE (F0 h), a storage device IDENTIFIER (STOR ID), and a volume IDENTIFIER (LUN ID) of the logical volume.
The predetermined protocol is a Small Computer system interface (Small Computer SYSTEM INTERFACE, SCSI) protocol.
The response information is used to indicate whether each logical volume supports data transfer characteristics and transfer data block size information set by each logical volume. The format of the response information is a CDB structure format. The length of the response information is byte-6.
As shown in table 2, table 2 is a schematic diagram of response information, and in table 2, the response information includes whether each logical volume SUPPORTs a data transfer characteristic (SUPPORT TYPE) and transfer data BLOCK SIZE information (BLOCK SIZE).
TABLE 2
Whether each logical volume supports data transfer characteristics may include supported, unsupported, and unknown three states. The value in RESULT in table 2 is 1 when each logical volume supports the data transfer characteristic, 0 when each logical volume does not support the data transfer characteristic, and 3 when each logical volume does not determine whether the data transfer characteristic is supported (i.e., unknown).
It will be appreciated that if each logical volume supports a data transfer characteristic is in an "unknown" state, it will also be considered that the data transfer characteristic is not supported.
In some optional embodiments, the cloud platform acquires the size information of the transmission data block set by the second logical volume, determines that the unit of the storage device for transmitting the target file is the size of the target data block according to the size information of the transmission data block and a preset strategy, acquires the target file, and divides the target file into at least one transmission file with the size of the target data block.
In one example, the cloud platform determines that the target data block size is the first value when the transmission data block size information indicates that the transmission data block size set by the second logical volume is greater than or equal to the first value, and determines that the target data block size is the set transmission data block size when the transmission data block size information indicates that the transmission data block size set by the second logical volume is greater than or equal to the second value and less than the first value.
The first value or the second value may be set according to actual needs, and is not limited. For example, the first value may be 4MB and the second value may be 1MB.
In some optional embodiments, if the transport data block size information indicates a transport data block size not set by the second logical volume, the cloud platform determines that the target data block size is a second value.
It can be understood that when the size of the transmission data block is not set by the second logical volume, the cloud platform determines the size of the target data block to be a smaller second value, so that the cloud platform can complete transmission of each transmission file in a shorter time, and waiting time of each read-write operation is reduced, thereby reducing overall delay.
In one example, a cloud platform obtains a target file from a first logical volume based on a source start address and a source end address in a data transfer request.
And S303, when the first logic volume and the second logic volume support the data transmission characteristics of data transmission between the logic volumes, generating at least one transmission request based on each transmission file, and sequentially sending each transmission request to the storage device through the transmission channel, so that the storage device transmits each transmission file to the second logic volume until the transmission of at least one transmission file is completed.
Wherein each transfer request is for transferring one transfer file to a second logical volume. The format in each transmission request is a CDB structure format. Each transmission request is byte-12 in length.
As shown in table 3, table 3 is a schematic diagram of one transmission request, and in table 3, each transmission request includes an OPERATION CODE (F1 h)) and a parameter list length (PARAMETER LIST LENGTH) corresponding to each transmission file.
TABLE 3 Table 3
As shown in table 4, each transmission request also includes a list of parameters. In table 4, the parameter list includes a SOURCE start address (SOURCE BEGIN LBA) at the first logical volume, a SOURCE END address (SOURCE END LBA) at the first logical volume, a TARGET start address (TARGET BEGIN LBA) at the second logical volume, a TARGET END address (TARGET END LBA) at the second logical volume, and a file size (LENGTH) of each transport file.
TABLE 4 Table 4
It can be appreciated that the cloud platform sends each transmission request to the storage device in turn through the transmission channel.
Based on the method of fig. 3, the cloud platform may receive a data transmission request sent from the virtual machine, obtain a transmission channel between the cloud platform and the storage device and at least one transmission file with a target data block size as a unit, generate at least one transmission request based on each transmission file when the first logical volume and the second logical volume support data transmission characteristics of data transmission between the logical volumes, and sequentially send each transmission request to the storage device through the transmission channel, so as to transmit the target file.
In addition, the first logical volume and the second logical volume support the data transmission characteristics of data transmission between the logical volumes, so that the storage device can directly transmit the target file in the first logical volume to the second logical volume in the storage device. When the data transmission is carried out between the cloud platform and the storage device, the switch is not needed, and the file transmission can be directly carried out in the storage device, so that the data transmission efficiency can be improved, the computing resources and the memory resources of the virtual machine are saved, the network resources among the virtual machines and among the virtual machine, the switch and the storage device are saved, the performances of the storage device and the cloud platform are improved, and the user experience is improved.
In this embodiment, there is provided a further data transmission method, which may be used in the above-mentioned storage device, and fig. 4 is a flowchart of a further data transmission method according to an embodiment of the present invention, as shown in fig. 4, where the flowchart includes the following steps:
s401, under the condition that a transmission channel between the cloud platform and the storage device is acquired, sequentially receiving at least one transmission request sent by the cloud platform through the transmission channel.
And S402, when the first logic volume and the second logic volume support the data transmission characteristics of data transmission between the logic volumes, transmitting each transmission file to the second logic volume according to each transmission request.
Wherein the at least one transmission request comprises a first transmission request. The first transmission request is for transmitting a first transmission file.
In an example, a storage device obtains a target address, carried in a first transmission request, of a first transmission file to be stored in a second logical volume, determines a target data block, corresponding to the target address, of the first transmission file to be stored according to the target address, detects whether the target data block is in a locked state, locks the target data block if not, and writes the first transmission file into the target data block.
The target address comprises a target starting address and a target ending address of the first transmission file.
In an example, if the target data block is in the locked state, the storage device sends a hint information to the cloud platform, where the hint information is used to indicate that the first transport file cannot be written to the target data block temporarily.
It will be appreciated that if the target data block is in the locked state, indicating that the target data block is performing other modification operations, the data block is being modified and temporarily inaccessible. During the time that a data block is locked, other operations attempting to modify the data block are prevented from being confused by multiple simultaneous modification operations.
In one example, a storage device writes source data (a first transfer file) at a source start address and a source end address in a first transfer request into a target start address and a target end address.
In one example, after the internal transmission of the first transmission file is completed, the storage device sends notification information to the cloud platform through the transmission channel, and sets the target data block to an unlocked state.
The notification information is used for notifying the cloud platform of the transmission result of the first transmission file. The format of the notification information is a CDB structure format. Each announcement message has a length of byte-6.
As shown in Table 5, table 5 is a schematic diagram of notification information, and in Table 5, the notification information includes a transmission result (DATA TRANSFER) of the first transmission file.
TABLE 5
Based on the method shown in fig. 4, the storage device may sequentially receive, through the transmission channel, at least one transmission request sent from the cloud platform when the transmission channel between the cloud platform and the storage device is acquired, and when the first logical volume and the second logical volume both support the data transmission characteristics of data transmission between the logical volumes, transmit each transmission file to the second logical volume according to each transmission request, so as to complete transmission of the target file.
In addition, the first logical volume and the second logical volume support the data transmission characteristics of data transmission between the logical volumes, so that the storage device can directly transmit the target file in the first logical volume to the second logical volume in the storage device. When the data transmission is carried out between the cloud platform and the storage device, the switch is not needed, and the file transmission can be directly carried out in the storage device, so that the data transmission efficiency can be improved, the computing resources and the memory resources of the virtual machine are saved, the network resources among the virtual machines and among the virtual machine, the switch and the storage device are saved, the performances of the storage device and the cloud platform are improved, and the user experience is improved.
In this embodiment, a cloud platform or a storage device is further provided, and the cloud platform or the storage device is used to implement the foregoing embodiments and preferred embodiments, which are not described herein. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The embodiment provides a cloud platform, as shown in fig. 5, fig. 5 is a structural block diagram of the cloud platform according to an embodiment of the present invention, where the cloud platform includes:
The transceiver module 501 is configured to receive a data transmission request sent from a virtual machine, where the data transmission request is used to request to transmit a target file in a first logical volume in a storage device to a second logical volume.
The transceiver module 501 is further configured to obtain a transmission channel between the cloud platform and the storage device, and obtain at least one transmission file with a size of a target data block as a unit, where the at least one transmission file forms the target file.
And the processing module 502 is configured to generate at least one transmission request based on each transmission file when the first logical volume and the second logical volume both support the data transmission characteristics of data transmission between the logical volumes, and sequentially send each transmission request to the storage device through the transmission channel, so that the storage device transmits each transmission file to the second logical volume until the transmission of at least one transmission file is completed.
In some optional embodiments, the transceiver module 501 is specifically configured to obtain a transmission channel between the cloud platform and the storage device in response to a selection operation of a user on a display interface of the cloud platform.
In some optional embodiments, before receiving the data transmission request sent by the virtual machine, the processing module 502 is further configured to send, when the cloud platform is started, a query instruction of a preset protocol to each logical volume in the storage device according to the granularity of the logical volume, and the transceiver module 501 is further configured to receive response information of each logical volume returned by the storage device, where the response information is used to indicate whether each logical volume supports the data transmission characteristic and the transmission data block size information set by each logical volume.
In some optional embodiments, the transceiver module 501 is further specifically configured to obtain the size information of the transmission data block set by the second logical volume, determine, according to the size information of the transmission data block and a preset policy, that a unit of the storage device for transmitting the target file is the size of the target data block, obtain the target file, and divide the target file into at least one transmission file with the size of the target data block.
In some optional embodiments, the transceiver module 501 is further specifically configured to determine that the target data block size is the first value when the transport data block size information indicates that the transport data block size set by the second logical volume is greater than or equal to the first value, and determine that the target data block size is the set transport data block size when the transport data block size information indicates that the transport data block size set by the second logical volume is greater than or equal to the second value and less than the first value.
The embodiment provides a storage device, as shown in fig. 6, fig. 6 is a block diagram of the storage device according to an embodiment of the present invention, where the storage device includes:
The receiving module 601 is configured to sequentially receive, through a transmission channel, at least one transmission request sent from the cloud platform, where the at least one transmission request is generated by the cloud platform based on the received data transmission request sent from the virtual machine and at least one transmission file in units of a target data block size, where the data transmission request is used to request to transmit a target file in a first logical volume in the storage device to a second logical volume, where the at least one transmission file constitutes the target file, where the transmission channel between the cloud platform and the storage device is acquired.
And a transmission module 602, configured to transmit each transmission file to the second logical volume according to each transmission request when the first logical volume and the second logical volume both support the data transmission characteristic of data transmission between the logical volumes.
In some alternative embodiments, the at least one transmission request includes a first transmission request, where the first transmission request is used to transmit a first transmission file, a transmission module 602 is specifically configured to obtain a target address of the first transmission file carried in the first transmission request to be stored in the second logical volume, determine, according to the target address, a target data block of the first transmission file corresponding to the target address to be stored, detect whether the target data block is in a locked state, and if not, lock the target data block and write the first transmission file into the target data block.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The cloud platform or storage device in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application SPECIFIC INTEGRATED Circuit) circuits, processors and memories that execute one or more software or firmware programs, and/or other devices that can provide the above-described functionality.
The embodiment of the invention also provides computer equipment, which is provided with the cloud platform shown in the figure 5 or the storage equipment shown in the figure 6.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, and as shown in fig. 7, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting components, including a high-speed interface and a low-speed interface. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 7.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.
The memory 20 may include a storage program area that may store an operating system, application programs required for at least one function, and a storage data area that may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The memory 20 may comprise volatile memory, such as random access memory, or nonvolatile memory, such as flash memory, hard disk or solid state disk, or the memory 20 may comprise a combination of the above types of memory.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random-access memory, a flash memory, a hard disk, a solid state disk, or the like, and further, the storage medium may further include a combination of the above types of memories. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Portions of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or aspects in accordance with the present application by way of operation of the computer. Those skilled in the art will appreciate that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, and the like, and accordingly, the manner in which computer program instructions are executed by a computer includes, but is not limited to, the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled programs, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed programs. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims (10)

CN202412000297.8A2024-12-312024-12-31 A data transmission method, cloud platform, computer equipment and storage mediumPendingCN119814770A (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202412000297.8ACN119814770A (en)2024-12-312024-12-31 A data transmission method, cloud platform, computer equipment and storage medium

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202412000297.8ACN119814770A (en)2024-12-312024-12-31 A data transmission method, cloud platform, computer equipment and storage medium

Publications (1)

Publication NumberPublication Date
CN119814770Atrue CN119814770A (en)2025-04-11

Family

ID=95278396

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202412000297.8APendingCN119814770A (en)2024-12-312024-12-31 A data transmission method, cloud platform, computer equipment and storage medium

Country Status (1)

CountryLink
CN (1)CN119814770A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US20050120093A1 (en)*2001-05-102005-06-02Hitachi, Ltd.Remote copy for a storgae controller in a heterogeneous environment
CN1642170A (en)*2004-01-162005-07-20英业达股份有限公司 Method of Accessing Logical Devices Using iSCSI Protocol
CN1645335A (en)*2003-06-182005-07-27国际商业机器公司Method, system, and article of manufacture for consistent copying of storage volumes
CN1790250A (en)*2002-09-182006-06-21株式会社日立制作所Storage system, and method for controlling the same
US8290911B1 (en)*2009-06-302012-10-16Symantec CorporationSystem and method for implementing data deduplication-aware copying of data
CN105549904A (en)*2015-12-082016-05-04华为技术有限公司 A data migration method and storage device applied in a storage system
US20180095667A1 (en)*2016-10-042018-04-05Pure Storage, Inc.Migrating data between volumes using virtual copy operation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US20050120093A1 (en)*2001-05-102005-06-02Hitachi, Ltd.Remote copy for a storgae controller in a heterogeneous environment
CN1790250A (en)*2002-09-182006-06-21株式会社日立制作所Storage system, and method for controlling the same
CN1645335A (en)*2003-06-182005-07-27国际商业机器公司Method, system, and article of manufacture for consistent copying of storage volumes
CN1642170A (en)*2004-01-162005-07-20英业达股份有限公司 Method of Accessing Logical Devices Using iSCSI Protocol
US8290911B1 (en)*2009-06-302012-10-16Symantec CorporationSystem and method for implementing data deduplication-aware copying of data
CN105549904A (en)*2015-12-082016-05-04华为技术有限公司 A data migration method and storage device applied in a storage system
US20180095667A1 (en)*2016-10-042018-04-05Pure Storage, Inc.Migrating data between volumes using virtual copy operation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
詹求翔;王春露;生拥宏;鞠大鹏;汪东升;: "一种基于数据差异的逻辑卷快速同步方法", 计算机工程与应用, no. 01, 1 January 2010 (2010-01-01)*

Similar Documents

PublicationPublication DateTitle
KR102427561B1 (en)A novel ssd architecture for fpga based acceleration
CN112948318B (en)RDMA-based data transmission method and device under Linux operating system
US11262916B2 (en)Distributed storage system, data processing method, and storage node
CN112596960B (en)Distributed storage service switching method and device
US20150106822A1 (en)Method and system for supporting resource isolation in multi-core architecture
US11789718B2 (en)System and method for subscription based solution modification implementation
US11010094B2 (en)Task management method and host for electronic storage device
US11588909B1 (en)System and method for subscription based solution data compatibility
US20220253252A1 (en)Data processing method and apparatus
CN111352899B (en)Path aggregation method, access method, communication device, and storage medium
CN115396422B (en) Data transmission method and device
CN112445412B (en) Data storage method and device
CN112148206B (en) Data reading and writing method, device, electronic equipment and medium
US20110246667A1 (en)Processing unit, chip, computing device and method for accelerating data transmission
JP2015158910A (en)Memory subsystem with wrapped-to-continuous read
CN109144403B (en)Method and equipment for switching cloud disk modes
CN118916312B (en) A data transmission method, device, system, computer equipment and storage medium
CN103500108A (en)System memory access method, node processor and multi-processor system
CN114356215A (en)Distributed cluster and control method of distributed cluster lock
CN115291998B (en)Rbd-nbd mapping method and device
CN119814770A (en) A data transmission method, cloud platform, computer equipment and storage medium
US20150089264A1 (en)Processing apparatus and processing method
JP2023509040A (en) DATA TRANSMISSION METHOD, TERMINAL AND COMPUTER-READABLE STORAGE MEDIUM
CN114077583A (en)Metadata processing method, storage device and storage medium
CN112003860B (en)Memory management method, system and medium suitable for remote direct memory access

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination

[8]ページ先頭

©2009-2025 Movatter.jp