






本发明属于计算机技术领域,尤其涉及基于云服务的数据存储方法、完整性检测方法及装置、终端设备。The invention belongs to the technical field of computers, and in particular relates to a data storage method based on a cloud service, an integrity detecting method and device, and a terminal device.
近些年来,以云服务为基础的数据存储服务蓬勃发展,大型的互联网公司都提出了各自的云存储服务,在这些云存储服务中,面对普通消费者的云存储服务大多是免费的,而面向开发者的云存储服务基本是收费的。在收费的云存储服务(例如,阿里云OSS、百度开放云BOS、亚马逊S3、Windows Azure)中,存储在云端的数据常会遇到数据安全方面的问题(如数据完整性问题),而这也是用户使用云存储服务考虑的重要因素之一。为了降低或消除用户对云端数据的安全性担忧,许多针对云端数据的完整性检测方案被提出,但由于当前的云存储服务在验证云端数据完整性方面所提供的计算能力有限,使得大部分现有的完整性检测方案无法在实际云存储环境下正常运行。In recent years, cloud service-based data storage services have flourished, and large Internet companies have proposed their own cloud storage services. Among these cloud storage services, cloud storage services for ordinary consumers are mostly free. Cloud storage services for developers are basically charged. In the cloud storage services that are charged (for example, Alibaba Cloud OSS, Baidu Open Cloud BOS, Amazon S3, Windows Azure), data stored in the cloud often encounters data security issues (such as data integrity issues), and this is also One of the important factors that users consider when using cloud storage services. In order to reduce or eliminate users' security concerns about cloud data, many integrity detection schemes for cloud data are proposed, but most of the current cloud storage services provide limited computing power in verifying cloud data integrity. Some integrity detection schemes cannot work properly in an actual cloud storage environment.
因此,在用户端或云存储客户端进行数据完整性检测的方案被提出。当用户下载其存储在云端的数据时,若需要对数据完整性进行验证,首先将待检测数据从云端下载到本地,然后通过哈希(Hash)函数或者消息认证码(MAC)对下载下来的数据进行完整性验证。在这种方式中,当待检测的数据较小时,可以把这些数据直接从云端下载下来进行检测。但是,当所要验证的数据较大时,直接下载的方式就不适用了。Therefore, a scheme for data integrity detection at the client or cloud storage client is proposed. When the user downloads the data stored in the cloud, if the data integrity needs to be verified, the data to be detected is first downloaded from the cloud to the local, and then downloaded through a hash function or a message authentication code (MAC). The data is integrity verified. In this way, when the data to be detected is small, the data can be downloaded directly from the cloud for detection. However, when the data to be verified is large, the direct download method does not apply.
现在对较大文件进行完整性检测的主流做法是采用随机方式进行分块,进行抽样检测,同样本地检测也需要考虑到云存储服务器的负载问题,例如,带宽、请求响应等方面的系统开销,对于一些重要用户,例如,银行用户等,在数据存储期间可能需要多次进行完整性检测或在每次下载时都需要进行完整性检测,这对云存储服务器的软、硬件提出了更高的要求,对应地需要支付更高的云存储服务费。因此,现有技术缺少一种有效的用于云服务的数据存储方法,以最大程度地减少云存储服务(包括数据存储、检测等)需要的软、硬件开销,从而降低云存储服务的成本。The current mainstream practice for integrity detection of large files is to perform block detection in a random manner, and the same local detection needs to take into account the load of the cloud storage server, such as bandwidth, request response, etc. For some important users, for example, bank users, etc.Integrity detection may be required multiple times during data storage or integrity detection is required for each download. This puts higher requirements on the software and hardware of the cloud storage server, and correspondingly needs to pay for higher cloud storage services. fee. Therefore, the prior art lacks an effective data storage method for cloud services to minimize the software and hardware overhead required for cloud storage services (including data storage, detection, etc.), thereby reducing the cost of cloud storage services.
发明内容Summary of the invention
本发明实施例的目的在于提供一种基于云服务的数据存储方法、完整性检测方法及装置,旨在解决由于现有技术无法提供一种有效的用于云服务的数据存储方法,以最大程度地减少云存储服务需要的软、硬件开销,从而降低云存储服务的成本。The object of the embodiments of the present invention is to provide a cloud service-based data storage method, an integrity detection method, and a device, which are intended to solve the problem that the prior art cannot provide an effective data storage method for cloud services to the maximum extent. Reduce the software and hardware overhead required by cloud storage services, thereby reducing the cost of cloud storage services.
一方面,本发明提供了一种基于云服务的数据存储方法,所述方法包括下述步骤:In one aspect, the present invention provides a cloud service-based data storage method, the method comprising the following steps:
接收将一文件上传到提供所述云服务的云存储服务器的请求;Receiving a request to upload a file to a cloud storage server providing the cloud service;
根据所述云服务的开销参数确定将所述文件划分为文件分块的数量,所述数量使得所述文件的存储开销最小;Determining, according to an overhead parameter of the cloud service, the number of files to be divided into file segments, the number of which minimizes storage overhead of the file;
将所述文件划分为所述数量的数据分块,并计算每个数据分块的基于Hash函数的消息认证码(Hash-based Message Authentication Code,缩写为HMAC);Dividing the file into the number of data blocks, and calculating a hash-based message authentication code (HMAC) for each data block;
将每个数据分块和对应的HMAC上传到所述云存储服务器存储。Each data chunk and corresponding HMAC are uploaded to the cloud storage server storage.
另一方面,本发明还提供了一种基于云服务的数据存储装置,所述装置包括:In another aspect, the present invention also provides a cloud service-based data storage device, the device comprising:
请求接收单元,用于接收将一文件上传到提供所述云服务的云存储服务器的请求;a request receiving unit, configured to receive a request to upload a file to a cloud storage server that provides the cloud service;
分块数量确定单元,用于根据所述云服务的开销参数确定将所述文件划分为文件分块的数量,所述数量使得所述文件的存储开销最小;a block number determining unit, configured to determine, according to an overhead parameter of the cloud service, the number of dividing the file into file blocks, where the number is such that a storage overhead of the file is minimized;
文件处理单元,用于将所述文件划分为所述数量的数据分块,并计算每个数据分块的HMAC;以及a file processing unit, configured to divide the file into the number of data blocks, and calculate eachHMAC of data chunking;
数据上传单元,用于将每个数据分块和对应的HMAC上传到所述云存储服务器存储。And a data uploading unit, configured to upload each data chunk and the corresponding HMAC to the cloud storage server storage.
另一方面,本发明还提供了一种终端设备,所述终端设备包括:In another aspect, the present invention also provides a terminal device, where the terminal device includes:
数据接收器,用于接收将一文件上传到提供所述云服务的云存储服务器的请求;a data receiver, configured to receive a request to upload a file to a cloud storage server that provides the cloud service;
中央处理单元,用于根据所述云服务的开销参数确定将所述文件划分为文件分块的数量,所述数量使得所述文件的存储开销最小,将所述文件划分为所述数量的数据分块,并计算每个数据分块的HMAC;以及a central processing unit, configured to determine, according to an overhead parameter of the cloud service, the number of dividing the file into file partitions, the quantity is such that a storage overhead of the file is minimized, and the file is divided into the quantity of data Blocking and calculating the HMAC for each data chunk; and
数据发送器,用于将每个数据分块和对应的HMAC上传到所述云存储服务器存储。a data sender, configured to upload each data block and corresponding HMAC to the cloud storage server storage.
另一方面,本发明还提供了一种基于云服务的完整性检测方法,所述完整性检测方法用于对前述数据存储方法存储的文件进行完整性检测,所述完整性检测方法包括步骤:In another aspect, the present invention further provides a cloud service-based integrity detection method, the integrity detection method for performing integrity detection on a file stored by the foregoing data storage method, the integrity detection method comprising the steps of:
从所述云存储服务器随机选择所述文件的文件分块,抽取的数量为所述样本数量;Randomly selecting a file partition of the file from the cloud storage server, and extracting the quantity is the sample quantity;
将选择的文件分块和所述选择的文件分块对应的HMAC下载到本地终端,将所述选择的文件分块对应的HMAC记为第一HMAC;Downloading the selected file partition and the HMAC corresponding to the selected file partition to the local terminal, and recording the HMAC corresponding to the selected file partition as the first HMAC;
计算所述选择的文件分块对应HMAC,将计算得到的HMAC计为第二HMAC;Calculating the selected file partition corresponding to the HMAC, and calculating the calculated HMAC as the second HMAC;
判断所述第一HMAC和第二HMAC是否相同,是则输出数据完整的结果,否则输出数据已损坏的结果。Determining whether the first HMAC and the second HMAC are the same, is the result of outputting the complete data, otherwise the output data is corrupted.
在本发明实施例中,在接收到将一文件上传到提供云服务的云存储服务器的请求后,根据云服务的开销参数确定将文件划分为文件分块的数量,该数量可使得文件的存储开销最小,根据确定的该数量将文件划分数据分块,并计算每个数据分块的HMAC,最后将每个数据分块和对应的HMAC上传到所述云存储服务器存储,从而最大程度地减少云存储服务需要的软、硬件开销,降低了云存储服务的成本。In the embodiment of the present invention, after receiving the request for uploading a file to the cloud storage server that provides the cloud service, determining, according to the overhead parameter of the cloud service, dividing the file into the number of file partitions, the quantity may cause the file to be stored. The overhead is minimal, the file partitioning data is divided according to the determined quantity, and the HMAC of each data block is calculated, and finally each data block and the corresponding HMAC are uploaded to the cloud.Storage server storage, which minimizes the software and hardware overhead required for cloud storage services and reduces the cost of cloud storage services.
图1是本发明实施例提供的基于云服务的数据存储方法的实现流程图;FIG. 1 is a flowchart of implementing a cloud service-based data storage method according to an embodiment of the present invention;
图2是本发明实施例提供的基于云服务的数据存储方法中确定将文件划分为文件分块的数量的实现流程图;FIG. 2 is a flowchart of an implementation of determining a number of file partitions into file partitions in a cloud service-based data storage method according to an embodiment of the present disclosure;
图3是本发明实施例提供的基于云服务的数据存储装置的结构图;3 is a structural diagram of a cloud service-based data storage device according to an embodiment of the present invention;
图4是本发明实施例提供的基于云服务的数据存储装置的结构图;4 is a structural diagram of a cloud service-based data storage device according to an embodiment of the present invention;
图5是本发明实施例提供的基于云服务的数据存储装置中参数获取单元的结构图;5 is a structural diagram of a parameter obtaining unit in a cloud service-based data storage device according to an embodiment of the present invention;
图6是本发明实施例提供的终端设备的结构图;以及FIG. 6 is a structural diagram of a terminal device according to an embodiment of the present invention;
图7是本发明实施例提供的基于云服务的完整性检测方法的实现流程图。FIG. 7 is a flowchart of an implementation of a cloud service-based integrity detection method according to an embodiment of the present invention.
为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
以下结合具体实施例对本发明的具体实现进行详细描述:The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:
图1示出了本发明实施例提供的基于云服务的数据存储方法的实现流程,详述如下:FIG. 1 is a flowchart showing an implementation process of a cloud service-based data storage method according to an embodiment of the present invention, which is described in detail as follows:
在步骤S101中,接收将一文件上传到提供云服务的云存储服务器的请求。In step S101, a request to upload a file to a cloud storage server providing a cloud service is received.
在本发明实施例中,当用户请求将文件上传到云存储服务器进行存储时,可通过云存储服务器的专用客户端或浏览器进行上传,云存储服务器用于对接收的文件数据进行存储,并处理文件数据存储期间用户对其存储的数据进行完整性验证或检测的请求。In the embodiment of the present invention, when the user requests to upload the file to the cloud storage server for storage, the file may be uploaded through a dedicated client or a browser of the cloud storage server, and the cloud storage server is configured to store the received file data, and A request by the user to perform integrity verification or detection of the data it stores during file data storage.
在步骤S102中,根据云服务的开销参数确定将文件划分为文件分块的数量,确定的该数量使得文件的存储开销最小。In step S102, the number of files divided into file partitions is determined according to the overhead parameter of the cloud service, and the determined number minimizes the storage overhead of the file.
在本发明实施例中,用户云存储服务的开销主要来自保存(Storage)、数据外流(Data Transfer)、请求(Request)三个方面,因此,当前国内外主流云存储服务提供商在这三个方面都采取收费方式,以用于补偿云存储服务器在这些方面的软、硬件开销。本发明实施例根据云服务的开销参数确定将文件划分为文件分块的数量,确定的该数量使得文件的存储开销最小,从而最大程度地降低云存储服务器在这些方面的软、硬件开销,减少用户使用云存储服务的成本。In the embodiment of the present invention, the overhead of the user cloud storage service mainly comes from three aspects: storage, data transfer, and request. Therefore, current mainstream cloud storage service providers at home and abroad are in the three. The charging method is adopted to compensate the software and hardware overhead of the cloud storage server in these aspects. In the embodiment of the present invention, the number of file partitions is determined according to the cost parameter of the cloud service, and the determined number is such that the storage overhead of the file is minimized, thereby minimizing the software and hardware overhead of the cloud storage server in these aspects, and reducing The cost of a user using a cloud storage service.
在步骤S103中,将文件划分为确定数量的数据分块,并计算每个数据分块的HMAC。In step S103, the file is divided into a determined number of data blocks, and the HMAC of each data block is calculated.
在本发明实施例中,根据步骤S102中确定的划分数量对用户上传的文件进行划分,得到对应的数据分块,并计算每个数据分块的HMAC。In the embodiment of the present invention, the file uploaded by the user is divided according to the number of divisions determined in step S102, and corresponding data blocks are obtained, and the HMAC of each data block is calculated.
在步骤S104中,将每个数据分块和对应的HMAC上传到云存储服务器存储。In step S104, each data block and corresponding HMAC are uploaded to the cloud storage server storage.
在本发明实施例中,在接收到将一文件上传到提供云服务的云存储服务器的请求后,根据云服务的开销参数确定将文件划分为文件分块的数量,该数量可使得文件的存储开销最小,根据确定的该数量将文件划分数据分块,并计算每个数据分块的HMAC,最后将每个数据分块和对应的HMAC上传到所述云存储服务器存储,从而最大程度地减少云存储服务需要的软、硬件开销,降低了云存储服务的成本。In the embodiment of the present invention, after receiving the request for uploading a file to the cloud storage server that provides the cloud service, determining, according to the overhead parameter of the cloud service, dividing the file into the number of file partitions, the quantity may cause the file to be stored. The overhead is minimal, the file partitioning data is divided according to the determined number, and the HMAC of each data chunk is calculated, and finally each data chunk and the corresponding HMAC are uploaded to the cloud storage server storage, thereby minimizing The software and hardware overhead required by cloud storage services reduces the cost of cloud storage services.
图2示出了本发明实施例提供的基于云服务的数据存储方法中确定将文件划分为文件分块的数量的实现流程,详述如下:FIG. 2 is a flowchart showing an implementation process for determining a number of files to be divided into file blocks in a cloud service-based data storage method according to an embodiment of the present invention, which is described in detail as follows:
在步骤S201中,获取文件的存储时间、在存储时间内请求对文件进行完整性检测的次数、文件的大小、对文件进行完整性检测时抽取的样本数量、云服务的存储开销、数据传送开销、请求开销、每个HMAC所占的字节。In step S201, the storage time of the file is acquired, the number of times the integrity check is requested for the file during the storage time, the size of the file, the number of samples extracted when the file is integrity-detected, the storage overhead of the cloud service, and the data transmission overhead are obtained. , request overhead, bytes occupied by each HMAC.
在本发明实施例中,为了使用户在使用云存储服务时占用最小的存储开销,付出最低的成本,需要预先确定将文件划分为文件分块的数量。其中,云存储服务器的存储开销包括保存(Storage)、数据外流(Data Transfer)、请求(Request)三个方面的开销,涉及到文件的存储、下载、完整性检测事务。因此,在确定将文件划分为文件分块的数量时,需要获取的开销参数包括:文件的存储时间、在存储时间内请求对文件进行完整性检测的次数、文件的大小、对文件进行完整性检测时抽取的样本数量、云服务的存储开销、数据传送开销、请求开销、每个HMAC所占的字节。In the embodiment of the present invention, in order to make the user occupy the minimum storage cost when using the cloud storage service and pay the lowest cost, it is necessary to determine in advance the number of files to be divided into file blocks. The storage overhead of the cloud storage server includes the overheads of storage, data transfer, and request, and involves storage, download, and integrity detection of files. Therefore, when determining the number of files to be divided into file partitions, the cost parameters that need to be acquired include: the storage time of the file, the number of times the integrity check is requested for the file during the storage time, the size of the file, and the integrity of the file. The number of samples extracted during detection, the storage overhead of the cloud service, the data transfer overhead, the request overhead, and the bytes occupied by each HMAC.
在步骤S202中,根据获取的存储时间、完整性检测的次数、文件的大小、样本数量、云服务的存储开销、数据传送开销、请求开销、每个HMAC所占的字节计算文件的存储开销。In step S202, the storage cost of the file is calculated according to the acquired storage time, the number of times of integrity detection, the size of the file, the number of samples, the storage overhead of the cloud service, the data transmission overhead, the request overhead, and the byte occupied by each HMAC. .
在本发明实施例中,云存储服务器的存储开销为保存(Storage)、数据外流(Data Transfer)、请求(Request)三个方面的开销之和,最终可使用公式(1)计算文件的存储开销:In the embodiment of the present invention, the storage overhead of the cloud storage server is the sum of the overheads of storage, data transfer, and request, and finally the storage overhead of the file can be calculated using formula (1). :
其中,p1表示所述存储开销、p2表示所述数据传送开销、p3表示所述请求开销、n表示所述文件的大小、c表示所述样本数量、s表示所述文件的大小、s’表示所述每个所述HMAC所占的字节、i表示所述存储时间、j表示所述完整性检测的次数、g表示一常数。Wherein p1 represents the storage overhead, p2 represents the data transfer overhead, p3 represents the request overhead, n represents the size of the file, c represents the number of samples, s represents the size of the file, s' denotes the byte occupied by each of the HMACs, i denotes the storage time, j denotes the number of times of integrity detection, and g denotes a constant.
在具体实施过程中,p1、p2、p3可由具体数值来表示,例如,存储所需的空间、传送的数据量、请求处理的次数(包括下载、检测等),可选地,也可以由对应需支付的费用来表示。In a specific implementation process, p1 , p2 , and p3 may be represented by specific numerical values, for example, storage required space, amount of data transmitted, number of requests for processing (including downloading, detection, etc.), optionally, also It can be expressed by the corresponding fee to be paid.
在具体实施过程中,获取对抽取的样本数量参数时,可首先获取云服务存储文件的损坏率以及检测准确率,之后根据损坏率、检测准确率确定对文件进行完整性检测时抽取的样本数量。其中,检测准确率是指当存在文件损坏时,检测到文件损坏的几率,用户可自行设定,也可以由系统默认设置。优选地,在获取云服务存储文件的损坏率以及检测准确率后,可通过等式p=1-(1-k)c确定抽取的样本数量,其中,p表示检测准确率,k表示损坏率,从而确定最佳的抽取样本数量,在保证检测准确率的同时,减少完整性检测方面的开销。In the specific implementation process, when the sample quantity parameter is extracted, the damage rate of the cloud service storage file and the detection accuracy rate may be first obtained, and then the number of samples extracted when the integrity check of the file is determined according to the damage rate and the detection accuracy rate is determined. . Among them, the detection accuracy rate refers to the probability of detecting file corruption when there is file corruption, which can be set by the user or set by the system by default. Preferably, after obtaining the damage rate of the cloud service storage file and the detection accuracy, the number of samples extracted may be determined by the equation p=1-(1-k)c , where p represents the detection accuracy and k represents the damage rate. In order to determine the optimal number of samples to be taken, the accuracy of the detection is reduced while reducing the overhead of integrity detection.
在步骤S203中,确定存储开销为最小存储开销时文件的文件分块的数量。In step S203, the number of file partitions of the file when the storage overhead is the minimum storage overhead is determined.
在本发明实施例中,通过对公式(1)进行化简,文件的存储开销可由公式(2)表示:In the embodiment of the present invention, by simplifying the formula (1), the storage overhead of the file can be expressed by the formula (2):
其中:among them:
可以得到,当M取得最小值时,存在一个唯一的n值,即可确定存储开销为最小存储开销时文件的文件分块的数量。It can be obtained that when M obtains the minimum value, there is a unique value of n, which can determine the number of file partitions of the file when the storage overhead is the minimum storage overhead.
本发明实施例根据云服务的开销参数根据预设的公式确定将文件划分为文件分块的数量,确定的该数量使得文件的存储开销最小,从而最大程度地降低云存储服务器在这些方面的软、硬件开销,减少了用户使用云存储服务的成本。According to the preset parameter of the cloud service, the embodiment of the present invention determines the number of files to be divided into file blocks according to a preset formula, and the determined number minimizes the storage overhead of the file, thereby minimizing the softness of the cloud storage server in these aspects. The hardware overhead reduces the cost of users using cloud storage services.
图3示出了本发明实施例提供的基于云服务的数据存储装置的结构,为了便于说明,仅示出了与本发明实施例相关的部分,其中包括:Figure 3 is a diagram showing the structure of a cloud service-based data storage device according to an embodiment of the present invention. For the convenience of description, only parts related to the embodiment of the present invention are shown, including:
请求接收单元31,用于接收将一文件上传到提供所述云服务的云存储服务器的请求;The
分块数量确定单元32,用于根据所述云服务的开销参数确定将所述文件划分为文件分块的数量,所述数量使得所述文件的存储开销最小;The block
文件处理单元33,用于将所述文件划分为所述数量的数据分块,并计算每个数据分块的HMAC;以及a
数据上传单元34,用于将每个数据分块和对应的HMAC上传到所述云存储服务器存储。The
本发明实施例中,基于云服务的数据存储装置的请求接收单元31具体可以为一数据接收器,分块数量确定单元32和文件处理单元33可以由一中央处理器(CPU)来实现,数据上传单元34可以为一数据发送器,各单元的具体实施可参考图1对应实施例的实施方式,在此不再赘述。In the embodiment of the present invention, the
图4示出了本发明实施例提供的基于云服务的数据存储装置的结构,为了便于说明,仅示出了与本发明实施例相关的部分。FIG. 4 shows a structure of a cloud service-based data storage device according to an embodiment of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown.
在本发明实施例中,基于云服务的数据存储装置包括请求接收单元31、分块数量确定单元32、文件处理单元33以及数据上传单元34,其中:In the embodiment of the present invention, the cloud service-based data storage device includes a
请求接收单元31,用于接收将一文件上传到提供所述云服务的云存储服务器的请求;The
分块数量确定单元32,用于根据所述云服务的开销参数确定将所述文件划分为文件分块的数量,所述数量使得所述文件的存储开销最小;The number-of-
文件处理单元33,用于将所述文件划分为所述数量的数据分块,并计算每个数据分块的HMAC;以及a
数据上传单元34,用于将每个数据分块和对应的HMAC上传到所述云存储服务器存储。The
在本发明实施例中,分块数量确定单元32可包括参数获取单元321、开销计算单元322以及分块数量确定子单元323,其中:In the embodiment of the present invention, the number-of-
参数获取单元321,用于获取所述文件的存储时间、在所述存储时间内请求对所述文件进行完整性检测的次数、所述文件的大小、对所述文件进行完整性检测时抽取的样本数量、所述云服务的存储开销、数据传送开销、请求开销、每个所述HMAC所占的字节;The
开销计算单元322,用于根据获取的所述存储时间、所述完整性检测的次数、所述文件的大小、所述样本数量、所述云服务的存储开销、数据传送开销、请求开销、每个所述HMAC所占的字节计算所述文件的存储开销;以及The
分块数量确定子单元323,用于确定所述存储开销为最小存储开销时所述文件的文件分块的数量。The block
在本发明实施例中,云存储服务器的存储开销为保存(Storage)、数据外流(Data Transfer)、请求(Request)三个方面的开销之和,最终可使用公式(1)计算文件的存储开销。通过对公式(1)进行化简,文件的存储开销可由公式(2)表示,从而可知当M取得最小值时,存在一个唯一的n值,即可确定存储开销为最小存储开销时文件的文件分块的数量。In the embodiment of the present invention, the storage overhead of the cloud storage server is the sum of the overheads of storage, data transfer, and request, and finally the storage overhead of the file can be calculated using formula (1). . By simplifying the formula (1), the storage overhead of the file can be expressed by the formula (2), so that when the M obtains the minimum value, there is a unique n value, and the file with the storage overhead as the minimum storage overhead can be determined. The number of chunks.
在具体实施过程中,如图5所示,参数获取单元321可包括:In a specific implementation process, as shown in FIG. 5, the
参数获取子单元3211,用于获取所述云服务存储文件的损坏率、检测准确率;以及a parameter obtaining sub-unit 3211, configured to acquire a damage rate and a detection accuracy rate of the cloud service storage file;
样本数量确定子单元3212,用于根据所述损坏率、检测准确率确定对所述文件进行完整性检测时抽取的样本数量。The sample quantity determining sub-unit 3212 is configured to determine, according to the damage rate and the detection accuracy rate, the number of samples extracted when performing integrity detection on the file.
在具体实施过程中,获取对抽取的样本数量参数时,可首先获取云服务存储文件的损坏率以及检测准确率,之后根据损坏率、检测准确率确定对文件进行完整性检测时抽取的样本数量。其中,检测准确率是指当存在文件损坏时,检测到文件损坏的几率,可由用户设定。优选地,在获取云服务存储文件的损坏率以及检测准确率后,可通过等式p=1-(1-k)c确定抽取的样本数量。In the specific implementation process, when the sample quantity parameter is extracted, the damage rate of the cloud service storage file and the detection accuracy rate may be first obtained, and then the number of samples extracted when the integrity check of the file is determined according to the damage rate and the detection accuracy rate is determined. . Among them, the detection accuracy rate refers to the probability of detecting file damage when there is file corruption, which can be set by the user. Preferably, after acquiring the damage rate of the cloud service storage file and the detection accuracy, the extracted sample number can be determined by the equation p=1-(1-k)c .
作为示例地,本发明实施例中基于云服务的数据存储装置各单元的具体实施可参考图2对应实施例的实施方式,在此不再赘述。For example, the specific implementation of each unit of the cloud service-based data storage device in the embodiment of the present invention may refer to the implementation manner of the corresponding embodiment in FIG. 2, and details are not described herein again.
图6示出了本发明实施例提供的终端设备的结构,为了便于说明,仅示出了与本发明实施例相关的部分。FIG. 6 shows the structure of a terminal device according to an embodiment of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown.
在本发明实施例中,终端设备6包括数据接收器61、中央处理单元62以及数据发送器63,其中:In the embodiment of the present invention, the
数据接收器61,用于接收将一文件上传到提供云服务的云存储服务器的请求;The
中央处理单元62,用于根据云服务的开销参数确定将文件划分为文件分块的数量,确定的该数量使得文件的存储开销最小,将文件划分为确定数量的数据分块,并计算每个数据分块的HMAC;以及The
数据发送器63,用于将每个数据分块和对应的HMAC上传到云存储服务器存储。The
在本发明实施例,中央处理单元62具体用于获取文件的存储时间、在存储时间内请求对文件进行完整性检测的次数、文件的大小、对文件进行完整性检测时抽取的样本数量、云服务的存储开销、数据传送开销、请求开销、每个HMAC所占的字节,根据获取的存储时间、完整性检测的次数、文件的大小、样本数量、云服务的存储开销、数据传送开销、请求开销、每个HMAC所占的字节计算文件的存储开销,确定存储开销为最小存储开销时文件的文件分块的数量。In the embodiment of the present invention, the
在具体实施例中,中央处理单元62用于获取云服务存储文件的损坏率、检测准确率,并根据损坏率、检测准确率确定对文件进行完整性检测时抽取的样本数量。In a specific embodiment, the
在本发明实施例中,使用下述公式(1)计算文件的存储开销。进一步终端设备6还可以包括存储设备,以用于存储前述的参数、文件数据、HMAC等。In the embodiment of the present invention, the storage overhead of the file is calculated using the following formula (1). Further
图7示出了本发明实施例提供的基于云服务的完整性检测方法的实现流程,详述如下:FIG. 7 is a flowchart showing an implementation process of a cloud service-based integrity detection method according to an embodiment of the present invention, which is described in detail as follows:
在步骤S701中,从云存储服务器随机选择文件的预设数量的文件分块。In step S701, a predetermined number of file partitions of the file are randomly selected from the cloud storage server.
本发明实施例提供的完整性检测方法用于对存储在云存储服务器上的文件完整性进行检测,其中,检测的文件为使用前述图1、图2对应实施例中基于云服务的数据存储方法存储的文件。The integrity detection method provided by the embodiment of the present invention is used to detect the integrity of a file stored on a cloud storage server, where the detected file is a cloud service-based data storage method in the corresponding embodiment of FIG. 1 and FIG. 2 Stored files.
在步骤S702中,将选择的文件分块和选择的文件分块对应的HMAC下载到本地终端,将选择的文件分块对应的HMAC记为第一HMAC。In step S702, the selected file partition and the HMAC corresponding to the selected file partition are downloaded to the local terminal, and the HMAC corresponding to the selected file partition is recorded as the first HMAC.
在步骤S703中,计算选择的文件分块对应HMAC,将计算得到的HMAC计为第二HMAC。In step S703, the selected file partition is calculated to correspond to the HMAC, and the calculated HMAC is calculated as the second HMAC.
在步骤S704中,判断第一HMAC和第二HMAC是否相同,是则输出数据完整的结果,否则输出数据已损坏的结果。In step S704, it is determined whether the first HMAC and the second HMAC are the same, and the result is that the output data is complete, otherwise the output data is corrupted.
在本发明实施例中,结合前述实施例中基于云服务的数据存储方法,对使用该方法存储的数据进行检测,可最大程度地减少云存储服务中完整性检测需要的软、硬件开销,降低了云存储服务的成本。In the embodiment of the present invention, in combination with the data storage method based on the cloud service in the foregoing embodiment, the data stored by using the method is detected, which can minimize the software and hardware overhead required for integrity detection in the cloud storage service, and reduce The cost of cloud storage services.
以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201580060151.9ACN107251523B (en) | 2015-12-29 | 2015-12-29 | Data storage method based on cloud service, integrity detection method and device and terminal equipment |
| PCT/CN2015/099377WO2017113088A1 (en) | 2015-12-29 | 2015-12-29 | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2015/099377WO2017113088A1 (en) | 2015-12-29 | 2015-12-29 | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device |
| Publication Number | Publication Date |
|---|---|
| WO2017113088A1true WO2017113088A1 (en) | 2017-07-06 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2015/099377CeasedWO2017113088A1 (en) | 2015-12-29 | 2015-12-29 | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device |
| Country | Link |
|---|---|
| CN (1) | CN107251523B (en) |
| WO (1) | WO2017113088A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109583899A (en)* | 2018-12-20 | 2019-04-05 | 鸿秦(北京)科技有限公司 | Data storage and management method and system based on cloud storage service device |
| CN109802828A (en)* | 2018-12-27 | 2019-05-24 | 陕西师范大学 | Support the cloud data integrity auditing method of open verifying |
| WO2020042130A1 (en)* | 2018-08-31 | 2020-03-05 | 深圳大学 | Dual-cloud-based secure data storage method and apparatus, and device and storage medium |
| CN120256096A (en)* | 2025-03-17 | 2025-07-04 | 广州天辰信息科技有限公司 | Data processing method for elderly care service platform and elderly care service platform |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108173958A (en)* | 2018-01-05 | 2018-06-15 | 东华大学 | A data optimization storage method based on ant colony algorithm in multi-cloud environment |
| CN112860189B (en)* | 2021-02-19 | 2022-12-30 | 山东大学 | Cost-driven cold and hot layered cloud storage redundancy storage method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102281321A (en)* | 2011-04-25 | 2011-12-14 | 程旭 | Data cloud storage partitioning and backup method and device |
| US20130041872A1 (en)* | 2011-08-12 | 2013-02-14 | Alexander AIZMAN | Cloud storage system with distributed metadata |
| CN103379133A (en)* | 2012-04-16 | 2013-10-30 | 陈士平 | Safe and reliable cloud storage system |
| CN103873505A (en)* | 2012-12-12 | 2014-06-18 | 中国电信股份有限公司 | Method, system and device for offline uploading to cloud storage server |
| CN104754055A (en)* | 2015-04-03 | 2015-07-01 | 易云捷讯科技(北京)有限公司 | Safety cloud storage method for use in multi-cloud environment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8549140B2 (en)* | 2010-10-15 | 2013-10-01 | Cmp.Ly | Method and system for indicating and documenting associations, disclosures and instructions using visually identifiable description references and a standardized framework of coded instructions, hyperlinks and related visual display elements |
| CN102546755A (en)* | 2011-12-12 | 2012-07-04 | 华中科技大学 | Data storage method of cloud storage system |
| CN103414690B (en)* | 2013-07-15 | 2016-05-11 | 北京航空航天大学 | One can openly be verified the high in the clouds data property held method of calibration |
| CN104038493B (en)* | 2014-06-10 | 2017-05-17 | 电子科技大学 | Bilinear pairing-free cloud storage data security audit method |
| CN104954390B (en)* | 2015-07-17 | 2019-04-02 | 青岛大学 | It can restore the cloud storage integrality detection method and system of Lost Security Key |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102281321A (en)* | 2011-04-25 | 2011-12-14 | 程旭 | Data cloud storage partitioning and backup method and device |
| US20130041872A1 (en)* | 2011-08-12 | 2013-02-14 | Alexander AIZMAN | Cloud storage system with distributed metadata |
| CN103379133A (en)* | 2012-04-16 | 2013-10-30 | 陈士平 | Safe and reliable cloud storage system |
| CN103873505A (en)* | 2012-12-12 | 2014-06-18 | 中国电信股份有限公司 | Method, system and device for offline uploading to cloud storage server |
| CN104754055A (en)* | 2015-04-03 | 2015-07-01 | 易云捷讯科技(北京)有限公司 | Safety cloud storage method for use in multi-cloud environment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020042130A1 (en)* | 2018-08-31 | 2020-03-05 | 深圳大学 | Dual-cloud-based secure data storage method and apparatus, and device and storage medium |
| CN109583899A (en)* | 2018-12-20 | 2019-04-05 | 鸿秦(北京)科技有限公司 | Data storage and management method and system based on cloud storage service device |
| CN109802828A (en)* | 2018-12-27 | 2019-05-24 | 陕西师范大学 | Support the cloud data integrity auditing method of open verifying |
| CN109802828B (en)* | 2018-12-27 | 2021-05-28 | 陕西师范大学 | Cloud Data Integrity Audit Methods Supporting Public Verification |
| CN120256096A (en)* | 2025-03-17 | 2025-07-04 | 广州天辰信息科技有限公司 | Data processing method for elderly care service platform and elderly care service platform |
| Publication number | Publication date |
|---|---|
| CN107251523A (en) | 2017-10-13 |
| CN107251523B (en) | 2020-03-27 |
| Publication | Publication Date | Title |
|---|---|---|
| WO2017113088A1 (en) | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device | |
| CN110888838B (en) | Request processing method, device, equipment and storage medium based on object storage | |
| US10754634B1 (en) | Customized application package with context specific token | |
| US8756247B2 (en) | System and method for sharing online storage services among multiple users | |
| US8019882B2 (en) | Content identification for peer-to-peer content retrieval | |
| US8108536B1 (en) | Systems and methods for determining the trustworthiness of a server in a streaming environment | |
| US10623470B2 (en) | Optimizing internet data transfers using an intelligent router agent | |
| US9921818B2 (en) | Method and apparatus of downloading and installing a client | |
| WO2019223443A1 (en) | Method and apparatus for processing database configuration parameter, and computer device and storage medium | |
| CN103685308A (en) | Detection method and system of phishing web pages, client and server | |
| US20180234234A1 (en) | System for describing and tracking the creation and evolution of digital files | |
| CN112199175B (en) | Task queue generating method, device and equipment | |
| CN110727892A (en) | Cache data updating method and device and electronic equipment | |
| CN108733385B (en) | Application installation method, device, device and storage medium | |
| CA3068345A1 (en) | Witness blocks in blockchain applications | |
| US20200004427A1 (en) | Method, apparatus and system for controlling mounting of file system | |
| KR20180005542A (en) | Apparatus and method for verifing data integrity | |
| CN111930753B (en) | Data retrieving method and device, electronic equipment and storage medium | |
| CN111988262A (en) | Authentication method, authentication device, server and storage medium | |
| CN105847364A (en) | Public cloud object storage method based on uniform domain name and public cloud object storage system based on uniform domain name | |
| US10616291B2 (en) | Response caching | |
| WO2020042130A1 (en) | Dual-cloud-based secure data storage method and apparatus, and device and storage medium | |
| TW201608388A (en) | Network storage deduplicating method and server using the same | |
| CN112688905B (en) | Data transmission method, device, client, server and storage medium | |
| CN107623692A (en) | Method and device for data verification |
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application | Ref document number:15911719 Country of ref document:EP Kind code of ref document:A1 | |
| NENP | Non-entry into the national phase | Ref country code:DE | |
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established | Free format text:NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17/10/2018) | |
| 122 | Ep: pct application non-entry in european phase | Ref document number:15911719 Country of ref document:EP Kind code of ref document:A1 |