CN106549963A - Safe storage system based on HDFS - Google Patents

Abstract

Translated fromChinese

基于HDFS的安全的存储系统属于云存储技术领域。随着网络建设的不断发展，数据正在呈爆炸性的增长，而且现在网络攻击手段层出不穷，基于此，如何高效安全的存储这些数据成为一个急需解决的问题。本发明使用SHA256算法取代CRC32做数据校验，这样碰撞攻击的成功性就会极大的降低；然后，在数据安全性方面，HDFS则没有相关的处理，所以对于网络数据安全的问题，本发明提出了基于AES算法和RSA算法两种算法相结合的方法来确保数据的保密性，使用AES算法对数据进行加密，然后使用RSA算法对AES的秘钥进行加密，这样不仅提高了数据的安全性，而且在加密速度上并不会耗用太长的时间。实验结果表明，该方法能够有效的实现数据安全的传输以及存储。

A secure storage system based on HDFS belongs to the technical field of cloud storage. With the continuous development of network construction, data is growing explosively, and now there are endless means of network attacks. Based on this, how to store these data efficiently and safely has become an urgent problem to be solved. The present invention uses SHA256 algorithm to replace CRC32 to do data checking, and the success of collision attack will be greatly reduced like this; Then, in terms of data security, HDFS has no relevant processing, so for the problem of network data security, the present invention A method based on the combination of the AES algorithm and the RSA algorithm is proposed to ensure the confidentiality of the data, and the AES algorithm is used to encrypt the data, and then the RSA algorithm is used to encrypt the AES key, which not only improves the security of the data , and it doesn't take too long to encrypt speed. The experimental results show that this method can effectively realize the secure transmission and storage of data.

Description

Translated fromChinese

基于HDFS的安全存储系统HDFS-based secure storage system

技术领域technical field

本发明属于云存储技术领域，涉及一种基于HDFS的安全的存储系统。The invention belongs to the technical field of cloud storage, and relates to a safe storage system based on HDFS.

背景技术Background technique

在2015年12月的第二届世界互联网大会上，提到在“十三五”时期，中国将大力实施网络强国战略、国家大数据战略、“互联网+”行动计划，可以看到互联网以及大数据的重要性。如今，互联网已经深度介入人们的日常生活，在过去的一年中体现得尤其明显。一提网络，它不光是刷朋友圈、网购商品，还包括因为移动互联网而迎来大发展的数据经济。人们的网络行为会产生海量数据，而对这些数据进行分析会产生巨大的价值，所以对海量数据的存储是非常有必要的。云存储则为数据存储提供了非常便利的条件，它降低了存储海量数据的成本，因为其可以利用非常低廉的服务器，因此，云存储成为了国内外各大云服务商首先发展的一项服务。At the second World Internet Conference in December 2015, it was mentioned that during the "Thirteenth Five-Year Plan" period, China will vigorously implement the strategy of network power, the national big data strategy, and the "Internet +" action plan. It can be seen that the Internet and the big The importance of data. Today, the Internet has been deeply involved in people's daily life, especially in the past year. When it comes to the Internet, it is not only about browsing Moments and online shopping, but also includes the data economy that has ushered in great development due to the mobile Internet. People's network behavior will generate massive data, and the analysis of these data will generate huge value, so the storage of massive data is very necessary. Cloud storage provides very convenient conditions for data storage. It reduces the cost of storing massive data because it can use very cheap servers. Therefore, cloud storage has become the first service developed by major cloud service providers at home and abroad. .

但是，伴随而来的便是数据安全的问题。Hadoop的云存储系统—HDFS，没有数据加密方面的功能，数据在传输到服务器的过程中往往以明文的形式出现，这在数据安全方面造成了极大的隐患。因此，现在急需一种加入保密算法的云存储系统。However, along with it comes the issue of data security. Hadoop's cloud storage system—HDFS, has no data encryption function, and data often appears in plain text during transmission to the server, which poses a great hidden danger in data security. Therefore, there is an urgent need for a cloud storage system that adds a secret algorithm.

在网络环境中，信息安全面临两大基本攻击：被动攻击和主动攻击。对付被动攻击的主要方法是加密和解密技术，而对付主动攻击的方法就是认证技术。所以要实现安全的云存储系统必须同时考虑加密和认证两方面的问题。In the network environment, information security faces two basic attacks: passive attack and active attack. The main method to deal with passive attacks is encryption and decryption technology, while the method to deal with active attacks is authentication technology. Therefore, in order to realize a secure cloud storage system, both encryption and authentication must be considered.

加密技术是提高网络通信系统的信息保密性，防止网络传输数据泄漏的主要技术手段。目前，广泛使用的两种加密体制是对称密钥加密体制和非对称密钥加密体制。对称密钥加密体制速度快，效率高，是网络通信系统中加密大量数据的行之有效的方法。采用对称密钥加密体制设计网络通信加密方案时，必须考虑到秘钥的安全性。Encryption technology is the main technical means to improve the information confidentiality of the network communication system and prevent the leakage of network transmission data. At present, two encryption schemes widely used are symmetric key encryption scheme and asymmetric key encryption scheme. The symmetric key encryption system is fast and efficient, and it is an effective method for encrypting large amounts of data in network communication systems. When designing a network communication encryption scheme using a symmetric key encryption system, the security of the secret key must be considered.

认证技术是提供网络通信系统中通信双方身份和通信内容、过程可信度保证的技术手段。当前，在金融交易、电子商务、电子信件、手机用户信息的确认等领域，网络信息通信比较频繁，数据完整性确认和数据来源的真伪鉴定都是很重要的安全服务。因此，对认证技术的研究和实践是网络信息安全领域的一项重要内容。认证技术的应用主要包括身份认证，消息认证和数字签名。消息认证是验证信息来源和内容的基本技术，主要解决数据在通信和存储过程中的完整性问题，以确保信息不受非法攻击和篡改。所以不难看出，消息认证是认证技术最主要的应用，它对网络通信安全具有至关重要的意义，是网络信息安全领域中非常值得关注和研究的问题。Authentication technology is a technical means to provide the identity, communication content and process credibility assurance of the communication parties in the network communication system. At present, in the fields of financial transactions, e-commerce, electronic letters, mobile phone user information confirmation and other fields, network information communication is relatively frequent, and data integrity confirmation and authenticity identification of data sources are very important security services. Therefore, the research and practice of authentication technology is an important content in the field of network information security. The application of authentication technology mainly includes identity authentication, message authentication and digital signature. Message authentication is a basic technology to verify the source and content of information, mainly to solve the integrity problem of data in the process of communication and storage, to ensure that information is not illegally attacked and tampered with. Therefore, it is not difficult to see that message authentication is the most important application of authentication technology. It is of vital significance to network communication security, and it is a problem worthy of attention and research in the field of network information security.

发明内容Contents of the invention

针对现有HDFS无法保障用户数据的安全问题，本发明公开了一种改进的AES算法，本发明还公开了一种云存储加密方法，同时还公开了一种云存储数据完整性认证的方法，本发明还公开了结合加密方法和完整性认证相对应的云存储系统。通过上述方法，对传输的数据进行处理，极大的保证了数据的安全性。Aiming at the problem that the existing HDFS cannot guarantee the security of user data, the present invention discloses an improved AES algorithm, a cloud storage encryption method, and a cloud storage data integrity authentication method. The invention also discloses a cloud storage system corresponding to the combination of the encryption method and the integrity authentication. Through the above method, the transmitted data is processed, which greatly guarantees the security of the data.

本发明的技术方案如下：Technical scheme of the present invention is as follows:

本发明公开了一种云存储加密系统，其具体包括云存储安全客户端和云存储服务器端；所述云存储安全客户端用于在客户端上定义秘钥种子，根据秘钥种子得到对称加密算法的秘钥，然后通过对称加密算法的秘钥为文件进行加密，形成加密的用户文件。之后客户端接收服务器端传输过来的会话密钥，通过该会话密钥对对称加密算法的秘钥进行加密，形成对称加密算法的秘钥的密文形式，以便于安全的为服务器端传输；所述云存储服务器端用于与服务器端进行连接，连接成功后，服务器端为客户端传输会话密钥，也就是非对称加密算法的公钥或是私钥。The invention discloses a cloud storage encryption system, which specifically includes a cloud storage security client and a cloud storage server; the cloud storage security client is used to define a secret key seed on the client, and obtain symmetric encryption according to the secret key seed The secret key of the algorithm, and then encrypt the file with the secret key of the symmetric encryption algorithm to form an encrypted user file. Afterwards, the client receives the session key transmitted from the server, and encrypts the secret key of the symmetric encryption algorithm through the session key to form the ciphertext form of the secret key of the symmetric encryption algorithm, so as to be safely transmitted to the server; The above-mentioned cloud storage server is used to connect with the server. After the connection is successful, the server transmits the session key to the client, which is the public key or private key of the asymmetric encryption algorithm.

更进一步地，上述文件对称加密算法采用改进的AES算法，非对称加密算法采用RSA算法，客户端和服务器端的连接采用Socket方法。Furthermore, the above-mentioned file symmetric encryption algorithm adopts the improved AES algorithm, the asymmetric encryption algorithm adopts the RSA algorithm, and the connection between the client and the server adopts the Socket method.

1.其中AES算法涉及到4种操作：字节代换、行移位、列混淆和轮密钥加，其特征在于：字节代换是通过S盒完成一个字节到另外一个字节的映射，将S盒用一个16x16的字节组成的矩阵来表示，通过查表即可实现该步骤。1. The AES algorithm involves 4 operations: byte substitution, row shifting, column confusion and round key addition. It is characterized in that byte substitution is completed from one byte to another through the S box For mapping, the S box is represented by a matrix composed of 16x16 bytes, and this step can be realized by looking up a table.

2.如权利要求3所述的系统，对于行移位和列混淆这两个步骤合并为一个操作步骤；设经过字节代换后的状态为2. system as claimed in claim 3, these two steps are merged into one operation step for row shifting and column confusion; Let the state after byte substitution be

经过行移位和列混淆变换后的状态为The state after row shifting and column confusion transformation is

那么，So,

矩阵中的每个元素的计算过程如下：The calculation process for each element in the matrix is as follows:

由此，将其写成一个向量变换的形式为Therefore, it can be written in the form of a vector transformation as

在此计算过程中，只涉及到与运算，运算通过左移一位实现，运算通过后再与本身进行⊕实现；运算将明文分组后的16个字节循环左移一位，运算先将明文分组后的16个字节循环左移一位，之后再与本身数据进行异或操作。In this calculation, only the and operation, The operation is implemented by shifting one bit to the left, operation through Then carry out ⊕ realization with itself; The operation rotates the 16 bytes of the plaintext grouped to the left by one bit, The operation first rotates the 16 bytes of the plaintext grouped to the left by one bit, and then performs an XOR operation with its own data.

AES加密过程涉及到4种操作：字节代换、行移位、列混淆和轮密钥加，以AES-128为例，要进行10轮的轮变换，除了最后一轮不进行列混淆以外，其余前9轮都一次进行了4个变换，本发明针对其加密过程进行优化，目的是提高AES算法的加密速度。The AES encryption process involves four operations: byte substitution, row shifting, column obfuscation, and round key addition. Taking AES-128 as an example, 10 rounds of round transformation are required, except that the last round does not perform column obfuscation. , the remaining first 9 rounds have carried out 4 transformations at a time, and the present invention optimizes the encryption process, with the aim of improving the encryption speed of the AES algorithm.

首先是对字节代换的优化，字节代换的主要功能是通过S盒完成一个字节到另外一个字节的映射，不同于固有的实现方式(由字节在GF(2^N)域中求其乘法逆并外加一个仿射变换实现)，由于该步骤是一种非线性面向字节的变换，是将一个8位二进制数据转换为另一个不同的8位二进制数据，这里要求一一对应，具体实现时，将S盒用一个16x16的置换表来表示，通过查表即可实现该步骤，避免了复杂的乘法运算。The first is the optimization of byte substitution. The main function of byte substitution is to complete the mapping from one byte to another byte through the S box, which is different from the inherent implementation (by byte in GF(2^N ) domain Find its multiplicative inverse and add an affine transformation to realize), because this step is a nonlinear byte-oriented transformation, it is to convert an 8-bit binary data into another different 8-bit binary data, here requires one by one Correspondingly, in the specific implementation, the S box is represented by a 16x16 permutation table, and this step can be realized by looking up the table, avoiding complicated multiplication operations.

本发明公开了一种云存储数据完整性认证的系统，其具体包括云存储安全客户端和云存储服务器端；所述云存储安全客户端用于在客户端上允许用户选定需要传送到服务器端的文件，使用哈希算法对该文件进行处理，以形成该文件的哈希值。之后，利用之前所得到的服务器端传输过来的会话密钥，对刚才形成的文件的哈希值进行加密，形成一个经过非对称加密算法处理过的哈希值，以保证在传输到服务器的过程中的安全性。所述云存储服务器端用于在服务器端每次在接收用户文件后，对其计算哈希值。然后通过与客户端发来的哈希值进行比较来验证消息的完整性，若验证成功，保存文件并上传至HDFS，若验证失败，则丢弃文件。The invention discloses a system for authenticating cloud storage data integrity, which specifically includes a cloud storage security client and a cloud storage server; The file on the terminal is processed by a hash algorithm to form a hash value of the file. After that, use the previously obtained session key transmitted from the server to encrypt the hash value of the file just formed to form a hash value processed by an asymmetric encryption algorithm to ensure security in . The cloud storage server end is used to calculate a hash value for each user file after the server end receives it. Then verify the integrity of the message by comparing it with the hash value sent by the client. If the verification is successful, save the file and upload it to HDFS. If the verification fails, discard the file.

本文还公开了一种结合云存储加密方案和云存储数据完整性认证方案的云存储安全系统。用户在云存储安全客户端上选定需要传送到服务器端的文件，使用哈希算法对该文件进行处理，以形成该文件的哈希值。之后，用户在云存储安全客户端上选择是否定制自己的秘钥种子，客户端根据用户的选择生成对称加密算法的秘钥，并且使用对称加密算法的秘钥对用户要上传的文件进行加密，形成一个加密的用户文件。随后，云存储安全客户端与云存储服务器端进行连接，连接成功后，服务器端为客户端传输会话秘钥。这时，云存储安全客户端接收服务器端传输过来的会话密钥，使用该会话密钥对对称加密算法的秘钥进行加密，形成对称加密算法的秘钥的密文文件，以便于安全的为服务器端传输。同时，云存储安全客户端通过该会话密钥对该文件的哈希值进行加密，形成哈希值的密文文件。此时将加密的用户文件、对称加密算法的秘钥的密文和哈希值的密文文件压缩为一个压缩包，将此压缩包通过云存储安全客户端上传到云存储服务器端。在云存储服务器端，每次在接收客户端传送过来的压缩包后，解压此压缩包，并且使用非对称加密算法的私钥或公钥对对称加密算法的秘钥的密文和哈希值的密文文件进行解密，得到对称加密算法的秘钥以及用户文件的哈希值。之后使用对称加密算法的秘钥对加密的用户文件进行解密，得到明文的文件，再使用哈希算法对明文的文件计算哈希值，通过与客户端发来的哈希值进行比较来验证消息的完整性。对两次的哈希值进行比较，若相匹配，则验证成功，保存文件并上传至HDFS，否则丢弃用户上传的文件。This paper also discloses a cloud storage security system combined with a cloud storage encryption scheme and a cloud storage data integrity authentication scheme. The user selects the file to be transmitted to the server on the cloud storage security client, and uses the hash algorithm to process the file to form the hash value of the file. After that, the user chooses whether to customize his own key seed on the cloud storage security client, and the client generates the secret key of the symmetric encryption algorithm according to the user's choice, and uses the secret key of the symmetric encryption algorithm to encrypt the file to be uploaded by the user. Form an encrypted user file. Subsequently, the cloud storage security client connects with the cloud storage server, and after the connection is successful, the server transmits the session key to the client. At this time, the cloud storage security client receives the session key transmitted from the server, uses the session key to encrypt the secret key of the symmetric encryption algorithm, and forms a ciphertext file of the secret key of the symmetric encryption algorithm, so as to provide security Server-side transfer. At the same time, the cloud storage security client encrypts the hash value of the file through the session key to form a ciphertext file of the hash value. At this time, the encrypted user file, the ciphertext of the secret key of the symmetric encryption algorithm, and the ciphertext file of the hash value are compressed into a compressed package, and the compressed package is uploaded to the cloud storage server through the cloud storage security client. On the cloud storage server side, each time after receiving the compressed package sent by the client, decompress the compressed package, and use the private key or public key of the asymmetric encryption algorithm to pair the ciphertext and hash value of the secret key of the symmetric encryption algorithm The ciphertext file is decrypted to obtain the secret key of the symmetric encryption algorithm and the hash value of the user file. Then use the secret key of the symmetric encryption algorithm to decrypt the encrypted user file to obtain the plaintext file, then use the hash algorithm to calculate the hash value of the plaintext file, and verify the message by comparing it with the hash value sent by the client integrity. Compare the two hash values, if they match, the verification is successful, save the file and upload it to HDFS, otherwise discard the file uploaded by the user.

通过采用以上的技术方案，本发明的优势在于：本方法采用基于完整性和保密性的算法来构建安全的云存储系统，一方面考虑了系统性能，对文件的加密采用了改进的AES对称加密算法，对会话密钥的加密采用了RSA非对称加密算法，并在云存储安全客户端只进行加密运算，在云存储服务器端只进行解密运算。By adopting the above technical scheme, the advantage of the present invention is that: the method adopts an algorithm based on integrity and confidentiality to construct a safe cloud storage system, on the one hand, system performance is considered, and the encryption of files adopts improved AES symmetric encryption Algorithm, the encryption of the session key adopts the RSA asymmetric encryption algorithm, and only the encryption operation is performed on the cloud storage security client side, and only the decryption operation is performed on the cloud storage server side.

附图说明Description of drawings

图1为云存储系统的结构示意图。FIG. 1 is a schematic structural diagram of a cloud storage system.

图2为云存储加密的实现流程示意图。Fig. 2 is a schematic diagram of the implementation process of cloud storage encryption.

图3为云存储完整性认证的实现流程示意图。FIG. 3 is a schematic diagram of an implementation flow of cloud storage integrity authentication.

图4为云存储系统的实现流程示意图。FIG. 4 is a schematic diagram of the implementation flow of the cloud storage system.

具体实施方式detailed description

为了使本发明的目的、技术方案及优点更加清楚明白，一下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

3.本发明公开了一种改进的AES算法，其具体内容如下所示：AES加密过程涉及到4种操作：字节代换、行移位、列混淆和轮密钥加，字节代换是通过S盒完成一个字节到另外一个字节的映射，将S盒用一个16x16的字节组成的矩阵来表示3. The present invention discloses a kind of improved AES algorithm, and its specific content is as follows: AES encryption process relates to 4 kinds of operations: byte substitution, line shift, column confusion and round key add, byte substitution It is to complete the mapping from one byte to another byte through the S box, and represent the S box with a matrix composed of 16x16 bytes

通过查表即可实现该步骤，避免了复杂的乘法运算。This step can be realized by looking up a table, avoiding complicated multiplication operations.

4.还有就是针对于行移位和列混淆这两个步骤进行了优化，本发明将这两个步骤合并为一个操作步骤，可以进一步简化实现过程。本优化的基本原理如下所示，设经过字节代换后的状态为4. In addition, the two steps of row shifting and column confusion are optimized. The present invention combines these two steps into one operation step, which can further simplify the implementation process. The basic principle of this optimization is as follows, assuming that the state after byte substitution is

经过行移位和列混淆变换后的状态为The state after row shifting and column confusion transformation is

那么，So,

矩阵中的每个元素的计算过程如下：The calculation process for each element in the matrix is as follows:

由此，将其写成一个向量变换的形式为Therefore, it can be written in the form of a vector transformation as

在此计算过程中，只涉及到与运算，运算可以通过左移一位实现，运算可以通过后再与本身进行⊕实现(运算将明文分组后的16个字节循环左移一位，运算先将明文分组后的16个字节循环左移一位，之后再与本身数据进行异或操作)。此步骤的变换合并拟称为行列变换，替换行移位和列混淆这两个步骤，实现了AES加密过程的优化。In this calculation, only the and operation, The operation can be realized by shifting one bit to the left, operation can be done by And then carry out ⊕ realization with itself ( The operation rotates the 16 bytes of the plaintext grouped to the left by one bit, The operation first rotates the 16 bytes of the plaintext grouped to the left by one bit, and then performs an XOR operation with its own data). The transformation and combination of this step is proposed to be called row-column transformation, which replaces the two steps of row shifting and column confusion, and realizes the optimization of the AES encryption process.

5.一种云存储加密系统，其具体包括云存储安全客户端和云存储服务器端；所述云存储安全客户端用于在客户端上定义秘钥种子，根据秘钥种子得到对称加密算法的秘钥，然后通过对称加密算法的秘钥为文件进行加密，形成加密的用户文件。之后客户端接收服务器端传输过来的会话密钥，通过该会话密钥对对称加密算法的秘钥进行加密，形成对称加密算法的秘钥的密文形式，以便于安全的为服务器端传输；所述云存储服务器端用于与服务器端进行连接，连接成功后，服务器端为客户端传输会话密钥，也就是非对称加密算法的公钥或是私钥。5. A cloud storage encryption system, which specifically includes a cloud storage security client and a cloud storage server; the cloud storage security client is used to define a secret key seed on the client, and obtain the symmetric encryption algorithm according to the secret key seed The secret key, and then encrypt the file with the secret key of the symmetric encryption algorithm to form an encrypted user file. Afterwards, the client receives the session key transmitted from the server, and encrypts the secret key of the symmetric encryption algorithm through the session key to form the ciphertext form of the secret key of the symmetric encryption algorithm, so as to be safely transmitted to the server; The above-mentioned cloud storage server is used to connect with the server. After the connection is successful, the server transmits the session key to the client, which is the public key or private key of the asymmetric encryption algorithm.

6.如权利要求5所述的云存储加密方法，其特征在于文件对称加密算法采用改进的AES算法，非对称加密算法采用RSA算法，客户端和服务器端的连接采用Socket方法。6. The cloud storage encryption method according to claim 5, wherein the file symmetric encryption algorithm adopts the improved AES algorithm, the asymmetric encryption algorithm adopts the RSA algorithm, and the connection between the client and the server adopts the Socket method.

7.一种云存储数据完整性认证的系统，其具体包括云存储安全客户端和云存储服务器端；所述云存储安全客户端用于在客户端上允许用户选定需要传送到服务器端的文件，使用哈希算法对该文件进行处理，以形成该文件的哈希值。之后，利用之前所得到的服务器端传输过来的会话密钥，对刚才形成的文件的哈希值进行加密，形成一个经过非对称加密算法处理过的哈希值，以保证在传输到服务器的过程中的安全性。所述云存储服务器端用于在服务器端每次在接收用户文件后，对其计算哈希值。然后通过与客户端发来的哈希值进行比较来验证消息的完整性，若验证成功，保存文件并上传至HDFS，若验证失败，则丢弃文件。7. A cloud storage data integrity authentication system, which specifically includes a cloud storage security client and a cloud storage server; the cloud storage security client is used to allow users to select files that need to be transmitted to the server on the client , process the file using a hash algorithm to form a hash value for the file. After that, use the previously obtained session key transmitted from the server to encrypt the hash value of the file just formed to form a hash value processed by an asymmetric encryption algorithm to ensure security in . The cloud storage server end is used to calculate a hash value for each user file after the server end receives it. Then verify the integrity of the message by comparing it with the hash value sent by the client. If the verification is successful, save the file and upload it to HDFS. If the verification fails, discard the file.

8.本文还公开了一种结合云存储加密方案和云存储数据完整性认证方案的云存储安全系统。用户在云存储安全客户端上选定需要传送到服务器端的文件，使用哈希算法对该文件进行处理，以形成该文件的哈希值。之后，用户在云存储安全客户端上选择是否定制自己的秘钥种子，客户端根据用户的选择生成对称加密算法的秘钥，并且使用对称加密算法的秘钥对用户要上传的文件进行加密，形成一个加密的用户文件。随后，云存储安全客户端与云存储服务器端进行连接，连接成功后，服务器端为客户端传输会话秘钥。这时，云存储安全客户端接收服务器端传输过来的会话密钥，使用该会话密钥对对称加密算法的秘钥进行加密，形成对称加密算法的秘钥的密文文件，以便于安全的为服务器端传输。同时，云存储安全客户端通过该会话密钥对该文件的哈希值进行加密，形成哈希值的密文文件。此时将加密的用户文件、对称加密算法的秘钥的密文和哈希值的密文文件压缩为一个压缩包，将此压缩包通过云存储安全客户端上传到云存储服务器端。在云存储服务器端，每次在接收客户端传送过来的压缩包后，解压此压缩包，并且使用非对称加密算法的私钥或公钥对对称加密算法的秘钥的密文和哈希值的密文文件进行解密，得到对称加密算法的秘钥以及用户文件的哈希值。之后使用对称加密算法的秘钥对加密的用户文件进行解密，得到明文的文件，再使用哈希算法对明文的文件计算哈希值，通过与客户端发来的哈希值进行比较来验证消息的完整性。对两次的哈希值进行比较，若相匹配，则验证成功，保存文件并上传至HDFS，否则丢弃用户上传的文件。8. This paper also discloses a cloud storage security system that combines a cloud storage encryption scheme and a cloud storage data integrity authentication scheme. The user selects the file to be transmitted to the server on the cloud storage security client, and uses the hash algorithm to process the file to form the hash value of the file. After that, the user chooses whether to customize his own key seed on the cloud storage security client, and the client generates the secret key of the symmetric encryption algorithm according to the user's choice, and uses the secret key of the symmetric encryption algorithm to encrypt the file to be uploaded by the user. Form an encrypted user file. Subsequently, the cloud storage security client connects with the cloud storage server, and after the connection is successful, the server transmits the session key to the client. At this time, the cloud storage security client receives the session key transmitted from the server, uses the session key to encrypt the secret key of the symmetric encryption algorithm, and forms a ciphertext file of the secret key of the symmetric encryption algorithm, so as to provide security Server-side transfer. At the same time, the cloud storage security client encrypts the hash value of the file through the session key to form a ciphertext file of the hash value. At this time, the encrypted user file, the ciphertext of the secret key of the symmetric encryption algorithm, and the ciphertext file of the hash value are compressed into a compressed package, and the compressed package is uploaded to the cloud storage server through the cloud storage security client. On the cloud storage server side, each time after receiving the compressed package sent by the client, decompress the compressed package, and use the private key or public key of the asymmetric encryption algorithm to pair the ciphertext and hash value of the secret key of the symmetric encryption algorithm The ciphertext file is decrypted to obtain the secret key of the symmetric encryption algorithm and the hash value of the user file. Then use the secret key of the symmetric encryption algorithm to decrypt the encrypted user file to obtain the plaintext file, then use the hash algorithm to calculate the hash value of the plaintext file, and verify the message by comparing it with the hash value sent by the client integrity. Compare the two hash values, if they match, the verification is successful, save the file and upload it to HDFS, otherwise discard the file uploaded by the user.

本发明中的云存储加密及完整性验证系统主要由图1所示的部分组成，详细如下：Cloud storage encryption and integrity verification system among the present invention are mainly made up of the part shown in Fig. 1, detail is as follows:

(1)云存储安全客户端：实现与云存储服务器端的对接。具备加密功能，包括明文文件的加密和会话秘钥的加密。同时还具有生成文件哈希码的功能，包括利用安全散列对明文文件的计算求值。(1) Cloud storage security client: realize the connection with the cloud storage server. It has encryption function, including the encryption of plaintext files and the encryption of session keys. At the same time, it also has the function of generating file hash codes, including the calculation and evaluation of plaintext files using secure hashes.

(2)云存储服务器端：具有存储功能，负责用户上传的数据文件的存储；具有解密功能，包括对密文文件的解密和会话秘钥的解密；具有生成文件哈希码的功能，包括利用安全散列对明文文件的计算求值；具有文件哈希码验证的功能，包括对从云存储安全客户端接收到的哈希码与云存储服务器端重新生成文件哈希码的比较。(2) Cloud storage server side: with storage function, responsible for the storage of data files uploaded by users; with decryption function, including decryption of ciphertext files and decryption of session keys; with the function of generating file hash codes, including using Secure hash calculation and evaluation of plaintext files; it has the function of file hash code verification, including the comparison of the hash code received from the cloud storage security client and the file hash code regenerated by the cloud storage server.

基于加密算法(数据加密采用的算法经常是DES和AES，随着硬件和网络的发展，DES算法被破解的可能性越来越大，而且所需的时间也越来越少。而AES算法相比于DES算法，具有更好的安全性、效率以及灵活性。对于对称加密所使用的密钥我们可以通过非对称加密的方式发送出去，虽然非对称加密更加安全，但是和对称加密比起来，它加密的速度非常慢，所以本发明还是采用对称加密算法加密消息。因此，本发明采用了改进的AES算法和RSA算法相结合的方式)和消息认证功能(消息认证常用的方法有CRC，MD5和SHA1，其中，CRC多项式是线性结构，很容易通过改变数据方式达到CRC碰撞。随着计算机运算能力的提高，MD5和SHA1找到碰撞的几率也越来越大了。因此，本发明使用更为安全的SHA256算法)的云存储系统构建完成后，用户通过云存储安全客户端向云存储服务器端发送连接请求，云存储服务器端产生用户公钥和私钥，并将会话秘钥，用于用户对对称加密算法的秘钥进行加密，以保证数据的保密性。Based on the encryption algorithm (the algorithms used for data encryption are often DES and AES, with the development of hardware and networks, the DES algorithm is more and more likely to be cracked, and the time required is less and less. The AES algorithm is relatively Compared with the DES algorithm, it has better security, efficiency and flexibility. For the key used in symmetric encryption, we can send it through asymmetric encryption. Although asymmetric encryption is more secure, compared with symmetric encryption, The speed of its encryption is very slow, so the present invention still adopts symmetric encryption algorithm to encrypt message.Therefore, the present invention has adopted the improved AES algorithm and the mode that RSA algorithm combines) and message authentication function (the method commonly used for message authentication has CRC, MD5 And SHA1, wherein, CRC polynomial is linear structure, is easy to reach CRC collision by changing data mode. Along with the raising of computing power of computer, MD5 and SHA1 find the probability of collision also more and more big.Therefore, the present invention uses more After the cloud storage system with secure SHA256 algorithm is built, the user sends a connection request to the cloud storage server through the cloud storage security client, and the cloud storage server generates the user's public key and private key, and uses the session key for the user Encrypt the secret key of the symmetric encryption algorithm to ensure the confidentiality of the data.

用户进行云存储加密的过程如图2所示，其详细步骤如下：The process of cloud storage encryption by users is shown in Figure 2, and the detailed steps are as follows:

步骤一：用户在云存储安全客户端上选择是否定制自己的秘钥种子(使用自己定义的密码形成秘钥)，使用自定的秘钥种子，生成唯一的秘钥，若不使用秘钥种子，每次上传文件都生成一个随机的秘钥。Step 1: The user chooses on the cloud storage security client whether to customize his own secret key seed (using a self-defined password to form a secret key), and uses the self-defined secret key seed to generate a unique secret key. If the secret key seed is not used , a random secret key is generated each time a file is uploaded.

步骤二：云存储安全客户端根据用户的选择生成改进的对称加密算法AES的秘钥。Step 2: The cloud storage security client generates the secret key of the improved symmetric encryption algorithm AES according to the user's choice.

步骤三：云存储安全客户端使用改进的AES算法的秘钥对用户要上传的文件进行加密，形成一个加密的用户文件。Step 3: The cloud storage security client uses the secret key of the improved AES algorithm to encrypt the file to be uploaded by the user to form an encrypted user file.

步骤四：云存储安全客户端与云存储服务器端使用Socket方法进行连接，连接成功后，服务器端为客户端传输会话秘钥，也就是非对称加密算法RSA的公钥或是私钥。Step 4: The cloud storage security client connects with the cloud storage server using the Socket method. After the connection is successful, the server transmits the session key to the client, which is the public key or private key of the asymmetric encryption algorithm RSA.

步骤五：云存储安全客户端接收服务器端传输过来的会话密钥。Step 5: The cloud storage security client receives the session key transmitted from the server.

步骤六：云存储安全客户端通过该会话密钥对对称加密算法的秘钥(改进的AES的秘钥)进行加密，形成对称加密算法的秘钥的密文形式，以便于安全的为服务器端传输。Step 6: The cloud storage security client encrypts the secret key of the symmetric encryption algorithm (the secret key of improved AES) through the session key to form the ciphertext form of the secret key of the symmetric encryption algorithm, so as to securely provide the server-side transmission.

用户进行云存储完整性认证的过程如图3所示，其详细步骤如下：The process of the user's cloud storage integrity authentication is shown in Figure 3, and the detailed steps are as follows:

步骤一：用户在云存储安全客户端上选定需要传送到服务器端的文件，使用哈希算法SHA256对该文件进行处理，以形成该文件的哈希值。Step 1: The user selects a file on the cloud storage security client to be transmitted to the server, and uses the hash algorithm SHA256 to process the file to form a hash value of the file.

步骤二：使用在文件加密过程中所得到的会话密钥(非对称加密算法RSA的公钥或是私钥)，对上一步生成的哈希值进行加密，形成一个经过非对称加密算法处理过的哈希值，以保证在传输到服务器的过程中的安全性。Step 2: Use the session key (public key or private key of the asymmetric encryption algorithm RSA) obtained during the file encryption process to encrypt the hash value generated in the previous step to form an asymmetric encryption algorithm processed hash value to ensure security during transmission to the server.

步骤三：在云存储服务器端，每次在接收客户端传送过来的用户文件后，使用哈希算法SHA256对其计算哈希值。Step 3: On the cloud storage server side, each time after receiving the user file sent by the client, use the hash algorithm SHA256 to calculate the hash value.

步骤四：通过与客户端发来的哈希值进行比较来验证消息的完整性。Step 4: Verify the integrity of the message by comparing it with the hash value sent by the client.

步骤五：对两次的哈希值进行比较，若相匹配，则验证成功，保存文件并上传至HDFS，否则丢弃用户上传的文件。Step 5: Compare the two hash values. If they match, the verification is successful, save the file and upload it to HDFS, otherwise discard the file uploaded by the user.

本文还公开了一种结合云存储加密方案和云存储数据完整性认证方案的云存储安全系统，其存储过程如图4所示，详细步骤如下：This paper also discloses a cloud storage security system that combines a cloud storage encryption scheme and a cloud storage data integrity authentication scheme. The storage process is shown in Figure 4, and the detailed steps are as follows:

步骤一：用户在云存储安全客户端上选定需要传送到服务器端的文件，使用哈希算法SHA256对该文件进行处理，以形成该文件的哈希值。Step 1: The user selects a file on the cloud storage security client to be transmitted to the server, and uses the hash algorithm SHA256 to process the file to form a hash value of the file.

步骤二：用户在云存储安全客户端上选择是否定制自己的秘钥种子(使用自己定义的密码形成秘钥)，使用自定的秘钥种子，生成唯一的秘钥，若不使用秘钥种子，每次上传文件都生成一个随机的秘钥。Step 2: The user chooses on the cloud storage security client whether to customize his own secret key seed (using a self-defined password to form a secret key), use the self-defined secret key seed to generate a unique secret key, if not use the secret key seed , a random secret key is generated each time a file is uploaded.

步骤三：云存储安全客户端根据用户的选择生成对称加密算法改进的AES的秘钥。Step 3: The cloud storage security client generates a secret key of the improved AES of the symmetric encryption algorithm according to the user's choice.

步骤四：云存储安全客户端使用改进的AES算法的秘钥对用户要上传的文件进行加密，形成一个加密的用户文件。Step 4: The cloud storage security client uses the secret key of the improved AES algorithm to encrypt the file to be uploaded by the user to form an encrypted user file.

步骤五：云存储安全客户端与云存储服务器端使用Socket方法进行连接，连接成功后，服务器端为客户端传输会话秘钥，也就是非对称加密算法RSA的公钥或是私钥。Step 5: The cloud storage security client and the cloud storage server use the Socket method to connect. After the connection is successful, the server transmits the session key to the client, which is the public key or private key of the asymmetric encryption algorithm RSA.

步骤六：云存储安全客户端接收服务器端传输过来的会话密钥。Step 6: The cloud storage security client receives the session key transmitted from the server.

步骤七：云存储安全客户端通过该会话密钥对对称加密算法的秘钥(改进的AES的秘钥)进行加密，形成对称加密算法的秘钥的密文文件，以便于安全的为服务器端传输。Step 7: The cloud storage security client encrypts the secret key of the symmetric encryption algorithm (the secret key of improved AES) through the session key to form a ciphertext file of the secret key of the symmetric encryption algorithm, so as to securely provide the server-side transmission.

步骤八：云存储安全客户端通过该会话密钥对该文件的哈希值进行加密，形成哈希值的密文文件。Step 8: The cloud storage security client encrypts the hash value of the file through the session key to form a ciphertext file of the hash value.

步骤九：将加密的用户文件、对称加密算法的秘钥的密文和哈希值的密文文件压缩为一个压缩包。Step 9: Compress the encrypted user file, the ciphertext of the secret key of the symmetric encryption algorithm, and the ciphertext of the hash value into a compressed package.

步骤十：将此压缩包通过云存储安全客户端上传到云存储服务器端。Step 10: Upload the compressed package to the cloud storage server through the cloud storage security client.

步骤十一：在云存储服务器端，每次在接收客户端传送过来的压缩包后，解压此压缩包，并且使用RSA的私钥或公钥对对称加密算法的秘钥的密文和哈希值的密文文件进行解密，得到对称加密算法的秘钥以及用户文件的哈希值。Step 11: On the cloud storage server side, each time after receiving the compressed package sent by the client, decompress the compressed package, and use the RSA private key or public key to pair the ciphertext and hash of the secret key of the symmetric encryption algorithm The ciphertext file of the value is decrypted to obtain the secret key of the symmetric encryption algorithm and the hash value of the user file.

步骤十二：使用对称加密算法的秘钥对加密的用户文件进行解密，得到明文的文件。Step 12: Use the secret key of the symmetric encryption algorithm to decrypt the encrypted user file to obtain the plaintext file.

步骤十三：使用哈希算法SHA256对明文的文件计算哈希值。Step 13: Use the hash algorithm SHA256 to calculate the hash value for the plaintext file.

步骤十四：通过与客户端发来的哈希值进行比较来验证消息的完整性。Step 14: Verify the integrity of the message by comparing it with the hash value sent by the client.

步骤十五：对两次的哈希值进行比较，若相匹配，则验证成功，保存文件并上传至HDFS，否则丢弃用户上传的文件。Step 15: Compare the two hash values, if they match, the verification is successful, save the file and upload it to HDFS, otherwise discard the file uploaded by the user.

上述的实施例中所给出的系数和参数，是提供给本领域的技术人员来实现或使用发明的，发明并不限定仅取前述公开的数值，在不脱离发明的思想的情况下，本领域的技术人员可以对上述实施例作出种种修改或调整，因而发明的保护范围并不被上述实施例所限，而应该是符合权利要求书提到的创新性特征的最大范围。The coefficients and parameters given in the above-mentioned embodiments are provided for those skilled in the art to implement or use the invention. The invention is not limited to only take the aforementioned disclosed values. Without departing from the idea of the invention, this Those skilled in the art can make various modifications or adjustments to the above-mentioned embodiments, so the protection scope of the invention is not limited by the above-mentioned embodiments, but should meet the maximum scope of the innovative features mentioned in the claims.

Claims (6)

1. a kind of cloud storage encryption system, including cloud storage security client and cloud storage service device end；It is characterized in that：It is describedCloud storage security client is used to define key seed on the client, obtains the secret of symmetric encipherment algorithm according to key seedKey, is then encrypted for file by the key of symmetric encipherment algorithm, forms the user file of encryption；Client is received afterwardsThe session key that server end is transmitted, is encrypted to the key of symmetric encipherment algorithm by the session key, and it is right to be formedClaim the ciphertext form of the key of AES, transmit for server end in order to safety；The cloud storage service device end is used forBe attached with server end, after successful connection, server end be client transmissions session key, that is, asymmetric encryption calculateThe public key or private key of method.

2. the system as claimed in claim 1, it is characterised in that：Symmetric encipherment algorithm adopts aes algorithm, rivest, shamir, adelmanUsing RSA Algorithm, client and server end is connected by Socket methods.

3. system as claimed in claim 2, wherein aes algorithm are related to 4 kinds of operations：Byte substitution, row displacement, row obscure andInvAddRoundKey, it is characterised in that：Byte substitution is to complete a byte to the mapping of another byte by S boxes, and S boxes are usedThe matrix of the byte composition of one 16x16 representing, by tabling look-up step of be capable of achieving.

4. system as claimed in claim 3, for row displacement and row are obscured the two steps and merge into an operating procedure；IfState after byte substitution is

$S=\left(\begin{array}{cccc}{s}_0& s_1& s_2& s_3\\ s_4& s_5& s_6& s_7\\ s_8& s_9& s_{10}& s_{11}\\ s_{12}& s_{13}& s_{14}& s_{15}\end{array}\right)$

Through row displacement and the state arranged after obscuring conversion it is

$S^{\′}=\left(\begin{array}{cccc}{s_0}^{\′}& {s_1}^{\′}& {s_2}^{\′}& {s_3}^{\′}\\ {s_4}^{\′}& {s_5}^{\′}& {s_6}^{\′}& {s_7}^{\′}\\ {s_8}^{\′}& {s_9}^{\′}& {s_{10}}^{\′}& {s_{11}}^{\′}\\ {s_{12}}^{\′}& {s_{13}}^{\′}& {s_{14}}^{\′}& {s_{15}}^{\′}\end{array}\right)$

So,

The calculating process of each element in matrix is as follows：

$\begin{array}{cc}{s_0}^{\′}=02s_0+03s_5+01s_{10}+01s_{15}& {s_1}^{\′}=02s_1+03s_6+01s_{11}+01s_{12}\\ {s_2}^{\′}=02s_2+03s_7+01s_8+01s_{13}& {s_3}^{\′}=02s_3+03s_4+01s_9+01s_{14}\\ {s_4}^{\′}=01s_0+02s_5+03s_{10}+01s_{15}& {s_5}^{\′}=01s_1+02s_6+03s_{11}+01s_{12}\\ .& .\\ .& .\\ .& .\\ {s_{14}}^{\′}=03s_2+01s_7+01s_8+02s_{13}& {s_{15}}^{\′}=03s_3+01s_4+01s_9+02s_{14}\end{array}$

Thus, the form for being write as a vector transformation is

$\left(\begin{array}{c}{s_0}^{\′}\\ {s_1}^{\′}\\ {s_2}^{\′}\\ {s_3}^{\′}\\ {s_4}^{\′}\\ {s_5}^{\′}\\ {s_6}^{\′}\\ {s_7}^{\′}\\ {s_8}^{\′}\\ {s_9}^{\′}\\ {s_{10}}^{\′}\\ {s_{11}}^{\′}\\ {s_{12}}^{\′}\\ {s_{13}}^{\′}\\ {s_{14}}^{\′}\\ {s_{15}}^{\′}\end{array}\right)=\left(\begin{array}{cccccccccccccccc}02& 00& 00& 00& 00& 03& 00& 00& 00& 00& 01& 00& 00& 00& 00& 01\\ 00& 02& 00& 00& 00& 00& 03& 00& 00& 00& 00& 01& 01& 00& 00& 00\\ 00& 00& 02& 00& 00& 00& 00& 03& 01& 00& 00& 00& 00& 01& 00& 00\\ 00& 00& 00& 02& 03& 00& 00& 00& 00& 01& 00& 00& 00& 00& 01& 00\\ 01& 00& 00& 00& 00& 02& 00& 00& 00& 00& 03& 00& 00& 00& 00& 01\\ 00& 01& 00& 00& 00& 00& 02& 00& 00& 00& 00& 03& 01& 00& 00& 00\\ 00& 00& 01& 00& 00& 00& 00& 02& 03& 00& 00& 00& 00& 01& 00& 00\\ 00& 00& 00& 01& 02& 00& 00& 00& 00& 03& 00& 00& 00& 00& 01& 00\\ 01& 00& 00& 00& 00& 01& 00& 00& 00& 00& 02& 00& 00& 00& 00& 03\\ 00& 01& 00& 00& 00& 00& 01& 00& 00& 00& 00& 02& 03& 00& 00& 00\\ 00& 00& 01& 00& 00& 00& 00& 01& 02& 00& 00& 00& 00& 03& 00& 00\\ 00& 00& 00& 01& 01& 00& 00& 00& 00& 02& 00& 00& 00& 00& 03& 00\\ 03& 00& 00& 00& 00& 01& 00& 00& 00& 00& 01& 00& 00& 00& 00& 02\\ 00& 03& 00& 00& 00& 00& 01& 00& 00& 00& 00& 01& 02& 00& 00& 00\\ 00& 00& 03& 00& 00& 00& 00& 01& 01& 00& 00& 00& 00& 02& 00& 00\\ 00& 00& 00& 03& 01& 00& 00& 00& 00& 01& 00& 00& 00& 00& 02& 00\end{array}\right)\left(\begin{array}{c}{s}_0\\ s_1\\ s_2\\ s_3\\ s_4\\ s_5\\ s_6\\ s_7\\ s_8\\ s_9\\ s_{10}\\ s_{11}\\ s_{12}\\ s_{13}\\ s_{14}\\ s_{15}\end{array}\right)$

In this calculating process, relate only toWithComputing,Computing by moving to left a realization,Computing passes throughCarry out with itself again afterwardsRealize；Computing by clear packets after 16 byte cycles move to left one,Computing first will be in plain text16 byte cycles after packet move to left one, carry out xor operation with data itself again afterwards.

5. a kind of system of cloud storage data integrity certification, including cloud storage security client and cloud storage service device end；ItsIt is characterised by：The cloud storage security client is used for permission user on the client to be selected needs the text for being sent to server endPart, is processed to this document using hash algorithm, to form the cryptographic Hash of this document；Afterwards, clothes resulting before are utilizedThe session key that business device end transmits, the cryptographic Hash of the file to being formed just now are encrypted, and form one through asymmetricThe cryptographic Hash that AES was processed；The cloud storage service device end in server end every time after receive user file,Cryptographic Hash is calculated to which；Then the cryptographic Hash by sending with client is compared to the integrity for verifying message, if checkingSuccess, preserves file and is uploaded to HDFS, if authentication failed, abandon file.

6. a kind of cloud storage security system, it is characterised in that：User selectes needs on cloud storage security client and is sent to clothesThe file at business device end, is processed to this document using hash algorithm, to form the cryptographic Hash of this document；Afterwards, user is in cloudChoose whether to customize the key seed of oneself on storage security client, client generates symmetric cryptography according to the selection of user and calculatesThe key of method, and be encrypted using the secret key pair user of symmetric encipherment algorithm file to be uploaded, form an encryptionUser file；Subsequently, cloud storage security client is attached with cloud storage service device end, and after successful connection, server end isClient transmissions session key；At this moment, the session key that cloud storage security client the reception server end transmits, using shouldSession key is encrypted to the key of symmetric encipherment algorithm, forms the cryptograph files of the key of symmetric encipherment algorithm；Meanwhile, cloudStorage security client is encrypted to the cryptographic Hash of this document by the session key, forms the cryptograph files of cryptographic Hash；ThisWhen by the compression of cryptograph files boil down to one of the user file, the ciphertext of the key of symmetric encipherment algorithm and cryptographic Hash of encryptionThis compressed package is uploaded to cloud storage service device end by cloud storage security client by bag；At cloud storage service device end, exist every timeAfter receiving the compressed package that client sends, this compressed package is decompressed, and using the private key or public key of rivest, shamir, adelmanThe cryptograph files of the ciphertext and cryptographic Hash of the key of symmetric encipherment algorithm are decrypted, obtain the key of symmetric encipherment algorithm withAnd the cryptographic Hash of user file；The user file encrypted using the secret key pair of symmetric encipherment algorithm afterwards is decrypted, and obtains brightThe file of text, is reused hash algorithm and calculates cryptographic Hash to the file of plaintext, compared by the cryptographic Hash sent with clientRelatively verifying the integrity of message；Cryptographic Hash twice is compared, if matching, is proved to be successful, preserve file and onHDFS is reached, the file that user uploads otherwise is abandoned.