技术领域Technical Field
本申请涉及数据安全技术领域，尤其是涉及到一种基于注意力机制的联邦学习方法、客户端、服务器和系统。The present application relates to the field of data security technology, and in particular to a federated learning method, client, server and system based on an attention mechanism.
背景技术Background technique
联邦机器学习(Federated machine learning/Federated Learning)，又名联邦学习，联合学习，联盟学习。联邦学习是一种分布式机器学习技术，在不集中各参与方各自对应的样本数据的前提下，使各参与方能够协同训练并共享全局网络模型，以达到全局优化的目的。联邦学习的数据安全性问题是联邦学习应用、发展过程中的关键因素，它已引起了国内外政府、工业界和学术界的广泛关注和重视。然而，由于联邦学习分布式的特点，恶意客户端会试图通过向训练数据中注入恶意样本来修改模型的行为，从而导致全局模型性能降低，甚至是系统崩溃，严重影响系统的预测结果、整体性能和可信度。Federated machine learning (Federated machine learning/Federated Learning), also known as federated learning, joint learning, and alliance learning. Federated learning is a distributed machine learning technology that enables all participants to collaboratively train and share global network models without centralizing their corresponding sample data to achieve the goal of global optimization. The data security issue of federated learning is a key factor in the application and development of federated learning, and it has attracted widespread attention and attention from governments, industries, and academia at home and abroad. However, due to the distributed characteristics of federated learning, malicious clients will attempt to modify the behavior of the model by injecting malicious samples into the training data, thereby reducing the performance of the global model or even causing the system to crash, seriously affecting the prediction results, overall performance, and credibility of the system.
相关技术中，利用拜占庭鲁棒的异步联邦学习框架来抵御投毒攻击，但其在明文上进行分析，增加了客户端隐私泄露的风险，并且丢失了部分客户端模型更新，易导致全局模型精度的损失。In related technologies, a Byzantine robust asynchronous federated learning framework is used to resist poisoning attacks, but it performs analysis on plain text, which increases the risk of client privacy leakage and loses some client model updates, which can easily lead to loss of global model accuracy.
发明内容Summary of the invention
有鉴于此，本申请提供了一种基于注意力机制的联邦学习方法、客户端、服务器和系统，能够在面向恶意环境下训练高质量的全局模型，可有效抵抗来自恶意客户端的投毒攻击且不影响模型精度。In view of this, the present application provides a federated learning method, client, server and system based on an attention mechanism, which can train high-quality global models in malicious environments, effectively resist poisoning attacks from malicious clients without affecting the model accuracy.
根据本申请的第一方面实施例，提供了一种基于注意力机制的联邦学习方法，适用于聚合服务器，该方法包括：According to the first aspect of the present application, a federated learning method based on an attention mechanism is provided, which is applicable to an aggregation server, and the method includes:
将第t-1轮的第一密文全局模型发送至客户端，其中，所述第一密文全局模型为采用CKKS同态加密算法依据第一公钥加密的全局模型，以使所述客户端依据第t-1轮的所述第一密文全局模型解密后的第t-1轮的所述全局模型和本地数据确定所述客户端的梯度信息；Sending the first ciphertext global model of the t-1th round to the client, wherein the first ciphertext global model is a global model encrypted by the first public key using the CKKS homomorphic encryption algorithm, so that the client determines the gradient information of the client according to the global model of the t-1th round decrypted by the first ciphertext global model of the t-1th round and the local data;
接收所述客户端发送的密文梯度信息，其中，所述密文梯度信息为采用CKKS同态加密算法依据第二公钥对加密的所述梯度信息；Receive the ciphertext gradient information sent by the client, wherein the ciphertext gradient information is the gradient information encrypted by using the CKKS homomorphic encryption algorithm according to the second public key pair;
将所述密文梯度信息发送至协作服务器，以使所述协作服务器依据所述梯度信息和客户端的根梯度信息，计算注意力权重；Sending the ciphertext gradient information to the collaboration server, so that the collaboration server calculates the attention weight according to the gradient information and the root gradient information of the client;
接收所述协作服务器发送的密文权重，其中，所述密文权重为采用CKKS同态加密算法依据所述第二公钥加密的所述注意力权重；Receiving the ciphertext weight sent by the collaboration server, wherein the ciphertext weight is the attention weight encrypted according to the second public key using the CKKS homomorphic encryption algorithm;
依据所述密文权重和所述密文梯度信息，对第t-1轮的所述第一密文全局模型进行更新，形成第t轮的所述第一密文全局模型。The first ciphertext global model of the t-1th round is updated according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the tth round.
可选地，所述依据所述密文权重和所述密文梯度信息，对第t-1轮的所述第一密文全局模型进行更新，包括：Optionally, updating the first ciphertext global model of the t-1th round according to the ciphertext weight and the ciphertext gradient information includes:
依据所述密文权重聚合所述密文梯度信息，得到第t轮的第二密文全局模型，其中，所述第二密文全局模型为采用CKKS同态加密算法依据所述第二公钥加密的全局模型；Aggregating the ciphertext gradient information according to the ciphertext weight to obtain a second ciphertext global model of the tth round, wherein the second ciphertext global model is a global model encrypted according to the second public key using the CKKS homomorphic encryption algorithm;
将第t轮的所述第二密文全局模型发送至协作服务器，以使所述协作服务器依据所述第一公钥对通过所述第二公钥绑定的第二私钥解密后得到的全局模型进行加密处理，得到第t轮的所述第一密文全局模型；Sending the second ciphertext global model of the tth round to the collaboration server, so that the collaboration server encrypts the global model obtained by decrypting the second private key bound to the second public key according to the first public key, to obtain the first ciphertext global model of the tth round;
接收所述协作服务器发送的第t轮的所述第一密文全局模型。Receive the first ciphertext global model of the tth round sent by the collaborative server.
可选地，所述将所述密文梯度信息发送至协作服务器之前，所述方法还包括：Optionally, before sending the ciphertext gradient information to the collaboration server, the method further includes:
依据所述第二公钥生成第一密文噪声值；generating a first ciphertext noise value according to the second public key;
将所述第一密文噪声值添加至所述密文梯度信息，以所述密文梯度信息进行扰动处理。The first ciphertext noise value is added to the ciphertext gradient information to perform disturbance processing on the ciphertext gradient information.
可选地，所述将第t轮的所述第二密文全局模型发送至协作服务器之前，所述方法还包括：Optionally, before sending the second ciphertext global model of the tth round to the collaboration server, the method further includes:
依据所述第二公钥生成第二密文噪声值；generating a second ciphertext noise value according to the second public key;
将所述第二密文噪声值添加至第t轮的所述第二密文全局模型，以进行第t轮的所述第二密文全局模型的扰动处理。The second ciphertext noise value is added to the second ciphertext global model of the t-th round to perform perturbation processing on the second ciphertext global model of the t-th round.
可选地，所述接收所述协作服务器发送的第t轮的所述第一密文全局模型之后，所述方法还包括：Optionally, after receiving the first ciphertext global model of the tth round sent by the collaboration server, the method further includes:
若第t轮的所述第一密文全局模型添加有所述第二密文噪声值，对所述第t轮的所述第一密文全局模型进行去噪处理。If the first ciphertext global model of the t-th round is added with the second ciphertext noise value, denoising is performed on the first ciphertext global model of the t-th round.
根据本申请的第二方面实施例，提供了一种基于注意力机制的联邦学习方法，适用于协作服务器，该方法包括：According to a second aspect of the present application, a federated learning method based on an attention mechanism is provided, which is applicable to a collaborative server, and the method includes:
接收聚合服务器发送的密文梯度信息；Receive the ciphertext gradient information sent by the aggregation server;
依据第二公钥绑定的第二私钥对所述密文梯度信息进行解密处理，得到梯度信息；Decrypting the ciphertext gradient information according to the second private key bound to the second public key to obtain gradient information;
依据所述梯度信息和客户端的根梯度信息，计算注意力权重；Calculating attention weights based on the gradient information and the root gradient information of the client;
依据所述第二公钥对所述注意力权重进行加密处理，得到密文权重；Encrypting the attention weight according to the second public key to obtain a ciphertext weight;
将所述密文权重发送至所述聚合服务器，以使所述聚合服务器依据所述密文权重和所述密文梯度信息，对第t-1轮的第一密文全局模型进行更新，形成第t轮的所述第一密文全局模型。The ciphertext weight is sent to the aggregation server, so that the aggregation server updates the first ciphertext global model of the t-1th round according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the tth round.
可选地，所述依据所述梯度信息和客户端的根梯度信息，计算注意力权重，包括：Optionally, calculating the attention weight according to the gradient information and the root gradient information of the client includes:
获取客户端对应的根数据，并对所述根数据进行训练，得到所述客户端的根梯度信息；Obtain root data corresponding to the client, and train the root data to obtain root gradient information of the client;
依据同一客户端对应的所述梯度信息和所述根梯度信息进行点积计算，确定所述客户端的缩放点积分数；Performing dot product calculation according to the gradient information and the root gradient information corresponding to the same client to determine the scaling dot product score of the client;
计算所述客户端的缩放点积分数在所述协作服务器关联的至少一个客户端的总缩放点积分数的占比，作为所述注意力权重。The proportion of the scaling point score of the client to the total scaling point score of at least one client associated with the collaboration server is calculated as the attention weight.
可选地，基于注意力机制的联邦学习方法还包括：Optionally, the attention mechanism-based federated learning method also includes:
接收所述聚合服务器发送的第t轮的第二密文全局模型；Receiving a second ciphertext global model of the tth round sent by the aggregation server;
依据所述第二私钥对第t轮的所述第二密文全局模型进行解密处理，得到第t轮的所述全局模型；Decrypting the second ciphertext global model of the tth round according to the second private key to obtain the global model of the tth round;
依据第一公钥对第t轮的所述全局模型进行加密处理，得到第t轮的所述第一密文全局模型；Encrypting the global model of the tth round according to the first public key to obtain the first ciphertext global model of the tth round;
将第t轮的所述第一密文全局模型发送至所述聚合服务器；Sending the first ciphertext global model of the tth round to the aggregation server;
其中，采用CKKS同态加密算法进行加密处理。Among them, CKKS homomorphic encryption algorithm is used for encryption processing.
根据本申请的第三方面实施例，提供了一种基于注意力机制的联邦学习方法，适用于客户端，该方法包括：According to a third aspect of the present application, a federated learning method based on an attention mechanism is provided, which is applicable to a client, and the method includes:
接收聚合服务器发送的第t-1轮的第一密文全局模型，其中，所述第一密文全局模型为采用CKKS同态加密算法依据第一公钥加密的全局模型；Receive a first ciphertext global model of round t-1 sent by the aggregation server, wherein the first ciphertext global model is a global model encrypted by using a CKKS homomorphic encryption algorithm according to a first public key;
依据所述第一公钥绑定的第一私钥对第t-1轮的所述第一密文全局模型进行解密处理，得到第t-1轮的所述全局模型；Decrypting the first ciphertext global model of the t-1th round according to the first private key bound to the first public key to obtain the global model of the t-1th round;
依据第t-1轮的所述全局模型对本地数据进行迭代训练，确定第t轮的梯度信息；Iteratively train the local data according to the global model of the t-1th round to determine the gradient information of the tth round;
依据第二公钥对所述梯度信息进行加密处理，得到密文梯度信息；Encrypting the gradient information according to the second public key to obtain ciphertext gradient information;
将所述密文梯度信息发送至所述聚合服务器，以使所述聚合服务器依据密文权重和所述密文梯度信息，对第t-1轮的所述第一密文全局模型进行更新，形成第t轮的所述第一密文全局模型，其中，所述密文权重由协作服务器依据对注意力权重的加密处理得到，所述注意力权重依据所述梯度信息计算得到。The ciphertext gradient information is sent to the aggregation server, so that the aggregation server updates the first ciphertext global model of the t-1th round according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the tth round, wherein the ciphertext weight is obtained by the collaborative server based on the encryption processing of the attention weight, and the attention weight is calculated based on the gradient information.
根据本申请第四方面实施例，提供了一种服务器，包括存储介质、处理器及存储在存储介质上并可在处理器上运行的计算机程序，处理器执行程序时实现上述基于注意力机制的联邦学习方法的步骤。According to the fourth aspect embodiment of the present application, a server is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the program, the steps of the above-mentioned federated learning method based on the attention mechanism are implemented.
根据本申请第五方面实施例，提供了一种客户端，包括存储介质、处理器及存储在存储介质上并可在处理器上运行的计算机程序，处理器执行程序时实现上述基于注意力机制的联邦学习方法的步骤。According to the fifth aspect embodiment of the present application, a client is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the program, the steps of the above-mentioned federated learning method based on the attention mechanism are implemented.
根据本申请第六方面实施例，提供了一种联邦学习系统，包括：上述实施例提供的服务器和客户端。According to the sixth aspect of the present application, a federated learning system is provided, including: the server and client provided in the above embodiment.
根据本申请第七方面实施例，提供了可读存储介质，其上存储有程序或指令，程序或指令被处理器执行时实现上述基于注意力机制的联邦学习方法的步骤。According to the seventh aspect embodiment of the present application, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above-mentioned federated learning method based on the attention mechanism are implemented.
借由上述技术方案，在本地客户端基于前一轮得到的全局模型进行本地训练，并将训练好的梯度信息加密算法进行加密，并将加密后的密文梯度信息上传至聚合服务器。聚合服务器将加密后的密文梯度信息转发给协作服务器。协作服务器保留一部分小而干净的可信任的根数据，并基于缩放点击注意力机制对客户端上传的密文梯度进行权重值计算，再将基于注意力机制得到的注意力权重加密后返回聚合服务器。由聚合服务器利用密文权重和所述密文梯度信息完成加权聚合，以完成当前轮次全局模型的更新。一方面，使用缩放点击注意力机制对客户端上传的密文梯度进行权重计算和密文梯度信息聚合，便于识别出梯度信息是否存在恶意攻击，并降低可能存在恶意数据对于模型更新的重要程度，实现高效且安全的联邦学习过程。又一方面，在联邦学习过程中，采用轻量级CKKS同态加密算法对客户端隐私数据的梯度进行加密保护，以确保恶意客户端不能推断成员上传恶意梯度，保证在恶意环境下可有效抵抗来自恶意客户端的投毒攻击且不影响模型精度，同时提供对客户端数据的增强型隐私保护，实现高效且安全的联邦学习过程。再一方面，由协作服务器进行注意力机制的相关运算，无需双云服务器针对同一安全协议同时工作，降低了通信开销。By means of the above technical solution, local training is performed on the local client based on the global model obtained in the previous round, and the trained gradient information encryption algorithm is used to encrypt the trained gradient information, and the encrypted ciphertext gradient information is uploaded to the aggregation server. The aggregation server forwards the encrypted ciphertext gradient information to the collaboration server. The collaboration server retains a small, clean and trusted portion of the root data, and calculates the weight value of the ciphertext gradient uploaded by the client based on the zoom-click attention mechanism, and then encrypts the attention weight obtained based on the attention mechanism and returns it to the aggregation server. The aggregation server uses the ciphertext weight and the ciphertext gradient information to complete the update of the global model of the current round. On the one hand, the weight calculation and ciphertext gradient information aggregation of the ciphertext gradient uploaded by the client are performed using the zoom-click attention mechanism, which facilitates the identification of whether the gradient information is maliciously attacked, and reduces the importance of possible malicious data for model updates, thereby achieving an efficient and secure federated learning process. On the other hand, during the federated learning process, the lightweight CKKS homomorphic encryption algorithm is used to encrypt and protect the gradient of the client's private data to ensure that malicious clients cannot infer that members upload malicious gradients, and to ensure that poisoning attacks from malicious clients can be effectively resisted in a malicious environment without affecting the accuracy of the model. At the same time, enhanced privacy protection for client data is provided to achieve an efficient and secure federated learning process. On the other hand, the collaborative server performs related operations of the attention mechanism, and there is no need for dual cloud servers to work simultaneously for the same security protocol, which reduces communication overhead.
上述说明仅是本申请技术方案的概述，为了能够更清楚了解本申请的技术手段，而可依照说明书的内容予以实施，并且为了让本申请的上述和其它目的、特征和优点能够更明显易懂，以下特举本申请的具体实施方式。The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
图1示出了本申请实施例提供的基于注意力机制的联邦学习方法的流程示意图；FIG1 is a schematic diagram showing a flow chart of a federated learning method based on an attention mechanism provided in an embodiment of the present application;
图2示出了本申请具体实施例提供的基于注意力机制的联邦学习方法的流程示意图；FIG2 shows a schematic diagram of a process flow of a federated learning method based on an attention mechanism provided in a specific embodiment of the present application;
图3为根据本申请实施例提供的联邦学习系统的结构示意图。FIG3 is a schematic diagram of the structure of a federated learning system provided according to an embodiment of the present application.
具体实施方式Detailed ways
下文中将参考附图并结合实施例来详细说明本申请。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本申请的限制。The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“相接”到另一元件时，它可以直接连接或相接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“相接”可以包括无线连接或无线稠接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。Those skilled in the art will appreciate that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof. It should be understood that when we refer to an element as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, or there may be intermediate elements. In addition, the "connection" or "connection" used herein may include wireless connection or wireless fusion. The term "and/or" used herein includes all or any unit and all combinations of one or more associated listed items.
现在，将参照附图更详细地描述根据本申请的示例性实施例。然而，这些示例性实施例可以多种不同的形式来实施，并且不应当被解释为只限于这里所阐述的实施例。应当理解的是，提供这些实施例是为了使得本申请的公开彻底且完整，并且将这些示例性实施例的构思充分传达给本领域普通技术人员。Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present application thorough and complete, and to fully convey the concepts of these exemplary embodiments to those of ordinary skill in the art.
在本实施例中提供了一种基于注意力机制的联邦学习方法，以联邦学习系统为例进行说明，该系统包括至少一个客户端、聚合服务器和协作服务器，至少一个客户端分别与聚合服务器通信连接，如图1所示，该方法包括：In this embodiment, a federated learning method based on an attention mechanism is provided. A federated learning system is used as an example for explanation. The system includes at least one client, an aggregation server, and a collaboration server. At least one client is respectively connected to the aggregation server for communication. As shown in FIG1 , the method includes:
步骤101，聚合服务器将第t-1轮的第一密文全局模型发送至客户端；Step 101, the aggregation server sends the first encrypted global model of the t-1th round to the client;
其中，第一密文全局模型为采用CKKS同态加密算法依据第一公钥加密的全局模型。t表示当前迭代更新的轮次，T表示全局迭代轮数，t为正整数，t≤T。The first ciphertext global model is a global model encrypted by the first public key using the CKKS homomorphic encryption algorithm. t represents the current iteration update round, T represents the number of global iteration rounds, t is a positive integer, t≤T.
在该实施例中，采用轻量级CKKS同态加密算法加密全局模型，使得加密后的全局模型支持在加密状态下的运算，而无需解密密文，在保护数据隐私的同时实现计算功能，并且可以减少数据传输量，降低通信开销。In this embodiment, a lightweight CKKS homomorphic encryption algorithm is used to encrypt the global model, so that the encrypted global model supports operations in an encrypted state without decrypting the ciphertext, thereby realizing computing functions while protecting data privacy, reducing data transmission volume, and reducing communication overhead.
可以理解的是，若为首轮学习任务，可省略聚合服务器发送第一密文全局模型发送至客户端的步骤，由客户端直接依据本地数据进行训练，或者随机初始化聚合服务器已有的全局模型，将全局模型加密后发送至客户端进行本地训练。It is understandable that if this is the first round of learning tasks, the step of the aggregation server sending the first ciphertext global model to the client can be omitted, and the client can directly perform training based on local data, or randomly initialize the existing global model of the aggregation server, encrypt the global model and send it to the client for local training.
本申请实施例中客户端是联邦学习中的数据拥有者，可以是分布式的终端设备或用户。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机等。每个客户端都拥有自己的本地数据集，并在本地执行模型训练。客户端负责对本地数据进行模型训练，生成梯度，并上传到聚合服务器。In the embodiment of the present application, the client is the data owner in federated learning, which can be a distributed terminal device or user. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, etc. Each client has its own local data set and performs model training locally. The client is responsible for model training on local data, generating gradients, and uploading them to the aggregation server.
聚合服务器即中央服务器，负责协调和整合来自各个客户端的梯度。聚合服务器接收到来自多个客户端的梯度后，使用加密技术进行权重聚合和添加扰动噪声，生成更新全局模型。然后，聚合服务器将更新的全局模型分发回各个客户端。The aggregation server is a central server that coordinates and integrates the gradients from various clients. After receiving the gradients from multiple clients, the aggregation server uses encryption technology to aggregate the weights and add perturbation noise to generate an updated global model. The aggregation server then distributes the updated global model back to each client.
协作服务器是用于安全计算的服务器，它协助客户端和聚合服务器进行安全计算操作，如加密和解密。协作服务器负责控制加密算法、密钥管理和其他安全相关的操作，确保在联邦学习过程中的数据隐私和安全性。The collaboration server is a server for secure computing, which assists clients and aggregation servers in performing secure computing operations such as encryption and decryption. The collaboration server is responsible for controlling encryption algorithms, key management, and other security-related operations to ensure data privacy and security during the federated learning process.
聚合服务器和协作服务器可以配置成独立的物理服务器，也可以配置成多个物理服务器构成的服务器集群或者分布式系统，还可以配置成提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN以及大数据和人工智能平台等基础云计算服务的云服务器。Aggregation servers and collaborative servers can be configured as independent physical servers, or as server clusters or distributed systems consisting of multiple physical servers. They can also be configured as cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
步骤102，客户端接收聚合服务器发送的第t-1轮的第一密文全局模型；Step 102, the client receives the first ciphertext global model of the t-1th round sent by the aggregation server;
步骤103，客户端依据第一公钥绑定的第一私钥对第t-1轮的第一密文全局模型进行解密处理，得到第t-1轮的全局模型；Step 103: The client decrypts the first ciphertext global model of the t-1th round according to the first private key bound to the first public key to obtain the global model of the t-1th round;
需要说明的是，在联邦学习任务启动时，可生成两份公私钥对，即绑定的第一公钥、第一私钥以及绑定的第二公钥、第二私钥，两份公私钥对用于聚合服务器、协作服务器和客户端之间进行模型参数加密与解密。第一公钥、第一私钥可作为客户端与聚合服务器交互的公私钥对，第二公钥、第二私钥可作为协作服务器与聚合服务器交互的公私钥对，通常情况下客户端和聚合服务器无权获知第二私钥，协作服务器和聚合服务器无权获知第一私钥。It should be noted that when the federated learning task is started, two public-private key pairs can be generated, namely the bound first public key, the first private key and the bound second public key, the second private key. The two public-private key pairs are used to encrypt and decrypt model parameters between the aggregation server, the collaboration server and the client. The first public key and the first private key can be used as the public-private key pair for the client to interact with the aggregation server, and the second public key and the second private key can be used as the public-private key pair for the collaboration server to interact with the aggregation server. Under normal circumstances, the client and the aggregation server have no right to know the second private key, and the collaboration server and the aggregation server have no right to know the first private key.
步骤104，客户端依据第t-1轮的全局模型对本地数据进行迭代训练，确定第t轮的梯度信息；Step 104, the client iteratively trains the local data based on the global model of the t-1th round to determine the gradient information of the tth round;
其中，本地迭代轮数以及迭代训练学习率等训练参数可按需设置，本申请实施例不做具体限定。梯度信息即客户端本地模型相对于损失函数的梯度，用于反映了客户端本地数据对于模型参数的优化方向和程度。本地数据即该客户端在本地采集的数据，可以应用于全局模型来进行模型预测、模型训练以及其它合适的多方数据联合处理。本地数据随着应用场景以及业务的不同而不同，例如，应用场景可以包括金融、保险、政务、医疗、电商等，则本地数据可以为商品交易数据、金融交易数据、医疗健康数据、用户特征数据、关系特征数据等。Among them, training parameters such as the number of local iteration rounds and iterative training learning rate can be set as needed, and the embodiments of the present application do not make specific limitations. Gradient information is the gradient of the client's local model relative to the loss function, which is used to reflect the optimization direction and degree of the client's local data for the model parameters. Local data, that is, data collected locally by the client, can be applied to the global model for model prediction, model training, and other appropriate multi-party data joint processing. Local data varies with different application scenarios and businesses. For example, application scenarios may include finance, insurance, government affairs, medical care, e-commerce, etc., and local data may be commodity transaction data, financial transaction data, medical and health data, user feature data, relationship feature data, etc.
在该实施例中，以前一轮得到的全局模型作为基础，以本地数据训练客户端本地模型，使得本地模型可以从全局模型中学习到一些高质量的特征表示，并且可利用全局模型的模型参数，更快地调整自己的权重，以便于本地模型快速收敛到最优解。In this embodiment, the global model obtained in the previous round is used as a basis, and the client local model is trained with local data, so that the local model can learn some high-quality feature representations from the global model, and can use the model parameters of the global model to adjust its own weights more quickly, so that the local model can quickly converge to the optimal solution.
可以理解的是，系统不同的客户端的性能可能并不相同，如不同的客户端可能具有不同的计算能力和/或通信能力等。It is understandable that the performance of different clients of the system may not be the same. For example, different clients may have different computing capabilities and/or communication capabilities.
步骤105，客户端采用CKKS同态加密算法，依据第二公钥对梯度信息进行加密处理，得到密文梯度信息；Step 105: The client uses the CKKS homomorphic encryption algorithm to encrypt the gradient information according to the second public key to obtain ciphertext gradient information;
步骤106，客户端将密文梯度信息发送至聚合服务器；Step 106, the client sends the ciphertext gradient information to the aggregation server;
在该实施例中，客户端的本地数据可以在本地客户端上进行训练，将梯度信息进行加密传输。如此，聚合服务器依据密文梯度信息得到全局模型的更新方向，并进行全局模型的参数更新。一方面，通过传输梯度信息反应本地训练情况，可以避免将敏感数据传输至聚合服务器，保护用户数据隐私，同时还节约了客户端与聚合服务器之间的通信资源。又一方面，通过第二公钥对梯度信息进行加密，可以确保在传输过程中梯度信息的机密性，可以有效地防止恶意参与者对梯度信息的窃取或篡改。再一方面，采用CKKS同态加密算法进行加密处理，可以保护梯度信息中的隐私，实现安全的信息共享和隐私保护的联合学习。In this embodiment, the local data of the client can be trained on the local client, and the gradient information can be encrypted and transmitted. In this way, the aggregation server obtains the update direction of the global model based on the ciphertext gradient information, and updates the parameters of the global model. On the one hand, by transmitting gradient information to reflect the local training situation, it is possible to avoid transmitting sensitive data to the aggregation server, protect user data privacy, and save communication resources between the client and the aggregation server. On the other hand, by encrypting the gradient information with a second public key, the confidentiality of the gradient information during the transmission process can be ensured, and malicious participants can be effectively prevented from stealing or tampering with the gradient information. On the other hand, the CKKS homomorphic encryption algorithm is used for encryption processing to protect the privacy in the gradient information and realize secure information sharing and privacy-protected joint learning.
步骤107，聚合服务器接收客户端发送的密文梯度信息；Step 107, the aggregation server receives the ciphertext gradient information sent by the client;
步骤108，聚合服务器将密文梯度信息发送至协作服务器；Step 108, the aggregation server sends the ciphertext gradient information to the collaboration server;
可以理解的是，考虑到系统中客户端的数量可能为多个，多个客户端均会向聚合服务器上传密文梯度信息。聚合服务器在转发密文梯度信息发送至协作服务器时，可以按照接收到密文梯度信息的顺序逐个将不同客户端上传的密文梯度信息发送至协作服务器，聚合服务器还可以等待全部客户端均上传密文梯度信息后，将多个客户端的密文梯度信息集合打包后一并发送至协作服务器。It is understandable that, considering that there may be multiple clients in the system, multiple clients will upload ciphertext gradient information to the aggregation server. When the aggregation server forwards the ciphertext gradient information to the collaboration server, it can send the ciphertext gradient information uploaded by different clients to the collaboration server one by one in the order in which the ciphertext gradient information is received. The aggregation server can also wait until all clients have uploaded the ciphertext gradient information, and then package the ciphertext gradient information sets of multiple clients and send them to the collaboration server together.
在一可选的实施例中，步骤108之前，基于注意力机制的联邦学习方法还包括：聚合服务器依据第二公钥生成第一密文噪声值；聚合服务器将第一密文噪声值添加至密文梯度信息，以密文梯度信息进行扰动处理。In an optional embodiment, before step 108, the attention mechanism-based federated learning method also includes: the aggregation server generates a first ciphertext noise value based on the second public key; the aggregation server adds the first ciphertext noise value to the ciphertext gradient information to perform perturbation processing on the ciphertext gradient information.
在该实施例中，通过在密文梯度信息中添加密文噪声值进行扰动处理，由于添加了噪声值，可以增加梯度信息的随机性和不确定性，攻击者无法准确推断出原始梯度信息，有效对抗隐私攻击，从而提高数据的隐私保护程度，降低了敏感信息泄露的风险，进一步提高了模型训练和更新的安全性。In this embodiment, disturbance processing is performed by adding ciphertext noise values to the ciphertext gradient information. The addition of noise values can increase the randomness and uncertainty of the gradient information, and attackers cannot accurately infer the original gradient information, effectively countering privacy attacks, thereby improving the privacy protection of the data, reducing the risk of sensitive information leakage, and further improving the security of model training and updating.
可以理解的是，若密文梯度信息添加了第一密文噪声值，使得后续协作服务器采用带有扰动的密文梯度信息计算注意力权重。It is understandable that if the first ciphertext noise value is added to the ciphertext gradient information, the subsequent collaborative server uses the disturbed ciphertext gradient information to calculate the attention weight.
步骤109，协作服务器接收聚合服务器发送的密文梯度信息；Step 109, the collaboration server receives the ciphertext gradient information sent by the aggregation server;
步骤110，协作服务器依据第二公钥绑定的第二私钥对密文梯度信息进行解密处理，得到梯度信息；Step 110, the collaboration server decrypts the ciphertext gradient information according to the second private key bound to the second public key to obtain gradient information;
步骤111，协作服务器依据梯度信息和客户端的根梯度信息，计算注意力权重；Step 111, the collaborative server calculates the attention weight according to the gradient information and the root gradient information of the client;
在该实施例中，协作服务器保留一部分小而干净的可信任的根数据，并利用根数据训练得到的根梯度信息和客户端本地训练的梯度信息，基于缩放点击注意力机制进行权重值计算。从而通过注意力机制确定根数据和客户端本地数据之间的关联性，进而可确定出客户端是否遭受投毒攻击。再通过注意力权重的大小对后续模型训练的初始权重进行调整，以减少全局模型更新过程中对投毒数据的敏感性，从而提高全局模型的性能和鲁棒性。In this embodiment, the collaborative server retains a small, clean, and trusted portion of root data, and uses the root gradient information obtained by training the root data and the gradient information trained locally on the client to calculate the weight value based on the zoom-click attention mechanism. Thus, the correlation between the root data and the client's local data is determined through the attention mechanism, and then it can be determined whether the client is under a poisoning attack. The initial weight of the subsequent model training is adjusted by the size of the attention weight to reduce the sensitivity to poisoned data during the global model update process, thereby improving the performance and robustness of the global model.
在一可选的实施例中，步骤111，也即协作服务器依据梯度信息和客户端的根梯度信息，计算注意力权重，具体包括如下步骤：In an optional embodiment, step 111, that is, the collaborative server calculates the attention weight according to the gradient information and the root gradient information of the client, specifically includes the following steps:
步骤111-1，协作服务器获取客户端对应的根数据，并对根数据进行训练，得到客户端的根梯度信息；Step 111-1, the collaborative server obtains the root data corresponding to the client, and trains the root data to obtain the root gradient information of the client;
其中，根数据是在客户端应用程序中的没有经过修改或处理的原始数据，代表了最初的数据状态，没有被其他操作改变过。The root data is the original data in the client application that has not been modified or processed, represents the initial data state, and has not been changed by other operations.
步骤111-2，协作服务器依据同一客户端对应的梯度信息和根梯度信息进行点积计算，确定客户端的缩放点积分数；Step 111 - 2, the collaborative server performs dot product calculation based on the gradient information and root gradient information corresponding to the same client to determine the scaling dot product score of the client;
具体地，计算缩放点积分数的公式如下：Specifically, the formula for calculating the scaling point integral score is as follows:
式中，g0表示根梯度信息，d表示梯度维数，⊙表示内积运算，Gi表示梯度信息，αi表示第i个客户端相应的缩放点积分数。Whereg0 represents the root gradient information, d represents the gradient dimension, ⊙ represents the inner product operation,Gi represents the gradient information, andαi represents the corresponding scaling point integral score of the i-th client.
值得一提的是，如果客户端的梯度信息与干净的根梯度信息差异较大，那么其缩放点积分数可能较低，从而降低其在模型聚合中的影响力。It is worth mentioning that if the client's gradient information is significantly different from the clean root gradient information, its scaled point product score may be low, thus reducing its influence in model aggregation.
步骤111-3，协作服务器计算客户端的缩放点积分数在协作服务器关联的至少一个客户端的总缩放点积分数的占比，作为注意力权重。Step 111 - 3 : The collaboration server calculates the proportion of the zoom point score of the client to the total zoom point score of at least one client associated with the collaboration server as the attention weight.
具体地，计算注意力权重的公式如下：Specifically, the formula for calculating the attention weight is as follows:
式中，ωi表示第i个客户端的注意力权重，αi表示第i个客户端相应的缩放点积分数，m表示联邦学习系统中与聚合服务器交互的客户端数量，exp(·)表示指数函数(如，exp(a)＝e^a)。whereωi represents the attention weight of the ith client,αi represents the corresponding scaled point integral score of the ith client, m represents the number of clients interacting with the aggregation server in the federated learning system, and exp(·) represents an exponential function (e.g., exp(a) = e^a).
在该实施例中，协作服务器利用干净的根数据集训练生成根梯度信息。通过属于同一客户端的根梯度信息和梯度信息，计算该客户端相应的缩放点积分数，以分析出客户端是否遭受投毒攻击，并量化客户端对模型参数的影响程度。再利用该客户端的缩放点积分数在所有客户端的总缩放点积分数中的占比确定注意力权重。从而更准确地分配权重，有助于排除恶意客户端对模型更新的干扰，可更安全的更新模型参数，保证联邦学习的安全性和可靠性。In this embodiment, the collaborative server generates root gradient information using clean root data set training. The corresponding scaling point score of the client is calculated through the root gradient information and gradient information belonging to the same client to analyze whether the client is under poisoning attack and quantify the degree of influence of the client on the model parameters. The attention weight is then determined by the proportion of the scaling point score of the client in the total scaling point score of all clients. In this way, the weights are allocated more accurately, which helps to eliminate the interference of malicious clients on model updates, and the model parameters can be updated more safely, ensuring the security and reliability of federated learning.
步骤112，协作服务器采用CKKS同态加密算法，依据第二公钥对注意力权重进行加密处理，得到密文权重；Step 112: The collaborative server uses the CKKS homomorphic encryption algorithm to encrypt the attention weight according to the second public key to obtain a ciphertext weight.
在该实施例中，以CKKS同态加密算法对注意力权重进行加密，可以在不暴露明文的情况下，通过注意力权重进行计算和处理。从而可以保护数据隐私，防止信息泄漏，即使协作服务器受到攻击或泄漏，攻击者也无法获取到明文的注意力权重，有效降低因服务器被攻击而导致的信息泄漏风险，实现安全计算，进而有助于提高联邦学习的安全性、可信度和可扩展性。In this embodiment, the attention weights are encrypted using the CKKS homomorphic encryption algorithm, and calculations and processing can be performed using the attention weights without exposing the plaintext. This protects data privacy and prevents information leakage. Even if the collaborative server is attacked or leaked, the attacker cannot obtain the plaintext attention weights, effectively reducing the risk of information leakage caused by server attacks, achieving secure computing, and thus helping to improve the security, credibility, and scalability of federated learning.
步骤113，协作服务器将密文权重发送至聚合服务器；Step 113, the collaboration server sends the ciphertext weight to the aggregation server;
步骤114，聚合服务器接收协作服务器发送的密文权重；Step 114, the aggregation server receives the ciphertext weight sent by the collaboration server;
步骤115，聚合服务器依据密文权重和密文梯度信息，对第t-1轮的第一密文全局模型进行更新，形成第t轮的第一密文全局模型。Step 115: The aggregation server updates the first ciphertext global model of the t-1th round according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the tth round.
本申请实施例提供的基于注意力机制的联邦学习方法，在本地客户端基于前一轮得到的全局模型进行本地训练，并将训练好的梯度信息加密算法进行加密，并将加密后的密文梯度信息上传至聚合服务器。聚合服务器将加密后的密文梯度信息转发给协作服务器。协作服务器保留一部分小而干净的可信任的根数据，并基于缩放点击注意力机制对客户端上传的密文梯度进行权重值计算，再将基于注意力机制得到的注意力权重加密后返回聚合服务器。由聚合服务器利用密文权重和所述密文梯度信息完成加权聚合，以完成当前轮次全局模型的更新，聚合服务器在将更新后的密文全局模型分发给各客户端进行下一轮训练，直至达到预设的全局迭代轮数后停止。一方面，使用缩放点击注意力机制对客户端上传的密文梯度进行权重计算和密文梯度信息聚合，便于识别出梯度信息是否存在恶意攻击，并降低可能存在恶意数据对于模型更新的重要程度，实现高效且安全的联邦学习过程。又一方面，在联邦学习过程中，采用轻量级CKKS同态加密算法对客户端隐私数据的梯度进行加密保护，以确保恶意客户端不能推断成员上传恶意梯度，保证在恶意环境下可有效抵抗来自恶意客户端的投毒攻击且不影响模型精度，同时提供对客户端数据的增强型隐私保护，实现高效且安全的联邦学习过程。再一方面，由协作服务器进行注意力机制的相关运算，无需双云服务器针对同一安全协议同时工作，降低了通信开销。The federated learning method based on the attention mechanism provided in the embodiment of the present application performs local training on the local client based on the global model obtained in the previous round, and encrypts the trained gradient information encryption algorithm, and uploads the encrypted ciphertext gradient information to the aggregation server. The aggregation server forwards the encrypted ciphertext gradient information to the collaboration server. The collaboration server retains a small, clean and trusted root data, and calculates the weight value of the ciphertext gradient uploaded by the client based on the zoom click attention mechanism, and then encrypts the attention weight obtained based on the attention mechanism and returns it to the aggregation server. The aggregation server uses the ciphertext weight and the ciphertext gradient information to complete the weighted aggregation to complete the update of the global model of the current round. The aggregation server distributes the updated ciphertext global model to each client for the next round of training until the preset number of global iterations is reached. On the one hand, the weight calculation and ciphertext gradient information aggregation of the ciphertext gradient uploaded by the client are performed using the zoom click attention mechanism, which is convenient for identifying whether the gradient information has malicious attacks, and reduces the importance of possible malicious data for model updates, so as to achieve an efficient and secure federated learning process. On the other hand, during the federated learning process, the lightweight CKKS homomorphic encryption algorithm is used to encrypt and protect the gradient of the client's private data to ensure that malicious clients cannot infer that members upload malicious gradients, and to ensure that poisoning attacks from malicious clients can be effectively resisted in a malicious environment without affecting the accuracy of the model. At the same time, enhanced privacy protection for client data is provided to achieve an efficient and secure federated learning process. On the other hand, the collaborative server performs related operations of the attention mechanism, and there is no need for dual cloud servers to work simultaneously for the same security protocol, which reduces communication overhead.
在一可选的实施例中，基于注意力机制的联邦学习方法还包括：若当前轮次t大于或等于预设全局迭代轮数，停止对全局模型进行更新。In an optional embodiment, the attention mechanism-based federated learning method further includes: if the current round t is greater than or equal to a preset global iteration number, stopping updating the global model.
在该实施例中，在全局模型的更新轮次达到预设的全局迭代轮数后，不再对全局模型进行更新。如此，全局模型将保持不变，不会再受到来自局部模型的更新影响，以避免过拟合或节省计算资源。停止更新全局模型后，可以进行模型评估或其他后续操作。In this embodiment, after the update round of the global model reaches the preset global iteration round number, the global model is no longer updated. In this way, the global model will remain unchanged and will no longer be affected by the update from the local model to avoid overfitting or save computing resources. After stopping updating the global model, model evaluation or other subsequent operations can be performed.
在一可选的实施例中，对第t-1轮的第一密文全局模型进行更新，形成第t轮的第一密文全局模型，具体包括如下步骤：In an optional embodiment, updating the first ciphertext global model of the t-1th round to form the first ciphertext global model of the tth round specifically includes the following steps:
步骤201，聚合服务器依据密文权重聚合密文梯度信息，得到第t轮的第二密文全局模型；Step 201, the aggregation server aggregates the ciphertext gradient information according to the ciphertext weight to obtain the second ciphertext global model of the tth round;
其中，第二密文全局模型为采用CKKS同态加密算法依据第二公钥加密的全局模型。Among them, the second ciphertext global model is a global model encrypted according to the second public key using the CKKS homomorphic encryption algorithm.
步骤202，聚合服务器将第t轮的第二密文全局模型发送至协作服务器；Step 202, the aggregation server sends the second ciphertext global model of the tth round to the collaboration server;
值得一提的是，在聚合服务器将第t轮的第二密文全局模型发送至协作服务器之前，注意力机制的联邦学习方法还包括：聚合服务器依据第二公钥生成第二密文噪声值；聚合服务器将第二密文噪声值添加至第t轮的第二密文全局模型，以进行第t轮的第二密文全局模型的扰动处理。如此，通过在第二密文全局模型中引入密文噪声值来进行扰动处理，从而使第二密文全局模型的具体信息变得更加模糊，从而保护了隐私数据，降低第二密文全局模型的敏感性和隐私泄露的风险，增加攻击者的难度，提高模型的安全性。It is worth mentioning that before the aggregation server sends the second ciphertext global model of the tth round to the collaboration server, the federated learning method of the attention mechanism also includes: the aggregation server generates a second ciphertext noise value based on the second public key; the aggregation server adds the second ciphertext noise value to the second ciphertext global model of the tth round to perform perturbation processing on the second ciphertext global model of the tth round. In this way, by introducing the ciphertext noise value into the second ciphertext global model for perturbation processing, the specific information of the second ciphertext global model becomes more vague, thereby protecting the privacy data, reducing the sensitivity of the second ciphertext global model and the risk of privacy leakage, increasing the difficulty of attackers, and improving the security of the model.
其中，若已生成第一密文噪声值，可直接将第一密文噪声值作为第二密文噪声值对第二密文全局模型进行扰动，从而省略依据第二公钥生成第二密文噪声值的步骤。Among them, if the first ciphertext noise value has been generated, the first ciphertext noise value can be directly used as the second ciphertext noise value to perturb the second ciphertext global model, thereby omitting the step of generating the second ciphertext noise value according to the second public key.
可以理解的是，若第二密文全局模型添加了第二密文噪声值，使得后续协作服务器解密和加密产生密文全局模型也存在扰动。It is understandable that if the second ciphertext global model adds a second ciphertext noise value, the subsequent ciphertext global model generated by decryption and encryption by the collaborative server will also be disturbed.
步骤203，协作服务器接收聚合服务器发送的第t轮的第二密文全局模型；Step 203, the collaboration server receives the second ciphertext global model of the tth round sent by the aggregation server;
步骤204，协作服务器依据第二私钥对第t轮的第二密文全局模型进行解密处理，得到第t轮的全局模型；Step 204: The collaborative server decrypts the second ciphertext global model of the tth round according to the second private key to obtain the global model of the tth round;
步骤205，协作服务器采用CKKS同态加密算法，依据第一公钥对第t轮的全局模型进行加密处理，得到第t轮的第一密文全局模型；Step 205: The collaborative server uses the CKKS homomorphic encryption algorithm to encrypt the global model of the tth round according to the first public key to obtain the first ciphertext global model of the tth round;
步骤206，协作服务器将第t轮的第一密文全局模型发送至聚合服务器Step 206: The collaborative server sends the first encrypted global model of round t to the aggregation server
步骤207，聚合服务器接收协作服务器发送的第t轮的第一密文全局模型。Step 207: The aggregation server receives the first encrypted global model of the tth round sent by the collaboration server.
在该实施例中，聚合服务器基于密文权重对所有客户端的本地梯度信息进行密文加权聚合，以更新当前轮次的密文全局模型，聚合过程中以密文形式进行数据处理，降低了恶意攻击者窃取敏感信息的可能性，提高了数据的安全性。此外，由于密文权重和密文梯度信息均采用协作服务器约定的第二公钥进行加密，使得客户端无法解密密文聚合后的第二密文全局模型。协作服务器可依据第一公钥对通过第二私钥解密后得到的全局模型进行加密处理，从而在保证数据传输和处理安全性的基础上，通过重加密处理将密文全局模型转换成客户端可以解密密文形式，使得客户端能够直接使用转换后的第一密文全局模型进行推理和预测，而无需解密模型或与服务器进行通信，减少了通信的开销，提高了模型的可用性和实际应用的便利性。In this embodiment, the aggregation server performs ciphertext weighted aggregation on the local gradient information of all clients based on the ciphertext weight to update the ciphertext global model of the current round. Data is processed in ciphertext form during the aggregation process, which reduces the possibility of malicious attackers stealing sensitive information and improves data security. In addition, since both the ciphertext weight and the ciphertext gradient information are encrypted using the second public key agreed upon by the collaborative server, the client cannot decrypt the second ciphertext global model after ciphertext aggregation. The collaborative server can encrypt the global model obtained after decryption by the second private key based on the first public key, thereby converting the ciphertext global model into a ciphertext form that can be decrypted by the client through re-encryption processing on the basis of ensuring the security of data transmission and processing, so that the client can directly use the converted first ciphertext global model for reasoning and prediction without decrypting the model or communicating with the server, reducing communication overhead and improving the availability of the model and convenience of practical application.
可以理解的是，若第t轮的第一密文全局模型添加有第二密文噪声值，对第t轮的第一密文全局模型进行去噪处理。以去除第一密文全局模型中噪声扰动，得到干净的第一密文全局模型以便于后续使用。It can be understood that if the first ciphertext global model of the tth round is added with the second ciphertext noise value, the first ciphertext global model of the tth round is subjected to denoising to remove the noise disturbance in the first ciphertext global model and obtain a clean first ciphertext global model for subsequent use.
在一具体的实施例中，如图2所示，提出了一种基于注意力机制的隐私联邦学习方法，包括如下步骤：In a specific embodiment, as shown in FIG2 , a privacy federated learning method based on an attention mechanism is proposed, comprising the following steps:
步骤301，系统初始化。Step 301, system initialization.
该系统包括聚合服务器S0，协作服务器S1和m个客户端C，m为正整数。The system includes an aggregation server S0 , a collaboration server S1 and m clients C, where m is a positive integer.
具体地，设置系统的全局迭代轮数T、本地迭代轮数E以及学习率η。随机初始化聚合服务器的全局模型w1。协作服务器收集小而干净的根数据集D0，并将其训练生成根梯度g0。生成两份公私钥对，用于聚合服务器、协作服务器和客户端之间进行模型参数加密与解密，分别为协作服务器的公私钥对和客户端的公私钥对(pkC，skC)。若为首次更新，聚合服务器使用客户端的第一公钥pkC对初始全局模型w1进行加密，得到第一密文全局模型Specifically, set the global iteration number T, local iteration number E and learning rate η of the system. Randomly initialize the global model w1 of the aggregation server. The collaborative server collects a small and clean root data set D0 and trains it to generate root gradient g0. Generate two public and private key pairs for encryption and decryption of model parameters between the aggregation server, the collaborative server and the client, which are the public and private key pairs of the collaborative server. If it is the first update, theaggregation server uses the client's first public key pk Cto encrypt the initial global model w1 and obtainthe first ciphertext global model
其中，聚合服务器使用客户端的第一公钥pkC对初始全局模型w1进行加密的步骤如下：The steps of the aggregation server using the first public key pkC of the client to encrypt the initial global modelw1 are as follows:
(1)基于CKKS同态加密算法的性质，对模型参数进行分片，服务器节点将初始全局模型w1分为个片段，即(1) Based on the properties of the CKKS homomorphic encryption algorithm, the model parameters are sharded. The server node divides the initial global modelw1 into fragments, i.e.
(2)基于CKKS同态加密算法，用公钥pkC对每个分片进行加密，得到初始全局模型密文(2) Based on the CKKS homomorphic encryption algorithm, each shard is encrypted with the public key pkC to obtain the initial global model ciphertext
式中，梯度长度为n，可加密长度为k，表示向上取整操作，表示向下取整操作，表示CKKS加密算法。In the formula, the gradient length is n, the encryption length is k, Indicates rounding up operation. Indicates a round-down operation. Indicates the CKKS encryption algorithm.
步骤302，客户端本地训练。Step 302: client local training.
具体地，由于联邦学习过程需要多轮次训练，在本节中以第t轮单个客户端Ci的训练过程为例进行说明，i表示客户端Ci(Ci∈C)的索引值。客户端Ci获取聚合服务器分发的第一密文全局模型并使用第一私钥skC对该参数进行解密，获得明文的全局模型wt。客户端Ci使用本地数据迭代E次完成本地训练，获得本地梯度Specifically, since the federated learning process requires multiple rounds of training, this section takes the training process of a single client Ci in the tth round as an example to illustrate, where i represents the index value of client Ci (Ci∈C). Client Ci obtains the first ciphertext global model distributed by the aggregation server The first private key skC is used to decrypt the parameter to obtain the global model wt in plain text. The client Ci uses the local data to iterate E times to complete the local training and obtain the local gradient
其中，客户端Ci使用本地数据进行本地训练的步骤如下：The steps for clientCi to perform local training using local data are as follows:
(1)在数据集中选择参与本次迭代训练的随机样本Di表示客户端Ci的本地数据集，j表示迭代的轮次。(1) Select random samples from the dataset to participate in this iterative trainingDi represents the local dataset of clientCi , and j represents the round of iteration.
(2)按照下式迭代训练本地模型：(2) Iterate and train the local model according to the following formula:
式中，η表示本地训练的学习率，表示损失函数。Where η represents the learning rate of local training, Represents the loss function.
步骤303，客户端上传本地更新。Step 303: The client uploads the local update.
具体地，客户端Ci使用协作服务器的第二公钥对本地梯度进行加密得到密文梯度并将其作为本轮次的本地更新。客户端Ci将加密后的密文梯度发送给聚合服务器。Specifically, the client Ci uses the second public key of the collaboration server For local gradient Encrypt to get the ciphertext gradient And use it as the local update of this round. ClientCi will encrypt the ciphertext gradient Sent to the aggregation server.
步骤304，聚合服务器扰动本地梯度。Step 304: The aggregation server perturbs the local gradient.
具体地，聚合服务器获取参与训练的客户端的密文梯度聚合服务器随机生成一个非零的整数作为噪声值，并使用协作服务器的第二公钥对其进行加密，得到第一个密文噪声值聚合服务器为参与训练的客户端的密文梯度添加的密文噪声值，得到扰动后的密文梯度聚合服务器将扰动后的密文梯度存入集合G中，并将扰动后的密文梯度发送给协作服务器。Specifically, the aggregation server obtains the ciphertext gradients of the clients participating in the training The aggregation server randomly generates a non-zero integer as the noise value and uses the second public key of the collaboration server Encrypt it to get the first ciphertext noise value The aggregation server adds the ciphertext noise value to the ciphertext gradient of the client participating in the training to obtain the disturbed ciphertext gradient The aggregation server stores the perturbed ciphertext gradients into the set G and sends the perturbed ciphertext gradients to the collaborative server.
其中，添加密文梯度噪声值的公式如下：The formula for adding the ciphertext gradient noise value is as follows:
步骤305，协作服务器计算权重。Step 305: The collaboration server calculates the weight.
具体地，协作服务器获取扰动后的密文梯度集合G。协作服务器遍历扰动后的密文梯度集合G，并使用自己的第二私钥对其中扰动后的密文梯度进行解密。协作服务器为每个扰动后的本地梯度计算相应的缩放点积分数值。协作服务器基于扰动后的本地梯度的缩放点击分数值计算相应的权重系数(注意力权重)。协作服务器使用自己的第二公钥对权重系数进行加密得到密文权重系数协作服务器将参与训练的客户端的密文权重系数发送给聚合服务器。Specifically, the collaborative server obtains the perturbed ciphertext gradient set G. The collaborative server traverses the perturbed ciphertext gradient set G and uses its own second private key The perturbed ciphertext gradient is decrypted. The collaborative server calculates the corresponding scaled point integral value for each perturbed local gradient. The collaborative server calculates the corresponding weight coefficient (attention weight) based on the scaled click score value of the perturbed local gradient. The collaborative server uses its own second public key Encrypt the weight coefficient to obtain the ciphertext weight coefficient The collaborative server sends the ciphertext weight coefficients of the clients participating in the training to the aggregation server.
其中，计算客户端Ci相应的缩放点积分数值的公式如下：The formula for calculating the corresponding scaling point integral value of clientCi is as follows:
式中，g0表示根梯度信息，d表示梯度维数，⊙表示内积运算，Gi表示梯度信息，αi表示第i个客户端相应的缩放点积分数。Whereg0 represents the root gradient information, d represents the gradient dimension, ⊙ represents the inner product operation,Gi represents the gradient information, andαi represents the corresponding scaling point integral score of the i-th client.
计算相应的权重系数的公式如下：The formula for calculating the corresponding weight coefficient is as follows:
式中，ωi表示第i个客户端的注意力权重，αi表示第i个客户端相应的缩放点积分数，m表示联邦学习系统中与聚合服务器交互的客户端数量，exp(·)表示预设函数。whereωi represents the attention weight of the ith client,αi represents the corresponding scaled point integral score of the ith client, m represents the number of clients interacting with the aggregation server in the federated learning system, and exp(·) represents the preset function.
步骤306，聚合服务器进行全局模型聚合更新。Step 306: The aggregation server performs global model aggregation update.
具体地，聚合服务器获取客户端的密文权重系数。聚合服务器为扰动后的本地密文梯度去除噪声值，得到干净的密文梯度。聚合服务器基于密文权重系数聚合所有客户端的密文本地梯度，以更新密文全局模型，得到Specifically, the aggregation server obtains the ciphertext weight coefficient of the client. The aggregation server removes the noise value of the perturbed local ciphertext gradient to obtain a clean ciphertext gradient. The aggregation server aggregates the ciphertext local gradients of all clients based on the ciphertext weight coefficient to update the ciphertext global model and obtain
其中，去除密文梯度噪声的公式如下：Among them, the formula for removing ciphertext gradient noise is as follows:
聚合服务器基于密文权重系数聚合所有客户端的密文本地梯度的公式如下：The formula for the aggregation server to aggregate the ciphertext local gradients of all clients based on the ciphertext weight coefficient is as follows:
步骤307，聚合服务器扰动全局模型。Step 307: The aggregation server perturbs the global model.
具体地，聚合服务器随机生成一个新的非零整数作为噪声值，并使用协作服务器的第二公钥对其进行加密，得到第二个密文噪声值聚合服务器为密文全局模型添加新的密文噪声值得到扰动后的密文全局模型聚合服务器将扰动后的密文全局模型发送给协作服务器。Specifically, the aggregation server randomly generates a new non-zero integer as the noise value and uses the second public key of the collaboration server Encrypt it to get the second ciphertext noise value Aggregation server is a ciphertext global model Add new ciphertext noise value Get the perturbed ciphertext global model The aggregation server will perturb the ciphertext global model Sent to the collaboration server.
其中，为密文全局模型添加密文梯度噪声值的公式如下：Among them, is the ciphertext global model The formula for adding the ciphertext gradient noise value is as follows:
步骤308，协作服务器重加密。Step 308: The collaborative server re-encrypts.
具体地，协作服务器获取聚合服务器发送的扰动后的密文全局模型协作服务器使用自己的第二私钥对扰动后的密文全局模型进行解密，得到干净的扰动后的全局模型协作服务器使用客户端的第一公钥pkC重新加密扰动的全局模型，得到协作服务器发送重新加密的扰动后的密文全局模型给聚合服务器。Specifically, the collaborative server obtains the perturbed ciphertext global model sent by the aggregation server The collaborative server uses its own second private key Global model of perturbed ciphertext Decrypt and get a clean perturbed global model The collaborative server re-encrypts the perturbed global model using the client's first public key pkC to obtain The collaborative server sends the re-encrypted perturbed ciphertext to the global model To the aggregation server.
步骤309，聚合服务器分发全局模型。Step 309: The aggregation server distributes the global model.
具体地，聚合服务器获取扰动后的密文全局模型聚合服务器为扰动后的密文全局模型去除噪声值，得到干净的密文全局模型聚合服务器将密文全局模型分发给各客户端进行下一轮训练，全局模型的更新在达到设定的轮数T后停止。Specifically, the aggregation server obtains the perturbed ciphertext global model The aggregation server removes the noise value from the disturbed ciphertext global model to obtain a clean ciphertext global model. The aggregation server will cipher the global model Distribute to each client for the next round of training, and the update of the global model stops after reaching the set number of rounds T.
其中，聚合服务器为扰动后的密文全局模型去除噪声值的公式如下：The formula for removing noise value from the global model of the disturbed ciphertext by the aggregation server is as follows:
在该实施例中，首先在本地客户端基于全局模型进行本地训练，并将训练好的梯度使用CKKS同态加密算法加密并上传至聚合服务器，聚合服务器扰动梯度更新后发送给协作服务器，协作服务器保留一个小而干净的可信任根数据集，并基于缩放点击注意力机制对客户端上传的密文梯度进行权重值计算，然后由聚合服务器完成密文加权聚合，最后双云服务器使用重加密技术提供全局模型给客户端下载。从而可有效抵抗来自恶意客户端的投毒攻击，在面向恶意环境下训练高精度、高鲁棒、高可用性的全局模型，保证在恶意环境下可有效抵抗来自恶意客户端的投毒攻击且不影响模型精度，同时，提供对客户端数据的增强型隐私保护。实现高效且安全的联邦学习过程。In this embodiment, first, local training is performed on the local client based on the global model, and the trained gradient is encrypted using the CKKS homomorphic encryption algorithm and uploaded to the aggregation server. The aggregation server perturbs the gradient and sends it to the collaboration server after updating. The collaboration server retains a small and clean trusted root data set, and calculates the weight value of the ciphertext gradient uploaded by the client based on the zoom click attention mechanism. The aggregation server then completes the weighted aggregation of the ciphertext, and finally the dual cloud server uses re-encryption technology to provide the global model to the client for download. This can effectively resist poisoning attacks from malicious clients, train high-precision, high-robustness, and high-availability global models in malicious environments, and ensure that poisoning attacks from malicious clients can be effectively resisted in malicious environments without affecting the accuracy of the model. At the same time, enhanced privacy protection for client data is provided. An efficient and secure federated learning process is achieved.
需要说明的是，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
在一实施例中，基于上述如图1和图2所示的方法，为了实现上述目的，本申请实施例还提供了一种服务器，该包括存储介质和处理器；存储介质，用于存储计算机程序；处理器，用于执行计算机程序以实现上述应用于聚合服务器和协作服务器的基于注意力机制的联邦学习方法。In one embodiment, based on the above method as shown in Figures 1 and 2, in order to achieve the above purpose, the embodiment of the present application also provides a server, which includes a storage medium and a processor; the storage medium is used to store computer programs; the processor is used to execute the computer program to implement the above-mentioned attention-based federated learning method applied to the aggregation server and the collaborative server.
在一实施例中，基于上述如图1和图2所示的方法，为了实现上述目的，本申请实施例还提供了一种客户端，该包括存储介质和处理器；存储介质，用于存储计算机程序；处理器，用于执行计算机程序以实现上述应用于客户端的基于注意力机制的联邦学习方法。In one embodiment, based on the above method as shown in Figures 1 and 2, in order to achieve the above purpose, the embodiment of the present application also provides a client, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the above-mentioned attention mechanism-based federated learning method applied to the client.
可选地，该客户端还可以包括用户接口、网络接口、摄像头、射频(RadioFrequency，RF)电路，传感器、音频电路、WI-FI模块等等。用户接口可以包括显示屏(Display)、输入单元比如键盘(Keyboard)等，可选用户接口还可以包括USB接口、读卡器接口等。网络接口可选的可以包括标准的有线接口、无线接口(如蓝牙接口、WI-FI接口)等。Optionally, the client may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.
在一实施例中，如图3所示，作为上述基于注意力机制的联邦学习方法的具体实现，本申请实施例提供了一种联邦学习系统，该基于注意力机制的联邦学习系统包括：客户端41、聚合服务器42和协作服务器43。In one embodiment, as shown in Figure 3, as a specific implementation of the above-mentioned federated learning method based on the attention mechanism, an embodiment of the present application provides a federated learning system, which is based on the attention mechanism and includes: a client 41, an aggregation server 42 and a collaborative server 43.
其中，客户端41用于接收聚合服务器发送的第t-1轮的第一密文全局模型，其中，第一密文全局模型为采用CKKS同态加密算法依据第一公钥加密的全局模型；依据第一公钥绑定的第一私钥对第t-1轮的第一密文全局模型进行解密处理，得到第t-1轮的全局模型；依据第t-1轮的全局模型对本地数据进行迭代训练，确定第t轮的梯度信息；采用CKKS同态加密算法，依据第二公钥对梯度信息进行加密处理，得到密文梯度信息；将密文梯度信息发送至聚合服务器，以使聚合服务器依据密文权重和密文梯度信息，对第t-1轮的第一密文全局模型进行更新，形成第t轮的第一密文全局模型，其中，密文权重由协作服务器依据对注意力权重的加密处理得到，注意力权重依据梯度信息计算得到。Among them, the client 41 is used to receive the first ciphertext global model of the t-1 round sent by the aggregation server, wherein the first ciphertext global model is a global model encrypted according to the first public key using the CKKS homomorphic encryption algorithm; the first ciphertext global model of the t-1 round is decrypted according to the first private key bound to the first public key to obtain the global model of the t-1 round; local data is iteratively trained according to the global model of the t-1 round to determine the gradient information of the t round; the gradient information is encrypting according to the second public key using the CKKS homomorphic encryption algorithm to obtain the ciphertext gradient information; the ciphertext gradient information is sent to the aggregation server, so that the aggregation server updates the first ciphertext global model of the t-1 round according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the t round, wherein the ciphertext weight is obtained by the collaborative server based on the encryption processing of the attention weight, and the attention weight is calculated based on the gradient information.
聚合服务器42用于将第t-1轮的第一密文全局模型发送至客户端，其中，第一密文全局模型为采用CKKS同态加密算法依据第一公钥加密的全局模型，以使客户端依据第t-1轮的第一密文全局模型解密后的第t-1轮的全局模型和本地数据确定客户端的梯度信息；接收客户端发送的密文梯度信息，其中，密文梯度信息为采用CKKS同态加密算法依据第二公钥对加密的梯度信息；将密文梯度信息发送至协作服务器，以使协作服务器依据梯度信息和客户端的根梯度信息，计算注意力权重；接收协作服务器发送的密文权重，其中，密文权重为采用CKKS同态加密算法依据第二公钥加密的注意力权重；依据密文权重和密文梯度信息，对第t-1轮的第一密文全局模型进行更新，形成第t轮的第一密文全局模型。The aggregation server 42 is used to send the first ciphertext global model of the t-1th round to the client, wherein the first ciphertext global model is a global model encrypted according to the first public key using the CKKS homomorphic encryption algorithm, so that the client determines the client's gradient information based on the global model of the t-1th round and local data decrypted by the first ciphertext global model of the t-1th round; receive the ciphertext gradient information sent by the client, wherein the ciphertext gradient information is the gradient information encrypted according to the second public key pair using the CKKS homomorphic encryption algorithm; send the ciphertext gradient information to the collaborative server, so that the collaborative server calculates the attention weight based on the gradient information and the root gradient information of the client; receive the ciphertext weight sent by the collaborative server, wherein the ciphertext weight is the attention weight encrypted according to the second public key using the CKKS homomorphic encryption algorithm; update the first ciphertext global model of the t-1th round based on the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the tth round.
协作服务器43用于接收聚合服务器发送的密文梯度信息；依据第二公钥绑定的第二私钥对密文梯度信息进行解密处理，得到梯度信息；依据梯度信息和客户端的根梯度信息，计算注意力权重；采用CKKS同态加密算法，依据第二公钥对注意力权重进行加密处理，得到密文权重；将密文权重发送至聚合服务器，以使聚合服务器依据密文权重和密文梯度信息，对第t-1轮的第一密文全局模型进行更新，形成第t轮的第一密文全局模型。The collaborative server 43 is used to receive the ciphertext gradient information sent by the aggregation server; decrypt the ciphertext gradient information according to the second private key bound to the second public key to obtain the gradient information; calculate the attention weight according to the gradient information and the root gradient information of the client; use the CKKS homomorphic encryption algorithm to encrypt the attention weight according to the second public key to obtain the ciphertext weight; send the ciphertext weight to the aggregation server, so that the aggregation server updates the first ciphertext global model of the t-1 round according to the ciphertext weight and the ciphertext gradient information to form the first ciphertext global model of the t round.
在一实施例中，聚合服务器42具体用于依据密文权重聚合密文梯度信息，得到第t轮的第二密文全局模型，其中，第二密文全局模型为采用CKKS同态加密算法依据第二公钥加密的全局模型；将第t轮的第二密文全局模型发送至协作服务器，以使协作服务器依据第一公钥对通过第二私钥解密后得到的全局模型进行加密处理，得到第t轮的第一密文全局模型；接收协作服务器发送的第t轮的第一密文全局模型。In one embodiment, the aggregation server 42 is specifically used to aggregate the ciphertext gradient information according to the ciphertext weights to obtain the second ciphertext global model of the tth round, wherein the second ciphertext global model is a global model encrypted by the second public key using the CKKS homomorphic encryption algorithm; the second ciphertext global model of the tth round is sent to the collaboration server, so that the collaboration server encrypts the global model obtained after decryption by the second private key according to the first public key to obtain the first ciphertext global model of the tth round; and the first ciphertext global model of the tth round sent by the collaboration server is received.
在一实施例中，聚合服务器42还用于依据第二公钥生成第一密文噪声值；将第一密文噪声值添加至密文梯度信息，以密文梯度信息进行扰动处理；依据第二公钥生成第二密文噪声值；将第二密文噪声值添加至第t轮的第二密文全局模型，以进行第t轮的第二密文全局模型的扰动处理；若第t轮的第一密文全局模型添加有第二密文噪声值，对第t轮的第一密文全局模型进行去噪处理。In one embodiment, the aggregation server 42 is also used to generate a first ciphertext noise value based on the second public key; add the first ciphertext noise value to the ciphertext gradient information to perform perturbation processing on the ciphertext gradient information; generate a second ciphertext noise value based on the second public key; add the second ciphertext noise value to the second ciphertext global model of the tth round to perform perturbation processing on the second ciphertext global model of the tth round; if the first ciphertext global model of the tth round is added with the second ciphertext noise value, denoising is performed on the first ciphertext global model of the tth round.
在一实施例中，协作服务器43具体用于获取客户端对应的根数据，并对根数据进行训练，得到客户端的根梯度信息；依据同一客户端对应的梯度信息和根梯度信息进行点积计算，确定客户端的缩放点积分数；计算客户端的缩放点积分数在协作服务器关联的至少一个客户端的总缩放点积分数的占比，作为注意力权重。In one embodiment, the collaborative server 43 is specifically used to obtain root data corresponding to the client, and train the root data to obtain root gradient information of the client; perform dot product calculation based on the gradient information and root gradient information corresponding to the same client to determine the scaling point score of the client; calculate the proportion of the scaling point score of the client in the total scaling point score of at least one client associated with the collaborative server as the attention weight.
在一实施例中，协作服务器43还用于接收聚合服务器发送的第t轮的第二密文全局模型；依据第二私钥对第t轮的第二密文全局模型进行解密处理，得到第t轮的全局模型；采用CKKS同态加密算法，依据第一公钥对第t轮的全局模型进行加密处理，得到第t轮的第一密文全局模型；将第t轮的第一密文全局模型发送至聚合服务器。In one embodiment, the collaborative server 43 is also used to receive the second ciphertext global model of the tth round sent by the aggregation server; decrypt the second ciphertext global model of the tth round according to the second private key to obtain the global model of the tth round; use the CKKS homomorphic encryption algorithm to encrypt the global model of the tth round according to the first public key to obtain the first ciphertext global model of the tth round; and send the first ciphertext global model of the tth round to the aggregation server.
基于上述如图1和图2所示方法，相应的，本申请实施例还提供了一种可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述如图1和图2所示的基于注意力机制的联邦学习方法。Based on the above method as shown in Figures 1 and 2, accordingly, an embodiment of the present application also provides a readable storage medium on which a computer program is stored. When the program is executed by a processor, the above federated learning method based on the attention mechanism as shown in Figures 1 and 2 is implemented.
基于这样的理解，本申请的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施场景所述的方法。Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.
存储介质中还可以包括操作系统、网络通信模块。操作系统是管理和保存计算机设备硬件和软件资源的程序，支持信息处理程序以及其它软件和/或程序的运行。网络通信模块用于实现存储介质内部各组件之间的通信，以及与该实体设备中其它硬件和软件之间通信。The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and/or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the physical device.
本领域技术人员可以理解附图只是一个优选实施场景的示意图，附图中的模块或流程并不一定是实施本申请所必须的。本领域技术人员可以理解实施场景中的装置中的模块可以按照实施场景描述进行分布于实施场景的装置中，也可以进行相应变化位于不同于本实施场景的一个或多个装置中。上述实施场景的模块可以合并为一个模块，也可以进一步拆分成多个子模块。Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.
上述本申请序号仅仅为了描述，不代表实施场景的优劣。以上公开的仅为本申请的几个具体实施场景，但是，本申请并非局限于此，任何本领域的技术人员能思之的变化都应落入本申请的保护范围。The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410573353.4ACN118350452B (en) | 2024-05-10 | 2024-05-10 | Federated learning method, client, server and system based on attention mechanism |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410573353.4ACN118350452B (en) | 2024-05-10 | 2024-05-10 | Federated learning method, client, server and system based on attention mechanism |
| Publication Number | Publication Date |
|---|---|
| CN118350452Atrue CN118350452A (en) | 2024-07-16 |
| CN118350452B CN118350452B (en) | 2025-03-21 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410573353.4AActiveCN118350452B (en) | 2024-05-10 | 2024-05-10 | Federated learning method, client, server and system based on attention mechanism |
| Country | Link |
|---|---|
| CN (1) | CN118350452B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119449404A (en)* | 2024-11-01 | 2025-02-14 | 西南石油大学 | A federated learning method based on lightweight homomorphic encryption to resist poisoning attacks |
| CN119599095A (en)* | 2024-11-15 | 2025-03-11 | 电子科技大学 | Fair federal learning method based on function encryption |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114925848A (en)* | 2022-05-07 | 2022-08-19 | 广东石油化工学院 | Target detection method based on transverse federated learning framework |
| WO2022217914A1 (en)* | 2021-04-14 | 2022-10-20 | 深圳前海微众银行股份有限公司 | Blockchain-based federated learning method and apparatus |
| WO2023012230A2 (en)* | 2021-08-06 | 2023-02-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Generative adversarial-based attack in federated learning |
| WO2023077627A1 (en)* | 2021-11-03 | 2023-05-11 | 深圳前海微众银行股份有限公司 | Blockchain-based privacy protection scheme aggregation method and apparatus |
| CN116451804A (en)* | 2023-03-28 | 2023-07-18 | 上海同态信息科技有限责任公司 | A federated learning method and related equipment based on homomorphic encryption |
| CN116467691A (en)* | 2023-03-28 | 2023-07-21 | 中国电子科技集团公司第三十研究所 | Multi-Biometric Feature Fusion Authentication Method Based on Federated Learning and Attention Mechanism |
| CN116523080A (en)* | 2023-03-22 | 2023-08-01 | 贵州大学 | A privacy-preserving federated learning method, system and device |
| CN117350373A (en)* | 2023-11-30 | 2024-01-05 | 艾迪恩(山东)科技有限公司 | Personalized federal aggregation algorithm based on local self-attention mechanism |
| CN117875455A (en)* | 2024-03-08 | 2024-04-12 | 南京信息工程大学 | A data poisoning defense method for federated learning based on data enhancement |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022217914A1 (en)* | 2021-04-14 | 2022-10-20 | 深圳前海微众银行股份有限公司 | Blockchain-based federated learning method and apparatus |
| WO2023012230A2 (en)* | 2021-08-06 | 2023-02-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Generative adversarial-based attack in federated learning |
| WO2023077627A1 (en)* | 2021-11-03 | 2023-05-11 | 深圳前海微众银行股份有限公司 | Blockchain-based privacy protection scheme aggregation method and apparatus |
| CN114925848A (en)* | 2022-05-07 | 2022-08-19 | 广东石油化工学院 | Target detection method based on transverse federated learning framework |
| CN116523080A (en)* | 2023-03-22 | 2023-08-01 | 贵州大学 | A privacy-preserving federated learning method, system and device |
| CN116451804A (en)* | 2023-03-28 | 2023-07-18 | 上海同态信息科技有限责任公司 | A federated learning method and related equipment based on homomorphic encryption |
| CN116467691A (en)* | 2023-03-28 | 2023-07-21 | 中国电子科技集团公司第三十研究所 | Multi-Biometric Feature Fusion Authentication Method Based on Federated Learning and Attention Mechanism |
| CN117350373A (en)* | 2023-11-30 | 2024-01-05 | 艾迪恩(山东)科技有限公司 | Personalized federal aggregation algorithm based on local self-attention mechanism |
| CN117875455A (en)* | 2024-03-08 | 2024-04-12 | 南京信息工程大学 | A data poisoning defense method for federated learning based on data enhancement |
| Title |
|---|
| FAN YITAO, JI MINGTAO, QIAN ZHUZHONG,: "Accelerating Federated Learning with Adaptive Extra Local Updates upon Edge Networks", 2023 IEEE 29TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS (ICPADS), 25 April 2024 (2024-04-25), pages 2588 - 2595* |
| 周俊;方国英;吴楠;: "联邦学习安全与隐私保护研究综述", 西华大学学报(自然科学版), no. 04, 10 July 2020 (2020-07-10), pages 9 - 17* |
| 杨丽，朱凌波，于越明，苗银宾，: "联邦学习与攻防对抗综述", 信息网络安全, 31 December 2023 (2023-12-31), pages 69 - 90* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119449404A (en)* | 2024-11-01 | 2025-02-14 | 西南石油大学 | A federated learning method based on lightweight homomorphic encryption to resist poisoning attacks |
| CN119449404B (en)* | 2024-11-01 | 2025-09-30 | 西南石油大学 | A federated learning method resistant to poisoning attacks based on lightweight homomorphic encryption |
| CN119599095A (en)* | 2024-11-15 | 2025-03-11 | 电子科技大学 | Fair federal learning method based on function encryption |
| Publication number | Publication date |
|---|---|
| CN118350452B (en) | 2025-03-21 |
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | A training-integrity privacy-preserving federated learning scheme with trusted execution environment | |
| US12309127B2 (en) | End-to-end secure operations using a query vector | |
| JP7388445B2 (en) | Neural network update method, terminal device, calculation device and program | |
| CN110719158A (en) | Edge calculation privacy protection system and method based on joint learning | |
| CN111428887B (en) | Model training control method, device and system based on multiple computing nodes | |
| JP7152909B2 (en) | Systems and methods for secure two-party evaluation of data-sharing utility | |
| US20150381349A1 (en) | Privacy-preserving ridge regression using masks | |
| CN118350452B (en) | Federated learning method, client, server and system based on attention mechanism | |
| CN114223175A (en) | Generate sequences of network data while preventing acquisition or manipulation of temporal data | |
| CN113922944A (en) | Quantum homomorphism encryption and decryption method based on multi-value single quantum state | |
| Zhang et al. | A hybrid encryption approach for efficient and secure data transmission in IoT devices | |
| Nita et al. | Homomorphic encryption | |
| Zhang et al. | PQSF: post-quantum secure privacy-preserving federated learning | |
| Liu et al. | ESA-FedGNN: Efficient secure aggregation for federated graph neural networks | |
| Moazzam et al. | Evolution and advancement of quantum computing in the era of networking and cryptography | |
| Sun et al. | FL-EASGD: Federated Learning Privacy Security Method Based on Homomorphic Encryption. | |
| CN118509157B (en) | Privacy protection federal learning method for resisting back door attack | |
| Kokare et al. | Post quantum cryptography: A survey of past and future | |
| CN112637233B (en) | Safe averaging method based on multi-user data | |
| Filaly et al. | Hybrid Encryption Algorithm for Information Security in Hadoop | |
| Wang et al. | Secret sharing scheme with dynamic size of shares for distributed storage system | |
| WO2022244129A1 (en) | Management apparatus, financial institution terminal, and management method | |
| Sun et al. | Gradient privacy-preserving in federated learning via proxy re-encryption | |
| Correia et al. | Federated Learning: An approach with Hybrid Homomorphic Encryption | |
| Reddy et al. | Multitier security for IoMT using HECC with DSA in cloud environment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |