Movatterモバイル変換


[0]ホーム

URL:


CN114764707B - Federated learning model training method and system - Google Patents

Federated learning model training method and system
Download PDF

Info

Publication number
CN114764707B
CN114764707BCN202110001486.0ACN202110001486ACN114764707BCN 114764707 BCN114764707 BCN 114764707BCN 202110001486 ACN202110001486 ACN 202110001486ACN 114764707 BCN114764707 BCN 114764707B
Authority
CN
China
Prior art keywords
user
organization
training
model parameters
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110001486.0A
Other languages
Chinese (zh)
Other versions
CN114764707A (en
Inventor
刘诗阳
史家康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
Research Institute of China Mobile Communication Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
Research Institute of China Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, Research Institute of China Mobile Communication Co LtdfiledCriticalChina Mobile Communications Group Co Ltd
Priority to CN202110001486.0ApriorityCriticalpatent/CN114764707B/en
Publication of CN114764707ApublicationCriticalpatent/CN114764707A/en
Application grantedgrantedCritical
Publication of CN114764707BpublicationCriticalpatent/CN114764707B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Classifications

Landscapes

Abstract

Translated fromChinese

本发明提供一种联邦学习模型训练方法和系统,该方法包括:创建区块链网络,以用于联邦学习模型的训练;在每一轮联邦学习模型的训练过程中,接收组织用户上传的本地训练模型参数,组织用户为在本地使用本地用户数据对联邦学习模型进行训练的区块链用户;将所有组织用户上传的本地训练模型参数发送给工人用户,由工人用户利用所有本地训练模型参数对联邦学习模型进行联合训练;采用成功创建区块的工人用户上传的联合训练模型参数更新联邦学习模型;将联合训练模型参数发送给组织用户以更新组织用户的本地训练模型参数。本发明中,通过引入除数据提供方之外的其他参与方用户参与联邦学习模型的训练,从而扩大了用户范围。

The present invention provides a method and system for training a federated learning model, the method comprising: creating a blockchain network for training a federated learning model; receiving local training model parameters uploaded by an organization user during each round of training of the federated learning model, the organization user being a blockchain user who trains the federated learning model locally using local user data; sending the local training model parameters uploaded by all organization users to worker users, who jointly train the federated learning model using all local training model parameters; updating the federated learning model using the joint training model parameters uploaded by the worker users who successfully created the block; and sending the joint training model parameters to the organization user to update the local training model parameters of the organization user. In the present invention, by introducing other participating users other than the data provider to participate in the training of the federated learning model, the user range is expanded.

Description

Federal learning model training method and system
Technical Field
The embodiment of the invention relates to the technical field of business support, in particular to a federal learning model training method and system.
Background
Federal learning (FEDERATED LEARNING) is an emerging artificial intelligence basic technology for developing efficient machine learning among multiple parties or computing nodes on the premise of guaranteeing information security during large data exchange, protecting terminal data and personal data privacy, and guaranteeing legal compliance. Currently, some federal learning models are trained by federal learning.
However, the current method of training a federal learning model through federal learning has problems in that federal learning participants are limited to data providers, resulting in a small user scope and low training efficiency.
Disclosure of Invention
The embodiment of the invention provides a federal learning model training method and a federal learning model training system, which are used for solving the problems of small user range and low training efficiency of the existing federal learning model training method.
In order to solve the technical problems, the invention is realized as follows:
In a first aspect, an embodiment of the present invention provides a federal learning model training method, including:
Creating a blockchain network for training of a federal learning model;
in the training process of each round of federation learning model, receiving local training model parameters uploaded by an organization user, wherein the organization user is a blockchain user for training the federation learning model by using local user data locally;
Transmitting the local training model parameters uploaded by all the organization users to worker users, and carrying out joint training on a federal learning model by the worker users by utilizing the local training model parameters uploaded by all the organization users;
Updating the federal learning model by adopting joint training model parameters uploaded by the worker users who successfully create the blocks;
and sending the combined training model parameters to the organization user to update the local training model parameters of the organization user.
Optionally, the local training model parameters uploaded by the organization user are encrypted by adopting a homomorphic encryption algorithm.
Optionally, the organization user adopts a customized key to homomorphic encrypt the local training model parameters.
Optionally, during each round of training of the federal learning model, the block link points of the block chain network receive local training model parameters uploaded by the organization user, including:
When receiving the local training model parameters uploaded by the organization user, judging whether the organization user uploads encryption certificates at the same time, wherein the encryption certificates are issued by any N endorsement nodes of the blockchain network based on zero knowledge certificates;
And if the organization user does not send the encryption certificate, judging that the organization user is a malicious party.
Optionally, the method further comprises:
Before training of a federation learning model, receiving additional samples uploaded by an organization user to form an additional sample library, wherein the additional samples comprise a plurality of input samples and output sample pairs, the input samples are generated by the organization user by utilizing a random algorithm, and the output samples are output obtained by adding the input samples into local user data of the organization user to train the federation learning model;
Receiving local training model parameters uploaded by an organization user;
Randomly extracting K samples from the extra sample library by any N endorsement nodes of the blockchain network, and judging the accuracy of the local training model parameters according to the K samples;
and if the average accuracy of the N endorsement nodes is higher than a preset threshold, issuing encryption evidence for the organization user.
Optionally, the method further comprises:
After creating the blockchain network, assigning an initial asset to each user;
After the converged federal learning model is obtained, each user is subjected to asset settlement based on transactions that occur in each round.
Optionally, before the asset settlement for each user according to the transaction occurring in each round, the method further comprises:
After a user joins the blockchain network, deploying an intelligent contract set on a blockchain node of the user, wherein the intelligent contract set comprises at least one of a state query contract, a model update contract and a model download contract if the user is an organization user;
when the state query contract is called by a user, submitting a query transaction to query the current remaining available assets of the user side;
When the model updating contract is called by the organization user, uploading the local training model parameters of the organization user to the blockchain network to complete one model parameter updating transaction;
Downloading all local training model parameters uploaded by the organization users from the blockchain network when the model downloading contracts are called by the worker users, and downloading joint training model parameters uploaded by the worker users from the blockchain network when the model downloading contracts are called by the organization users so as to complete one-time model parameter downloading transaction;
when the model processing contract is called by the worker user, the combined training model parameters are uploaded to the blockchain network so as to finish one-time processing model parameter transaction.
In a second aspect, an embodiment of the present invention provides a federal learning model training system, including:
the creation module is used for creating a blockchain network for training of the federal learning model;
The first receiving module is used for receiving local training model parameters uploaded by an organization user in the training process of each round of federal learning models, wherein the organization user is a blockchain user for training the federal learning models by using local user data locally;
The first sending module is used for sending the local training model parameters uploaded by all the organization users to the worker users, and the worker users jointly train the federal learning model by utilizing the local training model parameters uploaded by all the organization users;
The updating module is used for updating the federal learning model by adopting the joint training model parameters uploaded by the worker users who successfully create the blocks;
and the second sending module is used for sending the combined training model parameters to the organization user so as to update the local training model parameters of the organization user.
Optionally, the local training model parameters uploaded by the organization user are encrypted by adopting a homomorphic encryption algorithm.
Optionally, the organization user adopts a customized key to homomorphic encrypt the local training model parameters.
Optionally, the system further comprises:
And the encryption module is used for judging whether the organization user uploads encryption proof at the same time when receiving the local training model parameters uploaded by the organization user, wherein the encryption proof is issued by any N endorsement nodes of the blockchain network based on zero knowledge proof, and judging that the organization user is a malicious party if the organization user does not send the encryption proof.
Optionally, the system further comprises:
The first receiving module is used for receiving additional samples sent by an organization user before training of the federation learning model, forming an additional sample library, wherein the additional samples comprise a plurality of input samples and output sample pairs, the input samples are generated by the organization user by utilizing a random algorithm, and the output samples are output obtained by adding the input samples into local user data of the organization user to train the federation learning model;
the second receiving module is used for receiving the local training model parameters uploaded by the organization user;
The judging module is used for randomly extracting K samples from the additional sample library and judging the accuracy of the local training model parameters according to the K samples;
And the proving module is used for providing encryption proving for the organization user if the average accuracy of the N endorsement nodes is higher than a preset threshold value.
Optionally, the system further comprises:
An initial asset assignment module for assigning an initial asset to each user after creating the blockchain network;
and the settlement module is used for carrying out asset settlement on each user according to the transaction occurring in each round after the converged federal learning model is obtained.
Optionally, the system further comprises:
The contract deployment module is used for deploying an intelligent contract set on a blockchain node of a user after the user joins the blockchain network, wherein the intelligent contract set comprises at least one of a state query contract, a model update contract and a model download contract if the user is an organization user;
when the state query contract is called by a user, submitting a query transaction to query the current remaining available assets of the user side;
When the model updating contract is called by the organization user, uploading the local training model parameters of the organization user to the blockchain network to complete one model parameter updating transaction;
Downloading all local training model parameters uploaded by the organization users from the blockchain network when the model downloading contracts are called by the worker users, and downloading joint training model parameters uploaded by the worker users from the blockchain network when the model downloading contracts are called by the organization users so as to complete one-time model parameter downloading transaction;
when the model processing contract is called by the worker user, the combined training model parameters are uploaded to the blockchain network so as to finish one-time processing model parameter transaction.
In a third aspect, an embodiment of the present invention provides a computing device, including a processor, a memory, and a program stored on the memory and executable on the processor, the program implementing the steps of the federal learning model training method of the first aspect when executed by the processor.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the federal learning model training method of the first aspect described above.
In the embodiment of the invention, the blockchain and federal learning are combined, and users without enough user data or without sharing the user data also have the right of participating in federal learning by introducing other party users except the data provider, so that the user range can be enlarged, the model training efficiency can be effectively improved, and rewards can be obtained by carrying out model training to generate blocks, and the rewards can be used for deducting the model use fee and the like, and are superior to the prior art in the aspects of the user range and the incentive mechanism.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a schematic flow chart of a federal learning model training method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a federal learning model training system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computing device according to an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The block chain technology is a joint innovative application of four technologies including intelligent contracts, point-to-point networks, encryption algorithms and consensus mechanisms. Blockchains are essentially a linked list storage structure based on time stamps, also known as a distributed ledger or a decentralized database, because the nodes of the blockchain form a point-to-point network, exhibiting the property of being decentralized. The block chain decides a writer based on a consensus mechanism, ensures the consistency of node data of the whole network, ensures the safety and reliability of the data based on an encryption algorithm, and simultaneously utilizes intelligent contracts deployed on the nodes to carry out automatic transaction processing. In summary, the blockchain has the characteristics of decentralization, collective maintenance, cross-validation, disclosure transparency, difficult tampering, traceability of data and the like, and is applied to various application scenes which need novel database support, multi-main-body business, non-mutual trust of all main bodies and no trusted third party, and typical examples are digital finance, food and medicine tracing, judicial evidence storage and the like.
The blockchain is used as a decentralized, data encryption and non-tamperable distributed shared database, can provide data confidentiality for federally learned data exchange to ensure user privacy, ensure data safety among all participants, can also ensure a value-driven incentive mechanism of a data consistency blockchain for model training by providing data by multiple participants, and can also increase enthusiasm for providing data among all participants and updating network model parameters.
In the invention, the blockchain and the federal learning are combined, and the Union learning model is trained. The federal learning model may be a business recommendation model, etc.
Referring to fig. 1, an embodiment of the present invention provides a federal learning model training method, including:
Step 11, creating a blockchain network for training of a federal learning model;
in the embodiment of the invention, the participants of the blockchain network are collectively called as blockchain users, and the blockchain users can be divided into organization users and worker users according to the contribution mode.
(A) An organization user refers to a business vendor that owns user data and is willing to use the user data to participate in federal learning. The contribution mode of the organization is a shared sample and participates in the training of the federal learning model. The organization submits updated model parameters each time and can obtain corresponding rewards.
(B) Worker users refer to users who do not have sufficient user data or who possess user data but are reluctant to use their own user data to participate in federal learning, but still wish to use federal learning models, such as small-scale business vendors and the like. The contribution mode of the worker user comprises work such as federal learning model training, model parameter updating and the like, and a new block is generated. If a plurality of worker users compete to complete the work of one block, the worker users who complete the block first and are confirmed by consensus can be rewarded.
In the embodiment of the invention, a plurality of service providers in the same field commonly negotiate a deep learning model and use the deep learning model as a federal learning model. Considering that services provided by a plurality of service providers belong to the same field, the feature fields of the data of the services are often higher in overlap ratio, and the user group overlap ratio is lower. Therefore, a sample-based federal learning method is adopted, and partial data with coincident characteristic fields and non-coincident user samples is taken as a shared sample.
When providing service, the service provider sets a local data collection module in software (such as APP of e-commerce, news and the like) or hardware (such as equipment of home theater video on demand, cloud game service and the like) according to the actual demand of the service, and the local data collection module is used for collecting user data and uploading the user data to a manufacturer server to form a manufacturer local user database.
Step 12, receiving local training model parameters uploaded by an organization user in the training process of each round of federal learning models, wherein the organization user is a blockchain user for training the federal learning models by using local user data locally;
The organization user uses the local user data to train the federation learning model locally, and the local user data is not required to be uploaded, so that the privacy of the user data can be protected, and the leakage of the user data is avoided.
Step 13, transmitting the local training model parameters uploaded by all the organization users to worker users, and carrying out joint training on the federal learning model by the worker users by utilizing the local training model parameters uploaded by all the organization users;
step 14, updating the federal learning model by adopting joint training model parameters uploaded by the worker users who successfully create the blocks;
and step 15, the joint training model parameters are sent to the organization user to update the local training model parameters of the organization user.
In the embodiment of the invention, the blockchain and federal learning are combined, and users without enough user data or without sharing the user data also have the right of participating in federal learning by introducing other party users except the data provider, so that the user range can be enlarged, the model training efficiency can be effectively improved, and rewards can be obtained by carrying out model training to generate blocks, and the rewards can be used for deducting the model use fee and the like, and are superior to the prior art in the aspects of the user range and the incentive mechanism.
The following describes a reward mechanism according to an embodiment of the present invention.
In an embodiment of the present invention, optionally, the method further includes:
After creating the blockchain network, each user is given an initial asset, wherein the participants include organizational users and worker users, the initial asset being used to reflect the costs (monetary costs or data costs) paid by the users to participate in federal learning.
After the converged federal learning model is obtained, each user is subjected to asset settlement based on transactions that occur in each round.
In the embodiment of the present invention, optionally, before the asset settlement is performed for each user according to the transaction occurring in each round, the method further includes:
After the user joins the blockchain network, an intelligent contract set S is deployed on the blockchain node of the user, wherein if the user is an organization user, the intelligent contract set S comprises at least one of a state query contract S1, a model update contract S2 and a model download contract S3, and if the user is a worker user, the intelligent contract set comprises at least one of a state query contract S1, a model processing contract S4 and a model download contract S3, and it is required that the intelligent contract set S has extensibility and adds other intelligent contracts according to actual service requirements besides the contract.
When the state query contract S1 is called by the user, a query transaction is submitted to query the world state on all the distributed account books owned by the participant, i.e. to query the current remaining available assets of the user, i.e. the participant may submit a query transaction with the state query contract S1.
When the model updating contract S2 is called by the organization user, uploading the local training model parameters of the organization user to the blockchain network to complete one model parameter updating transaction, namely, the organization user can upload the encrypted local training model parameters to the blockchain by calling the model updating contract S2. In each iteration of federal learning, all organizational users (organizational users) need to complete model parameter uploading for the worker user to use all model parameters for joint training.
When the model downloading contract S3 is called by the worker user, the local training model parameters uploaded by all organization users are downloaded from the blockchain network, and when the model downloading contract S3 is called by the organization users, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network so as to complete one model parameter downloading transaction, namely, a participant can submit one model parameter downloading transaction by calling the model downloading contract S3.
When the model processing contract S4 is called by the worker user, the combined training model parameters are uploaded to the blockchain network to finish one processing model parameter transaction. Upon completion of the joint training, the worker user who successfully created the block may invoke model processing contract S4 to update the parameters of the federal learning model with the joint training model parameters.
A complete federal learning iteration consists of the following steps:
(a) And training the federal learning model by using the local user data by the organization user to obtain local training model parameters. And calling a model update contract S2 to generate a transaction, encrypting the local training model parameters and uploading the encrypted local training model parameters to a blockchain, namely exchanging the encrypted local training model parameters by all users. For a transaction initiated by multiple users, each user needs to pay for the transaction based on the amount of data that they own. Generally, the larger the amount of data a party has, the less the fee is paid.
(B) The worker user collects model update contracts S2 for transaction proposed by all organization users, calls a model download contract S3 to download all encrypted model parameters, utilizes all model parameters to complete joint training of the federal learning model, and the worker user who successfully creates the block calls a model processing contract S4 to complete parameter update of the model.
(C) The organization user invokes the model download contract S3 to download the combined training model parameters, and updates the local training model parameters after decryption.
So far, a complete federal learning iteration is completed. At this time, one block in the blockchain is successfully created, the worker user who creates the block gets rewarded, and the organization user participating in training completes one update of the local model. At this time, each organization user calculates whether the loss function meets the requirement, if not, the next iteration is continuously initiated, and the three steps (a), (b) and (c) are repeated until the loss function converges.
After all users complete multiple rounds of federal learning iteration to obtain a converged federal learning model, actual asset settlement is performed according to user asset variation caused by transactions occurring in each round of iteration.
Although the organization user performs the training of the federal learning model locally, the model parameters obtained by training contain partial characteristics of the local user data. If a malicious party obtains plaintext model parameters and a predetermined federal learning model, the characteristics of the local user data can be deduced from the plaintext model parameters and the predetermined federal learning model, so that data leakage is caused. Therefore, in the embodiment of the invention, in order to avoid the leakage of the data characteristic information caused by the leakage of the model parameters, optionally, the local training model parameters uploaded by the organization user are encrypted by adopting a homomorphic encryption algorithm. Further optionally, the organization user employs a custom key to homomorphic encrypt the local training model parameters.
Homomorphic encryption is characterized in that homomorphic encrypted data is processed to obtain an output, and the output is decrypted, and the result is the same as the output result obtained by processing unencrypted original data by the same method. Therefore, as a worker user who actually performs joint training, the training task of the model can be completed without decrypting the gradient. The worker user cannot touch any unencrypted data in the whole process, and leakage of characteristic information of local model data is avoided.
In the embodiment of the disclosure, the homomorphic encryption algorithm is utilized to organize the model parameters uploaded by the user, namely, the model parameters are encrypted in one layer, so that information leakage caused by plaintext model parameters is avoided. The data contacted by the worker user during model training are encrypted data, and the worker user cannot decrypt the data, so that the problem that the organization user and the worker user are not mutually trusted is solved.
In the federal learning process, there are multiple participants, and malicious participants may exist in each participant, and the malicious participants obtain the joint training model parameters without paying or paying a small cost by maliciously uploading wrong or forged model parameters, and simultaneously cause the reduction of the precision of the joint training model.
In order to avoid a malicious party attack, in the embodiment of the present invention, in a training process of each round of federal learning model, receiving, by a block link point of the block chain network, local training model parameters uploaded by a user includes:
When receiving the local training model parameters uploaded by the organization user, judging whether the organization user uploads encryption certificates at the same time, wherein the encryption certificates are issued by any N endorsement nodes of the blockchain network based on zero knowledge certificates;
And if the organization user does not send the encryption certificate, judging that the organization user is a malicious party.
In an embodiment of the present invention, optionally, the method further includes:
Before training of a federation learning model, receiving additional samples uploaded by an organization user to form an additional sample library, wherein the additional samples comprise a plurality of input samples and output sample pairs, the input samples are generated by the organization user by utilizing a random algorithm, and the output samples are output obtained by adding the input samples into local user data of the organization user to train the federation learning model;
Receiving local training model parameters uploaded by an organization user;
Randomly extracting K samples from the extra sample library by any N endorsement nodes of the blockchain network, and judging the accuracy of the local training model parameters according to the K samples;
and if the average accuracy of the N endorsement nodes is higher than a preset threshold, issuing encryption evidence for the organization user.
Wherein N is a positive integer greater than 1, and K is a positive integer greater than 1.
That is, prior to the onset of federal learning, the organization user generates additional samples using a random algorithm, and all organization users collectively generate an additional sample library. The data in the additional sample library is automatically generated, independent of any organization user's own user data. Each user needs to add this additional sample library to his own user database as a supplemental training set for model training. In each federal learning iteration, each organization user needs to submit an encryption proof at the same time in the step of uploading the encrypted model parameters by the organization user. The encrypted proof is issued by any N endorsement nodes based on zero knowledge proof. The specific method is that the endorsement node randomly extracts K samples from the extra sample library, and judges the encrypted model parameters uploaded by the organization user. And if the average accuracy of the N endorsement nodes is higher than the threshold lambda, the encrypted model parameters uploaded by the organization user are considered to be correct. The input samples in the samples are input into a federal learning model formed by the encrypted model parameters, output samples are obtained, and the average accuracy of the output samples of the N endorsement nodes is judged.
In the embodiment of the invention, an encryption proving mechanism is provided based on the zero knowledge proving idea. The mechanism can verify the correctness of the encrypted model parameters on the premise of not acquiring the model parameters uploaded by the organization user, prevent malicious parties from uploading error parameters or fake parameters, and solve the problem that malicious attacks possibly faced by the Union study partner are difficult to trace and punish in the prior art.
Referring to fig. 2, an embodiment of the present invention further provides a federal learning model training system 20, including:
a creation module 21 for creating a blockchain network for training of the federal learning model;
The first receiving module 22 is configured to receive, during each round of training of the federal learning model, local training model parameters uploaded by an organization user, where the organization user is a blockchain user that trains the federal learning model locally using local user data;
a first sending module 23, configured to send the local training model parameters uploaded by all the organization users to a worker user, where the worker user performs joint training on the federal learning model by using the local training model parameters uploaded by all the organization users;
an update module 24 for updating the federal learning model using the joint training model parameters uploaded by the worker user who successfully created the block;
A second sending module 25, configured to send the joint training model parameters to the organization user to update local training model parameters of the organization user.
Optionally, the local training model parameters uploaded by the organization user are encrypted by adopting a homomorphic encryption algorithm.
Optionally, the organization user adopts a customized key to homomorphic encrypt the local training model parameters.
Optionally, the method further comprises:
And the encryption module is used for judging whether the organization user uploads encryption proof at the same time when receiving the local training model parameters uploaded by the organization user, wherein the encryption proof is issued by any N endorsement nodes of the blockchain network based on zero knowledge proof, and judging that the organization user is a malicious party if the organization user does not send the encryption proof.
Optionally, the system further comprises:
The first receiving module is used for receiving additional samples sent by an organization user before training of the federation learning model, forming an additional sample library, wherein the additional samples comprise a plurality of input samples and output sample pairs, the input samples are generated by the organization user by utilizing a random algorithm, and the output samples are output obtained by adding the input samples into local user data of the organization user to train the federation learning model;
the second receiving module is used for receiving the local training model parameters uploaded by the organization user;
The judging module is used for randomly extracting K samples from the additional sample library and judging the accuracy of the local training model parameters according to the K samples;
And the proving module is used for providing encryption proving for the organization user if the average accuracy of the N endorsement nodes is higher than a preset threshold value.
Optionally, the system further comprises:
An initial asset assignment module for assigning an initial asset to each user after creating the blockchain network;
and the settlement module is used for carrying out asset settlement on each user according to the transaction occurring in each round after the converged federal learning model is obtained.
Optionally, the system further comprises:
The contract deployment module is used for deploying an intelligent contract set on a blockchain node of a user after the user joins the blockchain network, wherein the intelligent contract set comprises at least one of a state query contract, a model update contract and a model download contract if the user is an organization user;
when the state query contract is called by a user, submitting a query transaction to query the current remaining available assets of the user side;
When the model updating contract is called by the organization user, uploading the local training model parameters of the organization user to the blockchain network to complete one model parameter updating transaction;
Downloading all local training model parameters uploaded by the organization users from the blockchain network when the model downloading contracts are called by the worker users, and downloading joint training model parameters uploaded by the worker users from the blockchain network when the model downloading contracts are called by the organization users so as to complete one-time model parameter downloading transaction;
when the model processing contract is called by the worker user, the combined training model parameters are uploaded to the blockchain network so as to finish one-time processing model parameter transaction.
In summary, the scheme of the invention mainly comprises the following parts:
A federal learning network. Multiple service offerings within the same service domain (e.g., cloud service, video or communication, etc.) provide user data collected from each, and perform deep learning model training locally. Encrypting model parameters of the intermediate model obtained by training, uploading the model parameters to a blockchain network formed by all manufacturers together for joint training, and downloading the model parameters after joint training for local model updating, so that the federal learning model in the service field, such as a service recommendation model, is obtained by repeated iteration, and each manufacturer can use the model for more accurate service recommendation to users. In the whole training process, all user privacy data are stored in the local of a service manufacturer, so that user privacy data leakage caused by sharing samples is avoided.
And the blockchain network is used for guaranteeing that uplink data of all parties of the federal learning participation are not tamperable and traceable. When a malicious party tries to destroy the overall model effect by using wrong parameter updating, the malicious party can be quickly positioned by using a cross-validation mechanism of the blockchain network, so that wrong updating is avoided. In addition, the incentive mechanism inherent in the blockchain network may allow the participants with more contributions to gain additional revenue compensation.
And the encryption proving mechanism is used for guaranteeing that the local data characteristics of each participant are not revealed and avoiding the attack of malicious participants.
The federal learning model training system is used for realizing all functions related to the technical scheme, such as local data acquisition, local modeling, model parameter plus multiparty combined training final model, data uploading block chain, block chain contribution excitation, tracing of malicious attacks possibly faced by the federal learning partner and the like.
In one embodiment, the federal learning model training system comprises a user information management module, a local model training module, a blockchain operation module, a settlement module and an encryption module.
And the user information management module is used for performing basic information management such as user registration, information recording, information modification and the like. The module also has the function authority of classifying the users according to the asset certification (monetary asset or data asset) of the users and limiting the users according to the categories of the users.
And the local model training module is used for completing the tasks of training, updating, maintaining and the like of the local model of the user. The organization user invokes the module to perform preliminary model training by using the user data in the local own database, and updates the local model by using federal learning model parameters after the worker user completes the joint training, and the worker user invokes the module to perform joint training on all the collected encrypted models.
And the blockchain operation module is used for providing the user with the functions of joining or exiting the blockchain by the node, initiating asset transaction, uploading or downloading model parameters, submitting encryption certification, tracing data and the like, and submitting the operation result of the user to the blockchain.
And the encryption module is used for providing the user with a homomorphic encryption function according to the customized secret key and an encryption correctness auditing function based on zero knowledge proof. In the step of uploading the encrypted model parameters, the function of presenting the encrypted certificates is completed by the module.
Referring to fig. 3, an embodiment of the present invention further provides a computing device 30, including a processor 31, a memory 32, and a computer program stored in the memory 32 and capable of running on the processor 31, where the computer program when executed by the processor 31 implements the processes of the above-mentioned federal learning model training method embodiment, and can achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the above-mentioned federal learning model training method embodiment, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a computing device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.

Claims (8)

CN202110001486.0A2021-01-042021-01-04 Federated learning model training method and systemActiveCN114764707B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202110001486.0ACN114764707B (en)2021-01-042021-01-04 Federated learning model training method and system

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202110001486.0ACN114764707B (en)2021-01-042021-01-04 Federated learning model training method and system

Publications (2)

Publication NumberPublication Date
CN114764707A CN114764707A (en)2022-07-19
CN114764707Btrue CN114764707B (en)2025-04-04

Family

ID=82363262

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202110001486.0AActiveCN114764707B (en)2021-01-042021-01-04 Federated learning model training method and system

Country Status (1)

CountryLink
CN (1)CN114764707B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN111125779A (en)*2019-12-172020-05-08山东浪潮人工智能研究院有限公司 A blockchain-based federated learning method and device
CN111368319A (en)*2020-03-042020-07-03西安电子科技大学Block chain-based data security access method in federated learning environment
EP3846062A1 (en)*2020-01-062021-07-07Tata Consultancy Services LimitedMethod and system for processing transactions in a blockchain network
WO2021208721A1 (en)*2020-11-232021-10-21平安科技(深圳)有限公司Federated learning defense method, apparatus, electronic device, and storage medium
CN113645020A (en)*2021-07-062021-11-12北京理工大学 A Consortium Chain Privacy Protection Method Based on Secure Multi-Party Computation
CN114338045A (en)*2022-01-142022-04-12中国人民解放军战略支援部队信息工程大学Information data verifiability safety sharing method and system based on block chain and federal learning
WO2022116421A1 (en)*2020-12-042022-06-09深圳先进技术研究院Federated learning worker node selecting method and system, terminal, and storage medium
CN115361145A (en)*2022-10-192022-11-18北京理工大学 A Consortium Chain Privacy Protection Method Supporting Supervision Based on Zero-Knowledge Proof
CN115599799A (en)*2022-11-302023-01-13中南大学(Cn)Block chain and federal learning fusion method for medical big data
WO2024016548A1 (en)*2022-07-202024-01-25天津科技大学Blockchain-based ai model training method
CN117540376A (en)*2023-11-082024-02-09浙江师范大学Federal learning method and system supporting anti-poisoning and reasoning attack
CN117875456A (en)*2023-12-222024-04-12重庆邮电大学Federal learning method based on CKS homomorphic encryption and Fabric

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
US10270599B2 (en)*2017-04-272019-04-23Factom, Inc.Data reproducibility using blockchains
US11489672B2 (en)*2018-11-062022-11-01International Business Machines CorporationVerification of conditions of a blockchain transaction
CN110262819B (en)*2019-06-042021-02-26深圳前海微众银行股份有限公司 A method and device for updating model parameters of federated learning
US11694110B2 (en)*2019-06-122023-07-04International Business Machines CorporationAggregated machine learning verification for database
CN111598254B (en)*2020-05-222021-10-08深圳前海微众银行股份有限公司 Federated learning modeling method, device and readable storage medium
CN111966698B (en)*2020-07-032023-06-13华南师范大学 A blockchain-based trusted federated learning method, system, device and medium
CN111552986B (en)*2020-07-102020-11-13鹏城实验室Block chain-based federal modeling method, device, equipment and storage medium
CN111914277B (en)*2020-08-072023-09-01平安科技(深圳)有限公司Intersection data generation method and federal model training method based on intersection data
CN112132198B (en)*2020-09-162021-06-04建信金融科技有限责任公司Data processing method, device and system and server

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN111125779A (en)*2019-12-172020-05-08山东浪潮人工智能研究院有限公司 A blockchain-based federated learning method and device
EP3846062A1 (en)*2020-01-062021-07-07Tata Consultancy Services LimitedMethod and system for processing transactions in a blockchain network
CN111368319A (en)*2020-03-042020-07-03西安电子科技大学Block chain-based data security access method in federated learning environment
WO2021208721A1 (en)*2020-11-232021-10-21平安科技(深圳)有限公司Federated learning defense method, apparatus, electronic device, and storage medium
WO2022116421A1 (en)*2020-12-042022-06-09深圳先进技术研究院Federated learning worker node selecting method and system, terminal, and storage medium
CN113645020A (en)*2021-07-062021-11-12北京理工大学 A Consortium Chain Privacy Protection Method Based on Secure Multi-Party Computation
CN114338045A (en)*2022-01-142022-04-12中国人民解放军战略支援部队信息工程大学Information data verifiability safety sharing method and system based on block chain and federal learning
WO2024016548A1 (en)*2022-07-202024-01-25天津科技大学Blockchain-based ai model training method
CN115361145A (en)*2022-10-192022-11-18北京理工大学 A Consortium Chain Privacy Protection Method Supporting Supervision Based on Zero-Knowledge Proof
CN115599799A (en)*2022-11-302023-01-13中南大学(Cn)Block chain and federal learning fusion method for medical big data
CN117540376A (en)*2023-11-082024-02-09浙江师范大学Federal learning method and system supporting anti-poisoning and reasoning attack
CN117875456A (en)*2023-12-222024-04-12重庆邮电大学Federal learning method based on CKS homomorphic encryption and Fabric

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
一种基于区块链与联邦学习的数据隐私保护方法;张桉;《企业科技与发展》;20221031(第10期);全文*

Also Published As

Publication numberPublication date
CN114764707A (en)2022-07-19

Similar Documents

PublicationPublication DateTitle
Singh et al.Cross-domain secure data sharing using blockchain for industrial IoT
CN110599181B (en)Data processing method, device and equipment based on block chain and storage medium
Panda et al.Bitcoin and blockchain: history and current applications
CN109474584B (en)Rule-based block chain network automatic permission joining method
US20230004970A1 (en)Distributed Ledgers with Ledger Entries Containing Redactable Payloads
CN108650270B (en)Data sharing method and system based on alliance chain and incentive mechanism
CN111125779A (en) A blockchain-based federated learning method and device
CN111340494B (en)Asset type consistency evidence generation, transaction and transaction verification method and system
CN114565386B (en) Blockchain escrow transaction method and system for multi-party collaborative privacy protection
EP3376708A1 (en)Anonymous communication system and method for subscribing to said communication system
CN114386043B (en)Decentralised privacy maintenance reputation evaluation method for crowd sensing
Oude Roelink et al.Systematic review: Comparing zk‐SNARK, zk‐STARK, and bulletproof protocols for privacy‐preserving authentication
CN114187006A (en)Block chain supervision-based federal learning method
Zhang et al.SABlockFL: a blockchain-based smart agent system architecture and its application in federated learning
Bhatia et al.WorkerRep: Building trust on crowdsourcing platform using blockchain
CN118133349A (en) Privacy computing method, device, equipment, storage medium and computer program product
CN118214761A (en) Attribute-based encrypted data sharing method, system and device based on blockchain
CN114764707B (en) Federated learning model training method and system
CN113673893A (en) Retirement power battery management method and system
CN118264471A (en) A fair and reliable method for providing cross-domain services of Internet of Vehicles based on blockchain
CN116489637B (en) A privacy-preserving mobile edge computing method for the metaverse
Liu et al.pRate: anonymous star rating with rating secrecy
CN116166743A (en)Digital asset inheritance system and method based on Hyperledger Fabric super ledger
Sekhar et al.An integrated secure scalable blockchain framework for iot communications
PakkiEverything you ever wanted to know about Bitcoin Mixers (But were Afraid to Ask)

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp