CN113487351A - Privacy protection advertisement click rate prediction method, device, server and storage medium - Google Patents
Privacy protection advertisement click rate prediction method, device, server and storage mediumDownload PDFInfo
- Publication number
- CN113487351A CN113487351ACN202110755722.8ACN202110755722ACN113487351ACN 113487351 ACN113487351 ACN 113487351ACN 202110755722 ACN202110755722 ACN 202110755722ACN 113487351 ACN113487351 ACN 113487351A
- Authority
- CN
- China
- Prior art keywords
- client
- cluster
- clients
- model
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0242—Determining effectiveness of advertisements
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
- G06Q30/0269—Targeted advertisements based on user profile or attribute
- G06Q30/0271—Personalized advertisement
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Accounting & Taxation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Evolutionary Biology (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Entrepreneurship & Innovation (AREA)
- Medical Informatics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Databases & Information Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及一种隐私保护广告点击率预测方法、装置、服务器及存储介质，属于用户隐私保护技术领域。The invention relates to a method, device, server and storage medium for predicting the click-through rate of privacy protection advertisements, and belongs to the technical field of user privacy protection.
背景技术Background technique
广告点击率的高效预测，对于提高广告投放的效率有着至关重要的作用。为了给用户提供个性化点击率预测，捕获不同特征之间的交互关系，以此估计用户和广告之间的相关性，工业界和学术界将深度学习引入了这一领域。谷歌公司提出了一种“Wide&Deep”模型，该模型并行通过带有叉积的线性算法与深度神经网络层来进行特征学习，以此捕获特征交互关系来进行广告推荐。Efficient prediction of ad click-through rate plays a vital role in improving the efficiency of ad delivery. In order to provide users with personalized click-through rate predictions and capture the interaction between different features to estimate the correlation between users and advertisements, industry and academia have introduced deep learning into this field. Google proposed a "Wide&Deep" model, which performs feature learning through a linear algorithm with a cross-product and a deep neural network layer in parallel, so as to capture the feature interaction for ad recommendation.
为了更好地捕获特征之间的高阶交互，在“Wide&Deep”的基础上，DeepFM模型了结合分解机和深度神经网络来对特征交互进行建模。与其他广告点击率预测策略相比，DeepFM不仅具有FM的功能，可以在稀疏数据中学习特征的交互关系，而且可以使用深度学习来构建用于特征学习的神经网络。In order to better capture the high-order interactions between features, on the basis of "Wide & Deep", the DeepFM model combines factorization machines and deep neural networks to model feature interactions. Compared with other advertising click-through rate prediction strategies, DeepFM not only has the function of FM, which can learn the interaction relationship of features in sparse data, but also can use deep learning to build a neural network for feature learning.
传统的广告点击率预测在模型训练时会将用户数据直接上传给中心服务器进行集中式训练。用户数据中包含了很多隐私敏感的信息，原始数据不加保护就上传给服务器会造成隐私泄露。The traditional advertisement click rate prediction will upload user data directly to the central server for centralized training during model training. User data contains a lot of privacy-sensitive information, and uploading the original data to the server without protection will cause privacy leakage.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明提供了一种隐私保护广告点击率预测方法、装置、服务器及存储介质，其可以权衡广告点击率预测算法在不同客户端数据非独立同分布场景下的准确度和隐私性，即保持联邦学习模型可用性的同时保护客户端数据的隐私安全。In view of this, the present invention provides a privacy-preserving advertisement click-through rate prediction method, device, server, and storage medium, which can weigh the accuracy and privacy of the advertisement click-through rate prediction algorithm in different client data non-IID scenarios , that is, to maintain the availability of the federated learning model while protecting the privacy and security of client data.
本发明的第一个目的在于提供一种隐私保护广告点击率预测方法。The first object of the present invention is to provide a method for predicting the click-through rate of a privacy-preserving advertisement.
本发明的第三个目的在于提供一种隐私保护广告点击率预测装置。The third object of the present invention is to provide a device for predicting the click-through rate of a privacy-preserving advertisement.
本发明的第三个目的在于提供一种服务器。The third object of the present invention is to provide a server.
本发明的第四个目的在于提供一种存储介质。A fourth object of the present invention is to provide a storage medium.
本发明的第一个目的可以通过采取如下技术方案达到：The first purpose of the present invention can be achieved by adopting the following technical solutions:
一种隐私保护广告点击率预测方法，应用于服务器，所述方法包括：A method for predicting a click-through rate of a privacy protection advertisement, applied to a server, the method comprising:
将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量，并将权重更新向量上传到服务器；Distribute the global model to each client, so that each client trains the local model according to the local user data, obtains the weight update vector by calculating the gradient of the factorization machine component and the deep learning component, and uploads the weight update vector to the server;
接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度；Receive the weight update vector uploaded by each client, and calculate the similarity between each client according to the weight update vector uploaded by each client;
根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型；According to the similarity between each client, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model;
在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，直至全局模型收敛或达到最大轮次；In each cluster, the global model is issued to all clients in the cluster, so that all clients in the cluster update the local model until the global model converges or reaches the maximum round;
接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。A request sent by a client of a certain user is received, and the global model is delivered to the client of the user in the corresponding cluster, so that the client of the user calculates the advertisement click rate of the candidate advertisement of the user through the local model.
进一步的，所述采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型，具体包括：Further, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model, which specifically includes:
采用聚类联邦学习算法，对所有客户端进行聚类，判断是否发生分裂；Clustering federated learning algorithm is used to cluster all clients to determine whether a split occurs;
若发生分裂，则将所有客户端分成两个聚类，使每个聚类生成一个全局模型；If a split occurs, all clients are divided into two clusters, so that each cluster generates a global model;
若不发生分裂，则判断全局模型是否收敛；If no split occurs, judge whether the global model converges;
若全局模型未收敛，且未达到最大轮次，则将所有客户端作为一个聚类，使该聚类生成一个全局模型。If the global model does not converge and the maximum number of rounds is not reached, all clients are regarded as a cluster, and the cluster generates a global model.
进一步的，所述聚类发生分裂是指：当前聚类的联邦学习目标函数的接近驻点，以及聚类中存在某一客户端没有到达本地损失函数的一个稳定点。Further, the cluster splitting refers to: the current clustered federated learning objective function is close to a stagnation point, and there is a stable point in the cluster where a certain client does not reach the local loss function.
进一步的，所述方法还包括：Further, the method also includes:
根据该用户的候选广告的广告点击率，将所选的部分广告列表发送给该用户的客户端，实现对该用户的个性化广告推荐。According to the advertisement click-through rate of the candidate advertisements of the user, the selected partial advertisement list is sent to the client of the user, so as to realize the personalized advertisement recommendation for the user.
进一步的，计算因子分解机组件的梯度，如下式：Further, the gradient of the factorization machine components is calculated as follows:
其中，
进一步的，计算深度学习组件的梯度，如下式：Further, the gradient of the deep learning component is calculated as follows:
其中，t表示迭代轮次，Dk表示第k个客户端的用户数据，SGD()表示随机梯度下降法。Among them, t represents the iteration round, Dk represents the user data of the kth client, and SGD() represents the stochastic gradient descent method.
进一步的，所述计算各个客户端之间的相似度，如下式：Further, the calculation of the similarity between each client is as follows:
其中，αi,j表示第i个客户端与第j个客户端之间的余弦相似度，Δθi表示第i个客户端的权重更新向量，Δθj表示第j个客户端的权重更新向量。Among them, αi,j represents the cosine similarity between the ith client and the j th client, Δθi represents the weight update vector of the ith client, and Δθj represents the weight update vector of the j th client.
本发明的第二个目的可以通过采取如下技术方案达到：The second object of the present invention can be achieved by adopting the following technical solutions:
一种隐私保护广告点击率预测装置，应用于服务器，所述装置包括：A device for predicting a click-through rate of a privacy protection advertisement, applied to a server, the device comprising:
模型训练模块，用于将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量，并将权重更新向量上传到服务器；The model training module is used to send the global model to each client, so that each client can train the local model according to the local user data, obtain the weight update vector by calculating the gradient of the factorization machine component and the deep learning component, and put the weight The update vector is uploaded to the server;
相似度计算模块，用于接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度；The similarity calculation module is used to receive the weight update vector uploaded by each client, and calculate the similarity between each client according to the weight update vector uploaded by each client;
聚类模块，用于根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型；The clustering module is used to cluster all the clients according to the similarity between each client, using the clustering federated learning algorithm, so that each cluster generates a global model;
模型更新模块，用于在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，直至全局模型收敛或达到最大轮次；The model update module is used in each cluster to issue the global model to all clients in the cluster, so that all clients in the cluster update the local model until the global model converges or the maximum round is reached Second-rate;
广告点击率预测模块，用于接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。The advertising click-through rate prediction module is used to receive the request sent by the client of a certain user, and issue the global model to the client of the user in the corresponding cluster, so that the client of the user can calculate the user's client through the local model. The ad CTR of the candidate ad.
本发明的第三个目的可以通过采取如下技术方案达到：The third object of the present invention can be achieved by adopting the following technical solutions:
一种服务器，包括处理器以及用于存储处理器可执行程序的存储器，所述处理器执行存储器存储的程序时，实现上述的隐私保护广告点击率预测方法。A server includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned method for predicting the click-through rate of an advertisement for privacy protection is implemented.
本发明的第四个目的可以通过采取如下技术方案达到：The fourth object of the present invention can be achieved by adopting the following technical solutions:
一种存储介质，存储有程序，所述程序被处理器执行时，实现上述的隐私保护广告点击率预测方法。A storage medium stores a program, and when the program is executed by a processor, the above-mentioned method for predicting the click-through rate of a privacy protection advertisement is realized.
本发明相对于现有技术具有如下的有益效果：The present invention has the following beneficial effects with respect to the prior art:
1、本发明基于联邦因子分解机实现，可以权衡广告点击率预测算法在不同客户端数据非独立同分布场景下的准确度和隐私性，即保持联邦学习的模型可用性的同时保护客户端数据的隐私安全；对现有的集中式的因子分解机进行了优化，引入了联邦因子分解机，具体来说，联邦学习的分布式训练使客户端不用直接将用户原始数据上传给服务器，仅利用客户端的梯度信息更新模型；在因子分解机中引入聚类联邦学习的思想，从而实现能保护用户隐私的广告点击率预测，可以解决用户数据异构性带来的线性聚合模型损失。1. The present invention is implemented based on the federated factorization machine, which can weigh the accuracy and privacy of the advertisement click-through rate prediction algorithm in different client data non-IID scenarios, that is, maintain the availability of the federated learning model while protecting the client data. Privacy and security; the existing centralized factorization machine is optimized, and a federated factorization machine is introduced. Specifically, the distributed training of federated learning enables the client to not directly upload the user's original data to the server, but only uses the client The gradient information update model at the end; the idea of clustering federated learning is introduced into the factorization machine, so as to realize the prediction of advertisement click rate that can protect user privacy, and solve the loss of linear aggregation model caused by the heterogeneity of user data.
2、本发明保持了较高的模型精度，在Tencent2019训练集下，最终全局模型精度相比传统的联邦矩阵分解模型提高8％，比单一全局模型的联邦学习准确率提高了2.5％。2. The present invention maintains a high model accuracy. Under the Tencent2019 training set, the final global model accuracy is 8% higher than the traditional federated matrix decomposition model, and the federated learning accuracy rate of a single global model is increased by 2.5%.
3、本发明增强了广告点击率预测算法的隐私性，为数据异构场景的联邦广告推荐下的隐私保护提供了良好的解决方案。3. The present invention enhances the privacy of the advertisement click rate prediction algorithm, and provides a good solution for the privacy protection under the federal advertisement recommendation in the data heterogeneous scene.
4、本发明设计了一个用户级的分布式因子分解机，可以应用于联邦学习框架，保证了模型训练时用户原始数据不出本地，从而减轻了用户隐私泄露的风险。4. The present invention designs a user-level distributed factorization machine, which can be applied to the federated learning framework to ensure that the user's original data is not localized during model training, thereby reducing the risk of user privacy leakage.
5、本发明采用了聚类联邦学习的机制，提高了客户端数据异构时的广告点击率预测算法的准确性。5. The present invention adopts the mechanism of clustering federated learning, which improves the accuracy of the advertisement click rate prediction algorithm when the client data is heterogeneous.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图示出的结构获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained according to the structures shown in these drawings without creative efforts.
图1为本发明实施例1的隐私保护广告点击率预测框架的原理图。FIG. 1 is a schematic diagram of a privacy-preserving advertisement click-through rate prediction framework according to Embodiment 1 of the present invention.
图2为本发明实施例1的隐私保护广告点击率预测方法的流程图。FIG. 2 is a flowchart of a method for predicting a click-through rate of a privacy-preserving advertisement according to Embodiment 1 of the present invention.
图3为本发明实施例1的对所有客户端进行聚类的流程图。FIG. 3 is a flowchart of clustering all clients according to Embodiment 1 of the present invention.
图4为本发明实施例2的广告点击率预测装置的结构框图。FIG. 4 is a structural block diagram of an apparatus for predicting an advertisement click rate according to Embodiment 2 of the present invention.
图5为本发明实施例3的服务器的结构框图。FIG. 5 is a structural block diagram of a server according to Embodiment 3 of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部的实施例，基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention. .
实施例1：Example 1:
为了保护用户隐私，将联邦学习框架引入广告点击率预测，联邦学习的安全聚合策略学习客户端的模型参数，同时确保原始数据不出本地，由于用户数据的异构性，简单的模型聚合策略(如FedSGD、FedAvg)会导致性能降低，甚至在用户数据极度Non-IID的情况下导致模型发散。In order to protect user privacy, the federated learning framework is introduced into advertisement click rate prediction. The secure aggregation strategy of federated learning learns the model parameters of the client, while ensuring that the original data is not local. Due to the heterogeneity of user data, simple model aggregation strategies (such as FedSGD, FedAvg) can lead to performance degradation and even model divergence when user data is extremely Non-IID.
面对联邦场景下的数据异构的挑战，客户端共享的数据子集、在目标函数中加入近端项等的安全聚合策略能够帮助解决上述挑战。然而，由于此类计算的时间开销较大，技术的实用性并不高。Sattler等人提出的聚类损失项，使用余弦相似度来克服客户端具有不同数据分布时造成的模型发散问题。在此基础上，本实施例提供了一种基于联邦因子分解机的广告点击率预测框架，该广告点击率预测框架用多中心化的联邦学习来减少数据异构对模型训练的影响。Faced with the challenges of data heterogeneity in federated scenarios, security aggregation strategies such as data subsets shared by clients and adding near-end items to the objective function can help solve the above challenges. However, due to the high time overhead of such computations, the practicality of the technique is not high. The clustering loss term proposed by Sattler et al. uses cosine similarity to overcome the model divergence problem caused when clients have different data distributions. On this basis, this embodiment provides an advertisement click-through rate prediction framework based on a federated factorization machine, and the advertisement click-through rate prediction framework uses multi-center federated learning to reduce the impact of data heterogeneity on model training.
如图1所示，本实施例的隐私保护广告点击率预测框架包括两个部分，第一个部分是广告平台，采用服务器实现，它存在多个聚类，在同一聚类的客户端数据分布相同，通过聚类中的全局模型来预测用户的点击概率；第二个部分是用户的客户端，它主要在本地收集、分析用户数据并将模型渐变上传到广告平台的相应聚类。As shown in Figure 1, the privacy protection advertisement click-through rate prediction framework of this embodiment includes two parts. The first part is the advertisement platform, which is implemented by the server. It has multiple clusters, and the client data distribution in the same cluster In the same way, the user's click probability is predicted through the global model in the clustering; the second part is the user's client, which mainly collects and analyzes user data locally and uploads the model gradient to the corresponding clustering of the advertising platform.
在广告平台中，假设有一组候选广告，它们表示为D＝[d1,d2,...,dm]，广告平台中有多个广告聚类，由相似的用户组成，可以从广告的ID，标题等特征中学习广告模型，当用户u的客户端向广告平台发送请求时，广告平台将在相应聚类中计算用户u的候选广告的广告点击率，分别表示为[y1,y2,...,ym]，并将所选的部分广告列表发送给用户，来实现对用户u的个性化广告推荐；在客户端中，本地模型训练的用户特征包含用户个人信息和广告点击行为数据，在本地使用用户数据来训练本地模型，并将该本地模型的梯度发送到广告平台以更新全局模型。In the advertising platform, suppose there is a set of candidate advertisements, which are represented as D=[d1,d2,...,dm]. There are multiple advertisement clusters in the advertising platform, which are composed of similar users, which can be obtained from the ID of the advertisement, When the client of user u sends a request to the advertising platform, the advertising platform will calculate the advertisement click rate of the candidate advertisement of user u in the corresponding cluster, which are respectively expressed as [y1, y2, .. .,ym], and send the selected part of the advertisement list to the user to implement personalized advertisement recommendation for user u; in the client, the user characteristics trained by the local model include the user's personal information and advertisement click behavior data. User data is used locally to train a local model, and the gradients of this local model are sent to the advertising platform to update the global model.
如图1和图2所示，本实施例提供了一种隐私保护广告点击率预测方法，该方法基于上述隐私保护广告点击率预测框架的广告平台(即服务器)实现，包括以下步骤：As shown in FIG. 1 and FIG. 2 , the present embodiment provides a method for predicting the click-through rate of privacy-preserving advertisements. The method is implemented based on the advertising platform (ie, the server) of the above-mentioned privacy-preserving advertisement click-through rate prediction framework, and includes the following steps:
S201、将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量，并将权重更新向量上传到服务器。S201. Distribute the global model to each client, so that each client trains the local model according to local user data, obtains the weight update vector by calculating the gradients of the factorization machine component and the deep learning component, and uploads the weight update vector to server.
本实施例中，全局模型为需要训练的模型，由于模型训练需要多个轮次，在第一个轮次中，需要训练的模型为初始全局模型，在后续的每个训练轮次中，需要训练的模型为上一轮训练得到的全局模型。In this embodiment, the global model is the model that needs to be trained. Since model training requires multiple rounds, in the first round, the model to be trained is the initial global model. The trained model is the global model obtained from the previous round of training.
在服务器将全局模型下发至各个客户端后，各个客户端接收到全局模型，根据本地用户数据训练本地模型，从而获得权重更新向量，并将权重更新向量上传到服务器，使服务器通过计算获得新一轮的全局模型；进一步地，本地用户数据的用户特征包括用户个人信息和广告点击行为数据，由于原始用户数据的用户特征非常稀疏，因此需要通过一个嵌入层将其转换成密集向量才能进一步学习低阶和高阶特征交互，然后根据服务器发送的统一映射获取新的连续向量；更进一步地，与集中式的深度因子分解机(DeepFM)模型预估相比，分布式场景下的模型预估公式更为复杂，采用链式计算法则，通过分别计算因子分解机(Factorization Machine，简称FM)组件和深度学习组件的梯度来获得每个客户端的权重更新向量，其中深度学习组件为深度神经网络(Deep Neural Networks，简称DNN)组件。After the server sends the global model to each client, each client receives the global model, trains the local model according to the local user data, thereby obtains the weight update vector, and uploads the weight update vector to the server, so that the server obtains the new weight through calculation. One round of global model; further, the user features of the local user data include user personal information and advertisement click behavior data. Since the user features of the original user data are very sparse, it needs to be converted into a dense vector through an embedding layer for further learning. Low-order and high-order features interact, and then obtain a new continuous vector according to the unified map sent by the server; further, compared with the centralized deep factorization machine (DeepFM) model prediction, the model prediction in distributed scenarios The formula is more complicated. The chain calculation rule is used to obtain the weight update vector of each client by separately calculating the gradients of the Factorization Machine (FM) component and the deep learning component. The deep learning component is a deep neural network ( Deep Neural Networks, DNN for short) component.
计算因子分解机组件的梯度，如下式：Calculate the gradient of the factorization machine components as follows:
其中，
计算深度学习组件的梯度，每个客户端执行多次随机梯度下降来迭代更新本地模型，第t轮第k个客户端的梯度如下式：The gradient of the deep learning component is calculated, and each client performs multiple stochastic gradient descents to iteratively update the local model. The gradient of the k-th client in the t-th round is as follows:
其中，t表示迭代轮次，Dk表示第k个客户端的用户数据，SGD()表示随机梯度下降法。Among them, t represents the iteration round, Dk represents the user data of the kth client, and SGD() represents the stochastic gradient descent method.
通过计算得到的因子分解机组件和深度学习组件的梯度即为权重更新向量。The gradient of the factorization machine component and the deep learning component obtained by calculation is the weight update vector.
S202、接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度。S202. Receive the weight update vector uploaded by each client, and calculate the similarity between each client according to the weight update vector uploaded by each client.
本实施例中，各个客户端之间的相似度为余弦相似度，如下式：In this embodiment, the similarity between each client is the cosine similarity, as follows:
其中，αi,j表示第i个客户端与第j个客户端之间的余弦相似度，Δθi表示第i个客户端的权重更新向量，Δθj表示第j个客户端的权重更新向量。Among them, αi,j represents the cosine similarity between the ith client and the j th client, Δθi represents the weight update vector of the ith client, and Δθj represents the weight update vector of the j th client.
S203、根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型。S203. According to the similarity between each client, a clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model.
进一步地，该步骤S203如图3所示，具体包括：Further, this step S203, as shown in FIG. 3, specifically includes:
S2031、采用聚类联邦学习算法，对所有客户端进行聚类，判断是否发生分裂，若发生分裂，则进入步骤S2032，若不发生分裂，则进入步骤S2033。S2031. Use the clustering federated learning algorithm to cluster all the clients to determine whether a split occurs. If a split occurs, go to step S2032, and if no split occurs, go to step S2033.
本实施例中，通过观察客户端在一个固定点(驻点)θ*梯度变化，当聚类内的数据分布不一致时，聚类中联邦学习目标函数的平稳解在单个客户中不可能是平稳的；反之，如果数据分布一致，聚类中的目标函数优化将能够能到所有客户的局部风险函数的最优解。因此，在目标函数接近平稳点时，客户的梯度的范数将趋近于零，因此，发生分裂通过以下两点确定：In this embodiment, by observing the gradient change of the client at a fixed point (stagnation point) θ* , when the data distribution within the cluster is inconsistent, the stationary solution of the federated learning objective function in the cluster cannot be stationary in a single client On the contrary, if the data distribution is consistent, the optimization of the objective function in the clustering will be able to reach the optimal solution of the local risk function of all customers. Therefore, as the objective function approaches a stationary point, the norm of the customer's gradient will approach zero, so the split occurs is determined by the following two points:
(1)当前聚类的联邦学习目标函数的接近驻点θ*，如下式：(1) The approximate stagnation point θ* of the federated learning objective function of the current cluster is as follows:
其中，Di表示第i个客户端的用户数据，ε1表示超参数，具体取值通过实验确定，gk()表示第k个客户端的目标函数。Among them, Di represents the user data of the ith client, ε1 represents the hyperparameter, and the specific value is determined through experiments, and gk () represents the objective function of the k th client.
(2)聚类中存在某一客户端没有到达本地损失函数的一个稳定点，如下式：(2) There is a stable point where a client does not reach the local loss function in the clustering, as follows:
maxi＝1,...,M||gk(θ*)||＞ε2＞0 (5)maxi=1,...,M ||gk (θ* )||>ε2 >0 (5)
其中，ε2表示超参数，具体取值通过实验确定，gk()表示第k个客户端的目标函数。Among them, ε2 represents the hyperparameter, and the specific value is determined through experiments, and gk () represents the objective function of the kth client.
S2032、将所有客户端分成两个聚类，使每个聚类生成一个全局模型。S2032: Divide all clients into two clusters, so that each cluster generates a global model.
本实施例中，聚类联邦学习自上而下递归将所有客户端分成两个聚类，可以使不同聚类的客户端之间的最大相似性最小化。In this embodiment, cluster federated learning recursively divides all clients into two clusters from top to bottom, which can minimize the maximum similarity between clients in different clusters.
S2033、判断全局模型是否收敛，若全局模型未收敛，且未达到最大轮次，则进入步骤S2034，若全局模型收敛或达到最大轮次，则结束模型训练，模型训练结束后可以进入步骤S205。S2033. Determine whether the global model has converged. If the global model has not converged and the maximum number of rounds has not been reached, proceed to step S2034. If the global model has converged or reached the maximum number of rounds, the model training is ended. After the model training is completed, step S205 can be entered.
S2034、将所有客户端作为一个聚类，使该聚类生成一个全局模型，进入步骤S204。S2034 , take all clients as a cluster, and make the cluster generate a global model, and then go to step S204 .
S204、在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，将权重更新向量上传到服务器，返回步骤S202，直至全局模型收敛或达到最大轮次。S204, in each cluster, issue the global model to all clients in the cluster, so that all clients in the cluster update the local model, upload the weight update vector to the server, and return to step S202, Until the global model converges or the maximum number of rounds is reached.
S205、接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。S205: Receive a request sent by a client of a certain user, and issue the global model to the client of the user in the corresponding cluster, so that the client of the user can calculate the advertisement click rate of the candidate advertisement of the user through the local model .
本实施例中，某个用户的客户端向服务器发送请求，服务器在相应聚类中将全局模型下发给该用户的客户端，该用户的客户端接收到全局模型后，使用本地存储的原始用户数据(用户个人信息和点击数据)，通过本地模型计算该用户的候选广告的广告点击率，从而实现个性化点击率预测。In this embodiment, the client of a certain user sends a request to the server, and the server delivers the global model to the client of the user in the corresponding cluster. After receiving the global model, the client of the user uses the original stored locally User data (user personal information and click data), the advertisement click rate of the candidate advertisements of the user is calculated by the local model, so as to realize the personalized click rate prediction.
S206、根据该用户的候选广告的广告点击率，将所选的部分广告列表发送给该用户的客户端，实现对该用户的个性化广告推荐。S206: Send the selected partial advertisement list to the client of the user according to the advertisement click-through rate of the candidate advertisement of the user, so as to implement personalized advertisement recommendation for the user.
本领域技术人员可以理解，实现上述实施例的方法中的全部或部分步骤可以通过程序来指令相关的硬件来完成，相应的程序可以存储于计算机可读存储介质中。Those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.
应当注意，尽管在附图中以特定顺序描述了上述实施例的方法操作，但是这并非要求或者暗示必须按照该特定顺序来执行这些操作，或是必须执行全部所示的操作才能实现期望的结果。相反，描绘的步骤可以改变执行顺序。附加地或备选地，可以省略某些步骤，将多个步骤合并为一个步骤执行，和/或将一个步骤分解为多个步骤执行。It should be noted that although the method operations of the above-described embodiments are depicted in a particular order in the drawings, this does not require or imply that the operations must be performed in that particular order, or that all illustrated operations must be performed to achieve the desired results . Conversely, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined to be performed as one step, and/or one step may be decomposed into multiple steps to be performed.
实施例2：Example 2:
如图4所示，本实施例提供了一种隐私保护广告点击率预测装置，该装置应用于服务器，包括模型训练模块401、相似度计算模块402，聚类模块403、模型更新模块404、广告点击率预测模块405和广告推荐模块406，各个模块的具体功能如下：As shown in FIG. 4 , this embodiment provides an apparatus for predicting the click-through rate of privacy-preserving advertisements. The apparatus is applied to a server and includes a
模型训练模块401，用于将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量。The
相似度计算模块402，用于接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度；The
聚类模块403，用于根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型；The
模型更新模块404，用于在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，直至全局模型收敛或达到最大轮次；The
广告点击率预测模块405，用于接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。The advertisement click-through
广告推荐模块406，用于根据该用户的候选广告的广告点击率，将所选的部分广告列表发送给该用户的客户端，实现对该用户的个性化广告推荐。The
本实施例中各个模块的具体实现可以参见上述实施例1，在此不再一一赘述；需要说明的是，本实施例提供的装置仅以上述各功能模块的划分进行举例说明，在实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。For the specific implementation of each module in this embodiment, reference may be made to the above-mentioned Embodiment 1, which will not be repeated here. It should be noted that the device provided in this embodiment is only illustrated by the division of the above-mentioned functional modules, and in practical applications , the above-mentioned function distribution can be completed by different function modules according to the needs, that is, the internal structure is divided into different function modules, so as to complete all or part of the functions described above.
实施例3：Example 3:
如图5所示，本实施例提供了一种服务器，其通过系统总线501连接的处理器502、存储器和网络接口503，该处理器用于提供计算和控制能力，该存储器包括非易失性存储介质504和内存储器505，该非易失性存储介质504存储有操作系统、计算机程序和数据库，该内存储器505为非易失性存储介质中的操作系统和计算机程序的运行提供环境，处理器502执行存储器存储的计算机程序时，实现上述实施例1的隐私保护广告点击率预测方法，如下：As shown in FIG. 5 , the present embodiment provides a server with a
将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量，并将权重更新向量上传到服务器；Distribute the global model to each client, so that each client trains the local model according to the local user data, obtains the weight update vector by calculating the gradient of the factorization machine component and the deep learning component, and uploads the weight update vector to the server;
接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度；Receive the weight update vector uploaded by each client, and calculate the similarity between each client according to the weight update vector uploaded by each client;
根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型；According to the similarity between each client, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model;
在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，直至全局模型收敛或达到最大轮次；In each cluster, the global model is issued to all clients in the cluster, so that all clients in the cluster update the local model until the global model converges or reaches the maximum round;
接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。A request sent by a client of a certain user is received, and the global model is delivered to the client of the user in the corresponding cluster, so that the client of the user calculates the advertisement click rate of the candidate advertisement of the user through the local model.
进一步地，所述采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型，具体包括：Further, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model, which specifically includes:
采用聚类联邦学习算法，对所有客户端进行聚类，判断是否发生分裂；Clustering federated learning algorithm is used to cluster all clients to determine whether a split occurs;
若发生分裂，则将所有客户端分成两个聚类，使每个聚类生成一个全局模型；If a split occurs, all clients are divided into two clusters, so that each cluster generates a global model;
若不发生分裂，则判断全局模型是否收敛；If no split occurs, judge whether the global model converges;
若全局模型未收敛，且未达到最大轮次，则将所有客户端作为一个聚类，使该聚类生成一个全局模型。If the global model does not converge and the maximum number of rounds is not reached, all clients are regarded as a cluster, and the cluster generates a global model.
进一步地，所述方法还可包括：Further, the method may also include:
根据该用户的候选广告的广告点击率，将所选的部分广告列表发送给该用户的客户端，实现对该用户的个性化广告推荐。According to the advertisement click-through rate of the candidate advertisements of the user, the selected partial advertisement list is sent to the client of the user, so as to realize the personalized advertisement recommendation for the user.
实施例4：Example 4:
本实施例提供了一种存储介质，该存储介质为计算机可读存储介质，其存储有计算机程序，所述计算机程序被处理器执行时，实现上述实施例1的隐私保护广告点击率预测方法，如下：This embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program. When the computer program is executed by a processor, the method for predicting the click-through rate of a privacy-preserving advertisement in the foregoing embodiment 1 is implemented, as follows:
将全局模型下发至各个客户端，以使各个客户端根据本地用户数据训练本地模型，分别通过计算因子分解机组件和深度学习组件的梯度获得权重更新向量，并将权重更新向量上传到服务器；Distribute the global model to each client, so that each client trains the local model according to the local user data, obtains the weight update vector by calculating the gradient of the factorization machine component and the deep learning component, and uploads the weight update vector to the server;
接收各个客户端上传的权重更新向量，根据各个客户端上传的权重更新向量，计算各个客户端之间的相似度；Receive the weight update vector uploaded by each client, and calculate the similarity between each client according to the weight update vector uploaded by each client;
根据各个客户端之间的相似度，采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型；According to the similarity between each client, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model;
在每个聚类中，将全局模型下发给该聚类中的所有客户端，以使该聚类中的所有客户端更新本地模型，直至全局模型收敛或达到最大轮次；In each cluster, the global model is issued to all clients in the cluster, so that all clients in the cluster update the local model until the global model converges or reaches the maximum round;
接收某个用户的客户端发送的请求，在相应聚类中将全局模型下发给该用户的客户端，以使该用户的客户端通过本地模型计算该用户的候选广告的广告点击率。A request sent by a client of a certain user is received, and the global model is delivered to the client of the user in the corresponding cluster, so that the client of the user calculates the advertisement click rate of the candidate advertisement of the user through the local model.
进一步地，所述采用聚类联邦学习算法，对所有客户端进行聚类，使每个聚类生成一个全局模型，具体包括：Further, the clustering federated learning algorithm is used to cluster all clients, so that each cluster generates a global model, which specifically includes:
采用聚类联邦学习算法，对所有客户端进行聚类，判断是否发生分裂；Clustering federated learning algorithm is used to cluster all clients to determine whether a split occurs;
若发生分裂，则将所有客户端分成两个聚类，使每个聚类生成一个全局模型；If a split occurs, all clients are divided into two clusters, so that each cluster generates a global model;
若不发生分裂，则判断全局模型是否收敛；If no split occurs, judge whether the global model converges;
若全局模型未收敛，且未达到最大轮次，则将所有客户端作为一个聚类，使该聚类生成一个全局模型。If the global model does not converge and the maximum number of rounds is not reached, all clients are regarded as a cluster, and the cluster generates a global model.
进一步地，所述方法还可包括：Further, the method may also include:
根据该用户的候选广告的广告点击率，将所选的部分广告列表发送给该用户的客户端，实现对该用户的个性化广告推荐。According to the advertisement click-through rate of the candidate advertisements of the user, the selected partial advertisement list is sent to the client of the user, so as to realize the personalized advertisement recommendation for the user.
需要说明的是，本实施例的计算机可读存储介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。It should be noted that the computer-readable storage medium in this embodiment may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
综上所述，本发明基于联邦因子分解机实现，可以权衡广告点击率预测算法在不同客户端数据非独立同分布场景下的准确度和隐私性，即保持联邦学习的模型可用性的同时保护客户端数据的隐私安全；对现有的集中式的因子分解机进行了优化，引入了联邦因子分解机，具体来说，联邦学习的分布式训练使客户端不用直接将用户原始数据上传给服务器，仅利用客户端的梯度信息更新模型；在因子分解机中引入聚类联邦学习的思想，从而实现能保护用户隐私的广告点击率预测，可以解决用户数据异构性带来的线性聚合模型损失。To sum up, the present invention is implemented based on the federated factorization machine, which can weigh the accuracy and privacy of the advertisement click-through rate prediction algorithm in different client data non-IID scenarios, that is, to maintain the availability of the federated learning model while protecting the client. The privacy and security of terminal data; the existing centralized factorization machine is optimized, and the federated factorization machine is introduced. Specifically, the distributed training of federated learning allows the client to not directly upload the user's original data to the server. Only the gradient information of the client is used to update the model; the idea of clustering federated learning is introduced into the factorization machine, so as to realize the prediction of advertisement click rate that can protect user privacy, and can solve the loss of linear aggregation model caused by the heterogeneity of user data.
以上所述，仅为本发明专利较佳的实施例，但本发明专利的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明专利所公开的范围内，根据本发明专利的技术方案及其发明构思加以等同替换或改变，都属于本发明专利的保护范围。The above is only a preferred embodiment of the patent of the present invention, but the protection scope of the patent of the present invention is not limited to this. The technical solution and the inventive concept of the invention are equivalently replaced or changed, all belong to the protection scope of the patent of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755722.8ACN113487351A (en) | 2021-07-05 | 2021-07-05 | Privacy protection advertisement click rate prediction method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755722.8ACN113487351A (en) | 2021-07-05 | 2021-07-05 | Privacy protection advertisement click rate prediction method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113487351Atrue CN113487351A (en) | 2021-10-08 |
Family
ID=77939950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110755722.8APendingCN113487351A (en) | 2021-07-05 | 2021-07-05 | Privacy protection advertisement click rate prediction method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113487351A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113988207A (en)* | 2021-11-09 | 2022-01-28 | 长春理工大学 | Item recommendation method and system |
| CN113988314A (en)* | 2021-11-09 | 2022-01-28 | 长春理工大学 | Cluster federal learning method and system for selecting client |
| CN114595831A (en)* | 2022-03-01 | 2022-06-07 | 北京交通大学 | A Federated Learning Approach Fusing Adaptive Weight Assignment and Personalized Differential Privacy |
| CN115081003A (en)* | 2022-06-29 | 2022-09-20 | 西安电子科技大学 | A Gradient Leak Attack Method under the Sampling Aggregation Framework |
| CN115168902A (en)* | 2022-07-26 | 2022-10-11 | 武汉大学 | Grouping verifiable chain type privacy protection federal learning method and device |
| CN115311692A (en)* | 2022-10-12 | 2022-11-08 | 深圳大学 | Federal pedestrian re-identification method, system, electronic device and storage medium |
| CN116527362A (en)* | 2023-05-06 | 2023-08-01 | 北京邮电大学 | A Data Protection Method Based on LayerCFL Intrusion Detection |
| CN116579443A (en)* | 2023-05-22 | 2023-08-11 | 广东工业大学 | A personalized federated learning method and storage medium for data heterogeneity |
| CN116933866A (en)* | 2023-08-16 | 2023-10-24 | 中国人民解放军总医院 | Personalized federated learning methods, systems and storage media for data heterogeneity |
| CN117077817A (en)* | 2023-10-13 | 2023-11-17 | 之江实验室 | A personalized federated learning model training method and device based on label distribution |
| CN117422154A (en)* | 2023-11-29 | 2024-01-19 | 重庆市科学技术研究院 | Cluster federation learning realization method and system with feature alignment |
| WO2025087218A1 (en)* | 2023-10-27 | 2025-05-01 | 烟台大学 | Method and system for detecting industrial internet abnormal node, medium, and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310047A (en)* | 2020-02-20 | 2020-06-19 | 深圳前海微众银行股份有限公司 | Information recommendation method, device and equipment based on FM model and storage medium |
| CN111507765A (en)* | 2020-04-16 | 2020-08-07 | 厦门美图之家科技有限公司 | Advertisement click rate prediction method and device, electronic equipment and readable storage medium |
| WO2020229684A1 (en)* | 2019-05-16 | 2020-11-19 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Concepts for federated learning, client classification and training data similarity measurement |
| CN112364943A (en)* | 2020-12-10 | 2021-02-12 | 广西师范大学 | Federal prediction method based on federal learning |
| CN112396099A (en)* | 2020-11-16 | 2021-02-23 | 哈尔滨工程大学 | Click rate estimation method based on deep learning and information fusion |
| CN112508203A (en)* | 2021-02-08 | 2021-03-16 | 同盾控股有限公司 | Federated data clustering method and device, computer equipment and storage medium |
| WO2021115480A1 (en)* | 2020-06-30 | 2021-06-17 | 平安科技（深圳）有限公司 | Federated learning method, device, equipment, and storage medium |
- 2021
- 2021-07-05CNCN202110755722.8Apatent/CN113487351A/enactivePending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020229684A1 (en)* | 2019-05-16 | 2020-11-19 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Concepts for federated learning, client classification and training data similarity measurement |
| CN111310047A (en)* | 2020-02-20 | 2020-06-19 | 深圳前海微众银行股份有限公司 | Information recommendation method, device and equipment based on FM model and storage medium |
| CN111507765A (en)* | 2020-04-16 | 2020-08-07 | 厦门美图之家科技有限公司 | Advertisement click rate prediction method and device, electronic equipment and readable storage medium |
| WO2021115480A1 (en)* | 2020-06-30 | 2021-06-17 | 平安科技（深圳）有限公司 | Federated learning method, device, equipment, and storage medium |
| CN112396099A (en)* | 2020-11-16 | 2021-02-23 | 哈尔滨工程大学 | Click rate estimation method based on deep learning and information fusion |
| CN112364943A (en)* | 2020-12-10 | 2021-02-12 | 广西师范大学 | Federal prediction method based on federal learning |
| CN112508203A (en)* | 2021-02-08 | 2021-03-16 | 同盾控股有限公司 | Federated data clustering method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 张朝阳 著: "深入浅出：工业机器学习算法详解与实战", 31 January 2020, 北京：机械工业出版社, pages: 214 - 219* |
| 汪雄飞 等译: "机器学习编程：从编程到深度学习", 30 June 2021, 北京：机械工业出版社, pages: 117 - 120* |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113988314A (en)* | 2021-11-09 | 2022-01-28 | 长春理工大学 | Cluster federal learning method and system for selecting client |
| CN113988207A (en)* | 2021-11-09 | 2022-01-28 | 长春理工大学 | Item recommendation method and system |
| CN113988207B (en)* | 2021-11-09 | 2024-09-24 | 长春理工大学 | Article recommendation method and system |
| CN113988314B (en)* | 2021-11-09 | 2024-05-31 | 长春理工大学 | Clustering federation learning method and system for selecting clients |
| CN114595831A (en)* | 2022-03-01 | 2022-06-07 | 北京交通大学 | A Federated Learning Approach Fusing Adaptive Weight Assignment and Personalized Differential Privacy |
| CN115081003B (en)* | 2022-06-29 | 2024-04-02 | 西安电子科技大学 | A gradient leakage attack method under the sampling aggregation framework |
| CN115081003A (en)* | 2022-06-29 | 2022-09-20 | 西安电子科技大学 | A Gradient Leak Attack Method under the Sampling Aggregation Framework |
| CN115168902A (en)* | 2022-07-26 | 2022-10-11 | 武汉大学 | Grouping verifiable chain type privacy protection federal learning method and device |
| CN115311692A (en)* | 2022-10-12 | 2022-11-08 | 深圳大学 | Federal pedestrian re-identification method, system, electronic device and storage medium |
| CN115311692B (en)* | 2022-10-12 | 2023-07-14 | 深圳大学 | Federal pedestrian re-identification method, system, electronic device and storage medium |
| CN116527362A (en)* | 2023-05-06 | 2023-08-01 | 北京邮电大学 | A Data Protection Method Based on LayerCFL Intrusion Detection |
| CN116579443A (en)* | 2023-05-22 | 2023-08-11 | 广东工业大学 | A personalized federated learning method and storage medium for data heterogeneity |
| CN116933866A (en)* | 2023-08-16 | 2023-10-24 | 中国人民解放军总医院 | Personalized federated learning methods, systems and storage media for data heterogeneity |
| CN116933866B (en)* | 2023-08-16 | 2025-03-04 | 中国人民解放军总医院 | Personalized federal learning method, system and storage medium for data heterogeneity |
| CN117077817B (en)* | 2023-10-13 | 2024-01-30 | 之江实验室 | Personalized federal learning model training method and device based on label distribution |
| CN117077817A (en)* | 2023-10-13 | 2023-11-17 | 之江实验室 | A personalized federated learning model training method and device based on label distribution |
| WO2025087218A1 (en)* | 2023-10-27 | 2025-05-01 | 烟台大学 | Method and system for detecting industrial internet abnormal node, medium, and device |
| CN117422154A (en)* | 2023-11-29 | 2024-01-19 | 重庆市科学技术研究院 | Cluster federation learning realization method and system with feature alignment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113487351A (en) | Privacy protection advertisement click rate prediction method, device, server and storage medium | |
| US20220391771A1 (en) | Method, apparatus, and computer device and storage medium for distributed training of machine learning model | |
| CN114902273A (en) | System and method for optimizing resource allocation using GPU | |
| US12033074B2 (en) | Vertical federated learning with compressed embeddings | |
| CN108549909B (en) | Object classification method and object classification system based on crowdsourcing | |
| CN115907056A (en) | Prediction model training method, information prediction method and corresponding devices | |
| US20220383036A1 (en) | Clustering data using neural networks based on normalized cuts | |
| US20220318639A1 (en) | Training individually fair machine learning algorithms via distributionally robust optimization | |
| CN111522923A (en) | A multi-round task-based dialogue state tracking method | |
| CN111209105A (en) | Capacity expansion processing method, device, device and readable storage medium | |
| CN111667069A (en) | Pre-training model compression method and device and electronic equipment | |
| CN118446316A (en) | Model reasoning method, device, computer equipment and computer readable storage medium | |
| CN116957128A (en) | Business indicator prediction methods, devices, equipment and storage media | |
| Yu et al. | Performance evaluation of integrated strategy of vehicle route guidance and traffic signal control using traffic simulation | |
| CN111310462A (en) | User attribute determination method, device, equipment and storage medium | |
| CN108256077B (en) | A Dynamic Extended Knowledge Graph Reasoning Method for China Mobile Intelligent Customer Service | |
| Zhang et al. | DAG scheduling with communication delays based on graph convolutional neural network | |
| Zhang et al. | Small files storing and computing optimization in Hadoop parallel rendering | |
| Zhang et al. | A new sequential prediction framework with spatial-temporal embedding | |
| CN116719607A (en) | Model updating method and system based on federal learning | |
| CN116266814A (en) | Prediction of resource demand and adjustment of resource allocation for nodes within a cluster | |
| KR102815217B1 (en) | Learning method of neural network model for language generation and apparatus for the learning method | |
| CN111241826B (en) | Entity name recognition method, device, equipment and storage medium | |
| CN115496175A (en) | Newly-built edge node access evaluation method and device, terminal equipment and product | |
| Hu et al. | HGC: Hybrid gradient compression in distributed deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |