Computer Science > Machine Learning
arXiv:2303.10837 (cs)
[Submitted on 20 Mar 2023 (v1), last revised 17 Jun 2024 (this version, v3)]
Title:FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System
Authors:Weizhao Jin,Yuhang Yao,Shanshan Han,Jiajun Gu,Carlee Joe-Wong,Srivatsan Ravi,Salman Avestimehr,Chaoyang He
View a PDF of the paper titled FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System, by Weizhao Jin and 7 other authors
View PDFHTML (experimental)Abstract:Federated Learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as the aggregated local models on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present FedML-HE, the first practical federated learning system with efficient HE-based secure model aggregation. FedML-HE proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing customizable privacy preservation. Our optimized system demonstrates considerable overhead reduction, particularly for large foundation models (e.g., ~10x reduction for ResNet-50, and up to ~40x reduction for BERT), demonstrating the potential for scalable HE-based FL deployment.
| Subjects: | Machine Learning (cs.LG); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2303.10837 [cs.LG] |
| (orarXiv:2303.10837v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2303.10837 arXiv-issued DOI via DataCite |
Submission history
From: Weizhao Jin [view email][v1] Mon, 20 Mar 2023 02:44:35 UTC (4,121 KB)
[v2] Mon, 30 Oct 2023 21:40:35 UTC (4,867 KB)
[v3] Mon, 17 Jun 2024 15:39:21 UTC (4,867 KB)
Full-text links:
Access Paper:
- View PDF
- HTML (experimental)
- TeX Source
- Other Formats
View a PDF of the paper titled FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System, by Weizhao Jin and 7 other authors
References & Citations
Bookmark
Bibliographic and Citation Tools
Bibliographic Explorer(What is the Explorer?)
Connected Papers(What is Connected Papers?)
Litmaps(What is Litmaps?)
scite Smart Citations(What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv(What is alphaXiv?)
CatalyzeX Code Finder for Papers(What is CatalyzeX?)
DagsHub(What is DagsHub?)
Gotit.pub(What is GotitPub?)
Hugging Face(What is Huggingface?)
Papers with Code(What is Papers with Code?)
ScienceCast(What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower(What are Influence Flowers?)
CORE Recommender(What is CORE?)
IArxiv Recommender(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community?Learn more about arXivLabs.