Google deployed a production machine learning model using federated learning with a rigorous differential privacy guarantee for Gboard's next-word prediction.
This article announces Google's deployment of a production machine learning model that combines federated learning (FL) with formal differential privacy (DP) guarantees. Federated learning allows mobile devices to collaboratively train ML models while keeping raw training data on user devices, decoupling ML from cloud data storage. Differential privacy provides a quantifiable measure of data anonymization, addressing concerns about models memorizing sensitive user data. This deployment is a proof-of-concept utilizing the DP-FTRL algorithm to train a recurrent neural network for Spanish-language Gboard's next-word prediction, marking it as the first production neural network trained directly on user data with a formal DP guarantee (ρ=0.81 zero-Concentrated-Differential-Privacy). The system incorporates data minimization principles by transmitting minimal updates, limiting data access, processing data early, and minimal retention. For anonymization, user-level differential privacy is employed, ensuring the model's output distribution doesn't change even if all training examples from a single user are added or removed. This is achieved by limiting user contributions and adding negatively correlated noise during aggregation. The DP-FTRL algorithm addresses challenges in real-world cross-device FL systems where device availability varies, by focusing on the accuracy of cumulative sums of gradients rather than individual gradients, and using negatively correlated noise to maintain model quality while providing strong privacy.
Integrations: Gboard, Google Assistant, Google Messages, TensorFlow Privacy, Pipeline DP
Platforms: Android, iOS
View full Federated Learning with Formal Differential Privacy Guarantees profile on Tools-Radar | Browse Coding & Developer Tools tools | Alternatives to Federated Learning with Formal Differential Privacy Guarantees
Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com