secure aggregation
In plain wordsA cryptographic trick that lets a server add up updates from many devices while learning only the total, never any single device's contribution.
A cryptographic protocol, a special case of secure multi-party computationsecure multi-party computation, n.: Cryptographic methods that let several parties compute a result from their combined private data — say, an average salary — without any of them revealing its own data to the others., by which many clients jointly reveal the sum of their private vectors to a server while each vector on its own stays hidden. In the standard construction, each pair of clients agrees on random masks that one adds and the other subtracts, so the masks cancel in the total; secret-sharing of mask seeds lets the server finish the sum even when some clients drop out mid-round. In federated learningfederated learning, n.: Training one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates. it hides each device's model update from the server, and combined with differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person. noise it limits what the aggregate itself can reveal. It does not stop malicious clients from submitting poisoned updates; indeed, it makes them harder to inspect.
With secure aggregation enabled, the server saw only the summed update from the 500 phones in each round.
Origin
Keith Bonawitz and colleagues at Google introduced the practical protocol in 'Practical Secure Aggregation for Privacy-Preserving Machine Learning' (ACM CCS 2017), designed for mobile federated learning; later work cut its per-client overhead to grow only polylogarithmically with the number of clients.
Related
- federated learningfederated learning, n.: Training one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates.
- secure multi-party computationsecure multi-party computation, n.: Cryptographic methods that let several parties compute a result from their combined private data — say, an average salary — without any of them revealing its own data to the others.
- differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
- local differential privacylocal differential privacy, n.: A privacy method in which each person's device scrambles their data with random noise before sending it anywhere, so the company collecting it never sees anyone's true answer but can still estimate overall trends.
- gradient inversion attackgradient inversion attack, n.: Rebuilding someone's private training data — such as photos or text — just from the model updates their device shared during collaborative training.
Sources
- Bonawitz, K. et al. (2017). Practical Secure Aggregation for Privacy-Preserving Machine Learning. ACM CCS 2017.
Mentioned in
federated learningfederated learning, n.: Training one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates., privacy-preserving machine learningprivacy-preserving machine learning, n.: The family of techniques for training and using AI models while keeping the underlying personal or confidential data hidden — from adding careful noise to computing on encrypted data., gradient inversion attackgradient inversion attack, n.: Rebuilding someone's private training data — such as photos or text — just from the model updates their device shared during collaborative training., local differential privacylocal differential privacy, n.: A privacy method in which each person's device scrambles their data with random noise before sending it anywhere, so the company collecting it never sees anyone's true answer but can still estimate overall trends., secure multi-party computationsecure multi-party computation, n.: Cryptographic methods that let several parties compute a result from their combined private data — say, an average salary — without any of them revealing its own data to the others.