Academic Keynote: Differentially Private Covariance-Adaptive Mean Estimation, Adam Smith (BU)

Subscribers:
349,000
Published on ● Video Link: https://www.youtube.com/watch?v=QXTH7oN_aYw



Duration: 36:16
241 views
0


A Google TechTalk, presented by Adam Smith, 2021/11/9
ABSTRACT: Differentially Private Covariance-Adaptive Mean Estimation

Covariance-adaptive mean estimation is a fundamental problem in statistics, where we are given n i.i.d. samples from a d-dimensional distribution with mean $\mu$ and covariance $\Sigma$ and the goal is to find an estimator $\hat\mu$ with small error $\|\hat\mu-\mu\|_{\Sigma}\leq \alpha$, where $\|\cdot\|_{\Sigma}$ denotes the Mahalanobis distance. (We call this "covariance-adaptive" since the accuracy metric depends on the data distribution.)

It is known that the empirical mean of the dataset achieves this guarantee if we are given at least $n=\Omega(d/\alpha^2)$ samples. Unfortunately, the empirical mean and other statistical estimators can reveal sensitive information about the samples of the training dataset. To protect the privacy of the individuals who participate in the dataset, we study statistical estimators which satisfy differential privacy, a condition that has become a standard criterion for individual privacy in statistics and machine learning.

We present two new differentially private mean estimators for d-dimensional (sub)Gaussian distributions with unknown covariance whose sample complexity is optimal up to logarithmic factors and matches the non-private one in many parameter regimes. Previous estimators with the same guarantee either require strong a priori bounds on the covariance matrix or require $\Omega(d^{3/2})$ samples.

Based on the paper https://arxiv.org/pdf/2106.13329.pdf, which will appear as a spotlight paper at NeurIPS 2021 and is joint work with Gavin Brown, Marco Gaboardi, Jonathan Ullman, and Lydia Zakynthinou.

About the Speaker: Adam Smith, Boston University
Adam Smith is a professor of computer science at Boston University. From 2007 to 2017, he served on the faculty of the Computer Science and Engineering Department at Penn State. His research interests lie in data privacy and cryptography, and their connections to machine learning, statistics, information theory, and quantum computing. He obtained his Ph.D. from MIT in 2004 and has held postdoc and visiting positions at the Weizmann Institute of Science, UCLA, Boston University and Harvard. He received a Presidential Early Career Award for Scientists and Engineers (PECASE) in 2009; a Theory of Cryptography Test of Time award in 2016; the Eurocrypt 2019 Test of Time award; and the 2017 Gödel Prize.

For more information about the workshop: https://events.withgoogle.com/2021-workshop-on-federated-learning-and-analytics/#content




Other Videos By Google TechTalks


2022-02-08Privacy Amplification by Decentralization
2022-02-08Experimenting w/ Local & Central Differential Privacy for Both Robustness & Privacy in Fed.Learning
2022-02-08Differentially Private Fine-tuning of Language Models
2022-02-08When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?
2022-02-08FedVault: Efficient Gradient Outlier Detection for Byzantine-Resilient and Privacy-Preserving FedML
2022-02-08Federated Learning and Analytics Research Using TensorFlow Federated
2022-02-08Day 2 Lightning Talks: Federated Optimization and Analytics
2022-02-08Academic Keynote: Systems Support for Federated Computation, Mosharaf Chowdhury (U of Michigan)
2022-02-08Day 2 Lightning Talks: Privacy & Security
2022-02-08Google Keynote: Federated Learning & Federated Analytics-Research, Applications, & System Challenges
2022-02-08Academic Keynote: Differentially Private Covariance-Adaptive Mean Estimation, Adam Smith (BU)
2022-02-08Academic Keynote: Mean Estimation with User-level Privacy under Data Heterogeneity, Rachel Cummings
2022-02-08Day 1 Lightning Talks: Federated Optimization and Analytics
2022-02-08Day 1 Lightning Talks: Privacy & Security
2022-02-08Academic Keynote: Federated Learning with Strange Gradients, Martin Jaggi (EPFL)
2022-02-08Google Keynote: Federated Aggregation and Privacy
2022-02-08Welcome and Opening Remarks
2022-01-25Warehouse-Scale Video Acceleration: Co-Design and Deployment in the Wild
2022-01-06What Could Be the Data-Structures of the Mind?
2021-12-21Differential Privacy and the 2020 Census in the United States
2021-12-21Covariance-Aware Private Mean Estimation Without Private Covariance Estimation