Abstract: Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. (2013), yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning.

Speaker:  Ossi Räisä is a doctoral researcher in the Trustworthy Machine Learning group at the University of Helsinki. His research interests are centred around differential privacy and synthetic data generation, and include differential privacy theory, uncertainty estimation in private data analysis, and synthetic data evaluation.

Affiliation: University of Helsinki

Place of Seminar:  Kumpula Exactum BK106 (in person) & zoom ( Meeting ID: 640 5738 7231 ; Passcode: 825217)