Priv'IT: Private and Sample Efficient Identity Testing

We develop differentially private hypothesis testing methods for the small sample regime. Given a sample D from a categorical distribution p over some domain Sigma, an explicitly described distribution q over Sigma, some privacy parameter epsilon, accuracy parameter alpha, and requirements A and beta(I) for the type I and type beta(II) errors of our test, the goal is to distinguish between p = q and d(Tv) (p, q) >= alpha. We provide theoretical bounds for the sample size vertical bar D vertical bar so that our method both satisfies (epsilon, 0)-differential privacy, and guarantees beta(I) and beta(II) type I and type II errors. We show that differential privacy may come for free in some regimes of parameters, and we always beat the sample complexity resulting from running the chi(2)-test with noisy counts, or standard approaches such as repetition for endowing nonprivate chi(2)-style statistics with differential privacy guarantees. We experimentally compare the sample complexity of our method to that of recently proposed methods for private hypothesis testing (Gaboardi et al., 2016; Kifer & Rogers, 2017).