Revisiting Perceptron: Efficient and Label-Optimal Learning of Halfspaces

It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the bounded noise condition [49], where each label is flipped with probability at most eta < 1/2, our algorithm achieves a near-optimal label complexity of <(O)over tilde>(d/(1-2 eta)(2) ln 1/epsilon)(2) in time (O) over tilde (d(2)/epsilon(1-2 eta)(3)). Under the adversarial noise condition [6, 45, 42], where at most a (Omega) over tilde (epsilon) fraction of labels can be flipped, our algorithm achieves a near-optimal label complexity of (O) over tilde (d ln 1/epsilon) in time (O) over tilde (d(2)/epsilon) Furthermore, we show that our active learning algorithm can be converted to an efficient passive learning algorithm that has near-optimal sample complexities with respect to epsilon and d.