Nonlinear knowledge in kernel machines

We give a unified presentation of recent work in applying prior knowledge to nonlinear kernel approximation [26] and nonlinear kernel classification [25]. In both approaches, prior knowledge over general nonlinear sets is incorporated into nonlinear kernel approximation or classification problems as linear constraints in a linear program. The key tool in this incorporation is a theorem of the alternative for convex functions that converts nonlinear prior knowledge implications into linear inequalities without the need to kernelize these implications. Effectiveness of the proposed approximation formulation is demonstrated on two synthetic examples as well as an important lymph node metastasis prediction problem arising in breast cancer prognosis. Effectiveness of the proposed classification formulation is demonstrated on two publicly available datasets, including a breast cancer prognosis dataset. Upon the introduction of prior knowledge, all these problems exhibit marked improvements over nonlinear kernel approximation and classification approaches that do not utilize such knowledge.