Optimal memoryless control in Gaussian noise: A simple counterexample

In this paper, we investigate control strategies for a scalar, one-step delay system in discrete-time, i.e., the state of the system is the input delayed by one time unit. In contrast with classical approaches, here the control action must be a memoryless function of the output of the plant, which comprises the current state corrupted by measurement noise. We adopt a first order state-space representation for the delay system, where the initial state is a Gaussian random variable. In addition, we assume that the measurement noise is drawn from a white and Gaussian process with zero mean and constant variance. Performance evaluation is carried out via a finite-time quadratic cost that combines the second moment of the control signal, and the second moment of the difference between the initial state and the state at the final time. We show that if the time-horizon is one or two then the optimal control is a linear function of the plant's output, while for a sufficiently large horizon a control taking on only two values will outperform the optimal affine solution. This paper complements the well-known counterexample by Hans Witsenhausen, which showed that the solution to a linear, quadratic and Gaussian optimal control paradigm might be nonlinear. Witsenhausen's counterexample considered an optimization horizon with two time-steps (two stage control). In contrast with Witsenhausen's work, the solution to our counterexample is linear for one and two stages but it becomes nonlinear as the number of stages is increased. The fact that our paradigm leads to nonlinear solutions, in the multi-stage case, could not be predicted from prior results. In contrast to prior work, the validity of our counterexample is based on analytical proof methods. Our proof technique rests on a simple nonlinear strategy that is useful in its own right, since it outperforms any affine solution. (c) 2010 Elsevier Ltd. All rights reserved.