Model-based reinforcement learning by pyramidal neurons: Robustness of the learning rule

Reinforcement learning of control is studied in computational models of pyramidal neurons. We have previously demonstrated that a neuron can not only learn direct control, in which the neuron's activity is directly associated with later rewards, but also indirect control, in which the association is based on a model of the controlled dynamics. Here we will summarize these learning principles: the central task of the learning rule consists in detecting small transient changes in the postsynaptic firing rate. Then the robustness of the learning rule is demonstrated by applying it to a set of cyclically activated synapses, which can detect cyclic changes in the postsynaptic firing rate. The learned response is insensitive to inhibition, dendritic nonlinearities, dendritic time constants, overlaps of distributed input patterns, or input noise. Predictions are made for the learning principles employed by pyramidal neurons.