The role of training variability for model-based and model-free learning of an arbitrary visuomotor mapping

A fundamental feature of the human brain is its capacity to learn novel motor skills. This capacity requires the formation of vastly different visuomotor mappings. Using a grid navigation task, we investigated whether training variability would enhance the flexible use of a visuomotor mapping (key-to-direction rule), leading to better generalization performance. Experiments 1 and 2 show that participants trained to move between multiple start-target pairs exhibited greater generalization to both distal and proximal targets compared to participants trained to move between a single pair. This finding suggests that limited variability can impair decisions even in simple tasks without planning. In addition, during the training phase, participants exposed to higher variability were more inclined to choose options that, counterintuitively, moved the cursor away from the target while minimizing its actual distance under the constrained mapping, suggesting a greater engagement in model-based computations. In Experiments 3 and 4, we showed that the limited generalization performance in participants trained with a single pair can be enhanced by a short period of variability introduced early in learning or by incorporating stochasticity into the visuomotor mapping. Our computational modeling analyses revealed that a hybrid model between model-free and model-based computations with different mixing weights for the training and generalization phases, best described participants' data. Importantly, the differences in the model-based weights between our experimental groups, paralleled the behavioral findings during training and generalization. Taken together, our results suggest that training variability enables the flexible use of the visuomotor mapping, potentially by preventing the consolidation of habits due to the continuous demand to change responses. The development of new motor skills often requires the learning of novel associations between actions and outcomes. These novel mappings can be flexible and generalize to new situations, or more local with narrow generalization, similar to stimulus-action associations. In a series of experiments using a navigation task, we showed that generalizable mappings are favored under a training variability regime, while local mappings with narrow generalization are developed in the absence of variability. Training variability was generated in our experiments either with multiple goals or with stochasticity in the action-outcome mapping, with both regimes leading to successful generalization. In addition, we showed that the benefits in generalization from training variability can be observed even when participants are subsequently exposed to no variability for a prolonged period of time. These results were best described by a mixture of model-free and model-based reinforcement learning algorithms, with different mixture weights for the training and generalization phases.