Adaptive long-term control of biological neural networks with Deep Reinforcement Learning

Driving activity patterns in the brain to desired levels is an important therapeutic strategy for many neurological disorders. The highly dynamic nature of neuronal networks and changes with disease progression create an urgent need for closed-loop control. Without adequate mathematical models of such complex networks, it remains unclear how tractable control problems can be formulated and solved for neurobiological systems. Reinforcement Learning (RL) is a promising tool to address such challenges, but has rarely been used for the long-term control of live, plastic neural networks. This is a difficult problem since it requires the controller to adapt to poorly characterized non-stationary background processes that alter stimulus-response relations over time. We captured these challenges in a novel control task, defined as clamping response strengths to predefined levels over long durations in a living model system, namely generic BNNs in vitro grown on microelectrode arrays. We show that by defining appropriate state-action spaces and employing powerful nonlinear RL methods such as Deep RL, adaptivity to non-stationary background processes can be achieved in a series of experiments: In 27/29 networks, the learned controllers were able to improve performance compared to a random and a (linear) LSPI controller by a large margin. (C) 2019 Elsevier B.V. All rights reserved.