Structured exploration allows the biological brain to learn faster than AI

Neuroscientists have uncovered how the act of exploration allows animals to learn more efficiently about their spatial environment. Their findings could help build better AI agents that can learn faster and require less experience.

Researchers at the Sainsbury Wellcome Center and the Gatsby Computational Neuroscience Unit at UCL found that animals’ exploratory instincts are not random. This purposeful action allows the mouse to study the world map efficiently. This study, published today in Neuronsdescribes how neuroscientists tested their hypothesis that specific exploratory actions that animals perform, such as darting rapidly towards objects, are important in helping them learn how to navigate their environment.

“There are many theories in psychology about how performing certain actions facilitates learning. In this study, we tested whether simply observing obstacles in an environment was sufficient to learn about them, or whether sensory-guided actions helped animals build a cognitive map of the world,” said Professor Tiago Branco, Group Leader at the Sainsbury Wellcome Center and corresponding author on the paper. .

In previous work, scientists at SWC observed a correlation between how well animals learned to overcome obstacles and the number of times they ran into those objects. In this study, Philip Shamash, SWC PhD student and first author of the paper, conducted an experiment to test the effect of preventing animals from taking exploratory runs. By expressing a light-activating protein called channelrhodopsin in a part of the motor cortex, Philip was able to use an optogenetic tool to prevent animals from embarking on exploratory journeys towards obstacles.

The team found that even though the rats had spent a significant amount of time observing and sniffing out obstacles, if they were prevented from running towards them, they did not learn. This suggests that the act of instinctive exploration itself helps animals learn a map of their environment.

To explore algorithms the brain might use to learn, the team worked with Sebastian Lee, a PhD student in Andrew Saxe’s lab at SWC, to run various reinforcement learning models that people have developed for artificial agents, and observe which one most closely reproduced mouse behavior. .

There are two main classes of reinforcement learning models: model-independent and model-based. The team found that under some conditions, the mice acted in a model-independent manner, but under other conditions, they appeared to have a model of the world. So the researchers implemented an agent that could mediate between model-independent and model-based. This isn’t necessarily how rat brains work, but it helps them understand what it takes in a learning algorithm to explain the behavior.

“One of the problems with artificial intelligence is that agents need a lot of experience to learn something. They have to explore the environment thousands of times, whereas real animals can learn about the environment in less than ten minutes. We think this is partly because, unlike artificial agents, animal exploration is not random and instead focuses on objects that stand out. This kind of directed exploration makes learning more efficient so they need less experience to learn,” explained Professor Branco.

The next step for researchers is to explore the relationship between the execution of exploratory actions and subgoal representations. The team is now recording in the brain to discover which areas are involved in representing the subgoals and how the act of exploration leads to the formation of the representation.

This research was funded by the Wellcome Senior Research Fellowship (214352/Z/18/Z) and by the Sainsbury Wellcome Center Core Grant from the Gatsby Charitable Foundation and Wellcome (090843/F/09/Z), the Sainsbury Wellcome Center PhD Program and the Sir Henry Dale Scholarship of the Wellcome Trust and Royal Society (216386/Z/19/Z).

Source:

Read the full paper at Neurons: ‘Rats identify sub-destination sites through an action-based mapping process’ DOI:10.1016/j.neuron.2023.03.034

Media contacts:

For more information or to speak with the researchers involved, please contact:

April Cashin-Garbutt
Head of Research and Engagement Communications, Sainsbury Wellcome Center
E: (email protected) Q: +44 (0)20 3108 8028

About the Sainsbury’s Wellcome Centre
The Sainsbury Wellcome Center (SWC) brings together the world’s leading neuroscientists to come up with theories about how neural circuits in the brain give rise to the fundamental processes underlying behavior, including perception, memory, expectation, decision, cognition, volition and action. Funded by the Gatsby and Wellcome Charitable Foundations, SWC is located within UCL and closely linked to the School of Life Sciences and Brain Sciences. For more information, please visit: www.sciencesburywellcome.org