Machine learning methods used for self-driving cars can improve the lives of patients with type-1 diabetes

Scientists at the University of Bristol have shown that reinforcement learning, a type of machine learning in which computer programs learn to make decisions by trying different actions, significantly outperforms commercial blood glucose controllers in terms of safety and effectiveness. Using offline reinforcement learning, where the algorithm learns from patient records, the researchers improved on previous work, showing that good blood glucose control can be achieved by learning from patient decisions rather than by trial and error.

Type 1 diabetes is one of the most common autoimmune conditions in the UK and is characterized by a deficiency of the hormone insulin, which is responsible for blood glucose regulation.

Many factors affect a person’s blood glucose and therefore it can be a challenging and onerous task to choose the right insulin dosage for a particular scenario. Current artificial pancreas devices provide automatic insulin dosing but are limited by simple decision-making algorithms.

But a new study, published in Journal of Biomedical Informatics, indicating offline reinforcement learning could represent an important milestone of care for people living with the condition. The increase was greatest in children, who experienced an additional hour and a half in the target glucose range per day.

Children represent a very important group because they often cannot manage their diabetes without help and increasing this measure will lead to much better long-term health outcomes.

Main author Harry Emerson from Bristol Department of Mathematical Engineeringexplains: “My research explores whether reinforcement learning can be used to develop safer and more effective insulin dosing strategies.

“This machine learning-driven algorithm has demonstrated superhuman performance in playing chess and driving self-driving cars, and can therefore be feasibly learned to perform highly personalized insulin dosing from previously collected blood glucose data.

“This particular work focuses specifically on offline reinforcement learning, where an algorithm learns to act by observing examples of good and poor blood glucose control.

“Previous reinforcement learning methods in this field have largely used a process of trial and error to identify good actions that could expose real-world patients to unsafe insulin doses.”

Due to the high risk associated with incorrect insulin dosing, the experiment was performed using the FDA-approved UVA/Padova simulator, which creates a virtual patient series for testing type 1 diabetes control algorithms. The state-of-the-art offline reinforcement learning algorithm was evaluated against one of the existing artificial pancreas control algorithms. most used. This comparison was performed on 30 virtual patients (adults, adolescents, and children) and considered data for 7,000 days, with performance evaluated according to current clinical guidelines. The simulator is also extended to take into account realistic implementation challenges, such as measurement errors, incorrect patient information, and the limited amount of data available.

This work provides a basis for advanced reinforcement learning research in glucose control; demonstrates the potential of the approach to improve the health outcomes of people with type 1 diabetes, while highlighting the method’s shortcomings and areas of future development that are needed.

The researchers’ main goal is to deploy reinforcement learning in real-world artificial pancreatic systems. These devices operate under limited patient supervision and will consequently require significant evidence of safety and effectiveness to achieve regulatory approval.

Harry added: “This research demonstrates the potential of machine learning to learn effective insulin dosing strategies from previously collected data on type 1 diabetes. The method explored outperforms one of the most widely used commercial artificial pancreas algorithms and demonstrates the ability to leverage a person’s habits and schedules to respond more quickly to harmful events.”