Exploration of Reinforcement Learning Based Method for Managing Type 1 Diabetes
ABSTRACT
Background:
Type 1 diabetes Mellitus (TIDM) is characterized by chronic insulin deficiency and consequent hyperglycemia. Patients with TIDM require long term exogenous insulin therapy to regulate blood glucose levels and prevent the long-term complications of the disease. Currently, there are no effective algorithms that takes into consideration the TIDM patient’s unique characteristics to automatically recommend personalized insulin dosage levels.
Objective:
The objective of this work is to develop and validate a general reinforcement learning framework for the personalized treatment of TIDM.
Methods:
This research presents a model-free data-driven reinforcement learning (RL) algorithm, namely Q-learning, that recommends insulin doses to regulate the blood glucose level of a TIDM patient considering his/her state defined by, hemoglobin A1C, body mass index, engagement in physical activity, and alcohol usage. In this approach, the RL agent learns from its exploration of the patient’s responses when he/she is subjected to varying insulin doses. As a result of a treatment action at time step t, the RL agent receives a numeric reward, positive or negative. The reward is calculated as a function of the difference between the actual blood glucose level and the targeted level. The RL agent was trained on ten years of clinical data of patients treated at the Mass General Hospital.
Results:
Eighty-seven patients were included in the training set. The mean age of these patients was 53 years, 59% were male, 86% were Caucasians and 47% of them were married. The performance of the RL agent was evaluated on 60 test cases. RL agent recommended insulin dosage interval includes the actual dose prescribed by the physician in 88% of the cases.
Conclusions:
This exploratory study demonstrates that a RL algorithm can be used to automatically personalize insulin doses to adequate glycemic control in patients with TIDM. However, further investigation in a larger sample of patients is needed to confirm these findings.
Citation
