JMIR Publications

ABSTRACT

Background:

Optimizing insulin dosing and predicting future glucose levels for type 1 diabetes (T1D) patients is challenging due to the dynamic nature of glucose metabolism. Traditional static insulin regimens fail to adapt to individual variability in diet, physical activity, stress, and metabolic fluctuations, leading to suboptimal glycemic control. Reinforcement Learning (RL) offers a promising alternative by enabling personalized, real-time insulin adjustments that improve the balance between hyperglycemia and hypoglycemia.

Objective:

This study aims to develop a Deep Q-Network (DQN)-based RL system that dynamically personalizes insulin dosing recommendations using continuous glucose monitoring (CGM) data, meal intake, and physical activity levels. By leveraging real-time data, the model adapts to patients’ evolving physiological states, enhancing glucose control and patient safety.

Methods:

We utilized the OhioT1DM dataset (2018 & 2020), which contains eight weeks of continuous glucose measurements, insulin dosing records, and physical activity data for twelve T1D patients. The RL agent was designed with a state representation consisting of recent blood glucose levels, insulin doses, and lifestyle factors over a 2-hour window. The 2-hour window was selected based on the known pharmacodynamic profile of rapid-acting insulin (peak action within 90–120 minutes), as well as the typical lag in glycemic response following meals or exercise. This window size captures both recent and delayed physiological effects while balancing data density and model stability. The action space included discrete insulin dose recommendations (e.g., 0.5U, 1U, 1.5U). A reward function incentivized glucose levels within the target range (70-180 mg/dL) while penalizing extreme deviations. The DQN model was trained to maximize reward by learning optimal dosing strategies through iterative trial and error.

Results:

Performance evaluation was conducted using both qualitative and quantitative metrics. Time-series analysis compared actual and predicted glucose levels, demonstrating effective glucose regulation. The RL model achieved a mean glucose level of 80.06 mg/dL, with a reward score of 10 during evaluation, indicating that most glucose predictions were maintained within the desired clinical range. This suggests the model has learned to regulate blood glucose effectively through adaptive insulin dosing. The RMSE (12.39 mg/dL) was slightly higher than the MAE (9.85 mg/dL), indicating stable predictions. Additionally, the Percentage Time in Target Range (TIR) was 64.06%, suggesting that the model-maintained glucose within the clinically safe range for a majority of the time.

Conclusions:

The DQN-based RL model demonstrated its effectiveness in personalized insulin dosing while minimizing the risk of hypo- and hyperglycemia. This suggests the model has learned to regulate blood glucose effectively through adaptive insulin dosing. This approach represents a significant advancement over conventional methods, offering a scalable and adaptive strategy for real-world diabetes management along with enhancing clinical trust and transparency through explainability techniques. Clinical Trial: NA