A New Glycemic closed-loop control based on Dyna-Q for Type-1-Diabetes. (March 2023)

• Record Type: Journal Article
• Authors: Del Giorno, Silvia; D'Antoni, Federico; Piemonte, Vincenzo; Merone, Mario
• Is Part Of: Biomedical signal processing and control. Volume 81(2023)
• Publication Date: 2023-03
• Subjects: Deep learning -- Reinforcement Learning -- Diabetes -- Glycemic control -- Time series forecasting -- Dyna-Q; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104492
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: Objective: Type 1 Diabetes Mellitus is an autoimmune disease which requires constant care from patients. Continuous Glucose Monitoring (CGM) devices allow to keep track of the glycemic trend for 24 h a day. Control algorithms are necessary to automate the therapy and to develop an artificial pancreas system. The objective of this study is to develop a fully-automated glycemic control system based on a Dyna-Q Reinforcement Learning algorithm that is able to automatically decide the insulin infusion without the need of carbohydrate information from the patient. Methods: A Dyna-Q Reinforcement Learning architecture is proposed to automate glycemic control while using only information on past CGM and insulin data, validated on data from 10 in silico patients. Results: The proposed glycemic predictor achieves an average RMSE and a MARD of 13.2 mg/dl and 6.9% on 10 virtual adults, and of 15.0 mg/dl and 7.2% on 12 real patients, while over 98.8% of the forecasts fall within the safe zones of the Clarke Error Grid. The controller is able to maintain the glucose levels of the virtual subjects in the target range for 60.7% of the simulation time on a 24 h scenario, without causing hypoglycemic events in 8 out of 10 patients. Conclusion: The proposed architecture is able to achieve good performance without exploiting information on carbohydrates and using a much smaller amount of training data compared to models in the literature. Significance: We proved that model-based Reinforcement Learning could be a valid approach for a human-safe fully-automated artificial pancreas. Highlights: Current T1DM care resorts to pre-set meal-insulin schedules, which must be improved A new method is presented for simultaneous prediction and safe control of glycemia The proposed method uses Dyna-Q Reinforcement Learning to automate insulin therapy We propose a valid approach for human-safe fully-automated artificial pancreas design The system is validated on in silico patient data generated with the UVA/PD simulator