Personalized vital signs control based on continuous action-space reinforcement learning with supervised experience. (August 2021)

• Record Type: Journal Article
• Authors: Sun, Chenxi; Hong, Shenda; Song, Moxian; Shang, Junyuan; Li, Hongyan
• Is Part Of: Biomedical signal processing and control. Volume 69(2021)
• Publication Date: 2021-08
• Subjects: Personalized vital signs control -- Clinical decision support -- Safety control -- Reinforcement learning -- Supervised learning; PVSC Personalized Vital Signs Control -- SL Supervised Learning -- RL Reinforcement Learning -- DL Deep Learning -- RNN Recurrent Neural Network -- MDP Markov Decision Process -- AC Actor-Critic network -- DDPG Deep Deterministic Policy Gradient -- PVSC-RL Our method -- ICU Intensive Care Unit -- EHR Electronic Health Record -- BG Blood Glucose -- HP Blood Pressure -- HF Heart Failure -- sepsis Sepsis -- SOFA Sequential Organ Failure Assessment -- BNP Brain Natriuretic Peptide; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.102847
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: Personalized vital signs control is essential for medical decision support. Data-driven deep learning methods assist clinicians consider more complex factors. Reinforcement learning methods infer optimal vital sign based on non-optimized data. Combining reinforcement learning and supervisor makes control better and safer. Abstract: Vital signs reflect patients' current health status. Different patients are suitable for distinct references of vital signs due to the disease type and individual physique. Personalized Vital Signs Control (PVSC) helps clinicians with the recommendation of the optimal references for real-time treatment. However, a data-driven approach needs to overcome the unclear ground truth problem, requires complex feature integration and continuous value space, and is expected to ensure the safety of fragile patients. But none of the existing approaches can overcome all of these challenges simultaneously. This work emphasizes PVSC as a sequence decision-making problem and applies multiple reinforcement learning methods to it. We propose a novel adaptive medical control model. The model combines the deep deterministic policy gradient reinforcement learning algorithm, supervised experience knowledge, and recurrent neural network module, named PVSC-RL. We test the model on a real medical database, MIMIC-III, with 15, 232 sepsis and 13, 608 heart failure records. Experimental results show that using the policy of PVSC-RL, the survival rate of sepsis and heart failure patients by blood glucose control and blood pressure control can be increased by 3.45%, 15.21%, 4.19%, 12.13%. Meanwhile, the safety rate is also be increased by 3.09%, 1.02%, 3.54%, 4.04%.