대한신장학회

Skip Navigation: Skip to contents

KOR
ENG
전체메뉴보기

간행물 검색

현재 페이지 경로

HOME
간행물
간행물 검색

논문분류	춘계학술대회 초록집
제목	Development of a Real-time Autonomous Ultrafiltration Control Agent Using Deep Reinforcement Learning
저자	Min Woo Kang
출판정보	2024; 2024(1):
키워드
초록	Objectives: The global prevalence of end-stage kidney disease (ESKD) patients is increasing, emphasizing a growing need for home dialysis. However, since home hemodialysis lacks the close monitoring by healthcare professionals as in clinical settings, the development of artificial intelligence capable of autonomously adjusting dialysis in real-time is essential. Methods: We utilized data from hemodialysis patients at MacKay Memorial Hospital in Taiwan, an open data source from Scientific Data. The aim of this study is to develop reinforcement learning (RL) agent capable of autonomously adjusting ultrafiltration in real-time during dialysis. Deep learning model predicting blood pressure during dialysis was developed. Using this model, a virtual dialysis simulator was created to establish a Markov Decision Process environment. We utilized the Fully Parameterized Quantile Function (FQF) for online RL and Conservative Q-Learning (CQL) for offline RL. The CQL model was created by adding CQL loss to IQN (Implicit Quantile Networks). Both models were evaluated using 4,480 virtual patients through the simulator. Results: Data from a total of 1,072 patients and 27,508 dialysis sessions were used in this study. The simulator's blood pressure prediction abilities showed Mean Absolute Errors (MAE) of 5.12 and 2.87 mmHg for SBP and DBP, respectively, on the train data, while on the test data, the MAE was 6.02 and 3.29 mmHg, respectively. The online RL model successfully achieved an average weight reduction with mean error of 0.26 kg compared to the target, and 896 cases of hypotension occurred. The offline RL model achieved weight reduction with mean error of 0.04 kg compared to the target, and 909 cases of hypotension occurred. Conclusions: Using an open data source, we developed RL agent capable of autonomously adjusting ultrafiltration during dialysis through reinforcement learning. It appears that with refined data in the future, the development of a self-regulating dialysis machine is plausible.
원문(PDF)	PDF 원문보기

목록

위로가기