대한신장학회

Skip Navigation: Skip to contents

KOR
ENG
전체메뉴보기

간행물 검색

현재 페이지 경로

HOME
간행물
간행물 검색

논문분류	춘계학술대회 초록집
제목	Reliability Comparison between a Large Language Model and Nephrologist in Analyzing Acid-Base Disorders in Critically Ill Patients
저자	So Young Lee
출판정보	2024; 2024(1):
키워드
초록	Objectives: The generative pre-trained transformer (GPT), a type of large language model (LLM), is now playing a huge role in driving innovation in fields of medical education, diagnosis and treatment and with a process of continuous improvement it will itself be able to become beacons of sustainability. Medical professionals expect that they may have help from LLMs to diagnose patients more accurately and efficiently, but it is unclear whether current LLMs are well-trained and validated on real-world clinical data. In this study, we compared the performance of ChatGPT, a representative LLM, MDCalc, an online medical calculator, and human nephrologist for their diagnostic accuracies of acid-base gas analysis in critically ill cases. Methods: This study included 150 patients admitted to intensive care unit with varying medical conditions. All variables were obtained during the first 24 hours after admission. Results: The Fleiss’ Kappa between the interpretations of nephrologist, ChatGPT and MDCalc with the acid-base gas analysis for acid-base disorders showed that Fleiss’ Kappa value was -0.138 (95% CI -0.216 to -0.059), indicating no agreement among the judgments of human doctor, LLM and online medical calculator in critically ill patients for the interpretation of acid-base status. MDCalc showed that all patients had mixed acid-base disorders while nephrologist that 4 patients had simple acid-base disorder and 1 patient had normal acid-base balance. By contrast, ChatGPT reported that 51 patients had only simple acid-base disorder. Furthermore, according to ChatGPT, normal acid-base balance was found in 27 patients, who were all diagnosed as having acid-base disorders by MDCalc or nephrologist. Conclusions: We found that current chatGPT does not yet provide better diagnostic performance for interpreting acid-base balance in critically ill patients who with usually have mixed acid-base disorders compared with an existing online medical calculator or nephrologist.
원문(PDF)	PDF 원문보기

목록

위로가기