Abstract: Performance of a Clinic-deployed Model to Predict Diabetic Ketoacidosis (DKA) Risk in Type 1 Diabetes (T1D)
C.A. Vandervelden1, B. Lockee1, M. Barnes1, E.M. Tallon1, D.D. Williams1, J. Kohlenberg2, R. Sonabend3, S.R. Patton4, S. Mehta5, M.A. Clements1
1Children’s Mercy Kansas City, Kansas City, United States, 2University of Minnesota, Minneapolis, United States, 3Texas Children’s Hospital, Houston, United States, 4Nemours Children’s Health, Orlando, United States, 5Joslin Diabetes Center, Boston, United States
Introduction: DKA is a severe, potentially life-threatening complication of T1D. Predicting DKA risk may allow for preventive interventions. Objectives: We assessed the performance of a clinic-deployed machine learning model that identifies youth with T1D at risk for DKA-related hospitalization.
Methods: DKA risk predictions using production data for a cohort of 2605 youth with T1D (53% male) aged=13.5±4.3 yr. (mean±SD) receiving care from a network of US diabetes centers between 9 Jun 2022 and 6 Mar 2023. Each week, the deployed model predicted 6-month DKA risk for all youth who had a clinic visit within the prior 90 days. The weekly cohort was rank-ordered by probability of hospitalization. We considered the top R=100, 50, 25, or 10 highest-risk individuals as potential thresholds for a clinical intervention. We calculated the weekly precision, the fraction of people in the top R who experienced DKA, and recall, the fraction of DKA events successfully placed in the top R, for each threshold. This approach replicates how a clinic may prioritize people for preventive interventions based on each individual’s risk. An updated model including average time between DKA episodes, added on 28 Dec 2022, was also evaluated.
Results: Overall, 74 DKA events occurred within 6 months of a baseline prediction for 5158 clinic visits (1.9%). Using the top 100 as the DKA threshold, 21% of youth in the top 100 experienced DKA (precision) and 57% of all DKA events were in the top 100 (recall), on average. Of the 41 DKA events which occurred following the deployment of the updated model, 24% of youth in the top 100 experienced DKA (precision) and 79% of all DKA events were in the top 100 (recall), on average
Conclusions: We previously developed and clinically deployed a model to predict 6-month DKA risk. The model successfully flagged the majority of DKA cases as requiring additional care. Future efforts will assess the efficacy of preventive interventions for reducing DKA-related hospitalization.
Link: https://www.react-profile.org/upload/KIT/system/uploads/ISPAD2023_Abstracts.pdf