Objective: Hospital-acquired pressure injury (HAPrI) risk assessment is necessary for allocating limited preventative resources. This study is a response to a professional wound society call for accurate, efficient, and clinician-interpretable HAPrI risk prediction algorithms.
Method: We used EHR data from a surgical ICU to develop and test a series of machine-learning-based algorithms to predict HAPrI events. In addition to assessing the performance of the models, we employed model-agnostic interpretation methods to assess models in a human-interpretable way.
Results: Our final sample consisted of 5,101 adult surgical and cardiovascular surgical ICU patients (split into 80:20 train:test sets).The sample was predominantly male (n = 3302, 65%) and White (n = 4256, 83%). The mean age was 58 years (SD = 17) and HAPrI developed among 333 patients (6.5%). Predictions from three machine-learning models (extreme gradient boosted machines, random forests, and logistic regression) had similar performance on the held-out test data (AUC: 0.80-0.81; F1: 0.28-0.33). However, model-agnostic interpretation methods (which take models and additional metadata for uniform comparisons between models) indicate differential importance of variables (feature importance), (individual) conditional expectations, and local effect fluctuations. That is, although models perform similarly in aggregate, they make different use of variables and can predict different outcomes for clinically similar patients.
Graphically, caeteris paribus (“all other things being equal”) plots, generated for simulated patients, provide clinician-interpretable demonstrations of instance-level feature importance for each model. Figures will show the comparative effect of key variables, e.g., logistic regression’s smooth, domain-long leverage of hours in ICU vs both other models' sharp steps and disparate HAPrI predictions for this variable. Additionally, partial dependence profiles show the dataset-level performance for each specific feature and indicate differential use of variables for predictions.
Conclusion: Among critical care patients, most are designated high-risk when using industry-standard assessment tools, such as the Braden Scale. Machine learning approaches have been used to develop algorithms that may better discriminate high-risk critical care patients than these standard tools. Better discrimination would allow for more judicious use of finite resources. In 2019, the National Pressure Injury Advisory Panel clinical practice guidelines called for human interpretable machine learning for HAPrI risk prediction. The objective is to demonstrate clinician-interpretable HAprI risk prediction via explainable-artificial intelligence (XAi).
Using existing EHR data to develop risk algorithms allows for near real-time risk assessment and reduction in redundant charting. However, given the black-box nature of many machine-learning algorithms, it is difficult to assess the influence of each feature on instance-level outcome performance or on the dataset-level model performance.
Using XAi techniques, we were able to understand how specific features in the model influenced the predicted outcome. Interestingly, some features (e.g., a protective effect of acidosis and hypoxemia) seemed counterintuitive to the clinical understanding of HAPrI development, and each of the three models dealt with them differently. Knowing this, it may help guide which variables to keep in a model, and lead to expert-augmented machine learning approaches which have better clinical value.
Clinicians are reluctant to trust and incorporate Ai, even when it outperforms clinician judgment. By enabling clinicians to understand algorithm decision-making on the patient level, XAi will enhance trust and also enable clinicians to recognize exceptions-- cases where clinician judgment should be considered alongside the algorithm. Additionally, XAi may support collaborations between clinicians and data scientists to detect anomalies and augment algorithms to enable expert-augmented machine learning.

Machine learning for clinician-interpretable algorithm development: pressure injury safety surveillance using explainable-Ai

[Back to the conference](https://diaeurope.cmvirtual.eu/)

DIA Europe 2021

A global, member-driven non-profit organization mobilizing life sciences and healthcare. DIA fosters innovation to improve health and well-being worldwide. Professionals from 80 countries continue to engage with DIA through our unparalleled membership network, educational offerings, and professional development opportunities.