Acute Care Learning Laboratory—Reducing Threats to Diagnostic Fidelity in Critical Illness
Principal Investigator: Brian Pickering, M.B., B.Ch., M.Sc., Mayo Clinic, Rochester, MN
AHRQ Grant No.: HS26609
Project Period: 09/30/18-11/30/22
Description: The overall goal of this learning lab was to reduce the rate of diagnostic error or delay (DEOD) in acutely ill patients by establishing and using an in situ acute-hospital learning lab.1-2 Researchers engaged key stakeholders to identify threats to the diagnostic process and guide the design, development, testing, implementation, and evaluation of interventions targeting system vulnerabilities.
The specific aims were to:
- Develop and validate automated phenotypes of DEOD that can be applied in near-real time to medical record data.
- Engage stakeholders using mixed methods and systems engineering approaches to identify factors that contribute to DEOD, then design and develop applicable system-based interventions.
- Evaluate the feasibility and preliminary effectiveness of learning lab interventions on the rate of DEOD in patients with emerging critical illness.
This PSLL developed a unique clinical informatics platform, Acute Care Multi-Patient Platform (Control Tower/AMP), that enables health information technology (IT) innovations to be developed with stakeholder input and implemented within the hospital. The resulting applications are built and supported locally, using agile methodologies and user-centered design principles.3
A pilot study was conducted to determine if the Control Tower/AMP could reduce the time to clinical decision of 10 pairs of clinicians caring for acutely ill patients. Results showed that it significantly reduced the time to clinical task completion and clinician task load compared with the standard electronic medical record (EMR). While the Control Tower/AMP is not available for use by other organizations/institutions, it may be in the future.
Additional research is needed to determine clinicians’ performance while using the Control Tower/AMP in the “live” intensive care unit (ICU) setting. But this project highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to DEOD. Researchers conducted 10 studies, with some of the findings3-6 showing that:
- Twenty percent of patients were considered to have experienced DEOD as a primary contributor to the deterioration event. A similar percentage (20%) could have benefited from earlier engagement of a specialist in diagnostic evaluation and treatment planning.
- Four types of factors interact in complex ways to impede diagnostic performance:
- Organizational (e.g., infrastructure, workload, tools, processes).
- Interactional (e.g., communication, coordination, roles, power).
- Individual clinician (e.g., bias, experience, ego).
- Individual patient (e.g., health literacy, medical complexity, acuity of illness).
- ICU operational conditions may contribute to cognitive overload and affect clinical decision making. ICU operational factors, such as admission rates and patient severity of illness, may affect the critical care team’s cognitive function and result in changes in the production of medication orders.
- EMR manual review is of limited reliability in the real-time identification of DEOD in hospitalized patients.
- Health IT for early detection of patient deterioration in acute care settings was not significantly associated with improved mortality or length of stay in a meta-analysis of randomized controlled trials.
The lab is collaborating with institutional clinical, administrative, and technology leadership to further develop AMP as a resource for the real-time identification of patient vulnerability to physiologic deterioration resulting from DEOD. Taking advantage of institutional data resources, real-time data processing capabilities, and patient clinical data, the laboratory is testing novel approaches to the detection of diagnostic error within the context of a complex social-technical system.
As it prepares for pilot testing, the lab plans to apply for funding to evaluate the preliminary effectiveness of AMP to positively affect the diagnostic performance of rapid response systems that have access to AMP interventions in clinical practice. The lab is ideally placed to capture lessons learned from the resulting work system reorganization before, during, and after implementation.
To date, this PSLL’s work has resulted in more than 20 peer-reviewed journal publications related to diagnostic error (selected articles presented below), with more than 300 citations in other publications. The work has directly contributed to the team’s increased understanding of the role technology can play in the identification of patients vulnerable to diagnostic error and informed proposals that have been funded by the institution to improve patient care.
Publications
2023
- Herasevich S, et al. Diagnostic error among vulnerable populations presenting to the emergency department with cardiovascular and cerebrovascular or neurological symptoms: a systematic review. BMJ Qual Saf 2023 Nov;32(11):676-88. Epub 2023 Mar 27.
- Herasevich S, et al. Evaluation of digital health strategy to support clinician-led critically ill patient population management: a randomized crossover study. Crit Care Explor 2023 May 3;5(5):e0909.
- Pinevich Y, et al. Time to diagnostic certainty for saddle pulmonary embolism in hospitalized patients. Biomol Biomed 2023 Jul 3;23(4):671-79.
- Tekin A, et al. Diagnostic delay in pulmonary blastomycosis: a case series reflecting a referral center experience. Infection 2023 Feb;51(1):193-201. Epub 2022 Jul 1.
- Valik JK, et al. Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data. Sci Rep 2023 Jul 20;13(1):11760.
- Wilson PM, et al. Effect of an artificial intelligence decision support tool on palliative care referral in hospitalized patients: a randomized clinical trial. J Pain Symptom Manage 2023 Jul;66(1):24-32. Epub 2023 Feb 24.
2022
- Herasevich S, et al. The impact of health information technology for early detection of patient deterioration on mortality and length of stay in the hospital acute care setting: systematic review and meta-analysis. Crit Care Med 2022 Aug 1;50(8):1198-209. Epub 2022 Apr 12.
- Huang C, et al. Bedside clinicians’ perceptions on the contributing role of diagnostic errors in acutely ill patient presentation: a survey of academic and community practice. J Patient Saf 2022;18(2):e454-e462.
- Johnson SW, et al. Hospital variation in management and outcomes of acute respiratory distress syndrome due to COVID-19. Crit Care Explor 2022 Feb 18;10(2):e0638.
- Lindroth HL, et al. Information and data visualization needs among direct care nurses in the intensive care unit. Appl Clin Inform 2022;13(5):1207-13. Epub 20221228.
- Park JP, et al. Investigating the cognitive capacity constraints of an ICU care team using a systems engineering approach. BMC Anesthesiology 2022.
- Redmond S, et al. Contributors to diagnostic error or delay in the acute care setting: a survey of clinical stakeholders. Health Serv Insights 2022;15:11786329221123540.
- Zhong X, et al. A multidisciplinary approach to the development of digital twin models of critical care delivery in intensive care units. Int J Prod Res 2022;60(13):4197-213. https://www.tandfonline.com/doi/abs/10.1080/00207543.2021.2022235.
2021
- Barwise A, et al. What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the United States. J Patient Saf 2021;17(4):239-48.
- Domecq JP, et al. Outcomes of patients with coronavirus disease 2019 receiving organ support therapies: the International Viral Infection and Respiratory Illness Universal Study Registry. 2021 Mar 1;49(3):437-48. Erratum in: Crit Care Med 2021 May 1;49(5):e562.
- Garcia MA, et al. Variation in use of repurposed medications among patients with coronavirus disease 2019. From the Society of Critical Care Medicine Discovery Viral Infection and Respiratory Illness Universal Study: Coronavirus Disease 2019 Registry Investigator Group. Crit Care Explor 2021 Nov 2;3(11):e0566.
- Huang C, et al. Clinical characteristics, treatment, and outcomes of critically ill patients with COVID-19: a scoping review. Mayo Clin Proc 2021;96(1):183-202.
- Wilson PM, et al. Improving time to palliative care review with predictive modeling in an inpatient adult population: study protocol for a stepped-wedge, pragmatic randomized controlled trial. Trials 2021;22(1):635. Epub 20210916.
2020
- Soleimani J, et al. Feasibility and reliability testing of manual electronic health record reviews as a tool for timely identification of diagnostic error in patients at risk. Appl Clin Inform 2020;11(03):474-82.
2019
- Jayaprakash N, et al. Improving diagnostic fidelity: an approach to standardizing the process in patients with emerging critical illness. Mayo Clin Proc Innov Qual Outcomes 2019;3(3):327-34. Published 2019 Jul 19.
2018
- Gajic O, et al. Outcomes of critical illness: what is meaningful? Curr Opin Crit Care 2018;24(5):394-400.
References
- AFYA, PSLL Learning Network Webinar #12 Report. [Internal Communication]: AHRQ; 2020.
- Pickering BW. Research Plan: Acute Care Learning Laboratory - Reducing Threats to Diagnostic Fidelity in Critical Illness. Rochester, MN: Mayo Clinic; 2018.
- Pickering B. Final Report: Acute Care Learning Laboratory - Reducing Threats to Diagnostic Fidelity in Critical Illness. AHRQ, ed. Rochester, MN: Mayo Clinic; 2023.
- Soleimani J, et al. Feasibility and reliability testing of manual electronic health record reviews as a tool for timely identification of diagnostic error in patients at risk. Appl Clin Inform 2020;11(3):474-82.
- Redmond S, et al. Contributors to diagnostic error or delay in the acute care setting: a survey of clinical stakeholders. Health Serv Insights 2022;15:11786329221123540.
- Park J, et al. Investigating the cognitive capacity constraints of an ICU care team using a systems engineering approach. BMC Anesthesiol 2022;22(1):10.
