Human–AI Collaboration for Clinical Decision Support: An HCI Design Framework

Abstract

Artificial intelligence (AI) is increasingly embedded in clinical decision support (CDS) systems for triage, diagnosis, and care planning. Yet, improvements in model performance do not automatically translate into safer or better clinical decisions. In high-stakes environments such as emergency departments (EDs), the success of CDS hinges on human–computer interaction (HCI): how information is presented, when and how recommendations arrive, and how accountability, oversight, and feedback are managed within complex workflows. This paper advances a comprehensive HCI framework for human–AI collaboration in CDS across three layers: (1) information design (uncertainty-forward summaries, contrastive explanations, progressive disclosure), (2) coordination design (workflow-aligned timing, interruption management, handoff support), and (3) governance design (provenance, auditability, and clinician override as first-class operations). We report findings from a mixed-methods program: 36 hours of contextual inquiry in two EDs; two controlled studies with 72 clinicians comparing uncertainty encodings and explanation patterns; and an eight-week field deployment of a modular interface layer running in shadow mode over an existing risk-prediction model. The interface variants with uncertainty-aware summaries and counterfactual explanations produced a statistically significant reduction in over-treatment (–14%, p < .05), improved trust calibration (+27% reduction in calibration error, p < .01), and decreased handoff screen-switching (–22%). The deployment preserved decision time within operational thresholds (median change +6s, n.s.) and reduced post-hoc diagnostic revisions (–19%). We surface risks, including automation bias induced by persuasive explanations and alert fatigue from mistimed prompts, and we propose concrete mitigations. The paper contributes: a rigorously evaluated design framework; reusable UI patterns with parameterizations for uncertainty and contrastive reasoning; and a governance checklist to support safe adoption, audit, and continual improvement.