Search

The integration of large language models (LLMs) into clinical healthcare systems represents a transformative shift in how data analytics, decision support, and operational infrastructure are conceptualized and deployed. This narrative review synthesizes recent advancements in LLMs within healthcare, focusing on their roles in enhancing clinical analytics, infrastructural frameworks, and oversight mechanisms while addressing inherent risk dynamics. Drawing from peer-reviewed literature, we examine how LLMs facilitate the processing of vast unstructured clinical data, such as electronic health records and patient narratives, to generate actionable insights that inform diagnostics, treatment planning, and resource allocation. Key infrastructural elements include scalable deployment pipelines that integrate LLMs with existing hospital information systems, enabling real-time analytics and predictive modeling without disrupting legacy workflows. Oversight is emphasized through regulatory frameworks that ensure ethical deployment, data privacy compliance, and bias mitigation, as LLMs amplify risks related to misinformation, algorithmic opacity, and equitable access in diverse clinical settings. Risk dynamics are explored in terms of model hallucinations, dependency on training data quality, and potential for exacerbating healthcare disparities if not properly governed. The review highlights systems-level analytics where LLMs contribute to closed-loop healthcare ecosystems, from data ingestion and inference to feedback-driven recalibration, fostering adaptive intelligence in clinical decision-making. For instance, LLMs have been adapted for tasks like text summarization, diagnostic reasoning, and patient communication, outperforming traditional methods in efficiency while requiring robust validation to maintain clinical fidelity. We underscore the need for interdisciplinary collaboration between clinicians, data scientists, and policymakers to harness LLMs' potential in optimizing healthcare delivery. By synthesizing cross-study evidence, this review proposes an original interpretive framework for LLM-enabled healthcare systems, structured around data-model-deployment-governance cycles, to guide future implementations. Ultimately, while LLMs promise enhanced analytics and infrastructural resilience, their clinical adoption demands vigilant oversight to balance innovation with patient safety and ethical integrity. This synthesis not only maps the current landscape but also identifies infrastructural gaps in scaling LLMs for equitable, high-stakes clinical environments, paving the way for more resilient healthcare analytics paradigms.

The integration of artificial intelligence into clinical decision support systems offers improved diagnostic accuracy and efficiency, but the opacity of many machine learning models raises concerns about trust, accountability, and regulatory compliance. Explainable artificial intelligence (XAI) has been proposed to address this by making model predictions interpretable to clinicians; however, its true clinical value remains uncertain, and evaluation has not kept pace with methodological development. This systematic review aimed to identify XAI methods used in clinical decision support systems, assess how they are evaluated with clinicians, and determine whether explanations improve diagnostic accuracy, trust, mental models, and efficiency. Following PRISMA guidelines, we searched PubMed, Web of Science, IEEE Xplore, ACM Digital Library, and Scopus for studies published between 2017 and 2024. Eligible studies included original research evaluating XAI in clinical decision support systems with clinician participants and reporting quantitative or qualitative outcomes. Risk of bias was assessed using adapted QUADAS-2 and ROBIS tools, and findings were synthesized narratively with subgroup analyses. From 2,847 records, 68 studies were included. The most common XAI methods were SHAP-based feature attribution (38%), saliency or heatmap methods (29%), concept-based approaches such as TCAV (15%), and counterfactual or example-based explanations (12%). Radiology was the dominant field (54%), followed by dermatology (18%) and pathology (12%). Evaluation approaches were highly inconsistent, with few validated instruments and most studies relying on Likert-scale trust measures or qualitative feedback. Only 16% of studies showed improved diagnostic accuracy with explanations, 67% showed no significant effect, and 17% reported reduced accuracy due to over-reliance or misinterpretation. Although 82% of studies reported increased clinician trust, trust rarely correlated with actual diagnostic performance. Overall, while XAI methods are widely studied in clinical decision support, their evaluation is inconsistent and their benefits are limited. Explanations tend to increase clinician trust without reliably improving diagnostic accuracy, and may sometimes worsen performance, highlighting a trust–accuracy gap that poses important safety concerns for clinical deployment.