Clinical Intelligence Research Press Clinical Intelligence Research Press

Predictive Model for Identifying Delayed Diagnostic Follow-Up After Abnormal Screening Results Using Patient Portal Messages, Scheduling Attempts, Primary Care Workload, Result Severity, and Reminder History

Original Research | Open access | Published: 20 July 2024
Volume 4, article number 96, (2024) Cite this article
You have full access to this open access article.
Download PDF
, , ,
  1. Department of Digital Healthcare Analytics, Faculty of Medicine, University of Toronto, Toronto, Canada
  2. Department of Health Informatics Engineering, Faculty of Engineering, McGill University, Montreal, Canada
132 Accesses

Abstract

Failure to follow up after abnormal screening results is a persistent ambulatory safety problem. Because the diagnostic process often spans patient notification, scheduling, primary care review, and reminder outreach, delays may emerge gradually before they become visible in registry reports. Existing care gap reports commonly classify results as closed or open after a defined time window. This retrospective framing limits the ability to detect patients who are currently moving toward delayed diagnostic resolution. The objective of this article is to describe a predictive model that estimates the probability of delayed diagnostic follow-up after an abnormal screening result. The proposed model integrates patient engagement signals, scheduling activity, clinician workload, result severity, and reminder history. The model would use a gradient-boosted classification framework trained on historical abnormal results and longitudinal care-process features. Inputs would be extracted from patient portals, scheduling systems, primary care workload records, laboratory and radiology result metadata, and reminder logs. Conceptually, the model would generate a daily updated risk score for each unreconciled abnormal result. It would also identify the dominant drivers of risk, such as unread portal messages, repeated appointment cancellations, limited primary care availability, high-severity findings, or escalating reminder activity. A predictive approach could shift delayed follow-up management from retrospective audit to proactive prioritization. By identifying patients at greatest risk before the diagnostic window closes, care teams could better allocate outreach and navigation resources.

Explore related subjects
Discover the latest articles in related subjects:

Introduction

Delayed follow-up after abnormal screening results represents a preventable pathway to diagnostic delay, particularly when abnormal stool tests, mammograms, lung screening results, imaging findings, or critical laboratory values require timely diagnostic resolution. Studies of colorectal cancer screening have shown that failure to complete follow-up colonoscopy after an abnormal stool-based test can increase downstream cancer risk, and similar safety concerns arise when abnormal imaging recommendations are not reconciled in routine care [1-7]. The ambulatory setting is especially vulnerable because responsibility for follow-up may be distributed across laboratories, radiology departments, primary care clinicians, specialists, scheduling staff, and patients. A predictive model that identifies which unreconciled results are most likely to become delayed could support earlier intervention before preventable harm occurs [8, 9].

Current approaches to care gap management often rely on registries, manual chart review, or retrospective quality reports that identify whether diagnostic follow-up has already been missed. These methods are useful for accountability but are less effective for real-time prioritization because they generally do not distinguish a patient who is likely to complete follow-up tomorrow from a patient who is drifting toward loss to follow-up [1, 2, 4]. Population-level measures of follow-up completion can reveal performance gaps, yet they may not provide enough detail about patient engagement, appointment friction, or clinician capacity to guide immediate outreach. Predictive analytics could complement these systems by estimating risk while there is still time to close the diagnostic loop [10, 11].

Electronic health records now capture transactional signals that may precede diagnostic delay, including portal message delivery, result viewing, appointment offers, cancellations, rescheduling, no-shows, reminder delivery, and clinician in-basket workload. Patient portal studies suggest that message content, message timing, and access to test results can reflect engagement patterns that are relevant to follow-up behavior [12-17]. Scheduling studies similarly indicate that missed appointments, no-show history, and appointment-management transactions can be modeled as predictors of future adherence [18-24]. Primary care workload and electronic message burden add another layer, because even an engaged patient may experience delayed follow-up when appointment access, panel complexity, or clinician task load is constrained [25-27].

This article describes a predictive model for identifying patients with delayed diagnostic follow-up after abnormal screening results using patient portal messages, scheduling attempts, primary care workload, result severity, and reminder history. The model is conceptualized as a dynamic risk score attached to each unreconciled abnormal result, rather than as a static patient-level risk label. It would combine patient-level behaviors, clinician-level workload pressure, result-level urgency, and system-level outreach history to identify cases needing escalation. Such a model could help care coordinators prioritize outreach, support safety-netting workflows, and make follow-up management more proactive without replacing clinical judgment.

Background

Diagnostic follow-up failures and their causes

Diagnostic follow-up failures emerge from interacting patient, provider, and system factors rather than from a single missed action. A positive stool-based colorectal cancer screening test, an abnormal mammogram, or a suspicious lung screening result may require additional diagnostic procedures, but completion depends on notification, comprehension, scheduling, transportation, insurance, referral coordination, and clinician oversight [1-7]. Ambulatory diagnostic delay analyses also show that health information technology can contribute when alerts, result notifications, or handoffs fail to produce closed-loop action [8]. Therefore, a useful predictive model should represent delay as a trajectory shaped by evolving signals rather than as a simple final status [4, 9].

Patient portal engagement and message logs

Patient portals can provide early evidence of engagement because they record whether results or messages were delivered, opened, and sometimes answered. Randomized and observational work on portal outreach indicates that portal messages may influence visit completion and reattendance, while message-classification studies show that portal content can be structured into clinically meaningful categories for prediction [12-15]. Immediate access to results may also change the timing and nature of patient-clinician communication, making read receipts and message replies potentially informative markers of follow-up readiness or confusion [17]. For a delayed follow-up model, portal inactivity should not be interpreted as patient unwillingness, but it may identify a need for non-portal outreach or navigator support [16].

Scheduling attempts and appointment adherence

Scheduling data provide a behavioral trace of whether the patient is moving toward diagnostic resolution. Machine-learning studies of no-shows and appointment adherence show that prior missed visits, appointment timing, access constraints, and visit characteristics can support risk stratification for attendance behavior [18-24]. For abnormal-result follow-up, repeated cancellations, failed scheduling attempts, long lead times, or rescheduling after diagnostic appointments may indicate increasing risk of delay. These features are especially valuable because they update over time and can therefore convert a static registry entry into a dynamic prediction problem [19, 21].

Primary care workload and access constraints

Primary care workload can affect follow-up timeliness because abnormal results often require review, ordering, referral coordination, patient counseling, and response to portal messages. Event-log and in-basket studies show that electronic health record workload and message burden are substantial components of primary care work, while clinician workload can vary across panels and practice contexts [25-27]. When a primary care clinician has limited appointment availability or high in-basket burden, a patient with an unresolved abnormal result may experience delays even if the need for follow-up is clinically clear. A predictive model should therefore include clinician and panel-level features as contextual risk modifiers rather than relying only on patient behavior [25, 26].

Reminder history and result severity

Reminder history and result severity describe both the intensity of prior outreach and the clinical urgency of the unresolved finding. Interventions to increase follow-up after abnormal cervical cancer screening and other abnormal results show that reminders, navigation, and outreach can improve loop closure, but repeated reminders may also signal that standard communication channels have not worked [9]. Result severity systems such as BI-RADS, Lung-RADS, positive stool-based testing, or critical laboratory flags provide a structured basis for urgency, and studies of imaging and laboratory abnormality prediction illustrate the importance of result-level clinical context [7, 28]. A model should treat high-severity results as time-sensitive while recognizing that low engagement or repeated failed reminders can elevate risk even when the initial result category is less severe [3, 5].

Model Development Overview

High-level predictive pipeline

The proposed pipeline begins when an abnormal screening, imaging, or laboratory result is indexed as requiring diagnostic follow-up. Each night, the system would extract updated features from the electronic health record, patient portal, scheduling platform, primary care panel data, result metadata, and reminder logs, then generate a probability that follow-up will be delayed if no additional intervention occurs [1, 10, 11]. The outcome definition would be tied to failure to complete or initiate the appropriate diagnostic action within a clinically defined window, while the prediction would be made before the window closes. This structure converts follow-up management from a retrospective open-versus-closed registry into an anticipatory risk-monitoring process [4, 8].

Figure 1 illustrates the proposed dynamic prediction pipeline that transforms abnormal-result events and longitudinal care-process signals into explainable, tiered follow-up actions.

Figure 1. Dynamic Predictive Pipeline for Identifying Delayed Diagnostic Follow-Up after Abnormal Screening Results

Figure 1. Dynamic Predictive Pipeline for Identifying Delayed Diagnostic Follow-Up after Abnormal Screening Results

Core input features

Core inputs would include patient portal activity, scheduling attempts, primary care workload, result severity, and reminder history. Portal features could include whether the result or message was viewed, elapsed time since notification, whether a patient replied, and whether message content suggests confusion or barriers [12-17]. Scheduling features could summarize appointment invitations, accepted slots, cancellations, no-shows, rescheduling frequency, and time to the next available diagnostic or primary care visit [18-24]. Workload, severity, and reminder features would add context by representing clinician capacity, urgency of the abnormal result, the number and type of prior reminders, and whether standard outreach has already failed [9, 25-28].

Design principles

The model should be dynamic, transparent, program-specific, and privacy-preserving. Dynamic updating is necessary because risk can change when a patient opens a portal message, cancels a diagnostic appointment, receives a reminder, or completes a primary care visit [12, 19, 21]. Transparency is essential because care coordinators need to understand whether the risk score is driven by patient nonresponse, scheduling friction, clinician workload, or result severity before choosing an outreach strategy [8, 9]. Program specificity is also important because mammography, colorectal screening, lung screening, and laboratory follow-up differ in urgency, workflow, terminology, and acceptable diagnostic windows [3, 5, 7, 28].

Data Sources and Feature Engineering

Indexing abnormal results and defining delayed follow-up

Abnormal results would be indexed from structured laboratory fields, radiology result categories, screening registries, and coded recommendations when available. For colorectal cancer screening, the model could define an index event as a positive stool-based test requiring colonoscopy, while mammography and lung screening use structured severity categories and follow-up recommendations to determine diagnostic expectations [1-7]. For laboratory and imaging follow-up, critical flags, abnormality severity, and documented recommendation text could be mapped to a program-specific follow-up window [8, 28]. The delayed follow-up label should reflect failure to complete the clinically indicated next step within the specified window, while preserving a distinction between completed diagnostic resolution and merely scheduled future care [2, 4].

Patient portal and scheduling features

Portal features would transform communication logs into time-aware predictors, including days since result release, whether the message was opened, whether the patient replied, and whether additional messages were sent by the care team. Prior studies of portal outreach and message classification support the feasibility of extracting structured signals from message metadata and content, although these signals require careful interpretation because portal use varies across populations [12-17]. Scheduling features would include the number of appointment offers, scheduled diagnostic visits, cancellations, reschedules, no-shows, and elapsed time between the abnormal result and each scheduling transaction [18-24]. Time-decayed versions of these features would allow recent cancellations or unread messages to influence risk more strongly than older events that may have already been resolved [19, 21].

Table 1 presents the proposed feature architecture linking each data domain to its predictive role, temporal meaning, and operational interpretation.

Table 1. Conceptual Feature Architecture for Predicting Delayed Diagnostic Follow-Up

Feature domain

Example variables

Temporal interpretation

Predictive contribution

Operational interpretation

Patient portal engagement

Result viewed, message opened, patient reply, time since notification

Indicates whether the patient has encountered or responded to the abnormal-result communication

Detects early nonengagement or possible communication failure

May suggest need for phone outreach, language support, or nonportal contact

Scheduling activity

Appointment offers, accepted slots, cancellations, reschedules, no-shows, lead time

Shows whether the patient is moving toward diagnostic completion

Identifies friction in appointment completion before the deadline closes

May suggest scheduling assistance, transportation support, or navigator involvement

Primary care workload

Panel size, in-basket burden, appointment availability, clinician task volume

Captures system capacity around the follow-up episode

Distinguishes patient-level delay from clinic-capacity constraints

May suggest team-based escalation rather than repeated patient reminders

Result severity

BI-RADS, Lung-RADS, positive stool test, critical lab flag, recommendation urgency

Defines clinical deadline and harm potential

Prevents urgent findings from being treated as routine open loops

May trigger clinician review or accelerated diagnostic coordination

Reminder history

Number of reminders, modality, timing, response after reminder, failed outreach

Reflects intensity and effectiveness of prior outreach

Identifies patients not responding to standard communication channels

May justify escalation from automated reminders to live outreach

Follow-up status signals

Ordered test, scheduled diagnostic visit, completed exam, outside care documentation

Separates true delay from active progression toward closure

Reduces false prioritization of already progressing cases

Supports accurate worklist triage and closure tracking

Primary care workload and reminder intensity

Primary care workload features would summarize clinician and panel context, such as panel size, panel complexity, visit availability, in-basket volume, and recent electronic health record work intensity. Evidence on electronic health record event logs, physician workload, and in-basket burden suggests that workload can be quantified from operational data and may affect the timeliness of ambulatory follow-up [25-27]. Reminder intensity would be represented by the number, timing, and modality of reminders, including portal notifications, automated calls, text messages, letters, or staff outreach attempts [9, 12, 13]. In the model, repeated reminders without scheduling progress would be interpreted as a sign that usual outreach may be insufficient, whereas a recent successful contact or scheduled diagnostic appointment would be expected to reduce predicted delay risk [4, 21].

Predictive Model Architecture

Model choice and rationale

A gradient-boosted classification model is an appropriate conceptual architecture because abnormal-result follow-up data combine structured categories, time intervals, counts, missing values, and nonlinear interactions. Prior predictive modeling studies in screening follow-up, appointment adherence, and care-gap contexts show that machine-learning approaches can use electronic health record and operational data to identify patients at risk for noncompletion or missed visits [10, 11, 18-24]. Gradient-boosted trees are especially suited to capturing interactions such as unread portal messages becoming more concerning when paired with repeated cancellations or limited appointment availability. The model would be developed as a decision-support tool that supports prioritization, not as a replacement for clinical review or patient-centered communication [8, 9].

Input feature vector and preprocessing

The input feature vector would include patient-level, result-level, clinician-level, and system-level variables measured at a defined prediction time after the abnormal result. Missing portal activity would be encoded carefully, because absence of a portal read could mean nonuse, access barriers, message nonreceipt, or use of an alternative communication channel rather than disengagement [12, 16, 17]. Result severity categories would be encoded from structured screening or radiology fields, while scheduling and reminder features would be transformed into counts, elapsed times, recent-event indicators, and time-decayed measures [7, 18-21]. Workload features would be standardized within clinic or clinician context so that high panel burden or message volume is interpreted relative to the operational environment in which follow-up occurs [25-27].

Output: delayed follow-up probability

The model output would be a calibrated probability that an unreconciled abnormal result will become delayed without additional action. This score could be recomputed daily and grouped into operational tiers so that care coordinators can distinguish routine monitoring from cases needing live outreach, navigation, or clinician escalation [1, 4, 9]. The output should be accompanied by the leading risk contributors, such as unread portal notification, multiple failed scheduling attempts, high-severity result, heavy primary care workload, or repeated reminders without response [7, 8, 12, 13, 25-27]. In practice, the value of the model would depend on whether the score is embedded into a closed-loop workflow that tracks outreach, diagnostic completion, and unresolved risk over time [2, 8].

Handling Temporal Signals and Censoring

Defining the observation window

The observation window should be defined relative to the date and time of the abnormal result, with prediction made at a specific day after that result and outcome assessed at a later diagnostic deadline. This structure is important because patient portal views, scheduling attempts, reminder delivery, and primary care actions occur in sequence, and using information that happens after the prediction time would create future leakage [12, 13, 18, 19]. For example, a model predicting delay early after a positive stool test or abnormal imaging result should use only signals available up to that point, not later evidence that colonoscopy, diagnostic imaging, or specialist evaluation eventually occurred [1-7]. Time-based splitting between development and validation periods would also be needed so that the model reflects prospective use rather than random mixing of older and newer care processes [8, 10, 11].

Updating risk scores with new information

Risk should be treated as dynamic because each new event can change the likelihood of delayed diagnostic follow-up. A patient who opens a portal result, replies to a message, or schedules a diagnostic appointment may move toward lower risk, while repeated cancellations, no-shows, or reminders without response may increase concern [12-15, 18-24]. Primary care workload can also change over time, because clinician availability, in-basket burden, and appointment access may fluctuate across weeks or clinic sessions [25-27]. Daily updating would allow the worklist to reflect current trajectory rather than a fixed baseline assessment at the moment the abnormal result was released [4, 8, 9].

Handling censored cases

Censoring arises when the follow-up window has not fully elapsed, when a patient transfers care, when insurance or network status changes, or when diagnostic follow-up may occur outside the captured health system. These cases should be handled in a way that avoids labeling still-actionable patients as delayed or completed simply because observation is incomplete [1, 2, 4]. Survival-oriented approaches, landmark modeling, or carefully defined exclusion rules could be considered depending on the data structure and intended deployment setting. Because loss to observation may itself be associated with portal nonuse, scheduling friction, or access barriers, sensitivity analyses should examine whether censoring patterns differ across patient groups and screening programs [9, 16, 17, 21].

Model Interpretability and Outreach Guidance

Explaining predictions to care coordinators

Model explanations should show why a specific unreconciled result is considered high risk, not merely assign a numerical score. Feature-attribution displays could identify whether the main drivers are unread portal messages, absence of a patient reply, repeated rescheduling, high result severity, limited primary care availability, or reminder fatigue [7, 12-15, 18-21]. This interpretability is especially important in ambulatory safety workflows because coordinators must decide whether to send another electronic reminder, call the patient, involve a nurse navigator, or request clinician review [8, 9]. Explanations should be framed as decision support and should avoid implying that patient behavior alone caused the delay when system workload and access constraints may be contributing [25-27].

Actionable patient lists and tiered interventions

The practical output of the model would be a daily prioritized worklist organized by predicted risk and dominant modifiable barrier. Lower-risk patients might continue to receive standard automated reminders, while higher-risk patients could be escalated to phone outreach, navigation, scheduling assistance, or clinician review depending on the explanation attached to the score [9, 12, 13]. Scheduling-derived risk signals would be particularly useful for distinguishing patients who need appointment access support from those who need clarification about the abnormal result or the recommended next step [18-24]. The worklist should also preserve clinical urgency so that a severe imaging or laboratory result is not deprioritized simply because the patient appears engaged in the portal [7, 8, 28].

Table 2 translates predicted delay risk and dominant explanatory drivers into tiered follow-up actions that preserve clinical urgency and workflow accountability.

Table 2. Risk-Tiered Intervention Logic for Model-Guided Follow-Up Management

Risk tier

Typical model pattern

Dominant explanation profile

Recommended action

Accountability owner

Safety consideration

Tier 1: Routine monitoring

Low predicted delay risk; follow-up scheduled or patient engaged

Portal viewed, appointment accepted, low-severity result, no failed reminders

Continue standard reminder pathway and monitor closure

Registry or population health staff

Avoid unnecessary outreach burden

Tier 2: Communication concern

Moderate risk with weak portal engagement

Unread message, no reply, delayed result viewing

Use nonportal outreach such as phone call, letter, interpreter-supported contact, or proxy clarification

Care coordinator or medical assistant

Do not interpret portal nonuse as patient unwillingness

Tier 3: Scheduling friction

Moderate-to-high risk with appointment instability

Cancellations, no-shows, long lead time, repeated rescheduling

Provide scheduling assistance, navigation, transportation screening, or appointment access support

Navigator or scheduling team

Address access barriers rather than blaming adherence

Tier 4: System-capacity risk

High risk with workload or access constraints

Heavy in-basket burden, limited visit availability, clinician panel pressure

Escalate to team-based workflow, covering clinician, or centralized follow-up pool

Primary care team lead or clinic manager

Ensure workload features guide support rather than penalize clinicians

Tier 5: Urgent unresolved finding

High severity regardless of engagement status

Critical lab, suspicious imaging, high-priority screening category

Immediate clinician review and accelerated diagnostic coordination

Responsible clinician or safety escalation pathway

Severity should override routine prioritization rules

Tier 6: Refractory outreach failure

High risk with repeated reminders and no progress

Multiple reminders, no scheduling movement, failed contact attempts

Switch from automated reminders to live outreach, navigation, or documented escalation

Care coordinator and clinician reviewer

Track unresolved status and document outreach attempts clearly

Operational Deployment and Care Gap Workflow

Integration into population health and closing-the-loop systems

Deployment should occur within an existing population health or closing-the-loop infrastructure rather than as a standalone dashboard. The model would ingest nightly feeds from screening registries, laboratory and radiology systems, scheduling platforms, portal logs, reminder systems, and primary care workload sources, then return prioritized abnormal results to the team responsible for follow-up [1, 2, 8]. Integration with current registries is important because many organizations already track open abnormal results, but those systems often need stronger prioritization logic to guide same-day outreach decisions [4, 9]. The workflow should make clear which staff member owns each action, how escalation occurs, and when clinician review is required for ambiguous or severe findings [7, 28].

Closed-loop tracking and outcome monitoring

Closed-loop tracking should record not only whether follow-up was eventually completed but also which outreach actions were attempted, when they occurred, and whether they changed the patient’s trajectory. Such tracking would allow the system to distinguish a reminder that led to scheduling from one that produced no response, and it would help identify when repeated automated reminders should give way to live outreach or navigation [9, 12, 13]. Outcome monitoring should capture diagnostic exam ordering, appointment completion, documented outside follow-up, patient refusal, transfer of care, or unresolved status so that the model does not treat all open loops as equivalent [1-5]. These logged outcomes would also support periodic model updating as screening programs, portal adoption, appointment availability, and primary care workload patterns change over time [10, 11, 25-27].

Evaluation Strategy

Predictive performance

The model should be evaluated with metrics that reflect the clinical purpose of prioritizing rare but consequential delayed follow-up events. Discrimination, precision-recall behavior, calibration, and performance at operationally meaningful thresholds should be assessed, while avoiding reliance on a single summary metric that may obscure whether high-risk worklists are usable for care coordinators [10, 11, 18-24]. Calibration is especially important because the model output is intended to guide outreach intensity, not simply rank patients in abstract order. Performance should also be examined across result types, severity categories, portal-use patterns, scheduling histories, and primary care workload contexts to ensure that risk scores remain clinically interpretable [7, 9, 12-17, 25-28].

Temporal and external validation

Temporal validation should test whether a model developed using earlier abnormal results remains useful on later results after workflows, staffing patterns, portal use, or reminder practices have changed. This is essential because screening follow-up processes can evolve as organizations introduce centralized registries, new portal policies, automated reminders, or navigation programs [1, 2, 9, 12, 13]. External validation in a different health system would be needed to assess whether the same feature logic generalizes across scheduling platforms, patient populations, primary care capacity, and diagnostic follow-up pathways [4, 8, 10, 11]. Particular attention should be paid to whether workload and access variables retain meaning across systems, because panel structure and in-basket practices may differ substantially [25-27].

Prospective impact on diagnostic delay rates

A prospective evaluation should examine whether model-guided outreach improves diagnostic loop closure compared with usual registry-based care. A pragmatic randomized or stepped-wedge design could compare standard follow-up workflows with prioritized outreach guided by the model, while measuring time to diagnostic resolution, unresolved abnormal results, care coordinator workload, and patient experience [1-5, 8, 9]. Because the model is intended to support intervention rather than simply prediction, evaluation should also assess whether explanations lead to appropriate outreach choices, such as scheduling help for repeated cancellations or nonportal contact for unread messages [12-24]. Any prospective study should monitor unintended consequences, including overburdening staff, over-contacting patients, or widening inequities among patients with limited portal access [16, 17, 25-27].

Limitations

Data completeness and behavioral assumptions

The model would depend on the completeness and meaning of electronic traces, and these traces may not fully represent patient intent, access, or communication outside the health system. A portal message that is not opened may indicate digital exclusion, proxy access, language barriers, or preference for phone communication rather than lack of concern [12, 16, 17]. Scheduling records may miss patient-initiated calls, external appointments, transportation barriers, or informal coordination by clinic staff, while reminder logs may confirm delivery attempts without proving comprehension [9, 18-24]. Workload features also require cautious interpretation because electronic health record activity and in-basket volume are imperfect proxies for clinician capacity and team-based follow-up support [25-27].

Generalizability across screening programs and populations

A model trained for one abnormal-result pathway may not generalize directly to another because follow-up urgency, workflows, patient expectations, and diagnostic endpoints vary across mammography, lung screening, colorectal screening, cervical screening, imaging recommendations, and laboratory abnormalities. Severity systems such as Lung-RADS, radiology follow-up categories, positive stool testing, and critical laboratory values differ in clinical meaning, making program-specific feature definitions and outcome windows necessary [1-7, 9, 28]. Generalizability may also be limited by differences in patient portal adoption, reminder modalities, appointment supply, primary care panel structure, and navigator availability across health systems [12-17, 25-27]. Therefore, local validation, careful recalibration, and workflow-specific implementation planning would be needed before operational use [8, 10, 11].

Conclusion

A predictive model for delayed diagnostic follow-up after abnormal screening results could help ambulatory care teams identify risk while intervention is still possible. By treating each abnormal result as a dynamic follow-up episode, the model would move beyond static open-loop reporting and toward prospective safety management. Its purpose would be to support timely diagnostic resolution through earlier recognition of emerging delay patterns. Such a model would be most useful when embedded in a workflow that assigns responsibility, documents outreach, and tracks closure.

The key strength of the proposed approach is its integration of patient engagement, scheduling activity, primary care workload, result severity, and reminder history into a single actionable signal. Portal views and replies can suggest whether the patient has received and engaged with the result, while scheduling activity can show whether follow-up is progressing or stalling. Workload measures add system context, and severity measures preserve clinical urgency. Reminder history then helps distinguish patients who are responding to routine outreach from those who may need escalation.

Important challenges remain before such a model could be trusted in routine care. Data heterogeneity, incomplete capture of outside follow-up, inconsistent portal use, and variable scheduling workflows may affect both prediction and fairness. Patient-centered privacy safeguards would be essential because the model relies on communication and behavior traces that require careful governance. Prospective evidence would also be needed to show that model-guided outreach closes diagnostic loops rather than merely producing risk scores.

Future work should prioritize pragmatic clinical trials in accountable care organizations, integrated delivery systems, safety-net clinics, and public health programs. These studies should test whether predictive prioritization improves timely diagnostic resolution while preserving patient trust and avoiding unnecessary outreach burden. Implementation should be paired with careful monitoring of equity, staff workload, and unintended consequences. If validated and responsibly deployed, this approach could make abnormal-result follow-up more proactive, transparent, and safer.

Acknowledgements

None

Conflict of interest

None

Financial support

None

Ethics statement

None

References

Ciemins EL, Mohl JT, Moreno CA, Colangelo F, Smith RA, Barton M. Development of a follow-up measure to ensure complete screening for colorectal cancer. JAMA Netw Open. 2024;7(3):e242693.
Mohl JT, Ciemins EL, Miller-Wilson LA, Gillen A, Luo R, Colangelo F. Rates of follow-up colonoscopy after a positive stool-based screening test result for colorectal cancer among health care organizations in the US, 2017-2020. JAMA Netw Open. 2023;6(1):e2251384.
San Miguel Y, Demb J, Martinez ME, Gupta S, May FP. Time to colonoscopy after abnormal stool-based screening and risk for colorectal cancer incidence and mortality. Gastroenterology. 2021;160(6):1997-2005.
Coronado GD, Kihn-Stang A, Slaughter MT, Petrik AF, Thompson JH, Rivelli JS, et al. Follow-up colonoscopy after an abnormal stool-based colorectal cancer screening result: analysis of steps in the colonoscopy completion process. BMC Gastroenterol. 2021;21(1):356.
Zorzi M, Battagello J, Selby K, Capodaglio G, Baracco S, Rizzato S, et al. Non-compliance with colonoscopy after a positive faecal immunochemical test doubles the risk of dying from colorectal cancer. Gut. 2022;71(3):561-7.
O'Connor EA, Nielson CM, Petrik AF, Green BB, Coronado GD. Prospective cohort study of predictors of follow-up diagnostic colonoscopy from a pragmatic trial of FIT screening. Sci Rep. 2020;10(1):2441.
Barbosa EJ Jr, Yang R, Hershman M. Real-world lung cancer CT screening performance, smoking behavior, and adherence to recommendations: Lung-RADS category and smoking status predict adherence. AJR Am J Roentgenol. 2021;216(4):919-26.
Powell L, Sittig DF, Chrouser K, Singh H. Assessment of health information technology-related outpatient diagnostic delays in the US Veterans Affairs health care system: a qualitative study of aggregated root cause analysis data. JAMA Netw Open. 2020;3(6):e206752.
Varon ML, Geng Y, Fellman BM, Troisi C, Fernandez ME, Li R, et al. Interventions to increase follow-up of abnormal cervical cancer screening results: a systematic literature review and meta-analysis. PLoS One. 2024;19(2):e0291931.
Davis M, Simpson K, Lenert LA, Diaz V, Alekseyenko AV. Predicting mammogram screening follow through with electronic health record and geographically linked data. Cancer Res Commun. 2023;3(10):2126-32.
Bastani M, Chiuzan C, Silvestri G, Raoof S, Chusid J, Diefenbach M, et al. A predictive model for lung cancer screening nonadherence in a community setting health-care network. JNCI Cancer Spectr. 2023;7(2):pkad019.
Atta S, Brown RB, Wasser LM, Mayer N, Cassidy J, Liu PJ, et al. Effect of a patient portal reminder message after no-show on appointment reattendance in ophthalmology: a randomized clinical trial. Am J Ophthalmol. 2024;263:93-8.
Berset AE, Burkhardt MC, Xu Y, Mescher A, Brinkman WB. Effect of electronic outreach using patient portal messages on well child care visit completion: a randomized clinical trial. JAMA Netw Open. 2022;5(11):e2242853.
Cronin RM, Fabbri D, Denny JC, Rosenbloom ST, Jackson GP. A comparison of rule-based and machine learning approaches for classifying patient portal messages. Int J Med Inform. 2017;105:110-20.
Sulieman L, Gilmore D, French C, Cronin RM, Jackson GP, Russell M, et al. Classifying patient portal messages using convolutional neural networks. J Biomed Inform. 2017;74:59-70.
Dharod A, Bellinger C, Foley K, Case LD, Miller D. The reach and feasibility of an interactive lung cancer screening decision aid delivered by patient portal. Appl Clin Inform. 2019;10(1):19-27.
Steitz BD, Turer RW, Lin CT, MacDonald S, Salmi L, Wright A, et al. Perspectives of patients about immediate access to test results through an online patient portal. JAMA Netw Open. 2023;6(3):e233572.
Liu D, Shin WY, Sprecher E, Conroy K, Santiago O, Wachtel G, et al. Machine learning approaches to predicting no-shows in pediatric medical appointments. NPJ Digit Med. 2022;5(1):50.
Lee SY, Eagleson RM, Hearld LR, Gibson MJ, Hearld KR, Hall AG, et al. Leveraging machine learning to enhance appointment adherence at a novel post-discharge care transition clinic. JAMIA Open. 2024;7(4):ooae086.
Yang Y, Madanian S, Parry D. Enhancing health equity by predicting missed appointments in health care: machine learning study. JMIR Med Inform. 2024;12(1):e48273.
Deina C, Fogliatto FS, da Silveira GJ, Anzanello MJ. Decision analysis framework for predicting no-shows to appointments using machine learning algorithms. BMC Health Serv Res. 2024;24(1):37.
Elvira C, Ochoa A, Gonzalvez JC, Mochón F. Machine-learning-based no show prediction in outpatient visits. Int J Interact Multimed Artif Intell. 2018;4(7):29-34.
AlMuhaideb S, Alswailem O, Alsubaie N, Ferwana I, Alnajem A. Prediction of hospital no-show appointments through artificial intelligence algorithms. Ann Saudi Med. 2019;39(6):373-81.
Chong LR, Tsai KT, Lee LL, Foo SG, Chang PC. Artificial intelligence predictive analytics in the management of outpatient MRI appointment no-shows. AJR Am J Roentgenol. 2020;215(5):1155-62.
Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, Sinsky CA, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med. 2017;15(5):419-26.
Rittenberg E, Liebman JB, Rexrode KM. Primary care physician gender and electronic health record workload. J Gen Intern Med. 2022;37(13):3295-301.
Baxter SL, Saseendrakumar BR, Cheung M, Savides TJ, Longhurst CA, Sinsky CA, et al. Association of electronic health record inbasket message characteristics with physician burnout. JAMA Netw Open. 2022;5(11):e2244363.
Ayad A, Hallawa A, Peine A, Martin L, Fazlic LB, Dartmann G, et al. Predicting abnormalities in laboratory values of patients in the intensive care unit using different deep learning models: comparative study. JMIR Med Inform. 2022;10(8):e37658.

Author information

Emily Johnson, Robert Smith, Laura Brown & Kevin Miller contributed to this work.

Authors and affiliations

Department of Digital Healthcare Analytics, Faculty of Medicine, University of Toronto, Toronto, Canada
Emily Johnson, Robert Smith & Kevin Miller

Department of Health Informatics Engineering, Faculty of Engineering, McGill University, Montreal, Canada
Laura Brown

Corresponding author

Correspondence to Emily Johnson

Rights and permissions

Open Access The author(s) retain copyright. This article is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. It may be shared and adapted for non-commercial purposes with appropriate attribution, an indication of changes, and distribution of adaptations under the same license. Third-party material may be subject to separate terms identified in its credit line. View the license at https://creativecommons.org/licenses/by-nc-sa/4.0/.

About this article

Cite this article

Vancouver
Johnson E, Smith R, Brown L, Miller K. Predictive Model for Identifying Delayed Diagnostic Follow-Up After Abnormal Screening Results Using Patient Portal Messages, Scheduling Attempts, Primary Care Workload, Result Severity, and Reminder History. J. Health Inform. Digit. Syst.. 2024;4:96.
https://doi.org/10.68159/z800757605
APA
Johnson, E., Smith, R., Brown, L., & Miller, K. (2024). Predictive Model for Identifying Delayed Diagnostic Follow-Up After Abnormal Screening Results Using Patient Portal Messages, Scheduling Attempts, Primary Care Workload, Result Severity, and Reminder History. Journal of Health Informatics and Digital Systems, 4, 96.
https://doi.org/10.68159/z800757605
Received
23 December 2023
Revised
06 February 2024
Accepted
10 March 2024
Published
20 July 2024
Version of record
20 July 2024

Share this article

Easily share this article with others using the link below:

Predictive Model for Identifying Delayed Diagnostic Follow-Up After Abnormal Screening Results Using Patient Portal Messages, Scheduling Attempts, Primary Care Workload, Result Severity, and Reminder History
Scan to access
this article

Ready to submit?
Start a new submission or continue a submission in progress:
Submission Portal Author Guidelines

Follow this journal
Get notified of new updates and articles.