Abstract
The objectives of this project were to evaluate a large healthcare system’s telemedicine no-show risk model, assessing accuracy in identifying (1) high-risk for no-show patients and (2) refinements to the model that would improve the prediction accuracy and ease of use. The original model factors—Portal Activation, Last Portal Log-In < 90 days, E-Check-In < 90 days, E-Check-In for Visit Completed, and Last Video Visit—showed limited predictive power, making the model challenging to apply in practice. Leadership seeks a more accurate and streamlined model to support targeted interventions aimed at reducing no-shows.
A literature review revealed limited evidence on predictive scoring models for telemedicine visit no-shows. We analyzed 525,333 telemedicine appointments from 30 clinics between 2021 and 2024. Using descriptive statistics and binary logistic regression, we: (1) assessed no-show rates by risk score; (2) calculated standardized Z scores for model factors; and (3) evaluated overall model performance with Receiver Operating Characteristic (ROC) area under the curve (AUC).
Statistical analysis showed poor overall accuracy in predicting no-shows. Binary logistic regression identified significant predictors: Portal Activation (Z=-7.17), E-Check-In Completed (Z=296.29), Last Telemedicine Visit Connection (Z=44.55), E-Check-In < 30 days (Z=13.11), and Last Portal Log-In < 90 days (Z=6.12). The Portal Activation was inversely associated with no-shows, penalizing patients who accessed the portal with a higher risk score, lowering predictive accuracy. Removing this factor improved ROC AUC from 0.86 to 0.92, enhancing both accuracy and usability.
Removing Portal Activation from the original model significantly improved the prediction algorithm of patients at risk for telemedicine no-shows (ROC AUC 0.92). The refined model is more accurate and easier to apply in practice, supporting leadership’s longer-term goal for targeted interventions. During Phase II of the project, leadership will pilot the revised model in select clinics.
Notes
References:
Adepoju, O. E., Chae, M., Liaw, W., Angelocci, T., Millard, P., & Matuk-Villazon, O. (2022). Transition to telemedicine and its impact on missed appointments in community-based clinics. Annals of Medicine, 54(1), 98-107.
Crotty, B. H., Hyun, N., Polovneff, A., Dong, Y., Decker, M. C., Mortensen, N., . . . Somai, M. M. (2021). Analysis of clinician and patient factors and completion of telemedicine appointments using video. JAMA Network Open, 4(11), e2132917-e2132917.
Incze, E., Holborn, P., Higgs, G., & Ware, A. (2021). Using machine learning tools to investigate factors associated with trends in ‘no-shows’ in outpatient appointments. Health & place, 67, 102496.
Narwal-Kasmani, R., Vaughan, T. J., Ulrich, C. A., & Stausmire, J. M. (2023). Performance improvement, telemedicine, patient engagement, and comparative no-show rates. Journal of Healthcare Risk Management, 43(1), 9-17.
Valero-Bover, D., González, P., Carot-Sans, G., Cano, I., Saura, P., Otermin, P., . . . Piera-Jiménez, J. (2022). Reducing non-attendance in outpatient appointments: predictive model development, validation, and clinical assessment. BMC health services research, 22(1), 451.
Sigma Membership
Rho
Type
Presentation
Format Type
Text-based Document
Study Design/Type
Quality Improvement
Research Approach
Translational Research/Evidence-based Practice
Keywords:
Primary Care, Health Equity or Social Determinants of Health, History
Recommended Citation
Medvec, Barbara R. and Knight, John D., "Predicting Telemedicine Patient Visit No-Shows: Discovering an Actionable Patient Risk Model" (2026). International Nursing Research Congress (INRC). 5.
https://www.sigmarepository.org/inrc/2026/presentations_2026/5
Conference Name
37th International Nursing Research Congress
Conference Host
Sigma Theta Tau International
Conference Location
Toronto, Ontario, Canada
Conference Year
2026
Rights Holder
All rights reserved by the author(s) and/or publisher(s) listed in this item record unless relinquished in whole or part by a rights notation or a Creative Commons License present in this item record. All permission requests should be directed accordingly and not to the Sigma Repository. All submitting authors or publishers have affirmed that when using material in their work where they do not own copyright, they have obtained permission of the copyright holder prior to submission and the rights holder has been acknowledged as necessary.
Review Type
Abstract Review Only: Reviewed by Event Host
Acquisition
Proxy-submission
Date of Issue
2026-07-20
Predicting Telemedicine Patient Visit No-Shows: Discovering an Actionable Patient Risk Model
Toronto, Ontario, Canada
The objectives of this project were to evaluate a large healthcare system’s telemedicine no-show risk model, assessing accuracy in identifying (1) high-risk for no-show patients and (2) refinements to the model that would improve the prediction accuracy and ease of use. The original model factors—Portal Activation, Last Portal Log-In < 90 days, E-Check-In < 90 days, E-Check-In for Visit Completed, and Last Video Visit—showed limited predictive power, making the model challenging to apply in practice. Leadership seeks a more accurate and streamlined model to support targeted interventions aimed at reducing no-shows.
A literature review revealed limited evidence on predictive scoring models for telemedicine visit no-shows. We analyzed 525,333 telemedicine appointments from 30 clinics between 2021 and 2024. Using descriptive statistics and binary logistic regression, we: (1) assessed no-show rates by risk score; (2) calculated standardized Z scores for model factors; and (3) evaluated overall model performance with Receiver Operating Characteristic (ROC) area under the curve (AUC).
Statistical analysis showed poor overall accuracy in predicting no-shows. Binary logistic regression identified significant predictors: Portal Activation (Z=-7.17), E-Check-In Completed (Z=296.29), Last Telemedicine Visit Connection (Z=44.55), E-Check-In < 30 days (Z=13.11), and Last Portal Log-In < 90 days (Z=6.12). The Portal Activation was inversely associated with no-shows, penalizing patients who accessed the portal with a higher risk score, lowering predictive accuracy. Removing this factor improved ROC AUC from 0.86 to 0.92, enhancing both accuracy and usability.
Removing Portal Activation from the original model significantly improved the prediction algorithm of patients at risk for telemedicine no-shows (ROC AUC 0.92). The refined model is more accurate and easier to apply in practice, supporting leadership’s longer-term goal for targeted interventions. During Phase II of the project, leadership will pilot the revised model in select clinics.
Description
Focus: Clinical
Status: Completed Work/Project
Evaluating telemedicine visits over a four-year period, Phase I of this Quality Improvement project refined a patient no-show risk model to improve accuracy and ease of use. The original five-factor model achieved an ROC AUC of 0.86 but generated inaccurate predictions of no-shows. By removing the lowest-performing factor, predictive accuracy improved to an ROC AUC of 0.92.