Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: Journal of Medical Internet Research

Date Submitted: Oct 1, 2026

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Patient concordance following an urgent virtual physician consultation in a provincial triage program in British Columbia, Canada

  • Kurtis Stewart; 
  • Zenith A Zyn; 
  • Swati Kanwal; 
  • Devon Currie; 
  • Daniel Raff; 
  • Ross Duncan; 
  • Lindsay Hedden; 
  • Brandon Wong; 
  • Sandra Sundhu; 
  • Frank X Scheuermeyer; 
  • Riyad B Abu-Laban; 
  • Roger Tam; 
  • Linda Riches; 
  • Lillian Dennis; 
  • Kendall Ho; 
  • Sonya Cressman

ABSTRACT

Background:

Virtual nurse-physician collaborations in pre-hospital triage program such as HealthLink BC’s 8-1-1 service can reduce unnecessary emergency department (ED) by referring patients to the appropriate level of follow-up care, ranging from “Go to ED Now” to “Home Treatment”. However, limited evidence exists on the level to which patients follow through with the advice provided.

Objective:

We sought to describe the patient-level and geographic factors associated with caller concordance to 8-1-1 physician advice and develop predictive machine learning (ML) models of concordance.

Methods:

All calls to the 8-1-1 virtual physician service made between April 6, 2020 and March 31, 2023 with a follow-up care recommendation were eligible. Caller records were linked to provincial health administrative databases to determine demographic and pre-/post-call health service use information and geographic information based on health or census area. Caller concordance was calculated by physician discharge disposition. Descriptive analyses summarized concordance rates by demographic characteristics and health regions. ML models, including support vector machines, neural networks, and extreme gradient boosting (XGBoost), were developed using over 380 preprocessed features to predict concordance and evaluated using F1-scores.

Results:

113,090 calls were included in the analysis. Concordance rates varied by physician disposition: they were highest for “Go to ED Now” calls (61.3%). When stratified, concordance rates were most strongly associated with health region and rurality. The XGBoost model had the strongest predictive performance for the “Go to ED Now” disposition (F1-score=0.78), while other models and other dispositions had F1-scores<=0.63. Health region, rurality, sex, and primary care attachment were consistently the strongest features influencing model predictions. In a sensitivity analysis expanding the concordance definitions, XGBoost models’ F1-scores increased up to 0.90.

Conclusions:

Caller concordance with virtual physician advice was highest for the most urgent care recommendation, but varied greatly by health region, indicating significant disparities for rural patients trying to access immediate ED care. XGBoost models had strong predictive performance for more urgent care recommendations, especially when considering concordance over longer timeframes. ML models could be leveraged to support patients at risk of discordance and subsequent potentially negative outcomes by personalizing virtual triage interventions.


 Citation

Please cite as:

Stewart K, Zyn ZA, Kanwal S, Currie D, Raff D, Duncan R, Hedden L, Wong B, Sundhu S, Scheuermeyer FX, Abu-Laban RB, Tam R, Riches L, Dennis L, Ho K, Cressman S

Patient concordance following an urgent virtual physician consultation in a provincial triage program in British Columbia, Canada

JMIR Preprints. 01/10/2026:112394

DOI: 10.2196/preprints.112394

URL: https://preprints.jmir.org/preprint/112394

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.