JMIR Publications

ABSTRACT

Background:

Millions of people have limited access to specialty care. The problem is exacerbated by ineffective specialty visits due to incomplete pre-referral workup, leading to delay in diagnosis and treatment. Existing processes to guide pre-referral diagnostic workup are labor-intensive (i.e. building a consensus guideline between primary care doctors and specialists) and require availability of the specialists (i.e. electronic consultation).

Objective:

Using pediatric endocrinology as an example, we developed a recommender algorithm to anticipate patients’ initial workup needs at the time of specialty referral and compared it to a reference benchmark using the most common workup orders. We also evaluated the clinical appropriateness of the algorithm recommendations.

Methods:

Electronic health record data were extracted from 3424 pediatric patients with new outpatient endocrinology referrals. Using item co-occurrence statistics, we predicted the initial workup orders that would be entered by specialists and assessed the recommender’s performance in a holdout dataset. We surveyed endocrinologists to assess the clinical appropriateness of the predicted orders and to understand the initial workup process.

Results:

Specialists indicate that < 50% of new patient referrals arrive with complete initial workup for common referral reasons. The algorithm achieved AUC of 0.95 (95% CI 0.95 - 0.96). Compared to a reference benchmark using the most common orders, precision and recall improved from 37% to 48% (P < 0.001) and 27% to 39% (P < 0.001) for the top 4 recommendations, respectively. The top 4 recommendations generated for common referral conditions (abnormal thyroid studies, obesity, amenorrhea) were considered clinically appropriate the majority of the time by specialists surveyed and practice guidelines reviewed.

Conclusions:

An item-association based recommender algorithm can predict appropriate specialist’s workup orders with high discriminatory accuracy. This could support future clinical decision support tools to increase effectiveness and access to specialty referrals. Our study demonstrates important first steps towards a data-driven paradigm for outpatient specialty consultation with a tier of automated recommendations that proactively enable initial workup that would otherwise be delayed by awaiting an in-person visit.