JMIR Publications

ABSTRACT

Background:

Online patient reviews are widely used by consumers to assess the quality of direct-to-consumer teleconsultation (DTCT) services, particularly in settings where objective quality information is limited. However, whether these reviews validly reflect actual clinical and patient-centered care quality remains unclear.

Objective:

This study aimed to evaluate the validity of online physician reviews in reflecting the quality of care delivered on China’s three largest DTCT platforms.

Methods:

We conducted a cross-sectional study using unannounced standardized patients (USPs) to objectively assess the quality of DTCT services. Thirty-three USPs were trained to present 11 standardized clinical cases and completed 542 DTCT consultations between physicians on three major Chinese platforms. Technical quality was assessed using a clinical guideline adherence checklist, and patient-centered quality was measured using the Patient–Patient-Centered Care Chinese version (PPPC-CN) scale. Online review quality was defined as the positive review rate displayed on each physician’s profile. Agreement between online reviews and measured quality was evaluated using Intraclass Correlation Coefficients (ICCs), with additional rank correlation analyses.

Results:

Of the 542 consultations initiated, 530 were completed and 404 physicians had publicly available review data. Among all encounters, 53.14% (288/542) were phone-based and 46.86% (254/542) were text-based consultations. The median positive review rate was 99.9% (interquartile range [IQR], 99.4%–100%). Median guideline adherence was low (0.16; IQR, 0.08–0.26), and median patient-centered quality was modest (PPPC-CN score 2.1; IQR, 1.98–2.79). Diagnoses were completely correct in 40.92% (196/530) of consultations. Unnecessary examinations occurred in 1.7% of encounters, and medication prescribing was appropriate in 79.04%. The median consultation time was 13 minutes (IQR, 7–64.69), and the median registration fee was 29.9 yuan (IQR, 26.1–39.9). Agreement between positive review rate and guideline adherence (ICC= 0.002; 95% CI, −0.006 to 0.013) and between positive review rate and patient-centered quality (ICC= 0.014; 95% CI, −0.043 to 0.083) was negligible and far below accepted validity thresholds. Correlations between positive review rate and diagnostic accuracy were weak but statistically significant (Spearman ρ = 0.168; Kendall τ = 0.141; both P < 0.05). Limitations include the use of standardized cases rather than real patients and the focus on publicly visible review metrics.

Conclusions:

Online reviews on major platforms were overwhelmingly positive but showed almost no alignment with actual provider performance. DTCT providers demonstrated low guideline adherence and modest patient-centered quality. More research on improving the review frameworks is urgently needed to fill the gap between patient feedback and service quality. Clinical Trial: The study has been approved by the Southern Medical University Ethics Committee ([2022] No. 013) and registered with the China Clinical Trial Registry (ChiCTR2200062975).