JMIR Publications

ABSTRACT

Background:

Understanding client perspectives of virtual mental health care quality will be critical to inform future policies and practices. This is especially true given the recent surge in the use of virtual care over the last three years, and a more recent emphasis on how sustain access to virtual care while balancing this with patient and health systems needs.

Objective:

This paper outlines the process to redesign and validate the Virtual Client Experience Survey (VCES) that can be used to evaluate client and family experiences of virtual care, specifically virtual mental health and addictions care, across disciplines.

Methods:

The VCES was adapted from a previously validated telepsychiatry survey. All items were reviewed and updated; in particular with the need to ensure transdisciplinary relevance that accorded with the expansion of virtual care across mental healthcare sectors and settings. The survey was then re-validated using the Institute of Medicine’s (IOM) six domains of health care quality as a guiding framework. These six domains include: safe, effective, patient-centered, efficient, timely, and equitable. The VCES was piloted with a convenience sample of clients and family members accessing outpatient care via video or telephone. A confirmatory factory analysis (CFA) was used to test the factorial structures of the VCES, with minor re-specification of the model based on modification indices, factor loadings, reliability and item-total correlation. The re-specifications were checked for alignment with the construct definitions and item interpretation. The reliability of the constructs was estimated by Cronbach’s alpha coefficient.

Results:

The survey was completed 181 times. The construct reliability was generally high. Timely was the only subscale with alpha lower than 0.7; all others were above 0.8. In all cases, the corrected item-total correlation was higher than 0.3. For the CFA, the model was adjusted after multiple imputations with 20 datasets. The mean Chi-square value was 437.5, with 199 degrees of freedom (p<0.001). The mean root mean square error of approximation (RMSEA) was 0.08 (SD = 0.002), the mean confirmatory fit index (CFI) was 0.987 (SD=0.001), the mean Tucker-Lewis Index (TLI) was 0.985 (SD=0.001), and the mean standard room mean square residual (SRMR) was 0.04 (SD = 0.001).

Conclusions:

This paper reports on the validation of a transdisciplinary measure to evaluate client and family experiences of virtual mental health and addictions care. Given the widespread uptake of virtual care, this survey has broad applicability across settings that provide mental health and addictions care. The VCES can be used to guide targeted quality improvement initiatives across health care quality domains. By effectively addressing challenges as they emerge, it is anticipated that we will continue to move towards hybrid modalities of practice that leverage the strengths and benefits of telephone, video, and in-person care to effectively respond to unique client and family needs and circumstances. Clinical Trial: N/A