JMIR Publications

ABSTRACT

Background:

The prevalence of mobile health devices for remote monitoring has experienced rapid expansion, leading to more people being able to continuously monitor their asthma. However, discrepancies between patient-reported outcome measures and electronically recorded device data pose challenges in the use of mHealth systems for disease management.

Objective:

This study investigates the concordance between the longitudinal self-reported relief inhaler usage and the smart-inhaler recorded usage data to identify and characterize "concordant users".

Methods:

We analyzed a subset (n=15) of the AAMOS-00 dataset (N=22) who had sufficient data. To improve robustness of our results to data processing choices, we carried out a multiverse analysis across 132 unique analytical configurations combined with patient-specific permutation testing (10,000 permutations each) to assess the statistical significance of concordance and identify concordant users. We also conducted an exploratory analysis of end-of-study feedback using a multi-method large language model (LLM) ensemble to evaluate user mindsets.

Results:

Longitudinal analysis revealed these patients showed stable, strong concordance over the study period, while the remaining patients (n=12) showed variable and low agreement. While demographic characteristics did not distinguish these groups, exploratory analysis of end-of-study feedback using a multi-method LLM ensemble provided a possible distinction in user mindset. These concordant users emphasized the data's utility for "Clinical Care Value" and "Insights & Self-Discovery," whereas remaining users primarily emphasized “Integration & Interoperability”. A sensitivity analysis using median Fisher's Z identified one additional borderline patient but did not alter the comparison findings.

Conclusions:

These findings, while hypothesis-generating given the limited sample size, suggest that high-quality self-reporting might be a sustained trait associated with a patient’s motivation to derive clinical utility from their data. This highlights the potential for future research to validate whether designing digital health interventions that appeal to user motivations can improve data concordance between self-reports and device records.