Currently submitted to: Journal of Medical Internet Research
Date Submitted: Jul 14, 2026
Open Peer Review Period: Jul 15, 2026 - Sep 9, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Beyond average effect: An embedded evaluation of mechanisms, contextual fit, outcomes, and opportunities for adaptation in a digital health intervention for emerging adults living with type 1 diabetes
ABSTRACT
Background:
Multicomponent digital health interventions are increasingly used to support type 1 diabetes (T1D) self-management and are generally acceptable to patients. However, existing evaluations primarily report average effects at the group level, with limited understanding of how outcomes arise across individuals, intervention components, and engagement patterns.
Objective:
To identify the contextual profiles under which favourable self-efficacy was observed within a multicomponent digital health intervention for emerging adults living with T1D.
Methods:
We conducted an embedded evaluation using Coincidence Analysis, a configurational method that examines how combinations of factors are associated with outcomes in complex interventions. The analysis was embedded within a randomised controlled trial of Keeping in Touch (KiT), a 12-month personalised text message-based intervention for emerging adults living with T1D in Canada. Self-efficacy was dichotomised as favourable or unfavourable based on baseline level and change over 12 months. We first examined contextual profiles associated with outcomes in the full sample, including the role of overall intervention exposure within these profiles. Among intervention-group participants, we further examined how patterns of exposure to intervention components and participant engagement were associated with outcomes.
Results:
Among 168 participants in the full-sample analysis and 89 intervention-group participants in the engagement analysis, multiple distinct profiles were associated with favourable and unfavourable outcomes. Exposure to the intervention was associated with favourable outcomes only within a specific profile characterized by shorter diabetes duration (<10 years) and lower baseline HbA1c (<9.0%). Higher engagement, characterised by a higher prompt response rate (≥Q2) and use of optional features, featured in favourable profiles, whereas lower engagement featured in unfavourable profiles. However, engagement alone was neither necessary nor sufficient for benefit. Instead, its association with outcomes depended on personal context, including diabetes duration, sex, and exposure to specific educational topics.
Conclusions:
For emerging adults with T1D, a one-size-fits-all approach may be insufficient, and evaluations focus solely on average intervention effect may obscure important differences in effectiveness across subgroups. By identifying the multiple context-dependent pathways associated with both outcomes, the findings underscore the need for context‑responsive, person‑centred approaches that tailor intervention content and engagement strategies to diverse population.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.