JMIR Publications

ABSTRACT

Background:

The rapid evolution and integration of digital technologies have transformed (mental) health and social care, offering new modalities of digital care, assistance, and support through web-based platforms, mobile apps, extended reality (XR), wearables, and artificial intelligence (AI) systems. Despite this proliferation, there is little consensus on what constitutes “high-quality” digital care. Challenges persist around data security, interoperability, accessibility, sustainability, and professional competence, while existing standards and regulations provide fragmented guidance.

Objective:

This study aimed to develop a contextualized, consensus-based quality assessment framework for digital care, assistance, and support in Flanders, Belgium. For this purpose, perspectives across technology, organizational processes, and professional competencies were integrated.

Methods:

The study employed a multi-phase design comprising (1) ten expert interviews with Flemish government officials, (2) a narrative literature review of 303 peer-reviewed and grey literature sources, (3) a three-round Delphi study with 50 experts across five domains (end users, facilitators, technology developers, deontology/ethics, and digital inclusion/media literacy), and (4) four complementary focus groups and three interviews with specialists in AI, regulation, social work, mental health, and IT. The Delphi rounds (May 2024 - January 2025) gathered iterative feedback through open-ended elicitation, structured rating, and classification of quality criteria. Quantitative data were analyzed using descriptive statistics, while qualitative feedback was subjected to thematic analysis.

Results:

Fifty experts (58%) participated in Round 1, 40 (80%) in Round 2, and 34 (68%) in Round 3. Round 1 generated 577 unique quality criteria, consolidated into 26 clusters organized under three pillars: technology, organization, and professional competencies. The relative importance across pillars was balanced: Technology (M = 37.29, SD = 12.38), Professionals (M = 33.33, SD = 10.39), and Organizations (M = 29.80, SD = 10.45). Accessibility, reliability, and safety ranked highest for the technology pillar; vision, quality monitoring, and infrastructure for organization; and support, digital competences, and ethics for professionals. The finalized framework included 112 criteria, of which 35 (31.25%) were designated as optional and 77 as minimum requirements. Focus groups (n = 4) and interviews (n = 3) validated the framework’s comprehensiveness and usability, emphasizing proportional implementation, user centrality, and alignment with European Union (EU) regulations. Stakeholders highlighted the need for practical tools, training, and governance mechanisms to ensure adoption and sustainability.

Conclusions:

This study produced a co-developed, context-sensitive quality framework that balances technological robustness, organizational readiness, and professional competence in digital care, assistance, and support. The framework can serve both as a quality safeguard and a developmental roadmap, promoting inclusivity, transparency, and accountability. Accompanying self-assessment and governance tools enhance practical applicability. Implementation success will depend on governmental support, resource allocation, and structured feedback loops. Future research should pilot the framework in real-world settings, assess its impact, and establish mechanisms for continuous updates to maintain relevance in a rapidly evolving digital landscape.