JMIR Publications

ABSTRACT

Background:

Speech sound disorders (SSDs) are common in children and are associated with an increased risk of reading academic difficulties. The COVID-19 pandemic further highlighted the need for remote and digitalized assessment tools. In Korea, standardized instruments, such as the Urimal Test of Articulation and Phonation and Assessment of Phonology and Articulation of Children are widely used but have limitations, including reliance on face-to-face evaluation, absence of automated scoring, and limited applicability for remote assessments.

Objective:

This study aimed to develop and establish the content validity of an articulation assessment tool that can overcome these limitations and be integrated into digital therapeutics (DTx).

Methods:

A three-round modified Delphi survey was conducted between July and September 2025, with 23 of 25 invited experts (recruitment rate, 92.0%), including 12 physiatrists and 11 speech-language pathologists, with a mean professional experience of 10.69 years. All participants completed all rounds (retention rate, 100%).Panelists evaluated the appropriateness of word lists, phonological environments, and scoring criteria. Quantitative analyses, including calculations of Content Validity Ratio (CVR), Content Validity Index (CVI), and median and interquartile range (IQR) were performed. Consensus thresholds were set at CVR ≥0.39, CVI ≥0.78, median ≥3.5, and IQR ≤1.0. Items were retained only when all four criteria were satisfied. While formal qualitative analysis was not performed, the research team internally reviewed and synthesized core keywords and themes from the experts' open-ended responses to guide the refinement of items.

Results:

These qualitative findings converged into four key themes: (1) modernization of word stimuli, (2) expansion of phonological coverage, (3) refinement of scoring criteria to reduce ambiguity, and (4) enhancement of result interpretability through visualization. In round 2, a revised 35-word list was evaluated across 25 items, of which 20 (80.0%) met all consensus criteria. Five items (20.0%) failed to meet at least one threshold, including phonological environment adequacy (CVR=0.48, CVI=0.74), scoring redundancy (CVR=0.13, CVI=0.57), usefulness of PWC/PWP (CVR=0.39, CVI=0.70), contribution of mean phonological length (CVR=0.22, CVI=0.61), and usefulness of feature-based indices (CVR=0.30, CVI=0.65; IQR=2). Items that reached consensus showed CVR values of 0.57–0.91, CVI values of 0.78–0.96, median scores of 4, and IQR values of 0–1. In round 3, all remaining items achieved consensus.

Conclusions:

This Delphi study developed a novel articulation assessment tool robust content validity. This tool includes updated word stimuli, diverse analysis indices, and visualization features, thereby enhancing its clinical utility and suitability for integration into artificial intelligence-based DTx. By standardizing and digitalizing articulation assessments, this tool has the potential to support personalized and accessible interventions for children with SSDs. Future studies should focus on non-inferiority testing against existing standardized tools and clinical validation using digital therapeutic platforms.