Accepted for/Published in: JMIR Neurotechnology
Date Submitted: May 10, 2022
Open Peer Review Period: May 10, 2022 - Jul 5, 2022
Date Accepted: Jul 29, 2022
(closed for review but you can still tweet)
Interrater agreement on National Institutes of Health Stroke Scale between paramedics and stroke physicians: a validation study for the digital training model in the Paramedic Norwegian Acute Stroke Prehospital Project
ABSTRACT
Background:
Time spent in the prehospital phase of acute stroke care is multifactorial and have an effect on possibilities for acute treatment. Communication between paramedics and the in-hospital stroke team directly affects time to treatment. A mutual stroke scale like National Institutes of Health Stroke Scale (NIHSS) may improve communication quality. The Paramedic Norwegian Acute Stroke Prehospital Project (ParaNASPP) was a stepped wedge randomised trial of stroke screening using NIHSS in the ambulance where the intervention was training paramedics in stroke and NIHSS, use of NIHSS made into a mobile application guiding the examination and facilitating communication with the in-hospital stroke team.
Objective:
The aim of this study was to validate the digital training model from the ParaNASPP clinical trial.
Methods:
Twenty-four paramedics were recruited from Oslo University Hospital (OUS) in Norway, completing the ParaNASPP training model. Twenty exclusive videos with pre-defined NIHSS scores were recorded. Four stroke physicians from OUS were included for reference. Bland Altman plots with 95% Limits of Agreement (LoA) were calculated comparing first paramedics and stroke physicians to the pre-defined scores, and then with each other. A pre-defined LoA was set to +/-3 points. To align with clinical practice NIHSS scores were also dichotomized into 0-5 (minor stroke) or ≥6 (moderate and major stroke), and agreement calculated using Cohen’s Kappa.
Results:
The videos (n=20) had a median (range) NIHSS score 7 (0-31). The paramedics’ scores were slightly higher than the pre-defined scores with a mean difference of -0.38 and LoA ranged from -4.04 to 3.29. The paramedics scored higher than the stroke physicians with a mean difference of -0.39, with LoA ranging from -4.58 to 3.80. Dichotomizing, Cohen’s Kappa was 0.89 between pre-defined scores and paramedics, 0.92 between pre-defined scores and stroke physicians, and 0.81 between paramedics and stroke physicians, all indicating very good agreement.
Conclusions:
The standardized digital training model used in the ParaNASPP clinical trial seems to result in very good agreement in NIHSS scoring between paramedics and both pre-defined scores and stroke physician scores. This study also demonstrates possibilities for transfer of clinical competence in digital simulation training. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.