JMIR Publications

ABSTRACT

Background:

Web-based computerized adaptive testing (CAT) implementation of the skin cancer (SC) risk scale could substantially reduce participant burden without compromising measurement precision. However, the CAT of SC classification has not been reported in academics so far.

Objective:

We aim to build a CAT-based model using machine learning to develop an app for automatic classification of SC to help patients assess the risk at an early stage.

Methods:

We extracted data from a population-based Australian cohort study of SC risk (n=43,794) using the Rasch simulation scheme onto data. All 30 feature items were calibrated with the Rasch partial credit model (PCM). A total of 1000 cases following a normal distribution (mean =0 and SD=1) based on the item and threshold difficulties were simulated using three techniques of machine learning, including Naïve Bayes(NB), K-nearest Neighbors Algorithm(KNN), and Logistic regression(LR) to compare model accuracy in training and testing datasets with a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, and receiver operating characteristic curve (area under the curve [AUC]) and confidence intervals (CIs), along with the accuracy and precision across proposed models for comparison. An app classifying the SC risk of the respondent was developed.

Results:

We observed that: (1) the 30-item KNN model yields higher AUC=99% and 91% for the 700 training and 300 testing cases, respectively, beyond the other two counterparts, and (2) an app that predicts SC-classification for patients was successfully developed and demonstrated in this study.

Conclusions:

The 30-item SC-prediction model, combined with the Rasch online CAT, is recommended for improving the accuracy of the patient SC assessment. An app developed for helping patients self-assess SC risk at an early stage is required for application in the future.