Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: Iproceedings

Date Submitted: Jan 28, 2022
Date Accepted: Jan 28, 2022

The final, peer-reviewed published version of this preprint can be found here:

Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Polesie S, Paoli J

Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Iproc 2022;8(1):e36895

DOI: 10.2196/36895

Interobserver and Human-AI Concordance in Differentiating Between Invasive and In Situ Melanoma

  • Sam Polesie; 
  • John Paoli

ABSTRACT

Background:

Machine learning (ML) algorithms including convolutional neural networks (CNNs) have recently made significant advances in a research setting. Even though several algorithms nowadays are targeted directly to the consumer market, their implementation in clinical practice is still pending. Most melanomas are easy to recognize even without the aid of dermoscopy. Nonetheless, it is often more challenging to discriminate between invasive and in situ melanomas (MIS) in a preoperative setting even with the assistance of dermoscopy. Although several dermoscopic features suggestive of MIS and invasive melanomas have been presented, their usefulness in a larger setting is limited by how well physicians agree on their presence or absence.

Objective:

The overarching aim of this research project is to identify useful dermoscopic features to help dermatologists in predicting the melanoma thickness and to develop CNNs that can assist dermatologists in the preoperative assessment of melanoma thickness. The ultimate aim is to develop algorithms that can strengthen patient care, clinical decision-making and use in routine healthcare.

Methods:

We have included dermoscopic images as well as clinical close-up images of invasive melanomas and MIS from our department during the time period of January 1, 2016 to December 31, 2020. Using this image material, we have trained, validated and tested two separated CNNs based on dermoscopic and clinical close-up images. We have also invited dermatologists to review the test sets and, for a subset of the dermoscopic images, asked them to specify the presence of prespecified dermoscopic features. Subsequently, we compared CNN outputs to the combined dermatologists’ output for all lesions and assessed the interobserver agreement for several dermoscopic features.

Results:

The CNN developed using dermoscopic images performed in par with the invited dermatologists whereas the CNN using clinical close-up images was outperformed by the group of dermatologists. Two dermoscopic features, atypical blue white structures and shiny white lines, both displayed a moderate to substantial interobserver agreement and were both indicative of invasive melanomas >1.0 mm.

Conclusions:

CNNs used to differentiate between invasive melanomas and MIS might be an example of a clinically relevant ML application, but needs further refinement and evaluation in prospective clinical trials. Only a few dermoscopic features are helpful in distinguishing melanoma thickness.


 Citation

Please cite as:

Polesie S, Paoli J

Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Iproc 2022;8(1):e36895

DOI: 10.2196/36895

Per the author's request the PDF is not available.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.