JMIR Publications

ABSTRACT

Background:

Deep learning offers great benefits in classification tasks such as medical imaging diagnostics or stock trading, especially when compared to human-level performances and can be a viable option for classifying distinct levels within Community-engaged research (CEnR). CEnR is a collaborative approach between academics and community partners with the aim of conducting research that is relevant to community needs while incorporating diverse forms of expertise. In the field of deep learning and AI, training multiple models to get the highest validation accuracy is common practice; however, it can overfit towards that specific dataset and not generalize well to a real-world population which creates issues of bias and potentially dangerous algorithmic decisions. Because of this there is a need for creating techniques and exhaustive evaluative processes for these powerful unexplainable models if we plan on automating human decision-making so that we can ensure we do not incorporate and blindly trust poor AI models with real world decisions.

Objective:

We propose an evaluation study to see if our most accurate transformer-based models derived from previous studies can emulate our own classification spectrum for tracking community-engaged research studies as well as if the use of calibrated confidence scores are meaningful.

Methods:

We compare the results from three domain experts, classifying a sample of 45 studies derived from our university’s IRB database to three previously trained transformer-based models, as well as showing if calibrated confidence scores can be a viable technique for using AI as a support role for complex decision-making systems.

Results:

Our findings show that certain models display greater confidence despite not having the highest validation accuracy.

Conclusions:

There is a need to further explore methods that allow domain experts to “trust” our model more.