JMIR Publications

ABSTRACT

Background:

During the COVID-19 pandemic, the rapid spread of misinformation created significant public health challenges. Large language models (LLMs), pre-trained on extensive textual data, have shown potential in identifying misinformation, but their accuracy is influenced by factors such as prompt engineering (i.e., modifying LLM requests to assess changes in output). One form of prompt engineering is role-playing, where ChatGPT imitates specific social roles or identities upon request. This research explores how ChatGPT's accuracy in detecting COVID-19-related misinformation is affected by role-playing prompts when assigned specific social identities in the request prompt.

Objective:

To investigate the impact of role-playing prompts on the accuracy of ChatGPT in detecting misinformation.

Methods:

This study used 36 real-world tweets about COVID-19 that were categorized into misinformation, sentiment, corrections, and neutral reporting. ChatGPT was tested with prompts incorporating different combinations of multiple social identities (i.e., political beliefs, education levels, locality, religiosity, and personality traits), resulting in a total of 51,840 runs. Conditions only including no identities in prompts and only including political identity were used for control comparisons.

Results:

Findings reveal that including social identities in prompts significantly reduces detection accuracy, with a notable drop from 68.1% (no identities) to 29.3% (all identities included). Prompts with political identities alone resulted in the lowest accuracy (19.2%). ChatGPT was also able to distinguish between sentiments that express opinions not aligned with public health guidelines from misinformation that makes declarative statements. While findings show that inclusion of identities decreases detection accuracy, it remains uncertain whether ChatGPT adopts views aligned with social identities: when assigned a conservative identity, ChatGPT identifies misinformation with nearly the same frequency as it does under a liberal identity. Additionally, the study evaluated ChatGPT's explanations for its classification decisions, revealing inconsistent reasoning across study conditions.

Conclusions:

These results indicate that ChatGPT's ability to classify misinformation is compromised when role-playing social identities, highlighting the complexity of integrating human biases and perspectives in LLMs. While LLMs show promising resulting in misinformation detection, the authors conclude that human oversight is still needed based on current capabilities. Further research is needed to understand how LLMs weigh social identities in prompt-based tasks and to explore their application in different cultural contexts and information domains.