JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) technologies are increasingly being integrated into mental health settings to support tasks such as clinical documentation and decision-making. In parallel, AI-enabled deception detection, which leverages multimodal behavioral cues like facial expressions, vocal tone, and body movements, is also a growing research area. These technologies may hold significant relevance in mental health contexts, where deception, whether intentional or unintentional, can compromise treatment outcomes and therapeutic trust. However, most research on AI-based deception detection has focused on law enforcement and security domains, resulting in limited understanding of its applicability to mental health. Consequently, the ethical, relational, and practical implications of using such technologies in clinical settings remain underexplored.

Objective:

This study explored stakeholder perspectives on the responsible integration of AI-enabled deception detection in therapeutic contexts. Specifically, we examined: (RQ1) what ethical frameworks and safeguards are needed to guide the use of such tools in therapy; (RQ2) what technical and procedural protections are necessary to uphold client confidentiality; and (RQ3) what design and evaluation strategies can mitigate bias and promote fairness in clinical applications of AI-based deception detection.

Methods:

We conducted six virtual focus groups (N = 18) with individuals who held dual roles as both mental health clinicians and current therapy clients. Participants responded to a hypothetical scenario describing the integration of AI deception detection into therapy. A semi-structured guide facilitated conversation, and transcripts were analyzed thematically using a combination of inductive and deductive coding strategies.

Results:

Participants expressed a range of concerns about the integration of AI-enabled deception detection in therapy, highlighting potential ethical, relational, and contextual challenges. In response to RQ1, participants described fears of a “Big Brother” atmosphere and concerns about distraction from in-session notifications. However, many viewed telehealth as a less intrusive context and emphasized respecting disclosure timing and maintaining client agency. For RQ2, participants raised concerns about unconscious data capture, subpoena risks, and unclear data protections. For RQ3, participants cautioned that such tools may exacerbate power imbalances, erode trust through false positives, and lack cultural or contextual sensitivity. Informed by these findings, the research team developed design and policy recommendations, including minimizing in-session notifications; ensuring ongoing consent; using encrypted, HIPAA-compliant systems; establishing transparent data policies; training models on diverse populations; exploring modeling personalization; and developing equitable use policies.

Conclusions:

While AI-enabled deception detection technology holds promise for augmenting clinical insight, its integration into therapy must be guided by a commitment to safe, ethical practice. Researchers and clinicians should collaborate to design systems that: (1) integrate seamlessly into therapy without disrupting therapeutic relationships, (2) prioritize data security and transparency to protect client confidentiality, and (3) implement fairness safeguards that address cultural representation and power dynamics. Addressing these challenges will be essential for ensuring that AI deception detection enhances, rather than undermines, therapeutic practice.