JMIR Publications

ABSTRACT

Background:

The recent increase in online health information-seeking prompted extensive user appraisal of encountered content. Information consumption depends crucially on the quality of encountered information and the user’s ability to evaluate it; yet, within the context of online, organic search behavior, few studies take into account both these aspects simultaneously.

Objective:

We aimed to explore a method to bridge these two aspects and grant even consideration to both the stimulus (webpage content) and the user (ability to appraise encountered content). We examined novices and experts in information retrieval and appraisal to demonstrate a novel approach to studying Information Foraging Theory: Stimulus-Engagement Alignment (SEA).

Methods:

We sampled from experts and novices in information retrieval and assessment, asking participants to conduct a 10-minute search task with a specific information goal. We employed an observational and a retrospective think-aloud protocol to collect data within the framework of an interview. Data from three streams (think-aloud, human-computer interaction, and screen content) were manually coded in the ROCK standard and subsequently aligned and represented in a tabularized format with the R package {rock}. SEA scores were derived from designated code co-occurrences in specific segments of data within the stimulus data stream versus the think-aloud and human-computer interaction data streams.

Results:

SEA scores represented a meaningful comparison of what participants encountered and what they engaged with. Operationalizing codes as either “present” or “absent” in a particular data stream allowed us to inspect not only which credibility cues participants engaged with the most frequency, but also whether participants noticed the absence of cues. Code co-occurrence frequencies could thus indicate case-, time-, and context-sensitive information appraisal that also takes into account the quality of information encountered.

Conclusions:

Employing SEA allowed us to retain epistemic access to idiosyncratic manifestations of both stimuli and engagement, but also, via utilizing the same coding scheme and designated co-occurrences across participants, be able to pinpoint trends within our sample and subsamples. We believe our approach lends a powerful analysis encompassing the breadth and depth of data, both on par with each other in the feat of understanding organic, online search behavior.