Utility of comics to support member-checking in realist evaluation

Context: In realist evaluation (RE), as in many qualitative approaches, member-checking significantly enhances the credibility of findings. Because RE is relatively novel and continuously evolving, integrating new tools is pivotal in democratizing its practice. However, despite the significance of member-checking within the RE framework, researchers currently face a lack of tools tailored to supporting this crucial aspect of the evaluation process.

Objective: This study aims to develop and test a novel tool based on comics to explore perceptions of Context-Mechanism-Outcome (CMO) configurations and propose a new method for conducting member-checking in the RE context.

Study Design and Analysis: Based on a refined program theory, comics were created to represent each CMO configuration. These comics were used in group discussions to determine whether the CMOs accurately reflected participants' experiences.

Setting: The refined program theory emerged from a Canadian study on the role of the champion in promoting continuous quality improvement (CQI) initiatives in primary healthcare (PHC).

Population Studied: Group discussions with PHC professionals and experts from across Canada with experience in CQI were organized to evaluate the use of comics for member-checking in RE studies.

Instrument: Researchers employed a creative combination of comic strips and open-ended questions to facilitate participants' articulation of their experiences about the depicted scenarios.

Results: These engaging methods fostered a conducive environment for sharing insights. Three focus groups, each spanning 90 minutes, were conducted via Teams. They comprised 3 to 5 participants, allowing for interactive discussions and rich data exchange. Comics effectively contextualized the discussions and allowed participants to easily understand CMO configurations without needing to grasp the definitions of the mechanisms, a core component of CMO configurations that can be challenging for non-experts. Comics provided a playful, engaging way to lead the groups and encourage exchanges.

Conclusions: This method streamlines the focus on CMO configurations without having to explain all RE principles to participants. Simplifying explanations for participants fosters an effective member-checking process.