Enhance Your Team-Based Qualitative Research

Thank you, Greg and Kathleen, for your thoughtful comments. We appreciate your additional recommendations and would like to invite others to contribute any additional ideas.

The reliability of the analysis is of definite concern when work is diffused across team members; your recommendation for standardized protocols and on-site training are excellent.

We look forward to seeing your handbook next year. Good luck!

Competing interests:   None declared

Fernald and Duclos’ recent article on enhancing rigor in team-based qualitative research[1] is long overdue. Qualitative research studies, particularly those carried out in team settings, have increasingly become a part of health science research. Yet, as Fernald and Duclos accurately note, few published works exist that provide researchers with practical suggestions for improving the quality and integrity of such research. As designers and managers of multi-site qualitative research, we have written many procedures and guidelines for enhancing qualitative research, but they have always been developed for specific research projects and taken the form of unpublished documents or manuals. So, we too have lamented the lack of a set of general methods for improving the management and outcome of team-based qualitative studies. We agree with many of the article’s suggestions, and applaud the authors’ initiative to bring the challenges and issues associated with team-based qualitative research to the forefront. From our own experience with multi-site research, especially multi-national projects, we offer some additional key points.

Maintaining team cohesion and study integrity under increasingly globalized conditions and geographic dispersal of health research can not be easily achieved with frequent face-to-face team meetings. Conference calls are an option, but are limited in their ability to identify and solve the plethora of problems that inexorably arise during large research initiatives. In some parts of the world, they are also difficult to conduct due to poor communications infrastructure and extreme time zone differences. We have found that regular, face-to-face on-site visits are essential in multi-site projects, despite the costs, as is timely feedback to site collaborators on transcripts, field notes, summaries, and other materials. These overall coordination strategies are added to the kinds of steps outlined by Fernald and Duclos, which are in turn implemented at each field site as well as at our central project offices.

Providing field teams with explicit structure can also mitigate many problems. Fernald and Duclos offer some general suggestions in this regard (e.g., scheduling regular team meetings, appointing one person as data manager, writing down code definitions). However, for team-based qualitative research, especially that which involves multiple sites, there are additional concerns with regard to the critical issue of reliability. When multiple interviewers and data analysts are involved, achieving a meaningful and reliable comparison of data across participants and sites can be problematic; as the authors point out, in qualitative research each individual may have a unique interviewing or analytical style. They go on to recommend that a team leader needs to “accommodate” these differences. We agree that qualitative research needs to be iterative and flexible, but we also recommend standardizing certain processes, within study protocols and through comprehensive training, that facilitate consistent and reliable data collection and analysis. In our research projects, we extend Fernald and Duclos’ recommendations for building a sense of common purpose by providing methodological training on-site to field staff (e.g., interviewing skills, data management procedures, research ethics), so team members at all sites share a similar understanding of the study objectives and data collection procedures. Subsequent training in data analysis augments that team building effort and the shared experience of implementing a common protocol. We also seek opportunities to bring together at least some members of the research team from each site. The monetary costs are high for international work but the end result is data that are richer and suitable for cross-site analysis.

Other ways in which methodological consistency and data comparability can be achieved are by using a semi-structured instrument, to ensure that similar questions are asked of all participants, and using a standard transcription protocol[2]. Structure enhances reliability during data analysis as well. As Fernald and Duclos suggest, team members should write down code definitions, but we further recommend that they be given, or be asked to develop, a specific structure to guide how this is done[3]. We also note that one of the advantages of team-based qualitative research is the ability to work with multiple data coders and rigorously assess inter-coder agreement, an important issue in qualitative data analysis[4].

With each new project we have undertaken, we have felt the need to consolidate the kind of systematic guidance that Fernald and Duclos outline in their article. To this end, we are in the process of writing and editing a collection of papers for a handbook for team-based qualitative research, which will provide an in-depth and comprehensive treatment of the subject. We expect the volume to be available in late 2006 (AltaMira Press).

1. Fernald D, Duclos C. Enhance your team-based qualitative research. Annals of Family Medicine. 2005;3:360-364.

2. McLellan E, MacQueen KM, Niedig J. Beyond the qualitative interview: data preparation and transcription. Field Methods. 2003;15:63- 84.

3. MacQueen K, McLellan E, Kay K, Milstein B. Codebook development for team-based qualitative analysis. Cultural Anthropology Methods. 1998;10:31-36.

4. Hruschka D, Schwartz D, Cobb St.John D, Picone-Decaro E, Jenkins R, Carey J. Reliability in coding open-ended data: lessons learned from HIV behavioral research. Field Methods. 2004; 16:307-331.

Competing interests:   None declared