Designing A Mixed Methods Study In Primary Care

The article "Designing A Mixed Methods Study In Primary Care" by Creswell, Fetters and Ivankova (1) is a valuable step in the evolution of rigorous methods for combining qualitative and quantitative approaches to primary care research. Their coding template represents a useful means for applying rigorous criteria both to the design of mixed methods studies in primary care, and to the assessment and comparison of such studies. The article might also serve to further stimulate consideration of some of the underlying epistemological issues at the root of combining qualitative and quantitative methods of research.

The processes of primary care are characterized by layered complexity, much of which is missed when examined with only one method, or way of knowing. It is as if one were asked to describe a birthday cake using only the sense of sight. Visual knowledge of the cake would reveal its size, shape, color, and even the flickering candles. But, visual knowledge would not reveal the cake's chocolate smell, the smell of the candles after they are blown out, the moistness or sweet taste of the frosting, nor the sounds of singing as the cake is carried into the room. To have the fullest experience and understanding of the cake, we would need to involve many or all of the possible senses, or ways of knowing…a mixed methods approach. As Stange and Borkan note in their editorials, mixed methods approaches hold promise for primary care research as a means for providing new insights on complex phenomena, and for understanding specific phenomena within the contexts in which they are embedded.

As many have recognized, however, the mixing of qualitative and quantitative methods is not without potential problems related to differences in the theoretical and epistemological assumptions behind each method (2,3). To stretch the birthday cake analogy, the integration of visual information from both eyes to produce binocular vision is more straight forward than the integration of information of different logical types, such as that from both visual and acoustic sources. Many questions arise: How are low context quantitative data integrated with the higher context qualitative data to produce a coherent understanding of the phenomenon being studied? For a given type of quantitative data, do different qualitative methods produce different contextual information, and, if so, do the different possible combinations of methods yield different meanings? Under what circumstances are different mixes of particular qualitative and quantitative methods appropriate? How do we best analyze the data of different logical types, which result from the application of both qualitative and quantitative methods?

Morgan contends that questions of the compatibility of qualitative and quantitative knowledge are empirical questions, and not purely philosophical ones. As primary care researchers "lead the charge of integration and comprehensiveness", as Borkan (4) encourages us to do, we will have the opportunity to explore how these different ways of understanding can be brought together. Creswell et al have given us a framework that contributes substantially to the development of greater rigor in mixed methods as applied to primary care.

References: 1) Creswell, JW, Fetters MD, Ivankova NV. Designing a mixed methods study in primary care. Ann Fam Med. 2004;2:7-12. 2) Morgan DL. Practical strategies for combining qualitative and quantitative methods: applications to health research. Qual Health Res. 1998;8:362-76. 3) Barbour, RS. Mixing qualitative methods: quality assurance or quality quagmire? Qual Health Res. 1998;8:352-61. 4) Borkan, JM. Mixed methods studies: a foundation for primary care research. Ann Fam Med. 2004;2:4-6.

David I. Buckley, MD, Assistant Professor, Family Medicine, Oregon Health and Science University, Portland, OR

Competing interests:   None declared

In the latest issue of Annales, the article by JW Cresswell, MD Fetters, and NV Ivankova is interesting reading. No doubt the complexity of primary care is well suited for the use of qualitative research. And the positive comments on the article indicate that there is indeed a place for qualitative research (or mixed or integrated qualitative/quantitative research) in primary care research. This is all very well.

But there is a problem, which is hinted at in the article but not reflected on, and that is the difference between qualitative method, or data collection, and qualitative research. In reviewing the 5 articles that form the basis for the article Creswell et al point out, that the way qualitative end quantitative methods are combined, most often qualitative methods become helpers in quantitative research. That qualitative data is mainly used to establish questions to put into a questionnaire or to find common denominators in statements from interviewees.

But is this the kind of qualitative research that is needed? As a qualitative researcher within family medicine I wonder whatever happened to the other part of qualitative research: analysis of the data?

Qualitative analysis is more than coding and commonalities. In qualitative research in disciplines like anthropology and ethnography this would mean analysing the empirical data to find domains and categories coming from the informants and not the researchers. Furthermore it would mean looking for diversity and individuality as well as shared attitudes and actions. And most often the really interesting aspects in research are found in the exceptions, the breaking of the principles etc., or when finds are analysed in relation to their context. And empirical finds would be put in context in relation to theories that could put them into a wider perspective and generalise the analysis. A historian or a sociologist would do it slightly differently, but a qualitative analysis has a long tradition within other disciplines. Traditions and knowledge with a potential, that ought to be used in medical research.

But does it matter? Being a primary care researcher with experience in integrating qualitative and quantitative methods I find it sad that the potential of qualitative method is not used to its full. The article by McVea, et al. gives insight into the knowledge that may be gained by a thorough qualitative analysis. Often qualitative analysis is reduced to a couple of citations with no indication of context or variation. One of the reasons as to why the qualitative is mainly put into service by the quantitative could be found in Dean's article describing the role of methods in maintaining orthodox beliefs in health research. Dean's field is wider than primary care but the reductionism she finds I see reflected in the use and understanding of qualitative research in primary health care.

References: JW Cresswell, MD Fetters, and NV Ivankova. Designing A Mixed Methods Study In Primary Care. Annales of Family Medicine 2 (1):7-12, 2004. K Dean. The role of methods in maintaining orthodox beliefs in health research. Social Science & Medicine 58:675 685, 2004. K McVea, B Crabtree, JD Medder, and et al. An ounce of prevention? Evaluation of 'Put prevention into practice' program. Journal of Family Practice 1996 (43):361-369, 1996.

Competing interests:   None declared

We recently initiated a series of projects designed to answer the questions: 1. What is the prevalence of diabetes mellitus among the aboriginal and non-aboriginal populations living in the rural, remote community of Bella Coola? 2. Are there differences in blood sugar control (hemoglobin A1c) between aboriginal and non-aboriginal diabetics? We sent out thousands of surveys, reviewed thousands of charts, and we found that prevalence of diabetes was higher among aboriginal population and they also tended to have poorer blood sugar control. One of the things I observed while doing the study was that some of the poorest controlled diabetics were also those who refused to come in regularly and they were the same who refused to complete surveys. I knew these diabetics and I knew I had told them to come in, but I did not know why they would not come in, or why they would not participate in surveys. Were they seeing doctors elsewhere? Were they angry with care received? So I decided to simply ask them next I saw them – at the store, at social event, wherever. One said they were waiting to be called to come in ; the other said she loved soda pop and was not yet ready to quit; and the third said he was still smoking and did not want to come in until he had quit. So I told them to all to come in and they did; and their blood sugars are improving. I learned that quantitative research provided useful information as to extent of the local diabetic problem; but it was qualitative inquiry that provided insight into why the poorest controlled diabetics were not receiving recommended treatments. And it was the qualitative inquiry that brought these diabetics back into the clinic, and onto a proper diabetes management program. This is a simple example of Creswell’s statement “…neither quantitative nor qualitative methods are sufficient in themselves to capture the trends and details of the situation.” Quantitative research will provide statistics as to the extent of a health problem. Quantitative research will also provide the statistics that can be used in applications for additional funding to deal with that health problem. But it will be qualitative stuff that provides insights and directions as to how primary care health providers can best improve the health of our patients / clients; e.g. how do we improve blood sugar control in our clinic diabetic population?

Competing interests:   None declared

The article "Designing A Mixed Methods Study In Primary Care" by Creswell, Fetters and Ivankova, has presented a basic cross section of what is out there already as far as mixed methods research in primary care. This article raised the discussion about the criteria that is needed to do this type of research. This is very important to me as a junior researcher in having the criteria as reference and also that Family Medicine has found its niche in the world of research.

Competing interests:   None declared

The article by Creswell, Fetters and Ivankova brings into light two major points we must consider when dealing with family medicine research. It cleverly proposes a pragmatical framework for designing and evaluating mixed methods studies. This framework includes a taxonomical approach on which family medicine researchers can build on, as well as a usefull exemple of an evaluation process for five articles based on criteria that fit studies’ characteristics. In their discussion the authors also present three different models of mix method study. I agree with Borkan’s comments that lots of primary care mix method studies are not published as such, for various reasons including academic turf. Traditionnal “positivist rigor” adopted by grant agency or medical publisher in their evaluation processes is slowly evolving toward interest for more complex and meaningful studies. Work like the one reported in this article will enable them to progress in this direction with a rigourous approach, as opposed to a rigid one.

As a family physician practitionner and researcher, and in the light of this article and related editorials by Stange and Borkan, I can see how clinical work and research present similar processes. First primary care physicians begin their clinical evaluation with an inductive phase of generating a list of differential diagnosis based on qualitative and quantitative data collected with the medical interview, physical observation and chart content analysis. Then, they test their diagnosis hypothesis with a more deductive process in order to confirm or reject them with quantitative data like lab test or qualitative triangulation process like searching for colleague’s medical opinions. They often complete iterative evaluation processes that allow them to reexamine their hypothesis and generate new one if necessary. It does sound like a mix method evaluation, doesn’t it?

I would like to emphasize that work by Creswell and his colleague, as well as those already published by many authors make me feel confident that knowledge generated by mixed method studies is valid and very relevant for family physicians, and will ultimately benefit all the community.

Competing interests:   None declared

The article “Designing A Mixed Methods Study In Primary Care” by Creswell, Fetters and Ivankova in January/February 2004 Annals of Family Medicine is a well written piece that furthers the efforts of this internationally known team to educate primary care researchers in multi- methods research. The approach of defining multi-methods research, citing its relevance to primary care research and acceptance by the National Institutes of Health as an area of broad appeal in public health research lays groundwork for the following analysis of 5 articles which utilized three variations of multi-methods research. Presenting each of 5 requisites for the multi-methods approach (rationale for mixing qualitative and quantitative data, types of data collected, priority of data, implementation sequence and data integration) provides a conceptual framework for researchers to develop multi-methods studies. Within the results section, each of these areas is addressed with examples from the reviewed articles and with additional examples provided, leaving the reader with an excellent set of tools and examples to begin conceptualizing a study. The discussion provides three models for multi- methods research with ideas on where best utilization as well as challenges a researcher can expect in planning and execution of a study.

This is an important work for primary care research. It begins a labelling process for multi-methods studies with excellent teaching examples for their implementation. The authors call for future work to be accomplished to label other models, and this would certainly add to the primary care researcher’s toolbox. The readibility of the presentation makes this a source document for the budding as well as the experienced researcher. It builds on the previous works of this team to promote this important approach to our research. As primary care research continues to grow, as noted by the continued growth of forums for primary care research such as the North American Primary Care Research Group, education and promotion of this approach to research, as is eloquently presented in this paper, is key to sound research. Kudos for their work.

WILLIAM J. CURRY, MD Associate Professor, Family & Community Medicine M. S. Hershey Medical Center College of Medicine, Pennsylvania State Univerity Hershey, Pennsylvania 17033

Competing interests:   None declared

I want to commend Creswell et al for their presentation of a framework for understanding the role of qualitative components within a mixed method design. As a staunch quantitative researcher, this article helped me to better conceptualize how qualitative and quantitative methods can complement each other. It almost makes me want to become a qualitative researcher. Almost.

Competing interests:   None declared