|
|
||||||||
Article |
1 Department of Educational Psychology, University of Nebraska-Lincoln, Lincoln, Neb
2 Department of Family Medicine, University of Michigan Health System, Ann Arbor, Mich
3 Office of Qualitative and Mixed Methods Research, University of Nebraska-Lincoln, Lincoln, Neb
CORRESPONDING AUTHOR: John W. Creswell, PhD, Department of Educational Psychology, 241 Teachers College Bldg, University of Nebraska-Lincoln, Lincoln, NE 68588, jcreswell1{at}unl.edu
| ABSTRACT |
|---|
|
|
|---|
METHODS We first identified criteria from the social and behavioral sciences to analyze mixed methods studies in primary care research. We then used the criteria to evaluate 5 mixed methods investigations published in primary care research journals.
RESULTS Of the 5 studies analyzed, 3 included a rationale for mixing based on the need to develop a quantitative instrument from qualitative data or to converge information to best understand the research topic. Quantitative data collection involved structured interviews, observational checklists, and chart audits that were analyzed using descriptive and inferential statistical procedures. Qualitative data consisted of semistructured interviews and field observations that were analyzed using coding to develop themes and categories. The studies showed diverse forms of priority: equal priority, qualitative priority, and quantitative priority. Data collection involved quantitative and qualitative data gathered both concurrently and sequentially. The integration of the quantitative and qualitative data in these studies occurred between data analysis from one phase and data collection from a subsequent phase, while analyzing the data, and when reporting the results.
DISCUSSION We recommend instrument-building, triangulation, and data transformation models for mixed methods designs as useful frameworks to add rigor to investigations in primary care. We also discuss the limitations of our study and the need for future research.
Key Words: Research design/methods data collection, methods investigative techniques social sciences qualitative research
| INTRODUCTION |
|---|
|
|
|---|
The concept of mixed methods research is not new to primary care. Almost 15 years ago, authors introduced the importance of integrating quantitative and qualitative research in a single study or program of inquiry.2,3 More recently, investigators emphasized that quantitative and qualitative methods can be mixed, such as in collecting qualitative data before quantitative data where variables are unknown, or using qualitative methods to expand quantitative results to advance study aims.4,5 This integration calls for collecting quantitative and qualitative data concurrently or in parallel or gathering information sequentially.6
Mixed methods inquiry as a new research paradigm has found a breeding ground in the North American Primary Care Research Group.7 Additionally, in 1999 a National Institutes of Health (NIH) task force in the Office of Behavioral and Social Sciences Research issued guidelines for conducting rigorous qualitative and multimethod investigations.8 In a brief section at the end of the NIH document, the task force noted the "broad appeal" of combining qualitative and quantitative methods in public health research, and it recommended that investigators be specific about how their methods will be combined and how the findings will be integrated.
Although the NIH document suggested several models for combining quantitative and qualitative data, it did not describe specific criteria that primary care researchers might use to design mixed methods investigations. Furthermore, it did not refer to published studies that illustrate mixed methods research or address specific models of mixed methods investigations. Thus the NIH document overlooked discussions about criteria, examples, and models widely accessible in recent social and behavioral science literature.911 The purpose of the present study was to use criteria available in the literature to analyze 5 published mixed methods studies in primary care. Based on our analysis, we recommend 3 models as organizing frameworks for primary care investigators.
| METHODS |
|---|
|
|
|---|
To locate these studies, we conducted a MEDLINE search of the years 1990 to 2001 using the key word descriptors of "multimethod and primary care," "qualitative and quantitative methods and family medicine/primary care," and such MeSH terms as "research design" and "research methodology." From among the 15 studies found, we selected 5 that met the criteria of our coding template.1216 They (1) used quantitative and qualitative data collection and analysis; (2) integrated, combined, or related both quantitative and qualitative data at some stage in the research process; (3) collected the data in a primary care setting; and (4) integrated or combined the data within a single study. We excluded for future research sustained programs of inquiry that consisted of multiple quantitative and qualitative investigations, such as the Family Practice Smoking Cessation Project17 or the Direct Observation of Primary Care Study.18
| RESULTS |
|---|
|
|
|---|
|
Only 3 of the 5 studies clearly articulated a rationale.1214 The study by Kutner et al mentioned that qualitative data were collected to identify questions relevant for developing and testing an instrument.13 The 2 other studies by Baskerville et al and McVea et al stated that gathering both forms of data contributed to a comprehensive and complete understanding of the results.12,14 One study mentioned this rationale in the introduction,12 one in the methods,13 and one in the discussion section at the end of the article.14
Other rationales mentioned in mixed methods studies might state that qualitative data help explore statistical results from quantitative data, or that quantitative outlier or extreme results can be better understood through qualitative data collection. Quantitative results might also help researchers select qualitative cases so they can examine the results in greater depth.1
Data Collection and Analysis
The forms of data collection and analysis in the 5 studies showed data was collected by structured instruments and more open-ended field observations and interviews. Quantitative data collection relied on structured instruments administered by telephone or face-to-face interviews and structured checklists completed by observers. Chart audits also provided numeric data for these studies. The statistical treatment of the data reflected descriptive analysis and some limited inferential analyses (eg, correlation, regression). Qualitative data were collected through open-ended or semistructured interviews and field observations of practices. The qualitative data were coded using predetermined templates and grouped into codes, themes, and categories. These results suggest typical procedures for both quantitative and qualitative data collection and analysis.19,20
Other types of qualitative data collection procedures, such as videotapes, photographs, or e-mails, did not surface in these projects.20 Aside from case studies,14 the application of other distinct traditions of qualitative inquiry, such as ethnographies, grounded theory studies, or biographies, also were not present.19 Advanced statistical procedures were largely absent.
Priority
Priority is determined by the researchers, who place an emphasis on quantitative data, qualitative data, or an equal priority shared between the 2 forms of data.1,10 Such priority is detected at the beginning of the study by noting the relative emphasis given to framing the research problem (eg, intent to test a theory, study variables, or explore constructs) or the subservient use of 1 form of data to the other (eg, qualitative data helps to build an instrument). Alternatively, in some studies, investigators might give equal emphasis and status by providing both detailed quantitative and qualitative data collection and data analysis.
Turning to the 5 studies, Kutner et al emphasized equally the quantitative and qualitative components in the study.13 McVea et al emphasized qualitative data collection and analysis,14 whereas the other 3 emphasized quantitative data collection and analysis.12,15,16 For example, the study by Nutting et al16 illustrates priority given to quantitative research. In this 2-phase study of barriers encountered by physicians and nurses to implementing guidelines for depression, the investigators focused on developing a structured checklist of 45 barriers to treatment. Their results highlight this checklist and the weighting of the factors; only a small initial qualitative interview phase to identify the barriers cast this study as a mixed methods investigation. For all 5 of the studies, a quantitative rather than a qualitative orientation was given priority.
Implementation
Implementation refers to whether the quantitative and qualitative data are collected in sequential phases or gathered concurrently at roughly the same time during the study.10 In a sequential approach, quantitative or qualitative data collection serves as a basis for the next data collection and analysis stage. This approach is ideal when one phase can contribute to the next phase and enhance the entire study. In the concurrent approach, quantitative and qualitative data are collected at the same time and are brought together in the results or interpretation of the results. Given time limitations for primary care research, concurrent approaches that enable the collection of multiple forms of data at one time might be more practical than a sequential approach. Other important considerations include the types of research questions, sampling goals, and the unit of analysis.
A mixed picture of implementation emerged from the 5 studies analyzed. Two of the studies reported concurrent approaches12,14 and 3 reported sequential approaches.13,15,16 McVea et al used a concurrent approach in which they emphasized the qualitative data collection and analysis, but they collected quantitative encounter data and chart audits at the same time they were in the field gathering qualitative field observations and office interviews.14 Alternatively, Kutner et al conducted a 2-phase sequential investigation in which the initial qualitative phase of data collection and analysis was followed by a second quantitative phase.13 Specifically, they began the study by collecting qualitative interview data (in the first phase) to explore patients information needs, then analyzed the interview data and used the information to build a semistructured instrument that was administered to a sample (in the second phase).
Integration
Priority and implementation decisions in design lead to the logical place in research methods (data collection, analysis, interpretation, or discussion19) where the investigators bring together the quantitative and qualitative data. Integration refers to the point in the process of research procedures at which the investigator mixes or integrates the quantitative and qualitative data collection and analysis.11 Based on the 5 studies, we identified 2 stages in the process where integration is possible: when data analysis leads to further data collection decisions,13,15,16 and when results are reported.12,14
For example, in the study by Nutting et al, integration occurred during data analysis, when the qualitative interviews were used to develop a quantitative checklist.16 Integration thus fl owed from qualitative data analysis into quantitative data collection. Similarly, McIlvain et al transformed the qualitative data by assigning it quantitative codes, thus illustrating another approach to integration at the data analysis stage of research.15 Alternatively, McVea et al integrated the qualitative and quantitative data in their results to portray 3 different types of case office practices.14 The issue of integrationthe stage of the research process at which the data are combinedillustrates the complexity of mixed methods research and the need to be explicit about the model of inquiry being used.
| DISCUSSION |
|---|
|
|
|---|
Instrument Design Model
In the instrument design model, priority is given to quantitative data collection and analysis. Implementation is a 2-phase project that begins with qualitative data collection and analysis and moves to quantitative instrument design and testing. Integration occurs at the data analysis stage, when the researchers analyze the qualitative data and use this information to develop an instrument for data collection. The intent of this model is to develop an instrument that is grounded in the views of participants (eg, patients) rather than use an off-the-shelf instrument that might not reflect their views. With a sequential approach, the study is logical and easily conducted, but expertise is needed to code and analyze qualitative data, as well as develop a psychometrically sound instrument. The studies by Kutner et al13 and Nutting et al16 illustrate this model of mixed methods design.
Triangulation Design Model
The triangulation design model is frequently used in primary care research, although it is more difficult to implement than the sequential instrument design model because of the need to reconcile and bring together numeric (quantitative) and text (qualitative) data. The intent of this model is to triangulate or gather both quantitative and qualitative data at the same time, and to integrate the two forms of data to best understand a research problem.11 This model typically gives equal priority to quantitative and qualitative data and analysis (often found in separate sections of the report), involves concurrent or simultaneous collection of data, and integrates both quantitative and qualitative data in the results, interpretation, or conclusion phase.
A typical structure for a triangulation study is to have separate sections on quantitative data collection and qualitative data collection, as well as separate sections on quantitative data analysis and qualitative data analysis. The investigators then provide a results, discussion, or conclusion section in which they discuss the results of both analyses. Typically investigators present the 2 forms of results as supporting or conflicting evidence for results, or they might transform one type of data into another form (quantitatively count the codes from qualitative results) to converge results.1 The studies by Baskerville et al12 and the McVea et al14 illustrate the triangulation model in primary care.
Data Transformation Design Model
The data transformation design model is well suited for the primary care investigator because it encompasses correlational (observational) designs, such as prevalence studies, retrospective studies, or prospective studies.21 This model allows the researcher to gather qualitative data, analyze it for codes and themes according to a predetermined codebook or conceptual framework, and (typically) numerically count the codes and themes. The priority favors quantitative data collection and analysis, the implementation is concurrent, and the integration occurs at the data analysis stage of the research process. The study by McIlvain et al15 illustrates this model.
These 3 models illustrate some of the possibilities for mixed methods models in primary care. Others not illustrated, but discussed in the social and behavioral science literature, include the explanatory model and the nested model. In the explanatory model, an initial quantitative phase is conducted to obtain statistical results.1 In a second phase, the investigator gathers qualitative data (eg, open-ended interviews) to help explain the quantitative results. In the nested model, a smaller qualitative data collection phase is embedded within a larger quantitative intervention trial.11 Although this model often fits the criteria for a concurrent and quantitative-priority design, it represents a variation in which the larger component addresses one question (eg, was the intervention successful?) and the smaller component another question (eg, how did patients react from a cultural perspective during the trial?).
| CONCLUSIONS |
|---|
|
|
|---|
We recognize several limitations in our discussion of the 5 studies. Our illustrative studies fit the inclusion criteria for selection but limit the number of studies actually discussed and the generalizability of our findings. Furthermore, although the authors from the 5 target articles clearly intended to gather and analyze both quantitative and qualitative data in each study, they might not have intended (or considered) combining or integrating the data, which is a centerpiece of mixed methods designs in the social sciences.1,20 Our coding template for analysis, drawn from the social sciences, might need to be modified to better address the nuances of mixed methods investigations in primary care. This template, however, seems to add rigor to design decisions that are needed in primary care and advocated by NIH.
The future analysis of primary care mixed methods investigations might focus on models addressed in the literature but not discussed here. The taxonomy should be examined for appropriateness in other health disciplines, such as nursing, allied health, and critical care. Additional studies could help researchers not only develop a better understanding of mixed methods approaches that are used in sustained programs of inquiry consisting of multiple studies but also refine further the criteria for evaluating the quality of primary care mixed methods studies. The discussion initiated here holds promise for designing rigorous proposals for funding and clarifies the complex designs inherent in this form of inquiry.
| FOOTNOTES |
|---|
Funding support: Dr. Fetters involvement in this work was made possible in part by the generous support of the Robert Wood Johnson Generalist Faculty Scholars Program.
Received for publication August 30, 2002. Revision received December 12, 2002. Accepted for publication January 4, 2003.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
A. Taylor Substance Use and Abuse: Women's Criminal Reoffending in New Zealand Affilia, May 1, 2008; 23(2): 167 - 178. [Abstract] [PDF] |
||||
![]() |
A. O'Cathain, E. Murphy, and J. Nicholl The quality of mixed methods studies in health services research J Health Serv Res Policy, April 1, 2008; 13(2): 92 - 98. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Mendlinger and J. Cwikel Spiraling Between Qualitative and Quantitative Data on Women's Health Behaviors: A Double Helix Model for Mixed Methods Qual Health Res, February 1, 2008; 18(2): 280 - 293. [Abstract] [PDF] |
||||
![]() |
J. Wu, T. Gipson, N. Chin, L. L. Wynn, K. Cleland, C. Morrison, and J. Trussell Women Seeking Emergency Contraceptive Pills by Using the Internet Obstet. Gynecol., July 1, 2007; 110(1): 44 - 52. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. I. Solberg Improving Medical Practice: A Conceptual Framework Ann. Fam. Med, May 1, 2007; 5(3): 251 - 256. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Handley, C. Hall, E. Sanford, E. Diaz, E. Gonzalez-Mendez, K. Drace, R. Wilson, M. Villalobos, and M. Croughan Globalization, Binational Communities, and Imported Food Risks: Results of an Outbreak Investigation of Lead Poisoning in Monterey County, California Am J Public Health, May 1, 2007; 97(5): 900 - 906. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. K. Barg, R. Huss-Ashmore, M. N. Wittink, G. F. Murray, H. R. Bogner, and J. J. Gallo A mixed-methods approach to understanding loneliness and depression in older adults. J. Gerontol. B. Psychol. Sci. Soc. Sci., November 1, 2006; 61(6): S329 - S339. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Katerndahl and B. Crabtree Creating innovative research designs: the 10-year methodological think tank case study. Ann. Fam. Med, September 1, 2006; 4(5): 443 - 449. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. C. Stange, B. F. Crabtree, and W. L. Miller Publishing Multimethod Research Ann. Fam. Med, July 1, 2006; 4(4): 292 - 294. [Full Text] [PDF] |
||||
![]() |
L. I. Solberg, A. L. Crain, J. M. Sperl-Hillen, M. C. Hroscikoski, K. I. Engebretson, and P. J. O'Connor Care Quality and Implementation of the Chronic Care Model: A Quantitative Study Ann. Fam. Med, July 1, 2006; 4(4): 310 - 316. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. L. Sussman, R. L. Williams, R. Leverence, P. W. Gloyd Jr, B. F. Crabtree, and On Behalf Of Rios Net Clinicians The Art and Complexity of Primary Care Clinicians' Preventive Counseling Decisions: Obesity as a Case Study Ann. Fam. Med, July 1, 2006; 4(4): 327 - 333. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. McEvoy and D. Richards A critical realist rationale for using a combination of quantitative and qualitative methods Journal of Research in Nursing, January 1, 2006; 11(1): 66 - 78. [Abstract] [PDF] |
||||
![]() |
K. C. Stange On TRACK Ann. Fam. Med, March 1, 2004; 2(2): 177 - 179. [Full Text] [PDF] |
||||
![]() |
K. C. Stange In this Issue: Multimethod Research Ann. Fam. Med, January 1, 2004; 2(1): 2 - 3. [Full Text] [PDF] |
||||
![]() |
J. M. Borkan Mixed Methods Studies: A Foundation for Primary Care Research Ann. Fam. Med, January 1, 2004; 2(1): 4 - 6. [Full Text] [PDF] |
||||
Read all TRACK Comments
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |