Research ArticleOriginal Research

Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations

Deborah J. Cohen and Benjamin F. Crabtree
The Annals of Family Medicine July 2008, 6 (4) 331-339; DOI: https://doi.org/10.1370/afm.818

Abstract

PURPOSE We wanted to review and synthesize published criteria for good qualitative research and develop a cogent set of evaluative criteria.

METHODS We identified published journal articles discussing criteria for rigorous research using standard search strategies then examined reference sections of relevant journal articles to identify books and book chapters on this topic. A cross-publication content analysis allowed us to identify criteria and understand the beliefs that shape them.

RESULTS Seven criteria for good qualitative research emerged: (1) carrying out ethical research; (2) importance of the research; (3) clarity and coherence of the research report; (4) use of appropriate and rigorous methods; (5) importance of reflexivity or attending to researcher bias; (6) importance of establishing validity or credibility; and (7) importance of verification or reliability. General agreement was observed across publications on the first 4 quality dimensions. On the last 3, important divergent perspectives were observed in how these criteria should be applied to qualitative research, with differences based on the paradigm embraced by the authors.

CONCLUSION Qualitative research is not a unified field. Most manuscript and grant reviewers are not qualitative experts and are likely to embrace a generic set of criteria rather than those relevant to the particular qualitative approach proposed or reported. Reviewers and researchers need to be aware of this tendency and educate health care researchers about the criteria appropriate for evaluating qualitative research from within the theoretical and methodological framework from which it emerges.

INTRODUCTION

Until the 1960s, the scientific method—which involves hypothesis testing through controlled experimentation—was the predominant approach to research in the natural, physical, and social sciences. In the social sciences, proponents of qualitative research argued that the scientific method was not an appropriate model for studying people (eg, Cicourel,1 Schutz,2,3 and Garfinkel4), and such methods as observation and interviewing would lead to a better understanding of social life in its naturally occurring, uncontrolled form. Biomedical and clinical research, with deep historical roots in quantitative methods, particularly observational epidemiology5 and clinical trials,6 was on the periphery of this debate. It was not until the late 1960s and 1970s that anthropologists and sociologists began introducing qualitative research methods into the health care field.4,7–14

Since that time, qualitative research methods have been increasingly used in clinical and health care research. Today, both journals (eg, Qualitative Health Research) and books are dedicated to qualitative methods in health care,15–17 and a vast literature describes basic approaches of qualitative research,18,19 as well as specific information on focus groups,20–23 qualitative content analysis,24 observation and ethnography,25–27 interviewing,28–32 studying stories33,34 and conversation,35–37 doing case study,38,39 and action research.40,41 Publications describe strategies for sampling,42–45 analyzing, reporting,45–49 and combining qualitative and quantitative methods50; and a growing body of health care research reports findings from studies using in-depth interviews,51–54 focus groups,55–57 observation,58–60 and a range of mixed-methods designs.61–63

As part of a project to evaluate health care improvements, we identified a need to help health care researchers, particularly those with limited experience in qualitative research, evaluate and understand qualitative methodologies. Our goals were to review and synthesize published criteria for “good” qualitative research and develop a cogent set of evaluative criteria that would be helpful to researchers, reviewers, editors, and funding agencies. In what follows, we identify the standards of good qualitative research articulated in the health care literature and describe the lessons we learned as part of this process.

METHODS

A series of database searches were conducted to identify published journal articles, books, and book chapters offering criteria for evaluating and identifying rigorous qualitative research.

Data Collection and Management

With the assistance of a librarian, a search was conducted in December 2005 using the Institute for Science (ISI) Web of Science database, which indexes a wide range of journals and publications from 1980 to the present. Supplemental Appendix 1, available online-only at http://www.annfammed.org/cgi/content/full/6/4/331/DC1, describes our search strategy. This search yielded a preliminary database of 4,499 publications. Citation information, abstracts, and the number of times the article was cited by other authors were exported to a Microsoft Excel file and an Endnote database.

After manually reviewing the Excel database, we found and removed a large number of irrelevant publications in the physical and environmental sciences (eg, forestry, observational studies of crystals), and further sorted the remaining publications to identify publications in health care. Among this subset, we read abstracts and further sorted publications into (1) publications about qualitative methods, and (2) original research using qualitative methods. For the purposes of this analysis, we reviewed in detail only publications in the first category. We read each publication in this group and further subdivided the group into publications that (1) articulated criteria for evaluating qualitative research, (2) addressed techniques for doing a particular qualitative method (eg, interviewing, focus groups), or (3) described a qualitative research strategy (eg, sampling, analysis). Subsequent analyses focused on the first category; however, among publications in the second category, a number of articles addressed the issue of quality in, for example, case study,39 interviewing,28 focus groups,22,64,65 discourse,66 and narrative67,68 research that we excluded as outside the scope of our analysis.

Books and book chapters could not be searched in the same way because a database cataloging these materials did not exist. Additionally, few books on qualitative methods are written specifically for health care researchers, so we would not be able to determine whether a book was or was not contributing to the discourse in this field. To overcome these challenges, we used a snowball technique, identifying and examining books and book chapters cited in the journal articles retrieved. Through this process, a number of additional relevant journal articles were identified as frequently cited but published in non–health care or nonindexed journals (eg, online journals). These articles were included in our analysis.

Analysis

We read journal articles and book chapters and prepared notes recording the evaluative criteria that author(s) posited and the world view or belief system in which criteria were embedded, if available. When criteria were attributed to another work, this information was noted. Books were reviewed and analyzed differently. We read an introductory chapter or two to understand the authors’ beliefs about research and prepared summary notes. Because most books contained a section discussing evaluative criteria, we identified and read this section, and prepared notes in the manner described above for journal articles and book chapters.

An early observation was that not all publications offered explicit criteria. Publications offering explicit evaluative criteria were treated as a group. Publications by the same author were analyzed and determined to be sufficiently similar to cluster. We examined evaluative criteria across publications, listing similar criteria in thematic clusters (eg, importance of research, conducting ethically sound research), identifying the central principle or theme of the cluster, and reviewing and refining clusters. Publications that discussed evaluative criteria for qualitative research but did not offer explicit criteria were analyzed separately.

Preliminary findings were synthesized into a Web site for the Robert Wood Johnson Foundation (http://www.qualres.org). This Web site was reviewed by Mary Dixon-Woods, PhD, a health care researcher with extensive expertise in qualitative research, whose feedback regarding the implications of endorsing or positing a unified set of evaluative criteria encouraged our reflection and influenced this report.

RESULTS

We identified 29 journal articles19,26,45,69–94 and 16 books or book chapters95–110 that offered explicit criteria for evaluating the quality of qualitative research. Supplemental Appendix 2, available online-only at http://www.annfammed.org/cgi/content/full/6/4/331/DC1, contains a table listing citation information and criteria posited in these works. An additional 29 publications were identified that did not offer explicit criteria but informed discourse on this topic and our analysis.111–139

Seven evaluative criteria were identified: (1) carrying out ethical research; (2) importance of the research; (3) clarity and coherence of the research report; (4) use of appropriate and rigorous methods; (5) importance of reflexivity or attending to researcher bias; (6) importance of establishing validity or credibility; and (7) importance of verification or reliability. There was general agreement observed across publications on the first 4 quality dimensions; however, on the last 3 criteria, disagreement was observed in how the concepts of researcher bias, validity, and reliability should be applied to qualitative research. Differences in perspectives were grounded in paradigm debates regarding the nature of knowledge and reality, with some arguing from an interpretivist perspective and others from a more pragmatic realist perspective. Three major paradigms and their implications are described in Table 1⇓.

Fundamental Criteria

It was widely agreed that qualitative research should be ethical, be important, be clearly and coherently articulated, and use appropriate and rigorous methods. Conducting ethically sound research involved carrying out research in a way that was respectful,69 humane,95 and honest,77 and that embodied the values of empathy, collaboration, and service.77,84 Research was considered important when it was pragmatically and theoretically useful and advanced the current knowledge base.* Clarity and coherence of the research report were criteria emphasizing that the report itself should be concise and provide a clear and adequate description of the research question, background and contextual material, study design (eg, study participants, how they were chosen, how data are collected and analyzed), and rationale for methodological choices. Description of the data should be unexaggerated, and the relationship between data and interpretation should be understandable.†

Researcher Bias

The majority of publications discussed issues of researcher bias, recognizing researchers’ preconceptions, motivations, and ways of seeing shape the qualitative research process. (It should be noted there is ample evidence to suggest researcher motivations and preconceptions shape all research.)140 One perspective (interpretivist) viewed researcher subjectivity as “something used actively and creatively through the research process” rather than as a problem of bias.72 A hallmark of good research was understanding and reporting relevant preconceptions through reflexive processing (ie, reflective journal-keeping).‡ A second perspective (realist) viewed researcher bias as a problem affecting the trustworthiness, truthfulness, or validity of the account. In addition to understanding researchers’ motivations and preconceptions, value and rigor were enhanced by controlling bias through techniques to verify and confirm findings, as discussed in more detail below.* Thus, whereas all publications agreed that researcher bias was an important consideration, the approach for managing bias was quite different depending on the paradigm grounding the work.

Validity

A number of publications framed the concept of validity in the context of quantitative research, where it typically refers to the “best available approximation to the truth or falsity of propositions.”142(p37) Internal validity refers to truth about claims made regarding the relationship between 2 variables. External validity refers to the extent to which we can generalize findings. Across publications, different ideas emerged.

Understanding the concept of validity requires understanding beliefs about the nature of reality. One may believe that there can be multiple ways of understanding social life and reality, even multiple realities. This view of reality emerges from an interpretivist perspective. Hallmarks of high-quality qualitative research include producing a rich, substantive account with strong evidence for inferences and conclusions and then reporting the lived experiences of those observed and their perspectives on social reality, while recognizing that these could be multiple and complex and that the researcher is intertwined in the portrayal of this experience. The goal is understanding and providing a meaningful account of the complex perspectives and realities studied.†

In contrast, research may be based on the belief that there is one reality that can be observed, and this reality is knowable through the process of research, albeit sometimes imperfectly. This perspective is typically associated with a positivist paradigm that underlies quantitative research, but also with the realist paradigm found in some qualitative research. Qualitative research based on this view tends to use alternative terms for validity (eg, adequacy, trustworthiness, accuracy, credibility) and emphasizes striving for truth through the qualitative research process, for example, by having outside auditors or research participants validate findings. An important dimension of good qualitative research, therefore, is plausibility and accuracy.‡

Verification or Reliability

Divergent perspectives were observed on the appropriateness of applying the concept of verifiability or reliability when evaluating qualitative research. As is validity, this concept is rooted in quantitative and experimental methods and refers to the extent to which measures and experimental treatments are standardized and controlled to reduce error and decrease the chance of obtaining differences.142 Two distinct approaches to evaluating the reliability of qualitative research were articulated. In the first, verification was a process negotiated between researchers and readers, where researchers were responsible for reporting information (eg, data excerpts, how the researcher dealt with tacit knowledge, information about the interpretive process) so readers could discern for themselves the patterns identified and verify the data, its analysis and interpretation.§ This interpretivist perspective contrasts with the second, realist, perspective. Rather than leaving the auditing and confirming role to the reader, steps to establish dependability should be built into the research process to repeat and affirm researchers’ observations. In some cases, special techniques, such as member checking, peer review, debriefing, and external audits to achieve reliability, are recommended and posited as hallmarks of quality in qualitative research.|| In Table 2⇓ we provide a brief description of these techniques.

Perspectives on the Value of Criteria

Health care researchers also discuss the usefulness of evaluative criteria. We observed 3 perspectives on the utility of having unified criteria for assessing qualitative research.

One perspective recognized the importance of validity and reliability as criteria for evaluating qualitative research.132,133 Morse et al make the case that without validity and reliability, qualitative research risks being seen as nonscientific and lacking rigor.88,125 Their argument is compelling and suggests reliability and validity should not be evaluated at the end of the project, but should be goals that shape the entire research process, influencing study design, data collection, and analysis choices. A second approach is to view the criteria of validity and reliability as inappropriate for qualitative research, and argue for the development of alternative criteria relevant for assessing qualitative research.*

This position is commonly based on the premise that the theoretical and methodological beliefs informing quantitative research (from whence the criteria of reliability and validity come) are not the same as the methodological and theoretical beliefs informing qualitative research and are, therefore, inappropriate.136 Cogent criteria for evaluating qualitative research are needed. Without well-defined, agreed-upon, and appropriate standards, qualitative research risks being evaluated by quantitative standards, which can lead to assimilation, preferences for qualitative research that are most compatible with quantitative standards, and rejection of more radical methods that do not conform to quantitative criteria.94 From this perspective emerged a number of alternative criteria for evaluating qualitative research.

Alternative criteria have been open to criticism. We observed such criticism in publications challenging the recommendation that qualitative research using such techniques as member checking, multiple coding, external audits, and triangulation is more reliable, valid, and of better quality.72,82,90,91,112,127,143 Authors challenging this recommendation show how techniques such as member checking can be problematic. For example, it does not make sense to ask study participants to check or verify audio-recorded transcribed data. In other situations, study participants asked to check or verify data may not recall what they said or did. Even when study participants recall their responses, there are a number of factors that may account for discrepancies between what participants recall and the researcher’s data and preliminary findings. For instance, the purpose of data analysis is to organize individual statements into themes that produce new, higher-order insights. Individual contributions may not be recognizable to participants, and higher-order insights might not make sense.82 Similar issues have been articulated about the peer-review and auditing processes127,143 and some uses of triangulation.130 Thus, alternative criteria for evaluating qualitative research have been posited and criticized on the grounds that such criteria (1) cannot be applied in a formulaic manner; (2) do not necessarily lead to higher-quality research, particularly if these techniques are poorly implemented; and (3) foster the false expectation among evaluators of research that use of one or more of these techniques in a study is a mark of higher quality.72,81,90,91,112,123,127

A third approach suggests the search for a cogent set of evaluative criteria for qualitative research is misguided. The field of qualitative research is broad and diverse, not lending itself to evaluation by one set of criteria. Instead, researchers need to recognize each study is unique in its theoretical positioning and approach, and different evaluative criteria are needed. To fully understand the scientific quality of qualitative research sometimes requires a deep understanding of the theoretical foundation and the science of the approach. Thus, evaluating the scientific rigor of qualitative research requires learning, understanding, and using appropriate evaluative criteria.123,124,135,137

DISCUSSION

There are a number of limitations of this analysis to be acknowledged. First, although we conducted a comprehensive literature review, it is always possible for publications to be missed, particularly with our identification of books and book chapters, which relied on a snowball technique. In addition, relying on publications and works cited within publications to understand the dialogue about rigor in qualitative methods is imperfect. Although these discussions manifest in the literature, they also arise at conferences, grant review sessions, and hallway conversations. One’s views are open to revision (cf, Lincoln’s103,144), and relationships with editors and others shape our ideas and whom we cite. In this analysis, we cannot begin to understand these influences.

Our perspectives affect this report. Both authors received doctoral training in qualitative methods in social science disciplines (sociology/communication and anthropology) and have assimilated these values into health care as reviewers, editors, and active participants in qualitative health care studies. Our training shapes our beliefs, so we feel most aligned with interpretivism. This grounding influences how we see qualitative research, as well as the perspectives and voices we examine in this analysis. We have been exposed to a wide range of theoretical and methodological approaches for doing qualitative research, which may make us more inclined to notice the generic character of evaluative criteria emerging in the health care community and take note of the potential costs of this approach.

In addition, we use 3 common paradigms—interpretivism, realism, and positivism—in our analysis. It is important to understand that paradigms and debates about paradigms are political and used to argue for credibility and resources in the research community. In this process, underlying views about the nature of knowledge and reality have been simplified, sometimes even dichotomized (interpretivism vs positivism). We recognize our use of these paradigms as an oversimplification and limitation of our work, but one that is appropriate if only because these categories are so widely used in the works we analyze.

Our analysis reveals some common ground has been negotiated with regard to establishing criteria for rigorous qualitative research. It is important to notice that the criteria that have been widely accepted—carrying out ethical research and important research, preparing a clear and coherent research report, and using appropriate and rigorous methods—are applicable to all research. Divergent perspectives were observed in the field with regard to 3 criteria: researcher bias, validity, and verification or reliability. These criteria are more heavily influenced by quantitative and experimental approaches142 and, not surprisingly, have met with resistance. To understand the implications of these influences, our analysis suggests the utility of examining how these criteria are embedded in beliefs about the nature of knowledge and reality.

Central to the interpretivist paradigm, which historically grounds most qualitative traditions, is the assumption that realities are multiple, fluid, and co-constructed, and knowledge is taken to be negotiated between the observer and participants. From this framework emerge evaluative criteria valuing research that illuminates subjective meanings and understands and articulates multiple ways of seeing a phenomenon. Rich substance and content, clear delineation of the research process, evidence of immersion and self-reflection, and demonstration of the researcher’s way of knowing, particularly with regard to tacit knowledge, are essential features of high-quality research.

In contrast, fundamental to a positivist paradigm, which historically grounds most quantitative approaches, is the assumption that there is a single objective reality and the presumption that this reality is knowable. The realist paradigm softens this belief by suggesting knowledge of reality is always imperfect. Within the realist framework the goal of qualitative research is to strive for attaining truth, and good research is credible, confirmable, dependable, and transferable. Thus, rigorous qualitative research requires more than prolonged engagement, persistent observation, thick description, and negative case analysis, but it should use such techniques as triangulation, external auditing, and member checking to promote attainment of truth or validity through the process of verifying findings.

One reason for the centrality of the realist paradigm in health care research may be its ability to assimilate the values, beliefs, and criteria for rigorous research that emerge from the positivist paradigm. In a community that values biomedical bench research, sees the randomized controlled trial as a reference standard, holds a belief in an objective reality, and values research that is reliable, valid, and generalizable (typically positivist ideals), it is not surprising that realist views with regard to qualitative research have found favor. Unlike interpretivism, realism adopts a philosophy of science not at odds with the commonly held ideals of positivism. By maintaining a belief in an objective reality and positing truth as an ideal qualitative researchers should strive for, realists have succeeded at positioning the qualitative research enterprise as one that can produce research which is valid, reliable, and generalizable, and therefore, of value and import equal to quantitative biomedical research.

Although qualitative research emerging from a realist paradigm may have successfully assimilated into the clinical research community (as it has in other disciplines), it may be at a cost. Qualitative approaches most compatible with traditional values of quantitative research may be most likely to be accepted (published and funded). More radical methods (eg, feminist standpoint research, critical postmodern research), which can make innovative contributions to the field, may be marginalized because they do not fit the evaluative criteria that have emerged in the health care community.94,115 In addition, doing rigorous qualitative research in the way realists prescribe involves using a number of techniques that may foster the appearance of validity and reliability, but can be problematic if inappropriately applied.*

The search for a single set of criteria for good qualitative research is grounded in the assumption that qualitative research is a unified field.124,135,137,145 Qualitative research is grounded in a range of theoretical frameworks and uses a variety of methodological approaches to guide data collection and analysis. Because most manuscript and grant reviewers are not qualitative experts, they are likely to embrace a generic set of criteria. Reviewers and researchers need to be aware of the 7 criteria for good qualitative research, but also they need to be aware that applying the same standards across all qualitative research is inappropriate. Helping reviewers understand how an unfamiliar qualitative approach should be executed and standards for evaluating quality are essential, because reviewers, even qualitative experts, might not be well-versed in the particular qualitative method being used or proposed. Panel organizers and editors need to recognize that a qualitative expert may have only a very narrow range of expertise. Moreover, some researchers may be so entrenched in the dogma of their own approach that they are unable to value qualitative methods dissimilar from their own. This type of ax grinding harms not only the efforts of qualitative researchers, but the field more generally.

Future work needs to focus on educating health care researchers about the criteria for evaluating qualitative research from within the appropriate theoretical and methodological framework. Although the ideas posited here suggest there may be a connection between how quality is defined and the kind of work published or funded, this assumption is worthy of empirical examination. In addition, the field needs to reflect on the value of qualitative health care research and consider whether we have the space and models for adequately reporting interpretive research in our medical journals.

View this table:
Table 1.

Common Paradigms in Health Care Research

View this table:
Table 2.

Verification Techniques Used in Qualitative Research

Acknowledgments

We are indebted to Mary Dixon-Woods, PhD, for her insightful comments on earlier versions of this work.

Footnotes

  • Received for publication April 15, 2007.
  • Revision received November 7, 2007.
  • Accepted for publication November 14, 2007.

REFERENCES

  1. ↵
    Cicourel A. Method and Measurement in Sociology. New York, NY: Free Press; 1964.
  2. ↵
    Schutz A. Collected Papers. Vol 1. The Hague: Martinus Nijhoff; 1962.
  3. ↵
    Schutz A. Collected Papers. Vol 2. The Hague: Martinus Nijhoff; 1962.
  4. ↵
    Garfinkel H. Studies in Ethnomethodology. Cambridge, MA: Polity Press; 1967.
  5. ↵
    Feinstein AR. Additional basic approaches in clinical research. Clin Res. 1985;33(2 Pt 1):111–114.
    OpenUrlPubMed
  6. ↵
    Meinert CL. Clinical Trials: Design, Conduct and Analysis. New York, NY: Oxford University Press; 1986.
  7. ↵
    Goffman E. Asylums. New York, NY: Doubleday Anchor; 1961.
  8. Glaser B, Strauss A. Awareness of Dying. New York, NY: Aldine; 1965.
  9. Garfinkel H. ‘Good’ Organizational Reasons for ‘Bad’ Clinic Records. Studies in Ethnomethodology. Cambridge, MA: Polity Press; 1967:186–207.
  10. Mishler E. Discourse and meaning in medical interviews. Research grant proposal submitted to the Medical Founcation, Inc.; 1979.
  11. Waitzkin H. Medicine, superstructure and micropolitics. Soc Sci Med [Med Psychol Med Sociol]. 1979;13A(6):601–609.
    OpenUrl
  12. Kleinman A. Culture, and illness: a question of models. Cult Med Psychiatry. 1977;1(3):229–231.
    OpenUrlCrossRefPubMed
  13. Kleinman AM. Some issues for a comparative study of medical healing. Int J Soc Psychiatry. 1973;19(3):159–163.
    OpenUrlFREE Full Text
  14. ↵
    Kleinman AM. Toward a comparative study of medical systems: an integrated approach to the study of the relationship of medicine and culture. Sci Med Man. 1973;1(1):55–65.
    OpenUrlPubMed
  15. ↵
    Pope C, Mays N, eds. Qualitative Research in Health Care. 2nd ed. London: BMJ Books; 2000.
  16. Crabtree BF, Miller WL, eds. Doing Qualitative Research. Thousand Oaks, CA: Sage Publications; 1992.
  17. ↵
    Crabtree BF, Miller WL, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publication; 1999.
  18. ↵
    Holloway I. Basic Concepts for Qualitative Research. Oxford: Black-well Science; 1997.
  19. ↵
    Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001;358(9280):483–488.
    OpenUrlCrossRefPubMed
  20. ↵
    Asbury J. Overview of focus group research. Qual Health Res. 1995;5(4):414–420.
    OpenUrlAbstract/FREE Full Text
  21. Freeman T. ‘Best practice’ in focus group research: making sense of different views. J Adv Nurs. 2006;56(5):491–497.
    OpenUrlCrossRefPubMed
  22. ↵
    Kidd PS, Parshall MB. Getting the focus and the group: enhancing analytical rigor in focus group research. Qual Health Res. 2000;10(3):293–308.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    Sim J. Collecting and analysing qualitative data: issues raised by the focus group. J Adv Nurs. 1998;28(2):345–352.
    OpenUrlCrossRefPubMed
  24. ↵
    Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–112.
    OpenUrlCrossRefPubMed
  25. ↵
    Kuzel AJ. Naturalistic inquiry: An appropriate model for family medicine. Fam Med. 1986;18(6):369–374.
    OpenUrlPubMed
  26. ↵
    Mays N, Pope C. Qualitative research: observational methods in health care settings. BMJ. 1995;311(6998):182–184.
    OpenUrlFREE Full Text
  27. ↵
    Savage J. Ethnography and health care. BMJ. 2000;321(7273): 1400–1402.
    OpenUrlFREE Full Text
  28. ↵
    Britten N. Qualitative research: qualitative interviews in medical research. BMJ. 1995;311(6999):251–253.
    OpenUrlFREE Full Text
  29. Bauman LJ, Greenberg Adair E. The use of ethnographic interviewing to inform questionnaire construction. Health Educ Q. 1992;19(1):9–23.
    OpenUrlCrossRefPubMed
  30. Griffiths P, Gossop M, Powis B, Strang J. Researching hidden poulations of drug users by privileged access interviewers: methodological and practical issues. Addiction. 1993;88(12):1617–1626.
    OpenUrlCrossRefPubMed
  31. Hoddinott P, Roisin P. Qualitative research interviewing by general practitioners. A personal view of the opportunitites and pitfalls. Fam Pract. 1997;14(4):307–312.
    OpenUrlAbstract/FREE Full Text
  32. ↵
    Nunkoosing K. The problem with interviews. Qual Health Res. 2005;15(5):698–706.
    OpenUrlAbstract/FREE Full Text
  33. ↵
    Atkinson P. Narrative turn or blind alley? Qual Health Res. 1997;7(3):325–344.
    OpenUrlAbstract/FREE Full Text
  34. ↵
    Kleinman A. The Illness Narratives. New York, NY: Basic Books; 1998.
  35. ↵
    Mishler EG. The Discourse of Medicine: Dialectics of Medical Interviews. Norwood, NJ: Ablex; 1984.
  36. Traynor M. Discourse analysis: theoretical and historical overview and review of papers in the Journal of Advanced Nursing 1996–2004. J Adv Nurs. 2006;54(1):62–72.
    OpenUrlCrossRefPubMed
  37. ↵
    Heritage J, Maynard DW, eds. Communication in Medical Care. Cambridge: Cambridge University Press; 2006.
  38. ↵
    Crabtree B, Miller W. Researching practice settings: a case study approach. In: Crabtree B, Miller W, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications; 1999:293–312.
  39. ↵
    Yin RK. Enhancing the quality of case studies in health services research. Health Serv Res. 1999;34(5 Pt 2):1209–1224.
    OpenUrlPubMed
  40. ↵
    Hugentobler MK, Israel BA, Schurman SJ. An action research approach to workplace health: Integrating methods. Health Educ Q. 1992;19(1):55–76.
    OpenUrlCrossRefPubMed
  41. ↵
    Meyer J. Using qualitative methods in health-related action research. In: Pope C, Mays N, eds. Qualitative Research in Health Care. 2nd ed. London: BMJ Books; 2000:59–74.
  42. ↵
    Kuzel A. Sampling in qualitative inquiry. In: Crabtree B, Miller W, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications; 1999:33–45.
  43. Morse J. Strategies for sampling. In: Morse J, ed. Qualitative Nursing Research: A Contemporary Dialogue. Newbury Park, CA: Sage Publications; 1991:127–145.
  44. Sandelowski M. Sample size in qualitative research. Res Nurs Health. 1995;18(2):179–183.
    OpenUrlCrossRefPubMed
  45. ↵
    Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999;34(5):1189–1208.
    OpenUrlPubMed
  46. Malterud K. The art and science of clinical knowledge: evidence beyond measures and numbers. Lancet. 2001;358(9279):397–400.
    OpenUrlCrossRefPubMed
  47. Morse JM. Qualitative methods: The state of the art. Qual Health Res. 1999;9(3):393–406.
    OpenUrlAbstract/FREE Full Text
  48. Miller WL, Crabtree BF. Qualitative analysis: how to begin making sense. Fam Pract Res J. 1994;14(3):289–297.
    OpenUrlPubMed
  49. ↵
    Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ. 2000;320(7227):114–116.
    OpenUrlFREE Full Text
  50. ↵
    Morgan DL. Practical stategies for combining qualitative and quantitative methods: applications to health research. Qual Health Res. 1998;8(3):362–376.
    OpenUrlAbstract/FREE Full Text
  51. ↵
    Back AL, Starks H, Hsu C, Gordon JR, Bharucha A, Pearlman RA. Clinician-patient interactions about requests for physician-assisted suicide: a patient and family view. Arch Intern Med. 2002;162(11):1257–1265.
    OpenUrlCrossRefPubMed
  52. England M, Tripp-Reimer T. Imminent concerns of filial caregivers reporting recent experiences of crisis. Int J Aging Hum Dev. 2003;56(1):67–88.
    OpenUrlAbstract/FREE Full Text
  53. Gubrium JF, Rittman MR, Williams C, Young ME, Boylstein CA. Benchmarking as everyday functional assessment in stroke recovery. J Gerontol B Psychol Sci Soc Sci. 2003;58(4):S203–S211.
    OpenUrlAbstract/FREE Full Text
  54. ↵
    Wetle T, Shield R, Teno J, Miller SC, Welch L. Family perspectives on end-of-life care experiences in nursing homes. Gerontologist. 2005;45(5):642–650.
    OpenUrlAbstract/FREE Full Text
  55. ↵
    Duncan MT, Morgan DL. Sharing the caring: family caregivers’ views of their relationships with nursing home staff. Gerontologist. 1994;34(2):235–244.
    OpenUrlAbstract/FREE Full Text
  56. Irwin K, Bertrand J, Mibandumba N, et al. Knowledge, attitudes and beliefs about HIV infection and AIDS among healthy factory workers and their wives, Kinshasa, Zaire. Soc Sci Med. 1991;32(8):917–930.
    OpenUrlCrossRefPubMed
  57. ↵
    Bradley EH, McGraw SA, Curry L, et al. Expanding the Andersen model: the role of psychosocial factors in long-term care use. Health Serv Res. 2002;37(5):1221–1242.
    OpenUrlCrossRefPubMed
  58. ↵
    Jervis LL. The pollution of incontinence and the dirty work of care-giving in a U.S. nursing home. Med Anthropol Q. 2001;15(1):84–99.
    OpenUrlCrossRefPubMed
  59. Robinson JD. Closing medical encounters: two physician practices and their implications for the expression of patients’ unstated concerns. Soc Sci Med. 2001;53(5):639–656.
    OpenUrlCrossRefPubMed
  60. ↵
    Ware NC, Lachicotte WS, Kirschner SR, Cortes DE, Good BJ. Clinician experiences of managed mental health care: a rereading of the threat. Med Anthropol Q. 2000;14(1):3–27.
    OpenUrlCrossRefPubMed
  61. ↵
    Wittink MN, Barg FK, Gallo JJ. Unwritten rules of talking to doctors about depression: integrating qualitative and quantitative methods. Ann Fam Med. 2006;4(4):302–309.
    OpenUrlAbstract/FREE Full Text
  62. Rabago D, Barrett B, Marchand L, Maberry R, Mundt M. Qualitative aspects of nasal irrigation use by patients with chronic sinus disease in a multimethod study. Ann Fam Med. 2006;4(4):295–301.
    OpenUrlAbstract/FREE Full Text
  63. ↵
    Ahles TA, Wasson JH, Seville JL, et al. A controlled trial of methods for managing pain in primary care patients with or without co-occurring psychosocial problems. Ann Fam Med. 2006;4(4):341–350.
    OpenUrlAbstract/FREE Full Text
  64. ↵
    Reed J, Payton VR. Focus groups: issues of analysis and interpretation. J Adv Nurs. 1997;26(4):765–771.
    OpenUrlCrossRefPubMed
  65. ↵
    Webb C, Kevern J. Focus groups as a research method: a critique of some aspects of their use in nursing research. J Adv Nurs. 2001;33(6):798–805.
    OpenUrlCrossRefPubMed
  66. ↵
    Waitzkin H. On studying the discourse of medical encounters. A critique of quantitative and qualitative methods and a proposal for reasonable compromise. Med Care. 1990;28(6):473–488.
    OpenUrlCrossRefPubMed
  67. ↵
    Mishler E. Validation in inquiry-guided research: The role of exemplars in narrative studies. Harv Educ Rev. 1990;60(4):415–442.
    OpenUrl
  68. ↵
    Bailey PH. Assuring quality in narrative analysis. West J Nurs Res. 1996;18(2):186–194.
    OpenUrlAbstract/FREE Full Text
  69. ↵
    Angen MJ. Evaluating interpretive inquiry: reviewing the validity debate and opening the dialogue. Qual Health Res. 2000;10(3):378–395.
    OpenUrlAbstract/FREE Full Text
  70. ↵
    Britten N, Jones R, Murphy E, Stacy R. Qualitative research methods in general practice and primary care. Fam Pract. 1995;12(1):104–114.
    OpenUrlFREE Full Text
  71. ↵
    Cutcliffe JR, McKenna HP. Establishing the credibility of qualitative research findings: the plot thickens. J Adv Nurs. 1999;30(2):374–380.
    OpenUrlCrossRefPubMed
  72. ↵
    Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract. 2003;9(2):187–194.
    OpenUrlCrossRefPubMed
  73. ↵
    Elder NC, Miller WL. Reading and evaluating qualitative research studies. J Fam Pract. 1995;41(3):279–285.
    OpenUrlPubMed
  74. ↵
    Fitzpatrick R, Boulton M. Qualitative methods for assessing health care. Qual Health Care. 1994;3(2):107–113.
    OpenUrlFREE Full Text
  75. ↵
    Frankel RM, Devers K. Qualitative research: a consumer’s guide. Educ Health (Abingdon). 2000;13(1):113–123.
    OpenUrlCrossRefPubMed
  76. ↵
    Giacomini MK, Cook DJ. Users’ guides to the medical literature: XXIII. Qualitative research in health care B. What are the results and how do they help me care for my patients? Evidence-Based Medicine Working Group. JAMA. 2000;284(4):478–482.
    OpenUrlCrossRefPubMed
  77. ↵
    Hall JM, Stevens PE. Rigor in feminist research. ANS Adv Nurs Sci. 1991;13(3):16–29.
    OpenUrlCrossRefPubMed
  78. ↵
    Hamberg K, Johansson E, Lindgren G, Westman G. Scientific rigour in qualitative research--examples from a study of women’s health in family practice. Fam Pract. 1994;11(2):176–181.
    OpenUrlAbstract/FREE Full Text
  79. ↵
    Hinds PS, Scandrett-Hibden S, McAulay LS. Further assessment of a method to estimate reliability and validity of qualitative research findings. J Adv Nurs. 1990;15(4):430–435.
    OpenUrlPubMed
  80. ↵
    Inui TS, Frankel RM. Evaluating the quality of qualitative research: a proposal pro tem. J Gen Intern Med. 1991;6(5):485–486.
    OpenUrlPubMed
  81. ↵
    Koch T. Establishing rigour in qualitative research: the decision trail. 1993. [discussion 101–103]. J Adv Nurs. 2006;53(1):91–100.
    OpenUrlCrossRefPubMed
  82. ↵
    Koch T, Harrington A. Reconceptualizing rigour: the case for reflexivity. J Adv Nurs. 1998;28(4):882–890.
    OpenUrlCrossRefPubMed
  83. ↵
    Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther. 1991;45(3):214–222.
    OpenUrlCrossRefPubMed
  84. ↵
    Kuzel AJ, Engel JD, Addison RB, Bogdewic SP. Desirable features of qualitative research. Fam Pract Res J. 1994;14(4):369–378.
    OpenUrlPubMed
  85. ↵
    Engel JD, Kuzel AJ. On the idea of what constitutes good qualitative inquiry. Qual Health Res. 1992;2(4):504–510.
    OpenUrlFREE Full Text
  86. ↵
    Maxwell J. Understanding validity in qualitative research. Harv Educ Rev. 1992;62(3):279–300.
    OpenUrl
  87. ↵
    Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ. 2000;320(7226):50–52.
    OpenUrlFREE Full Text
  88. ↵
    Morse J, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability and validity in qualitative research. Intl J Qual Meth. 2002;1(2):1–19.
    OpenUrl
  89. ↵
    Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res. 1998;8(3):341–351.
    OpenUrlAbstract/FREE Full Text
  90. ↵
    Sandelowski M. The problem of rigor in qualitative research. ANS Adv Nurs Sci. 1986;8(3):27–37.
    OpenUrlCrossRefPubMed
  91. ↵
    Sandelowski M. Rigor or rigor mortis: the problem of rigor in qualitative research revisited. ANS Adv Nurs Sci. 1993;16(2):1–8.
    OpenUrlPubMed
  92. Seale C, Silverman D. Ensuring rigour in qualitative research. Eur J Public Health. 1997;7(4):379–384.
    OpenUrlAbstract/FREE Full Text
  93. ↵
    Shortell SM. The emergence of qualitative methods in health services research. Health Serv Res. 1999;34(5 Pt 2):1083–1090.
    OpenUrlPubMed
  94. ↵
    Yardley L. Dilemmas in qualitative health research. Psychol Health. 2000;15:215–228.
    OpenUrlCrossRef
  95. ↵
    Altheide D, Johnson J. Criteria for assessing interpretive validity in qualitative research. In: Denzin N, Lincoln Y, eds. Handbook of Qualitative Research. Thousand Oaks, CA: Sage Publications; 1994:485–499.
  96. ↵
    Creswell J. Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Thousand Oaks, CA: Sage Publications; 1988.
  97. Frankel R. Standards of qualitative research. In: Crabtree B, Miller WL, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications; 1999:341.
  98. ↵
    Hammersley M. Reading Ethnographic Research. New York, NY: Longman; 1990.
  99. Kirk J, Miller M. Reliability and Validity in Qualitative Research. Vol 1. Newbury Park, CA: Sage Publications; 1986.
  100. ↵
    Kuzel A, Like R. Standards for trustworthiness in qualitative studies in primary care. In: Norton P, Stewart M, Tudiver F, Bass M, Dunn E, eds. Primary Care Research Traditional and Innovative Approaches. Vol 1. Newbury Park, CA: Sage Publications; 1991:138–158.
  101. ↵
    Leninger M. Importance and use of ethnomethods: ethnography and ethnonursing research. Recent Advances in Nursing.1987;17: 12–36.
    OpenUrl
  102. Leninger M. Evaluation criteria and critque of qualitative research studies. In: Morse JM, ed. Critical Issues in Qualitative Research Methods. Thousand Oaks, CA: Sage Publications; 1994:95–115.
  103. ↵
    Lincoln Y, Guba E. Naturalistic Inquiry. Newbury Park, CA: Sage Publications; 1985.
  104. Marshall C, Rossman G. Designing Qualitative Research. 3rd ed. Thousand Oaks, CA: Sage Publications; 1999.
  105. ↵
    Mays N, Pope C. Quality in qualitative health research. In: Mays N, Pope C, eds. Qualitative Research in Health Care. London: BMJ Books; 2000:89–102.
  106. ↵
    Miles M, Huberman A. Qualitative Data Analysis: An Expanded Sourcebook. 2nd ed. Thousand Oaks, CA: Sage Publications; 1994.
  107. ↵
    Munhall P. Nursing Research: A Qualitative Perspective. 3rd ed. Boston, MA: Jones and Bartlett Publishers; 2001.
  108. ↵
    Patton M. Qualitative Evaluation and Research Methods. 2nd ed. Newbury Park, CA: Sage Publications; 1990.
  109. ↵
    Seale C. The Quality of Qualitative Research. Thousand Oaks, CA: Sage Publications; 1999.
  110. ↵
    Silverman D. Interpreting Qualitative Data Methods for Analyzing Talk, Text and Interaction. 2nd ed. Thousand Oaks, CA: Sage Publications; 2001.
  111. ↵
    Avis M. Valid arguments? A consideration of the concept of validity in establishing the credibility of research findings. J Adv Nurs. 1995;22(6):1203–1209.
    OpenUrlPubMed
  112. ↵
    Barbour RS. The newfound credibility of qualitative research? Tales of technical essentialism and co-option. Qual Health Res. 2003;13(7):1019–1027.
    OpenUrlAbstract/FREE Full Text
  113. Barbour RS, Barbour M. Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract. 2003;9(2):179–186.
    OpenUrlCrossRefPubMed
  114. ↵
    Beck CT. Qualitative research: the evaluation of its credibility, fittingness, and auditability. West J Nurs Res. 1993;15(2):263–266.
    OpenUrlFREE Full Text
  115. ↵
    Boulton M, Fitzpatrick R, Swinburn C. Qualitative research in health care: II. A structured review and evaluation of studies. J Eval Clin Pract. 1996;2(3):171–179.
    OpenUrlCrossRefPubMed
  116. Bryman A. Quantity and Quality in Social Research. Vol 18. New York, NY: Routledge; 1988.
  117. Chapple A, Rogers A. Explicit guidelines for qualitative research: a step in the right direction, a defence of the ‘soft’ option, or a form of sociological imperialism? Fam Pract. 1998;15(6):556–561.
    OpenUrlAbstract/FREE Full Text
  118. ↵
    Devers KJ. How will we know “good” qualitative research when we see it? Beginning the dialogue in health services research. Health Serv Res. 1999;34(5 Pt 2):1153–1188.
    OpenUrlPubMed
  119. Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitative research. Qual Saf Health Care. 2004;13(3):223–225.
    OpenUrlAbstract/FREE Full Text
  120. Dingwall R, Murphy E, Watson P, Greatbatch D, Parker S. Catching goldfish: quality in qualitative research. J Health Serv Res Policy. 1998;3(3):167–172.
    OpenUrlPubMed
  121. Emden C, Sandelowski M. The good, the bad and the relative, Part one: Conceptions of goodness in qualitative research. Int J Nurs Pract. 1998;4(4):206–212.
    OpenUrlCrossRefPubMed
  122. Emden C, Sandelowski M. The good, the bad and the relative, Part Two: Goodness and the criterion problem in qualitative research. Int J Nurs Pract. 1999;5(1):2–7.
    OpenUrlCrossRefPubMed
  123. ↵
    Harding G, Gantley M. Qualitative methods: beyond the cookbook. Fam Pract. 1998;15(1):76–79.
    OpenUrlAbstract/FREE Full Text
  124. ↵
    Johnson M, Long T, White A. Arguments for ‘British Pluralism’ in qualitative health research. J Adv Nurs. 2001;33(2):243–249.
    OpenUrlCrossRefPubMed
  125. ↵
    Morse JM. Myth 93: Reliability and validity are not relevant to qualitative inquiry. Qual Health Res. 1999;9(6):717–719.
    OpenUrlFREE Full Text
  126. Morse JM. considering the “peer” in peer review. Qual Health Res. 2002;12(5):579–580.
    OpenUrlFREE Full Text
  127. ↵
    Morse JM. The significance of standards. Qual Health Res. 2003;13(9):1187–1188.
    OpenUrlFREE Full Text
  128. Nolan M, Behi R. Validity: a concept at the heart of research. Br J Nurs. 1995;4(9):530–533.
    OpenUrlPubMed
  129. ↵
    Nolan M, Behi R. Alternative approaches to establishing reliability and validity. Br J Nurs. May 25–Jun 7 1995;4(10):587–590.
    OpenUrlCrossRefPubMed
  130. ↵
    Nolan M, Behi R. Triangulation: the best of all worlds? Br J Nurs. Jul 27–Aug 9 1995;4(14):829–832.
    OpenUrlPubMed
  131. Peck E, Secker J. Quality criteria for qualitative research: Does context make a difference. Qual Health Res. 1999;9(4):552–558.
    OpenUrlAbstract/FREE Full Text
  132. ↵
    Poses RM, Isen AM. Qualitative research in medicine and health care: questions and controversy. J Gen Intern Med. 1998;13(1):32–38.
    OpenUrlCrossRefPubMed
  133. ↵
    Poses RM, Levitt NJ. Qualitative research in health care. Antirealism is an excuse for sloppy work. BMJ. 2000;320(7251):1729–1730.
    OpenUrlFREE Full Text
  134. Ratcliffe JW, Gonzalez-del-Valle A. Rigor in health-related research: toward an expanded conceptualization. Int J Health Serv. 1988;18(3):361–392.
    OpenUrlPubMed
  135. ↵
    Rolfe G. Validity, trustworthiness and rigour: quality and the idea of qualitative research. J Adv Nurs. 2006;53(3):304–310.
    OpenUrlCrossRefPubMed
  136. ↵
    Sandelowski M, Barroso J. Reading qualitative studies. Intl J Qual Meth. 2002;1(1):1–47.
    OpenUrl
  137. ↵
    Sparkes AC. Myth 94: Qualitative health researchers will agree about validity. Qual Health Res. 2001;11(4):538–552.
    OpenUrlAbstract/FREE Full Text
  138. Ward MM. Study design in qualitative research: a guide to assessing quality. J Gen Intern Med. 1993;8(2):107–109.
    OpenUrlCrossRefPubMed
  139. ↵
    Whittemore R, Chase S, Mandle C. Validity in qualitative research. Qual Health Res. 2001;11(4):522–537.
    OpenUrlAbstract/FREE Full Text
  140. ↵
    Deetz S. Differences in approaches to organizational science: Rethinking Burell and Morgan and their legacy. Organ Sci. 1996;7(2):191–207.
    OpenUrlCrossRef
  141. ↵
    Leininger M. Ethnomethods: the philosophic and epistemic bases to explicate transcultural nursing knowledge. J Transcult Nurs. 1990;1(2):40–51.
    OpenUrlFREE Full Text
  142. ↵
    Cook T, Campbell D. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin Company; 1979.
  143. ↵
    Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115–1117.
    OpenUrlFREE Full Text
  144. ↵
    Lincoln Y. Emerging criteria for quality in qualitative and interpretive research [Keynote address]. Annual meeting, American Educational Research Association. San Francisco, CA; 1995.
  145. ↵
    Dixon-Woods M, Shaw R, Agarwal S, Smith J. The problem of appraising qualitative research. J Health Serv Res Policy. 2007;12(1):42–47.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Related Articles

Cited By...

More in this TOC Section

Similar Articles

Subjects