Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
EditorialEditorials

Publishing Multimethod Research

Kurt C. Stange, Benjamin F. Crabtree and William L. Miller
The Annals of Family Medicine July 2006, 4 (4) 292-294; DOI: https://doi.org/10.1370/afm.615
Kurt C. Stange
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Benjamin F. Crabtree
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
William L. Miller
MD, MA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • eLetters
  • Info & Metrics
  • PDF
Loading

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article
Compose eLetter

More information about text formats

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Statement of Competing Interests
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Vertical Tabs

Jump to comment:

  • Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    William L Miller
    Published on: 25 October 2006
  • NAPCRG Small Group Reflections on Mixed Methods
    Benjamin F Crabtree
    Published on: 25 October 2006
  • Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    Deborah J Cohen
    Published on: 23 October 2006
  • Launching a Discussion from a Workshop on Publishing Multimethod Research
    Kurt C. Stange
    Published on: 22 October 2006
  • The challenges of multimethod research.
    Jeevan P Marasinghe
    Published on: 25 September 2006
  • Published on: (25 October 2006)
    Page navigation anchor for Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    • William L Miller, Allentown, USA

    I am another facilitator from the Workshop on publishing mixed methods' research at this year's annual NAPCRG meeting (October, 2006). Our group unanimously endorsed the importance of finding better ways to publish mixed methods' research findings that preserves their wholeness and the benefits of their synergy.

    Six problem areas were identified: 1. Need for more journals willing and able to publish mixed metho...

    Show More

    I am another facilitator from the Workshop on publishing mixed methods' research at this year's annual NAPCRG meeting (October, 2006). Our group unanimously endorsed the importance of finding better ways to publish mixed methods' research findings that preserves their wholeness and the benefits of their synergy.

    Six problem areas were identified: 1. Need for more journals willing and able to publish mixed methods' studies. 2. Need for more reviewers skilled at evaluating mixed methods' studies. 3. Investigators often publish in multiple journals because of their perceived need for multiple publications as a promotions and tenure requirement. 4. Concern about journal space limitations. 5. Challenge of the time lag that is often created when there is a complex qualitative analysis that takes much longer than the quantitative one. 6. Need for more clarity regarding the intent of a possible third "synthesis" paper to accompany the quantitative and qualitative manuscripts.

    Five proposals for addressing these problems were also identified: 1. Encourage the Annals editors to promote mixed methods' publication among other primary care journal editors. 2. Encourage NAPCRG to help develop additional mixed methods' reviewers. 3. Publish quantitative work first, when appropriate, and then publish the qualitative work along with an integrative manuscript that links back to the earlier quantitative publication. 4. Take advantage of emerging on-line publication technologies that may help with the space limitation problems. 5. Several options for the third "synthesis" paper were shared.

    - A reflective paper by the combined research team that highlights anomalies, new understandings, new directions, and lessons learned as the mixed methods' results are compared.

    - A re-look at and analysis of the data given the new insights from the mixed methods' work.

    - A theory paper that explores the theoretical implications of the combined findings.

    - An independently written editorial.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (25 October 2006)
    Page navigation anchor for NAPCRG Small Group Reflections on Mixed Methods
    NAPCRG Small Group Reflections on Mixed Methods
    • Benjamin F Crabtree, Somerset, NJ USA

    Like Deborah Cohen and Kurt Stange who recently posted track comments based on their participation in the NAPCRG workshop on publishing mixed methods, I also “hosted” a break-out group of approximately 10 researchers. We took our chairs outside and met on the veranda to explore participants’ experiences with publishing mixed methods. Most of the participants were experienced researchers who had considerable experience i...

    Show More

    Like Deborah Cohen and Kurt Stange who recently posted track comments based on their participation in the NAPCRG workshop on publishing mixed methods, I also “hosted” a break-out group of approximately 10 researchers. We took our chairs outside and met on the veranda to explore participants’ experiences with publishing mixed methods. Most of the participants were experienced researchers who had considerable experience in publishing both qualitative and quantitative research manuscripts. Participants had earlier listened to the overview of publishing options covered in this Annals of Family Medicine editorial and sensed that ideally mixed methods studies would be published in a single manuscript, although they also like the option of publishing simultaneous manuscripts in the same journal like the Solberg/Hroscikoski combination published in July/August 2006 issue of Annals. Either of these options was seen as one of the ways for qualitative research to better reach clinicians who would not find the qualitative portions that are often published in social science journals.

    A number of challenges to publishing a single comprehensive manuscript quickly emerged. A junior faculty member felt that to be considered for promotion and tenure, it was important to get more publications, so separation of qualitative and quantitative manuscripts seemed more attractive. They also felt that this would require a team of researchers, but only the first author would be given credit for the publication. This was confirmed by a group member who serves on a P&T committee who emphasized that P&T committees look at numbers and impact values. The group also worried about which journals would publish manuscripts that are longer, with one group member noting that most journals are constrained by the print format, seeing the length being determined in the same way that older songs were constrained by the number of grooves on a record. While online options seemed like a potential solution, one of the participants noted that P&T committees wanted a hardcopy version, with online journals still not being valued.

    After deciding that reality seemed to favor the publication of simultaneous or consecutive qualitative and quantitative manuscripts, a number of ideas surfaced for enhancing the linkage of these. The group felt that the current approaches to alerting readers that there was a parallel manuscript could become stronger, particularly in journals with online versions. They felt it would be particularly helpful if the online version of an article that was previously published could be updated to include a “look forward” link to find other manuscripts from the same study that are published later than the initial publication (currently journals can put in a jump link to access other articles published in the same issue and could also put in a link to access past publications in the same journal).

    Other ideas that were considered included: putting in “pop-up” text boxes on online versions; adding in information in the “information for authors” that indicates that the journal encourages simultaneous publication; and having a summary section of bullet points for multiple manuscripts from a line of research.

    I would like to thank the members of the “outside veranda” group for such a free-ranging and stimulating conversation and invite them to post their impressions of publishing mixed methods.

    Competing interests:   Annals Associate Editor

    Show Less
    Competing Interests: None declared.
  • Published on: (23 October 2006)
    Page navigation anchor for Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    Re: Launching a Discussion from a Workshop on Publishing Multimethod Research
    • Deborah J Cohen, Somerset, NJ, USA

    I facilitated one of the groups at the Workshop on publishing mixed method research at NARCRG (October, 2006). The group was comprised of 10- 15 researchers, most of whom where experts in doing mixed method research. Below, I attempt to summarize their comments. Our discussions focused on barriers to publishing mixed method research in medical journals.

    Barrier: Many of the participants in the group found journ...

    Show More

    I facilitated one of the groups at the Workshop on publishing mixed method research at NARCRG (October, 2006). The group was comprised of 10- 15 researchers, most of whom where experts in doing mixed method research. Below, I attempt to summarize their comments. Our discussions focused on barriers to publishing mixed method research in medical journals.

    Barrier: Many of the participants in the group found journal limits regarding length of a manuscript prohibitive when attempting to publish mixed methods research. Authors have at least three options when publishing a mixed method study: publish qualitative only, publish quantitative only, and/or attempt to publish a mixed methods manuscript. With the latter, describing the methods and the findings is more complex and requires more space.

    Suggestion: Journals should have a more generous word or page limit for manuscripts that use and report on mixed methods research.

    Barrier: Doing and reporting studies that aim to synthesize qualitative research. I am not sure I will recount this discussion adequately and urge the expert in this area to log on to the Annals TRACK and do so. My understanding is the participant was interested in developing several different ways to conduct and publish qualitative meta-syntheses. She felt editors might not be sympathetic to such syntheses.

    Barrier: Medical audiences seem to be less interested in the description of theoretical and conceptual frameworks that inform the research being reported. When manuscripts include theory and conceptual development as well as empirical evidence then the manuscript is too long and complicated.

    Suggestions: It was suggested that one solution to this is to publish theory and philosophy separate from empirical findings. Most weren’t satisfied with this suggestion, remarking that it leads to publication of findings that are decontextualized.

    My notes: It seemed to me at this point that participants and others would benefit from examples of published mixed methods pieces. The examples from Social Science and Medicine are nice, but this journal has a more generous page limit and is read by researchers more than clinicians. There is the edition of Annals where the theme is multimethod research. This might be useful to hyperlink to the TRACK discussion.

    Question: How should one format a single paper that involves mixed methods?

    Responses from the group: There needs to be a research question that pulls together the analysis and results reported in the paper.

    Telling the story is important – how did the use of one method lead to the use of another method.

    Define the limits of your research question. One mixed method dataset may provide the data to publish multiple papers. Researchers need to define and limit the scope of their research questions so that a mixed method paper can be published that is relevant to the medical audience and small enough to meet stringent word limits.

    This advice doesn’t apply well when trying to publish the ‘big’ paper or central findings paper from a study with a multi-method design.

    When a big paper doesn’t lend itself to a smaller word limit, is there a way to keep the analysis and results whole?

    - can we use the web to do this (putting some methods information on the web or placing some supporting materials on the web)

    - can we develop a way to integrate multiple publications from one research study? For example, one large study has multiple publications – some quantitative, some qualitative, some mixed methods. A number of these publications may be complementary to the manuscript being reviewed. Is there a way to ask authors in advance of publication to list other publications from this project or grant that they’d like readers to be aware of and perhaps linked to? Then the website could offer hyperlinks to these other complementary publications.

    - It was pointed out that BMC puts the description of trial methods on the website. This might be an option when publishing mixed methods work (methods layout on web)

    One participant said (and I paraphrase) -- we need to recognize that the questions we ask in family medicine research aren’t questions that lend themselves to the use of single methods. Family medicine is about context and this requires mixed methods.

    In response to the concern about page limits that are too restrictive, there were others who did recognize that the process of revision and refining a manuscript to meet a tight page limit often got the author to recognize the kernel or message of a paper. There was the recognition that we can often be enamored with our own work (in all of its vastness) and winnowing it down to the key message for a particular audience is often a useful and rewarding process. It also seemed like this was a process that required outside collaborators because the author isn’t always in the best position to identify how to cut-down his or her own work.

    Participants also talked about the importance of having multiple quotes included in a manuscript to support a particular theme. Multiple quotes are important because they can show that an experience was ‘common’ occurred across a number of respondents and multiple quotes can be selected to bring out the nuances of an experience. However, participants reported being in a catch-22 type of situation with many manuscript and journal reviewers/editors. If they included too many quotes then the paper was reviewed as being too long. If they included only one quote, then the manuscript might be critiqued for being too opaque and not including enough support for claims or even questioning how wide-spread a theme reported in the manuscript might be. This seemed to be a balancing act.

    Many participants joked about the ways they found to manage these problems within the constraints of journal page limits and rules (rules which, by the way, lend themselves to the reporting of quantitative research). Many laughed about how small they could make the font if they put quotes in a table or put them on the web or in the appendix. [Note: tables are often not counted in page limits or word counts and it might be argued that tables lend themselves better to the reporting of quantitative data than qualitative data. Perhaps quotes shouldn’t be counted in page limits either???] Use of tables and the web for reporting qualitative data were typically not ideal options because the quote would often be removed or distanced from the point the author was making in the text.

    It seemed that under ideal circumstances, authors would carefully select multiple quotes that bring out the key features of an experience they are describing, and the manuscript and analysis would undergo several iterations in order to refine the key points, and authors would carefully select, identify and place quotes in the text. At the same time, these key qualitative data would not be relegated to a table, appendix or webpage. They would be part of the manuscript for readers to examine, evaluate and understand as they read the findings of the report.

    One participant asked this – if the manuscript is initially accepted do you have an intermediary to help you cut it down?

    One participant asked: Is this a point in evolution for mixed method research? Is there enough good mixed method research published that we can examine what has been reported in order to identify a format authors can follow to get to the ‘kernel?’ The comparison was made to consortium criteria for reporting RCTs. Could there be something analogous for reporting mixed methods research?

    Another participant reported a recent experience with JNCI (not sure this is correct journal) where he/she was given a word limit but the word limit did not include the methods section. There was some joking about this – one could find all kinds of ways to integrate qualitative data (quotes and the like) into the methods section.

    Participants talked about the many functions of publication. Publication in peer-reviewed journals is linked with advancement (promotion) as well as ability to get future grant funding. Publication is not necessarily connected to dissemination. Peer review is also associated with validation of the excellence of one’s research and the one’s excellence as a researcher. This was something the group thought about as they were discussing how and whether or not it was important to try to publish their mixed method work in places where clinicians (and even patients) might see it. This is an especially difficult decision if there is a tight page limit for peer reviewed journals or if these publications were not peer reviewed.

    One participant cautioned the others about attempting to publish two separate papers simultaneously in the same journal. The report (probably from personal experience) was that if they (editors) don’t like one of the papers they may turn both down. Also, it will be important for each paper to stand alone because both papers may not be reviewed by the same people.

    Others responded by saying – if publishing two papers simultaneously in the same journal is going to be an approach to publishing mixed method research then the opportunity needs to be ‘real’ – the journal needs to assign reviewers experienced in using and reviewing mixed method research and reviewers would need to read and review both manuscripts.

    Respectfully submitted on behalf of my group,

    Deborah Cohen

    PS - Here's a shameless plug for the qualitative website I'm working on - www.qualres.org. Please visit!!

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (22 October 2006)
    Page navigation anchor for Launching a Discussion from a Workshop on Publishing Multimethod Research
    Launching a Discussion from a Workshop on Publishing Multimethod Research
    • Kurt C. Stange, Cleveland, OH, USA

    On October 16, 2007, at the 34th Annual meeting of the North American Primary Care Research Group, the Annals editors led a workshop on publishing multimethod research. It was attended by approximately 50 participants from at least 6 different countries, the large majority of whom were experienced in performing mixed methods research. After Ben Crabtree and Will Miller gave an overview of multimethod research, Debbie C...

    Show More

    On October 16, 2007, at the 34th Annual meeting of the North American Primary Care Research Group, the Annals editors led a workshop on publishing multimethod research. It was attended by approximately 50 participants from at least 6 different countries, the large majority of whom were experienced in performing mixed methods research. After Ben Crabtree and Will Miller gave an overview of multimethod research, Debbie Cohen provided a new website with useful information on performing qualitative research: www.qualres.org.

    Workshop participants divided into 4 small groups to explore their experience with communicating the findings of multimethod research in ways that maximize the learning and cross-talk between the two methods. At the end, we invited participants to use the Annals online discussion to contribute their insights. We look forward to a lively discussion, and hope to be able to synthesize the resulting new ideas in the On TRACK feature in a future issue of the Annals.

    Below are my brief notes about ideas from my small group at the workshop. These are summarized as PROBLEMS related to publishing multimethod research, and potential SOLUTIONS.

    I look forward to hearing your additional ideas.

    Kurt Stange

    PROBLEMS • Word/space limit • Challenge of portraying exploratory work to a solution-oriented audience • Challenge of portraying complex and iterative - Methods (design & analysis) - Results - Interventions - Participatory approach • Different language and world views of many quantitative and qualitative reviewers & readers • Different audiences • Expert peer review for mixed methods • Timing (findings from quantitative and qualitative analyses that could inform each other often are not done at the same time.)

    SOLUTIONS • Quality work • Focusing the main message to draw readers in to being willing to read the complexity • Use hyperlinks for details - Write a very focused main report - Provide details online (in appendices or hot links to other sources) - Results • Use a logic model in a diagram to convey complex processes • Separate quantitative and qualitative papers with a 3rd paper that conveys integrative cross-cutting findings or new theory that develops as a result • Journal editors could try to come up with a consensus statement and/or templates for different approaches to integrating quantitative and qualitative methods. (Note the editorial on which this eletter is commenting is an attempt to do this.) - Methods (design & analysis) - Results

    Competing interests:   Annals Editor

    Show Less
    Competing Interests: None declared.
  • Published on: (25 September 2006)
    Page navigation anchor for The challenges of multimethod research.
    The challenges of multimethod research.
    • Jeevan P Marasinghe, Peradeniya,Sri Lanka.
    • Other Contributors:

    The arena of biomedical research has progressed with striking strides and with an exponential rate (1).Immense numbers of publications are released each day to the general reader and the clinician .Multimethod research is a further advancement of the field which involves integrating qualitative and quantitative approaches (2).They may be met with considerable skepticism since there seems to be a lack of integrity in publ...

    Show More

    The arena of biomedical research has progressed with striking strides and with an exponential rate (1).Immense numbers of publications are released each day to the general reader and the clinician .Multimethod research is a further advancement of the field which involves integrating qualitative and quantitative approaches (2).They may be met with considerable skepticism since there seems to be a lack of integrity in publishing multimethod research. Consequently the reader may find it difficult to trace the two ends of the story and can get confused after reading the separate publications.

    The editorial by Stange KC (3) has come out with certain solutions for this growing issue. Out of them, publishing an integrated single article that describes methods and findings seems to be more rational and practical than the other solutions since it would give the reader a satisfactory insight straightaway without the need to go through several journals or several articles in the same journal .It would be more user- friendly and would satisfy the busy general reader and the clinician. Relevant sections of the other articles and their conclusions can be attached in a separate window in an online version without harming the general arrangement of the publication. So the reader can open a window if necessary to get additional information of the studies which used the other methodology.

    Publishing concurrent or sequential qualitative and quantitative papers in the same journal also seems to be practicable. An overview of the contents of the issue at the beginning to highlight the connection of two articles is appreciated.

    Online discussion by means of rapid responses allows the reader to get an idea of thoughts from diverse individuals. But the effect of it on the growing issue of publishing mixed method research in a meaningful way is questionable. However the editorial (3) provides the reader an insight about the problems faced by publishing multimethod research and some practical solutions for it.

    References. (1)Zelen M.Biosatisticians,biostatistical science and the future. Stat Med.2006 Aug 23; [Epub ahead of print]

    (2)Stange KC,Miller WL,Crabtree BF,O’Connor PJ,Zyzanski SJ.Multimethod research: approaches for integrating qualitative and quantitative methods.J Gen Intern Med.1994 May;9(5):278-82.

    (3)Stange KC,Crabtree BF,Miller WL.Publishing multimethod research. Ann Fam Med.2006 Jul-Aug;4(4):292-4.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 4 (4)
The Annals of Family Medicine: 4 (4)
Vol. 4, Issue 4
1 Jul 2006
  • Table of Contents
  • Index by author
  • In Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Publishing Multimethod Research
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
4 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Publishing Multimethod Research
Kurt C. Stange, Benjamin F. Crabtree, William L. Miller
The Annals of Family Medicine Jul 2006, 4 (4) 292-294; DOI: 10.1370/afm.615

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
Publishing Multimethod Research
Kurt C. Stange, Benjamin F. Crabtree, William L. Miller
The Annals of Family Medicine Jul 2006, 4 (4) 292-294; DOI: 10.1370/afm.615
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • THE POTENTIAL AND THE PROBLEM
    • SOLUTIONS
    • Footnotes
    • REFERENCES
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays
  • Mixed Methods in Biomedical and Health Services Research
  • In this Issue: Mixed Methods and Diverse Perspectives
  • Google Scholar

More in this TOC Section

  • Thank You, Reviewers and Commenters
  • Recruiting, Educating, and Taking Primary Care to Rural Communities
  • Returning to a Patient-Centered Approach in the Management of Hypothyroidism
Show more Editorials

Similar Articles

Subjects

  • Methods:
    • Mixed methods
  • Other topics:
    • Research capacity building

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine