Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleMethodology

The 5 R’s: An Emerging Bold Standard for Conducting Relevant Research in a Changing World

C. J. Peek, Russell E. Glasgow, Kurt C. Stange, Lisa M. Klesges, E. Peyton Purcell and Rodger S. Kessler
The Annals of Family Medicine September 2014, 12 (5) 447-455; DOI: https://doi.org/10.1370/afm.1688
C. J. Peek
1Department of Family Medicine and Community Health, University of Minnesota Medical School, Minneapolis, Minnesota
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: cjpeek@umn.edu
Russell E. Glasgow
2Department of Family Medicine and Colorado Health Outcomes Program, University of Colorado, Denver, Colorado
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kurt C. Stange
3Department of Family Medicine & Community Health, Department of Epidemiology & Biostatistics, and Department of Sociology, Case Comprehensive Cancer Center, Cleveland Clinical & Translational Science Collaborative, Case Western Reserve University, Cleveland, Ohio
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lisa M. Klesges
4School of Public Health, University of Memphis, Memphis, Tennessee
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
E. Peyton Purcell
5Clinical Research Directorate/CMRP, SAIC-Frederick, Inc, Frederick National Laboratory for Cancer Research, Frederick, Maryland
MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rodger S. Kessler
6Department of Family Medicine and the Center for Clinical and Translational Science, University of Vermont College of Medicine, Burlington, Vermont
PhD, ABPP
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Article Figures & Data

Tables

  • Additional Files
    • View popup
    Table 1

    Examples of Projects Illustrating the R’s

    Example: Study/Project/Resource Title and Relevant R’sStudy/Project/Resource Details
    Particular studies and programs
    DIAMOND (Depression Improvement Across Minnesota–Offering a New Direction)
     1. Relevant to stakeholders
     2. Redefines rigor
     3. Replicability
    The DIAMOND initiative for depression in primary care was a statewide collaborative of practices and health plans accompanied by a separately funded NIMH research study using a stepped wedge/phased intervention design.15,25,39,40 Practices launched the DIAMOND care process in cohorts, 6 months apart, with baseline data collected for all. Outcomes that mattered to different stakeholders were compared before and after launch across the many practices launching at different times. Outcomes were tracked using both quantitative and qualitative measures, including clinical outcomes, health plan claims data, patient surveys, and practice leader surveys regarding implementation.
    An explicit balance of fidelity and adaptation to local situations—specifics that practices had to tailor for themselves (eg, choice of discipline for care managers, specific workflow for PHQ-9, type of data tracking system)—helped practices implement the DIAMOND intervention.41,42
    P4H (Prescription for Health)
     1. Relevant to stakeholders
     2. Recursive
     3. Redefines rigor
     4. Reports on resources
    P4H was an initiative of The Robert Wood Johnson Foundation (RWJF) with the Agency for Healthcare Research and Quality (AHRQ) to fund a collaboration of 17 PBRNs that developed and evaluated strategies to improve health behavior changes for multiple behaviors through linkage to community resources.43 Practices worked with researchers, and teams of researchers, and PBRN leaders worked with each other and with a cross-cutting research group to share evolving learning, and develop common measures and an evolving research agenda.44
    Using mixed quantitative and qualitative methods (including researcher diary data and interviews)45 and cost analyses, P4H showed that primary care practices have the ability to develop their linkages to connect patients with community resources46 to improve practice processes,47 health behavior counseling, and patient behavior change.48
    ¡Viva Bien!
     1. Relevant to stakeholders
     2. Reports on resources
     3. Replicability
    ¡Viva Bien!32,33 was a randomized trial that provided a clear description of methods, implementation costs for a diabetes self-management program, and estimates of costs to replicate the program under different conditions, calculating incremental costs per behavioral, biologic, and quality-of-life change. It discussed how to separate the costs of development and research from implementation, and how to conduct relatively straight-forward sensitivity analyses to estimate costs of replicating a program or policy under different conditions.
    MOHR (My Own Health Report)
     1. Relevant to stakeholders
     2. Rapid and recursive
     3. Redefines rigor
     4. Reports on resources
     5. Replicability
    MOHR23,49 is a pragmatic participatory trial in which diverse primary care practices implement the collection of patient-reported information and provide patients advice, goal setting, and counseling in response—with deliberate diversity of settings and populations to ensure greater generalizability of results. Practices, patients, funding agencies, and content experts were engaged throughout the study to take into account local resources and characteristics in design, implementation, evaluation, and dissemination.
    Core elements of the study protocol were identified, with local tailoring to ensure implementation was relevant to local culture and practice on issues such as workflows, eligible patients, when and where assessment would be completed, whether electronic or paper, and how clinicians would receive the feedback. The trial used mixed methods, including cost analyses.
    Research networks across studies
    PRC (Prevention Research Centers) of the Centers for Disease Control and Prevention
     1. Relevant to stakeholders
     2. Recursive
     3. Replicability
    PRC directs a national network of 37 academic research centers at public health or medical schools with a preventive medicine residency program, translating research results into policy and public health practice. Centers have capacity for community-based, participatory prevention research needed to drive community changes to prevent and control chronic disease.
    Research involves collaboration among partners bringing different expertise to the table, identifies research needs of partners, conducts research that builds on previous evidence for promising interventions, and recommends how interventions can be packaged for replication and adoption (http://www.cdc.gov/prc/index.htm).
    QUERI (Quality Enhancement Research Initiative)
     1. Relevant to stakeholders
     2. Rapid and recursive
     3. Redefines rigor
     4. Replicability
    QUERI is a Veterans Affairs initiative that brings together operations with research staff to address key gaps in quality and outcomes. It has contributed to remarkable and rapid improvements in the quality of care received by veterans across 10 conditions deemed high-risk or highly prevalent.
    This initiative uses a 6-step process to spot gaps in performance and to identify and implement interventions. QUERI studies and facilitates adoption of new treatments, tests, and models of care into routine clinical practice—feasibility, implementation, adoption, and impact (http://www.queri.research.va.gov/default.cfm).
    Research application tools and resources
    RTIPs (Research Tested Intervention Programs)
     1. Relevant to stakeholders
     2. Reports on resources
     3. Replicability
    RTIPs is a resource of the National Cancer Institute that provides information on the specific conditions under which each of their tested interventions has been evaluated and tools for addressing issues about applicability (http://rtips.cancer.gov/rtips).
    New features related to external validity using the RE-AIM framework are included to help users better determine the likely public health impact of a given program if replicated in their setting. RTIPs also reports on the resources required to implement these programs.
    PRECIS (Pragmatic Explanatory Continuum Indicator Summary)
     1. Relevant to stakeholders
     2. Redefines rigor
     3. Replicability
    PRECIS50 is a graphic representation of the extent to which a study is pragmatic (testing effect in usual conditions) vs explanatory (testing effect in ideal conditions) on 10 key dimensions.
    If used consistently, this tool could greatly help practitioners decide whether a study is likely to be reproducible in their setting and researchers to investigate the dimensions along which similarity is more vs less critical for replication.
    • NIMH = National Institute of Mental Health; PBRN = practice-based research network; PHQ-9 = 9-item Patient Health Questionnaire; RE-AIM = Reach Effectiveness–Adoption Implementation Maintenance.

    • View popup
    Table 2

    Questions to Apply the 5 R’s at Each Stage of the Research and Dissemination Process

    Stage of ResearchBold Standard 5 R’s
    Relevant to StakeholdersRapid and RecursiveRedefines RigorReports on ResourcesReplicability
    DesignEnd users of this research identified?
    Stakeholders who need to be involved identified?
    Plan in place to engage their perspectives?
    Plan in place to gather stakeholder questions and what is important to them?
    Rapid cycle measurement and assessment built into the design? How?
    Approach in place to allow early discoveries to shape the study?
    How is study systematic and pragmatic about concepts, measures, data collection procedures, and analysis plan?
    Multiple methods used? How?
    Internal and external validity balanced? How?
    Intervention costs (monetary and other) measured? How?
    A standard vocabulary for reporting on resources in place? What?
    Study designed to inform implementation and reinvention in different settings? How?
    Likely relevant settings for this research identified?
    ImplementationStakeholders involved in ongoing refinement? How?
    Changes they suggested along the way recorded?
    Changes suggested implemented? Which ones?
    Short-cycle learning taking place to refine design and measurement?
    Is learning influencing the study? How?
    Systematic approach being followed to concepts, tools, data collection, measures, procedures, analyses?
    Checks for bias and superfluous connections in place?
    Clear description of what is being done recorded?
    Cost data gathered on an ongoing basis?
    Using a consistent vocabulary for different kinds of costs?
    Contextual factors documented that are important to understanding what happened (and why) in the study setting?
    ReportingDiverse stakeholders involved in interpreting and reporting findings?
    Their different interpretations reported?
    Emergent findings shared on an ongoing basis throughout the study?
    Have adaptations made been reported?
    Study methods reported transparently and thoroughly?
    Reported how study checked for potential biases and superfluous connections?
    Reported how conclusions are justified by standards of evidence?
    Study reports useful cost data using a defined vocabulary for different kinds of costs?
    Estimates made for costs under different conditions?
    Contextual factors relevant to reinvention in new settings reported, including variation across settings or within settings?
    DisseminationTarget audiences, stakeholders, or likely users involved in next steps?
    Findings expressed in language and context that mean something to different stakeholders?
    Guidelines provided for adaptation and customization/tailoring for future use?Description included for how internal and external validity findings support wider use?Intervention cost data discussed as a factor in dissemination?Data-supported suggestions included about the contexts for which program or intervention is relevant or reproducible?
    • View popup
    Table 3

    Challenges and Changes for Routine Implementation of 5 R’s

    ChallengesChanges to Address Challenges
    Accelerating the pace and iterative nature of the research enterprise
    Decision-maker needs outpace current speed of review cycles: grant review; funding decision; IRB approval and modification processes
    Study implementation time frames
    Publication cycles not amenable to “just in time” decisions; slow review and release of findings (see more below on dissemination)
    Low priority assigned to designs that can speed research
    Harness stakeholder interest in timeliness to drive a cultural shift to shorten what is considered “rapid” or “timely” compared with present custom
    Implement a variety of technical changes to research processes already suggested in literature19,21,26,27,54
    Use rapid-cycle testing of hypotheses, allowing ineffective ideas to “fail fast” and successful innovations to spread quickly
    Link social media with traditional communications vehicles
    Expanding limited concepts of rigor (eg, preference for, confidence in, or insistence on certain designs such as RCTs) by:
    Funding agencies offering calls for proposals
    Grant application reviewers
    Researchers
    “Customers” of research (stakeholders who use the findings)
    Among all parties, build awareness of and comfort with a broader “palette” of research designs, so that research design is driven by the questions, rather than research questions driven by designs
    Use professional meetings/training events to more clearly articulate features, pros/cons of different designs—their appropriate or promising scope of application
    Ensuring a blend of research team skills and interests
    Skill and interest in stakeholder involvement in generating questions, articulating ultimate use of study findings, study design, implementation, reporting, and dissemination
    Awareness of and respect for political as well as scientific concerns of stakeholders such as policy makers
    Skill and comfort in building relationships with clinicians and clinics—consultative, cooperative, problem solving
    Experience and confidence with the broader “palette” of research designs, including rapid learning in real-world experiments
    Propose an enhanced “job description” for research teams—a checklist of skills, interests, and relationships required for specific studies
    Beyond essential methodologic, data-gathering, and analytic skills, include “softer” skills and methods such as shown in left column
    Build up those skills through examples, conferences, and training among both existing and new researchers
    Increasing clinician familiarity with being active research partners
    Negative experiences or preconceptions about feasibility or practical value of doing research in the practice
    Few or no current relationships with researchers
    Unfamiliarity of working with researchers to turn practice concerns and curiosity into researchable questions
    Unfamiliarity with building research data gathering into routine clinic systems rather than being an effortful “add on”
    Not connecting research with more familiar quality improvement, rapid-cycle learning
    Provide examples and assistance through professional venues and practice facilitation or technical assistance that help clinicians and researchers adjust mindset, methods, and interactions to create practical research partnerships along the lines described in the literature15,16,25
    Raising priority on collection and reporting on context and resources
    Limited researcher and reviewer expectation that data on resource use of interventions or on context information relevant to transportability or reinvention in new settings be gathered systematically or reported
    Space limitations and/or customary priorities in journals that reduce additional context and resource data reporting
    Adjust research announcements and grant review guidelines to ask for greater reporting on context and resources required; accompany by explanation of why
    For publication in limited space, consider other methods such as web supplements to access detailed context and resource use data if not in standard published article
    More powerfully bringing publication and dissemination to practical decision making
    Limited readiness to publish replications of key findings in original or new contexts or to publish negative results of replication
    Reaching those stakeholders who want to make research-based decisions at the time and place decisions are made
    Limited dissemination in publications or forms in which stakeholders are already engaged, knowing that different forms of publication/dissemination reach different stakeholders
    Publish replications (successful or not) in places where stakeholders will find them
    Reward researchers via funding and career paths for key replications, not only for new positive results
    Create a stakeholder map—which stakeholders need what information from the study, in what form, and where it is most likely to be read
    Create stakeholder-specific versions of core journal publications to increase reach of the information
    • IRB = institutional review board; RCT = randomized controlled trial.

Additional Files

  • Tables
  • In Brief

    The 5 R's: An Emerging Bold Standard for Conducting Relevant Research in a Changing World

    C.J. Peek , and colleagues

    Background The accelerated pressure for change in health care has created an exploding need for relevant and rapidly generated new information. The current slow and fragmented approach to research, however, often fails to address practical needs for decision making. This paper synthesizes several existing approaches and develops a new "5 R's" standard to guide research and help meet the changing needs of health care delivery.

    What This Study Found The 5Rs are intended to generate research that 1) is relevant to stakeholders, 2) is rapid and recursive in application, 3) redefines rigor, 4) reports on resources required, and 5) is replicable. The R's of the research process are mutually reinforcing and can be supported by training that fosters collaborative and reciprocal relationships among researchers, implementers, and other stakeholders.

    Implications

    • The 5R's can serve as a framework for discussion and adjustment of criteria for what is considered high-quality research. Consistent and bold application of this standard, the authors conclude, will increase the value, timeliness and applicability of the research enterprise.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 12 (5)
The Annals of Family Medicine: 12 (5)
Vol. 12, Issue 5
September/October 2014
  • Table of Contents
  • Index by author
  • Back Matter (PDF)
  • Front Matter (PDF)
  • The Issue in Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The 5 R’s: An Emerging Bold Standard for Conducting Relevant Research in a Changing World
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
4 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
The 5 R’s: An Emerging Bold Standard for Conducting Relevant Research in a Changing World
C. J. Peek, Russell E. Glasgow, Kurt C. Stange, Lisa M. Klesges, E. Peyton Purcell, Rodger S. Kessler
The Annals of Family Medicine Sep 2014, 12 (5) 447-455; DOI: 10.1370/afm.1688

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
The 5 R’s: An Emerging Bold Standard for Conducting Relevant Research in a Changing World
C. J. Peek, Russell E. Glasgow, Kurt C. Stange, Lisa M. Klesges, E. Peyton Purcell, Rodger S. Kessler
The Annals of Family Medicine Sep 2014, 12 (5) 447-455; DOI: 10.1370/afm.1688
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • THE NEED FOR RELEVANT RESEARCH IN A RAPIDLY CHANGING HEALTH CARE WORLD
    • THE 5 R’S OF HEALTH CARE DELIVERY RESEARCH
    • EXAMPLES
    • CONCLUSIONS
    • Footnotes
    • References
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • Cost-effectiveness of support for health professionals to implement physical activity promotion: a protocol for within-trial and modelled economic evaluations of the PROMOTE-PA effectiveness-implementation hybrid trial
  • Looking Back to Move Forward: Reflections of PBRN Directors
  • Harmonizing the Tripartite Mission in Academic Family Medicine: A Longitudinal Case Example
  • Getting started in research, redefined: five questions for clinically focused physicians in family medicine
  • Unfinished Business: The Role of Research in Family Medicine
  • Mapping Evidence of Patients Experiences in Integrated Care Settings: A Protocol for a Scoping Review
  • The Rise of Primary Care Physicians in the Provision of US Mental Health Care
  • How to Enter the Clinical Research Field
  • In This Issue: Raise the Gaze
  • Could 5R Research Help Achieve the Triple Aim?
  • Google Scholar

More in this TOC Section

  • Joint Display of Integrated Data Collection for Mixed Methods Research: An Illustration From a Pediatric Oncology Quality Improvement Study
  • Patient-Guided Tours: A Patient-Centered Methodology to Understand Patient Experiences of Health Care
  • Putting Evidence Into Practice: An Update on the US Preventive Services Task Force Methods for Developing Recommendations for Preventive Services
Show more Methodology

Similar Articles

Subjects

  • Methods:
    • Participatory / action research
  • Other research types:
    • Health policy
    • PBRN research
    • Translational research
  • Other topics:
    • Research capacity building

Keywords

  • research design
  • healthcare research
  • evaluation research
  • partnership research
  • rapid research

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine