Abstract
Primary care (PC) is a unique clinical specialty and research discipline with its own perspectives and methods. Research in this field uses varied research methods and study designs to investigate myriad topics. The diversity of PC presents challenges for reporting, and despite the proliferation of reporting guidelines, none focuses specifically on the needs of PC. The Consensus Reporting Items for Studies in Primary Care (CRISP) Checklist guides reporting of PC research to include the information needed by the diverse PC community, including practitioners, patients, and communities. CRISP complements current guidelines to enhance the reporting, dissemination, and application of PC research findings and results. Prior CRISP studies documented opportunities to improve research reporting in this field. Our surveys of the international, interdisciplinary, and interprofessional PC community identified essential items to include in PC research reports. A 2-round Delphi study identified a consensus list of items considered necessary. The CRISP Checklist contains 24 items that describe the research team, patients, study participants, health conditions, clinical encounters, care teams, interventions, study measures, settings of care, and implementation of findings/results in PC. Not every item applies to every study design or topic. The CRISP guidelines inform the design and reporting of (1) studies done by PC researchers, (2) studies done by other investigators in PC populations and settings, and (3) studies intended for application in PC practice. Improved reporting of the context of the clinical services and the process of research is critical to interpreting study findings/results and applying them to diverse populations and varied settings in PC.
Annals “Online First” article
- guidelines
- research report
- checklist
- primary care
- research
- consensus
- stakeholder participation
- Delphi studies
- research impact
- research design
- surveys and questionnaires
- authors
- editors
- reviewers
- article
- publishing
- journals
INTRODUCTION
Researchers, journals, and research users across many fields recognize the need to improve research reporting1,2 and have developed research reporting guidelines to assist researchers. Many guidelines have been widely adopted, with potential benefits including more effective dissemination, translation, implementation of new knowledge, and reduction of research waste.3 The EQUATOR (Enhancing the Quality and Transparency of Health Research) network catalogs a growing number of guidelines for reporting health research (https://www.equator-network.org). Still, the bulk of the 500-plus guidelines pertain specifically to select methods, disciplines, topics, or focused specialties.4-6 Primary care (PC) researchers use a variety of reporting guidelines to cover the breadth of their interests, methods, and topics; however, no published guideline focuses directly on PC’s defining features and perspectives.
Need for Reporting Guidelines for PC Research
Primary care is a distinct health care model7 that can improve patient and population health8 and has unique clinical perspectives, knowledge needs, and research questions. This field has developed distinct approaches emphasizing patient-centered and problem-oriented care of whole patients, multiple and chronic conditions, interdisciplinary teams, participatory models, mixed methods, synthesis, translation, and implementation.9 PC research engages many partners and serves many users. Investigators in this field work in interdisciplinary teams, using a broad array of research methods to investigate the entire palette of human health, illness, and care across various clinical and community settings.10 PC researchers use multiple reporting guidelines for specific study designs, such as CONSORT (Consolidated Standards of Reporting Trials)11 for trials, STROBE (Strengthening the Reporting of Observational Studies in Epidemiology)12 for observational studies, and COREQ (Consolidated Criteria for Reporting Qualitative Research)13 for qualitative research. These guidelines remain appropriate for many studies, but there is a need for additional information and context beyond that requested in method-centered guidelines, particularly to inform the implementation of research findings/results in a vast range of socioeconomic, cultural, and health system settings. Simply adding extensions to current guidelines will not meet the needs of PC research or its synthetic, multimethod, generalist research.
CRISP INITIATIVE
Aims and Scope
Consensus Reporting Items for Studies in Primary Care (CRISP) is an international, interprofessional, interdisciplinary initiative to help improve the reporting of PC research (https://www.crisp-pc.org). The goal is to improve the quality, usefulness, and dissemination of reports of PC research findings/results and aid their application to improve care and health outcomes for patients and communities.
The CRISP Working Group began with a scoping review of the literature14 and a formal assessment of current reporting practices and the needs of the multiple users of PC research.15,16 These included clinicians, researchers, editors, reviewers, educators, patients, study participants, communities, funders, and policy makers, each with needs for research communication. We purposefully surveyed people from diverse nations, languages, personal and professional backgrounds, professions, specialties, disciplines, and roles in research. At every stage, we included patients, community representatives, and study participants to be sure their voices were captured. We engaged all of these voices as experts in conducting PC research and communicating its results to optimize the communication, dissemination, and implementation of study findings and results.
We defined PC according to the 1996 report of the US Institute of Medicine7: “Primary care is the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”
We defined clinicians as physicians, other health professionals, and other primary care team members who deliver health services directly to patients face to face.
The target audience for the resultant CRISP guidelines includes everyone engaged in PC research (Table 1). This report summarizes the rationale, background research, development, and potential uses of the CRISP guidelines. See the CRISP Explanation and Examples Guide (Supplemental Appendix) for further details on each reporting item.
Development of the CRISP Guidelines
The CRISP Working Group completed a series of studies to assess current practices, needs, best practices, and potential for improving PC research reporting (Table 2).14-18,*
Literature Scoping Review
We conducted a scoping review to map the published literature on PC research reporting quality, strengths, weaknesses, recommendations, and efforts to improve reporting.14 Our search of 7 major databases for articles published in English during 2000-2020, supplemented by a secondary search of references and expert panel suggestions, yielded 2,847 unique titles, of which 126 underwent full-text review and 25 met predetermined inclusion criteria. All publications identified the need to improve reporting and recommended items to include in reports. Most cited the need for more detailed reporting on the context of study interventions, clinical settings, and health care systems. Most publications endorsed reporting guidelines and recognized the unique needs of PC research reporting.
Needs Assessment Surveys and Guideline Scan
We conducted a needs assessment of the international, interprofessional community of producers and users of PC research.15 Our online survey, conducted during 2018-2019, yielded 255 respondents across 24 nations, including physicians, scientists, educators, public health professionals, patients, study participants, and community members. Respondents indicated difficulty interpreting, synthesizing, and applying PC research reports “50% or more of the time.” Overall, 37% reported problems using current PC research reports. Regarding specific types of research, 49% reported difficulty for qualitative research, 46% for mixed methods research, and 38% for observational research. The most common problems were synthesizing findings/results (58%) and assessing generalizability (42%). The majority of users wanted richer reporting of theoretical foundations (54%); teams, roles, and organization of care (53%); and patient involvement in the research process (53%). Some described challenges with the reporting of the context of the health care setting; practical details of interventions; patient-clinician and team relationships; and generalizability, applicability, and impact in various PC settings. Respondents nominated a list of potential reporting items for PC research. We concluded that opportunities exist to improve the reporting of PC research to make it more useful for its many users, suggesting a role for new research reporting guidelines specific to PC.
We conducted a second international, interprofessional, online survey in 2019, focused on PC clinicians who provided clinical care to patients for more than one-half of their working week.16 The survey yielded 252 respondents across 29 nations, including 88% physicians, 5% nurses, and 3% physician assistants. Of these practicing clinicians, 33% accessed original reports of PC research in academic journals weekly or daily, but only 36% found reports met their needs “frequently” or “always.” We concluded that PC practitioners read original research reports, but current reports meet their information needs less than one-half the time. Practitioners desired improved reporting of study context, interventions, relationships, generalizability, and implementation. Respondents nominated potential reporting items, adding to our list.
After identifying the needs voiced by survey respondents, we scanned the 14 EQUATOR Network guidelines most relevant to PC research.14 We found that currently published guidelines do not adequately address many of the concerns voiced by our respondents.
In addition, we surveyed editors of 12 major journals publishing PC research in 2020, to collect their recommendations for improving the reporting of PC research.*
Our CRISP Working Group also gathered peer comments on our developing work through presentations, open meetings, and workshops at national and international conferences in Australia, Canada, Europe, and the United States during 2018-2022.†
At each stage of this research, participants nominated potential reporting items. Following a prespecified, iterative analytic plan, our CRISP Working Group reviewed new items to add to the growing list. Some potential items were combined, split into separate items, or reworded for clarity and interpretation across nations, health care systems, and PC settings. The revised aggregate list of items was then presented in the following survey. Fewer new items were suggested at each successive stage, suggesting we had reached saturation across the study groups.
Delphi Study
To move from this aggregate list to a consensus set of reporting items, we conducted a Delphi study.17 Using a prespecified, published protocol,18 we recruited an international, interdisciplinary, interprofessional Delphi panel of PC researchers and research users for an online survey in 2021. We presented the list of potential reporting items and asked participants to vote whether each item should be included, required, or recommended in a guideline. An item advanced to the next Delphi round if more than 50% of panelists voted to include it. Eighty-nine respondents completed both Round 1 (84% response rate) and Round 2 (91%), with representation of a wide variety of demographic characteristics, health professions, scientific disciplines, research roles, levels of experience, and world regions. Round 1 presented 29 potential items, of which 25 moved on to Round 2. After the 2 rounds, 11 items were endorsed for inclusion by at least 90% of panelists, and an additional 12 items were endorsed by more than 50%. The Delphi study thereby identified a consensus set of items with broad endorsement from the worldwide community of producers and users of PC research.
On the basis of the Delphi results and earlier studies, the CRISP Working Group made final edits for clarity and broad understanding to produce the draft CRISP Checklist with 24 reporting items.
Pilot Testing and Finalizing
We performed pilot testing with 10 PC researchers, authors, reviewers, patients, study participants, and community representatives.* Participants used near-final drafts of 3 documents—the CRISP Checklist, the CRISP Statement published here, and the CRISP Explanation and Examples Guide (Supplemental Appendix)—to write, revise, or review a PC research report. All completed an anonymous online survey. Overall, participants reported that the checklist helped in writing and reviewing, was easy to understand and use, and improved the final reports. All recommended the checklist to research colleagues and suggested editors of PC research journals encourage authors to use them. The working group used these pilot results to finalize the CRISP documents.
Guided by this total body of research, testing, and commentary, the working group ultimately produced the final CRISP Checklist of 24 reporting items (Figure 1).
The CRISP initiative followed published recommendations for developing research reporting guidelines.19 Our surveys required informed consent and were approved by institutional review bodies,15-17 and our publications followed the relevant reporting guidelines.14-18
CRISP GUIDELINES
The CRISP guidelines consist of the 24-item CRISP Checklist and detailed instructions for use.
Checklist Items
The CRISP Checklist (Figure 1) is an aid for researchers, authors, and editors to help produce PC research reports. It summarizes items the PC community feels are essential for transparency, quality, and usefulness.
PC research involves a wide variety of methods, study designs, topics, and settings; thus, not all CRISP items apply to all studies. Some items may be more relevant to quantitative, qualitative, or participatory research methods. Please refer to the CRISP Explanation and Examples Guide (Supplemental Appendix) for more details on each item.
The CRISP guidance does not constrain effective or creative research communication. Final decisions on content and form rest with authors and editors.
The central theme running through the CRISP Checklist is a call for richer descriptions of context in research reports. PC readers need more information to understand the context of the research team, participants, patients, populations, clinical conditions, clinical encounters, patient care teams, study interventions, and study measures. In every section of the research report—introduction, methods, results, discussion—users need information anchored in the realities and practicalities of PC. The breadth and depth of PC and the wide variety of settings in which it is practiced mean that research reports must provide rich contextual descriptions. For investigators and authors who hope to see their research read and their findings/results implemented in practice, the CRISP Checklist provides a new tool to make their reports relevant, relatable, and actionable for PC readers.
Instructions for Using the Checklist
For each of the 24 items on the CRISP Checklist, the checklist (1) asks authors to indicate whether the information is included in the report (yes, no, or not applicable), (2) suggests a location for the item in the report, and (3) asks authors to note the item’s location in the manuscript.
Authors should respond to each item. If an item is missing, the authors note its absence and explain briefly why it is not included (eg, data not available in the public domain, data not collected, information beyond the scope of the study design).
Authors should note the location of each item in the submitted report by line number, page number, or section. The location is the choice of the authors. Suggested locations follow the usual sections of a research report: IMRaD (introduction, methods, results, and discussion). Details for some items can be provided by citing appropriate references (eg, a reference describing the health care system).
It is important to recognize that other reporting guidelines may also apply when specific methods or settings are used in PC research. Authors could use the CRISP guidelines alongside these guidelines to help ensure that their reports include details on methods as well as the information needed by the PC community (Supplemental Table). When a PC study fits one of the existing reporting guidelines, such as CONSORT11 for a PC trial, STROBE12 for a PC cohort study, or COREQ13 for a PC qualitative study, CRISP would complement that guidance and contextualize elements for PC research. Examples include CRISP items that ask for information about multimorbidity and continuity of care, which are not suggested in other guidelines. Some CRISP reporting items may overlap with those in other guidelines.
As with other reporting guidelines, the CRISP guidelines can help guide the planning and conduct of research, as well as its reporting. They may also be helpful for research teachers, learners, advocates, and funders.
DISCUSSION
Guidelines by and for the PC Research Community
The CRISP Checklist is the culmination of a multimethod program of research that was prospectively designed, transparent, and iterative, from needs assessment and literature review through worldwide surveys and Delphi refinement. We have detailed study designs, results, analysis, and limitations in previous reports.14-18
In the CRISP initiative, we purposefully designed a development pathway different from that of most published reporting guidelines. Rather than relying on a small group of recognized experts in a specific research method, we empowered our whole community as experts in PC research and its communication. We engaged a broad range of voices across professions, specialties, disciplines, nations, personal and professional characteristics, and research roles. We included both producers and users of PC research. At each step, we benefited from the participation of patients and community representatives. Finally, we pilot tested the CRISP Checklist and supporting documents among a diverse group.*
The success of this innovative approach is supported by the high levels of survey response, the appearance of clear themes in reporting items suggested by multiple groups, and the achievement of consensus on a set of reporting items.
Our surveys recruited participants from across the broad PC community, aiming to invite all voices. We were inclusive but cannot claim to be statistically representative. Most clinicians were physicians, and most were family or general practitioners. As the CRISP guidelines evolve, engaging more participant voices will be important.
Our success in engaging family physicians, general practitioners, and others was aided by the endorsement of CRISP goals and methods by WONCA (World Organization of Family Doctors, https://www.globalfamilydoctor.com) and NAPCRG (https://www.napcrg.org).
Guideline Implementation
Although incorporating the CRISP Checklist items may require adding detail and length to reports, our studies show that readers need this information to make the best use of research findings and results. Adding these essential items might require newer approaches to publishing, including use of appendices, online materials, and other creative strategies. Ideally, reporting formats should follow function, and research reports should meet the needs of readers hoping to apply findings/results to research, patient care, health systems, and population health. Advances in publishing technology and dissemination strategies should empower more uses for more users.
The CRISP guidelines are designed specifically to meet the needs of PC, but their principles and guidance can help enhance the reporting of research in other areas of medicine and health sciences. We also encourage other groups to consider the inclusive user-oriented process for developing research reporting guidelines.
Next Steps
The CRISP guidelines are a living document that may be revised over time as PC and its research methods evolve, dissemination routes expand, and science, practice, and mission mature. We welcome the guidance of all to make the guidelines more useful for more users. We welcome comments and suggestions through the CRISP website (https://crisp-pc.org/).
The effectiveness of the CRISP guidelines in improving PC research reports deserves field testing and trials. We encourage others to develop and share best practices for reporting the CRISP items. Editors, journals, and educators can explore new methods for communicating research findings/results and their implications for patient care, practice, and health care systems.
The CRISP Working Group plans to translate the guidelines into multiple languages to assist researchers worldwide. We welcome contact from researchers who believe translation may be useful for their local context.
Journal editors can help authors improve the quality of PC research reports by encouraging the use of the CRISP Checklist in their information for authors and instructions for reviewers.20 Primary care research journals can publish editorials on the potential value of this CRISP Statement and the CRISP Checklist constructed by and for PC researchers.
We hope these CRISP guidelines help investigators, authors, editors, reviewers, readers, and other users improve the reporting of PC research in the service of stimulating inquiry, advancing care, and improving health.
Acknowledgments
The CRISP Working Group thanks Diana N. Louden (Life Sciences Librarian, University of Washington, Seattle, Washington) for managing the literature review, and Angela Yang (School of Dentistry, University of Washington, Seattle, Washington) and Liesbeth Hunik (Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, The Netherlands) for organizing surveys and conducting the review of reporting guidelines. For expert guidance on our Delphi study, we thank Frank Moriarty (Senior Lecturer, Royal College of Surgeons in Ireland, Dublin, Ireland); Peter Lucassen (Senior Researcher, Radboud University Medical Centre, Nijmegen, The Netherlands); and Hans van der Wouden (Associate Professor, Department of General Practice, Amsterdam Public Health Research Institute, Amsterdam UMC - Vrije Universiteit, The Netherlands). We thank Vivian Ramsden (Professor, University of Saskatchewan, Saskatoon, Saskatchewan, Canada) for her insightful review and helpful suggestions on the manuscript. We also thank the participants in the CRISP Delphi and pilot studies.
Footnotes
↵* Unpublished data (Phillips et al, 2020).
↵† Unpublished data.
↵* Unpublished data (Sturgiss et al, 2023).
Conflicts of interest: authors report none.
Author contributions: All authors are members of the CRISP Working Group. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. William R. Phillips is the guarantor and accepts full responsibility for this work, and the conduct of the study; had access to the data; and controlled the decision to publish. Contributors: William R. Phillips and Elizabeth Sturgiss conceived the project; led the international CRISP Working Group; designed the study; managed and analyzed the data; and drafted, revised, and finalized the report. Paul Glasziou, Tim olde Hartman, Aaron Orkin, Pallavi Prathivadi, Joanne Reeve, Grant M. Russell, and Chris van Weel revised the study design, analyzed the data, and revised and approved the final report.
Funding support: All authors and CRISP Working Group members are volunteers. This work was completed with no outside financial support. Part of Dr Phillips’s time was supported by the Helen Riaboff Whiteley Center, University of Washington, Friday Harbor, Washington. Part of Dr Sturgiss’s time was supported by a National Health and Medical Research Council Investigator Grant.
Disclaimer: The views expressed are solely those of the authors and do not necessarily represent official views of the authors’ affiliated institutions.
- Received for publication January 11, 2023.
- Revision received May 22, 2023.
- Accepted for publication May 31, 2023.
- © 2023 Annals of Family Medicine, Inc.