How and Why to Study the Practice Content of a Practice-Based Research Network

INTRODUCTION

Practice-based research networks (PBRNs) are sustained collaborations between practitioners and academicians dedicated to developing relevant research questions, working together on study design and conduct, and translating new knowledge into practice.^1–,³ Primary care PBRNs have been active in the United States since 1978. Expanded funding opportunities have recently spawned the emergence of many new PBRNs.^1,3 New PBRNs face the challenges of creating a sense of identity and achieving tangible involvement in research by their members. Selection and conduct of a first study by the network is critical to meeting these challenges. It is advisable to choose a practical and relevant project, avoid complex protocols, and have broad member input in study design and conduct. Methodologic rigor should be balanced against constraints that might seriously limit clinicians’ participation. Describing the clinician members and the practice content of a new PBRN can meet these criteria, establish the network’s identity, stimulate further research, and enhance the ability of the PBRN to attract funding. The aims of this article are to describe methods for and our experiences with performing a practice content study in a new PBRN. Limited study results are included to illustrate important lessons learned and the overall value of this endeavor. Our methods and selected results are available in greater detail as supplemental data in Appendix 1, which can be found online at http://www.annfammed.org/cgi/content/full/2/5/425/DC1).

METHODS

As we started the Kentucky Ambulatory Network (KAN), we obtained consultation from family physicians with expertise in practice-based research. We thereby perceived the usefulness of doing a practice content study with external comparability, balanced against the risk to member interest of launching the PBRN with a study that did not focus on a clinical question. All our consultants strongly advised that we involve network clinicians when planning any study. We decided to replicate the National Ambulatory Medical Care Survey (NAMCS) within KAN, inviting limited changes to its methods, and involving community-based clinicians in crafting the exact methods. The NAMCS is a survey of visits to office-based physicians in the United States, performed annually by the Centers for Disease Control and Prevention. Family and general practice physicians are combined in reports that are stratified by specialty.⁴ Results of NAMCS replication studies have been published by 2 other primary care PBRNs,^5,6 with the Ambulatory Sentinel Practice Network (ASPN) publishing the only such comprehensive report a decade ago.⁵

Much of our inaugural convocation of members (November 2000) was devoted to planning the first KAN study. During breakout sessions KAN members and our consultants discussed ideas for modifications to the NAMCS methods. A steering committee was composed of physicians in private practice and faculty from the 2 Kentucky medical schools. Guided by input from the general membership, this committee adopted focused modifications of the NAMCS methods. The KAN General Advisory Committee and the University of Kentucky (UK) Institutional Review Board (IRB) reviewed and approved the protocol. Participating practitioners were deemed the study subjects, because they were recording their impressions about patient visits after the visit was completed. All recorded patient-level data were free of identifiers, and our IRB agreed that patient consent was not required.

We used the 1997–1998 NAMCS data collection protocol as our starting point.⁷ The steering committee added the following data fields to the patient record data form (Appendix 1, Figure 1): tobacco use, weight and height measurements, and whether chronic pain, depression, or anxiety (each defined by the clinician) contributed to the visit. The numbers of patient complaints, diagnoses, and medications that could be recorded were expanded.

We invited all community-based primary care clinicians (physicians and their affiliated nurse-practitioners and physician assistants) enrolled as KAN members by December 1, 2001, to participate. One of 3 faculty members from the UK Department of Family Practice called at least 1 clinician at each of 40 practices to discuss the study and invite their participation. Subsequently, the KAN study coordinator arranged a site visit at each practice that was attended by a UK faculty member and the study coordinator, with lunch provided by KAN. During this visit, we shared visions for KAN with physicians and staff members, described the purpose and procedures for this first study, and delivered study materials. We identified a clinician leader at each practice site to invite his or her colleagues to the lunch meeting and encourage them to participate. Each participant completed a brief clinician profile. Participant characteristics were compared with those from a national database of physicians.⁸

Each practice site was provided a study manual and patient record (visit) data forms with serial numbers that facilitated tracking. A single staff person at each practice site was identified as the site coordinator. Participants were instructed to complete the patient record form immediately after each visit or by end of the day. The KAN coordinator proactively contacted the practices throughout the data collection period and encouraged the participants to contact her as needed.

Data were collected from May 2001 to June 2002. Each participating practice volunteered for 2 separate weeks in the calendar year during which the clinicians were expected to complete 25 ambulatory care patient records per participating clinician per week. Clinicians were instructed to spread data collection evenly and systematically throughout each week, recording a set ratio of visits toward the target number (eg, a physician who regularly saw 125 patients per week would fill out a patient record form after every fifth visit). Missing data were not imputed. Data management and analytic methods, including those related to clustering, are described as supplemental data in Appendix 1.

RESULTS

Fifty-six clinicians at 24 sites returned 2,228 patient record (visit) forms. Forty-six clinicians completed 2 weeks of data collection; the rest completed only 1 week. Initially, 23 clinicians agreed to participate. They helped involve 33 more of their colleagues in the study. Some clinicians completed fewer than 25 patient record forms per week because they overestimated the number of patients they would see that week. The distribution of patient record data among clinicians and practices is displayed in Appendix 1. Sixty eight percent of eligible KAN clinicians participated. Available data describing participating and nonparticipating KAN clinicians, plus comparisons with the American Academy of Family Physicians database, are displayed in Appendix 1, Table 1.

DISCUSSION

The KAN practice content study helped us establish and activate our new PBRN fairly rapidly, fueling growth of our active membership in the process. We believe that clinicians’ involvement in planning, our attention to keeping the protocol as user-friendly as possible, and the personalized attention given each participating practice by our study coordinator, each contributed to this project’s success. The data gained will inform, guide, and support our research activities for years. We are sharing this information with our members, potential collaborators, funding agencies, and other networks. By showing that we can describe our network at this level of detail, we have attracted other PBRNs as collaborators and are using our data to inform pilot projects and support grant applications.

Our findings are also being used to validate our members’ impressions about some of the problems members face and energize them to help design or participate in future studies. For example, the high prevalence of obesity and the rates of counseling about diet and exercise stratified by body mass index have sparked ideas for studying incentives and barriers to promoting healthy lifestyles.

Replicating established methods facilitates comparisons of KAN practice content with other data sets. It will help consumers of our findings discern similarities and differences of importance to them and make judgments about the applicability of these and future KAN study results to populations of interest. Some of our changes to NAMCS methods also yielded data that may be valuable to other researchers. For example, our findings on the types and numbers of diagnoses made and medications prescribed may be useful for further research into the complexity of primary care. Seminal studies into this critical area have been published, but much is left to do.^12,16–,¹⁸

This cross-sectional study provided only a snapshot of the practice content of our PBRN. Practice content may change with time, but broad similarities between our results and those of similar studies done years apart suggests some stability in the content of general and family practice. Our data reflect only the first 24 community-based practices to become active in KAN. We could not continue this data collection indefinitely because of insufficient funding and manpower. Clinician membership in KAN is open to all primary care physicians and their affiliated physician extenders in Kentucky (founding members include a few clinicians in neighboring Tennessee), but active KAN members do not represent a random sample of any population of primary care providers.

Because any individual practice collected data for no more than 2 separate weeks, questions requiring longitudinal data cannot be addressed, and individual practices cannot be described. Efficient systems for continuous collection of practice content data in PBRNs would circumvent these limitations and enhance collaborations among PBRNs, thereby empowering them to answer important questions collectively with high degrees of validity and generalizability.