Describing Primary Care Encounters: The Primary Care Network Survey and the National Ambulatory Medical Care Survey
===================================================================================================================

* Helen J. Binns
* David Lanier
* Wilson D. Pace
* James M. Galliher
* Theodore G. Ganiats
* Margaret Grey
* Adolfo J. Ariza
* Robert Williams

## Abstract

**PURPOSE** The purpose of this study was to describe clinical encounters in primary care research networks and compare them with those of the National Ambulatory Medical Care Survey (NAMCS).

**METHODS** Twenty US primary care research networks collected data on clinicians and patient encounters using the Primary Care Network Survey (PRINS) Clinician Interview (PRINS-1) and Patient Record (PRINS-2), which were newly developed based on NAMCS tools. Clinicians completed a PRINS-1 about themselves and a PRINS-2 for each of 30 patient visits. Data included patient characteristics; reason for the visit, diagnoses, and services ordered or performed. We compared PRINS data with data obtained from primary care physicians during 5 cycles of NAMCS (1997–2001). Data were weighted; PRINS reflects participating networks and NAMCS provides national estimates.

**RESULTS** By discipline, 89% of PRINS clinicians were physicians, 4% were physicians in residency training, 5% were advanced practice nurses/nurse-practitioners, and 2% were physician’s assistants. The majority (53%) specialized in pediatrics (34% specialized in family medicine, 9% in internal medicine, and 4% in other specialties). All NAMCS clinicians were physicians, with 20% specializing in pediatrics. When NAMCS and PRINS visits were compared, larger proportions of PRINS visits involved preventive care and were made by children, members of minority racial groups, and individuals who did not have private health insurance. A diagnostic or other assessment service was performed for 99% of PRINS visits and 76% of NAMCS visits (95% confidence interval, 74.9%–78.0%). A preventive or counseling/education service was provided at 64% of PRINS visits and 37% of NAMCS visits (95% confidence interval, 35.1%–38.0%).

**CONCLUSIONS** PRINS presents a view of diverse primary care visits and differs from NAMCS in its methods and findings. Further examinations of PRINS data are needed to assess their usefulness for describing encounters that occur in primary care research networks.

*   Practice-based research
*   health care delivery
*   health services research
*   primary care
*   ambulatory care
*   office visits
*   physicians
*   allied health personnel
*   prevention
*   survey methods

## INTRODUCTION

Practice-based research networks (PBRNs) are groups of practices affiliated for the purposes of research and quality improvement. PBRNs have increased in number and diversity over the past decade and are a prime environment for needed research.1,2 Primary care–focused PBRNs use the experience and insight of clinicians to identify and frame research questions for primary care settings,3 which are the key sites for health care delivery in the United States.4 A wide scope of research is conducted in primary care PBRNs, including clinical trials, epidemiologic studies, translational research, and health services research. At the same time, many of the 111 primary care PBRNs now known to be active in the United States report that the lack of adequate information systems in office practices is a major barrier to research.5 Many PBRNs do not have accurate, up-to-date data describing their members and patients, making it difficult to define the denominator from which individuals are recruited for studies. This lack of information creates a barrier in properly responding to concerns that the networks may not be representative of US primary care practice. Individual PBRNs have addressed these concerns by PBRN patient samples with the National Ambulatory Medical Care Survey (NAMCS) data.6–8

During the past 2 decades, the NAMCS has been used to describe US primary care.9–13 The NAMCS methodology is well established and provides nationally representative cross-sectional data at the visit level.14 Its view of primary care is limited in several ways, however. First, the NAMCS collects information only about visits to physicians and thus provides no information about visits to other disciplines common in primary care practice, such as advanced practice nurses/nurse-practitioners (APNs/NPs) and physician’s assistants. Second, the survey response options for diagnostic/assessment and counseling/education services include options pertinent to surgical and medical specialties and thus have fewer options to describe the primary care encounter. Lastly, very little information about the characteristics of physicians and their practices is publicly available, and the data that are publicly available cannot be used to examine effects of clustering of patient encounters within multiclinician practices. There is a need to obtain more comprehensive information about primary care clinicians, their patients, and services they provide that can be used by researchers, policy makers, and the public. Such information may help to identify strengths and additional areas for improvement in services.

In 2001, as part of its ongoing effort to foster development and enhance the diversity of PBRNs, the Agency for Healthcare Research and Quality (AHRQ) awarded cooperative agreements to 20 US primary care PBRNs to conduct network-defining surveys using a standardized instrument. Clinicians in these networks represented primary care disciplines (general internal medicine, family medicine, and pediatrics) and included practicing physicians, physicians in residency training, APNs/NPs, and physician’s assistants. The primary purpose of the surveys was to systematically collect data about clinicians and patient encounters in each PBRN, while testing a method and survey instruments that could potentially be used by other PBRNs. This report of the **PRI**mary Care **N**etwork **S**urvey (PRINS) describes and illustrates the PRINS method, including weighting of data to represent participating PBRNs’ patient visits. It shows the promise of a rich data set by presenting comparisons with NAMCS data, which may benefit from further investigation.

## METHODS

PBRN and AHRQ representatives jointly developed the PRINS Clinician Interview (PRINS-1) and the PRINS Patient Record (PRINS-2) survey instruments. These instruments were patterned after recent NAMCS data instruments, with an expanded number of items to describe assessments and services, and no attempt to collect the names of medications prescribed. The PRINS method was approved by the US Office of Management and Budget and by each PBRN’s institutional review board. Signed consent was required only for the several PBRNs that had supplemental studies requiring such.

### Subjects

#### PBRNs

The 20 primary care PBRNs had 7 to 1,142 clinician members (median, 145). Two were national networks (pediatric, family medicine), while the rest were organized along geographic boundaries or were groups of practices with similar characteristics (eg, similar computer systems). Four focused their research on issues relevant to children and primarily included pediatric clinicians, while the rest focused on issues relevant in family medicine or internal medicine settings. Many networks had members from a variety of disciplines.

#### Clinician Sampling Strategy

PBRNs used various sampling strategies, with the overall objective of surveying a representative sample of network clinicians. Smaller networks chose to invite all members to participate, whereas other networks selected a random or convenience sample subset. Clinicians were asked to complete a PRINS-1 form about themselves and PRINS-2 forms for patient encounters. Among the 915 clinicians completing PRINS-2 forms, 15 clinicians provided care only in school settings or residential facilities, and data for these clinicians were excluded.

#### Encounter Sampling Strategy

PRINS data were collected from January through August 2002, with participation dates determined by PBRNs and clinicians. Clinicians completed PRINS-2 surveys on a systematic sample of 30 encounters. After estimating the usual patient volume in a week, each clinician established an appropriate sampling strategy (every nth encounter) to accomplish data gathering within 5 to 10 workdays. Many clinicians (56%) completed PRINS-2 visits within 5 workdays (95% completed them within 10 workdays). Clinician compliance with maintenance of the intended sampling strategy was not assessed.

### Instruments

#### PRINS-1

The PRINS-1 survey is a 22-item self-report questionnaire.15 Fifteen items describe clinician demographic characteristics, number of patients seen in an average week, and business relationships. Seven items describe characteristics of the office. Each PRINS-1 form was pre-identified by a PBRN number and a clinician number. A variable to link clinicians within practices was assigned to each PRINS-1 questionnaire.

#### PRINS-2

The PRINS-2 survey was designed to characterize the clinician-patient encounter.16 It is a 1-page instrument with 19 items (4 concerning patient demographics, 2 on clinical relationships of the clinician and practice to the patient, 3 on sources of payment for the visit, and 10 concerning the visit). Clinicians noted “diagnostic and other assessment services ordered or provided” by checking any of 40 options grouped in 5 categories (physical examination, measurements, tests, imaging, and “assessment for”). They noted “preventive and counseling/educational services ordered or provided” by checking any of 16 options. Each PRINS-2 form was pre-identified by study-specific IDs for the PBRN, clinician, and visit.

### Data Procedures

#### Data Collection

Each PBRN oriented its clinicians to the study procedures. Some PBRNs asked clinicians to provide all PRINS-1 data themselves, whereas others gathered practice- and office-level information from administrators. Clinicians listing only obstetrics-gynecology or “other” as their specialty were assigned to the “other” specialty group.

Clinicians completed PRINS-2 forms during or immediately after each patient encounter, with or without the assistance of practice staff. A few PBRNs provided in-office assistance. To facilitate the ability to gather missing PRINS-2 information, clinicians maintained a log linking survey numbers to patients. Data collection was completed on paper, or electronically at the few practices testing computerized data collection. All networks were instructed to have clinicians ask patients their race and ethnicity.

Completed paper-based instruments were returned to each network office and then checked by PBRN staff for legibility and completeness; where possible, data issues were resolved by the PBRN staff and the clinicians. Paper or electronic data were supplied to Social and Scientific Systems, Washington, DC, the data-coordinating center.

#### Data Entry

Paper data were faxed or mailed to the data center and entered using optical character recognition Teleform software (Cardiff, Vista, Calif), and entries were verified by data center personnel. A 12% sample of PRINS-1 entries were subsequently reviewed for accuracy. Twelve instruments each had 1 error, for a PRINS-1 entry error rate of 0.05%. For PRINS-2, the percentage of unreasonable values was determined for 11 data fields. For 5 fields, less than 0.1% of values were unreasonable, and for 6 fields, 0.14% to 0.87% of values were unreasonable. Unreasonable values were considered as missing data.

#### Coding

We coded text entries for visit diagnoses by matching them to text from the *ICD-9-CM Tabular List of Diseases* that was provided with documentation from the 2002 NAMCS cycle.17 The coding was first done by computer matching, and then results were reviewed and edited by a coder and a physician. The PRINS-2 survey instruments provided space for 3 diagnoses; however, when clinicians indicated more, we retained a maximum of 6 International *Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)* codes in the data set. We used the same procedure to code the reason for a visit (list of the patient’s complaints, symptoms, or other reason for the visit) using the NAMCS *Reasons for Visit Classification for Ambulatory Care*.18 Data on reason for visit are not presented.

Two networks coded diagnoses in a 10% random sample of their network visits. They disagreed with the data center–assigned *ICD-9-CM* coding for 3 (1.6%) of 191 diagnoses reviewed. Other reliability or validity measures are not available for the surveys.

### NAMCS Methods

We analyzed data from 5 cycles of NAMCS (1997–2001).19 We restricted analyses to the subset of physicians whose specialty type was primary care20 and whose specialty was family practice or general practice, internal medicine, or pediatrics, as defined by NAMCS.21 The 2001 NAMCS definitions for these are given in the Supplemental Appendix, Table 1 (which can be found online-only at [http://www.annfammed.org/cgi/content/full/5/1/39/DC1](http://www.annfammed.org/cgi/content/full/5/1/39/DC1)). The NAMCS Patient Record form is modified every 2 years; therefore, some items listed under diagnostic/ screening services and counseling/education/therapeutic services vary across years.

The NAMCS uses a multistage sampling procedure that produces essentially unbiased national estimates. It involves a sampling of primary sampling units (representing geographic areas throughout the United States, physician practices within primary sampling units, and patient visits within physician practices). The NAMCS targets non–federally employed, office-based physicians listed in the American Medical Association and American Osteopathic Association master file as providing office-based patient care at sites that are not federally operated facilities or hospital-based outpatient departments.

Sampled physicians are asked to complete patient record forms22 to describe the medical encounter for a systematic random sample of 30 office visits during a randomly assigned 1-week reporting period. The weighted response rate for physicians sampled for the 2001 NAMCS was 64.7%.20 Trained field representatives of the US Census Bureau oversee data collection at physicians’ offices. Data collection is completed by physicians, their staff, or both. Additionally, field representatives may abstract data from the medical record. In the 2001 NAMCS cycle, field representatives, either alone or with physicians or office staff members, abstracted data from records in 47.6% of the offices.20

NAMCS analysts apply statistical adjustments for survey and item nonresponse. For some items, missing values were imputed by randomly assigning a value from a similar patient record. In 2001, the following items were among those imputed: patient age (1.7% of weighted visits), sex (2.6%), and race (21.4%).20 The National Center for Health Statistics provided national estimates of NAMCS physician characteristics (Table 1⇓). Data on race and ethnicity of NAMCS physicians are not available.

View this table:
[Table 1.](http://www.annfammed.org/content/5/1/39/T1)

Table 1.  
**Characteristics of PRINS Clinicians and NAMCS Physicians**

### Analysis

We limited PRINS and NAMCS comparisons to items that are similar on the respective survey instruments. The 95% confidence intervals (CIs) are presented for the NAMCS data. Weighted data represent the sampling frames of the 2 methods: NAMCS data were weighted to produce national estimates of physicians and encounters with office-based primary care physicians,20 and PRINS visits were weighted to represent the clinicians within the networks and encounters at their offices. We derived PRINS weights from network reports of the number of clinicians active within the network and PRINS-1 form estimates of the number of ambulatory patient visits and the number of half-days they provided direct patient care in an average week. Weighted data thus differ considerably from unweighted data.

We grouped primary diagnoses into 15 primary diagnosis groups according to *ICD-9-CM* codes. Visit diagnoses were also examined using *ICD-9-CM* clusters as described by Pace et al.23 For this analysis, we considered all diagnoses listed for the encounter. Because multiple diagnoses per patient encounter were coded within the NAMCS and PRINS databases (maximum of 3 diagnoses for NAMCS and 6 for PRINS), diagnoses were weighted so as to reflect the weighted number of encounters. For example, if 3 *ICD-9-CM* codes were provided, each was weighted by one third and then further weighted to reflect the PRINS or NAMCS sampling frames.

Analyses presented are limited to percentages and 95% CIs for NAMCS data. Because of differences in sampling frames and methods between PRINS and NAMCS, comparisons based on these data should be made with caution. In addition, analyses of PRINS data will need to account for the sampling design that includes clinician and visit clustering. Data from visits of comparable NAMCS and PRINS practicing physicians can be examined when PRINS data become publicly available. This report presents a preliminary overview of PRINS data, suggests how PRINS may differ from NAMCS, and offers areas for further investigation.

## RESULTS

### Clinicians

We analyzed data from 900 PRINS clinicians in 392 practices; 72% were in a practice with another PRINS participant. A median 40% of clinicians per network participated (range, 2%–100%). Most PRINS clinicians were physicians (89%); the remaining were physicians in residency training (4%), APNs/NPs (5%), or physician’s assistants (2%) (Table 1⇑). Most PRINS clinicians were in pediatrics or family medicine, about one half were female, and approximately one third were practice owners. Table 1⇑ also shows the weighted characteristics of the 1,491 primary care physicians participating in the 1997–2001 NAMCS.

### Patients and Encounters

We compared characteristics of 597,176 weighted encounters with PRINS clinicians (26,265 unweighted encounters) with those of approximately 2.1 billion weighted encounters with NAMCS physicians. Larger percentages of the PRINS visits were made by children and nonwhite patients, whereas smaller percentages were made by patients with private insurance (Table 2⇓).

View this table:
[Table 2.](http://www.annfammed.org/content/5/1/39/T2)

Table 2.  
**Characteristics of PRINS Patient Visits and NAMCS Patient Visits**

A diagnostic or other assessment service was ordered or performed at almost all of PRINS visits and at approximately three quarters of NAMCS visits (Table 2⇑). Overall, blood pressure measurement was performed at similar percentages of PRINS and NAMCS visits. In analyses stratified by patient age-group, however, proportions of PRINS patient encounters including a blood pressure measurement were about 7 to 13 percentage points higher than the upper 95% CI for NAMCS encounters among similarly aged patients. Some aspect of a physical examination was conducted during 92.1% of PRINS encounters. When physical examination and blood pressure were excluded, 87.1% of PRINS visits included a diagnostic or other assessment service, most commonly weight measurement (82.0%), height measurement (42.1%), and assessment for developmental or functional impairment (13.4%), for mood or behavior problems (10.1%), and for tobacco use (10.4%).

A preventive or counseling/educational service was ordered or provided during approximately two thirds of PRINS visits and one third of NAMCS visits (Table 2⇑). The percentages of visits involving services for diet/nutrition, growth/development, and tobacco use/exposure were approximately 2 times higher for PRINS visits than for NAMCS visits. Other common preventive and counseling/educational services taking place during PRINS visits were counseling about physical activity (29.0%), injury prevention/safety (16.6%), immunization (13.1%), and behavior/psychosocial issues (11.9%).

### Diagnoses

The percentages of visits with a given category for the primary diagnosis rarely differed by more than 2% between the PRINS and NAMCS samples (Table 3⇓). Overall, 46.7% of PRINS visits included a single diagnosis code (32.6% included 2 codes; 20.0%, 3 codes; 0.6%, 4–6 codes; and 0.2%, 0 codes). A routine health maintenance diagnosis was listed as one of the codes for 22.7% of PRINS visits.

View this table:
[Table 3.](http://www.annfammed.org/content/5/1/39/T3)

Table 3.  
**Percentage of Visits by Primary Diagnosis Group**

The top 15 diagnostic clusters for PRINS encounters were similar to those for NAMCS encounters (Table 4⇓). Data were adjusted for multiple diagnoses per visit as described in the Methods. In both PRINS and NAMCS, the highest percentages of health maintenance visits occurred among children aged 0 to 14 years (28.0% and 25.6%, respectively; 95% CI, 24.5%–26.7%) (Table 5⇓).

View this table:
[Table 4.](http://www.annfammed.org/content/5/1/39/T4)

Table 4.  
**Distribution of Visits for the Top 15 PRINS Diagnostic Clusters**

View this table:
[Table 5.](http://www.annfammed.org/content/5/1/39/T5)

Table 5.  
**Percentage of Visits Within Age-groups With the Diagnosis of Routine Health Maintenance**

### PRINS Clinician Disciplines

Data stratified by discipline for PRINS clinicians and encounters are given online (Supplemental Appendix: Tables 2⇑–5⇑⇑⇑, available online-only at [http://www.annfammed.org/cgi/content/full/5/1/39/DC1](http://www.annfammed.org/cgi/content/full/5/1/39/DC1)). PRINS clinician data are presented in an unweighted format to better show the impact of clinician weighting. Most APNs/NPs and physician’s assistants worked in the family practice and general practice settings. More than 90% of APN/NP visits included counseling services, compared with approximately 60% for other disciplines.

## DISCUSSION

The PRINS data represent the first systematic cataloging of care delivery in primary care offices by a large, diverse group of PBRN members. This national effort presents a new data set that can be used to examine primary care delivery. It uniquely includes data collected from practicing physicians, physicians in training, APNs/NPs, and physician’s assistants.

Diagnostic or other assessment services and preventive and counseling and educational services occurred more routinely during PRINS visits than during NAMCS visits. Further analyses may shed light on whether such differences are due to survey methods, patient populations, or actual service delivery. PRINS data were directly recorded by the providing clinician, whereas NAMCS data were sometimes derived from chart review, a method shown to underestimate the delivery of any counseling, education, or therapeutic service by about 30%, but not to significantly influence recording of diagnostic or screening services.24 In addition, the PRINS-2 form was specifically designed to capture relevant primary care activities, thus offering more options for capturing this group of services. An evaluation of NAMCS long and short forms found that listing more options for diagnostic and screening services increased the number of services reported, but an expansion of counseling, education, or therapeutic items did not.24 Both PRINS and NAMCS survey methods likely result in an of health behavior counseling compared with direct observation.25 Another possible difference is the extent to which clinicians and their practice staff members in PBRNs may more diligently comply with research protocols relative to clinicians and staff less experienced in research conducted in the office setting.

The number of clinician diagnoses and patient complaints allowed on PRINS and NAMCS forms may need to be expanded to better reflect the complexity of primary care office visits. Family physicians address more than 3 problems at approximately one third of office visits but bill for more than 3 problems only 21% of the time.26 Although neither NAMCS nor PRINS forms specifically requested more than 3 diagnoses or patient complaints, PRINS allowed up to 6 to be coded; nonetheless, nearly all PRINS visits had 3 or fewer diagnoses listed. The effect of allowing a greater number of codes is an important area for future research.

NAMCS data play an important role in understanding the delivery of ambulatory care within the United States. A strength of NAMCS is the representative sampling methodology, whereas a weakness is the problem inherent with chart abstraction and imputation. The strengths and weaknesses of PRINS appear to be exactly the reverse of those of NAMCS. Viewed in combination, the 2 data sets may provide an improved understanding of activities undertaken within primary care. With the growth of PBRNs and the desire of the National Institutes of Health to train 50,000 community-based-clinician researchers,27 other alternatives to PRINS or NAMCS may be emerging to provide even more robust views of ambulatory care with improved data collection directly from clinicians.

PRINS data may offer a more in-depth view of particular aspects of primary care. First, PRINS encounters included many children, members of minority racial groups, and patients without private health insurance. These patient groups have health concerns that make them key groups to target when addressing national health priorities. Second, physicians in training, APNs/NPs, and physician’s assistants were included in PRINS; thus, these data offer the opportunity to compare care delivery between types of clinicians.28,29 Third, the PRINS data set is one of the first large data sets that can be used to examine questions related to clustering of patient characteristics and service delivery within practices, which is an important aspect of analysis in PBRN research.30–34 Lastly, PRINS data are valuable to individual PBRNs for describing their clinicians, patients, and visits. Such data are useful during study planning or practice quality improvement efforts to estimate percentages of patients with particular characteristics or diagnoses. PBRNs attempting such research should include representative samples of their members in data collection processes.

The PRINS data describing primary care delivery among PBRNs are unparalleled in scope. These data have the potential to more clearly describe service delivery in that sector. In addition, these data offer the unique opportunity to examine research questions of particular relevance to PBRNs. We encourage investigators to pursue further evaluations using the PRINS public use data when they become available from AHRQ.

## Acknowledgments

We greatly appreciate the efforts of the numerous clinicians, their practice staff members, and the leaders and research staff of the 20 PBRNs who participated in this effort. We also thank Social and Scientific Systems (SSS), 8757 Georgia Ave, Silver Spring, MD 20910, for providing statistical analyses. Many individuals at the National Centers for Health Statistics thoughtfully reviewed a draft of this paper and provided helpful comments; we particularly thank Donald K. Cherry, MS, National Center for Health Statistics, for providing NAMCS physician data.

PRINS Participants: Participating networks and the individuals in each network who coordinated this research effort for their network: Dartmouth-Northern New England Primary Care Research Network (the COOP Project): John H. Wasson, MD, Deborah J. Johnson, Judy Knapp; San Diego Unified Family Medicine Research Network (SURF*NET): Martin L. Kabongo, MD, PhD, Theodore G. Ganiats, MD; Kansas Practice Research Network (KSPRN): K. James Kallail, PhD, Lolem Ngong, MPH, Rick Kellerman, MD; Cincinnati Pediatric Research Group (Cincinnati PRG): Robert M. Siegel, MD, Jeralyn Bernier, MD, MPH; Colorado Research Network (CaReNet): Wilson D. Pace, MD, Deborah Main, PhD; Oklahoma Physicians Resource/Research Network (OKPRN): James Mold, MD, MPH, Michelle A. Roberts, BA; Southern Primary-care Urban Research Network (SPUR-NET): Jeffrey R. Steinbauer, MD, Grace Kuo, PharmD; Pediatric Practice Research Group (PPRG): Adolfo J. Ariza, MD, Helen J. Binns, MD, MPH; Research Involving Outpatient Settings Network (RIOSNet): Robert Williams, MD, MPH, Mark Unverzagt, MD, Alison Baker, MS; Virginia Ambulatory Care Outcomes Research Network (ACORN): Stephen F. Rothemich, MD, MS, Amy Burgett, RN; Pediatric Research in Office Settings (PROS): Kathleen A. Thoma, MA, Eric J. Slora, PhD, Richard C. Wasserman, MD, MPH; Southeast Regional Clinicians Network (SERCN): Elvan Catherine Daniels, MD, Yvonne Fry-Johnson, MD; Advanced Practice Registered Nurse Research Network (APRNet): Margaret Grey, DrPH, CPNP, Jo Cecille Demarest, BS, Terry Deshefy-Longhi, MS, RN; American Academy of Family Physicians National Research Network (AAFP NRN): John Hickner, MD, MS, Jennifer A. Kappus, Thomas V. Stewart, James M. Galliher, PhD; Northeastern Ohio Network (NEON): Kurt Stange, MD, PhD, Valerie Gilchrist, MD, Gary McCord, MA; UCSF Collaborative Research Network (UCSF CRN): Margaret Handley, PhD, MD, Mary Croughan, PhD; Indiana University-Indiana University Medical Group Research Network (ResNet): William Tierney, MD, Brenda Hudson, MA, Atif Zafar, MD, Burke Mamlin, MD, Ada Yeung, MHSA; Mount Sinai Practice-Based Research Network (Mt Sinai PBRN): Thomas McGinn, MD, MPH; Rainbow Research Network (RRN), Amy M. Heneghan, MD, Sunny Morton; DeltaNet: Regina Benjamin, MD.

## Footnotes

*   *Conflicts of interest: none reported* 

*   **Funding support:** The project was supported through a series of cooperative agreements (HS-01-004) from the Agency for Healthcare Research and Quality.

*   This study was presented in part at the 1st Annual Practice-Based Research Networks (PBRN) Meeting, July 21, 2005, Washington, DC.

*   Received for publication July 28, 2005.
*   Revision received May 22, 2006.
*   Accepted for publication May 30, 2006.


*   © 2007 Annals of Family Medicine, Inc.

## REFERENCES

1.  Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290(12):1624–1632.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.290.12.1624&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14506122&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000185461300028&link_type=ISI) 

2.  Lenfant C. Shattuck lecture—clinical research to clinical practice—lost in translation? N Engl J Med. 2003;349(9):868–874.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMsa035507&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12944573&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000184968400009&link_type=ISI) 

3.  Agency for Healthcare Research and Quality, Rockville, Md. Primary Care Practice-Based Research Networks. Fact Sheet, June 2001. AHRQ publication 01-P020. Available at: [http://www.ahrq.gov/research/pbrnfact.htm](http://www.ahrq.gov/research/pbrnfact.htm). Accessed: 15 July 2005.
    
    

4.  Green LA, Fryer GE Jr, Yawn BP, Lanier D, Dovey SM. The ecology of medical care revisited. N Engl J Med. 2001;344(26):2021–2025.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJM200106283442611&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11430334&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000169508200011&link_type=ISI) 

5.  Tierney W, Hudson B, Oppenheimer C. A national survey of primary care practice-based research networks [Abstract]. J Gen Intern Med. 2004;19(Suppl 1):109.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1525-1497.2004.S1006_5.x&link_type=DOI) 

6.  Green LA, Miller RS, Reed FM, Iverson DC, Barley GE. How representative of typical practice are practice-based research networks? A report from the Ambulatory Sentinel Practice Network Inc (ASPN). Arch Fam Med. 1993;2(9):939–949.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/archfami.1993.01850100059010&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=8111525&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 

7.  Lavigne JV, Binns HJ, Christoffel KK, et al. Behavioral and emotional problems among preschool children in pediatric primary care: prevalence and pediatricians’ recognition. Pediatric Practice Research Group. Pediatrics. 1993;91(3):649–655.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiOTEvMy82NDkiO3M6NDoiYXRvbSI7czoyMToiL2FubmFsc2ZtLzUvMS8zOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  Pearce KA, Love MM, Barron MA, Matheny SC, Mahfoud Z. How and why to study the practice content of a practice-based research network. Ann Fam Med. 2004;2(5):425–428.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMi81LzQyNSI7czo0OiJhdG9tIjtzOjIxOiIvYW5uYWxzZm0vNS8xLzM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  Freid VM, Makuc DM, Rooks RN. Ambulatory health care visits by children: principal diagnosis and place of visit. Vital Health Stat 13. 1998;(137):1–23.
    
    

10. Schappert SM. Ambulatory care visits of physician offices, hospital outpatient departments, and emergency departments: United States, 1995. Vital Health Stat 13. 1997;(129):1–38.
    
    

11. Robison LM, Sclar DA, Skaer TL, Galin RS. National trends in the prevalence of attention-deficit/hyperactivity disorder and the prescribing of methylphenidate among school-age children: 1990–1995. Clin Pediatr (Phila). 1999;38(4):209–217.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BjcGoiO3M6NToicmVzaWQiO3M6ODoiMzgvNC8yMDkiO3M6NDoiYXRvbSI7czoyMToiL2FubmFsc2ZtLzUvMS8zOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

12. Namen AM, Dunagan DP, Fleischer A, et al. Increased physician-reported sleep apnea: the National Ambulatory Medical Care Survey. Chest. 2002;121(6):1741–1747.
    
    

13. Franks P, Clancy CM, Nutting PA. Defining primary care. Empirical analysis of the National Ambulatory Medical Care Survey. Med Care. 1997;35(7):655–668.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/00005650-199707000-00001&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=9219494&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1997XJ68400001&link_type=ISI) 

14. National Center for Health Statistics. National Center for Health Statistics Ambulatory Health Care Data. Available at: [http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm](http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm). Accessed: 3 July 2004.
    
    

15. Agency for Healthcare Research and Quality. Primary Care Network Survey Clinician Interview (PRINS_1). Available at: [http://extranet.ahrq.gov/pbrnet](http://extranet.ahrq.gov/pbrnet). Accessed: 15 July 2005.
    
    

16. Agency for Healthcare Research and Quality. Primary Care Network Survey Patient Record (PRINS_2). Available at: [http://extranet.ahrq.gov/pbrnet](http://extranet.ahrq.gov/pbrnet). Accessed: 15 July 2005.
    
    

17. Centers for Disease Control and Prevention, National Center for Health Statistics, Ambulatory Health Care Data. DataWarehouse. *International Classification of Diseases, 9th Revision, Clinical Modification*. Washington, DC: Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: [ftp://ftp.cdc.gov/pub/Health\_Statistics/NCHS/Publications/ICD9-CM/2002/](ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/ICD9-CM/2002/). Accessed: 3 July 2004.
    
    

18. Centers for Disease Control and Prevention, National Center for Health Statistics, Ambulatory Health Care Data. 1997 Reasons for Visit Classification. Available at: [http://www.cdc.gov/nchs/data/ahcd/rvc97.pdf](http://www.cdc.gov/nchs/data/ahcd/rvc97.pdf). Accessed: 3 July 2004.
    
    

19. Centers for Disease Control and Prevention, National Center for Health Statistics, Ambulatory Health Care Data. Public use data on CD-ROM. Available at: [http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm](http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm). Accessed: 3 July 2004.
    
    

20. Centers for Disease Control and Prevention, National Center for Health Statistics. Advance Data from Vital and Health Statistics. Number 337, August 11, 2003. Available at: [http://www.cdc.gov/nchs/data/ad/ad337.pdf](http://www.cdc.gov/nchs/data/ad/ad337.pdf). Accessed: 27 May 2005.
    
    

21. Centers for Disease Control and Prevention, National Center for Health Statistics. 2001 NAMCS Micro-Data File Documentation. P52-4. Available at: [ftp://ftp.cdc.gov/pub/Health\_Statistics/NCHS/Data-set\_Documentation/NAMCS/doc01.pdf](ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Data-set_Documentation/NAMCS/doc01.pdf). Accessed: 13 January 2006.
    
    

22. Centers for Disease Control and Prevention, National Center for Health Statistics, Ambulatory Health Care Data. Survey Instrument NAMCS. Available at: [http://www.cdc.gov/nchs/about/major/ahcd/surinst.htm#Survey%20Instrument%20NAMCS](http://www.cdc.gov/nchs/about/major/ahcd/surinst.htm#Survey%20Instrument%20NAMCS). Accessed: 24 July 2005.
    
    

23. Pace WD, Dickinson LM, Staton EW. Seasonal variation in diagnoses and visits to family physicians. Ann Fam Med. 2004;2(5):411–417.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMi81LzQxMSI7czo0OiJhdG9tIjtzOjIxOiIvYW5uYWxzZm0vNS8xLzM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

24. Hing E, Schappert SM, Burt CW, Shimizu IM. Effects of form length and item format on response patterns and estimates of physician office and hospital outpatient department visits. National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey, 2001. Vital Health Stat 2. 2005;(139):1–32.
    
    

25. Gilchrist VJ, Stange KC, Flocke SA, McCord G, Bourguet CC. A comparison of the National Ambulatory Medical Care Survey (NAMCS) measurement approach with direct observation of outpatient visits. Med Care. 2004;42(3):276–280.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/01.mlr.0000114916.95639.af&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15076827&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000189293000010&link_type=ISI) 

26. Beasley JW, Hankey TH, Erickson R, et al. How many problems do family physicians manage at each encounter? A WReN study. Ann Fam Med. 2004;2(5):405–410.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMi81LzQwNSI7czo0OiJhdG9tIjtzOjIxOiIvYW5uYWxzZm0vNS8xLzM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

27. Zerhouni EA. US biomedical research: basic, translational, and clinical sciences. JAMA. 2005;294(11):1352–1358.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.294.11.1352&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16174693&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000231987800008&link_type=ISI) 

28. Roblin DW, Howard DH, Becker ER, Adams EK, Roberts MH. Use of midlevel practitioners to achieve labor cost savings in the primary care practice of an MCO. Health Serv Res. 2004;39(3):607–626.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1475-6773.2004.00247.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15149481&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221636400011&link_type=ISI) 

29. Hooker RS, McCaig LF. Use of physician assistants and nurse practitioners in primary care, 1995–1999. Health Aff (Millwood). 2001;20(4):231–238.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjIwLzEvMjMxIjtzOjQ6ImF0b20iO3M6MjE6Ii9hbm5hbHNmbS81LzEvMzkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

30. Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17(2):192–196.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiZmFtcHJhY3QiO3M6NToicmVzaWQiO3M6ODoiMTcvMi8xOTIiO3M6NDoiYXRvbSI7czoyMToiL2FubmFsc2ZtLzUvMS8zOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

31. Moerbeek M. Randomization of clusters versus randomization of persons within clusters: which is preferable? Am Stat. 2005;59:72–78.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16250177&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 

32. Medical audit in general practice. I: Effects on doctors’ clinical behaviour for common childhood conditions. North of England Study of Standards and Performance in General Practice. BMJ. 1992;304(6840):1480–1484.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMDQvNjg0MC8xNDgwIjtzOjQ6ImF0b20iO3M6MjE6Ii9hbm5hbHNmbS81LzEvMzkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

33. Campbell M, Grimshaw J, Steen N. Sample size calculations for cluster randomised trials. Changing Professional Practice in Europe Group (EU BIOMED II Concerted Action). J Health Serv Res Policy. 2000;5(1):12–16.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10787581&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 

34. Adams G, Gulliford MC, Ukoumunne OC, et al. Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004;57(8):785–794.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.jclinepi.2003.12.013&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15485730&link_type=MED&atom=%2Fannalsfm%2F5%2F1%2F39.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000224719400005&link_type=ISI)