Delivery of Clinical Preventive Services in Family Medicine Offices
===================================================================

* Benjamin F. Crabtree
* William L. Miller
* Alfred F. Tallia
* Deborah J. Cohen
* Barbara DiCicco-Bloom
* Helen E. McIlvain
* Virginia A. Aita
* John G. Scott
* Patrice B. Gregory
* Kurt C. Stange
* Reuben R. McDaniel, Jr

## Abstract

**BACKGROUND** This study aimed to elucidate how clinical preventive services are delivered in family practices and how this information might inform improvement efforts.

**METHODS** We used a comparative case study design to observe clinical preventive service delivery in 18 purposefully selected Midwestern family medicine offices from 1997 to 1999. Medical records, observation of outpatient encounters, and patient exit cards were used to calculate practice-level rates of delivery of clinical preventive services. Field notes from direct observation of clinical encounters and prolonged observation of the practice and transcripts from in-depth interviews of practice staff and physicians were systematically examined to identify approaches to delivering clinical preventive services recommended by the US Preventive Services Task Force.

**RESULTS** Practices developed individualized approaches for delivering clinical preventive services, with no one approach being successful across practices. Clinicians acknowledged a 3-fold mission of providing acute care, managing chronic problems, and prevention, but only some made prevention a priority. The clinical encounter was a central focus for preventive service delivery in all practices. Preventive services delivery rates often appeared to be influenced by competing demands within the clinical encounter (including between different preventive services), having a physician champion who prioritized prevention, and economic concerns.

**CONCLUSIONS** Practice quality improvement efforts that assume there is an optimal approach for delivering clinical preventive services fail to account for practices’ propensity to optimize care processes to meet local contexts. Interventions to enhance clinical preventive service delivery should be tailored to meet the local needs of practices and their patient populations.

*   Prevention
*   mass screening
*   office visits
*   family medicine offices
*   professional practice
*   health care quality, access, and evaluation
*   quality assurance, health care
*   quality improvement
*   qualitative research
*   Papanicolaou smear
*   cholesterol testing
*   smoking/prevention and control
*   mammography
*   immunizations

## INTRODUCTION

The *Healthy People 2010* report calls for a major effort to improve rates of preventive service delivery by primary care clinicians.1 Despite prevention being a core component of primary care practice,2 studies show that clinical preventive service delivery rates are low. 3,4 A range of interventions to enhance the delivery of clinical preventive services have been tried with varying degrees of success, including the use of continuing medical education, audit and feedback, computerized reminder systems, the involvement of nursing staff, the use of chart-based algorithms, and the implementation of continuous quality improvement programs.5–10

Efforts to improve clinical preventive services delivery are limited by our understanding of how clinicians and practices actually incorporate and deliver preventive services within the competing demands of care.11–13 Although recent research provides insight into the emergent nature of practices as human organizations,14,15 much remains to be understood about how the specifics of patients, their families, communities, clinicians, and health systems affect the approaches used in practices to provide clinical preventive services.13,16,17

The goal of this research is to better understand the organizational features of primary care practice that help explain how and why practices deliver clinical preventive services. Specifically, we asked: What competing demands are imposed by carrying out clinical prevention and illness care? What approaches are used by practices with high vs low rates of preventive service delivery? What organizational features support preventive services? Answers may suggest ways for enhancing the delivery of clinical preventive services.

## METHODS

The Prevention & Competing Demands in Primary Care study was an observational comparative case study designed to examine the organizational contexts and features that support or inhibit the delivery of clinical preventive services in family medicine offices. Practices were selected as cases, and the study design involved extensive observation of clinical encounters and office systems by a field researcher who spent 4 weeks or more in each practice. This comparative case study design sought to optimize qualitative description at the practice level through an iterative, purposeful sampling strategy, and quantitative data were used selectively to enrich the qualitative data. The design is ideal for identifying possible patterns across cases, but the qualitative sampling strategy makes it inappropriate to perform quantitative comparisons across practices.

The study protocol was reviewed and approved by the appropriate institutional review board.

### Sampling

Practice selection was based on results from a previous study of 91 practices from a single Midwestern state.18 These practices were ranked according to tobacco-related services delivery rates and then purposefully chosen in an iterative process to include a range in tobacco-related prevention rates, practice size, and geographic location (urban, suburban, and rural). Eighteen practices participated in the study. Approximately 30 patients per clinician were sampled using a strategy of seeking consent from consecutive patients in the waiting room. Details of practice and patient sampling can be found in Supplemental Appendix 1 available online only at [http://www.annfammed.org/cgi/content/full/3/5/430/DC1](http://www.annfammed.org/cgi/content/full/3/5/430/DC1).

### Data Collection

A field researcher trained in qualitative methods collected data through direct observations of the practice and patient encounters, brief patient exit cards to obtain self-reported smoking status and other prevention activities, patient pathways in which a patient gives consent and is observed throughout the visit to the practice,19 individual in-depth interviews with physicians and key staff members, and chart audits. These data contained detailed descriptions of local clinic environments, patient characteristics, nursing stations, examination rooms, waiting areas, physician offices, and patient education materials. Practice personnel and their roles, duties, and relationships with other staff were characterized in practice genograms.20 Physical office systems including charts, flow sheets, and computer systems, as well as functional office routines and procedures, were described. Individual in-depth interviews assessed clinician and staff perceptions of prevention philosophy, knowledge, and delivery. Field researchers directly observed and dictated descriptions of approximately 30 patient encounters with each of the more than 50 clinicians from the 18 practices, and charts were abstracted on each patient. Details of these methods, including many of the data collection instruments, have been previously published.21

### Data Management and Analysis

Interviews and field notes were transcribed and imported into FolioViews 4.2 (Open Market, Inc, Salt Lake City, 1999), a text management software program. Chart audit and structured checklist data were entered into Microsoft Excel and imported into SPSS (Statistical Package for the Social Sciences, SPSS Inc, Chicago, Ill).

Data were analyzed and interpreted in several iterative phases,22 leading to summaries of each practice’s approach to preventive service delivery. Chart reviews, encounter descriptions, and patient exit card responses were combined to calculate practice rates for 3 types of preventive services: screening (combination of cholesterol screening, Papanicolaou [Pap] test, and mammogram), counseling (smoking cessation counseling), and immunization (combination of childhood series, and adult tetanus, influenza, and pneumonia). Practice summaries, calculated rates for preventive services, and descriptions of individual clinician’s approaches were combined to create a summary characterizing each practice’s strategy for delivering preventive services. These summaries were compared to identify larger patterns (see Supplemental Appendix 1 for details).

## RESULTS

The 18 practices included a range of small and large practices located in different-sized communities, including 8 rural, 6 suburban, and 4 urban. The 57 clinicians were from practices that ranged from 1 solo practice to a large multispecialty practice, with most practices being small groups of 2 to 4 clinicians (n = 13). Eight practices included both physicians (doctors of medicine or doctors of osteopathy) and other allied health clinicians (nurse practitioner or physician’s assistant). In the largest practice, a multispecialty practice that included relatively independent obstetric-gynecology and pediatric sections, we focused only on the family physician unit that was housed in its own wing. With the exception of 1 general surgeon, 1 medicine-pediatrics physician, and 2 general internists, all participating physicians had training in family medicine. One half of the practices were owned by larger hospital-based health systems, whereas one half were independently-owned private practices. Not unexpectedly, most of the system-owned practices were located in suburban or urban settings. Throughout this Midwestern state fee-for-service predominated, with less than 15% of the state’s population enrolled in a health maintenance organization.

The sampling strategy ensured a considerable range in clinical preventive services delivery rates. Rates were calculated from 1,637 patients whose encounters were observed and charts audited; the number of patients per practice ranged from 40 for the solo practice to 272 for the largest practice. All but 2 practices had screening rates of more than 50%, with 3 practices having rates for screening services of more than 80%. Conversely, rates for both smoking counseling and immunizations were less than 50% for most practices. Immunization rates were often strongly affected by low rates of adult tetanus vaccinations because of the large denominator for this service relative to other immunizations. Whereas 2 practices provided clinical preventive services at relatively high rates across screening (89% and 76%), counseling (50% and 63%), and immunizations (60% and 60%), most practices displayed variation across these 3 types of services. For example, a small, low-volume suburban practice that emphasized wellness had the highest rate of counseling smokers (69%) but was among the lowest in administering immunizations (16%) and modest in screening (58%).

A case-by-case summary of key practice characteristics, clinical preventive service delivery rates, and service delivery approaches can be found in tabular form in Supplemental Table 1, available online only at [http://www.annfammed.org/cgi/content/full/3/5/430/DC1](http://www.annfammed.org/cgi/content/full/3/5/430/DC1). In addition, there are expanded case reports of 3 of the practices to provide more detailed context in Supplemental Appendix 2, available online only at [http://www.annfammed.org/cgi/content/full/3/5/430/DC1](http://www.annfammed.org/cgi/content/full/3/5/430/DC1).

The qualitative analyses focused on hypothesis generation and describing and understanding differences in rates of clinical preventive services delivery. The first results section below focuses on understanding the degree of prioritization of preventive services in relation to providing illness care and other competing demands, after which there are a summary of different approaches used by practices and a section on the organizational features that appeared to influence prioritization of clinical preventive services delivery.

### Competing Demands of Care

During in-depth interviews, every physician, nurse practitioner, and physician’s assistant expressed his or her endorsement of preventive services as an integral part of a 3-fold mission that included acute care and chronic illness management. All these clinicians knew of the USPSTF guidelines and were able to recall, with some accuracy, the recommendations for the common screening procedures and immunizations used in this analysis. All believed smoking was a major risk factor and that they had a role in helping patients to stop. Nevertheless, none of these practices delivered clinical preventive services at high rates across the board, although all practices performed these services at some level. For example, the small 4-clinician rural practice with the highest rates on screening (89%) and immunization (60%) had a somewhat lower rate of counseling smokers (50%). Only 1 practice had rates below 50% for all 3 of the calculated services, and most practices tended to do relatively well in 1 or 2 areas, but rarely all 3. Multiple competing demands within each practice were observed to affect these rates. Acute illness visits accounted for most encounters in all these practices, so to be financially viable, practice systems were usually geared toward maximizing efficiency in 10- to 15-minute illness visits. The result, all too commonly, was that the acute issues crowded out prevention. How practices prioritized clinical prevention relative to acute and chronic illness care appeared to be influenced by several other factors discussed in the following paragraphs.

Physicians generally prioritized 1 to several specific services, rather than the full range of recommended clinical preventive services. For example, in a small rural practice in which 1 of the physicians championed smoking counseling, but not other services, the practice had relatively high counseling rates (50%); however, the rates for screening (44%) and immunizations (17%) were quite low. Thus, different specific preventive services compete with each other, in addition to competing with illness visits, for time on the agenda.

In all practices, regardless of approach or system innovation, the clinical encounter with the physician or allied health clinician was the primary locus for delivering preventive services. While several practices had nursing and/or front office staff involved in reminders or patient intake forms, it was up to clinicians to follow through in the examination room. Even a large rural practice with a part-time health educator depended on internal referrals from clinicians. Thus, rates of delivery of preventive services were ultimately tied to clinician decision making during direct care of patients.

Differing patient needs and expectations resulted in separate competing demands. For example, in a cramped, 2-clinician inner-city practice, many patients complained of multiple problems at every visit, and few scheduled health care maintenance (HCM) visits. The lead physician stated that their patients were burdened with diseases of excess and needed better diets, more exercise, and smoking counseling. The clinicians used a mental checklist to discuss regularly what they saw as these patients’ most pressing issues: sexual practices, tine test for tuberculosis, diet, and caffeine use. A focus on women’s health resulted in respectable screening rates (65%); however, tobacco counseling (31%) and immunization (31%) were much lower.

Health care system expectations provided other separate types of competing demands. For example, in a busy 3-clinician practice in a highly competitive suburban community, the clinicians perceived that their hospital system owners expected high volume and referrals. Many of the women in the practice allegedly received their annual preventive care from the system-owned nearby obstetrics and gynecology practice, so comprehensive HCM visits often were not scheduled; however, the practice had no documentation that these services were provided. The net effect was low screening (47%), smoking counseling (36%), and immunization (21%) rates. It is interesting to note that another 2-clinician practice faced with the same system expectations, had similarly low screening (44%) and immunization (17%) rates, but did better in tobacco counseling (50%) because 1 physician championed tobacco counseling.

### Variation in Approaches for Preventive Service Delivery

Many of the office systems and innovations introduced during the past 20 years were in evidence in 1 or more of the practices. They all utilized health care maintenance (HCM) visits, particularly annual visits for birth control, childhood wellness protocols, and mandated school physicals. Many practices had prevention flow sheets, HCM visit forms and protocols, reminder systems, patient history forms, and educational materials targeting preventive issues. In 2 practices, staff completed intake forms or entered patient data during the intake process; however, in most practices, staff members were seen as overworked, and practices hesitated to consider additional prevention-related tasks. Several practices had immunization guidelines posted in examination rooms and nursing stations, and 1 practice gave nursing staff standing orders for mammogram referrals when scheduling HCM visits. Two practices had tried chart alert stickers but found they were rarely used. Most of the practices had introduced prevention flow sheets, but, except for 2 practices, these flow sheets were usually not filled out even when in charts.

Specific approaches did not differentiate practices with higher rates from those with lower rates. The differential adoption of approaches with time led to unique eclectic strategies that incorporated various office system approaches. Examples from different practices include meticulous charting of preventive services and reviewing charts before each encounter, completion of prevention intake forms by office staff, use of electronic medical record reminders, hiring of a part-time health educator, and review of screening services by the office nurse.

Most practices had not articulated systematic strategies for delivering preventive services; instead, they defaulted to a combination of opportunistic delivery during illness visits and periodic HCM visits (mostly for children and adult women). Clinical preventive services in these practices were often provided only in response to patient request or obvious need (eg, a smoker with a respiratory complaint). Whereas some system approaches noted above were present in these practices, their use was sporadic. Few practices had systems to help patients schedule HCM visits, making reliance on HCM visits an ineffective strategy by itself.

### Organizational Features that Support Clinical Preventive Services

A common pattern among practices with higher rates of clinical preventive service delivery was having one or more physician champions who made particular preventive services a practice priority. Many times the champion was the senior physician, but in a large rural practice a new partner just out of residency brought new ideas and enthusiasm that were adopted by senior physicians. Commitment to prevention often had origins in residency, but frequently it stemmed from personal history and formative experiences. For example, several physicians recounted instances in which a patient had cancer diagnosed early as a result of screening, reinforcing the value of screening. Conversely, 1 physician vividly described the impact of failing to test a stool sample for occult blood in a woman who was subsequently had colon cancer diagnosed. Others had personal or family experiences that underscored the importance of stressing prevention (eg, a physician who suffered severe head trauma in a bicycle accident while not wearing a helmet). Having a physician champion appeared to be necessary, but it was not always sufficient for success. For example, in one 3-physician practice, 1 physician who was extremely enthusiastic about implementing strategies for enhancing prevention met resistance and apathy from partners. It should also be noted that in the 8 practices with allied health clinicians, none of those clinicians was observed to have taken a lead in championing prevention at a practice level, even though several were specifically hired by a physician champion to focus on prevention.

Most practices with higher rates of clinical preventive service delivery either found ways to make prevention financially viable or made conscious decisions to compromise incomes to provide clinical preventive services. For example, the practice with the highest screening rates had a Clinical Laboratory Improvement Amendments (CLIA)-certified laboratory that provided an economic self-interest for cholesterol screening. On the other hand, 2 practices reduced patient volume and accepted greatly reduced incomes to have more time with patients, while a couple of practices invested in hiring a health educator or nurse practitioner to enhance clinical preventive service delivery. Economic disincentives were frequently cited as an explanation for less than optimal delivery of clinical preventive services. Most practices faced financial challenges, and clinical preventive services were often perceived as not being reimbursed proportionately to the amount of time expended, particularly when they were opportunistically added into illness visits, which were generally billed according to the major focus of the encounter.

## DISCUSSION

This study provides in-depth examinations of how preventive services are delivered in primary care practices; however, the findings must be interpreted in the context of the study’s limitations. The data were cross-sectional. The study examines clinical prevention only from a practice perspective; it does not examine the broader frame of prevention at the community and population levels. Because the sample is high in rural practices and low in large practices, reflecting the Midwestern location, it is possible that these patterns are not present in other areas of the country. To overcome these limitations, the sampling strategy deliberately selected practices representing a wide range of practice types, geographic locations, and levels of preventive service delivery. The research team spent weeks observing and interviewing in each practice to get as complete a picture as possible of the practice and its history.

All practices embrace prevention as part of a core mission and are doing it to some degree. They are often creative, adaptive, and responsive to local needs and expectations, including those of patients, community, local health care institutions, staff, and physician past experiences—these practices learn. On the other hand, only 7 practices delivered screening services at rates of 65% or above, and even the best only had rates of 69% for smoking counseling and 60% for immunizations.

What are the competing demands related to carrying out clinical prevention? Perceived patient and health system needs and expectations were at times supportive, but often at odds with a practice’s intent to provide clinical preventive services. More importantly, the encounter-centric approach to delivering preventive services assured that prevention was ensnarled in the site of greatest competing demands, including competition with acute and chronic illness care needs, patient concerns, billing issues, and pressure on physician time and productivity. Preventive services are thus squeezed into an already overcrowded clinical encounter so that even preventive services compete with each other for attention. The structure of practices may need redesign as described in the recent Institute of Medicine report 23 and Future of Family Medicine recommendations,24 so some preventive services are accomplished outside the encounter whereas others are integrated into illness visits,13,25 and there is greater use of information systems.

What approaches are used by practices with high vs low rates of clinical preventive service delivery? There was no best approach. Practices tended not to use systems thinking and had not developed systematic strategies for care improvement. Instead, we saw unique, eclectic strategies. These data suggest that future interventions need to raise system-level thinking and awareness and to be individualized. This suggestion is consistent with several current intervention approaches being studied.26–30

What organizational features support clinical preventive services? Having a physician champion and making strategic economic choices were important features shared by many practices with higher clinical preventive service delivery rates. Although this finding might suggest that every practice needs a physician champion for prevention, it might also raise concerns about the limitations of this physician-centeredness. The physician-centeredness evident in these practices could complicate efforts at creating a team approach to care, empowering staff, and enhancing practice-level reflection. This study’s findings also support those who claim that national health care finance reform is needed to obtain better alignment between prevention recommendations and practice reimbursement.23,31,32

A practice systems perspective suggests that efforts at getting diverse clinical offices to adopt a standardized set of processes for implementing preventive services are likely to fail regardless of the quality of the process. Future interventions need to recognize factors leading to practice variability and use this understanding to tailor interventions to the local needs of practices, their patients, and their communities. Recent approaches to systems change using participatory learning, complexity theory, and appreciative inquiry may be helpful.33–35 Future intervention studies should consider these strategies as part of longitudinal designs that allow evolution of change over time.

## Acknowledgments

The authors are grateful to the physicians, staff and patients from the 18 practices, whose participation made this study possible. The authors also wish to thank dedicated work of project staff that spent countless hours collecting, transcribing and managing an extremely large amount of data.

## Footnotes

*   *Conflicts of interest: none reported* 

*   **Funding support:** This study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians.

*   Received for publication June 26, 2004.
*   Revision received April 8, 2005.
*   Accepted for publication May 9, 2005.


*   © 2005 Annals of Family Medicine, Inc.

## REFERENCES

1.  US Department of Health and Human Services. *Healthy People 2010: Understanding and Improving Health.* Washington, DC: US Government Printing Office; 2000.
    
    

2.  US Preventive Services Task Force. *Guide to Clinical Preventive Services: Report of the US Preventive Services Task Force.* 2nd ed. Baltimore, Md: Williams & Wilkins; 1996.
    
    

3.  Kottke TE, Solberg LI, Brekke ML, Cabrera A, Marquez MA. Delivery rates for preventive services in 44 Midwestern clinics. Mayo Clin Proc. 1997;72:515–523.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.4065/72.6.515&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=9179135&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1997XC43200004&link_type=ISI) 

4.  Stange KC, Flocke SA, Goodwin MA, Kelly RB, Zyzanski SJ. Direct observation of rates of preventive service delivery in community family practice. Prev Med. 2000;31:167–176.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1006/pmed.2000.0700&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10938218&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000088742000012&link_type=ISI) 

5.  Greco PJ, Eisenberg JM. Changing physicians’ practices. N Engl J Med. 1993;329:1271–1273.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJM199310213291714&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=8413397&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1993MB98800014&link_type=ISI) 

6.  Grol R, Grimshaw J. Evidence-based implementation of evidence-based medicine. Jt Comm J Qual Improv. 1999;25:503–513.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10522231&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

7.  Stone EG, Morton SC, Hulscher ME, et al. Interventions that increase use of adult immunization and cancer screening services: a meta-analysis. Ann Intern Med. 2002;136:641–651.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11992299&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000175357900001&link_type=ISI) 

8.  Solberg LI, Brekke ML, Fazio CJ, et al. Lessons from experienced guideline implementers: attend to many factors and use multiple strategies. Jt Comm J Qual Improv. 2000;26:171–188.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10749003&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

9.  Thompson RS, Taplin SH, McAfee TA, Mandelson MT, Smith AE. Primary and secondary prevention services in clinical practice. Twenty years’ experience in development, implementation, and evaluation. JAMA. 1995;273:1130–1135.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.1995.03520380066037&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=7707602&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1995QR05900033&link_type=ISI) 

10. Davis D, O’Brien MA, Freemantle N, et al. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.282.9.867&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10478694&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000082272200013&link_type=ISI) 

11. Jaen CR, Stange KC, Nutting PA. Competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract. 1994;38:166–171.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=8308509&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1994MW68000015&link_type=ISI) 

12. Stange KC. The paradox of the parts and the whole in understanding and improving general practice. Int J Qual Health Care. 2002;14:267–268.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaW50cWhjIjtzOjU6InJlc2lkIjtzOjg6IjE0LzQvMjY3IjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS8zLzUvNDMwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13. Grol R. Improving the quality of medical care: building bridges among professional pride, payer profit, and patient satisfaction. JAMA. 2001;286:2578–2585.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.286.20.2578&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11722272&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000172385200032&link_type=ISI) 

14. Miller WL, Crabtree BF, McDaniel R, Stange KC. Understanding change in primary care practice using complexity theory. J Fam Pract. 1998;46:369–376.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=9597994&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000073544100011&link_type=ISI) 

15. Miller WL, McDaniel RR, Jr., Crabtree BF, Stange KC. Practice jazz: understanding variation in family practices using complexity science. J Fam Pract. 2001;50:872–878.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11674890&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000171523500012&link_type=ISI) 

16. Berwick DM. Disseminating innovations in health care. JAMA. 2003;289:1969–1975.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.289.15.1969&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12697800&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000182253900030&link_type=ISI) 

17. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93:1261–1267.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.2105/AJPH.93.8.1261&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12893608&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000184549100023&link_type=ISI) 

18. McIlvain HE, Crabtree BF, Backer EL, Turner PD. Use of office-based smoking cessation activities in family practices. J Fam Pract. 2000;49:1025–1029.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11093569&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

19. Pommerenke FA, Dietrich A. Improving and maintaining preventive services. Part 1: Applying the patient path model. J Fam Pract. 1992;34:86–91.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=1728659&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1992GZ44700019&link_type=ISI) 

20. McIlvain H, Crabtree B, Medder J, Stange KC, Miller WL. Using practice genograms to understand and describe practice configurations. Fam Med. 1998;30:490–496.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=9669161&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

21. Crabtree BF, Miller WL, Stange KC. Understanding practice from the ground up. J Fam Pract. 2001;50:881–887.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11674891&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000171523500013&link_type=ISI) 

22. Miller WL, Crabtree BF. The dance of interpretation. In: Crabtree BF, Miller WL, eds. *Doing Qualitative Research.* Thousand Oaks, Calif: Sage Publications; 1999:127–143.
    
    

23. Berwick DM, Donaldson MS. *Crossing the Quality Chasm: A New Health System for the 21st Century.* Washington, DC: National Academy Press; 2001.
    
    

24. Future of Family Medicine Project Leadership Committee. The future of family medicine: a collaborative project of the family medicine community. Ann Fam Med. 2004;2(Suppl 1):S3–S32.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTI6IjIvc3VwcGxfMS9zMyI7czo0OiJhdG9tIjtzOjIyOiIvYW5uYWxzZm0vMy81LzQzMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

25. Grol R. Personal paper. Beliefs and evidence in changing clinical practice. BMJ. 1997;315:418–421.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTUvNzEwNS80MTgiO3M6NDoiYXRvbSI7czoyMjoiL2FubmFsc2ZtLzMvNS80MzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

26. Solberg LI, Hroscikoski MC, Sperl-Hillen JM, O’Connor PJ, Crabtree BF. Key issues in transforming health care organizations for quality: the case of advanced access. Jt Comm J Qual Saf. 2004;30:15–24.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14738032&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

27. Cohen D, McDaniel RR, Jr., Crabtree BF, et al. A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004;49:155–168; discussion 169–170.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15190858&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221496600005&link_type=ISI) 

28. Stange KC, Goodwin MA, Zyzanski SJ, Dietrich AJ. Sustainability of a practice-individualized preventive service delivery intervention. Am J Prev Med. 2003;25:296–300.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0749-3797(03)00219-8&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14580630&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000186048300005&link_type=ISI) 

29. Feifer C, Ornstein SM. Strategies for increasing adherence to clinical guidelines and improving patient outcomes in small primary care practices. Jt Comm J Qual Saf. 2004;30:432–441.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15357133&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 

30. Stroebel CK, McDaniel RRJ, Crabtree BF, et al. Using complexity science to inform a reflective practice improvement process. Jt Comm J Qual Saf. 2005;31:438–446.
    
    

31. Town R, Kane R, Johnson P, Butler M. Economic incentives and physicians’ delivery of preventive care: a systematic review. Am J Prev Med. 2005;28:234–240.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.amepre.2004.10.013&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15710282&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000226986600014&link_type=ISI) 

32. Grumbach K, Bodenheimer T. A primary care home for Americans: putting the house in order. JAMA. 2002;288:889–893.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.288.7.889&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12186609&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000177513100034&link_type=ISI) 

33. Murray SA, Tapson J, Turnbull L, McCallum J, Little A. Listening to local voices: adapting rapid appraisal to assess health and social needs in general practice. BMJ. 1994;308:698–700.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMDgvNjkzMC82OTgiO3M6NDoiYXRvbSI7czoyMjoiL2FubmFsc2ZtLzMvNS80MzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

34. McDaniel RR, Jr., Jordan ME, Fleeman BF. Surprise, surprise, surprise! A complexity science view of the unexpected. Health Care Manage Rev. 2003;28:266–278.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12940348&link_type=MED&atom=%2Fannalsfm%2F3%2F5%2F430.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000184488900007&link_type=ISI) 

35. Cooperrider DL, Sorensen PF, Yaeger TF, Whitney D. *Appreciative Inquiry: An Emerging Direction for Organization Development.* Champaign, Ill: Stipes; 2001.