Recommendations for a Mixed Methods Approach to Evaluating the Patient-Centered Medical Home
============================================================================================

* Roberta E. Goldman
* Donna R. Parker
* Joanna Brown
* Judith Walker
* Charles B. Eaton
* Jeffrey M. Borkan

## Abstract

**PURPOSE** There is a strong push in the United States to evaluate whether the patient-centered medical home (PCMH) model produces desired results. The explanatory and contextually based questions of how and why PCMH succeeds in different practice settings are often neglected. We report the development of a comprehensive, mixed qualitative-quantitative evaluation set for researchers, policy makers, and clinician groups.

**METHODS** To develop an evaluation set, the Brown Primary Care Transformation Initiative convened a multidisciplinary group of PCMH experts, reviewed the PCMH literature and evaluation strategies, developed key domains for evaluation, and selected or created methods and measures for inclusion.

**RESULTS** The measures and methods in the evaluation set (survey instruments, PCMH meta-measures, patient outcomes, quality measures, qualitative interviews, participant observation, and process evaluation) are meant to be used together. PCMH evaluation must be sufficiently comprehensive to assess and explain both the context of transformation in different primary care practices and the experiences of diverse stakeholders. In addition to commonly assessed patient outcomes, quality, and cost, it is critical to include PCMH components integral to practice culture transformation: patient and family centeredness, authentic patient activation, mutual trust among practice employees and patients, and transparency, joy, and collaboration in delivering and receiving care in a changing environment.

**CONCLUSIONS** This evaluation set offers a comprehensive methodology to enable understanding of how PCMH transformation occurs in different practice settings. This approach can foster insights about how transformation affects critical outcomes to achieve meaningful, patient-centered, high-quality, and cost-effective sustainable change among diverse primary care practices.

*   patient-centered medical home
*   evaluation
*   practice transformation
*   context
*   mixed methods
*   qualitative methods
*   quantitative methods

## INTRODUCTION

The patient-centered medical home (PCMH) movement on the American health care scene is relatively new, though it has roots in both pediatrics and general practice.1–3 The conceptualization of PCMH began with a broad notion of transforming primary care practices by combining the best primary care attributes with new means of structuring patient care, enhancing patient engagement and care coordination, achieving improved health outcomes, providing a better patient experience of care, improving efficiency and use of health information technology, and, ultimately, reducing costs.4–9 Stakeholders disagree about exactly what a medical home is, to what transformation should aspire, and how “medical home-ness” should be assessed.10–13 Evaluations of such new organizational structures were intended to consider quality, cost, and experience of patients, clinicians, and medical staff,5,14,15 though the elements most pertinent to fostering payers’ support of these initiatives were increased efficiency and lowered costs.16

The informative richness of evaluation efforts can be compromised if their strategies have too narrow a focus. Practices’ participation in payer- or government-driven PCMH involves high stakes, with payment dependent upon required performance reporting. In this environment, evaluators tend to focus data collection on a constrained set of concrete quality measures to limit the reporting burden on primary care sites and to allow for an objective, comparative assessment of progress.17–19 Moreover, there has been a reluctance to collect data that are not directly tied to incentives, such as the patient, family,12,17 and clinician’s20 experience of care or the transformation process itself.21 Often neglected in evaluations is a qualitative exploration of the mediating mechanisms and modifying conditions22 of transformation. Qualitative inquiry into these conditions helps explain and provide context for how PCMH implementation can be variable and affected by the political, organizational, philosophical, cultural, community, financial, practice, clinician, staff, and patient factors within the highly stressed atmosphere of primary care delivery.19,21,23

Efforts to evaluate the PCMH model in the United States have occurred within the rapidly evolving environment of PCMH transformation. Diverse stakeholders are enthusiastic about the potential of the PCMH to improve health care delivery and reduce disparities.11,24–28 Recent reports point to the success of PCMH initiatives in achieving various subsets of desired aims.18,21,23,29–32 Yet, as Hoff et al note,23 there is “conceptual sponginess” to the PCMH for which concepts and labels are variously defined and differentially applied. As a result, the field—possibly prematurely—is seeking a specified set of clinical benchmarks19,21,33,34 on which to focus to determine whether PCMH works. Such a focus leaves behind the important questions of how and why PCMH may work in particular contexts, and in what ways a practice has or has not undergone fundamental and enduring transformation in the way it functions.21,23,35 Given the variable manifestations inherent in the politics, economics, and organizational contexts of real-world primary care practice,21,35,36 it is important to consider whether ideal transformation processes can be universally implemented and whether at this stage they can be evaluated with universally standardized measures.23,33,36–42 Furthermore, it is important to consider what constitutes a comprehensive set of ingredients for successful transformation and how to evaluate the processes contributing to its success.4,43

To address these gaps, we set out to elucidate the domains and evaluation strategies that constitute a comprehensive methodology to determine how transformation occurs within the context of practice and to explain why such transformation happens. We wanted to establish a set of metrics and methods that, when used together, address the many contextual issues involved in PCMH transformation. The set would be designed for use by researchers, policy makers, and primary care clinician groups when conducting and evaluating PCMH initiatives. Their findings could then be used to monitor change within practices, compare practices’ abilities to promote and sustain change, and to understand what conditions lead to stronger practice transformation.

## METHODS

In 2011, the Brown Primary Care Transformation Initiative at Brown University’s Department of Family Medicine convened a PCMH evaluation think tank to begin development of a comprehensive PCMH evaluation data collection set that could be implemented feasibly. The think tank included 28 national and regional authorities in PCMH evaluation from family medicine, internal medicine, pediatrics, nursing, epidemiology, anthropology, sociology, psychology, health insurance, e-health, health systems transformation, the Rhode Island Department of Health, and other public health entities.

After the think tank convened, our Brown Primary Care Transformation Initiative team engaged in an immersion-crystallization process of qualitative data analysis,44 meeting repeatedly to review the think tank notes, recordings, and transcripts. Before and after the think tank convened, 1 coauthor (D.R.P.) used the MEDLINE database to identify and review for quality and relevancy the English language articles addressing PCMH evaluation. Based on the literature findings and the published and online evaluation tools and methods, we selected tools and adapted and created others to compile a contextually comprehensive, qualitative and quantitative mixed methods PCMH evaluation set for use at baseline and follow-up. Our objective was to assemble an evaluation set that could be implemented according to PCMH project needs to uncover and explain how and why transformation occurs (or is impeded) within the cultural and philosophical construct of primary care practice and care seeking.

## RESULTS

### Compiling a Feasible, Contextually Comprehensive Evaluation Set

The organizing framework outlined by think tank participants called for a comprehensive evaluation set that is explanatory, attends to the context of transformation, and elicits the experiences of the diverse stakeholders’ (patients, parents or caregivers, clinicians, staff) broad range of issues involved in PCMH transformation. Mixed methods were promoted to capture the requisite baseline and follow-up data: qualitative interviews, participant-observation, focus groups, surveys, other quantitative measures, and patient outcomes. The strategies included in the set assess practice workflow and interpersonal communication, practice baseline culture, and culture transformation; patient and family centeredness and authentic patient activation; enhanced communication and trust among practice employees and among employees and patients; transparency; meaning, joy, and collaboration in delivering and receiving health care; and community integration. The evaluation set is displayed in Tables 1 through 8. All patient instruments are available in English and Spanish because 12.9% of the US population spoke Spanish at home in 2011, and of those speakers, 5.9% reported they speak English less than very well.48

View this table:
[Table 1](http://www.annfammed.org/content/13/2/168/T1)

Table 1 
Quantitative PCMH Evaluation Measures: Practice, Clinician, and Staff Survey Instruments

### Quantitative Measures

We selected or adapted publicly available written survey instruments that include the critical PCMH components we identified (see Supplemental Appendixes 1 and 2, [http://www.annfammed.org/content/13/2/168/suppl/DC1](http://www.annfammed.org/content/13/2/168/suppl/DC1) for adapted instruments). Our goal was to include instruments that we believe, when used together, are comprehensive yet manageable. Instruments in the evaluation set address practice, clinician, and staff demographics; job satisfaction; burnout; and clinician support for patient activation. Our inclusion of 3 patient instruments results from a careful selection of those we consider to be the most suitable for PCMH evaluation; they are relatively quick to complete (15 to 30 minutes to complete all 3), and they address different measures that together provide a contextually based view of the patient’s perspective.

Practices seeking PCMH accreditation are required to use the National Committee for Quality Assurance (NCQA) accreditation measure.49 We suggest practices also use the practice self-evaluation Patient-Centered Medical Home Assessment (PCMH-A), an excellent tool for assessing baseline needs and monitoring transformation.50 Hospital utilization can be measured by emergency department and hospital admissions and include ambulatory care–sensitive conditions (medical problems that might be prevented or can be treated outside a hospital).51 Quality is measured with specific clinical benchmarks within broad areas of care (eg, diabetes would have performance goals for hemoglobin A1c levels, documented eye examinations, etc). These benchmarks are variously defined among PCMH projects according to local requirements and data availability, so only suggested areas are listed in Table 5 (Supplemental Appendixes 1–5, [http://www.annfammed.org/content/13/2/168/suppl/DC1](http://www.annfammed.org/content/13/2/168/suppl/DC1)).52

View this table:
[Table 2](http://www.annfammed.org/content/13/2/168/T2)

Table 2 
Quantitative PCMH Evaluation Measures: Patient Survey Instruments

View this table:
[Table 3](http://www.annfammed.org/content/13/2/168/T3)

Table 3 
Quantitative PCMH Evaluation Measures: Examples of Meta-Measures

View this table:
[Table 4](http://www.annfammed.org/content/13/2/168/T4)

Table 4 
Hospital Utilization Measures

View this table:
[Table 5](http://www.annfammed.org/content/13/2/168/T5)

Table 5 
Examples of Clinical Benchmark Categories

### Qualitative Data Collection

Qualitative data complement quantitative data to obtain a contextualized understanding of what occurs in practices that may influence transformation and to help explain quantitative findings. Several types of participant observation allow evaluators to compare interview and survey findings with direct observation of how the practice functions. Observation templates guide trained participant-observers in writing field notes regarding the office environment. Patient pathways are conducted where an evaluator accompanies patients from registration through checkout, and for staff pathways the evaluator shadows individual staff during typical workdays (Table 6).

View this table:
[Table 6](http://www.annfammed.org/content/13/2/168/T6)

Table 6 
PCMH Qualitative Observational Evaluation Methods

In-person individual interviews with clinicians, staff, and patients should be conducted by trained interviewers using semistructured question guides. Core questions are supplemented with spontaneous probes and follow-up questions to elicit information in participants’ own words about how they think about primary care and PCMH transformation (Table 7) (Supplemental Appendixes 6–12, [http://www.annfammed.org/content/13/2/168/suppl/DC1](http://www.annfammed.org/content/13/2/168/suppl/DC1)).

View this table:
[Table 7](http://www.annfammed.org/content/13/2/168/T7)

Table 7 
PCMH Qualitative Interview Evaluation Methods

### Implementation Process Evaluation Methods

The qualitative methods described above elicit information on transformation from clinicians, staff, and patients, who are internal to the practice. To understand practice evolution as part of an externally facilitated initiative, it is helpful to document the perceptions of the external team. Such process evaluation documents additional contextual and explanatory factors that may influence the quality and outcomes of a PCMH initiative. Our evaluation set includes guides for periodic reflection and progress notes for evaluators and facilitators and twice-yearly focus groups with these project staff. Written reflections allow staff to delve deeply into specific issues that feel important to their work and the transformation process.36 Focus groups, led by a trained focus group moderator, promote group interaction around matters concerning the overall transformation initiative beyond those that commonly dominate staff meetings (Table 8, and Supplemental Appendixes 13 and 14, [http://www.annfammed.org/content/13/2/168/suppl/DC1](http://www.annfammed.org/content/13/2/168/suppl/DC1)).

View this table:
[Table 8](http://www.annfammed.org/content/13/2/168/T8)

Table 8 
PCMH Implementation Process Evaluation Methods

### Application of Evaluation Data to Promote Practice Transformation

Individualized practice reports can be created by analyzing baseline and follow-up data from all methods. Reports summarize strengths and challenges of the PCMH transformation domains. We recommend that facilitation staff meet in person with practice representatives to discuss findings and devise transformation goals and strategies. Care must be taken in the reports to maintain respondent confidentiality by not linking comments with specific employees or job roles. Process evaluation data can be used to modify facilitation strategies during the transformation and, at the completion of the initiative, to better understand the impact of practice transformation on staff interactions.

## DISCUSSION

PCMH researchers have noted that a comprehensive evaluation must be developed to understand and explain not only how and why practices transform but also how stakeholders experience the transformation.19,23 Enhanced insights are needed so the field can develop recommendations for practice facilitation that best achieves the goals of the PCMH model. Mixed methods approaches that involve diverse stakeholders may be best suited to revealing these complex insights. Taken together, findings from each of these “different ways of knowing”53 help explain the findings from the others and lead to conclusions that would not be reached by any single approach.54–57

Often missing in PCMH evaluation is the analysis of the context and process of practice transformation efforts—data that help explain why anticipated changes were or were not achieved.35,36 Comprehensive use of mixed methods evaluations can harness the power of different types of inquiry and resulting data. Surveys capture only part of the story, and individuals in practices may be too embedded within their cultural milieu to describe accurately the transformation processes, facilitators, and barriers. Similarly, evaluating performance through the electronic health record data alone may misrepresent the actual quality of care provided by the practice.58 Despite our attempt to design an evaluation set that can be feasibly implemented, however, limitations may include lack of staff trained to implement the multiple methods, the burden of data collection, and the unpredictable consequences of change. The contextually comprehensive approach we describe may provide insights into some of the more fundamental changes that are needed to drive transformation toward the joy and enhanced quality outcomes and satisfaction that many are hoping to foster in the health care experienced by patients, clinicians, and staff.59

## Acknowledgements

The PCMH Evaluation Think Tank members provided insightful commentaries during the think tank discussions. We are grateful to the numerous conference attendees who provided thoughtful comments on our presentation of parts of this work at several conferences.

## Footnotes

*   Conflicts of interest: authors report none.

*   **Funding support:** This work was funded by a grant from the US Department of Health and Human Services, Health Resources and Services Administration, HRSA grant #1D54HP20675-01-00.

*   **Prior presentations:** Portions of the material included in this article were presented in:
    
    Borkan JM, Eaton C, Goldman RE. Anthropological and mixed-methods approaches to PCMH evaluation. Presented as part of a preconference workshop at: North American Primary Care Research Group Annual Meeting; December, 2012; New Orleans, LA.
    
    Goldman RE, Borkan JM. Anthropological approaches: using qualitative methods to uncover unexpected insights about the implementation and outcomes of patient-centered medical home models. Presented at: AcademyHealth Annual Research Meeting; June, 2013; Baltimore, MD.
    
    Goldman R, Borkan J. Understanding PCMH evaluation: modeling for learners and teaching anthropological mixed-methods approaches. Presented at: Society of Teachers of Family Medicine Annual Meeting; May, 2014; San Antonio, TX.
    
    Borkan JM, Dickinson P, Goldman RE, Grumbach K, Brown JD, Magill M. Multiple methodologies for evaluating the patient centered medical home and other practice transformation efforts: getting to a gold standard. Presented at: North American Primary Care Research Group Annual Meeting; November, 2014; New York, NY.

*   **Supplementary materials:** Available at [http://www.AnnFamMed.org/content/13/2/168/suppl/DC1/](http://www.AnnFamMed.org/content/13/2/168/suppl/DC1/)

*   Received for publication July 14, 2014.
*   Revision received December 16, 2014.
*   Accepted for publication January 5, 2015.


*   © 2015 Annals of Family Medicine, Inc.

## References

1.  American Academy of Pediatrics Ad Hoc Task Force on Definition of the Medical Home: The medical home. Pediatrics. 1992;90(5):774.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiOTAvNS83NzQiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Medical Home Initiatives for Children With Special Needs Project Advisory Committee. American Academy of Pediatrics. The medical home. Pediatrics. 2002;110(1 Pt 1):184–186.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTEwLzEvMTg0IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  McPhee J. Heirs of General Practice. New York, NY: Farrar, Strauss & Giroux; 1984.
    
    

4.  American Academy of Family Physicians (AAFP). Joint principles of the patient-centered medical home. [http://www.aafp.org/dam/AAFP/documents/practice_management/pcmh/initiatives/PCMHJoint.pdf](http://www.aafp.org/dam/AAFP/documents/practice_management/pcmh/initiatives/PCMHJoint.pdf). Published Feb 2007. Accessed Feb 28, 2013.
    
    

5.  Nutting PA, Crabtree BF, Miller WL, Stange KC, Stewart E, Jaén C. Transforming physician practices to patient-centered medical homes: lessons from the national demonstration project. Health Aff (Millwood). 2011;30(3):439–445.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMwLzMvNDM5IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

6.  Institute for Health Improvement. The Triple Aim Initiative. [http://www.ihi.org/offerings/Initiatives/TripleAim/Pages/default.aspx](http://www.ihi.org/offerings/Initiatives/TripleAim/Pages/default.aspx). Accessed Jul 19, 2013.
    
    

7.  Peikes D, Genevro JL, Scholle SH, Torda P. The patient-centered medical home: strategies to put patients at the center of primary care. [http://pcmh.ahrq.gov/page/patient-centered-medical-home-strategies-put-patients-center-primary-care](http://pcmh.ahrq.gov/page/patient-centered-medical-home-strategies-put-patients-center-primary-care). Published Feb 2011. Accessed Aug 1, 2013.
    
    

8.  Scholle SH, Torda P, Peikes D, et al. Engaging patients and families in the medical home. [http://pcmh.ahrq.gov/sites/default/files/attachments/Engaging%20Patients%20and%20Families%20in%20the%20Medical%20Home.pdf](http://pcmh.ahrq.gov/sites/default/files/attachments/Engaging%20Patients%20and%20Families%20in%20the%20Medical%20Home.pdf). Published Jun 2010. Accessed Feb 27, 2013.
    
    

9.  Sherman B, Parry T, Hanson J. Patient-centered medical home performance metrics for employers. [https://www.pcpcc.org/guide/medical-home-performance-metrics-employers](https://www.pcpcc.org/guide/medical-home-performance-metrics-employers). Published Jan 2011. Accessed Feb 27, 2013.
    
    

10. Vest JR, Bolin JN, Miller TR, Gamm LD, Siegrist TE, Martinez LE. Medical homes: “where you stand on definitions depends on where you sit”. Med Care Res Rev. 2010;67(4):393–411.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BtY3IiO3M6NToicmVzaWQiO3M6ODoiNjcvNC8zOTMiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

11. Berenson R, Devers K, Burton K. Will the Patient-Centered Medical Home Transform the Delivery Of Health Care? Washington, DC: Urban Institute; 2011.
    
    

12. Hoff T, Weller W, DePuccio M. The patient-centered medical home: a review of recent research. Med Care Res Rev. 2012;69(6):619–644.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BtY3IiO3M6NToicmVzaWQiO3M6ODoiNjkvNi82MTkiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13. Wagner EH, Coleman K, Reid RJ, Phillips K, Abrams MK, Sugarman JR. The changes involved in patient-centered medical home transformation. Prim Care. 2012;39(2):241–259.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.pop.2012.03.002&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22608865&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

14. Gray BM, Weng W, Holmboe ES. An assessment of patient-based and practice infrastructure-based measures of the patient-centered medical home: do we need to ask the patient? Health Serv Res. 2012;47(1 Pt 1):4–21.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1475-6773.2011.01302.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22092245&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

15. Stange KC, Miller WL, Nutting PA, et al. Context for understanding the National Demonstration Project and the patient-centered medical home. Ann Fam Med. 2010;8(Suppl 1):S2–S8; S92.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTI6IjgvU3VwcGxfMS9TMiI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

16. Rosenthal MB, Beckman HB, Forrest DD, Huang ES, Landon BE, Lewis S. Will the patient-centered medical home improve efficiency and reduce costs of care? A measurement and research agenda. Med Care Res Rev. 2010;67(4):476–484.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo1OiJzcG1jciI7czo1OiJyZXNpZCI7czo4OiI2Ny80LzQ3NiI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Burton R, Devers K, Berenson R. Patient-centered medical home recognition tools: a comparison of ten surveys’ content and operational details. [http://www.urban.org/uploadedpdf/412338-patient-centered-medical-home-rec-tools.pdf](http://www.urban.org/uploadedpdf/412338-patient-centered-medical-home-rec-tools.pdf). Published Mar 2012. Accessed Jul 29, 2013.
    
    

18. Rosenthal MB, Friedberg MW, Singer SJ, Eastman D, Li Z, Schneider EC. Effect of a multipayer patient-centered medical home on health care utilization and quality: the Rhode Island chronic care sustainability initiative pilot program. JAMA Intern Med. 2013;173(20):1907–1913.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jamainternmed.2013.10063&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24018613&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

19. Bitton A, Schwartz GR, Stewart EE, et al. Off the hamster wheel? Qualitative evaluation of a payment-linked patient-centered medical home (PCMH) pilot. Milbank Q. 2012;90(3):484–515.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1468-0009.2012.00672.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22985279&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000308943300003&link_type=ISI) 

20. Nielsen M, Olayiwola JN, Grundy P, Grumbach K. The patient-centered medical home’s impact on cost & quality: an annual update of the evidence, 2012–2013. [https://www.pcpcc.org/resource/medical-homes-impact-cost-quality](https://www.pcpcc.org/resource/medical-homes-impact-cost-quality). Accessed Jan 15, 2014.
    
    

21. Alexander JA, Bae D. Does the patient-centred medical home work? A critical synthesis of research on patient-centred medical homes and patient-related outcomes. Health Serv Manage Res. 2012;25(2):51–59.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Boc20iO3M6NToicmVzaWQiO3M6NzoiMjUvMi81MSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

22. Sorensen G, Emmons K, Hunt MK, et al. Model for incorporating social context in health behavior interventions: applications for cancer prevention for working-class, multiethnic populations. Prev Med. 2003;37(3):188–197.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0091-7435(03)00111-7&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12914824&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000184761300002&link_type=ISI) 

23. Hoff T. Practice Under Pressure. New Brunswick, NJ: Rutgers University Press; 2010.
    
    

24. Romeo S. The promise of the medical home: five essential principles for success. [http://medicaleconomics.modernmedicine.com/medical-economics/news/modernmedicine/modern-medicine-feature-articles/promise-medical-home](http://medicaleconomics.modernmedicine.com/medical-economics/news/modernmedicine/modern-medicine-feature-articles/promise-medical-home). Published Oct 25, 2011. Accessed Nov 26, 2013.
    
    

25. Mosquera M. Patient centered medical home pilots show promise. Healthcare Finance News. Sep 13, 2012.
    
    

26. Newman M. Patient-centered medical home program trims expected health care costs by $98 million in second year. CareFirst. [http://www.prweb.com/releases/2013/6/prweb10805110.htm](http://www.prweb.com/releases/2013/6/prweb10805110.htm). Published Jun 6, 2013. Accessed Nov 26, 2013.
    
    

27. Braddock CH III., Snyder L, Neubauer RL, Fischer GS; American College of Physicians Ethics, Professionalism and Human Rights Committee and The Society of General Internal Medicine Ethics Committee. The patient-centered medical home: an ethical analysis of principles and practice. J Gen Intern Med. 2013;28(1):141–146.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-012-2170-x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22829295&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

28. Wong W, Anderson K, Dankwa-Mullan I, et al. The Patient-centered medical home: A path toward health equity. 2012 September. [http://iom.edu/Global/Perspectives/2012/PatientCenteredMedicalHome.aspx](http://iom.edu/Global/Perspectives/2012/PatientCenteredMedicalHome.aspx). Accessed Nov 26, 2013.
    
    

29. Reid RJ, Coleman K, Johnson EA, et al. The Group Health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood). 2010;29(5):835–843.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjI5LzUvODM1IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

30. Nielsen M, Langner B, Zema C, et al. Benefits of implementing the primary care medical home: a review of cost & quality results. 2012. [http://www.pcpcc.org/guide/benefits-implementing-primary-care-medical-home](http://www.pcpcc.org/guide/benefits-implementing-primary-care-medical-home). Accessed Jan 25, 2013.
    
    

31. Peikes D, Zutshi A, Genevro JL, Parchman ML, Meyers DS. Early evaluations of the medical home: building on a promising start. Am J Manag Care. 2012;18(2):105–116.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22435838&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000301089200004&link_type=ISI) 

32. Maeng DD, Graham J, Graf TR, et al. Reducing long-term cost by transforming primary care: evidence from Geisinger’s medical home model. Am J Manag Care. 2012;18(3):149–155.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22435908&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000308442600003&link_type=ISI) 

33. Hoff T. The patient-centered medical home: what we need to know more about. Med Care Res Rev. 2010;67(4):383–392.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo1OiJzcG1jciI7czo1OiJyZXNpZCI7czo4OiI2Ny80LzM4MyI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

34. Crabtree BF, Chase SM, Wise CG, et al. Evaluation of patient centered medical home practice transformation initiatives. Med Care. 2011;49(1):10–16.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1097/MLR.0b013e3181f80766&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=21079525&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000285407100003&link_type=ISI) 

35. Stange KC, Glasgow R. Considering and reporting important contextual factors in research on the patient-centered medical home. Rockville, MD: Agency for Healthcare Research and Quality; June 2013. [http://www.pcmh.ahrq.gov/sites/default/files/attachments/ContextualFactors.pdf](http://www.pcmh.ahrq.gov/sites/default/files/attachments/ContextualFactors.pdf). Accessed Nov 26, 2013.
    
    

36. Tomoaia-Cotisel A, Scammon DL, Waitzman NJ, et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann Fam Med. 2013;11(Suppl 1):S115–S123.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTU6IjExL1N1cHBsXzEvUzExNSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

37. DePuccio MJ, Hoff TJ. Medical home interventions and quality outcomes for older adults: a systematic review. Qual Manag Health Care. 2013;22(4):327–340.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24088881&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

38. Betbeze P. How patients view your medical home. Health Leaders Media. 2012;17(8):36–39.
    
    

39. Hoff T. The shaky foundation of the patient-centered medical home. Am J Manag Care. 2010;16(6):e134–e136.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20536270&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000279023400008&link_type=ISI) 

40. Alexander JA, Paustian M, Wise CG, et al. Assessment and measurement of patient-centered medical home implementation: the BCBSM experience. Ann Fam Med. 2013;11(Suppl 1):S74–S81.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjExL1N1cHBsXzEvUzc0IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

41. McNellis RJ, Genevro JL, Meyers DS. Lessons learned from the study of primary care transformation. Ann Fam Med. 2013;11(Suppl 1):S1–S5.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjExL1N1cHBsXzEvUzEiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

42. Rittenhouse DR, Schmidt L, Wu K, Wiley J. Contrasting trajectories of change in primary care clinics: lessons from New Orleans safety net. Ann Fam Med. 2013;11(Suppl 1):S60–S67.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjExL1N1cHBsXzEvUzYwIjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

43. Stange KC, Nutting PA, Miller WL, et al. Defining and measuring the patient-centered medical home. J Gen Intern Med. 2010;25(6):601–612.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-010-1291-3&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20467909&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000277712200022&link_type=ISI) 

44. 1.  Miller W, 
    2.  Crabtree B
    
    Borkan J. Immersion/Crystallization. In: Miller W, Crabtree B, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage Publications; 1999:179–194.
    
    

45. Jaen CR, Crabtree BF, Palmer RF, et al. Methods for evaluating practice change toward a patient-centered medical home. Ann Fam Med. 2010;8(Suppl 1):S9–S20; S92.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTI6IjgvU3VwcGxfMS9TOSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTMvMi8xNjguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

46. Nutting PA, Crabtree BF, Stewart EE, et al. Effect of facilitation on practice outcomes in the National Demonstration Project model of the patient-centered medical home. Ann Fam Med. 2010;8(Suppl 1):S33–S44; S92.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjgvU3VwcGxfMS9TMzMiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

47. Maslach C, Jackson S, Leiter M. Maslach Burnout Inventory. 3rd ed. Palo Alto, CA: Consulting Psychologists Press; 1996. [http://www.mindgarden.com/products/mbi.htm](http://www.mindgarden.com/products/mbi.htm). Accessed Oct 29, 2014.
    
    

48. Ryan C. Language use in the United States: 2011, American community survey reports. 2013. [http://www.census.gov/prod/2013pubs/acs-22.pdf?eml=gd](http://www.census.gov/prod/2013pubs/acs-22.pdf?eml=gd). Accessed Oct 29, 2014.
    
    

49. NCQA. Patient-Centered Medical Home Recognition 2011 Application. [http://www.ncqa.org/Programs/Recognition/PatientCenteredMedicalHomePCMH.aspx](http://www.ncqa.org/Programs/Recognition/PatientCenteredMedicalHomePCMH.aspx). Accessed Jan 14, 2014.
    
    

50. Daniel DM, Wagner EH, Coleman K, et al. Assessing progress toward becoming a patient-centered medical home: an assessment tool for practice transformation. Health Serv Res. 2013;48(6 Pt 1):1879–1897.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/1475-6773.12111&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24138593&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

51. Brown AD, Goldacre MJ, Hicks N, et al. Hospitalization for ambulatory care-sensitive conditions: a method for comparative access and quality studies using routinely collected statistics. Can J Public Health. 2001;92(2):155–159.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11338156&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000168245600019&link_type=ISI) 

52. Bates DW, Bitton A. The future of health information technology in the patient-centered medical home. Health Aff (Millwood). 2010;29(4):614–621.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjI5LzQvNjE0IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xMy8yLzE2OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

53. Stange KC, Miller WL, McWhinney I. Developing the knowledge base of family practice. Fam Med. 2001;33(4):286–297.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11322522&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000167959600010&link_type=ISI) 

54. Scammon DL, Tomoaia-Cotisel A, Day RL, et al. Connecting the dots and merging meaning: using mixed methods to study primary care delivery transformation. Health Serv Res. 2013;48(6 Pt 2):2181–2207.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/1475-6773.12114&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24279836&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

55. Wisdom J, Creswell J. Mixed methods: integrating quantitative and qualitative data collection and analysis while studying patient-centered medical home models. [http://pcmh.ahrq.gov/page/mixed-methods-integrating-quantitative-and-qualitative-data-collection-and-analysis-while#h=13-0028-EF](http://pcmh.ahrq.gov/page/mixed-methods-integrating-quantitative-and-qualitative-data-collection-and-analysis-while#h=13-0028-EF). Published Mar 2013. Accessed Dec 2, 2013.
    
    

56. Goldman R, Borkan J. Anthropological approaches: uncovering unexpected insights about the implementation and outcomes of patient-centered medical home models. [http://pcmh.ahrq.gov/page/anthropological-approaches-uncovering-unexpected-insights-about-implementation-and-outcomes#h=13-0022-ef](http://pcmh.ahrq.gov/page/anthropological-approaches-uncovering-unexpected-insights-about-implementation-and-outcomes#h=13-0022-ef). Published March 2013. Accessed Dec 2, 2013.
    
    

57. Miller WL, Crabtree BF, Harrison MI, Fennell ML. Integrating mixed methods in health services and delivery system research. Health Serv Res. 2013;48(6 Pt 2):2125–2133.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/1475-6773.12123&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24279834&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

58. Danford CP, Navar-Boggan AM, Stafford J, McCarver C, Peterson ED, Wang TY. The feasibility and accuracy of evaluating lipid management performance metrics using an electronic health record. Am Heart J. 2013;166(4):701–708.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.ahj.2013.07.024&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24093850&link_type=MED&atom=%2Fannalsfm%2F13%2F2%2F168.atom) 

59. Sinsky CA, Willard-Grace R, Schutzbank AM, Sinsky TA, Margolius D, Bodenheimer T. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11(3):272–278.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTEvMy8yNzIiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzEzLzIvMTY4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)