Recommendations for a Mixed Methods Approach to Evaluating the Patient-Centered Medical Home

Roberta E. Goldman; Donna R. Parker; Joanna Brown; Judith Walker; Charles B. Eaton; Jeffrey M. Borkan

doi:10.1370/afm.1765

Article Figures & Data

Tables

Additional Files

View popup

Table 1

Quantitative PCMH Evaluation Measures: Practice, Clinician, and Staff Survey Instruments

Measurement Tool	Domains	Source, Version, Purpose, Availability
Baseline Practice Survey⁴⁵	Demographics and practice information	Original NDP questionnaire (96 items). Adapted questionnaire by BPCTI (27 items). Provides information for NCQA and meaningful use. No cost Supplemental Appendix 1, http://www.annfammed.org/content/13/2/168/suppl/DC1
PCMH Implementation Survey^45,46	Change capacity: teamwork, work environment, culture, trust, communication	Modified by BPCTI from 2 longer NDP scales: NDP Practice Adaptive Reserve (23 items), Modified Scale = 23 items (reflection item deleted and confidence item added), and Practice Environmental Checklist (123 items). Combined modified scale = 32 items. No cost Supplemental Appendix 2, http://www.annfammed.org/content/13/2/168/suppl/DC1
Practice Demographic Questionnaire for clinicians	Demographics of individual practice clinicians	BPCTI (9 items). For physicians, NPs, and PAs. No cost Supplemental Appendix 3, http://www.annfammed.org/content/13/2/168/suppl/DC1
Practice Demographic Questionnaire for staff	Demographics of individual practice staff	BPCTI (5 items). For nurses, medical assistants, receptionists, and other staff. No cost Supplemental Appendix 4, http://www.annfammed.org/content/13/2/168/suppl/DC1
The Clinician Activation Measure assessment (CS-PAM)	Clinician support for and beliefs about patient activation and patient self-management	Validated tool from Insignia Health (13 items). Requires purchase of a license http://www.insigniahealth.com/solutions/clinician-activation-measure
Maslach Burnout Inventory⁴⁷	Measure of burnout: emotional exhaustion, depersonalization, personal accomplishment	MBI-HSS (22 items). For clinicians and staff. Available in 25 languages, free with purchase of license for English version. Website disclaimer gives no warranty for translation quality http://www.mindgarden.com/products/mbi.htm

BPCTI = Brown Primary Care Transformation Initiative; MBI = Maslach Burnout Inventory; MBI-HSS = MBI-Human Services Survey; NCQA = National Center for Quality Assurance; NDP = National Demonstration Project; NP = nurse practitioner; PA = physician assistant; PCMH = patient-centered medical home.

View popup

Table 2

Quantitative PCMH Evaluation Measures: Patient Survey Instruments

Measurement Tool	Domains	Source, Version, Purpose, Availability
Patient Activation Measure (PAM)	Patient activation regarding patients’ knowledge, skills, and confidence for self-management	Validated tool from Insignia Health (13 items) to inform patient activation efforts English and Spanish. Requires purchase of a license http://www.insigniahealth.com/solutions/patient-activation-measure
HRSA Patient Satisfaction Survey	Adult experiences of care at the practice	HRSA (32 items). English and Spanish. No cost. HRSA version 12/25/2012 Supplemental Appendix 5, http://www.annfammed.org/content/13/2/168/suppl/DC1
Interpersonal Process of Care Survey: Short Form (IPC-18)	Communication, patient-centered decision making, and interpersonal style	University of California, San Francisco Department of Medicine, Center for Aging in Diverse Communities (18-item short form). For patients from diverse racial/ethnic groups to describe disparities in interpersonal care, predict patient outcomes, and examine outcomes disparity reduction efforts. English and Spanish. No cost http://dgim.ucsf.edu/cadc/mm/ipcare.html

HRSA = Health Resources and Services Administration; PCMH = patient-centered medical home.

View popup

Table 3

Quantitative PCMH Evaluation Measures: Examples of Meta-Measures

Measurement Tool	Domains	Source, Version, Purpose, Availability
Physician Practice Connections – Patient-Centered Medical Home (PPC-PCMH)	9 Standards: access and communication, patient tracking and registry functions, care management, self-management support, electronic prescribing, test tracking, referral tracking, performance reporting and improvement, advanced electronic communication	NCQA revised standards for January 1, 2014. Most commonly used measure of PCMH accreditation http://www.ncqa.org/Programs/Recognition/Practices/PatientCenteredMedicalHomePCMH.aspx
Meaningful Use	3 Core areas: data capture and sharing, advancing clinical processes, achieving improved patient outcomes	Standards defined by the CMS Incentive Programs to regulate use of electronic health records. Eligible providers and hospitals earn incentive payments by meeting criteria http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Meaningful_Use.html
Medical Home Implementation Quotient (MHIQ)	9 Modules: patient-centered medical home, practice management, health information technology, quality and safety, practice-based team care, care coordination, practice-based services, access to care and information, care management	TransforMED. Self-assessment tool to help a practice learn more about the medical home model and gauge status within the medical home continuum http://www.transformed.com/mhiq/welcome.cfm
Patient-Centered Medical Home Assessment (PCMH-A)	8 Change concept areas: engaged leadership, quality improvement strategy, empanelment, continuous team-based healing relationships, organized, evidence-based care, patient-centered interactions, enhanced access, care coordination	MacColl Center for Healthcare Innovation. Helps practices gauge progress implementing PCMH change concepts. Tested by 65 sites participating in the Safety Net Medical Home Initiative http://bsmod.dom.wustl.edu/documents/PCMH-A_SNMHI_080410.pdf

CMS = Centers for Medicare & Medicaid Services; NCQA = National Committee for Quality Assurance; PCMH = patient-centered medical home.

View popup

Table 4

Hospital Utilization Measures

Hospital admissions per 1,000 members

Ambulatory care–sensitive conditions admissions per 1,000 members

Emergency department visits per 1,000 members

Avoidable emergency visits (ambulatory care–sensitive admissions per 1,000 members)

Hospital readmissions within 30 days

View popup

Table 5

Examples of Clinical Benchmark Categories

Adult measurement category examples

Comprehensive diabetes care

Tobacco use assessment and counseling

Hypertension control

Breast cancer screening

Cervical cancer screening

Colorectal cancer screening

Depression screening and treatment

Weight, BMI screening

Asthma treatment

Cholesterol management

Pediatric measurement category examples

Well-child checks

Immunizations

Developmental screening

BMI measurement and classification

Patients with persistent asthma on controller medication

Screening for chlamydia in sexually active adolescent girls

Oral health risk assessment

Hearing and vision checks

Lead screening

BMI = body mass index.

View popup

Table 6

PCMH Qualitative Observational Evaluation Methods

Method	Domains	Source, Version, Purpose, Availability
Direct observation within the practice environment (example: medical practice waiting room observation guide)	Examples: use of space, people flow, and interpersonal interactions in waiting rooms, nurses’ stations and other back areas; décor and tone of each area; communication among clinicians, staff, and patients	BPCTI. Participant observation sessions conducted throughout the practice at varying times and days of the week Flexibly structured field note template Supplemental Appendix 6, http://www.annfammed.org/content/13/2/168/suppl/DC1
Pathway observations with staff	Staff experience of work. Observe work tasks and workflow for specific staff person and during interactions with coworkers: type of tasks, redundancy, efficiency, demeanor, behaviors, interactions	BPCTI. Observation template guide Supplemental Appendix 7, http://www.annfammed.org/content/13/2/168/suppl/DC1
Pathway observations with adult and pediatric patients	Patient (or parent or guardian) experience of visit. Observe workflow, patient data collection, confidentiality procedures, observation of clinician and staff communication with patient, parent, or guardian, time duration for visit segments	BPCTI. Observation template guide. Researcher takes notes while accompanying patients from check-in through checkout. Informal interviewing during wait times Supplemental Appendix 8, http://www.annfammed.org/content/13/2/168/suppl/DC1

BPCTI = Brown Primary Care Transformation Initiative; PCMH = patient-centered medical home.

View popup

Table 7

PCMH Qualitative Interview Evaluation Methods

Method	Domains	Source, Version, Purpose, Availability
Used with clinicians and staff
Individual baseline interviews with clinicians and staff	Role description, perspective of and experience working in practice, teamwork, conflict resolution, change processes, goals for change	BPCTI (20 core open-ended questions). In-person or telephone interview conducted with physicians, NPs, PAs, nurses, medical assistants, receptionists, and other staff. Supplemental Appendix 9, http://www.annfammed.org/content/13/2/168/suppl/DC1
Follow-up interviews with clinicians, PCMH practice champions, and staff	Perceptions of transformation progress, vision of practice as a PCMH, roles in transformation efforts, communication, patient engagement, changes in interpersonal interactions	BPCTI (19 core open-ended questions). In-person or telephone interview conducted with physicians, NPs and PAs, nurses, medical assistants, receptionists, and other staff. Supplemental Appendix 10, http://www.annfammed.org/content/13/2/168/suppl/DC1
Used with patients
Individual adult patient interviews	Patient’s experience and opinions about the practice, cognizance of practice transformation, understanding of PCMH	BPCTI (12 core open-ended questions). In-person interview conducted with patients aged >18 y. Supplemental Appendix 11, http://www.annfammed.org/content/13/2/168/suppl/DC1
Individual or pair: parent or guardian and pediatric patient interviews	Parent and child’s experience and opinions about the practice, cognizance of practice transformation, understanding of PCMH	BPCTI (14 core open-ended questions). In-person interview conducted with parent or guardian alone, parent-child pair if child is capable of participating, or child alone if child is capable of speaking completely for him/herself. Supplemental Appendix 12, http://www.annfammed.org/content/13/2/168/suppl/DC1

BPCTI = Brown Primary Care Transformation Initiative; NP = nurse practitioner; PA = physician assistant; PCMH = patient-centered medical home.

View popup

Table 8

PCMH Implementation Process Evaluation Methods

Method	Domains	Source, Version, Purpose, Availability
Written reflections and progress notes	Facilitation staff document the changing contextual circumstances in the practices, in the broader environment, and in their own facilitation roles	BPCTI (5 trigger questions). Facilitation staff keep an ongoing typed log of reflections Supplemental Appendix 13, http://www.annfammed.org/content/13/2/168/suppl/DC1
Focus groups	Moderated group discussions about enabling factors and barriers to achieving evaluation or facilitation goals, how staff roles and relationships with practices evolved and the impact of this evolution, and notions about how and why the practices are or are not transforming in specific domains	BPCTI (4 core questions about evaluation data collection; 22 core questions about transformation facilitation) Supplemental Appendix 14, http://www.annfammed.org/content/13/2/168/suppl/DC1

BPCTI = Brown Primary Care Transformation Initiative; PCMH = patient-centered medical home.

Additional Files

Tables

Supplemental Appendixes 1-14

PDF files. Last file listed is all 14 appendixes in a single PDF file.

Files in this Data Supplement:

Supplemental data: Appendix 1 - PDF file
Supplemental data: Appendix 10 - PDF file
Supplemental data: Appendix 11 - PDF file
Supplemental data: Appendix 12 - PDF file
Supplemental data: Appendix 13 - PDF file
Supplemental data: Appendix 14 - PDF file
Supplemental data: Appendixes 1-14 - PDF file
Supplemental data: Appendix 2 - PDF file
Supplemental data: Appendix 3 - PDF file
Supplemental data: Appendix 4 - PDF file
Supplemental data: Appendix 5 - PDF file
Supplemental data: Appendix 6 - PDF file
Supplemental data: Appendix 7 - PDF file
Supplemental data: Appendix 8 - PDF file
Supplemental data: Appendix 9 - PDF file
The Article in Brief

Recommendations for a Mixed Methods Approach to Evaluating the Patient-Centered Medical Home

Roberta E. Goldman , and colleagues

Background The Patient-Centered Medical Home (PCMH) is intended to transform primary care practices by combining the best primary care attributes with new ways of structuring and coordinating care, engaging patients, improving health outcomes, providing a better patient experience, improving efficiency and use of health information technology, and reducing costs. This study develops a methodology for identifying how and why transformation occurs in primary care practices.

What This Study Found PCMH evaluation must be comprehensive enough to assess and explain the context of transformation in different primary care practices and the experiences of diverse stakeholders. The methods and measures proposed in this study are intended to be used together and include survey instruments, PCMH meta-measures, patient outcomes, quality measures, qualitative interviews, participant observation, and process evaluation.

Implications

This approach, the authors conclude, can foster insights about how transformation affects critical outcomes to achieve meaningful, patient-centered, high-quality, and cost-effective sustainable change among diverse primary care practices. These insights, in turn, can inform recommendations for practice facilitation that can most effectively achieve the goals of the PCMH model.