Abstract
PURPOSE We describe changes over time in performance on measures of technical quality and patient experience as a group of primary care clinics transformed themselves into level III patient-centered medical homes.
METHODS A group of 21 Minnesota primary care clinics achieving level III recognition as medical homes by the National Committee for Quality Assurance has been tracking a variety of quality and patient satisfaction measures for years. We analyzed trends in these measures and compared them with those of other medical groups in the community to estimate what we might expect as other primary care sites gear up to achieve medical home status.
RESULTS The clinics in this group achieved a 1% to 3% increase per year in patient satisfaction and a 2% to 7% increase per year in performance on quality measures for diabetes, coronary artery disease, preventive services, and generic medication use. When compared with the average for other medical groups in the region, the rates of increase were greater for satisfaction, but similar for the quality measures.
CONCLUSIONS Achieving medical home recognition was associated with improvements in quality and patient satisfaction for these clinics, but the rate of improvement is slow and does not always exceed levels in the surrounding community in Minnesota (which are also improving). Expectations for large and rapid change are probably unrealistic.
- Organizational innovation
- patient-centered care
- patient satisfaction
- primary health care
- quality of health care
- practice-based research
INTRODUCTION
The redesign of primary care epitomized by the patient-centered medical home (PCMH) concept has become one of the most high profile topics in medical care today.1–4 Although its definition, measurement, and impacts are still uncertain, great hopes have focused on this change as critical to national attainment of the Triple Aim (improvement in quality, patient experience, and health care costs) as well as the survival of currently threatened primary care.1–3
The future of PCMH development is uncertain, but it is clear that large-scale deployment of this model will require the transformation of existing primary care practices rather than the creation of wholly new care organizations. Those who have studied this transformation process have concluded that it is very difficult work that requires years of time and usually external assistance to make the necessary major changes in both roles and management strategies.4–6 As it has only recently been given high priority, however, we know relatively little about those time trends and the best strategies and requirements for transformation.
One way to learn about transformation is to study the history of existing clinics that have achieved recognition as high-level medical homes while having excellent scores on measurements of performance. To support such a study, the Agency for Healthcare Research and Quality (AHRQ) funded a competitive contract to study the 21 primary care clinics of HealthPartners Medical Group (HPMG) in the Minneapolis and St Paul metropolitan area. These clinics had recently been recognized as level III PCMHs by the National Committee for Quality Assurance (NCQA), making it the first large group in the country to have all of its clinics so recognized. A previous study verified that these clinics had high levels of performance on measures of technical quality and patient satisfaction, but found no correlation between individual clinic scores on the NCQA measure and those performance measures.7 In this article, we report on subsequent analyses of temporal trends in performance measures in these clinics, which have now been working on the PCMH transformation process for nearly 10 years. Separate data sources and analyses allowed us to assess both the rate of change in internal measures over time and how trends in quality and satisfaction measures for HealthPartners-insured patients compared within HPMG clinics as a whole, and with those of competing medical groups with HealthPartners contracts throughout Minnesota.
METHODS
Setting
HPMG is a multispecialty group practice within an integrated health system that includes a health plan, several hospitals, and a wide range of other health care services. It provides care to 400,000 active patients in the metropolitan area of Minneapolis and St Paul, most of whom receive their primary care in the 21 clinics studied here. Importantly for generalizability, only 60% of HPMG patients have HealthPartners insurance; the other 40% either are covered by a variety of private or public insurance products or are uninsured (5%). The demographic characteristics of the entire patient population are very similar to those of the population in the metropolitan area. HPMG has had a paperless electronic health record (EHR) system in place in both primary and specialty care settings for 6 years.
Primary care in HPMG is provided by a mixture of pediatric, internal medicine, and family physicians, as well as a smaller number of nurse practitioners and physician assistants. Physicians are paid largely based on productivity, while the medical group is reimbursed on a fee-for-service basis from most payers (including the HealthPartners health plan). The medical group has been working on improving quality, costs, and patient experience since at least 2000, with a particular focus on primary care redesign and on use of information technology to support a variety of innovative communication methods with patients and population health improvement strategies. Except for implementing advanced access scheduling in 2000 and the EHR in 2002–2004, there have been no abrupt changes in approach to care worth noting, and performance measures during that time showed no dramatic change.
Clinic-Level Measures
Patient Experience—Picker Survey
Patient experience in HPMG has been measured since 2006 by a nationally standardized mailed survey of a random sample of all HPMG patients who had a recent office visit with a physician or midlevel clinician. We report on the 7 items from this survey in the years 2006–2009 with the most relevance. Two items use only Yes/No responses, whereas the other 5 have possible responses of Yes completely, Yes somewhat, and No. To remain aligned with organizational reporting, we focused on the top rating choice as the measure for satisfaction. The survey process and production of aggregated summaries at the department, clinic, and clinician level are managed by an outside organization (NRC Picker Institute, Lincoln, Nebraska). The contract requires a yearly total of 25 responses for each midlevel clinician (nurse practitioner or physician assistant) and 50 for each physician. Patients are attributed to the clinic at which they had the recent office visit that prompted the survey. Response rates are typically about 30%; although that rate is quite low, it has produced very consistent scores over time at all 3 organizational levels. We used all 21 HPMG clinics in the analysis because all had survey data for the years 2006–2009. Clinic data summary points were based on data from 36 to 244 patient surveys, with a median of 104.
Quality—EHR Data
Quality measures within HPMG come from ongoing performance data taken from the EHR and reported at both the clinic and individual clinician levels. Because specifications for measures have changed over time, we performed a new data extraction to construct an all-or-none measure of optimal diabetes care with a common set of specifications over the years 2005–2009. This measure included, for diabetic patients, control of hemoglobin A1c (HbA1c) (defined as a level <7%), blood pressure (<130/80 mm Hg), and lipids (low-density lipoprotein cholesterol level <100 mg/dL); documented nonsmoking status; and regular use of aspirin for those older than 40 years of age. We included in this analysis the 19 of 21 clinics that were a part of the HPMG system for all 5 years. Patients were attributed to a clinic based on the clinician they saw most in the year and that clinician’s home clinic. The number of patients for this measure at the 19 clinics across all 5 years ranges from 970 to 5,017, with a median of 2,697.
Medical Group–Level Measures
Patient Experience—Consumer Choice Survey
To compare patient satisfaction in HPMG with that in the other, non-HPMG medical groups in the state, we took advantage of a separate survey (Consumer Choice) developed and conducted by the HealthPartners health plan among a sample of its members (65% of whom are cared for in medical groups other than HPMG). The survey is conducted annually to assess member satisfaction and is not linked to an office visit. Response rates to the 1-time mailed survey were 25%. HealthPartners members were attributed to a medical group based on claims received from each group, and a random sample of patients was drawn from each medical group to yield roughly 200 completed surveys per group. We report on 7 items from this survey in the years 2005–2009. This analysis focused on the top rating choice (very satisfied or definitely yes) among the 4 to 5 alternatives available per item to maximize variation across medical groups, to avoid item ceiling effects found when collapsing top categories, and to align the approach with organizational reporting. The 20 medical groups (1 HPMG, 19 non-HPMG) used in this analysis consisted of all medical groups that had contracts to care for HealthPartners members and that also had enough survey data in all 5 years. Each medical group data summary point is based on data from 153 to 282 patients, with a median of 207.
Quality—Clinical Indicators Data
Just as with patient satisfaction, to compare quality trends with other, non-HPMG medical groups in the state, we took advantage of existing performance data for HealthPartners members who are cared for in any of the contracted medical groups. These data analyzed by the health plan, called clinical indicators, are attributed to the various groups based on analysis of where members actually receive most of their primary care. The measures analyzed include an optimal diabetes care measure that includes control of HbA1c, blood pressure, and lipids. A composite measure for adult preventive services includes screening for chlamydia, colorectal cancer, breast cancer, cervical cancer, and hyperlipidemia. There is also a coronary artery disease (CAD) control composite measure that assesses control of lipids, blood pressure, and smoking, and daily aspirin use in patients with CAD. Finally, a patient-level indicator of the rate of generic drug use serves as a measure of efficiency. Attribution of patients to group varied by measure. For optimal diabetes care, patients were attributed to the group with the most diabetes-coded visits in the measurement year; for preventive services and optimal CAD care, attribution was to the group with the most office visits in the measurement year; for generic drug use, attribution was to the group of the prescribing physician.
Changes in measure specifications over time as well as introduction and retirement of some measures resulted in variation by measure in the time periods over which it was available, patient sample sizes, and numbers of medical groups to serve as the comparison with HPMG. The contracted medical groups used in the analysis of each measure consisted of all groups wherein HealthPartners members were seen and the measure was available for enough members in the years of interest. For optimal diabetes care, data from 20 medical groups total (HPMG and non-HPMG) were available in the years 2005–2007, with patient sample sizes ranging from 173 to 2,635 (median = 257). For preventive services, there were 22 medical groups during 2006–2009 and patient sample sizes of 287 to 528 (median = 346). For optimal CAD care, there were 20 medical groups during 2005–2007 and patient sample sizes of 143 to 516 (median = 245). Finally, for generic drug use, there were 35 medical groups during 2005–2009 and patient sample sizes of 381 to 44,877 (median = 2,621).
Covariates
We used linkages to clinic administrative and clinical data to ascertain covariates for use in adjusting estimates of trends over time for each clinic. Picker patient satisfaction data for the clinics were linked to the following year-specific, clinic-level variables related to patient case mix: number of active patients, proportion of active patients covered by Medicaid, and proportions that were female, white, and aged 65 years or older. We also linked the EHR-based optimal diabetes quality measure for each clinic to year-specific, patient-level variables—sex, Medicaid status, age, race, English as primary language, and number of medications prescribed—and to a year-specific clinic-level variable—the number of active patients at the clinic.
Similarly, at the medical group level, demographic information was available for the health plan members who were attributed to each contracted medical group. We linked Consumer Choice data to the following year-specific, medical group–level variables related to patient case mix: number of patient visits per week, proportion covered by Medicaid, proportion female, mean age of patients, mean number of primary care visits per year, and mean number of medications prescribed. The clinical indicator quality measures were linked to administrative year-specific, patient-level variables of sex, Medicaid status, age, number of primary care visits per year, and number of medications prescribed for the medical group–level analysis.
Analysis
We summarized each patient satisfaction and quality measure at the clinic or medical group level, and then computed means, standard deviations, and ranges to describe central tendency and variation. For Picker items and the EHR-based optimal diabetes care measure for the clinic-level analysis, we used generalized linear mixed models with a logit link to predict each binary measure from the available patient covariates, clinic covariates, or both; a linear term for time; and a random term to account for patient nesting in clinics. We translated the time coefficient from these models into a model-predicted adjusted yearly change in each measure. For Consumer Choice items and the clinical indicator quality measures for the medical group–level analysis, we used generalized linear mixed models (identity link for the generic drug use measure, logit link for all others) to predict each measure from the available patient covariates, medical group covariates, or both; a linear term for time; an interaction between medical group (HPMG vs not) and time; and a random term to account for the nested data. Coefficients from the interaction model were translated into HPMG and non-HPMG model-predicted adjusted yearly changes for each measure. We used linear time trends based on visual inspection of trends, selecting them because of the limited number of time points for some data sources for estimating higher-order trends.
We examined the magnitude and direction of the covariance between the random slope and intercept terms to assess whether clinics or medical groups that started lower on a measure showed greater or lesser improvement over time on the measure.
RESULTS
Clinic and Medical Group Characteristics
Of the 21 clinics in the HPMG group, 6 were located in center city areas of the Twin Cities and 15 were suburban (Table 1). They had an average of 9.3 primary care physicians plus 2 nurse practitioners or physician assistants. About two-thirds of their patients had commercial insurance, 12% had Medicare, and the rest had various state program coverage or were self-pay. Although patient demographics varied by clinic, on average, 69% of patients were white and 96% were English speaking.
Of the 34 contracted non-HPMG medical groups serving as the comparison in medical group analyses, about one-half were also in the Minneapolis/St Paul metropolitan area, while the other one-half were in Minnesota but outside that area, and 2 were in Wisconsin and North Dakota (data not shown). Only 5% of the comparison medical groups were single-site groups, while 40% had 2 to 5 clinics and 27% had more than 15 clinics. Forty percent were primary care groups and 60% were multispecialty with considerable diversity in the proportions of their patients that had various insurance arrangements.
The few patient-level descriptors available for the analysis comparing the HPMG vs non-HPMG medical group suggested similar profiles, with 58% female patients in the HPMG group and 57% across the non-HPMG medical groups, a median age of 42 vs 43 years, a median of 2 primary care visits per year, and a median of 11 prescription medications in 2009.
HPMG Clinic Trends Over Time
Responses on the Picker satisfaction survey are shown in Table 2 along with unadjusted mean rates and clinic score ranges in 2006 and 2009. Although the changes were not large for measures that were already at fairly high levels, rates for all of the items rose by 0.6 to 4.3 percentage points from 2006 to 2009, primarily as a result of increases for the clinics with the lowest rates. The yearly increase in satisfaction after adjustment for clinic-level patient case mix in each year was positive for all 7 items but significant for only 3 of them. A survey used by HPMG before use of the Picker survey showed no change from 2003 to 2005 in “able to get appointment,” but improvement in “treated with respect” and “timely test results” (data not shown).
Trends for the quality measure of optimal diabetes care assessed from the EHR showed an increasing rate over time, with 11.2% of patients meeting the measure in 2005 (clinic range, 4.8%–19.9%), and 22.3% meeting the measure in 2009 (clinic range, 15.8%–31.1%) (data not shown). After adjusting for patient-level characteristics and clinic size in each year, the model-predicted increase in the rate of optimal diabetes care was 2.3% per year (P <.001).
Comparison of HPMG Trends with Secular Trends in Minnesota
Table 3 shows Consumer Choice satisfaction rates in 2005 and 2009, comparing HPMG scores with the average for the 19 non-HPMG medical groups having at least 153 member responses per year. (More-detailed data for the latter are given in Supplemental Table 1, available online at http://www.annfammed.org/content/9/6/515/suppl/DC1.) The percentage of patients choosing the top rating for satisfaction ranged from 30% to 47% for HPMG patients and from 28% to 64% for non-HPMG patients across the 7 items in 2005. Nearly all satisfaction items improved over time for both groups, but the magnitude of the HPMG improvements was comparatively greater. When adjusting for differences in the patient case mix at the level of the medical group in each year, the yearly increase in satisfaction rates was significantly higher for HPMG than for non-HPMG medical groups for 4 of 7 items that address access and general satisfaction. The pattern of increases for these 4 items indicated that HPMG started with lower rates in 2005 and then caught up to the non-HPMG rates by 2009. For the HPMG clinics, a comparison with the survey results in 2000 indicated no change from 2000 to 2005 in the items “satisfied with clinic” (37.8% in 2000), “recommend clinic” (48.1%), and “advice on staying healthy” (35.0%), and slow improvement in “convenient appointment” (27.3% in 2000) and “doctor of your choice” (28.4%) (data not shown).
Table 4 shows quality measures from clinical indicators, allowing comparison of trends over time between HPMG and the 20 to 35 non-HPMG medical groups that had data for these measures over the varying time intervals for which each measure had adequate data. (More-detailed data for the latter are given in Supplemental Table 2, available online at http://www.annfammed.org/content/9/6/515/suppl/DC1.) Changes from the baseline year to the last year (both of which varied depending on the measure) were all positive for both HPMG and non-HPMG groups. After adjusting for patient-level variables and medical group size in each year, the yearly change for each measure was positive for all 4 measures. The magnitude of yearly change suggested that HPMG exceeded the secular trend for the optimal diabetes care, optimal CAD care, and adult preventive services measures, but these difference in trends between groups were not significant. For generic drug use, the non-HPMG yearly increase was significantly greater than the HPMG yearly increase; non-HPMG groups started lower than HPMG groups and then nearly caught up. The very large sample size for the generic drug use analysis yielded a model with all highly significant terms, so the clinical significance of the difference in improvement for HPMG and non-HPMG groups needs to be assessed in addition to the statistical significance.
DISCUSSION
Our study is important in that it describes the trajectory over time with which clinics that have achieved level III PCMH recognition have been able to improve their patient satisfaction and quality scores as they transformed. Rates of improvement in satisfaction among HPMG clinics as a whole averaged only about 1% per year for measures having high rates at baseline (Picker mean of 87%), but about 3% per year for measures with lower baseline rates (Consumer Choice mean of 38%). For quality, we observed increases of 2% to 7% per year for various measures.
Where we had data to compare HPMG improvements with community averages over 4 to 5 years, the relative changes were mixed. For satisfaction, the rate of increase among HPMG patients was greater than for patients cared for in other groups, but it only caught up to the community average. For the quality measures, the picture was more heterogeneous and more similar to the patterns of the other medical groups that, on average, had improved too. Overall, the rate of improvement per year is probably not what national policy makers are hoping to see from transformation to medical homes. Our analyses are complicated by the fact that the average rates for most performance measures in Minnesota are higher than those in most of the United States, and because we share quality results and strategies openly among competing groups through the Institute for Clinical Systems Improvement, our regional quality improvement collaborative, it is not surprising that the trend is upward for all groups. In addition, there is considerable local interest and action around medical home transformation, but we have no way to ascertain the extent to which various competing medical groups have actually made such changes, and no other Minnesota medical groups have attained NCQA recognition as PCMHs.
Unfortunately, there is still very little information about the impact of medical home transformation on quality with which to compare these observed changes in quality.8 Most of the few studies of medical home impacts have focused on costs and utilization. The National Demonstration Project reported only small changes in quality but decreases in patient ratings of practices’ PCMH attributes over 2 years among its volunteer clinic participants.9 The Group Health pilot clinic study reported improvements in patient experience and aggregate quality measures in a similar time period, but the changes were rather modest.10
This study is limited by its focus on a single state and reliance on available information. We have no information about how comparison medical groups might have scored on the NCQA recognition measure, some of which might well have been similarly recognized if they had applied. Furthermore, the performance measures used to compare HPMG with other medical groups over time is limited to those patients in all groups who have HealthPartners insurance; however, this latter limitation also has the advantage that the patients included in the measure should be relatively similar, differing only in where they chose to seek medical care. This study also is limited by its observational design, use of existing data from a survey with low response rates, and use of a nonrandomized comparison group. But at a time during which we will have to get an increasing proportion of our research data from just such observational studies and existing data, this study provides an opportunity to demonstrate the ability of such studies to understand phenomena that cannot be tested in a controlled trial. In addition, a variety of local environmental influences have undoubtedly affected trends in quality and satisfaction rates, perhaps similarly for HPMG and other medical groups. For example, HealthPartners has a number of initiatives and programs designed to improve performance for all of its members that may affect both HPMG and contracted groups with large proportions of Health-Partners members. There is also an unusually strong cultural tradition in Minnesota of public reporting and sharing of successful improvement strategies that may contribute to the improving secular trends across all groups. Finally, all changes in care have been gradual, so we cannot identify particular times when something happened that could have resulted in sudden improvement in any of these measures.
As we move rapidly as a nation to encourage transformation of traditional primary care practices into patient-centered medical homes, this study adds to the reasons for avoiding unrealistic expectations about the rate of improvement in health or patient experience that will result. Some have found reason to believe that cost savings may be realized, but even for that outcome, we must await further evaluations to be convinced that the savings will be sizable and quickly achieved.10–12
Acknowledgments
We are indebted to Beth Averbeck and Nancy Salazar for their thoughtful review of drafts of this article, to Kay Schmitt for help with the Consumer Choice data, to Ann Hanson for assembling the data from the EHRs, to Penny Moran and Nancy Salazar for help with the Picker satisfaction data, and to Diane Wehrle and Rachel Woods for clarifying the clinical indicators data, as well as to Krista Van Vorst for assembling the health plan data used in these analyses.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This work was supported by Agency for Healthcare Research and Quality contract HHSA290 2007 10010 TO 4.
- Received for publication February 10, 2011.
- Revision received May 26, 2011.
- Accepted for publication June 8, 2011.
- © 2011 Annals of Family Medicine, Inc.