Abstract
PURPOSE Medicare beneficiary spending patterns reflect those of the 306 Hospital Referral Regions where physicians train, but whether this holds true for smaller areas or for quality is uncertain. This study assesses whether cost and quality imprinting can be detected within the 3,436 Hospital Service Areas (HSAs), 82.4 percent of which have only 1 teaching hospital, and whether sponsoring institution characteristics are associated.
METHODS We conducted a secondary, multi-level, multivariable analysis of 2011 Medicare claims and American Medical Association Masterfile data for a random, nationally representative sample of family physicians and general internists who completed residency between 1992 and 2010 and had more than 40 Medicare patients (3,075 physicians providing care to 503,109 beneficiaries). Practice and training locations were matched with Dartmouth Atlas HSAs and categorized into low-, average-, and high-cost spending groups. Practice and training HSAs were assessed for differences in 4 diabetes quality measures. Institutional characteristics included training volume and percentage of graduates in rural practice and primary care.
RESULTS The unadjusted, annual, per-beneficiary spending difference between physicians trained in high- and low-cost HSAs was $1,644 (95% CI, $1,253–$2,034), and the difference remained significant after controlling for patient and physician characteristics. No significant relationship was found for diabetes quality measures. General internists were significantly more likely than family physicians to train in high-cost HSAs. Institutions with more graduates in rural practice and primary care produced lower-spending physicians.
CONCLUSIONS The “imprint” of training spending patterns on physicians is strong and enduring, without discernible quality effects, and, along with identified institutional features, supports measures and policy options for improved graduate medical education outcomes.
INTRODUCTION
Public and private payers have embraced value-based health care, and are marching aggressively toward the Triple Aim—better health, better care, and lower costs.1 Simultaneously, the National Academy of Medicine, The Josiah Macy Jr Foundation, and others are calling for greater accountability for more than $15 billion currently spent on Graduate Medical Education (GME), and for more strategic allocation of GME funding.2–8 Most research supporting accountability measures and reform options has focused on the specialties and distribution of graduates, but our recent study exploring the relationship between training and future practice costs found that physician-associated Medicare costs were highly correlated with those of their the physicians’ Hospital Referral Regions (HRR).9 Educational imprinting is the idea that learners model witnessed behaviors and beliefs, sometimes despite what they are taught, and GME is not immune.10–18 Our study demonstrated a durable imprinting effect, lasting at least 16 years after training. A contemporary study revealed similar association between HRR patient management style and general internists’ choices of conservative vs aggressive management options in a certifying exam.19 Studies have also shown that imprinting effects are visible in quality of care for women treated by obstetricians20 and that practice intensity (aggressiveness) is largely predicted by residency affiliation.21
These studies demonstrate important relationships between training and subsequent quality and cost of care, but they are limited either by geography or by the number of programs studied. HRRs are geographic entities created by the Dartmouth Institute based on patterns of patient care-seeking behavior and on referral patterns.22 There are 306 HRRs in the United States, and these large areas have as much cost variation within as between them.23 This raised concerns that our previous study9 might be at risk for ecological fallacy; that is, the training/practice cost associations we found might be related to factors other than the training institution(s) within the same geographic area.
In this study, we drill down to the building blocks of HRRs, the 3,436 Hospital Service Areas (HSAs), which are built from care patterns associated with specific hospitals or closely located groups of hospitals, to test whether associations between training and practice can be shown at this smaller geographic scale. Most HSAs have only 1 acute care hospital (2,830; 82.4%), and another 322 (9.4%) have just 2 hospitals, so this study approaches institutional-level assessment.24
The previous study also raised questions about whether quality, another important Triple Aim goal, was similarly associated. In this study, we investigated this association by adding claims-based quality measures of diabetes care.25 We recognized that HSA and institution are contextual factors offering limited options for GME reform. So we also explored limited compositional factors—sponsoring institution characteristics—for potential training-outcome differences that might offer more specific options for GME reform.
METHODS
Training and Practice HSA Cost Analysis
The study patient sample consisted of all Medicare beneficiaries aged 65 years or older who obtained primary care services in 2011 from a nationally representative sample of primary care physicians. The sampling frame was all physicians listed in the American Medical Association (AMA) Physician Masterfile and identified as being in direct patient care in 2010, excluding residents, retirees, physicians with unknown practice type, and physicians who mainly teach or do research. We used full-year 2011 Medicare billing claims to determine beneficiary costs per physician. We used the AMA Masterfile to identify each physician’s residency training program, residency location, and graduation year. We used the most frequent patient zip code to assign each physician to a practice HSA. Location data were matched with Dartmouth Atlas HSA files to identify residency training HSA in the physician’s graduation year, practice HSA in 2010,and the average HSA Medicare spending per beneficiary in those years.
We assigned beneficiaries to primary care physicians to establish physician patient panels using established methods.9,26 We defined primary care as care provided by family physicians or general internists for this population, excluding claims from hospitalists, identified using the Kuo method.27 Based on Dartmouth data availability, we included primary care physicians who completed residency training from 1992 to 2010. Medicare expense data were adjusted for the age, sex, and race of beneficiaries from 1992 to 2002, and additionally for price starting in 2003.22 Sensitivity testing of our analysis for this difference in price adjustment revealed no differences across the 2 time spans, so the data were pooled in this analysis.
The physician sample was further limited to physicians with 40 or more Medicare beneficiaries to ensure reliable per-beneficiary average spending. For each year of available Dartmouth data, standardized spending for both practice and training HSAs were separately categorized into low, average, and high-cost groups of approximately equal beneficiary numbers.
We calculated each physician’s mean cost per beneficiary (total Medicare Part A + Part B allowed charges), excluding outlier patients with unusually high health care costs (greater than $100,000).
Training and Practice HSA Quality Analysis
We used diabetes ICD-9 codes (250.xx) in 2011 Part B claims data to identify eligible patients. In the same claims data, using federal definitions of quality measures,28 we used CPT codes to determine annual receipt of 4 preventive services (eye examination and testing for hemoglobin A1c, low-density lipoprotein screening, and nephropathy screening) from any provider. (For CPT codes used, see Supplemental Appendix 1, available at http://www.AnnFamMed.org/content/15/2/140/suppl/DC1/). We focused on diabetes because it is prevalent, related preventive services are not typically controversial, and the services selected were recommended to occur at least annually in the period studied.
Patient and Physician Characteristics
Analyses controlled for patient age, sex, race and ethnicity, number of primary care visits, and comorbidity, the latter using the weighted Charlson index.29 CMS obtains race and ethnicity information from the Social Security Administration and applies an imputation algorithm to improve coding accuracy.30
Physician characteristics captured included sex, specialty, international medical graduate status, rural or urban location, percentage of hospital-based care, and years of practice. We used the physician main practice zip codes and Rural-Urban Commuting Area (RUCA) codes to classify practices into 5 groups: urban, large rural, small rural, isolated rural, and frontier.31 Number of years since graduation from residency determined “years in practice,” with the data organized into 3 categories: 1–7 years, 8–15 years, and 16–19 years.
Sponsoring Institution Characteristics
Sponsoring institutions are the accredited entities that have responsibility for overseeing training regardless of where training occurs. We used the AMA Masterfile to identify trainees of sponsoring institutions, using previously described methods.2,5 We studied the relationship of training and practice costs with size of graduate pool, percentage of graduates in rural practice, and percentage practicing in primary care. Prior studies show that these characteristics are associated with overall costs and hospitalizations.32,33
Statistical Analysis
Patient and physician characteristics were summarized in low-, average-, and high-cost HSAs based on site of physician training. The unadjusted mean and median per-beneficiary spending of physicians were calculated for all training and practice HSA category combinations. Multivariable regression models examined the association between the following variables:
The natural log of mean spending
Whether the physicians trained in a low, average, or high spending HAS
Characteristics of the sponsoring institutions: percentage of graduates in rural practice, percentage in primary care, and the natural log of the number of graduates
All models included controls for patient, community, and physician characteristics detailed above. Multivariable logistic regression models were used to study hospitalizations as a potential cause of cost differences related to institutional characteristics. For beneficiaries with diabetes, we estimated logistic regression models predicting receipt of any of the selected preventive services in 2011. We also ran a Poisson regression after creating a composite score reflecting the number of these recommended services provided; the score ranged from 0 to 4.
Because we oversampled physicians for smaller states, all results were adjusted with physician-level weights to obtain national estimates. To correct for measures correlation across patients treated by the same physicians, the standard errors were adjusted to allow for correlation within clusters (physicians) using the vce (cluster clustvar) command in Stata 14.1 (StataCorp LP). All tests of significance were 2-sided; significance was defined as P <.05. The American Academy of Family Physicians Institutional Review Board approved this study without restrictions.
RESULTS
Physician and Patient Demographics
Application of inclusion criteria resulted in an analytic sample of 503,109 beneficiaries receiving care from 3,075 general internists and family physicians (Table 1 and Supplemental Appendixes 2 and 3, available at http://www.AnnFamMed.org/content/15/2/140/suppl/DC1/).
HSA and Physician Spending
Practice HSA cost categories were low cost ($6,102–$8,683), average cost ($8,691–$9,879), and high cost ($9,880–$16,542) (Table 2 and Appendix 3). Physicians practicing in low-cost HSAs were more likely to have trained in low-cost HSAs, and those practicing in high-cost HSAs were more likely to have trained in high-cost HSAs (Figure 1). Physicians trained in high-cost HSAs spent significantly more per patient than those trained in low-cost HSAs, no matter what the spending category of the practice HSA (Table 2 and Appendix 3). Averaged across all practice HSAs, this difference was $1,644 (95% CI, $1,253–$2,034), and the starkest difference was between physicians trained in low- and high-cost HSAs who practiced in high-cost HSAs ($1,972; 95% CI, $1,138–$2,805). This relationship held true for family physicians and general internists in our multivariable analysis (Table 3 and Appendix 3); general internists, however, made up two-thirds of sample physicians trained in high-cost HSAs, and family physicians made up two-thirds of those trained in low-cost HSAs (Figure 1).
Institutional Characteristics
Graduates were significantly more likely to be low-cost physicians if their sponsoring institution produced fewer total physicians, more rural physicians, or more primary care physicians (Table 4). These differences were not explained by fewer hospitalizations (Table 3).
Duration of Association
Cost outcomes varied significantly by years in practice. For physicians with 0 to 7 years of practice, those trained in high-cost HSAs spent 23% more per patient (95% CI, 10%–37%) than those trained in low-cost HSAs. The difference was 9% for physicians in practice 8 to 15 years (95% CI, 3%–14%), and there was no significant difference for those in practice 16 to 19 years (Table 4).
Quality
There were no significant single or composite differences in quality score across categories of training HSA spending (Table 5). Higher-cost practice HSAs had significantly better quality scores for 2 measures, significantly worse scores for 1 measure, and scores for the fourth and for the composite measure that were not significantly different. Larger training institutions did significantly better on the composite quality score but not any individual measures. There were no significant associations between training HSA and practice HSA quality measures.
DISCUSSION
Among general internists and family physicians who completed residency training in 1992 or later and were in practice in 2010, spending patterns for Medicare beneficiaries they provided care for in practice were similar to the patterns of the HSA in which the institution that sponsored their training was located. There was no similar relationship for quality, and spending pattern variations had no discernible relationship to quality. Certain features of training institutions were associated with lower practice spending.
Several of our findings strengthen the suggestion that cost-related behaviors are imprinted on physicians in training:
The consistency of the relationship between training and practice beneficiary costs across HRRs and HSAs
The fact that HSAs around individual training institutions reflect the patterns of their respective institutions more than 80% of the time
The fact that we can find predictive institution-level factors for desirable outcomes.
We also found that general internists are much more likely to train in high-cost HSAs and subsequently become high-cost physicians, potentially explaining why they are more likely to be associated with higher average expenditures than family physicians.34,35 The lack of correlation between the quality scores we assessed and HSA spending category does not weaken our argument, but it may suggest that the culture and systems needed to imprint quality-related behaviors may not be consistent across GME training institutions. Both findings offer options for GME policy, namely to strengthen educational and cultural norms associated with imprinting desirable practice behaviors, to create educational outcome measures for GME funding, and to support GME expansion where institutional characteristics are associated with more desirable GME outcomes. The Institute of Medicine (IOM) and others are calling for greater GME accountability and strategic allocation of GME funding, and we need reliable tools for these high stakes decisions.2–8
This notion of GME imprinting is hardly new, but this study demonstrates with greater reliability that it is potent and important as our health system moves toward value-based payments. Jordan Cohen, MD, former President of the Association of American Medical Colleges, once noted, “The residency experience inevitably brands all physicians with an indelible imprint of medicine’s lived values.”16 This study suggests that our imprinting of the things we aspire to imprint on trainees, particularly care quality, may need further development.
While the “hidden curriculum” in medical education exists and has long been acknowledged, most attention has focused on social and cultural imprinting and not on health care costs or quality. There is growing interest in and evidence supporting the idea that cost-related behaviors experienced in training imprint cost-related behaviors in practice, but only one geographically limited study of obstetricians has demonstrated a quality imprint.9,19,20 Citing the impact of the hidden curriculum in GME, Cohen suggests that complaints about how graduates do their jobs could test public support and faith in our training programs.16 This study offers methods for assessing imprinting related to important elements of the Triple Aim for Healthcare. These methods could serve as accountability measures and provide strategic options for GME that could help preserve public support and faith.
In addition to contextual training imprinting effects, we found that compositional factors—training institution characteristics—were associated with producing lower-cost physicians. These findings harken to an earlier IOM report that called for more training in community-based settings.36 Purposefully moving current training programs or expanding training programs into the community, and particularly into rural areas, could be an important focus of the proposed GME Transformation Fund, and measures similar to those we tested could support performance-based GME incentives.6 This study also suggests additional research and policy options for acculturating the workforce to provide prudent and cost-effective care.37 A recent systematic review recommends purposeful imprinting through effective knowledge transmission about high-value care, facilitation of reflective practice, and provision of a supportive environment with high-value role-modeling and culture.18,37 An educational intervention following these principles could be tested in training institutions identified as producing high-cost physicians.
Limitations
Study limitations include the possibility that physicians caring for Medicare beneficiaries may not be representative of all physicians. Compared with our previous study, this study reduces but does not eliminate the risk of ecologic fallacy. These HSA- and institution-level findings provide greater specificity while proving to be quantitatively consistent with our original exploratory HRR analysis. This strengthens the association, coming much closer to an institution-level analysis, given that 82.4% of HSAs have only 1 teaching hospital. These findings also reduce prior concerns over ecological fallacy because, as has been noted, there is considerable variation within HRRs.23 Specifically, HSAs account for 91% of total variation in hospital costs vs 78% for HRRs, and the average number of hospitals within an HSA is 1.6 while the average for for HRRs is 11.38 HSAs provide for variation assessment that is both more complete and more reliable, thus reducing the concern about ecological fallacy.
A concern raised about using HSA rather than HRR data is that unobserved variation in health status is better balanced across HRRs than HSAs. If HSA-level analysis reduces population health status homogenization, unobserved health status variation would become a greater influence on cost variation and reduce the association between training and practice costs. Still, the fact that the association between training-HSA cost and later practice cost remains quite strong suggests that unobserved health status has little influence on imprinting and later outcomes. Our study could not assess institutional variation within HSAs that had more than 1 training institution, but obtaining institution-specific beneficiary costs would eliminate ecological concerns and permit better assessment of institutional factors associated with more desirable outcomes.
We were able to include a handful of claims-based quality measures, but further testing of quality is warranted. It will be important to retest these associations in the period after the Physician Quality Reporting System and other environmental pressures make quality monitoring more systematic. We did not account for differences in GME payments or for Disproportionate Share Payments, of which the former can be a measure of hospital Medicare volume and the latter a proxy for Medicaid volume. Dartmouth, however, is well-regarded for its efforts to normalize spending within HSAs to reduce confounding of differences.
Whatever its limitations, this study increases the likelihood that the relationships identified are real and important, and it supports efforts to test interventions in residency training that may bend imprinting toward teaching and modeling behaviors that improve value in health care. Furthermore, it speaks favorably to the recommendations made by the IOM, both recently and more than 25 years ago, supporting GME transformation, strategic GME expansion, and accountability for training outcomes.
Footnotes
Conflicts of interest: authors report none.
Supplementary materials: Available at http://www.AnnFamMed.org/content/15/2/140/suppl/DC1/
- Received for publication September 5, 2016.
- Revision received November 17, 2016.
- Accepted for publication November 25, 2016.
- © 2017 Annals of Family Medicine, Inc.