Abstract
PURPOSE There is limited evidence for the effectiveness of pay for performance despite its widespread use. We assessed whether the introduction of a pay-for-performance scheme for primary care physicians in Ontario, Canada, was associated with increased cancer screening rates and determined the amounts paid to physicians as part of the program.
METHODS We performed a longitudinal analysis using administrative data to determine cancer screening rates and incentive costs in each fiscal year from 1999/2000 to 2009/2010. We used a segmented linear regression analysis to assess whether there was a step change or change in screening rate trends after incentives were introduced in 2006/2007. We included all Ontarians eligible for cervical, breast, and colorectal cancer screening.
RESULTS We found no significant step change in the screening rate for any of the 3 cancers the year after incentives were introduced. Colon cancer screening was increasing at a rate of 3.0% (95% CI, 2.3% to 3.7%) per year before the incentives were introduced and 4.7% (95% CI, 3.7% to 5.7%) per year after. The cervical and breast cancer screening rates did not change significantly from year to year before or after the incentives were introduced. Between 2006/2007 and 2009/2010, $28.3 million, $31.3 million, and $50.0 million were spent on financial incentives for cervical, breast, and colorectal cancer screening, respectively.
CONCLUSIONS The pay-for-performance scheme was associated with little or no improvement in screening rates despite substantial expenditure. Policy makers should consider other strategies for improving rates of cancer screening.
INTRODUCTION
Governments around the world are trying to align physician financial incentives with desired health system goals. In the United States, the National Commission on Physician Payment Reform recently stated that “[o]ur nation cannot control runaway medical spending without fundamentally changing how physicians are paid.”1 Over the last decade, pay for performance has been seen by many as the most promising approach to reducing health system cost and improving quality.2 Critics have questioned whether pay for performance can deliver on its promise, however, citing issues with measurement, target-setting, non-financial barriers to optimizing care, and intrinsic physician motivation.3–5
Reducing morbidity and mortality through cancer screening is an important component of primary care. Screening for cervical cancer6 and colorectal cancer7 is generally regarded as cost-effective. Screening for breast cancer via mammography is more controversial,8–10 but is still widely recommended by clinical practice guidelines.11,12 Nevertheless, a substantial proportion of individuals do not receive recommended cancer screening.13–16 It is unclear whether pay for performance can shrink gaps in preventive care specifically17, 18 or primary care more generally.19
We evaluated a large-scale pay-for-performance scheme introduced in Ontario in 2006 and aimed at improving cancer screening in primary care.
METHODS
Setting
Ontario is Canada’s largest province, with a population of approximately 12.8 million people in 2011. Physician visits and cancer screening tests are paid for by the Ontario Health Insurance Plan (OHIP), with no out-of-pocket payments, for all permanent residents in the province.
Between 2002 and 2010, approximately 75% of Ontario residents joined patient-centered medical homes.20 Key features of the medical homes include patient enrollment, after-hours coverage, and physician payment reform including the introduction of capitation payments (ranging from 20% to 75% of physician income) and financial incentives, including ones for preventive health care.21 Approximately 10% of Ontarians see a primary care physician who does not practice in a medical home and is paid fee-for-service. The remaining 15% either do not have a regular primary care physician or visit a primary care physician who practices in a medical home in which they are not formally enrolled.
Study Design
Cross-Sectional Analysis
We assessed patient characteristics associated with cervical, breast, and colorectal cancer screening as well as use of the preventive care incentives among all eligible primary care physicians from April 1, 2009 to March 31, 2010. Physicians were excluded from the analysis if they were in their first year of practice or had enrolled fewer than 100 patients.
Longitudinal Analysis
We calculated age- and sex-standardized screening rates for cervical, breast, and colorectal cancer in each fiscal year from 1999–2000 to 2009–2010 and the amount of money spent on preventive care incentives each fiscal year after the pay-for-performance program was introduced broadly on April 1, 2006. Because of limited data availability in the earliest years of this period, we assessed colorectal cancer screening rates (fecal occult blood testing or colonoscopy) only for the years from 2001–2002 to 2009–2010. We stratified cancer screening rates by neighborhood income quintile and by whether the individual was enrolled with a physician who practiced in a medical home.
We used administrative claims data accessed through a comprehensive research agreement between the Institute for Clinical Evaluative Sciences and the Ontario Ministry of Health and Long-Term Care (MOHLTC). Before data analysis, patient and provider identifiers were replaced with unique encrypted numbers. This study was approved by the Research Ethics Board of Sunnybrook Research Institute in Toronto, Ontario.
Cancer Screening
We calculated cancer screening rates using claims submitted to OHIP. We determined whether individuals were due for screening based on their ages and timelines indicated by the preventive care incentives, which specify a 30-month window for screening (Table 1). For cervical cancer screening rates, we determined the proportion of women aged 35 to 69 years who received a Papanicolaou (Pap) smear screening in the 30 months before the end of the relevant fiscal year, excluding those who had had a hysterectomy. For breast cancer screening rates, we determined the proportion of women aged 50 to 69 years who received a mammogram in the 30 months before the end of the relevant fiscal year, excluding those who had had a mastectomy or were being treated for breast cancer. And for colorectal cancer screening rates, we determined the proportion of adults aged 50 to 74 years who either received fecal occult blood testing (FOBT) in the 30 months before the end of the relevant fiscal year or had had a colonoscopy in the previous 10 years, excluding those with a history of colon cancer.
Preventive Care Incentives
We assessed use of the incentives for cervical, breast, and colorectal cancer screening using claims submitted to OHIP. Incentives for all 3 types of screening were introduced broadly in 2006–2007, with more incentives for colorectal cancer screening added in 2008–2009.
The incentive program allowed primary care physicians in patient-centered medical homes to bill 1 incentive code annually for each type of screening based on their own calculation of targets achieved (Table 1). In 2009–2010, physicians could receive a maximum of $8,400 if they met the highest targets for all 3 types of screening—about 3% of their gross income.22
Other Data Sources
Patient age, sex, and place of residence were obtained from the registry of people covered by OHIP. We derived neighborhood income by linking census data to the patients’ residential postal codes and stratified income into quintiles. For the longitudinal analysis, income quintile was calculated for patients for each fiscal year. Patient diagnoses were derived from physician billing claims submitted to OHIP and the hospitalization data collected by Discharge Abstract Database from the Canadian Institute for Health Information. We used the Johns Hopkins ACG software to assign patients to Resources Utilization Bands based on similar expected health care utilization (1=low, 5=high).23
Enrollment tables provided by the MOHLTC identified patients enrolled with physicians practicing in patient-centered medical homes as of 2009. The stratified longitudinal analysis followed these patients back in time, excluding them from the analysis for years when they were not eligible for screening.
We determined the amounts paid to physicians under the incentive program by multiplying the number of times each preventive care incentive code was billed by the financial value of the fee code. In addition to the codes described in Table 1, we included codes introduced at the time of the incentive program such as ones that reimbursed physicians for contacting patients to remind them that they were due for screening (see Supplemental Appendix). We did not include payments to physicians for conducting screening tests.
Statistical Analysis
We used segmented linear regression models to assess the change in age- and sex-standardized screening rates after 2006–2007, when the largest increase in billing of the screening incentive occurred. Models allowed 1 linear trend before the intervention (from the start of the study period to 2005–2006), a step change between 2005–2006 and the year of the intervention (2006–2007), and a different linear trend after the intervention (from 2006–2007 to 2009–2010).24 Segmented linear regression is a practical way to assess the impact of a health policy change.25
We used a Chow test to confirm that there was a structural break in the screening data between 2005–2006 and 2006–2007. We also performed a Durbin-Watson test to determine the presence of first-order and second-order autocorrelation in each of the 3 data series. We detected no significant autocorrelation, so we assumed independence of the cancer screening observations when performing the regression analysis.
RESULTS
The characteristics of Ontarians eligible for and receiving screening in 2009–2010 are described in Table 2.
In 2009–2010, 84% (4,992) of eligible physicians billed at least 1 of the 3 cancer screening incentive codes and 22% (1,278) billed the highest payment category for all 3 cancer screening incentives.
During the study period, the age-adjusted screening rate increased from 55% to 57% for cervical cancer, 60% to 63% for breast cancer, and 20% to 51% for colorectal cancer (Figure 1). There was no significant step change in the screening rate for any of the 3 cancers in the year immediately after the incentives were introduced (Table 3). Colon cancer screening was increasing at a rate of 3.0% (95% CI, 2.3% to 3.7%) per year before the incentives were introduced and 4.7% (95% CI, 3.7% to 5.7%) per year after. The cervical and breast cancer screening rates did not change significantly from year to year before or after the incentives were introduced. Between 2006–2007 and 2009–2010, a total of $28.3 million, $31.3 million, and $50.0 million in incentive payments was paid to physicians for cervical, breast, and colorectal cancer screening, respectively.
For all 3 types of cancer screening, disparities in screening related to neighborhood income persisted over time. The ratios of the screening rates in the highest income quintile to those in the lowest quintile for cervical, breast, and colorectal cancer, respectively, were 1.28, 1.25, and 1.36 at the start of the study period and 1.33, 1.23, and 1.30 at the end. Individuals who were enrolled in a patient-centered medical home in 2009 had higher cancer-screening rates for all 3 types of cancer over the previous decade than those not enrolled. The ratios of the screening rates for individuals enrolled in a medical home in 2009 to the rates for those not enrolled, for cervical, breast, and colorectal cancer, respectively, were 1.46, 1.40, and 1.43 at the start of the study period and 1.96, 1.77, and 1.95 at the end.
DISCUSSION
Our findings demonstrate that the introduction of a province-wide, primary care pay-for-performance scheme for cervical, breast, and colorectal cancer screening was associated with little or no increase in cancer screening rates despite relatively large expenditure and very good uptake of incentives among primary care physicians. More, we found persistent disparities related to neighborhood income for all 3 types of cancer screening. Only physicians practicing in medical homes were eligible for the incentives, but individuals enrolled in a medical home were more likely to receive cancer screening than others even before introduction of the incentives.
Many factors influence the impact of financial incentives on quality of care,19, 26 but we hypothesize that the size and structure of Ontario’s incentive program played key roles in limiting its impact. The preventive care incentives were among the largest financial incentives introduced for primary care physicians in Ontario but constituted only about 3% of their gross income. Incentives were paid directly to physicians, not to the group practices or medical homes in which they worked. By contrast, in a pay-for-performance scheme for primary care physicians in the United Kingdom, incentive payments made up approximately 25% of physicians’ income, were paid to the group practice, and accelerated improvements in the quality of care for some chronic diseases.27
Organizational and patient factors may also have influenced the impact of the incentives. Practices with electronic reminder systems have been shown to provide superior preventive care,28 yet only 66% of primary care physicians in Ontario reported using electronic medical records and only 37% said they could easily generate a list of patients due for preventive care.29 Cervical and breast cancer screening rates remained relatively stable over the last decade, as they have in the United States30 and the United Kingdom,31 which may relate to some patients making an informed choice not to be screened. Other patients may need access to a female provider32 or education to address misconceptions before agreeing to be screened.33 Small financial incentives directed to individual physicians are not likely to influence these factors.
We found that the largest improvement in screening associated with the pay-for-performance program was for colorectal cancer. This may relate to the lower baseline screening rate, the larger financial incentives, or a province-wide media campaign to promote colorectal cancer screening that launched in April 2008.
Income-related disparities in cancer screening persisted after the introduction of pay for performance. In contrast, the pay-for-performance scheme in the United Kingdom narrowed the gap in quality of care between practices serving low- and high-income populations.34 This difference may be due to the smaller size of Ontario’s incentives and their limitation to physicians practicing in patient-centered medical homes, which in Ontario are less likely to serve low-income patients.21 Limiting the incentives to physicians practicing in medical homes also meant that the incentives rewarded physicians with higher baseline performance, a known pitfall of pay-for-performance schemes.35
Overall, our findings are in keeping with published systematic reviews that have found limited evidence that provider incentives increase cancer screening rates.17,18 There is, however, some evidence to support the use of other interventions, including patient or provider reminders, patient financial incentives, and organizational changes such as the adoption of standing orders for non-physician staff.16,17,36 Population-based screening programs with jurisdiction-wide coverage and centrally organized recruitment and follow-up generally have higher screening rates than programs that rely solely on individual providers or organizations.16
Our study has 2 limitations that merit emphasis. First is our reliance on administrative data. We could not include fecal occult blood tests done in public hospitals and therefore likely underestimated screening rates. This limitation likely had minimal impact on our analysis of rates over time, though. Second, our observational study could not isolate the impact of the incentives from the impact of other interventions or definitively address causation. However, because we found little or no change in screening rates after the introduction of incentives, the issue of co-intervention is less relevant. In the case of colorectal cancer screening, the province-wide media campaign introduced during the same period as the physician incentives further limits inferences about the effects of either intervention on screening rates. There were no relevant changes in guidelines during the study period.
In summary, we found that a large scale pay-for-performance scheme for primary care physicians had limited impact on cancer screening rates 3 years after its widespread introduction despite substantial expenditures. Existing income-related disparities in screening rates persisted. Incentives were largely paid to physicians who had historically higher screening rates. Policy makers should consider other strategies for improving rates of cancer screening and reducing gaps in care.
Acknowledgments
We would like to thank Irfan Dhalla for his comments on the manuscript.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Dr Kiran was supported as a Clinician Investigator and Dr Glazier as a Clinician Scientist by the Department of Family and Community Medicine at the University of Toronto and at St. Michael’s Hospital.
-
Disclaimer: The opinions, results, and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
-
Prior presentations: Presented, in part, at the Family Medicine Forum Research Day, Toronto, Ontario, November 14, 2012.
-
Supplementary materials: Available at http://www.AnnFamMed.org/content/12/4/317/suppl/DC1
- Received for publication September 26, 2013.
- Revision received February 2, 2014.
- Accepted for publication March 10, 2014.
- © 2014 Annals of Family Medicine, Inc.