Abstract
PURPOSE We undertook a study to assess the impact of comparative feedback vs general reminders on practice-based referrals to a tobacco cessation quit line and estimated costs for projected quit responses.
METHODS We conducted a group-randomized clinical trial comparing the impact of 6 quarterly (18 months) feedback reports (intervention) with that of general reminders (control) on practice-based clinician referrals to a quit-line service. Feedback reports were based on an Achievable Benchmark of Care approach using baseline practice, clinician, and patient survey responses, and referrals per quarter. Comparable quit responses and costs were estimated.
RESULTS Three hundred eight clinicians participated (171 family medicine, 88 internal medicine, 49 obstetrics-gynecology) from 87 primary care practices in Michigan. After 18 months, there were more referrals from the intervention than from the control practices (484 vs 220; P <.001). Practice facsimile (fax) referrals (84%, n = 595) exceeded telephone referrals (16%, n = 109), but telephone referrals resulted in greater likelihood of enrollment (77% telephone vs 44% fax, P <.001). The estimated number of smokers who quit based on the level of services utilized by referred smokers was 66 in the feedback and 36 in the gentle reminder practices.
CONCLUSION Providing comparative feedback on clinician referrals to a quit-line service had a modest impact with limited increased costs.
INTRODUCTION
Smoking cessation interventions have proved to be effective in primary care settings according to the systematic reviews of controlled clinical trials that resulted in Clinical Practice Guidelines on Smoking Cessation by the Public Health Service.1,2 Clinicians report various reasons for not following the guidelines, such as focusing on acute or chronic care rather than preventive care, having little training in giving brief advice to quit smoking, and not being subject to accountability or feedback.3,4 Survey findings show that physicians understand the importance of smoking cessation and espouse its value, but they often do not implement the key elements of office-based methods.3,4 Physicians rarely schedule smokers for follow-up visits or arrange referrals for support services.1,3–5 Several community-based and health-system-based studies have shown fairly high rates of long-term smoking cessation (20% to 36%) by combining physician identification, advice, and referral for follow-up care with telephone support counseling.6–11 Telephone counseling services for smoking cessation (quit lines) have become widely available through health plans and state or national services,12 but they are often underutilized. Methods to enhance clinician referrals to quit-line services are needed.
Feedback on clinical performance has been studied in numerous randomized clinical trials and reported in systematic reviews to have modest effects.13 Comparing personal performance with peer performance should be a powerful motivator to change behavior.14–17 Keife et al18 reported improved preventive care for diabetic patients by providing clinicians feedback based on Achievable Benchmarks of Care (ABC). The ABC approach identifies the performance of the top 10% of clinicians as the benchmark.18 Achievable benchmarks that were based on actual performance seemed more realistic to clinicians than preset standards, and they resulted in improved care.18
We found 9 studies that involved feedback on physician performance in smoking cessation.18–26 All studies used chart audit to provide feedback, and none reported direct observation of practice. One based feedback on encounter and pharmacy claims.26 The Task Force on Community Preventive Services,27 after reviewing a few studies, reported insufficient evidence to conclude that feedback on smoking cessation increases clinician advice to quit smoking or patient tobacco cessation. No reported studies have based feedback on achievable benchmarks for clinician performance in smoking cessation. Business models of feedback to employees stress that the most effective feedback should be clear, specific to the expected performance, reliable, comparable with peer performance, and repeated over time.28 We developed ABC feedback based on actual clinician performance in making quit-line referrals. We assessed the impact of comparative feedback or general reminders on practice-based referrals to a tobacco cessation quit line and estimated the costs for the projected quit responses.
METHODS
Study Design
We conducted a cluster-randomized clinical trial assessing the impact of quarterly comparative feedback (intervention) vs general reminders (control) to primary care clinicians within practices on their performance in generating referrals to a telephone quit-line service over an 18-month period between January 1, 2003, to June 30, 2004. We randomly assigned each practice to comparative feedback or general reminders so that all participating clinicians within a practice received the same type of feedback. For practices assigned to the intervention group, clinicians were given quarterly comparative feedback on their individual and group referrals and how their referral activity compared with the mean referrals based on the overall top 10% of clinicians. Estimated quit responses and the associated incremental costs of the intervention were calculated based on the level of service obtained by referred smokers.
The quit-line services for this study were provided by Blue Cross Blue Shield of Michigan (BSBEM), which serves nearly 5 million enrollees in Michigan. The BCBSM Quit the Nic Quit Line is a 24-hour service for smoking cessation counseling and relapse prevention support provided by trained nurses using computer counseling software. Smokers are offered 2 levels of service. Level 1 requires participants to set a quit date in the next 14 days and includes an enrollment intake session (lasting 45 minutes with motivational support to set a quit date and prevent relapse), a mailed quit kit, and 6 proactive telephone sessions at intervals of 1, 3, 7, 14, 30, and 60 days after the quit date, with follow-up at 90, 180, and 360 days after the quit date.7 Level 2 includes a brief intake session (lasting 10 minutes), a mailed quit kit, and several callbacks inviting them into level-1 service. All participants are advised to use pharmacotherapy to assist with withdrawal symptoms if recommended by their physician. The Quit Line service reports long-term success rates of 25% to 30% for level-1 enrollees, a finding consistent with other reports.7,9–11 The Quit Line service agreed to provide counseling services to all referred smokers from study participants regardless of their health plan.
Interventions
ABC feedback was based on a method developed by Keife and colleagues18 and was determined to be the mean number of referrals per quarter for the top 10% of the clinicians. The benchmark calculation for individual clinician referrals was based on an estimate of smoker visits per quarter calculated to account specifically for each clinician’s unique baseline characteristics, including half-days worked per week, adult patients per clinic, and smoking prevalence within the clinic population. A baseline quarter (October 2002 to December 2002) was used to set the benchmark for the first quarter (beginning January 1, 2003) of the comparative feedback intervention. Clinicians received either a mailed comparative feedback report or a general reminder about Quit Line services each quarter for 6 quarters. The benchmark referrals were reset after each quarter based on the clinicians’ performance during the previous quarter.
The comparative feedback report (Figure 1⇓) was a 1-page report showing 2 graphs. The first graph displayed the quarter benchmark for referrals, the individual clinician, his or her practice group, and the performance of the study group in numbers of referrals and percentage relative to the benchmark. The second graph showed the actual number of referrals made by the individual clinician per quarter.
The general reminders, which were sent to control practices and their clinicians on the same quarterly basis as the feedback reports, were simple postcards informing them only of the Quit Line services.
During the baseline quarter before the feedback intervention, all 308 participating clinicians received a mailed compact disk (developed by W.C.W. and J.S.H.) that included an overview of the clinical practice guidelines on smoking cessation, case examples, and Quit Line referral information. Clinicians who completed a posttest questionnaire (n = 53, 17%) received 2 hours of CME credit, a response rate that was consistent for both intervention and control groups.
Recruitment and Participants
Because the study was a collaboration between Michigan State University and the health plan Quit Line service, all participating clinicians and practices accepted patients covered by the plan. From March to July 2002, we recruited primary care clinicians (in family medicine, internal medicine, and obstetrics-gynecology) from a health plan list of 3,777 primary care practices. Only 858 practices seemed to fit the definition for usual primary care. Of these 858 practices, 102 agreed to participate during telephone recruitment, and completed baseline survey forms were received from 87 (85.3%); 308 of 371 clinicians (83%) within the 87 practices completed baseline survey forms.
Practice directors completed a prestudy survey form assessing practice characteristics (Table 1⇓). Clinicians completed a prestudy survey form assessing their characteristics (Table 2⇓), including the number of half-days worked and adult patients seen in each practice.
The 87 participating practices were located across regions of Michigan, including rural, urban, and suburban locations (Tables 1⇑ and 2⇑). The practices comprised 308 clinicians, with 55% (171) practicing family medicine, 29% (88) general internal medicine, and 16% (49) obstetrics-gynecology.
Baseline exit surveys of patients were conducted in each practice before the study for 2 half-days in which most clinicians were in the practice. Research assistants approached all patients (aged 18 years and older) as they exited from their visit. A brief (5-minute), 1-page questionnaire assessed clinician’s use of the 5 As (ask, advise, assess, assist, arrange) during the visit; age; sex; insurance status; race/ethnicity; smoking status; and, if a current smoker, number of cigarettes smoked per day and readiness to quit. The questionnaire was pilot tested in non–study clinics, and questions were adapted from previously developed exit surveys of smoking cessation.29,30
Of the 3,686 eligible patients, 3,239 (87.9%) were approached for inclusion; 1,992 patients completed the questionnaire (54% of all eligible and 61.5% of all approached). Illness and lack of time were main reasons for not completing the questionnaire. Nineteen (19%) of surveyed patients were current smokers (n = 371). The percentages of clinician asking, advising, assessing, assisting, and arranging follow-up were 58%, 42%, 34%, 22%, and 8%, respectively. There were no significant differences in prevalence of smoking or delivering of the 5 As between intervention and control practices at baseline.
All referrals to the Quit Line by study clinicians were identified by the Quit Line staff and categorized based on clinician; practice; level of service utilized—level 0 (clinician advice alone, no Quit Line intake), level 1 (enrolled in Quit Line), and level 2 (intake, information only); and type of referral—facsimile (fax) or telephone.
The study design, questionnaires, and survey forms were all approved by the Michigan State University Institutional Review Board and 25 local boards.
We estimated 12-month quit responses based on the number of referrals in each group and the level of service obtained. We estimated the quit responses based on the expected quit responses reported in the literature for the various levels of service and on the previous results reported by the services for level 1 (enrollment and Quit Line use with usual quit rate of 25%), level 2 (brief intake and information only with usual quit rate of 15%), and level 0 (no intake and clinician advice alone with usual quit rate of 10%).7,11,29,31–33
Measures and Outcomes
Key measures and outcomes included (1) changes from baseline to postintervention (18 months) in clinician referrals in both intervention and control groups, (2) estimated quit responses of referred smokers, and (3) estimated incremental costs of the feedback intervention.
Analyses
Data were pooled and maintained by the health plan data manager and transferred to Michigan State University for evaluation and statistical analysis. We determined baseline differences between practice, clinician, and patient characteristics using χ2 tests and t tests, as appropriate. The Mantel-Haenszel χ2 test for repeated measures was used to assess the difference in the primary measure (number of referrals) between the intervention and control groups, over time, which was the main unit of analysis. The study had the power to detect a 25% difference, which required nearly 80 practices and 300 clinicians. Intraclass correlation (ICC) was calculated (see the online-only Supplemental Appendix, which can be found at http://www.annfammed.org/cgi/content/full/5/2/135/DC1). All analyses were conducted using SAS statistical software, version 9.1 (SAS Institute Inc, Cary, NC).
Resource costs were used to calculate the costs of services in both groups. The mean annual salary and benefit cost were used for nurse ($54,600), analyst ($72,602), clerk ($26,000), and statistician ($103,125) staff. These costs were applied to 173 hours of the nurses’ time for receiving and processing telephone and fax referrals and to 236 hours of administrative staff time to develop and send the feedback reports as well as other program-related tasks. Actual supply costs were also used for printing and mailing feedback reports, letters, quit kits, and smoking cessation program materials. Costs were categorized as fixed or variable, an important difference in that the latter costs are incurred and change with intervention volumes. Because the Quit Line program was already purchased, available, and staffed, its cost represented a fixed cost. Increased use of the program associated with the interventions did not result in an incremental marginal cost increase relative to the control group, although in the long run, more volume can be expected to generate higher mean costs.
RESULTS
Absolute Quit Line referrals from baseline to 18 months (Figure 2⇓) showed a significant difference between intervention and control groups over time. Both groups increased during the fifth quarter, which may be explained by usual increases in commitment to quit smoking during January. Overall, referrals were very modest, considering the number of estimated visits by smokers per quarter. Because of a high intraclass correlation (ICC = .902) and a lack of adequate power to detect significant differences, individual clinician- and practice-level analyses were not completed.
We also analyzed the type of referral (office fax vs telephone). There were far greater referrals by fax than by telephone. The intervention practices had 427 (88.2%) fax and 57 (11.8%) telephone referrals. The control practices had 168 (76.4%) fax and 52 (23.6%) telephone referrals. Telephone referrals (84 enrolled vs 25 not enrolled) compared with fax referrals (249 enrolled vs 342 not enrolled) were significantly more likely than fax referrals to result in Quit Line enrollment (77.1% vs 42.1%, P <.001).
More estimated quits (Table 3⇓) occurred in the intervention practices than in the control practices (66 vs 36 quits).
The relative cost per outcome (Table 4⇓) based on estimated quits shows a higher cost in the intervention group, which was expected because more costs were incurred as a result of program implementation. Incremental costs are summarized in the bottom half of the table, which exclude program development costs, such as feedback report design and testing and database compilation. Incremental costs reflect the ongoing expense of sustaining this type of smoking cessation service of comparable feedback to clinicians.
Figure 3⇓ shows the final summary of results, including practices and clinicians who received either intervention feedback or control announcements during the entire study period.
DISCUSSION
An ABC feedback approach for clinicians can be developed and implemented by a health plan based on comparative referrals to quit-line services and appears to increase referrals over time, though the impact is very modest considering the magnitude of estimated visits by smokers. The ABC benchmark requires data and mechanisms usually not developed by health plans, such as linking referred smokers to individual clinicians, estimating the prevalence of smokers in practices, and knowing clinical time spent in practice per clinician. Being at the interface of clinical care and public health, most quit lines do not feed back data to referring clinicians. The added cost to develop clinician-specific feedback in our study was about $65 per estimated quit, which is minimal when considering almost twice the number of quits in the feedback group and the long-term benefits of smoking cessation.
It has been reported that less than 5% of smokers will accept referrals to receive special smoking cessation services such as group counseling.1,34 Although we detected a significant difference in referrals between our intervention and control groups, 704 referrals from more than 300 clinicians during 18 months is a very small response even when a low percentage of smokers generally agree to be referred. Estimating that 10% to 20% of smokers in primary care practice are usually willing to quit, there were potentially 2,200 to 4,500 referrals per quarter. The practices and clinicians were representative of primary care practice across Michigan and had performances on the 5 As guidelines that were consistent with other reports.2–4
Even though the number of fax referrals was greater than telephone referrals, the probability of enrollment was greater for telephone referrals, which may be explained by the level of self-motivation of smokers who telephone on their own initiative. Some clinicians may have referred smokers by fax to insure their feedback credit.
There are several limitations to this study. A high intraclass correlation and lack of statistical power did not permit analysis of individual clinician or practice differences. The exit surveys assessed only 2 half-days per practice and may not accurately reflect longitudinal changes for individual clinicians, but they are valid for the entire group. We estimated the number of quits based on the reported experience of the Quit Line, which uses self-reports.35–37 Smoker visits per quarter were estimated from baseline surveys only.
We interviewed by telephone 28 of the top-referring clinicians (top 10%) in this study. They consistently reported having a method (usually by staff) to identify all smoking patients, offered referrals to all interested smokers, and appreciated having 1 telephone counseling service. Primary care offices should consider having clinical staff identify tobacco use, offer advice to quit, and initiate referrals,37 which is consistent with the new Ask and Act initiative of the AAFP (American Academy of Family Physicians).38 Electronic medical records could also be used to incorporate easy documentation of the 5 As, offer simple cueing supports for clinicians, and tailor pharmacotherapies and referral services based on individual patient health care coverage.39 Clinicians may also be more motivated to refer smokers for services if they receive information on the outcomes of their patients. Quit lines and health plans should provide reports to clinicians on referred smokers. In fact, the staff-model managed care plan within this study has decided to provide ongoing reports on referred smokers based on our findings.
In conclusion, providing simple, clear, and comparable feedback on referrals to a quit-line service had a modest impact on the referral performance of clinicians and may require a major integration of practice and system supports to become truly effective.
Acknowledgments
Sam Vansen, MSHA, project leader, Outcomes Measures and Evaluation of BCBSM for data coordination; T. Jann Caison-Sorey, MD, MSA, Medical Director, Blue Healthline of BCBSM; Michelle Armstrong, senior analyst, guided self-management and Quit the Nic Quit Line services of BCBSM; Linda White, RN, MPH, Project Director, formerly from Michigan State University Department of Family Practice; Arlene Weismantel, MS, academic librarian for systematic literature review; Pramod Pathak, PhD, biostatistics consultant, Michigan State University Department of Epidemiology; Thomas Ruane, MD, Medical Director, PPO and Health Systems of BCBSM; Tim Reed, MS, Blue Care Network Director of BCBSM; Great Lakes Research into Practice (GRIN) Network; and participating physicians and clinicians across Michigan.
Footnotes
-
Conflicts of interest: William Wadland, MD, MS, of MSU, and BCBSM have a license agreement for software usage for Quit the Nic (the cessation program used in this study), which was originally developed as “I’d rather cope than smoke.”
-
Funding support: Addressing Tobacco in Managed Care, Robert Wood Johnson Foundation, grant No. 43968. Department of Family Medicine support for CD-ROM, audiotapes “Enhancing Smoking Cessation in Your Medical Practice,” and data analysis.
- Received for publication February 2, 2006.
- Revision received September 11, 2006.
- Accepted for publication September 27, 2006.
- © 2007 Annals of Family Medicine, Inc.