Increasing Rates of Tobacco Treatment Delivery in Primary Care Practice: Evaluation of the Ottawa Model for Smoking Cessation
=============================================================================================================================

* Sophia Papadakis
* Adam G. Cole
* Robert D. Reid
* Mustafa Coja
* Debbie Aitken
* Kerri-Anne Mullen
* Marie Gharib
* Andrew L. Pipe

## Abstract

**PURPOSE** We report on the effectiveness of the Ottawa Model for Smoking Cessation (OMSC), a multicomponent knowledge translation intervention, in increasing the rate at which primary care providers delivered smoking cessation interventions using the 3 A’s model—*Ask*, *Advise*, and *Act*, and examine clinic-, provider-and patient-level determinants of 3 A’s delivery.

**METHODS** We examined the effect of the knowledge translation intervention in 32 primary care practices in Ontario, Canada, by assessing a cross-sectional sample of patients before the implementation of the OMSC and a second cross-sectional sample following implementation. We used 3-level modeling (clinic, clinician, patient) to examine the main effects and predictors of 3 A’s delivery.

**RESULTS** Four hundred eighty-one primary care clinicians and more than 3,500 tobacco users contributed data to the evaluation. Rates of delivery of the 3 A’s increased significantly following program implementation (*Ask*: 55.3% vs 71.3%, *P* <.001; *Advise*: 45.5% vs 63.6%, *P* <.001; *Act*: 35.4% vs 54.4%, *P* <.001). The adjusted odds ratios (AOR) for the delivery of 3 A’s between the pre- and post-assessments were AOR = 1.94; (95% CI, 1.61–2.34) for *Ask*, AOR = 1.92; (95% CI, 1.60–2.29) for *Advise*, and AOR = 2.03; (95% CI, 1.71–2.42) for *Act*. The quality of program implementation and the reason for clinic visit were associated with increased rates of 3 A’s delivery.

**CONCLUSIONS** Implementation of the OMSC was associated with increased rates of smoking cessation treatment delivery. High quality implementation of the OMSC program was associated with increased rates of 3 A’s delivery.

*   smoking cessation
*   primary care
*   evidence-based practice
*   knowledge translation
*   health services research

## INTRODUCTION

The importance of smoking cessation as a preventive strategy is unparalleled. Primary care practice is an important setting for intervening with tobacco users and supporting cessation.1,2 International clinical practice guidelines recommend 5 strategies as the basis for smoking cessation interventions in clinical settings.1,3–5 The 5 A’s strategies are *ask* (identify smoking status), *advise* patients to quit smoking, *assess* readiness to quit, *assist* with making a quit attempt, and *arrange* follow-up.

The 3 A’s (*Ask, Advise, Act*) model is an adaptation of the 5 A’s that is based on the involvement of multiple health professionals in delivering treatment.6,7 These evidence-based smoking cessation treatment models have been shown to increase quit attempts and the rates of successful cessation.1,3–5

Many providers find it challenging to deliver evidence-based cessation treatment in the context of a busy primary care practice. Meta-analyses have determined that multicomponent interventions, which combine patient-, provider-, and clinic-level support, are most effective in increasing rates of 5 A’s delivery in primary care practice settings and increasing long-term smoking abstinence among patients.8,9 Despite evidence from multiple well-designed randomized controlled trials, multicomponent interventions have not been generally implemented.10–12

The Ottawa Model for Smoking Cessation (OMSC) is a multicomponent intervention originally designed for use in hospitals; the model has now been adapted for use in primary care practices.6,7 The OMSC uses the 3 A’s (*Ask, Advise, Act)* model.6,7 We previously demonstrated the efficacy of the OMSC as part of a pilot study involving 8 primary care practices; that study, a randomized, controlled trial, demonstrated significant improvements in tobacco treatment delivery.13,14 The efficacy of interventions can be quite different, however, in day-to-day clinical practice. The importance of translating knowledge of and evaluating evidence-based practices in ‘real world’ practice settings has been widely acknowledged.15,16 The purpose of this evaluation was to examine the effectiveness of the OMSC in increasing the rate at which primary care providers delivered smoking cessation interventions using the 3 A’s model (*Ask*, *Advise*, and *Act*), and to examine clinic-, provider-and patient-level determinants of 3 A’s delivery.

## METHODS

### Evaluation Design

From each of the participating primary care practices, consecutive adult patients who smoked were surveyed twice—once before intervention and again following intervention—to assess changes in levels of 3 A’s delivery. Evaluation participants also completed a telephone follow-up assessment 6 months following the exit survey in order to assess rates of smoking abstinence. The results of the follow-up will be reported in a separate publication. Ethics clearance under the category of program evaluation was received from the University of Ottawa Heart Institute Human Research Ethics Board. We followed guidelines for the reporting of knowledge translation interventions and before-and-after evaluations.17–19

### Clinic and Provider Recruitment

All family health teams located in 5 of Ontario’s 14 health regions received a mailed invitation to participate in the OMSC program. (Family health teams are interdisciplinary teams that include physicians, nurses, and other health professionals.) The evaluation sample consisted of 32 practices, each with 1 family health team.

### Data Collection

The characteristics of participating clinics, including implementation of the OMSC 10 Best Practices for smoking cessation in clinical settings, were documented at baseline. All clinicians from participating practices completed surveys.

At each participating family health team, consecutive patients arriving for appointments were screened for eligibility. Patients were eligible to participate if they smoked 1 or more cigarettes per day, were at least 18 years of age, had scheduled an appointment with a physician or nurse practitioner, and were able to complete an exit survey in English or French. A trained research assistant coordinated all screening and data collection activities in clinic waiting rooms. Patients completed their surveys following their clinic appointments to reduce the likelihood of survey-prompted patient-provider discussions about smoking.20

After all participating primary care practices had implemented the OMSC program for at least 4 months, post-implementation data were collected using procedures identical to pre-implementation data collection.

### The Ottawa Model for Smoking Cessation Intervention Program

#### The 3 A’s Framework

The OMSC facilitates delivery of a standardized smoking cessation intervention based on the 3 A’s framework using an interdisciplinary approach to tobacco treatment delivery. In the study, responsibility for the 3 steps was divided. *Asking* about smoking status was the responsibility of nursing staff or medical assistants. *Advising* (delivering advice and a brief intervention) and *acting* (referring patients to a clinic nurse, nurse practitioner, or pharmacist for a dedicated cessation consult) were the responsibility of the physician or nurse practitioner. The health professional providing the dedicated cessation consult offered counseling, addressed issues of pharmacotherapy, and scheduled follow-up visits.

#### Multicomponent Intervention Implementation

The multicomponent knowledge translation intervention was designed to help primary care clinics by introducing the OMSC 10 Best Practices for delivering tobacco treatment in primary care settings. The 10 Best Practices and intervention components had been selected following a review of the literature of evidence-based strategies for integrating smoking cessation in primary care settings.9 Table 1 provides a summary of the intervention components.

View this table:
[Table 1](http://www.annfammed.org/content/14/3/235/T1)

Table 1 
Summary of the Multicomponent Knowledge Translation Intervention

### Measures

#### Provider Performance in 3 A’s Delivery

To assess delivery of the 3 A’s, patients were asked whether their physician or another health care provider had *asked* them about their smoking status, *advised* them to quit smoking, and *acted* to provide assistance with quitting or arrange follow-up support. Patient exit surveys have been used in several large trials in the primary care setting to assess tobacco treatment delivery.21–24

### Predictor Variables

Clinic-level variables included the geographic location of the clinic, the number of physicians in the practice, and the presence of a physician champion (defined as a physician who takes leadership for implementation of the OMSC program within the clinic and supports spread of the program, as assessed by the facilitator assigned to work with the clinic).

Provider-level variables included age, sex, previous participation in smoking cessation training, provider beliefs about the importance of smoking cessation, and self-efficacy in 3 A’s delivery.

Patient-level variables included age, sex, years of formal education, presence of comorbidities, years of tobacco use, readiness to quit, cessation self-efficacy, and nicotine dependence as measured by the Heaviness of Smoking Index.25

### Sample Size

Sample size was adjusted for the cluster design using an intra-class correlation coefficient (ICC) of 0.05 based on previously published data.26,27 Calculations were based on a 2-sided test and an α of 0.05 with 90% power, a minimum 10% difference between the preimplementation survey and postimplementation survey results, and a sample of 32 primary care practices. Sample size calculations indicated a minimal sample of n = 52 at both the pre- and post-implementation assessment.

### Statistical Analysis

Differences in the distribution of selected patient-level characteristics between the 2 time periods were determined using the Pearson χ2 test for categorical variables. Variables that differed significantly (*P* <.05) between pre- and postimplementation assessment were controlled for in subsequent analyses. Pre- and postimplementation assessment rates were calculated for each of the evaluation outcomes (the 3 A’s: *Ask, Advise, Act*). All models controlled for clinic- and provider-level clustering. The ICC was calculated to compare the variation between clusters to the total variation; this was measured on a scale from 0 to 1, with a value close to 0 indicating the clusters were all “similar.” Multi-level models were used to identify the clinic- and provider-level variance and the influence of assessment time on 3 A’s delivery. All analyses were completed using SAS software version 9.4 (SAS Institute, Inc).

## RESULTS

### Description of Recruitment

The flow diagram for the evaluation is presented as Figure 1. Working with 32 primary care practices and 481 clinicians, we collected preintervention surveys from 1,919 patients (62.1% of those eligible) and postintervention surveys from 1,951 patients (57.4% of those eligible).

![Figure 1](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/14/3/235/F1.medium.gif)

[Figure 1](http://www.annfammed.org/content/14/3/235/F1)

Figure 1 
Flow of practices and patients through the study.

### Clinic, Provider, and Patient Characteristics

Primary care practices included representation from urban (23%), suburban (24%), and rural (54%) clinics. Seventy-two percent of clinics had 10 or more physicians within the practice. Physicians had a mean age of 45.7 years (SD = 10.6 years). Thirty-seven percent of physicians had participated in smoking cessation training in the past.

Table 2 displays the demographic profiles of patients at the pre- and post-intervention assessments. Differences between the pre- and postintervention samples in the proportion of females, time to the day’s first cigarette, and type of visit were documented and controlled for in all analyses.

View this table:
[Table 2](http://www.annfammed.org/content/14/3/235/T2)

Table 2 
Patient Demographics at Times of Assessment

### Effect of Intervention on 3 A’s Delivery

Rates of delivery of the 3 A’s increased significantly following program implementation (Figure 2):

*   *Ask*: 55.3% vs 71.3%, *P* <.001

*   *Advise*: 45.5% vs 63.6%, *P* <.001

*   *Act* to assist with cessation: 35.4% vs 54.4%, *P* <.001

![Figure 2](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/14/3/235/F2.medium.gif)

[Figure 2](http://www.annfammed.org/content/14/3/235/F2)

Figure 2 
Clinic performance in the 3 A’s delivery before and after OMSC knowledge translation intervention.

AOR = adjusted odds ratio; OMSC = Ottawa Model for Smoking Cessation; 3 A’s = *Ask, Advise, Act*.

Note: The AORs presented control for clinic- and provider-level variance between clusters, availability of cost-free nicotine replacement therapy, gender of the patient, self-reported time of first cigarette, and purpose of visit; based on inclusion of 32 clinics and 481 providers. P values are based on the Wald statistic.

The adjusted odds ratios (AORs) and 95% confidence intervals for the delivery of 3 A’s were as follows:

*   *Ask*: AOR = 1.94; 95% CI, 1.61–2.34

*   *Advise*: AOR = 1.92; 95% CI, 1.60–2.29

*   *Act* to assist with cessation: AOR = 2.03; 95% CI, 1.71–2.42

Moderate to large intra-clinic and intra-provider variability was observed in the rates at which the 3 A’s were delivered to patients. The ICC for clinic-level variation ranged between 0.044 and 0.086; the ICC for provider-level variation ranged between 0.041 and 0.066.

### Patient-, Clinician-, and Clinic-level Predictors of 3 A’s Delivery

The presence of a physician champion predicted rates at which patients were asked about their smoking status, but did not predict the other A’s (Table 3). Clinicians’ beliefs regarding the importance of cessation predicted rates at which advice and support with cessation were delivered. Patients presenting for an annual periodic exam were more likely to receive the 3 A’s than patients who were seen for a follow-up appointment. Several patient-level factors were associated with 3 A’s delivery (Table 3).

View this table:
[Table 3](http://www.annfammed.org/content/14/3/235/T3)

Table 3 
Final Model for the Multi-Level Analysis of Clinic-, Clinician-, and Patient-Level Characteristics Associated With Rates of Clinician Delivery of the 3 A’s (*Ask, Advise, Act*)

### Implementation Factors

Table 4 displays the rates at which the OMSC 10 Best Practices were implemented across clinics at the pre- and postintervention points. Clinics that had more than 8 of the 10 Best Practices in place had higher rates of 3 A’s delivery: *Ask* (AOR = 1.9; 95% CI, 1.3–2.8); *Advise* (AOR = 1.8; 95% CI, 1.2–2.7); *Act* to assist with cessation (AOR = 1.7; 95% CI, 1.1–2.6).

View this table:
[Table 4](http://www.annfammed.org/content/14/3/235/T4)

Table 4 
Implementation of Ottawa Model for Smoking Cessation 10 Best Practices Before and After Intervention

## DISCUSSION

This knowledge translation evaluation examined the association between implementation of the OMSC program and rates at which the 3 A’s were delivered to tobacco users in primary care practices. The effect sizes documented in this evaluation are consistent with rates observed in our earlier randomized trial evaluation of the OMSC program.13,14 The present evaluation was not a randomized controlled trial but rather focused on the broader implementation of an evidence-based intervention in “real-world settings.” The large sample allowed us to test the generalizability of the OMSC program. The OMSC program combines outreach facilitation, training, EMR tools, and audit and feedback, all of which have been shown to be important modifiers of practice behavior in primary care.9,28,29 This evaluation lends support to existing evidence regarding the effectiveness of multicomponent interventions in influencing tobacco treatment delivery in primary care settings.9

Our analysis found that implementation of the OMSC 10 Best Practices was an important independent predictor of enhanced 3 A’s delivery. That high quality implementation of the program was associated with higher rates of 3 A’s delivery reinforces the importance of ensuring fidelity to the intervention model. Providers were significantly more likely to address tobacco use during periodic exams, indicating that clinicians may be missing opportunities when patients present for other reasons. The presence of a physician champion was associated with significantly improved clinic performance in the delivery of cessation interventions. Increasing levels of formal education were associated with a decreasing probability that a patient will be asked about tobacco use. Additionally, patients older than 55 years were significantly more likely to be advised to quit smoking than their younger counterparts.

A significant degree of provider-level variance was documented in the present evaluation, but the variation was not sufficiently explained by the provider-level characteristics examined as part of this investigation. Future research should seek to better understand the nature of this variation and investigate intervention strategies for addressing low providor performance. Likewise, further exploration of the role of the physician champion may be useful for informing future interventions.

These results should be interpreted in light of the evaluation limitations. The design allows us to speak of associations but not causality in observed differences in 3 A’s delivery. We adjusted for the covariation observed in the pre- and postintervention data in the multi-level analysis, but we have no way of knowing whether factors that were not assessed also contributed to this covariation. We had a 60% participation rate in the exit surveys. Limited data was available on the profile of non-respondents, so we are unable to accurately assess how the evaluation sample may differ from the overall population of primary care patients who smoke. This limited data thereby limits the generalizability of our evaluation findings. Given that this evaluation was conducted in Ontario, Canada, in one type of primary care setting (ie, multidisciplinary family health teams), the generalizability of the results to other settings and practice models must be considered. Clinics that chose to participate in the evaluation might have been more motivated than those who declined participation. Our evaluation examined outcomes 4 months following OMSC introduction in clinics. Additional evaluation data are needed to determine whether the initial improvements in 3 A’s delivery are maintained over time.

## Footnotes

*   Conflicts of interest: Dr Reid has received speaker and consulting fees from Pfizer, Inc and Johnson & Johnson; Dr Pipe has received speaker and consulting fees from Pfizer, Inc, Johnson & Johnson, and Amgen, Inc. All other authors report no conflicts of interest.

*   **Funding support:** The Ottawa Model for Smoking Cessation in Primary Care Ontario Expansion Program, a knowledge translation initiative, was funded by an educational grant from the Heart and Stroke Foundation of Ontario and Pfizer, Inc.

*   **Previous presentation:** Presented at the Annual Meeting of the Society for Nicotine and Tobacco Research; February 5–8, 2014; Seattle, Washington.

*   Received for publication May 26, 2015.
*   Revision received November 5, 2015.
*   Accepted for publication November 22, 2015.


*   © 2016 Annals of Family Medicine, Inc.

## References

1.  World Health Organization. WHO Report on the Global Tobacco Epidemic. 2008: The MPOWER package. 2008.
    
    

2.  Vardavas CI, Symvoulakis EK, Lionis C. Dealing with tobacco use and dependence within primary health care: time for action. [editorial]. Tob Induc Dis. 2013;11(1):6.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1617-9625-11-6&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23442803&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

3.  Fiore MC, Jaén CR, Baker TB, et al. Treating Tobacco Use and Dependence: 2008 Update. Clinical Practice Guideline. Rockville, MD: US Department of Health and Human Services. Public Health Service. May 2008.
    
    

4.  European Network for Smoking and Tobacco Prevention. European smoking cessation guidelines: The authoritative guide to a comprehensive understanding of the implications and implementation of treatments and strategies to treat tobacco dependence. October 2012.
    
    

5.  New Zealand Ministry of Health. The New Zealand guidelines for helping people to stop smoking. June 2014. [http://www.health.govt.nz/system/files/documents/publications/nz-guidelines-helping-people-stop-smoking-jun14.pdf](http://www.health.govt.nz/system/files/documents/publications/nz-guidelines-helping-people-stop-smoking-jun14.pdf).
    
    

6.  Reid RD, Pipe AL, Quinlan B. Promoting smoking cessation during hospitalization for coronary artery disease. Can J Cardiol. 2006;22(9):775–780.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0828-282X(06)70294-X&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16835672&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000239053400007&link_type=ISI) 

7.  Reid RD, Mullen KA, Slovinec D’Angelo ME, et al. Smoking cessation for hospitalized smokers: an evaluation of the “Ottawa Model”. Nicotine Tob Res. 2010;12(1):11–18.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoibmljdG9iIjtzOjU6InJlc2lkIjtzOjc6IjEyLzEvMTEiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzE0LzMvMjM1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

8.  Anderson P, Jané-Llopis E. How can we increase the involvement of primary health care in the treatment of tobacco dependence? A meta-analysis. Addiction. 2004;99(3):299–312.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1360-0443.2003.00672.x&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14982543&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000189379600009&link_type=ISI) 

9.  Papadakis S, McDonald P, Mullen KA, Reid R, Skulsky K, Pipe A. Strategies to increase the delivery of smoking cessation treatments in primary care settings: a systematic review and meta-analysis. Prev Med. 2010;51(3–4):199–213.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.ypmed.2010.06.007&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20600264&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

10. Papadakis S, Gharib M, Hambleton J, Reid RD, Assi R, Pipe AL. Delivering evidence-based smoking cessation treatment in primary care practice: experience of Ontario family health teams. Can Fam Physician. 2014;60(7):e362–e371.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjk6IjYwLzcvZTM2MiI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTQvMy8yMzUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. Reid JL, Hammond D, Rynard VL, Burkhalter R. Tobacco Use in Canada: Patterns and Trends. Waterloo, ON: Propel Centre for Population Health Impact, University of Waterloo; 2014.
    
    

12. Jamal A, Dube SR, Malarcher AM, Shaw L, Engstrom MC; Centers for Disease Control and Prevention (CDC). Tobacco use screening and counseling during physician office visits among adults—National Ambulatory Medical Care Survey and National Health Interview Survey, United States, 2005–2009. MMWR Morb Mortal Wkly Rep. 2012;61(Suppl):38–45.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22695462&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

13. Papadakis S, McDonald PW, Pipe AL, Letherdale ST, Reid RD, Brown KS. Effectiveness of telephone-based follow-up support delivered in combination with a multicomponent smoking cessation intervention in family practice: a cluster-randomized trial. Prev Med. 2013;56(6):390–397.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.ypmed.2013.02.018&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23480968&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

14. Papadakis S, Aitken D, Gocan S, et al. A randomised controlled pilot study of standardised counselling and cost-free pharmacotherapy for smoking cessation among stroke and TIA patients. BMJ Open. 2011;1(2):e000366.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiMS8yL2UwMDAzNjYiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzE0LzMvMjM1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. Zapka J, Goins KV, Pbert L, Ockene JK. Translating efficacy research to effectiveness studies in practice: lessons from research to promote smoking cessation in community health centers. Health Promot Pract. 2004;5(3):245–255.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BocHAiO3M6NToicmVzaWQiO3M6NzoiNS8zLzI0NSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTQvMy8yMzUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

16. Woolf SH. The meaning of translational research and why it matters. JAMA. 2008;299(2):211–213.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.2007.26&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=18182604&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000252182600022&link_type=ISI) 

17. National Heart, Lung and Blood Institute. Quality assessment tool for before-after (pre-post) studies with no control group. [http://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools/before-after](http://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools/before-after). Published 2014.
    
    

18. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (standards for QUality improvement reporting excellence): Revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2015.
    
    

19. Albrecht L, Archibald M, Arseneau D, Scott SD. Development of a checklist to assess the quality of reporting of knowledge translation interventions using the workgroup for intervention development and evaluation research (WIDER) recommendations. Implement Sci. 2013;8:52.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1748-5908-8-52&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=23680355&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

20. Coleman T, Wilson A, Barrett S, Wynne A, Lewis S. Distributing questionnaires about smoking to patients: impact on general practitioners’ recording of smoking advice. BMC Health Serv Res. 2007;7:153.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1472-6963-7-153&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17892574&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

21. Katz DA, Brown RB, Muehlenbruch DR, Fiore MC, Baker TB; AHRQ Smoking Cessation Guideline Study Group. Implementing guidelines for smoking cessation: comparing the efforts of nurses and medical assistants. Am J Prev Med. 2004;27(5):411–416.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15556742&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000225205400009&link_type=ISI) 

22. Pbert L, Adams A, Quirk M, Hebert JR, Ockene JK, Luippold RS. The patient exit interview as an assessment of physician-delivered smoking intervention: a validation study. Health Psychol. 1999;18(2):183–188.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1037/0278-6133.18.2.183&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10194054&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000079255500010&link_type=ISI) 

23. Katz DA, Muehlenbruch DR, Brown RB, Fiore MC, Baker TB; AHRQ Smoking Cessation Guideline Study Group. Effectiveness of a clinic-based strategy for implementing the AHRQ Smoking Cessation Guideline in primary care. Prev Med. 2002;35(3):293–301.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1006/pmed.2002.1073&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12202073&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000177992200013&link_type=ISI) 

24. Unrod M, Smith M, Spring B, DePue J, Redd W, Winkel G. Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med. 2007;22(4):478–484.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-006-0069-0&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17372796&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000245107800009&link_type=ISI) 

25. Perez-Rios M, Santiago-Perez MI, Alonso B, Malvar A, Hervada X, de Leon J. Fagerstrom test for nicotine dependence vs heavy smoking index in a general population survey. BMC Public Health. 2009;9:493.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1471-2458-9-493&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20042106&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

26. Baskerville NB, Hogg W, Lemelin J. The effect of cluster randomization on sample size in prevention research. J Fam Pract. 2001;50(3):W241–6.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=11252223&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 

27. Campbell MK, Thomson S, Ramsay CR, MacLennan GS, Grimshaw JM. Sample size calculator for cluster randomized trials. Comput Biol Med. 2004;34(2):113–125.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0010-4825(03)00039-8&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=14972631&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000220088400002&link_type=ISI) 

28. Boyle R, Solberg L, Fiore M. Use of electronic healthrecords to support smoking cessation. Cochrane Database of Systematic Reviews2014, Issue 12. Art. No.: CD008743.
    
    

29. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=22696318&link_type=MED&atom=%2Fannalsfm%2F14%2F3%2F235.atom)