Interventions to Improve Influenza and Pneumococcal Vaccination Rates Among Community-Dwelling Adults: A Systematic Review and Meta-Analysis

The authors have undertaken a huge, complex and commendable task in assessing best methods for increasing influenza and pneumococcal vaccination rates.

Methods:
Literature search. The search is restricted to English language studies, which introduces an unknown bias. The search concluded in August 2010, and was not updated for this 2012 publication.

Assessment of risk of bias: The authors included randomised and non-randomised studies, and chose the Downs and Black instrument, which assesses "methodological quality." This instrument was developed in 1998, and has not undergone further development. It was based on assessments of 10 randomised and 10 non-randomised studies. The pilot was found to have unsatisfactory qualities, and was revised using the same 20 articles. Inter-rater reliability correlations for the RCT studies were 0.73 for the "Quality index," 0.78 for Reporting, -0.07 for Confounding, and 0.78 for Bias (for the non-randomised studies 0.77, 0.51, 0.45, and 0.59).

Identification and correction for unknown confounders: A key responsibility of authors of non-randomised studies and their reviewers is to identify known confounders and correct for them. The authors of this review correctly considered previous vaccination status, health status and demographic status as some important confounders. They noted that 60% of the studies in their review reported and accounted for confounders by showing that randomisation achieved a balanced distribution of covariates, or by statistical adjustment. However, there is no comment on known confounders in the non-randomised studies and whether an attempt was made to correct for them, and unknown confounders remain unknown in such studies.

Unit of analysis errors: The authors appropriately corrected for unit of analysis errors, which they did in 51% of studies but did not state the intra-class correlations they used, and derived them from general publications dating from 1994 and 2001 and not from vaccination studies.

Meta-analysis: heterogeneity as measured by I2 was 50% or higher for 113/144 (78%) of comparisons for influenza vaccination (Figure 2), 64/87 (74%) for pneumococcal vaccination (Figure 3) and 14/26 (54%) for substrata of patient outreach and team change (Table 2). The Cochrane Handbook [1] assesses 50-90% heterogeneity as "substantial." The authors undertook meta-analyses to understand this heterogeneity, but the analyses and reporting do not elucidate clearly for the reader the sources of heterogeneity and how these modify any conclusions.

Conclusions: The authors do not reference the Cochrane review on pneumococcal vaccine [2] which found positive outcomes for vaccination and is more recent than the 2002 review they cite. The ACIP publications about influenza referenced by the authors (their references 4-7) as a motivator for conducting this review differ markedly from the conclusions of the Cochrane reviews on the effectiveness of influenza. [3-5] There is extensive ongoing controversy. The authors do not address these differences. Also, because the interventions advocated by their review sometimes depend on few studies, or on groups of studies with marked heterogeneity, funders remain unclear about the strength of the recommendation to conduct the expensive and time-consuming tasks required to implement these interventions in thousands of localities and medical offices.

References
1. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
2. Moberley S, Holden J, Tatham DP, Andrews RM. Vaccines for preventing pneumococcal infection in adults. Cochrane Database of Systematic Reviews 2008, Issue 1. Art. No.: CD000422. DOI: 10.1002/14651858.CD000422.pub2
3. Jefferson T, Di Pietrantonj C, Rivetti A, Bawazeer GA, Al-Ansary LA, Ferroni E. Vaccines for preventing influenza in healthy adults. Cochrane Database of Systematic Reviews 2010, Issue 7. Art. No.: CD001269. DOI: 10.1002/14651858.CD001269.pub4
4. Jefferson T, Di Pietrantonj C, Al-Ansary LA, Ferroni E, Thorning S, Thomas RE. Vaccines for preventing influenza in the elderly. Cochrane Database of Systematic Reviews 2010, Issue 2. Art. No.: CD004876. DOI: 10.1002/14651858.CD004876.pub3
5. Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database of Systematic Reviews 2010, Issue 2. Art. No.: CD005187. DOI: 10.1002/14651858.CD005187.pub3

Roger Thomas, MD, Ph.D, CCFP, MRCGP, Professor of Family Medicine
Department of Family Medicine, University of Calgary G012, Health Sciences Centre, 3330 Hospital Drive NW, Calgary, Alberta, T2N 4N1.
E-mail: rthomas@ucalgary.ca

Competing interests:   None declared