Research ArticleOriginal Research

Electronic Health Records for Intervention Research: A Cluster Randomized Trial to Reduce Antibiotic Prescribing in Primary Care (eCRT Study)

Martin C. Gulliford, Tjeerd van Staa, Alex Dregan, Lisa McDermott, Gerard McCann, Mark Ashworth, Judith Charlton, Paul Little, Michael V. Moore and Lucy Yardley
The Annals of Family Medicine July 2014, 12 (4) 344-351; DOI: https://doi.org/10.1370/afm.1659

Abstract

PURPOSE This study aimed to implement a point-of-care cluster randomized trial using electronic health records. We evaluated the effectiveness of electronically delivered decision support tools at reducing antibiotic prescribing for respiratory tract infections in primary care.

METHODS Family practices from England and Scotland participating in the Clinical Practice Research Datalink (CPRD) were included in the trial; 53 family practices were allocated to intervention and 51 practices were allocated to usual care. Patients aged 18 to 59 years consulting for respiratory tract infections were eligible. The intervention was through remotely installed, computer-delivered decision support tools accessed during the consultations. Control practices provided usual care. The primary outcome was the proportion of consultations for respiratory tract infections with an antibiotic prescribed based on electronic health records. Family practice-specific proportions were included in a cluster-level analysis.

RESULTS Data were analyzed for 603,409 patients: 317,717 at intervention practices and 285,692 at control practices. Use of the intervention was less than anticipated, varying among practices. There was a reduction in proportion of consultations with antibiotics prescribed of 1.85% (95% CI, 0.10%–3.59%, P = .038) and in the rate of antibiotic prescribing for respiratory tract infections (9.69%; 95% CI, 0.75%–18.63%, fewer prescriptions per 1,000 patient-years, P = .034). There were no adverse events.

CONCLUSIONS Cluster randomized trials may be implemented efficiently in large samples from routine care settings by using primary care electronic health records. Future studies should develop and test multicomponent methods for remotely delivered intervention.

INTRODUCTION

The randomized controlled trial design offers potentially unbiased estimates of health intervention effects, but implementing a randomized trial may be logistically challenging, costly, and time-consuming.1 Recruiting participants into a trial may be difficult, sample sizes can be too small, and the characteristics of participants included in a trial might differ from those encountered in the wider population. Interventions delivered in randomized trials may sometimes differ substantially from those that can be delivered into routine practice.2 Considerable attrition of the randomized participant sample may occur during the period of follow-up. Collection of data with which to measure trial outcomes may be costly. In cluster randomized designs, the degree of loss of efficiency resulting from correlation of outcomes within clusters may be difficult to anticipate, and the number of clusters available for allocation may often be limited.3

During the last 2 decades, the development and aggregation of well-coded electronic health records has provided large data sets for observational research. It is now being recognized that electronic health records also offer opportunities for intervention research.4 Allocation and intervention may be implemented at the point of care, with information routinely recorded into electronic health records used to follow up on participants and evaluate trial outcomes.5,6 Data collected into electronic health records may also be used to inform trial designs and to gauge the representativeness of participants recruited to a study. Using electronic health records in intervention research has the potential to allow large studies to be conducted at low cost in settings where care is routinely delivered.

The primary purpose of this research was to develop and evaluate methods for conducting cluster randomized trials in a primary care database that contains electronic patient records for large numbers of family practices.7 The substantive objective of the proof-of-concept trial was to evaluate the effectiveness of a computer-delivered intervention designed to reduce antibiotic prescribing at consultations for respiratory tract infections in primary care, with the primary outcome the proportion of consultations for respiratory tract infections with antibiotics prescribed.

METHODS

Design Overview

The study was a cluster randomized controlled trial with family practice as the unit of allocation.7 Family practices were sampled from the Clinical Practice Research Datalink (CPRD). The CPRD is a large database containing the electronic health records of about 650 family practices and more than 5 million currently active patients in the United Kingdom. Data available for each patient comprise the entire anonymized electronic medical record, including medical codes associated with consultations and referrals; details of all drugs prescribed; and records of weight, height, smoking, and alcohol use; and laboratory tests.8 CPRD clinical records have been shown to have a high predictive value for a range of specific medical diagnoses.9 We have reported on the epidemiology of respiratory tract infections and antibiotic prescribing in CPRD previously,10 with results that are consistent with other epidemiological data. In this study, CPRD family practices were allocated either to an active intervention trial arm, which received the computer-delivered prescribing support tools, or to a control trial arm that continued with usual care. The intervention was continued for 12 months at intervention trial arm practices. Outcomes were evaluated from patients’ electronic health records routinely collected into CPRD.

Setting and Participants

Between October 2010 and April 2011, 445 CPRD family practices in England and Scotland were invited to participate. Allocation of participating practices was performed at King’s College London using anonymized identifiers to ensure that allocation was separated from recruitment. Family practices were allocated to the intervention or control trial arms by minimization using the MINIM program,11 stratifying by region and practice list size. Individual patients included all those aged 18 to 59 years who were registered with the trial practices. Children and older adults were excluded from the study to provide a low-risk population for this large study, which was conducted remotely from the investigators. There were no other exclusion criteria.

Randomization and Interventions

Intervention was at the family practice level. Implementing the intervention required the development and deployment of computer-delivered electronic support tools that encouraged prescribers to adopt either a no-prescribing or a delayed-prescribing approach during consultations with adults with acute respiratory tract infections.12 Intervention development was informed by social cognitive theory,13 built on previous research that has identified barriers to reducing antibiotic prescribing14–16 and refined through a qualitative interview study with family practitioners at non-study practices.13

The decision support tools were installed remotely at the intervention arm practices and delivered during consultations through a system known as DXS Point-of Care, which is embedded in the family practice information system (VISION) used by CPRD practices. The decision support tools, which were activated when the family physician entered a medical code for the respiratory tract infection, provided information for education and decision support, including a summary of antibiotic prescribing recommendations, a single-sided patient information sheet, a summary of research evidence concerning no-antibiotic– or delayed-antibiotic–prescribing strategies, information on the definite indications for antibiotic prescription, and information and evidence on the risks from nonprescribing.13 Links to these tools appeared on an initial menu screen, allowing the physician to then select and view the screen of choice. The support tools included separate modules for sore throat, cough and bronchitis, otitis media, rhinosinusitis, and common colds. Intervention trial arm practices were sent a letter of information and a training video that provided an introduction to the prompts. Data on the use of the decision support tools were collected electronically.

Outcomes and Follow-up

The primary outcome measure was the proportion of consultations for respiratory tract infections with antibiotics prescribed during the 12-month intervention period. Secondary outcome measures included the proportion of consultations with antibiotics prescribed for each of cough and bronchitis, colds, otitis media, rhinosinusitis, and sore throat; the consultation rate for respiratory tract infection per 1,000 patient-years; and rate of antibiotic prescribing for respiratory tract infections per 1,000 patient-years.10 We also evaluated the total number of times the decision support tools were accessed divided by the total number of relevant consultations for 12 months and analyzed each outcome according to quartile of intervention utilization.

Sample Size

The study aimed to detect a difference smaller than the 7% reported by Ranji et al.17 We assumed that the coefficient of variation between practices, for the proportion of consultations with antibiotics prescribed, was 0.23 from Ashworth et al,18 with an α of .05 and a power of 80%. To detect a 5% difference in the proportion of consultations at which antibiotics are prescribed, 47 practices per trial arm were required.19 Equal cluster sizes were assumed.7

Statistical Analysis

We analyzed data from 12 months before to 12 months after the intervention began. Analyses were implemented according to the intention-to-treat principle, and we included in the analysis all eligible person-time for all allocated practices, including data for any practices that later withdrew from CPRD or patients who subsequently ended their registration during the study period. Analyses for primary and secondary outcomes estimated the difference (95% confidence interval) in the outcome between the intervention and control trial arms. The analyses were performed using the family practice–specific rates or proportions as observations. Analyses were adjusted for the preintervention value of the outcome, in an analysis of covariance framework, as well as the mean age of eligible patients at each practice and proportion of women at the practice. Minimum variance weights were used to allow for varying practice sizes.20 Intervention utilization was divided into quartiles, and a trend test was implemented. Analyses were implemented using Stata 12.0 (StataCorp LP).

The study was approved by the London Surry Borders Research Ethics Committee (09/H0806/81), and written informed consent was obtained for the participation of each family practice.7

RESULTS

Of the 445 family practices invited to participate in the trial, 104 practices consented, with 53 allocated to the intervention trial arm and 51 to the control trial arm. Analysis of preintervention data showed that participating family practices were similar to nonparticipating CPRD practices with respect to respiratory consultation rate, antibiotic prescribing rate, and proportion of consultations with antibiotics prescribed. There were 4 practices, 3 in the intervention trial arm and 1 in the control trial arm, that were excluded from analysis: 3 because the practice started contributing up-to-standard electronic health record data after the intervention start and 1 because it finished contributing data before the intervention start. Table 1 displays the characteristics of family practices and patients included in the analysis by trial arm.

View this table:
Table 1

Practice- and Patient-Level Characteristics

Figure 1 shows the selection of patients from participating practices for analysis. There were 317,717 patients for whom person-time data were contributed to analysis in the intervention trial arm either before or after the intervention start date, and 285,692 patients with person-time data in the control trial arm. There were 292,398 patients in the intervention trial arm, and 264,137 patients in the control trial arm contributing person-time data to analysis in the 12 months before intervention (Table 1). Fifty percent were women, and approximately 35% of patients were aged 45 to 59 years in both trial arms.

Figure 1
Figure 1

Flow diagram charting progress through the trial.

CPRD = Clinical Practice Research Datalink; HB =Health Board; PCT = Primary Care Trust; RTI = respiratory tract infection.

a Figure includes participants contributing to analysis either in 12-month preintervention or 12-month intervention periods.

Table 2 provides data for rates of consultations and antibiotic prescribing for respiratory tract infection per 1,000 patient-years, as well as the proportion of consultations for respiratory tract infection with antibiotics prescribed, from 12 months before and 12 months after the intervention start date. The rate of antibiotic prescriptions for respiratory tract infection per 1,000 patient-years declined from 116 to 108 per 1,000 in the intervention trial arm, but increased marginally in the control trial arm. The adjusted mean difference was −9.69 (95% CI, −18.63 to −0.75, P=.034) prescriptions per 1,000 patient-years. At practices in the intervention trial arm, the mean of the practice-specific proportion of consultations for respiratory tract infection with antibiotic prescribed declined marginally from 53% to 52%, while at control trial arm practices the proportion remained constant at 52%. The adjusted difference in proportion of consultations with antibiotic prescribed was −1.85% (95% CI, −3.59% to −0.10%, P= .038). This finding is consistent with about 1 or 2 fewer antibiotic prescriptions per 100 consultations for respiratory illness.

View this table:
Table 2

Consultation and Antibiotic Prescribing for Respiratory Tract Infection per 1,000 Registered Patients

Table 3 displays the proportion of consultations with antibiotics prescribed divided by the 5 groups of respiratory tract conditions identified in the decision support tools. There were only small differences in mean rates of antibiotic prescribing either between intervention and control trial arms or before and after the intervention. There was evidence that antibiotic prescribing was lower after intervention for cough and bronchitis, with a reduction of 2.49% (0.15% to 4.83%, P = .030). There was no clear evidence of a change in prescribing for the other 4 condition subgroups.

View this table:
Table 3

Proportion of Consultations for Different Types of Respiratory Tract Infection With Antibiotic Prescribed

Table 4 displays the level of intervention utilization, with intervention practices divided into quartiles of intervention. Overall utilization of the intervention was low, with one fourth of intervention family practices making little or no use of the intervention. Table 4 also shows changes in antibiotic prescribing divided by quartile of utilization of the decision support tools. Antibiotic prescribing was generally slightly lower at practices that made greater use of the intervention. The decrement in antibiotic prescribing associated with the intervention appeared to increase slightly with increasing intervention utilization. There was evidence of a linear trend between intervention utilization and change in antibiotic prescribing (adjusted reduction per quartile increase in utilization −0.64, 95% CI, −0.05 to −1.23, P = .034). The highest quartile of utilization, however, showed the lowest antibiotic prescribing even before intervention.

View this table:
Table 4

Intervention Utilization and Antibiotic Prescribing by Quartile of Intervention Utilization

DISCUSSION

The study showed that it is both feasible and extremely efficient to implement a cluster randomized trial within a primary care database such as CPRD. Using the CPRD as a sampling frame allowed us to recruit a large number of practices over a short period of time at low financial cost. More than one-half million individual patients were included in the trial, making this approach suitable for the evaluation of public health interventions. The trial was 1 of 2 studies funded through a research grant of £338,000 (approximately US $507,000), with a cost of recruitment to this trial of about 27 pence (£0.27, US $0.41) per patient. Outcomes may be evaluated through information recorded into electronic health records that are collected automatically into the primary care database, providing a precise estimate of an effect of small magnitude. Implementation of the study has shown that interventions may be delivered to family practices remotely at low cost and then used in consultations by family physicians. For this proof-of-concept study, we used a very simple form of electronic intervention that required physicians to click on a banner; we avoided using active alerts or pop-ups that might have led to difficulties of implementation. It appeared that the intervention was underutilized by some practices despite initial and follow-up advice offered. It was not necessary for physicians to access the support tools at every consultation, however, as a practitioner might only need to access the educational materials once or a few times to obtain the achievable benefit. We plan to evaluate more complex forms of intervention in future studies. The study provided evidence of a small reduction in antibiotic utilization, with the proportion of consultations for respiratory illness and antibiotic prescribed being approximately 2% lower in the intervention trial arm. This small change could be quantitatively important if it could be achieved over a wider population of practices or if a downward annual trend were to be established. There was a general trend toward reduced antibiotic prescribing with increased intervention utilization, but practices that used the intervention most tended to prescribe fewer antibiotics before the intervention.

Strengths and Limitations

The implementation of the intervention provided an assessment of intervention effectiveness in usual health care settings. The use of random allocation, an intention-to-treat analysis, and automated collection of data for all eligible patients should have minimized the potential for bias. The main limitation of this study was the low utilization of the intervention by some trial practices. There may be several explanations. The intervention was activated when a medical code was entered by the physician, but some family physicians enter data only after the consultation has ended and the patient has left the consultation room. We provided practices with an initial letter introducing them to the trial principles and followed up with a training video; however, we were not able to ensure that the letter and video were seen by all prescribers in a practice, and some physicians may be less receptive to messages in the electronic health record. In this respect, the circumstances of the trial closely resembled how a similar intervention might be rolled out in routine practice. It is unlikely, however, that clinicians need to view the prompts every time they consult with an eligible patient. All of these considerations identify challenges in the use of computer-delivered information to influence practitioner behavior.21 Mair et al22 suggested that e-health research may focus on organizational solutions at the expense of social and behavioral considerations. Our intervention was grounded in social cognitive theory and aimed to create a controllable and supportive environment, increasing self-efficacy and promoting expectations of positive outcomes, while reducing perceived negative risks, to support better adherence to prescribing recommendations.13 Even so, the low utilization of our intervention by some practices indicates the need to actively promote engagement with e-health interventions.

Comparison With Other Studies

Roshanov et al21 associated the use of computerized clinical decision support tools with only small effects on practitioner’s behavior. In the context of antibiotic prescribing, however, a number of recent trials have shown substantial effects using strategies that combined education and decision support with feedback of prescribing data.23,24 Multifaceted interventions are increasingly developed and tested in primary care, with evidence of effectiveness at reducing antibiotic prescribing.25–27 These more intensive and costly interventions may be impractical for widespread application and their long-term effectiveness uncertain. Our study shows that a system for computer-delivered decision support delivered remotely to many practices, so that prescribers would have links to evidence-based information, may be expected to have a smaller, but significant, effect in reducing antibiotic prescribing. Such an outcome could be important in a wide population or if the effect is sustained over time.

Implications for Policy and Practice

Implementing randomized intervention studies by utilizing the electronic health records of a primary care database offers a promising approach to the evaluation of clinical and public health interventions delivered through primary care and public health services. Our study shows that this approach may be used in the evaluation of interventions to reduce antibiotic prescribing in settings where care is routinely delivered. With this approach now proven to be feasible, it will offer a cost-effective and sustainable method of implementing cluster randomized trials across a wide range of subjects of public health importance.

Acknowledgments

The authors thank Tim Foster and colleagues at DXS (UK) Ltd for facilitating the implementation of the intervention through DXS Point-of-Care.

Footnotes

  • Conflicts of interest: authors report none.

  • Authors’ contributions: M.C.G., T.vS., P.L., M.A. and L.Y. designed the study; A.D. and G.M. contributed to the implementation of the study; L.M. and L.Y., with P.L., M.V.M., M.A. and M.C.G., were responsible for developing the trial interventions; A.D. and M.C.G. designed the analysis and J.C. and A.D. analyzed the data; A.D. and M.C.G. drafted the paper. All authors contributed to and approved the final version of the manuscript.

  • Data Monitoring Committee independent members: Sarah Meredith (Chair), Sally Kerry, Elizabeth Murray.

  • Trial Steering Committee independent members: Jonathan Mant (Chair), John Robson, Andrew Haywood, and Nanik Pursani.

  • Funding support: The study was supported by the Joint Initiative in Electronic Patient Records and Databases in Research, a partnership between the Wellcome Trust, Medical Research Council, Economics & Social Research Council, and Engineering & Physical Sciences Research Council. Drs Gulliford and Dregan were supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at Guy’s and St Thomas’ NHS Foundation Trust, and King’s College London. Clinical Practice Research Datalink (CPRD) has received funding from the Medicines and Healthcare Products Regulatory Agency (MHRA), Wellcome Trust, Medical Research Council, NIHR Health Technology Assessment program, Innovative Medicine Initiative, UK Department of Health, Technology Strategy Board, Seventh Framework Programme EU, various universities, contract research organizations, and pharmaceutical companies. The Department of Pharmacoepidemiology & Pharmacotherapy, Utrecht Institute for Pharmaceutical Sciences, has received unrestricted funding for pharmacoepidemiological research from GlaxoSmithKline, Novo Nordisk, the private-public funded Top Institute Pharma (www.tipharma.nl, includes co-funding from universities, government, and industry), the Dutch Medicines Evaluation Board, and the Dutch Ministry of Health.

  • Disclaimer: The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health, and they do not reflect the official policy or position of the MHRA.

  • Trial Registration: Current Controlled Trials ISRCTN 47558792.

  • Received for publication October 14, 2013.
  • Revision received March 4, 2014.
  • Accepted for publication March 29, 2014.

References

  1. ↵
    1. Collier R
    . Rapidly rising clinical trial costs worry researchers. CMAJ. 2009;180(3):277–278.
    OpenUrlFREE Full Text
  2. ↵
    1. Schwartz D,
    2. Lellouch J
    . Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol. 2009;62(5):499–505.
    OpenUrlCrossRefPubMed
  3. ↵
    1. Donner A,
    2. Klar N
    . Pitfalls of and controversies in cluster randomization trials. Am J Public Health. 2004;94(3):416–422.
    OpenUrlCrossRefPubMed
  4. ↵
    1. van Staa TP,
    2. Goldacre B,
    3. Gulliford M,
    4. et al
    . Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ. 2012;344:e55.
    OpenUrlFREE Full Text
  5. ↵
    1. Vickers AJ,
    2. Scardino PT
    . The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials. 2009;10:14.
    OpenUrlCrossRefPubMed
  6. ↵
    1. D’Avolio L,
    2. Ferguson R,
    3. Goryachev S,
    4. et al
    . Implementation of the Department of Veterans Affairs’ first point-of-care clinical trial. J Am Med Inform Assoc. 2012;19(e1):e170–e176.
    OpenUrlCrossRefPubMed
  7. ↵
    1. Gulliford MC,
    2. van Staa T,
    3. McDermott L,
    4. et al
    ; electronic Cluster Randomised Trial Research Team. Cluster randomised trial in the General Practice Research Database: 1. Electronic decision support to reduce antibiotic prescribing in primary care (eCRT study). Trials. 2011;12:115.
    OpenUrlCrossRefPubMed
  8. ↵
    1. Williams T,
    2. van Staa T,
    3. Puri S,
    4. Eaton S
    . Recent advances in the utility and use of the General Practice Research Database as an example of a UK Primary Care Data resource. Ther Adv Drug Saf. 2012;3(2):89–99.
    OpenUrlAbstract/FREE Full Text
  9. ↵
    1. Herrett E,
    2. Thomas SL,
    3. Schoonen WM,
    4. Smeeth L,
    5. Hall AJ
    . Validation and validity of diagnoses in the General Practice Research Database: a systematic review. Br J Clin Pharmacol. 2010;69(1):4–14.
    OpenUrlCrossRefPubMed
  10. ↵
    1. Gulliford M,
    2. Latinovic R,
    3. Charlton J,
    4. Little P,
    5. van Staa T,
    6. Ashworth M
    . Selective decrease in consultations and antibiotic prescribing for acute respiratory tract infections in UK primary care up to 2006. J Public Health (Oxf). 2009;31(4):512–520.
    OpenUrlAbstract/FREE Full Text
  11. ↵
    1. Altman DG,
    2. Bland JM
    . Treatment allocation by minimisation. BMJ. 2005;330(7495):843.
    OpenUrlFREE Full Text
  12. ↵
    National Institute for Health and Clinical Excellence. Prescribing of Antibiotics for Self-Limiting Respiratory Tract Infections in Adults and Children in Primary Care. London, England: National Institute for Health and Clinical Excellence; 2008.
  13. ↵
    1. McDermott L,
    2. Yardley L,
    3. Little P,
    4. Ashworth M,
    5. Gulliford M
    ; eCRT Research Team. Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010;11:90.
    OpenUrlCrossRefPubMed
  14. ↵
    1. Kumar S,
    2. Little P,
    3. Britten N
    . Why do general practitioners prescribe antibiotics for sore throat? Grounded theory interview study. BMJ. 2003;326(7381):138.
    OpenUrlAbstract/FREE Full Text
    1. Hrisos S,
    2. Eccles M,
    3. Johnston M,
    4. et al
    . An intervention modelling experiment to change GPs’ intentions to implement evidence-based practice: using theory-based interventions to promote GP management of upper respiratory tract infection without prescribing antibiotics #2. BMC Health Serv Res. 2008;8:10.
    OpenUrlCrossRefPubMed
  16. ↵
    1. Simpson SA,
    2. Wood F,
    3. Butler CC
    . General practitioners’ perceptions of antimicrobial resistance: a qualitative study. J Antimicrob Chemother. 2007;59(2):292–296.
    OpenUrlAbstract/FREE Full Text
  17. ↵
    1. Ranji SR,
    2. Steinman MA,
    3. Shojania KG,
    4. et al
    . Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies. Volume 4: Antibiotic Prescribing Behaviour. AHRQ Publication number 04(06)-0051-4. Rockville, MD: Agency for Healthcare Research and Quality; 2006.
  18. ↵
    1. Ashworth M,
    2. Charlton J,
    3. Ballard K,
    4. Latinovic R,
    5. Gulliford M
    . Variations in antibiotic prescribing and consultation rates for acute respiratory infection in UK general practices 1995–2000. Br J Gen Pract. 2005;55(517):603–608.
    OpenUrlAbstract/FREE Full Text
  19. ↵
    1. Hayes RJ,
    2. Bennett S
    . Simple sample size calculation for cluster-randomized trials. Int J Epidemiol. 1999;28(2):319–326.
    OpenUrlAbstract/FREE Full Text
  20. ↵
    1. Kerry SM,
    2. Bland JM
    . Unequal cluster sizes for trials in English and Welsh general practice: implications for sample size calculations. Stat Med. 2001;20(3):377–390.
    OpenUrlCrossRefPubMed
  21. ↵
    1. Roshanov PS,
    2. Fernandes N,
    3. Wilczynski JM,
    4. et al
    . Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    1. Mair FS,
    2. May C,
    3. O’Donnell C,
    4. Finch T,
    5. Sullivan F,
    6. Murray E
    . Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012; 90(5):357–364.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Little P,
    2. Stuart B,
    3. Francis N,
    4. et al
    ; GRACE consortium. Effects of internet-based training on antibiotic prescribing rates for acute respiratory-tract infections: a multinational, cluster, randomised, factorial, controlled trial. Lancet. 2013;382(9899):1175–1182.
    OpenUrlCrossRefPubMed
  24. ↵
    1. Gonzales R,
    2. Anderer T,
    3. McCulloch CE,
    4. et al
    . A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis. JAMA Intern Med. 2013;173(4):267–273.
    OpenUrlCrossRefPubMed
  25. ↵
    1. Ranji SR,
    2. Steinman MA,
    3. Shojania KG,
    4. Gonzales R
    . Interventions to reduce unnecessary antibiotic prescribing: a systematic review and quantitative analysis. Med Care. 2008;46(8):847–862.
    OpenUrlCrossRefPubMed
    1. Butler CC,
    2. Simpson SA,
    3. Dunstan F,
    4. et al
    . Effectiveness of multifaceted educational programme to reduce antibiotic dispensing in primary care: practice based randomised controlled trial. BMJ. 2012; 344:d8173.
    OpenUrlAbstract/FREE Full Text
  27. ↵
    1. Gjelstad S,
    2. Høye S,
    3. Straand J,
    4. Brekke M,
    5. Dalen I,
    6. Lindbæk M
    . Improving antibiotic prescribing in acute respiratory tract infections: cluster randomised trial from Norwegian general practice (prescription peer academic detailing (Rx-PAD) study). BMJ. 2013; 347:f4403.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Related Articles

Cited By...

More in this TOC Section

Similar Articles

Subjects

Keywords