Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleOriginal Research

Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care

Brian S. Alper, David S. White and Bin Ge
The Annals of Family Medicine November 2005, 3 (6) 507-513; DOI: https://doi.org/10.1370/afm.370
Brian S. Alper
MD, MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David S. White
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bin Ge
MA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article
Compose eLetter

More information about text formats

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Statement of Competing Interests
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Vertical Tabs

Jump to comment:

  • Are Hunting Tools Enough for Point-of-Care Learning?
    Scott Strayer
    Published on: 03 January 2006
  • Take two and e-mail me in the morning
    Brian S. Alper
    Published on: 20 December 2005
  • Take Five
    Richard O. Schamp
    Published on: 17 December 2005
  • The outcomes when clinicians use online evidence systems
    Johanna I Westbrook
    Published on: 13 December 2005
  • A landmark trial despite the lack of objective outcomes
    Pierre Pluye
    Published on: 07 December 2005
  • Published on: (3 January 2006)
    Page navigation anchor for Are Hunting Tools Enough for Point-of-Care Learning?
    Are Hunting Tools Enough for Point-of-Care Learning?
    • Scott Strayer, Charlottesville, VA, USA
    • Other Contributors:

    Dr. Alper and colleagues have made an important contribution to the literature on the use of point-of-care clinical evidence seeking using currently available databases and technologies with this manuscript. This study adds to other point-of-care literature that has shown improved patient outcomes using computer decision support in different settings. Several of these studies have involved handheld computers as an idea...

    Show More

    Dr. Alper and colleagues have made an important contribution to the literature on the use of point-of-care clinical evidence seeking using currently available databases and technologies with this manuscript. This study adds to other point-of-care literature that has shown improved patient outcomes using computer decision support in different settings. Several of these studies have involved handheld computers as an ideal hardware platform for clinical settings.

    A recent study completed in a critical care unit demonstrated that handheld computer-based decision support reduced lengths of stay for patients and also reduced antibiotic prescribing (1). Office-based studies with handheld computers have shown increased adherence to asthma guidelines (2), potential reduction in adverse drug effects (3), improved antibiotic prescribing for otitis media in children (4), and reduced inappropriate use of antibiotics for upper respiratory infections (5). A recent Robert Wood Johnson Foundation initiative, the Prescription for Health Project, was undertaken to improve patient care through innovative approaches and technologies. Several of the projects reported promising pilot results in addressing tobacco use, unhealthy diets, sedentary lifestyles and alcohol misuse (6).

    In addition to having a high quality “hunting” tool for finding clinical evidence efficiently when questions arise, we have proposed that there should also be a high quality “foraging” tool that updates physicians on the latest medical advances (and setbacks) so that they are aware of important information when it is needed. These two types of information tools are needed to practice medicine in the Information Age: 1) a first-alert tool that forages through vast amounts of new information and pulls out only the relevant and valid information; and 2) a source that will quickly hunt through a database of this same type of information at the time when you need to use the information. Both tools are needed; without both, you don’t know when new information is available and won’t be able to find it when you need it (7).

    The ideal next step for synthesized evidence studies using databases such as Dynamed, InfoRetriever, UpToDate, and others, will be to study patient outcomes in settings where these types of databases are in use. Additionally, these databases should be linked to electronic medical records and methods of displaying the pertinent evidence should be explored fully. The content of these databases should be constantly scrutinized to ensure validity and relevance of included information, inclusion of only patient-oriented evidence, assignment of levels of evidence, lack of bias, and comprehensiveness of the information.

    Scott M. Strayer, MD, MPH Assistant Professor of Family Medicine University of Virginia Department of Family Medicine Charlottesville, Virginia

    David C. Slawson, MD B. Lewis Barnett, Jr. Professor of Family Medicine University of Virginia Department of Family Medicine Charlottesville, Virginia

    Allen F. Shaughnessy, PharmD Adjunct Professor of Public Health and Family Medicine Tufts University Family Medicine Residency Tufts University School of Medicine Boston, Massachusetts

    1. Sintchenko V, Iredell JR, Gilbert GL, Coiera E. Handheld computer-based decision support reduces patient length of stay and antibiotic prescribing in critical care. J Am Med Inform Assoc. 2005 Jul- Aug;12(4):398-402. Epub 2005 Mar 31.

    2. Shiffman RN, Freudigman M, Brandt CA, Liaw Y, Navedo DD. A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics. 2000 Apr;105(4 Pt 1):767-73.

    3. Rothschild JM, Lee TH, Bae T, Bates DW. Clinician use of a palmtop drug reference guide. J Am Med Inform Assoc. 2002 May- Jun;9(3):223-9.

    4. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics. 2001 Feb;107(2):E15.

    5. Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, Haddadin B, Rubin MA, Williamson J, Stults B, Rupper R, Stevenson K. Clinical decision support and appropriateness of antimicrobial prescribing: a randomized trial. JAMA. 2005 Nov 9;294(18):2305-14.

    6. Green LA, et. al. Prescription for Health: Changing Primary Care Practice to Foster Healthy Behaviors. Ann Fam Med. 2005 Jul/Aug 3;supp:S1-S68.

    7. Shaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for lifelong learning? BMJ 1999; 319:1280.

    Competing interests:   Drs. Shaughnessy and Slawson are consultants for InfoPOEM, Inc., publisher of the InfoPOEMs, The Clinical Awareness System(tm) and originators of the Information Mastery(tm) process.

    Show Less
    Competing Interests: None declared.
  • Published on: (20 December 2005)
    Page navigation anchor for Take two and e-mail me in the morning
    Take two and e-mail me in the morning
    • Brian S. Alper, Ipswich, MA, USA

    Five minutes for finding an answer is clearly much better than the thirty minutes previously published using MEDLINE, and this justifies searching for the most important questions.

    Yet five minutes is still too long for many settings where patient visits can last less than fifteen minutes. The good news is that we can do better than five minutes.

    The research grant supporting this study had three object...

    Show More

    Five minutes for finding an answer is clearly much better than the thirty minutes previously published using MEDLINE, and this justifies searching for the most important questions.

    Yet five minutes is still too long for many settings where patient visits can last less than fifteen minutes. The good news is that we can do better than five minutes.

    The research grant supporting this study had three objectives, two of which were published in the original article and the supplementary appendix. The third objective was to determine the facilitators and barriers to finding information in DynaMed and derive plans for improving the content and user interface. We conducted numerous interviews and direct observations of regular DynaMed users and clinicians who had not previously used DynaMed.

    We learned that the alphabetical listing with "Jump To" feature and the standardized templates were among the key features facilitating rapid, easy navigation.

    We also learned many features we could add or improve upon. We derived two prioritized lists for changes from this qualitative research, a list for the user interface and a list for the content.

    The most important user interface changes included features like placing all navigation functions in one section of the navigation frame and providing a navigation frame that supports navigation between sections of our standardized template. The user interface was revised in July 2005 to reflect these and other changes.

    Content changes which will facilitate more rapid searching include making conclusions bold, adding level of evidence labels to key evidence, making internal links from Treatment Overviews to the detailed supporting evidence, and separating large summaries into small sub-summaries. These content changes are being applied on an ongoing basis.

    We believe we can currently fulfill most information needs within two to three minutes and often within one minute, and expect this to continue to improve as we apply changes to our content and user interface. We would welcome formal testing unencumbered by competing interests.

    And if you have trouble finding an answer or finding an answer quickly, please e-mail us at editor@dynamicmedical.com -- we learn from these situations and adapt.

    Competing interests:   Dr. Alper is the editor-in-chief of DynaMed and works full-time as medical director of EBSCO Publishing Inc. which publishes DynaMed.

    Show Less
    Competing Interests: None declared.
  • Published on: (17 December 2005)
    Page navigation anchor for Take Five
    Take Five
    • Richard O. Schamp, St. Louis, MO, USA

    As Dr. Alper and his colleagues provide an important contribution(1)to primary care by demonstrating the utility of an online database of synthesized evidence in answering real-time questions for clinicians. Most of us in primary care identify our own knowledge deficits in routine work, and we wish, for our patients' benefit, these deficits could be resolved quickly and accurately within a workplace learning environmen...

    Show More

    As Dr. Alper and his colleagues provide an important contribution(1)to primary care by demonstrating the utility of an online database of synthesized evidence in answering real-time questions for clinicians. Most of us in primary care identify our own knowledge deficits in routine work, and we wish, for our patients' benefit, these deficits could be resolved quickly and accurately within a workplace learning environment.

    Many obstacles(2) face doctors who seek evidence-based questions in practice. Several obstacles coalesce around the time required. What is remarkable in the present study is that the average time to clinically meaningful answers was only five minutes. Compared to Medline or similar searching options, this is a huge advantage and practically eliminates some of the barriers to practicing clinicians who ask important clinical questions.

    For over seven years, I've used DynaMed routinely, usually as my first search tool among several, witnessing its evolution into increasing utility for information management(3). Besides assisting with my own information searches (which usually take less than five minutes), it is valuable when I teach point-of-care information mastery to students and residents and other physicians. I use it to answer foreground (patient- specific) questions and to fill in background deficits. Though Alper, et al point out that their study is unable to determine which features or combinations of features are responsible for the advantageous outcome, the key features of DynaMed that facilitate the information management process include: --Surveillance methodology that provides continuous updates to topics --Robust search engine, with Boolean features --Hypertext links to PubMed or original articles

    Finally, time savings and ease of use are meaningless if the results are not valid. The question of whether answers found in DynaMed are valid is addressed in the supplemental appendix to the article(4). DynaMed was found to meet or exceed the four comparators for 87.3% of the questions.

    The article provides compelling support for practicing family physicians and educators to take the five minutes needed to answer the questions(5) that arise in our patients to improve our knowledge and ultimately its application toward excellent clinical care.

    Sincerely,

    Richard O. Schamp, MD

    References

    1. Alper B, White D, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synethsized evidence: A randomized trial in primary care. Annals of Family Medicine 2005, 3(6):507 -513.

    2. Ely JW, Osheroff JA, Ebell MH, Champliss ML et al. Obstacles to ansering doctors¡¦ questions about patient care with evidence: qualitative study. BMJ 2002 Mar 23;324(7339):710

    3. Slawson DC, Shaughness AF, Teaching Evidence-Based Medicine: Should We Be Teaching Information Management Instead? Acad. Med., July 1, 2005; 80(7): 685 - 689.

    4. Alper B, Schamp R, White D, Hoock J, Supplemental Appendix. Level of Evidence comparison, http://annalsfm.highwire.org/cgi/content/full/3/6/507/DC1, accessed 16 Dec 2005.

    5. Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319: 358-61.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (13 December 2005)
    Page navigation anchor for The outcomes when clinicians use online evidence systems
    The outcomes when clinicians use online evidence systems
    • Johanna I Westbrook, Sydney, Australia
    • Other Contributors:

    As the information demands on clinicians continue to escalate, research that investigates effective means of supporting their access to accurate and timely information is imperative. Alper et al.(1)have made an important contribution by examining the use of a specific system when used in everyday practice. There is now a body of evidence demonstrating that hospital-based health professionals and family physicians are abl...

    Show More

    As the information demands on clinicians continue to escalate, research that investigates effective means of supporting their access to accurate and timely information is imperative. Alper et al.(1)have made an important contribution by examining the use of a specific system when used in everyday practice. There is now a body of evidence demonstrating that hospital-based health professionals and family physicians are able to integrate the use of online evidence systems into everyday clinical practice and that users report benefits to patient management as a result(2-4).

    However measures of utilisation and self-reports of benefit only go so far. In acknowledging the limitations of their study, Alper et al demonstrate the inherent difficulties in studying this topic, highlighting the need to draw together research evidence from multiple trials of similar systems. A crucial question is the actual impact that these systems have on the decision-making process. We addressed this using an experimental study design(5, 6). Experienced hospital-based doctors, family physicians and specialist nurses were asked to answer a set of eight clinical scenario questions unaided. They were then randomized to an online evidence system allowing us to quantify the extent to which access to an online evidence system improved the performance of clinicians in correctly answering questions. We found that compared to answering questions unaided, online evidence retrieval systems improved correct answers to clinical scenarios by 21%.

    Our study also demonstrated the challenges clinicians can face in interpreting information retrieved. For example, in 7% of 557 scenarios answered we found that clinicians provided an initially correct answer which was then incorrectly changed after they had searched for evidence using the system (ie they went from a right answer to a wrong answer). Analysis of computer web logs revealed that several participants viewed the same evidence yet provided different answers. Further, 54% of initially wrong answers failed to be corrected, with clinicians in each case documenting the specific evidence which they had retrieved from the online evidence system to support these incorrect answers. Interestingly, experienced specialist nurses, who performed poorly unaided when compared with their medical colleagues, improved their performance to equal the doctors once they had access to an online evidence system.

    Thus the system design challenge is not just to provide clinicians with fast and accurate evidence, but to present it in such a way as to increase the chance that it will be accurately interpreted and have a positive impact upon decisions made.

    References

    1. Alper B, White D, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synethsized evidence: A randomized trial in primary care. Annals of Family Medicine 2005, 3(6):507 -513.

    2. Westbrook J, Gosling AS, Coiera E. Do clinicians use online evidence to support patient care? A study of 55,000 clinicians. Journal of American Medical Informatics Association 2004, 11(2):113-120.

    3. Pluye P, Grad R, Dunikowski L, Stephenson R. Impact of clinical information-retireval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies. International Journal of Medical Informatics 2005, 74(9):745-768.

    4. Magrabi F, Coiera E, Westbrook J, Gosling A. Clinician use of online evidence in primary care consultations. International Journal of Medical Informatics 2005, 74(1):1-12.

    5. Westbrook J, Coiera E, Gosling AS. Do online information retrieval systems help experienced clinicians answer clinical questions? Journal of the American Medical Informatics Association 2005, 12(3):315-321.

    6. Westbrook J, Gosling AS, Coiera E. The impact of an online evidence system on confidence in decision making in a controlled setting. Medical Decision Making 2005, 25(2):178-185.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (7 December 2005)
    Page navigation anchor for A landmark trial despite the lack of objective outcomes
    A landmark trial despite the lack of objective outcomes
    • Pierre Pluye, Montreal, Canada

    I would like to gratefully acknowledge the contribution of Alper and colleagues (1), and to comment this benchmarking study in line with our literature review on the impact assessment of databases (2). Alper and colleagues’ trial shows that information derived from a synopsis-based database improves clinical decision-making compared with information derived from other sources. However, outcomes were self-reported by par...

    Show More

    I would like to gratefully acknowledge the contribution of Alper and colleagues (1), and to comment this benchmarking study in line with our literature review on the impact assessment of databases (2). Alper and colleagues’ trial shows that information derived from a synopsis-based database improves clinical decision-making compared with information derived from other sources. However, outcomes were self-reported by participants, and these reports were apparently not controlled afterward using retrospective interviews or independant record assessment for example.

    Despite the lack of objective outcome, this randomized trial constitutes a landmark study in the field for three reasons. First, while this trial was conducted in everyday practice, experimental studies are more often conducted in laboratories using a pre-designed set of clinical questions. Second, while this trial used a per-question randomization, other trials conducted in every day practice has been based on per- participant or per-source randomization. Third, to our knowledge only one Finnish trial measured change in clinical decision-making as an outcome, and found no difference between the use of guidelines within a database and that of paper-based guidelines. This absence of difference was not surprising as the paper-based compilation of over 1,000 guidelines is a usual source of information for practitioners in Finland (3).

    Thus, Alper and colleagues are convincing in three ways: the generalizability of their results in Family Medicine, the use of searches for information as units of randomization, and the comparison between usual sources of information and a new database (new from the participants’ perspective).

    Sincerely yours,

    Pierre Pluye, MD, PhD

    References

    1. Alper BS, White DS, Ge B. Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care. Annals of Family Medicine. 2005;3(6):507-513.

    2.Pluye P, Grad R, Dunikowski L, Stephenson R. Impact of clinical information-retrieval technology on physicians: A literature review of quantitative, qualitative and mixed methods studies. International Journal of Medical Informatics. 2005;74(9):745-768.

    3. Jousimaa J, Makela M, Kunnamo I, MacLennan G, Grimshaw JM. Primary care guidelines on consultation practices: The effectiveness of computerized versus paper-based versions. A cluster randomized controlled trial among newly qualified primary care physicians. International Journal of Technology Assessment in Health Care. 2002;18(3):586-596.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 3 (6)
The Annals of Family Medicine: 3 (6)
Vol. 3, Issue 6
1 Nov 2005
  • Table of Contents
  • Index by author
  • Annual Indexes
  • In Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
4 + 14 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care
Brian S. Alper, David S. White, Bin Ge
The Annals of Family Medicine Nov 2005, 3 (6) 507-513; DOI: 10.1370/afm.370

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care
Brian S. Alper, David S. White, Bin Ge
The Annals of Family Medicine Nov 2005, 3 (6) 507-513; DOI: 10.1370/afm.370
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • Acknowledgments
    • Footnotes
    • REFERENCES
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • Successes, lessons and opportunities: 15-year follow-up of an integrated evidence-based medicine curriculum
  • Update of Strategies to Translate Evidence from Cochrane Musculoskeletal Group Systematic Reviews for Use by Various Audiences
  • Number Needed to Benefit From Information (NNBI): Proposal From a Mixed Methods Research Study With Practicing Family Physicians
  • Decision making in family medicine: Randomized trial of the effects of the InfoClinique and Trip database search engines
  • Evidence-based practice in action: how can your medical librarian help?
  • Information Needs and Information-Seeking Behavior of Primary Care Physicians
  • In This Issue: Trade-Offs, Time Use, Depression Care
  • Google Scholar

More in this TOC Section

  • Performance-Based Reimbursement, Illegitimate Tasks, Moral Distress, and Quality Care in Primary Care: A Mediation Model of Longitudinal Data
  • Adverse Outcomes Associated With Inhaled Corticosteroid Use in Individuals With Chronic Obstructive Pulmonary Disease
  • Family-Based Interventions to Promote Weight Management in Adults: Results From a Cluster Randomized Controlled Trial in India
Show more Original Research

Similar Articles

Subjects

  • Methods:
    • Quantitative methods
  • Other research types:
    • Professional practice
  • Other topics:
    • Health informatics
    • Communication / decision making

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine