Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care

Brian S. Alper; David S. White; Bin Ge

doi:10.1370/afm.370

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article

Compose eLetter

Title *

Author Information

Contributors
First Name and Middle Initial * First or given name, e.g. 'Peter'. Last Name * Your last, or family, name, e.g. 'MacMoody'. Email Address * Your email address, e.g. higgs-boson@gmail.com Occupation * Your role and/or occupation, e.g. 'Orthopedic Surgeon'. Affiliation * Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.

Statement of Competing Interests

Competing interests? *

Yes

Please describe the competing interests

Vertical Tabs

Jump to comment:

Are Hunting Tools Enough for Point-of-Care Learning?
Scott Strayer

Published on: 03 January 2006
Take two and e-mail me in the morning
Brian S. Alper

Published on: 20 December 2005
Take Five
Richard O. Schamp

Published on: 17 December 2005
The outcomes when clinicians use online evidence systems
Johanna I Westbrook

Published on: 13 December 2005
A landmark trial despite the lack of objective outcomes
Pierre Pluye

Published on: 07 December 2005

Published on: (3 January 2006)
Page navigation anchor for Are Hunting Tools Enough for Point-of-Care Learning?
Are Hunting Tools Enough for Point-of-Care Learning?
Scott Strayer, Charlottesville, VA, USA
Other Contributors:
Dr. Alper and colleagues have made an important contribution to the literature on the use of point-of-care clinical evidence seeking using currently available databases and technologies with this manuscript. This study adds to other point-of-care literature that has shown improved patient outcomes using computer decision support in different settings. Several of these studies have involved handheld computers as an idea...
Show More

Dr. Alper and colleagues have made an important contribution to the literature on the use of point-of-care clinical evidence seeking using currently available databases and technologies with this manuscript. This study adds to other point-of-care literature that has shown improved patient outcomes using computer decision support in different settings. Several of these studies have involved handheld computers as an ideal hardware platform for clinical settings.
A recent study completed in a critical care unit demonstrated that handheld computer-based decision support reduced lengths of stay for patients and also reduced antibiotic prescribing (1). Office-based studies with handheld computers have shown increased adherence to asthma guidelines (2), potential reduction in adverse drug effects (3), improved antibiotic prescribing for otitis media in children (4), and reduced inappropriate use of antibiotics for upper respiratory infections (5). A recent Robert Wood Johnson Foundation initiative, the Prescription for Health Project, was undertaken to improve patient care through innovative approaches and technologies. Several of the projects reported promising pilot results in addressing tobacco use, unhealthy diets, sedentary lifestyles and alcohol misuse (6).
In addition to having a high quality “hunting” tool for finding clinical evidence efficiently when questions arise, we have proposed that there should also be a high quality “foraging” tool that updates physicians on the latest medical advances (and setbacks) so that they are aware of important information when it is needed. These two types of information tools are needed to practice medicine in the Information Age: 1) a first-alert tool that forages through vast amounts of new information and pulls out only the relevant and valid information; and 2) a source that will quickly hunt through a database of this same type of information at the time when you need to use the information. Both tools are needed; without both, you don’t know when new information is available and won’t be able to find it when you need it (7).
The ideal next step for synthesized evidence studies using databases such as Dynamed, InfoRetriever, UpToDate, and others, will be to study patient outcomes in settings where these types of databases are in use. Additionally, these databases should be linked to electronic medical records and methods of displaying the pertinent evidence should be explored fully. The content of these databases should be constantly scrutinized to ensure validity and relevance of included information, inclusion of only patient-oriented evidence, assignment of levels of evidence, lack of bias, and comprehensiveness of the information.
Scott M. Strayer, MD, MPH Assistant Professor of Family Medicine University of Virginia Department of Family Medicine Charlottesville, Virginia
David C. Slawson, MD B. Lewis Barnett, Jr. Professor of Family Medicine University of Virginia Department of Family Medicine Charlottesville, Virginia
Allen F. Shaughnessy, PharmD Adjunct Professor of Public Health and Family Medicine Tufts University Family Medicine Residency Tufts University School of Medicine Boston, Massachusetts
1. Sintchenko V, Iredell JR, Gilbert GL, Coiera E. Handheld computer-based decision support reduces patient length of stay and antibiotic prescribing in critical care. J Am Med Inform Assoc. 2005 Jul- Aug;12(4):398-402. Epub 2005 Mar 31.
2. Shiffman RN, Freudigman M, Brandt CA, Liaw Y, Navedo DD. A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics. 2000 Apr;105(4 Pt 1):767-73.
3. Rothschild JM, Lee TH, Bae T, Bates DW. Clinician use of a palmtop drug reference guide. J Am Med Inform Assoc. 2002 May- Jun;9(3):223-9.
4. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. Pediatrics. 2001 Feb;107(2):E15.
5. Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, Haddadin B, Rubin MA, Williamson J, Stults B, Rupper R, Stevenson K. Clinical decision support and appropriateness of antimicrobial prescribing: a randomized trial. JAMA. 2005 Nov 9;294(18):2305-14.
6. Green LA, et. al. Prescription for Health: Changing Primary Care Practice to Foster Healthy Behaviors. Ann Fam Med. 2005 Jul/Aug 3;supp:S1-S68.
7. Shaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for lifelong learning? BMJ 1999; 319:1280.
Competing interests: Drs. Shaughnessy and Slawson are consultants for InfoPOEM, Inc., publisher of the InfoPOEMs, The Clinical Awareness System(tm) and originators of the Information Mastery(tm) process.
Show Less

Competing Interests: None declared.
Published on: (20 December 2005)
Page navigation anchor for Take two and e-mail me in the morning
Take two and e-mail me in the morning
Brian S. Alper, Ipswich, MA, USA
Five minutes for finding an answer is clearly much better than the thirty minutes previously published using MEDLINE, and this justifies searching for the most important questions.
Yet five minutes is still too long for many settings where patient visits can last less than fifteen minutes. The good news is that we can do better than five minutes.
The research grant supporting this study had three object...
Show More

Five minutes for finding an answer is clearly much better than the thirty minutes previously published using MEDLINE, and this justifies searching for the most important questions.
Yet five minutes is still too long for many settings where patient visits can last less than fifteen minutes. The good news is that we can do better than five minutes.
The research grant supporting this study had three objectives, two of which were published in the original article and the supplementary appendix. The third objective was to determine the facilitators and barriers to finding information in DynaMed and derive plans for improving the content and user interface. We conducted numerous interviews and direct observations of regular DynaMed users and clinicians who had not previously used DynaMed.
We learned that the alphabetical listing with "Jump To" feature and the standardized templates were among the key features facilitating rapid, easy navigation.
We also learned many features we could add or improve upon. We derived two prioritized lists for changes from this qualitative research, a list for the user interface and a list for the content.
The most important user interface changes included features like placing all navigation functions in one section of the navigation frame and providing a navigation frame that supports navigation between sections of our standardized template. The user interface was revised in July 2005 to reflect these and other changes.
Content changes which will facilitate more rapid searching include making conclusions bold, adding level of evidence labels to key evidence, making internal links from Treatment Overviews to the detailed supporting evidence, and separating large summaries into small sub-summaries. These content changes are being applied on an ongoing basis.
We believe we can currently fulfill most information needs within two to three minutes and often within one minute, and expect this to continue to improve as we apply changes to our content and user interface. We would welcome formal testing unencumbered by competing interests.
And if you have trouble finding an answer or finding an answer quickly, please e-mail us at editor@dynamicmedical.com -- we learn from these situations and adapt.
Competing interests: Dr. Alper is the editor-in-chief of DynaMed and works full-time as medical director of EBSCO Publishing Inc. which publishes DynaMed.
Show Less

Competing Interests: None declared.
Published on: (17 December 2005)
Page navigation anchor for Take Five
Take Five
Richard O. Schamp, St. Louis, MO, USA
As Dr. Alper and his colleagues provide an important contribution(1)to primary care by demonstrating the utility of an online database of synthesized evidence in answering real-time questions for clinicians. Most of us in primary care identify our own knowledge deficits in routine work, and we wish, for our patients' benefit, these deficits could be resolved quickly and accurately within a workplace learning environmen...
Show More

As Dr. Alper and his colleagues provide an important contribution(1)to primary care by demonstrating the utility of an online database of synthesized evidence in answering real-time questions for clinicians. Most of us in primary care identify our own knowledge deficits in routine work, and we wish, for our patients' benefit, these deficits could be resolved quickly and accurately within a workplace learning environment.
Many obstacles(2) face doctors who seek evidence-based questions in practice. Several obstacles coalesce around the time required. What is remarkable in the present study is that the average time to clinically meaningful answers was only five minutes. Compared to Medline or similar searching options, this is a huge advantage and practically eliminates some of the barriers to practicing clinicians who ask important clinical questions.
For over seven years, I've used DynaMed routinely, usually as my first search tool among several, witnessing its evolution into increasing utility for information management(3). Besides assisting with my own information searches (which usually take less than five minutes), it is valuable when I teach point-of-care information mastery to students and residents and other physicians. I use it to answer foreground (patient- specific) questions and to fill in background deficits. Though Alper, et al point out that their study is unable to determine which features or combinations of features are responsible for the advantageous outcome, the key features of DynaMed that facilitate the information management process include: --Surveillance methodology that provides continuous updates to topics --Robust search engine, with Boolean features --Hypertext links to PubMed or original articles
Finally, time savings and ease of use are meaningless if the results are not valid. The question of whether answers found in DynaMed are valid is addressed in the supplemental appendix to the article(4). DynaMed was found to meet or exceed the four comparators for 87.3% of the questions.
The article provides compelling support for practicing family physicians and educators to take the five minutes needed to answer the questions(5) that arise in our patients to improve our knowledge and ultimately its application toward excellent clinical care.
Sincerely,
Richard O. Schamp, MD
References
1. Alper B, White D, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synethsized evidence: A randomized trial in primary care. Annals of Family Medicine 2005, 3(6):507 -513.
2. Ely JW, Osheroff JA, Ebell MH, Champliss ML et al. Obstacles to ansering doctors¡¦ questions about patient care with evidence: qualitative study. BMJ 2002 Mar 23;324(7339):710
3. Slawson DC, Shaughness AF, Teaching Evidence-Based Medicine: Should We Be Teaching Information Management Instead? Acad. Med., July 1, 2005; 80(7): 685 - 689.
4. Alper B, Schamp R, White D, Hoock J, Supplemental Appendix. Level of Evidence comparison, http://annalsfm.highwire.org/cgi/content/full/3/6/507/DC1, accessed 16 Dec 2005.
5. Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319: 358-61.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (13 December 2005)
Page navigation anchor for The outcomes when clinicians use online evidence systems
The outcomes when clinicians use online evidence systems
Johanna I Westbrook, Sydney, Australia
Other Contributors:
As the information demands on clinicians continue to escalate, research that investigates effective means of supporting their access to accurate and timely information is imperative. Alper et al.(1)have made an important contribution by examining the use of a specific system when used in everyday practice. There is now a body of evidence demonstrating that hospital-based health professionals and family physicians are abl...
Show More

As the information demands on clinicians continue to escalate, research that investigates effective means of supporting their access to accurate and timely information is imperative. Alper et al.(1)have made an important contribution by examining the use of a specific system when used in everyday practice. There is now a body of evidence demonstrating that hospital-based health professionals and family physicians are able to integrate the use of online evidence systems into everyday clinical practice and that users report benefits to patient management as a result(2-4).
However measures of utilisation and self-reports of benefit only go so far. In acknowledging the limitations of their study, Alper et al demonstrate the inherent difficulties in studying this topic, highlighting the need to draw together research evidence from multiple trials of similar systems. A crucial question is the actual impact that these systems have on the decision-making process. We addressed this using an experimental study design(5, 6). Experienced hospital-based doctors, family physicians and specialist nurses were asked to answer a set of eight clinical scenario questions unaided. They were then randomized to an online evidence system allowing us to quantify the extent to which access to an online evidence system improved the performance of clinicians in correctly answering questions. We found that compared to answering questions unaided, online evidence retrieval systems improved correct answers to clinical scenarios by 21%.
Our study also demonstrated the challenges clinicians can face in interpreting information retrieved. For example, in 7% of 557 scenarios answered we found that clinicians provided an initially correct answer which was then incorrectly changed after they had searched for evidence using the system (ie they went from a right answer to a wrong answer). Analysis of computer web logs revealed that several participants viewed the same evidence yet provided different answers. Further, 54% of initially wrong answers failed to be corrected, with clinicians in each case documenting the specific evidence which they had retrieved from the online evidence system to support these incorrect answers. Interestingly, experienced specialist nurses, who performed poorly unaided when compared with their medical colleagues, improved their performance to equal the doctors once they had access to an online evidence system.
Thus the system design challenge is not just to provide clinicians with fast and accurate evidence, but to present it in such a way as to increase the chance that it will be accurately interpreted and have a positive impact upon decisions made.
References
1. Alper B, White D, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synethsized evidence: A randomized trial in primary care. Annals of Family Medicine 2005, 3(6):507 -513.
2. Westbrook J, Gosling AS, Coiera E. Do clinicians use online evidence to support patient care? A study of 55,000 clinicians. Journal of American Medical Informatics Association 2004, 11(2):113-120.
3. Pluye P, Grad R, Dunikowski L, Stephenson R. Impact of clinical information-retireval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies. International Journal of Medical Informatics 2005, 74(9):745-768.
4. Magrabi F, Coiera E, Westbrook J, Gosling A. Clinician use of online evidence in primary care consultations. International Journal of Medical Informatics 2005, 74(1):1-12.
5. Westbrook J, Coiera E, Gosling AS. Do online information retrieval systems help experienced clinicians answer clinical questions? Journal of the American Medical Informatics Association 2005, 12(3):315-321.
6. Westbrook J, Gosling AS, Coiera E. The impact of an online evidence system on confidence in decision making in a controlled setting. Medical Decision Making 2005, 25(2):178-185.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (7 December 2005)
Page navigation anchor for A landmark trial despite the lack of objective outcomes
A landmark trial despite the lack of objective outcomes
Pierre Pluye, Montreal, Canada
I would like to gratefully acknowledge the contribution of Alper and colleagues (1), and to comment this benchmarking study in line with our literature review on the impact assessment of databases (2). Alper and colleagues’ trial shows that information derived from a synopsis-based database improves clinical decision-making compared with information derived from other sources. However, outcomes were self-reported by par...
Show More

I would like to gratefully acknowledge the contribution of Alper and colleagues (1), and to comment this benchmarking study in line with our literature review on the impact assessment of databases (2). Alper and colleagues’ trial shows that information derived from a synopsis-based database improves clinical decision-making compared with information derived from other sources. However, outcomes were self-reported by participants, and these reports were apparently not controlled afterward using retrospective interviews or independant record assessment for example.
Despite the lack of objective outcome, this randomized trial constitutes a landmark study in the field for three reasons. First, while this trial was conducted in everyday practice, experimental studies are more often conducted in laboratories using a pre-designed set of clinical questions. Second, while this trial used a per-question randomization, other trials conducted in every day practice has been based on per- participant or per-source randomization. Third, to our knowledge only one Finnish trial measured change in clinical decision-making as an outcome, and found no difference between the use of guidelines within a database and that of paper-based guidelines. This absence of difference was not surprising as the paper-based compilation of over 1,000 guidelines is a usual source of information for practitioners in Finland (3).
Thus, Alper and colleagues are convincing in three ways: the generalizability of their results in Family Medicine, the use of searches for information as units of randomization, and the comparison between usual sources of information and a new database (new from the participants’ perspective).
Sincerely yours,
Pierre Pluye, MD, PhD
References
1. Alper BS, White DS, Ge B. Physicians Answer More Clinical Questions and Change Clinical Decisions More Often With Synthesized Evidence: A Randomized Trial in Primary Care. Annals of Family Medicine. 2005;3(6):507-513.
2.Pluye P, Grad R, Dunikowski L, Stephenson R. Impact of clinical information-retrieval technology on physicians: A literature review of quantitative, qualitative and mixed methods studies. International Journal of Medical Informatics. 2005;74(9):745-768.
3. Jousimaa J, Makela M, Kunnamo I, MacLennan G, Grimshaw JM. Primary care guidelines on consultation practices: The effectiveness of computerized versus paper-based versions. A cluster randomized controlled trial among newly qualified primary care physicians. International Journal of Technology Assessment in Health Care. 2002;18(3):586-596.
Competing interests: None declared
Show Less

Competing Interests: None declared.