Abstract
PURPOSE In this study, we developed and field tested the Medication Error and Adverse Drug Event Reporting System (MEADERS)—an easy-to-use, Web-based reporting system designed for busy office practices.
METHODS We conducted a 10-week field test of MEADERS in which 220 physicians and office staff from 24 practices reported medication errors and adverse drug events they observed during usual clinical care. The main outcomes were (1) use and acceptability of MEADERS measured with a postreporting survey and interviews with office managers and lead physicians, and (2) distributions of characteristics of the medication event reports.
RESULTS A total of 507 anonymous event reports were submitted. The mean reporting time was 4.3 minutes. Of these reports, 357 (70%) included medication errors only, 138 (27%) involved adverse drug events only, and 12 (2.4%) included both. Medication errors were roughly equally divided among ordering medications, implementing prescription orders, errors by patients receiving the medications, and documentation errors. The most frequent contributors to the medication errors and adverse drug events were communication problems (41%) and knowledge deficits (22%). Eight (1.6%) of the reported events led to hospitalization. Reporting raised staff and physician awareness of the kinds of errors that occur in office medication management; however, 36% agreed or strongly agreed that the event reporting “has increased the fear of repercussion in the practice.” Time pressure was the main barrier to reporting.
CONCLUSIONS It is feasible for primary care clinicians and office staff to report medication errors and adverse drug events to a Web-based reporting system. Time pressures and a punitive culture are barriers to event reporting that must be overcome. Further testing of MEADERS as a quality improvement tool is warranted.
- Medication error
- adverse drug event
- event reporting
- primary care
- adverse drug reaction reporting systems
- online systems
- practice-based research
INTRODUCTION
Seventy-five percent of office visits to family physicians and general internists are associated with the continuation or initiation of medications.1 Physicians in the United States write more than 3.6 billion prescriptions annually in the ambulatory setting.2 Medication errors and adverse drug events are common in this setting. A medication error is an error in medication prescribing, dispensing, or use that may lead to a preventable adverse drug event. An adverse drug event is an injury due to medication use.3 A meta-analysis of 29 studies found an incidence of about 15 adverse drug events per 100 outpatients per year, of which 20% were judged preventable.4 Because of the huge volume of medications used by ambulatory patients in unmonitored situations, the potential risk due to medication errors and adverse drug events from office prescribing is many times greater than that from hospital prescribing.
Various methods have been proposed for reducing medication errors and adverse drug events in ambulatory care, including chart reviews, patient interviews, electronic “trigger tools”5–7 (computer algorithms that identify high-risk situations), medication reconciliation processes, and event reporting. None of these interventions has a solid evidence base for effectiveness in improving medication safety in outpatient care.8 Event reporting, however, has been used effectively as a tool to improve safety in high-risk industries including hospital medicine. Event reporting is not designed to determine event or error rates. It is a safety improvement method that takes advantage of the astute observations of workers on the front line to detect problems that may be mitigated by changes in systems, policies, and procedures. It is complementary to other safety and quality improvement methods, such as chart audits, that can be used to estimate error rates.
All hospitals use medication event reporting for safety improvement because it is required to maintain accreditation and is firmly entrenched in hospital culture. Event reporting has had minimal uptake in ambulatory care practices, however, because of cost, busy work flows, liability concerns, and the complexity of outpatient prescribing, and also because event reporting is not a part of office culture.1 If medication event reporting is to become an effective safety tool in office practice, event reporting systems must be tailored to meet the needs and constraints of busy primary care practices that have many competing demands.
In 2006, the Agency for Healthcare Research and Quality initiated the development and field testing of a practical, national medication event reporting system for ambulatory practices. The project included 4 phases: a literature review, an expert panel meeting, system design and development, and pilot and field testing of the system, called the Medication Error and Adverse Drug Event Reporting System (MEADERS). In this report, we describe the design, development, and field testing of MEADERS in 4 primary care practice-based research networks in the United States.
METHODS
Design Principles of MEADERS
To inform design and development, the research team solicited advice from 19 experts in primary care patient safety and medication safety. On the basis of their opinions and an extensive literature review, we arrived at the following design principles:
-
The focus of the system should be on reducing errors and adverse events in ambulatory practices and providing useful feedback to those reporting the events.
-
It should collect a limited data set for medication errors and adverse drug events.
-
The system should be electronic (with a paper option), available either over the Internet or as a free-standing system.
-
The interface should be intuitive and easy to use with minimal training required.
-
Data entry should be as simple as possible, and logins should not be required because they slow people down and are a barrier to reporting.
-
The system should be able to capture the story of an event with open text boxes.
-
It should also have autoclassification features that allow instant aggregation of certain descriptors of the event, such as medications involved, frequencies and types of events, degree of harm, and contributing factors.
-
Aggregated data reports should be available immediately after event reporting and should allow a practice to compare their data with regional and national data reports.
-
The system should be secure, that is, compliant with the Health Insurance Portability and Accountability Act (HIPAA), and should allow confidential reporting to minimize fear of negative repercussions. (Since we wished to report patient-identifiable data back to the practice, an encrypted patient identifier was required.)
-
A reporter should have the option of sending the event report to the Food and Drug Administration (FDA) MedWatch program by clicking a button or faxing.
The technical specifications of MEADERS have been published previously.9 Practices using the Web version, the one we tested in this study, can collect data from many locations and aggregate it centrally. MEADERS contains a report generator that displays a summary of the medication errors and adverse drug events submitted, organized by practice. It also contains a tool for browsing national ambulatory error reports contained in the MedMarx database so that a practice or organization can compare its own medication events with national data.
The MEADERS Event Reporting Form
Data fields of the MEADERS event reporting form were adapted from the FDA MedWatch minimum data set, the US Pharmacopeia MedMarx database, and the event reporting form developed by the Robert Graham Center and the American Academy of Family Physicians National Research Network.10–13 For rapid entry of medications, we developed a drop-down pick list of 4,008 medications derived from the medication dictionary of the Regenstrief Medical Record System.14 The data fields of the MEADERS event reporting form are listed in Supplemental Table 1, available online at http://annfammed.org/cgi/content/full/8/6/517/DC1.
Study Setting and Testing Procedure
Before pilot testing, early versions of MEADERS were reviewed iteratively by representatives of 4 practice-based research networks to provide suggestions for user-friendly design. A pilot test was conducted in ResNet, a research network of primary care practices affiliated with Indiana University School of Medicine.15 After appropriate revisions, MEADERS was field tested in 4 practice-based research networks: (1) the Central Texas Primary Care Research Network (CenTexNet), the primary care research network of Scott & White Clinic; (2) the Los Angeles Network (LANet), a network of community health centers and Federally Qualified Health Centers in the Los Angeles area; (3) the Oregon Rural Practice-based Research Network (ORPRN), a rural network of primary care practices in Oregon; and (4) the Connecticut Center for Primary Care (CCPC), a network of primary care private practices in rural, suburban, and urban Connecticut. Characteristics of the 24 participating practices are summarized in Table 1⇓.
We used a train-the-trainer model for disseminating the MEADERS system and the study methods by holding a 1-day, intensive training session for the network leaders and research coordinators in Bethesda, Maryland, in June 2007. Each network recruited at least 5 practice sites to participate in the study, and each network did their own site training using standard materials provided them. All practice sites were required to have Internet-linked computers easily available throughout the office but were not required to have electronic medical records. We allowed (though discouraged) the use of paper event reports that could be entered into MEADERS on the Web later by the clinician or an assistant in the practice. The 10-week reporting period was October through mid-December 2007. Study coordinators visited each of the participating practices at least once and usually twice or more to assist with and observe implementation of MEADERS.
Data Analysis
We produced frequency distributions of the reported data elements, including type of reporter, type of event, contributing factors, harm to the patient, and medications involved in the events. Three investigators (J.H., G.M.K., W.D.P.) coded medication classes according to the Physicians’ Desk Reference drug classes; consensus for any discrepancy was reached by discussions among these investigators. The MEADERS Web site recorded how long the reporter was connected to the Web site to track reporting time. We excluded connections lasting more than 15 minutes because these were probably occasions when the reporter forgot to close out the site.
We used 2 methods to evaluate the experience of physicians and staff who participated in this field test of MEADERS. First, at the conclusion of the reporting period, we asked participants to complete a postreporting questionnaire either online or on paper. Second, during the study, research coordinators of the 4 networks kept field notes, and immediately after MEADERS field testing, they performed semistructured e-mail surveys, telephone interviews, or face-to-face interviews of a lead physician or office administrator from each practice to assess participants’ experience including barriers to and facilitators of reporting. The network directors (L.M.K., L.J.F., J.T.L., S.N.F.) and research coordinators reviewed their field notes, e-mail survey responses, and interview notes; summarized their major findings; and forwarded these to one of the authors (J.H.), who synthesized the data from all 4 networks into a single report. The network directors reviewed this report for accuracy, and appropriate corrections were made.
The study protocol was approved by the institutional review boards of Indiana University, Scott & White (CenTexNet), and Oregon Health & Science University (ORPRN).
RESULTS
Participants
A total of 220 physicians and office staff from 24 practices participated in this study (Table 2⇓). In CenTexNet, only clinicians were invited to submit event reports; in the other networks, both clinicians and staff were invited to do so. None of the physicians or office staff refused to participate, but not all submitted event reports.
Event Reports
During the 10-week reporting period, reporters submitted 507 medication event reports that involved patients aged 2 months to 96 years. Thirty-seven percent of the reports were submitted by nurses, 34% by physicians, 11% by nurse practitioners and physician assistants, 5% by pharmacists and pharmacy technologists (all from community health centers where there was an on-site licensed pharmacy or dispensary), and 13% by other staff including medical assistants, receptionists, and office managers. The mean time spent online submitting an event report was 4.3 minutes, with a range of 55 seconds to 15 minutes. (We excluded 10 reports taking more than 15 minutes.) The number of event reports per week per network is given in the Supplemental Figure, available online at http://annfammed.org/cgi/content/full/8/6/517/DC1. The number submitted during the entire study varied greatly by practice, ranging from 0 to 83 (median = 14, SD = 20.6).
The types of medications involved in the events are summarized in Table 3⇓. The type most commonly involved was cardiovascular medication. Events associated with 1, 2, and at least 3 medications occurred in 82%, 13%, and 5% of the reports, respectively. Twenty-two percent of the reports described events occurring the day of the report; 36%, events during the past week; and 22%, events during the past month.
The majority of the event reports, 357 (70%), described medication errors only, 138 (27%) described adverse events only, and 12 (2.4%) described a combination of both. The types and frequencies of the medication errors are given in Table 4⇓. The errors were roughly equally divided among ordering medications, implementing prescription orders, errors by patients receiving the medications, and documentation errors.
The factors that reporters believed contributed to the adverse drug events and medication errors are listed in Table 5⇓. The leading factors cited were communication problems (written, 30%; verbal, 11%) and knowledge deficits (22%). The most common written communication problems were wrong doses and wrong drugs on written communications between physician offices and pharmacists, and errors on medication lists. The most common verbal communication problems were misunderstood verbal physician orders and mis-communications between patients and pharmacists or between nurses and pharmacists regarding the correct medication or dosage. For the most part knowledge deficits were situations wherein the pharmacist or patient lacked knowledge regarding the currently prescribed dose or medication, or a physician prescribed an unavailable dosage or form of the medication. In one report, the patient could not read and took 4 times the correct dose of the medication.
In 11% of events, the reporters judged there was physical harm, emotional harm, or both to the patient; in 69% of the events, they judged there was no harm; and in 20%, harm was unknown or data were missing, as shown in Table 6⇓. Of the 50 patients (10%) whom reporters thought were harmed, the injury resolved without intervention in 33 patients, resolved with intervention short of hospitalization in 9 patients, and required hospitalization in 8 patients. The reporters thought 3 patients had sustained permanent injury as a result of the event. Stated another way, 8 of 507 (1.6%) of the reported events lead to hospitalization and 3 of 507 (0.6%) led to permanent injury. Of the 8 events leading to hospitalization, 3 were due to medication errors, 4 were due to an adverse drug event, and 1 involved both.
Although events could be reported to the FDA by checking a single box, reporters elected to submit only 8 event reports (1.6%) to the FDA MedWatch reporting system. Of these 8 submitted reports, 3 involved physical harm to patients and all 3 patients fully recovered.
Participants’ Evaluation of MEADERS
Of the 220 physicians and staff members who participated in the MEADERS field test, 164 (75%) returned the postreporting questionnaire describing their experience with the system. Thirty-one percent of the respondents were physicians, 19% were physician assistants or nurse practitioners, 33% were nurses, medical assistants, and laboratory technicians, 10% were office managers, 1% were pharmacists, and the remainder were other administrative staff. Important quantitative and qualitative findings from this follow-up survey are summarized in Tables 7⇓ and 8⇓, respectively. For the most part, respondents found MEADERS easy to use and understood what events they should report. Many agreed with statements that suggest reporters learned by reporting events, making them more aware of the kinds of errors that occur in medication management. A substantial minority reported that they had changed something in their practice based on the events reported; however, 36% agreed or strongly agreed that the event reporting “has increased the fear of repercussion in the practice,” an undesirable result of event reporting.
Responses to Open-Ended Questions About MEADERS Implementation
The post–field test interviews and field notes of the network study coordinators provided useful insights into MEADERS implementation in the practices. Experiences varied across the 4 practice-based research networks.
CenTexNet trained a physician champion in each of their practices, which facilitated reporting. The key barriers to reporting in this network were time constraints, coincidence of the project period with the influenza season, staff turnover, and conflict with clinic work flow. Because physicians feared interference with work flow, 95% of CenTexNet’s event reports were written on paper first, then entered later on the Web site by an office staff member. All reports from the other 3 networks were submitted directly to the Web site.
In the rural ORPRN network, a weekly electronic newsletter was distributed to the lead clinician and a staff member at each site responding to participant concerns and provided feedback on the number of event reports submitted. One-half of the practices periodically distributed MEADERS data reports to all clinic staff. In others, only clinicians reviewed the data. Some practices in ORPRN said they had used information from the event reports to improve current clinical practices. Others expressed the viewpoint that although reporting to MEADERS for a limited time was feasible, ongoing reporting would be difficult because of time constraints.
LANet provided a community health center perspective. Community health centers often operate dispensaries or pharmacies on site, and they must comply with a variety of reporting requirements on medication errors. Several had paper-based medication error and adverse drug event reporting systems in place that already recorded an average of 1 to 3 of these incidents per day, so the MEADERS system was redundant. But clinicians and dispensary/pharmacy personnel commented that MEADERS would be very useful if it could be modified to meet their current reporting requirements, especially if it could be incorporated into an electronic medical record. (None of these sites had such records at the time of this study.) Some physicians were not convinced of the value of reporting medication errors and adverse drug events that were not clinically important, and in the larger community health centers with full-time dispensary staff or pharmacists, the clinicians viewed such reporting as pharmacists’ responsibility.
CCPC participants reported that the system was surprisingly easy to use. Some, including those at a practice that did not submit any reports despite agreeing to participate, expressed concern about the potential legal liability of reporting errors to a national system. This concern was not a major issue in the other networks. Unlike the other networks, the Connecticut participants submitted more reports of adverse drug events (77) than medication errors (48). A number of participants did not consider reporting on medication errors to be safe. Only 25 (20%) of the Connecticut event reports identified an error committed by a physician. None of the participants reported an event to MedWatch because they were unfamiliar with this program, they felt it was a black hole, or they wanted to avoid a long follow-up call interrupting their patient care.
All network leaders commented that the role of the practice facilitators was crucial in installing the MEADERS icon on desktops, training physicians and staff how to use the system, and prompting physicians and staff to report medication errors.
DISCUSSION
We successfully field tested MEADERS, a new, easy-to-use, Web-based medication event reporting system designed specifically for office practice. Physicians and staff from urban, suburban, and rural primary care practices in 4 states were able to identify and report 507 medication events during a 10-week period with little difficulty and minimal time demand. Despite physicians’ fears in 1 network that reporting to the Web would interfere with their work flow, most physicians who submitted reports did so directly to the Web site. From the survey conducted after event reporting, we learned that reporting medication events had a real impact on the safety culture of some of the practices by increasing awareness of medication errors and adverse drug events. Anecdotally, participation in event reporting led to changes in some office routines for managing medications. We identified 2 important and expected barriers to medication event reporting, however: time pressure and fear of reprisal.
Reporters identified errors that were nearly equally distributed throughout the medication management spectrum in 4 major categories: ordering, dispensing, receiving, and documenting. Few errors were reported for drug monitoring, so this may be a weak link in the medication event report because this type of error has been important in studies of medication events that lead to hospital admissions. Vigilance is therefore required throughout the medication management process. It is not surprising that communication problems—including errors that were written (30%), verbal (11%), and transcribed (14%)—were reported as the most frequent contributor to the medication events. In a study from Colorado family medicine offices, communication problems contributed to 70% of all types of errors reported.16 In our study, more than 50% of the reports were generated by nonphysicians. Methods to improve communication are much needed in busy ambulatory primary care settings and should involve practice clinicians and staff. Clinician and staff education to improve medication knowledge is needed as well: in 22% of the MEADERS reports, knowledge deficits contributed to the error. We suspect that a number of these knowledge and process errors may be less frequent with electronic prescribing,10 although strong empirical data from ambulatory practices to support this assertion are lacking at present.
The types of medication most often associated with the events reported in this field test of MEAD-ERS are similar to those identified in 2 previous event reporting studies conducted in primary care settings10,16: medications used for cardiovascular disease, central nervous system conditions (including analgesics), and endocrine disease (mainly diabetes), and antibiotics. Interventions to prevent medication errors and adverse drug events in outpatient primary care practice could target these classes of medications, although a comprehensive reengineering approach may be needed in some practices.
Similar to findings of a prior study of event reporting from family physician offices,10 more than two-thirds of the medication event reports resulted in no harm, and serious harm occurred relatively infrequently. (Harm was either unknown or not reported in 19% of events, but it is typical in reporting studies to have a large percentage of cases wherein harm is unknown because the reporting system does not require any further investigation of the incident.) Nonetheless, 8 hospitalizations of patients reported by 110 clinicians over a 10-week period translates to many thousands of US hospital admissions per year due adverse drug events and medication errors. In a systematic review of 15 studies, Winterstein et al17 estimated that 4.3% of US hospital admissions are due to preventable drug-related hospital admissions.
MEADERS is a small step forward in improving medication safety in ambulatory care. We demonstrated that it is feasible to deploy a Web-based medication event reporting system that clinicians and staff can understand and use in busy primary care practices. At least 43% of those who agreed to participate in the study reported 1 or more medication events. This is an extraordinary participation rate for event reporting, but it was in the context of a research study in motivated practices having engaged practice facilitators, so such a high rate of reporting seems unlikely under ordinary circumstances. The real challenge, however, is to demonstrate that event reporting in ambulatory care is sustainable and that the data from event reporting can be used to identify and to correct systems problems in office practice to reduce medication errors, adverse drug events, hospital admissions, and patient harm. Our study suggests that a culture of safety has not yet taken hold in all primary care practices; some of our volunteer reporters feared possible reprisal from reporting errors. This situation must change before event reporting can be a catalyst for improvement in ambulatory care. The enabling legislation for Patient Safety Organizations now provides strong legal protections for event reporting related to quality improvement. (See details at http://www.pso.ahrq.gov.) In addition, sufficient time must be available for clinicians and staff to report events, reflect on their meaning, and take action to improve office processes.
Finally, MEADERS as tested was a prototype system and had technical limitations that can be addressed in a subsequent version. For example, the system is not able to link to and automatically prepopulate data contained within electronic medical record systems. Such a link could reduce the reporting burden as some of the data fields could be automatically filled in. Real-time reporting is not possible in the current version. For example, the system does not produce an alert that an error report was filed and that a particular drug should be discontinued.
Our test of MEADERS was limited to 10 weeks and took place in volunteer practices that are members of practice-based research networks. Whether other practices could use this medication event reporting system effectively is not known. Many questions remain. Is sustained reporting over time possible? Or, would it make more sense to do it in short bursts of quality improvement cycles? What types of practices would learn and use MEADERS? Would it be an effective tool in reducing medication errors and adverse events in a larger and more representative group of practices? MEADERS or a similar tool should be integrated into electronic medical records to make event reporting a seamless activity that is a part of normal work flow for clinicians and office staff. Further testing of MEADERS as a medication management quality improvement tool for office practice is warranted.
Acknowledgments
We thank the physicians and staff from the practices that participated in this study; the study coordinators Melinda Davis and Sonia Holleman; and the consultants: Jim Battles, PhD, AHRQ; Paul Biondich, MD, Indiana University and the Regenstrief Institute; Lynn Bosco, MD, MPH, AHRQ; Michael Cohen, RPh, MS, ScD, Institute for Safe Medication Practices; Diane Cousins, RPh, US Pharmacopeia; R. Scott Evans, PhD, Intermountain Health Care; Jeff Friedlin, DO, the Regenstrief Institute; Kim Galt, PharmD, Creighton University; Linda Greenberg, AHRQ; Harold Kaplan, MD, Columbia University; Mark Lehto, PhD, Purdue University; Alan Lembitz, MD, COPIC Insurance Company; Gurdev Singh, PhD, Patient Safety Research Center, SUNY Buffalo; Ranjit Singh, MD, MBA, Patient Safety Research Center; Scott Smith, PhD, MPH, Center for Outcomes and Evidence; Kathleen Stevens, RN, EdD, University of Texas Health Science Center; Ann Trontell, MD, MPH, AHRQ; Morris Weinberger, PhD, University of North Carolina; Carl Williams, MD, the Regenstrief Institute.
Footnotes
-
Conflicts of interest: none reported
-
Funding support: The study was funded by the Agency for Healthcare Research and Quality (AHRQ), contract No. 290880008.
- Received for publication August 11, 2009.
- Revision received April 19, 2010.
- Accepted for publication May 11, 2010.
- © 2010 Annals of Family Medicine, Inc.