Abstract
BACKGROUND We examined reports to a primary care, ambulatory, patient safety reporting system to describe types of errors reported and differences between anonymous and confidential reports.
METHODS Applied Strategies for Improving Patient Safety (ASIPS) is a demonstration project designed to collect and analyze medical error reports from clinicians and staff in 2 practice-based research networks: the Colorado Research Network (CaReNet) and the High Plains Research Network (HPRN). A major component of ASIPS is a voluntary patient safety reporting system that accepts reports of errors anonymously or confidentially. Reports are coded using a multiaxial taxonomy.
RESULTS Two years into this project, 33 practices with a total of 475 clinicians and staff have participated in ASIPS. Participants submitted 708 reports during this time (66% using the confidential reporting form). We successfully followed up on 84% of the confidential reports of interest within the allotted 10-day time frame. We ended up with 608 relevant, codable reports. Communication problems (70.8%), diagnostic tests (47%), medication problems (35.4%), and both diagnostic tests and medications (13.6%) were the most frequently reported errors. Confidential reports were significantly more likely than anonymous reports to contain codable data.
CONCLUSION A safe and secure reporting system that relies on voluntary reporting from clinicians and staff can be successfully implemented in primary care settings. Information from confidential reports appears to be superior to that from anonymous reports and may be more useful in understanding errors and designing interventions to improve patient safety.
INTRODUCTION
Despite the increased attention given to patient safety and medical errors, the major emphasis has been on hospital errors. Recent studies find that errors in primary care can have serious consequences,1, 2 though few published studies directly address medical errors and preventable adverse events in primary care.3– 6 Despite a growing awareness that errors in medicine are both important and preventable, several recent studies highlight considerable barriers to reporting and handling medical errors7 and to implementing effective strategies to reduce threats to patient safety.8– 10
After the landmark study by Brennan et al11 and the Institute of Medicine’s first report on medical errors,12 the US Congress authorized new funding for research in this area. Applied Strategies for Improving Patient Safety (ASIPS) is 1 of 24, 3-year demonstration projects funded by the Agency for Healthcare Research and Quality and 1 of 4 specifically focused on primary care. These projects are designed to collect and analyze medical error reports.13
Knowing the limitations in incident reporting systems, such as failure to recognize an event, failure to report because of time constraints, and concern with the safety of reporting,3, 14 we were interested in learning about (1) the types of events submitted to ASIPS using confidential or anonymous reports, (2) any differences in the ability of the 2 types of reports to capture information, and (3) differences among reporters using confidential or anonymous reports.
METHODS
The ASIPS project collects event reports from 2 practice-based research networks: the Colorado Research Network (CaReNet) and the High Plains Research Network (HPRN). The participating practices are located across Colorado and care for a diverse patient population in terms of age, race, ethnicity, socioeconomic status, and medical problems. The entire ASIPS project includes (1) a voluntary reporting system, (2) analysis of reported events, (3) analysis of data from such secondary sources as insurance claims, (4) educational feedback to practices, and (5) implementation of interventions to improve patient safety. The ASIPS protocol has been approved by the Colorado Multiple Institutional Review Board (COMIRB) and the institutional review boards of practices not covered by COMIRB.
The Patient Safety Reporting System
The ASIPS patient safety reporting system is a Web-based data collection and data management system, described in detail elsewhere.15 It is modeled on the Federal Aviation Administration (FAA) Aviation Safety Reporting System, which is widely recognized as a major success.12, 16, 17 Similar to the FAA reporting system, the ASIPS research team represents an outside third party: the project is not directly attached to any institutional quality improvement group, nor does any team member supervise those making reports.
We aimed to elicit wide participation from all clinical and nonclinical personnel in practices, assuming that anyone working in a practice might observe patient safety events. This approach represents a departure from most previous studies, which have examined patient safety in primary care from the physician’s perspective.1, 18
Reporting Events
We asked participants to report “any event you don’t wish to have happen again, that might represent a threat to patient safety.” Such a broad definition was used to allow participants freedom to use their interpretation of a safety event and is similar to the reporting criterion used in another primary care patient safety study.1 We emphasized that this definition includes near misses where no patient harm actually occurs.
ASIPS accepts confidential or anonymous reports by telephone hotline, secure Web site, or paper. All 3 modes use the same instruments. Anonymous reports consist of 11 multiple-choice items and 6 fill-in-the-blank questions (Table 1⇓). Confidential reports require minimal initial information—reporter’s name, telephone number, and a brief description of the event. By offering the 2 report types, we attempted to balance reporters’ desire for anonymity with our desire to collect complete event data for better understanding. Confidential reports also allowed us to track event types (not specific events) by practice for direct feedback for practice improvement.
Reporting Form Questions
For most confidential reports, we scheduled follow-up interviews with the reporter to gather further detailed information. We allowed 10 days from the time we received a report to contact the individual for an interview. In all cases, contact information was automatically expunged from the database within 10 days of submission.15 Practices (not individual reporters) were paid $50 for each completed confidential follow-up interview to offset the cost of time taken for the interview. Reporting forms and interviewers did not ask for any personally identifying information about the patient(s) involved or permanently identify the practice in which the event occurred. All reports were reviewed and coded by teams, composed of at least 3 members, including 1 physician.
Taxonomy
We used a multiaxial taxonomy of medical errors19 to code events. The portion of this taxonomy used by ASIPS contains 10 axes within 4 domains. A code must be applied from each axis for every report (taxonomy axes and domains are available in Appendix 1, which can be found online as supplemental data at http://www.annfammed.org/cgi/content/full/2/4/327/DC1). Multiple codes may be used within any single domain. The axes allowed us to classify the setting of each event, the clinical domain (eg, procedures, medications), patient outcome, course and cause(s), discoverer, and roles and contribution of participants involved. An event participant was someone who was involved in the event at the time the event occurred. Because many events occurred over an extended time, a single report could include multiple participants. Patients were included as participants only when their conscious action or inaction contributed to the event.
The taxonomy includes a code category of “unknown” (unknown codes) in all 10 axes to indicate insufficient information is available to code the event in greater detail. Analysis was performed using SAS version 8.2.
Analysis 1. Differences in the Nature of Reports Submitted
Figure 1⇓ shows the number of safety events that were received and how they were grouped for analysis. We used all 608 coded events for analysis. Anonymous reports were compared with confidential reports for differences in who made the report and the nature of the error reported.
Inclusion and outcome flow diagram for reports to Applied Strategies for Improving Patient Safety (ASIPS). * Not relevant to ASIPS analysis or not an ambulatory event (eg, occurred entirely in inpatient setting; no unexpected outcome; not related to patient care or safety).
Analysis 2. Ability to Capture Detailed Information
We compared anonymous reports with the subset of confidential reports that had completed follow-up interviews (Figure 1⇑). This analysis compared the ability of the 2 types of reports to capture event detail. We also compared anonymous reports with all confidential reports (Figure 1⇑) to understand the overall ability of confidential reports to capture event information. For each analysis, unknown codes were compared with all other known codes within these domains. Furthermore, we examined the number of reports with 1 unknown code and the number with more than 1 unknown code to examine the extent of lost information.
Analysis 3. Ability to Capture Harm Information
We classified event reports into 5 mutually exclusive categories of harm: clinical harm, future risk of clinical harm, nonclinical harm, unstable (too early to ascertain harm), and no known harm (a combination of no reported harm and unknown). (Appendix 2, which can be found online at http://www.annfammed.org/cgi/content/full/2/4/327/DC1, provides supplemental detail concerning the harm categories.) Each event report was coded into 1 and only 1 harm category. We also formed combined categories within the 5-category harm hierarchy: any clinical harm combined the first 2 categories (clinical harm and future risk of clinical harm), and any harm combined any clinical harm and nonclinical harm.
RESULTS
Since the inception of ASIPS in October 2001, we have enrolled 33 practices from which 475 clinicians and staff consented to participate in the study. As of August 2003, the ASIPS patient safety reporting system has received more than 708 reports, of which 608 were coded for further analysis. Reports were received through the following systems: telephone 9%, Web 28%, and paper 63%. We applied a total of 8,605 individual taxonomic codes to the 608 events.
Description of Type of Report by Reporter Type
Reporters used the confidential report form 66% of the time. We attempted to follow up on 91% of the confidential reports, with a success rate of 84% (76% of total). Table 2⇓ shows the role of the reporter and the percentage of confidential reports submitted by that reporter group. Office staff were more likely than clinicians to use the confidential reporting method.
Type of Error by Report Type
Of the 394 coded events for which we could determine event participants, 57.6% involved a physician or other independent provider (eg, nurse practitioner or physician assistant) as one of the participants, 32.2% involved another licensed clinician (eg, registered nurse, licensed practical nurse, medical assistant, radiology technician), 23.6% involved a nonlicensed caregiver or office staff, 14.2% involved a patient or someone related to the patient, and 23.6% involved a third party (eg, a payer, a supplier, a regulatory or government body, or a health care facility outside the practice). These figures add up to more than 100%, indicating that many events involved more than 1 type of participant. In fact, 42.4% involved 2 participants, and 13.1% involved 3 to 5 participants.
Anonymous vs Confidential Reports
Results from Analysis 1: Differences in the Nature of Reports Submitted
Table 3⇓ displays data on the nature of reported events stratified by reporting method. No particular type of event was reported more or less commonly by confidential reports. Overall, our data show the preponderance of errors associated with diagnostic tests, 47% of events reported. Medication errors appeared in 35.4% of reports, and both a diagnostic testing and a medication error appeared in 13.6% of reports. Communication errors were identified in 70.8% of reports
Reporting Groups and Percentage of Confidential Reports
Results from Analysis 2: Ability to Capture Detailed Information
Table 4⇓ displays the percentage of anonymous, completed confidential, and all confidential reports that had insufficient information to permit coding of the 9 examined axes or subaxes. In addition, the table displays the percentages of each type of report that had at least 1 unknown code, as well as the percentages of reports with more than 1 unknown code. Not surprisingly, completed confidential reports had the lowest rate of unknown codes compared with anonymous reports, with 8 of the 11 comparisons (72.7%) being statistically significant. Even after adding the reports with uncompleted follow-up interviews, confidential reports were more likely than anonymous reports to have complete information, although only 5 of the 11 comparisons (45.5%) were statistically significant.
Percentage of Unknown Codes by Report Type
Results from Analysis 3: Ability to Capture Harm Information
The distribution of events among anonymous, completed confidential, and all confidential reports by the 5 mutually exclusive harm categories is shown in Table 5⇓. All 3 distributions are similar, indicating that the reporting of various levels of harm does not vary by whether the report is made anonymously or confidentially. A comparison of anonymous with completed confidential reports and of anonymous with all confidential reports found no significant differences (P = .146 and P = .338, respectively). Cross-tabulations of anonymous-confidential reports to harm categories revealed only 1 statistically significant difference (unstable-yes or no vs anonymous-completed confidential reports, P = .045). (This information is available in Appendix 3, which can be found online as supplemental data at http://www.annfammed.org/cgi/content/full/2/4/327/DC1).
Harm Stratified by Anonymous vs Confidential Reports
DISCUSSION
The ASIPS 2-report patient safety reporting system strikes a balance between preserving anonymity for those who wish it while providing an opportunity to collect rich data about many events from those willing to submit confidential reports and complete follow-up interviews. Confidential reports are more likely to include sufficient information to allow detailed coding. Assuming that the purpose of a patient safety reporting system is to reduce errors through changes in clinical practice, confidential reports appear to offer greater potential to understand the processes that are likely targets for intervention, especially in complex cases. Anonymous reports appear equivalent to confidential reports for understanding the risk (harmful outcome or not) associated with safety events. If patient safety reporting system data are used primarily to identify patterns of events that are candidates for further data collection, then anonymous reports may suffice, even with a lower level of detail.
Similar to those of previous studies, our findings highlight the value of a multifactorial approach to coding errors, because many events involve more than 1 person or error.20, 21 Likewise, while communication errors appear as a component of many errors, they are infrequently viewed as the focus of an event; thus, the extent of the contribution of communication breakdowns to errors might not be reflected in coding systems that apply a single label.1, 14
Confidential reports require time for the research staff to collect a complete set of data. While a complete data set allows for clarification and detailed information, it is also costly. It is possible that combined systems (such as the FAA system and the Australian Patient Safety Foundation system)16, 22 requiring a great amount of structured data entry, even when submitting confidential reports, could maintain the effectiveness of confidential systems while decreasing their cost. Greater structured data entry may also improve anonymous reports. Early in the learning process, structured data collection often slows the data-entry process.23 It will be important to be careful when designing system interfaces for long-term use, because our participants indicated that lack of time to create a report is a major barrrier to system use.
CONCLUSION
Although much work lies ahead to improve patient safety in primary care, we successfully implemented a safe and secure reporting system that relies on voluntary reporting by clinicians and staff. Many participants are willing to submit confidential reports in which they are temporarily identified. The greater detail of these reports adds to our ability to understand errors and design interventions to decrease them. Further studies are needed to clarify the appropriate level of report detail for a patient safety reporting system to be useful and to clarify approaches for improving report detail through structured data entry. Further study is also needed to examine the optimal level of coding detail required when used in conjunction with qualitative analysis.
Acknowledgments
We gratefully acknowledge the contributions of all ASIPS participants, especially those who have entrusted us with their stories of medical errors.
Footnotes
Conflicts of interest: none reported
Funding support: Support was provided by the Agency for Healthcare Research and Quality, grant #U18-HS011878, Wilson D. Pace, MD, principal investigator.
- Received for publication October 17, 2003.
- Revision received February 27, 2004.
- Accepted for publication March 11, 2004.
- © 2004 Annals of Family Medicine, Inc.
REFERENCES
Jump to section
Related Articles
Cited By...
- Ambulatory Medication Safety in Primary Care: A Systematic Review
- Exploration of prescribing error reporting across primary care: a qualitative study
- Implementation and Evaluation of a Laboratory Safety Process Improvement Toolkit
- International recommendations for national patient safety incident reporting systems: an expert Delphi consensus-building process
- Patient safety principles in family medicine residency accreditation standards and curriculum objectives: Implications for primary care
- System hazards in managing laboratory test requests and results in primary care: medical protection database analysis and conceptual model
- Reporting and Using Near-miss Events to Improve Patient Safety in Diverse Primary Care Practices: A Collaborative Approach to Learning from Our Mistakes
- Laboratory Medicine Handoff Gaps Experienced by Primary Care Practices: A Report from the Shared Networks of Collaborative Ambulatory Practices and Partners (SNOCAP)
- A comparison of calls subjected to a malpractice claim versus 'normal calls' within the Swedish Healthcare Direct: a case-control study
- Managing Patient Populations in Primary Care: Points of Leverage
- Emotional Impact of Patient Safety Incidents on Family Physicians and Their Office Staff
- Safety incidents in family medicine
- Central or local incident reporting? A comparative study in Dutch GP out-of-hours services
- Feasibility of centre-based incident reporting in primary healthcare: the SPIEGEL study
- Field Test Results of a New Ambulatory Care Medication Error and Adverse Drug Event Reporting System--MEADERS
- Safety learning system development--incident reporting component for family practice
- What happens between visits? Adverse and potential adverse events among a low-income, urban, ambulatory population with diabetes
- Management of Test Results in Family Medicine Offices
- Medication errors reported by US family physicians and their office staff
- "Every error counts": a web-based incident reporting and learning system for general practice
- Testing process errors and their harms and consequences reported from family medicine practices: a study of the American Academy of Family Physicians National Research Network
- Mitigation of patient harm from testing errors in family medicine offices: a report from the American Academy of Family Physicians National Research Network
- Patient safety events reported in general practice: a taxonomy
- Evaluation of an intervention aimed at improving voluntary incident reporting in hospitals
- Ambulatory care adverse events and preventable adverse events leading to a hospital admission
- Rural Community Members' Perceptions of Harm from Medical Mistakes: A High Plains Research Network (HPRN) Study
- Stopping the error cascade: a report on ameliorators from the ASIPS collaborative
- Harm Resulting from Inappropriate Telephone Triage in Primary Care
- Anatomy of a patient safety event: a pediatric patient safety taxonomy
- Electronic Data Collection Options for Practice-Based Research Networks
- Primary Care Practice-Based Research Networks: Working at the Interface Between Research and Quality Improvement
- In This Issue: Practice Change and Patient Safety
- Why Isn't It Better?