Article Text
PDF
Abstract
Background: Incident reporting systems have been established primarily in the inpatient setting. Their goal is the identification of safety risks in healthcare as a precondition for improvements in the overall quality of care. Knowledge about medical errors in general practice is sparse, as are reporting systems for patient safety in this setting. This article describes the development, structure and initial results of an incident reporting system for general practices in German-speaking countries.
Methods: Jeder Fehler Zaehlt (JFZ; www.jeder-fehler-zaehlt.de) is a web-based reporting system that receives incident reports from anonymous German-speaking users. Reports are fed into a database, classified and analysed by a team of experts. Exemplary reports are published on the internet and in journals, and the reporting system provides the opportunity for users to comment on and learn from these incidents.
Results: The incident reporting system received 199 reports in the 17 months following its launch, of which 188 were classifiable. Of these, 72.9% were classified as process errors and 26.1% as knowledge/skills errors. The most frequently reported process errors were treatment errors (32.2%), communication errors (12.6%) and investigation errors (8.5%). Of the classified errors, 41.5% were associated with harm to the patient. More than 300 comments were made, mostly by anonymous users of the system.
Conclusions: JFZ is a well-functioning and growing incident reporting system. Future efforts to improve the benefits of incident reporting will concentrate on increasing the utilisation of the system and broadening the spectrum of reported incidents.
Statistics from Altmetric.com
Incident reporting systems in medicine were first introduced two decades ago following the example of reporting systems in other industries such as aviation, chemicals and nuclear power.1 Incident reporting systems systematically collect information on adverse incidents, with the aim of identifying risks and thus enabling healthcare providers to improve quality. To this end they should be non-punitive, confidential or anonymous, independent, timely, systems orientated, responsive and provide expert analysis.2
Currently, incident reporting systems cover a relatively large portion of the inpatient sector. They are often established for certain kinds of incidents (eg, the National Coordination Council for Medication Error Reporting and Prevention (NCC MERP), USA). Several healthcare providers, such as the Veterans Health Administration, or organisations such as the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), have made it obligatory for healthcare providers to report adverse events. The National Reporting and Learning System (NRLS; England and Wales) collects incident reports from all healthcare settings, including specific medical incidents in general practice.3 It constitutes the first appreciable routine application of this method in primary care. The NRLS was set up in 2003 and is receiving a steadily growing number of incident reports.
However, knowledge about medical errors in primary healthcare remains sparse. Between 1981 and 2004 few studies evaluated the frequency and nature of errors in primary care or general practices and their results ranged from fewer than 4 to 80 errors per 100 000 consultations.4–9 No data on the frequency of errors in German general practices are yet available.
The increasing complexity of care (and increasing probability of harm to the patient) in the ambulatory setting has intensified the need for measures to improve healthcare, and an opportunity to share experience and knowledge is fundamental to quality improvement. This may be a challenge for countries such as Germany where general practices are characteristically single handed, and healthcare professionals in general practice have little chance of peer review and learning opportunities.10 Internet-based reporting systems which are open to the public allow experience to be shared and knowledge from a small area to reach a broader audience.
In this article we illustrate the development, structure and goals, and first results of a reporting system developed specifically for general practices in German-speaking countries.
METHODS
The incident reporting and learning system—Jeder Fehler Zaehlt (ie, “every error counts” (JFZ); www.jeder-fehler-zaehlt.de) is a voluntary and anonymous incident reporting system. JFZ is provided by an academic institution, the Institute for General Practice at the University of Frankfurt am Main, Germany. It was specifically developed for general practices in German-speaking countries. Initially, the reporting system was only directed towards general practitioners, so the strategy to increase awareness of the system focused mainly on them.
Development
The system has been modelled on other reporting systems such as the AIMS (Advanced Incident Management System, Australia), the Critical Incident Reporting System CIRSmedical (Switzerland) and the NRLS. Its method of reporting was first used in the Primary Care International Study of Medical Errors (PCISME). Among other aims, PCISME was designed to test an electronic method of collecting incident reports from general practitioners11 and was therefore regarded as suitable for our purposes. JFZ is a modified version of the PCISME reporting system and, following a pilot phase, was launched in September 2004.
Goals
The goals of the reporting and learning system are:
to collect and analyse reports on medical incidents in primary care in German-speaking countries and thus to contribute to an increase in patient safety in general practice;
to provide a platform that healthcare professionals in general practice can use for the publication and discussion of particular incidents, to learn from them and to share knowledge on how to improve patient safety in primary care.
Incidents are defined as anything that “was a threat to patient well-being and should not happen. I don’t want it to happen again”. This definition was adapted from the PCISME study1112 and includes near-misses and adverse events. It is comparable to the NRLS definition of “patient safety incident” (“any unintended or unexpected incidents which could have or did lead to harm for one or more patients”).
Structure
Box 1 Example of incident report submitted to Jeder Fehler Zaehlt and comments received
Note that the evaluation of the incident reflects the judgement of the reporting doctor.
Results management
Incident report (no. 117) by an anonymous doctor
What happened?
A 78-year-old woman made an appointment because of pain in her shoulder. Physiotherapy was ordered, as was a blood count and haemoglobin because she was looking pale. Blood was taken the next day. She was given no information on how she would be informed of the results.
What was the outcome?
The haemoglobin level was 8.5 g/dl. The result was seen by two doctors and the practice nurse who entered the result into the chart. The patient did not show up and was not informed. Three weeks later, her daughter asked for the result.
Likely reasons?
Laboratory test routine. Reliance on the doctor who ordered the test to act on the result, and on the patient to return to the surgery.
How could the incident have been prevented?
Probably by making an appointment. Then the practice staff would have noticed the absence of the patient.
Contributing factors
Organisation, task factors and patient.
Frequency of this error
First time.
Comments and recommendations by anonymous participants of the reporting system
In our surgery we inform all patients of their results the same day or the day after we receive them. They are called by the doctor in person who gives them the abnormal and normal results.
The patient’s name is entered into a telephone list and he or she is called up to three times. If the patient has still not been reached, a letter is sent asking him or her to return the call. (Anonymous)
When an appointment for a laboratory test is made, another is immediately made for a further visit or telephone consultation within 8 days of the test. (Anonymous)
We have the following proved procedure in place: the patient is advised to ask for the results and the practice calls the patient when results are abnormal and action is needed. In this way, we avoid the patient relying on the practice and the practice relying on the patient. (Anonymous)
Box 2 Example of incident report submitted to Jeder Fehler Zaehlt
Note that the evaluation of the incident reflects the judgement of the reporting doctor.
Weakness and dyspnoea
Incident report (no. 251) by an anonymous doctor
What happened?
Patient complained about weakness, a sharp pain in the left shoulder and dyspnoea. Separation from partner some weeks ago. Lives with new partner. A lot of pressure at work: sales representative. Auscultation of the lungs normal, ECG normal. Ordered radiograph of the lungs.
Five days later: radiograph—no abnormal findings. Patient asked for sick note. Granted for 2 days. Blood sample: blood count, liver enzymes, C-reactive protein. Results arrived in the evening: liver enzymes elevated, C-reactive protein moderately raised. No action on the result. Next morning patient called: condition had worsened. Admission to hospital specialising in pulmonology.
What was the outcome?
Pulmonary embolism and pneumonia. Oxygen saturation 70% at admission to hospital. Brother of patient suffered a pulmonary embolism at the age of 20. Discharge of the patient with anticoagulative medication.
Likely reasons?
(None offered)
How could the incident have been prevented?
Bear pulmonary embolism in mind. Communicate the urgency of a radiograph and immediate reconsultation. Assess oxygen saturation in case of shortness of breath. Assess D-dimers? Family history?
Contributing factors
Education and training, communication, work and environment factors.
Frequency of this error
First time.
Following consultations with experts in medical law, we chose a completely anonymous rather than a confidential system. For legal reasons this was the only way to completely protect the anonymity of patients, reporters and other persons involved. The reports are encoded and electronically transferred (SSL protocol) to a database on a secure server. After decryption the reports are then filed on a second server. No data are saved on the reporter (IP address), and any information that might enable those involved to be traced is deleted.
The system is freely accessible and no registration is required. The reporting form consists of six screen pages (see supplementary data “Screenshots of reporting system”). Four empty text boxes are located in the centre, and the event can be further described using the check boxes and drop-down menus with specified categories and data fields—for example, to name drugs (see table 1). The number of contributing factors is restricted to a maximum of three: one main and two secondary factors. Users of JFZ therefore have to choose the most relevant three. There are no compulsory data fields.
Feedback and publication of incident reports
Feedback is provided via the weekly and monthly publication of selected incident reports (literally translated as “error of the week”, “error of the month”) on the website. The selection for publication is based on the frequency, probability of reoccurrence, severity of outcome of the reported error type and typical circumstances of the incident. Users of JFZ can comment on the reports directly while online. The comments are found on the webpages assigned to the reports (see examples in boxes 1 and 2). Reports and discussions are published weekly in a widely read medical newspaper (Ärzte Zeitung). Every “error of the month” is analysed by the Institute for General Practice and published in two journals for general practitioners (covering about 90% of practices in Germany and Austria) (see example “error of the month” in supplementary material).
Data analysis
Two physicians (BH, JR), both experts in the field of patient safety and general practice, classified each report independently. When results varied, a consensus was found. The incident reports were analysed for underlying errors in accordance with the Institute of Medicine’s definition (“failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim”).13
The reports were categorised according to the International Taxonomy of Medical Errors in Primary Care (version August 2004)111214 with its four domains (error type, impact, contributing factors, prevention strategies). Each domain has two to four levels. Since several reports contained more than one item in one or more of the three domains error type, contributing factors and prevention strategies, the respective domain was classified up to three times rather than once.
The reporting system was approved by the ethics committee of the medical faculty of the Christian-Albrecht-University Kiel, where the project was originally conceived, and by the Independent State Centre for Data Protection Schleswig-Holstein.
RESULTS
Between September 2004 and January 2006, JFZ received 199 incident reports. Of these, 11 (5.5%) could not be classified because of missing or confusing information, or because we could not find any evidence of an error. Thus the results from 188 reports (94.5%) were analysed: female patients were involved in 44.7% and males in 38.3% of the reports (17% of reports did not specify), which is in line with the population distribution of German general practices.15 Most general practitioners (56.5%) were familiar with the patient. Of the classifiable reports, 125 (66.5%) came from Germany, 8 (4.3%) from Austria, one from Switzerland and 53 (28.2%) did not specify the country. According to those filing the reports, 130 incidents occurred for the first time, 29 occurred once a year, 16 occurred once a month, 5 occurred once a week and another 5 occurred daily (3 were not specified). Nearly three-quarters of the reports were classified as process errors (see table 2).
In the reports of treatment errors, 92 different drugs were identified; those most often involved were anticoagulants (21 times), vaccines (18), analgesics (17), antihypertensives (9), antibiotics (8) and antidiabetics (4).
The severity of outcome was subdivided into incidents not resulting in harm (category 2 “error, no harm”) (n = 68, 36.2%) and incidents resulting in harm (categories 3, 4 “error, harm or death”) (n = 78, 41.5%); 42 (22.3%) reports contained insufficient information to determine the extent of harm. For more details see fig 1 and box 3.
Box 3 Domain “severity of outcome”, International Taxonomy of Medical Errors in Primary Care
1 No error
2.1 An error occurred, but the error did not reach the patient
2.2 An error occurred that reached the patient, but did not cause the patient harm
2.3 An error occurred that reached the patient and required monitoring to confirm that it resulted in no harm to the patient and/or required intervention to preclude harm
3.1 An error occurred that may have contributed to or resulted in emotional harm to the patient
3.2 An error occurred that may have contributed to or resulted in temporary harm to the patient and required intervention
3.3 An error occurred that may have contributed to or resulted in temporary harm to the patient and required initial or prolonged hospitalisation
3.4 An error occurred that may have contributed to or resulted in permanent patient harm
3.5 An error occurred that required intervention necessary to sustain life
4 An error occurred that may have contributed to or resulted in the patient’s death
5 Error, unable to determine harm
In 101 reports the most frequent contributing factors were provider factors (“relying on computer”, “seeing what you expect to see”, “lack of attention to detail”) and in 25 reports these were patient factors (eg, “patient presenting with many health problems”). Other contributing factors occurred rarely (provider team, task, working conditions, organisation, physical environment, regulatory/payment system factors).
So far, users of JFZ have made more than 300 comments on the 74 published incident reports, mainly anonymously. During the first year the number of visits to the webpages rose from an average of 68 to about 120 a day. To our knowledge, JFZ is one of the largest publicly accessible sources of exemplary incident reports in primary care on the internet.
DISCUSSION
We have designed and implemented an anonymous and voluntary reporting system for medical errors in general practice. After 17 months there is a small, but steadily growing database of 199 incident reports which have been categorised according to an international classification system; the majority of errors were process errors, but knowledge and skills errors also account for a considerable proportion of the incidents.
Comparable systems exist, which collect incident reports from the primary care setting: the NRLS in England and Wales (which was set up in November 2003 and which also reports from other settings), CIRSmedicalGP in Switzerland16 (which was launched exclusively for general practitioners), and the non-governmental CIRSmedical Deutschland17 in Germany (which was set up in May 2005 and includes reports from other ambulatory practices) (see table 3).
Reporting rates
Compared with systems in other healthcare settings, the reporting frequency in ambulatory care remains low (see table 3). The reports from general practitioners in the NRLS represent only 0.5% of more than 80,000 reports from the entire National Health Service (NHS) in England and Wales.3
Why are reporting rates so low? General practices are small organisations (with alleged “low-risk” technologies) in a barely regulated environment. In comparison with staff in the inpatient sector, doctors in the outpatient setting (in Germany) undergo professional/peer reviews less frequently. An awareness of medical errors in their own clinical practices has only recently developed. In addition, the fear of being prosecuted or losing one’s reputation is substantial.18 These are important barriers to reporting, especially to an open system such as JFZ.
Types of errors and severity of outcome
We found that 72.9% of the reported errors were process errors. Compared with previous studies in primary care this is an unexpectedly low percentage.1112 However, as doctors were the target audience there was a bias towards reporting significant events that primarily involved their professional knowledge and decisions.
Reports by general practices to the NRLS contained a high proportion of severe outcomes (permanent or long-term harm and death, each about 2%),3 and in the present study, these were even more common, with 12.8% in similar categories (3.4 and 4).
One possible explanation is the worry caused by severe outcomes,16 or the misconception that events where harm does not occur are not worth reporting because “nothing severe happened”. Further, the definition of an error as used in this study (“threat to patient well-being”, “I don’t want it to happen again”) emphasises the emotional aspect in the motivation to report. Since incidents with fatal outcomes are more easily recalled19—or rather less easily forgotten—this definition could explain the high proportion of reports with fatal outcomes made to JFZ.
The method of data collection presented here exerts a strong bias towards remembered incidents.16 Reporting systems depend on the recognition and recollection of incidents by participants. In addition, information on causes and contributing factors relies on the judgement of the incident by the reporter. The subjective nature of reporting may be even more pronounced when there is a lack of established methods for analysing significant events and identifying medical errors; a systematic approach to risk management in this care setting is still in its infancy.
In the primary care setting, the discrimination between error and non-error sometimes presents difficulties.20 Hence, we used a simple definition of incidents that could easily be understood and remembered and would therefore not present a hindrance to reporting. In the PCISME study the same pragmatic approach resulted in a comparable number of serious reported outcomes. However, near-misses have also been reported to our reporting system, as they were to the PCISME.11
Limitations of voluntary incident reporting
Incident reporting systems act retrospectively and can only be “a window on the system”, as Vincent put it.21 Hence, they offer specific learning opportunities (depending on the quality of feedback), but cannot adequately reflect the safety of a system if they are not combined with other methods of risk management such as incident analysis, safety audit, and prospective methods of analysis.
Other studies on incident reporting in various healthcare settings suggest that other medical staff may report less reluctantly and more often than doctors.22–25 Hence, the results of this study are not considered as representative of general practice.
The perceived advantages of an incident reporting system are the relative safety of reporting (confidentiality, anonymity), comparative data analysis and the identification of common factors. However, one substantial drawback is that since we do not log IP addresses or other information on the reporter we cannot request further information. Therefore further investigation of the incident is impossible, which distinguishes incident reporting systems from other quality improvement approaches that analyse events in a systematic and detailed way, such as the significant event audit.2627 Another disadvantage is the dependence of data on the structure of reporting forms.28
Since the goal of reporting systems is the identification of risks in medical practice as a precondition for improvement in the safety of care, efforts should be directed at increasing the quality and completeness of information reported to the systems. In addition, healthcare professionals should be encouraged to report incidents covering the whole spectrum of care in general practice.
CONCLUSIONS
JFZ is an efficient incident reporting system in an early stage of development. Future efforts to add to the benefits of incident reporting will concentrate on increasing the utilisation of the system, broadening the spectrum of error types and improving the quality of information reported. Therefore, the following steps are planned:
inclusion of the whole practice team (assistants, receptionists): encouraging practice assistants to report and motivating doctors to allow them to do so;
improvement in feedback (data on frequent error types, tools for analysis and prevention, closed specialist user groups) and educational activities.
Acknowledgments
Many thanks to R Hamblin (2006/7 Harkness Fellow, Centre for Health Studies, Group Health Cooperative of Puget Sound), P Voelker (University of Washington) and P Elliott for proofreading the manuscript.
REFERENCES
Supplementary materials
web only appendices 17/4/307
Files in this Data Supplement:
Footnotes
Funding: The development of the system was self-funded. Since October 2005 a grant provided by the Federal Ministry of Health (project number 212A-43794-1/117) has enabled further work on the system.
Competing interests: None.
Ethics approval: The reporting system was approved by the ethics committee of the medical faculty of the Christian-Albrecht-University Kiel, where the project was originally conceived, and by the Independent State Centre for Data Protection Schleswig-Holstein.