Abstract
PURPOSE We wanted to explore test results management systems in family medicine offices and to delineate the components of quality in results management.
METHODS Using a multimethod protocol, we intensively studied 4 purposefully chosen family medicine offices using observations, interviews, and surveys. Data analysis consisted of iterative qualitative analysis, descriptive frequencies, and individual case studies, followed by a comparative case analysis. We assessed the quality of results management at each practice by both the presence of and adherence to systemwide practices for each results management step, as well as outcomes from chart reviews, patient surveys, and interview and observation notes.
RESULTS We found variability between offices in how they performed the tasks for each of the specific steps of results management. No office consistently had or adhered to office-wide results management practices, and only 2 offices had written protocols or procedures for any results management steps. Whereas most patients surveyed acknowledged receiving their test results (87% to 100%), a far smaller proportion of patient charts documented patient notification (58% to 85%), clinician response to the result (47% to 84%), and follow-up for abnormal results (28% to 55%). We found 2 themes that emerged as factors of importance in assessing test results management quality: safety awareness—a leadership focus and communication that occurs around quality and safety, teamwork in the office, and the presence of appropriate policies and procedures; and technological adoption—the presence of an electronic health record, digital connections between the office and testing facilities, use of technology to facilitate patient communication, and the presence of forcing functions (built-in safeguards and requirements).
CONCLUSION Understanding the components of safety awareness and technological adoption can assist family medicine offices in evaluating their own results management processes and help them design systems that can lead to higher quality care.
- Delivery of health care
- quality of care
- patient safety
- medical errors, office management, qualitative research
INTRODUCTION
Primary care physicians order tests for a considerable number of patients. Recent estimates are that family physicians and general internists order laboratory tests in 29% and 38% of patient visits and imaging studies in 10% and 12%, respectively.1 These tests serve multiple purposes—some are for screening, others are diagnostic, and still others are used to manage and monitor medications and chronic health problems. Although some tests are performed in physicians’ offices while patients wait, most are sent to reference laboratories, hospitals, and outside facilities.
For more than a decade, research has documented that primary care physicians are concerned that their systems for managing test results are unsatisfactory.2–5 In 3 primary care patient safety studies, 15% to 54% of all errors reported by physicians were related to the testing process.6–8 There are multiple steps involved in the management of test results.1,9 These steps begin when offices track orders and the return of results to the physician’s office from the outside testing facility. If this step is not done automatically by means of an electronic health record (EHR), the results must then be given to the clinician who responds to the result. The patient is notified and followed up, and documentation is made in the medical record. A recent study of testing process errors reported by family physicians and their staff found that errors occurred in all the multiple results management steps.9 Serious harm can and has befallen patients by errors in results management.9–11 Although all results management steps are important and interrelated, errors in patient notification were predictive of more patient adverse events.9
Previous attempts to study office management of test results have depended heavily on surveys2,5,12 and reports of errors by physicians and staff,6–9 whereas chart reviews4 and interviews3,4 are used less often. Even though each of these techniques gives some insight into the problems faced in performing these tasks, none fully describes the complexity of the results management process. Fewer studies actually report on what works well.4 Using a multimethod protocol of observations, interviews, and surveys, as well as a comparative case design analysis, we intensively studied 4 purposefully chosen family medicine offices to determine and describe components of results management qualilty.
METHODS
To gather the maximum amount of data with minimal interference to patient care and productivity, a family physician (N.C.E.) and a human factors graduate student (T.R.M.) performed site visits at each office for 2 to 4 days and collected other data before and after each visit. This study received approval from the University of Cincinnati and Wright State University institutional review boards.
Participant Selection
We purposefully selected physcians’ offices within the southwest Ohio region to provide variation in geographic location (rural, suburban, urban), practice size, patient insurance status, technology level (EHR, no EHR), and residency program (program, no program). Physicians at 6 offices initially expressed interest, and 4 ultimately agreed to participate. All offices had some degree of affiliation with a larger health care system or a hospital, but each functioned independently. As our research was an exploratory study to describe potential results management quality determinants, we had no means of knowing an office’s results management level of quality before the study.
All office staff were observed and participated in think-aloud interviews. Key informants were identified from staff observations at each office and selected for individual interviews. Individuals interviewed included those who had interest or additional roles in test results management or leadership in the office or who were opinion leaders. There were a total of 17 key informant interviews at the 4 offices.
All adult patients (and parents of child patients) for whom laboratory tests were ordered during the site visit and for a 1- to 4-week period afterward (depending on office size) were asked to participate in a future mail-based survey. Medical assistants handed these patients a 1-page form describing the survey, and those who wished to participate supplied their name and address.
Data Collection
An office information form and Testing Process Survey questionnaire (adapted from the American Academy of Family Physicians National Research Network) 3,9 were completed by the office manager at each office. These forms elicited demographic data about the office and how test results are officially managed. A brief, paper, networking questionnaire was distributed to all staff and physicians, which asked to whom they turned for advice and assistance about results management issues.
At each office, we reviewed 25 random charts that contained laboratory or imaging results in the last 12 months. This review assessed documentation of test orders, results, clinician response, patient notification, and follow-up of abnormal results. Participating patients for whom laboratory testing was ordered during and after the site visits received a mailed, written questionnaire (3 mailings done) about their experience getting the test taken and the results back.
We observed staff and physicians performing their tasks related to the testing process in hallways, nursing stations, the laboratory area, medical records, and other office areas. Occasionally we asked participants to “tell us what you are doing” or to give opinions, ideas, and concerns about the testing process. We also collected written documents from each office, including billing sheets, order forms, interoffice communication forms, patient communication forms, and testing process protocols and procedures. During semistructured interviews with key informants in the offices, we inquired about the testing process, participants’ roles in it, and experiences of errors and safety. Using field notes, we documented how each office responded to an individual written report prepared for them.
Preliminary Data Analysis
During each site visit we met several times each day to discuss findings and to develop process flow diagrams that graphically displayed the detailed tasks each office member performed to manage test results. At the end of each day, we updated the process flow diagrams, which then guided observations the next day. At the end of the each site visit, we put the flow diagrams into a final form and summarized the observation notes for each office.
Quantitative data were entered into a database, and simple descriptive frequencies were summarized for each office.
Each interview was audio-recorded and reviewed by the interviewer (T.R.M.), who made extensive notes on each interview and transcribed selected sections verbatim. The other investigators then reviewed and discussed the notes, looking for important themes and stories about test results management.
Initially we analyzed each site as an independent case study. All data sources were read, reviewed, and discussed by all the researchers. From this discussion, each step in the testing process was described in detail, and safety threats and strengths for each office were delineated and summarized in individual reports, which we then presented to each office.
Data Analysis
The notes and selected quotes from the interviews, the observation notes, and the individual office case studies were entered into the qualitative software program NVivo 2.0 (QSR International Pty Ltc, Doncaster, Victoria, Australia). We focused the analysis on finding examples, descriptions, and comments about each step in results management, as well as safety threats and strengths. Coding was performed by the lead author (N.C.E.) and reviewed by the interviewer (T.R.M.). After discussion and agreement on coding categories, all data were reviewed and coded a second time. Then all the data sources described above, including the quantitative data, were reviewed, along with the summary categories from the qualitative coding. During this review, specific details about the management of each step in results management were collated, and factors of importance for test results management safety and quality for these offices were determined. To assist with thematic formation, existing recommendations from the literature1,4,5,9,12–29 about test results management were compared with the findings from the offices.
RESULTS
Characteristics of the participating family medicine offices are displayed in Table 1⇓.
Performance of Test Results Management Steps
We identified multiple ways the main steps of test results management are performed, beginning with tracking orders through follow-up of test results needing further care after the patient is initially notified. In Figure 1⇓, we list all the different ways these tasks are actually performed at the 4 family medicine offices. For no step did every office perform the results management process the same way. This variability was occasionally minimal (office A had physicians telephone patients with critical results, whereas office B had nurses make the telephone call), but for some steps, like tracking orders, completely different systems existed at each office (or there were no systems at all).
Test Results Management Quality Data
Because there are no reference standards in the medical literature for how best to manage test results, we used multiple sources of data to assess quality. Table 2⇓ outlines some of the specific data indicators for each office. We looked for written protocols for results management steps, which are an indication of attempts to standardize office practices. We also assessed the standardization of results management steps actually practiced office-wide. Office D had only 1 such practice that was routinely followed (the return of results to the office); for the other 3 offices, standardized practices existed for getting the results to the office and to the physician. Only office B, which had an EHR, had an office-wide system for getting physician signatures on all results. Offices A and C had standardized tracking systems. For example, tracking system at office A consisted of attaching copies of order forms to charts, which were then placed in a separate holding area in the office. Tracking review was performed by a medical assistant assigned to a support position for 1 week at a time. Office B, on the other hand, had no tracking system. Orders entered in the EHR were not automatically linked to results, nor did any staff routinely check for pending orders without results. None of the offices had a standardized way to notify patients of normal test results. Depending on the physicians or medical assistants, they mailed letters or copies of results, made telephone calls, had staff make telephone calls, waited for patients to make follow-up visits, or did not notify patients at all.
The patient questionnaire elicited information about notification of test results. Generally speaking, all the patient responses were relatively high, with 87.5% to 100% of respondents acknowledging they had received their test results. A smaller percentage actually received information, advice, or instructions about the results. The chart review assessed documentation of clinician response to results, patient notification, and follow-up instructions. As with the patient survey, results management steps were inconsistent across the offices, with all offices having some areas of strengths and weaknesses.
When the individual office findings were presented to the offices, offices B and C met with the study team to review the findings, office A acknowledged the findings in writing, and office D never commented on the report.
Assessment of Test Results Management Quality
No single office emerged as having notably higher quality test results management indicators, although office D had fewer standardized practices and less interest in learning about their results management assessment. We found a similar story from our interview and observation data. As we analyzed these qualitative data, we found that 2 themes, safety awareness and technological adoption, emerged as important for test results management safety. Table 3⇓ displays our observations of how the 4 offices used the 8 factors associated with these themes in test results management.
Safety awareness is the observed behaviors and expressed beliefs of office staff that show a commitment to safety and include a leadership focus on quality and safety, communication that occurs around quality and safety, teamwork in the office, and the presence of appropriate policies and procedures. For example, in office A, the leadership (medical director, office manager) frequently talked about safety and quality, the staff remarked on their commitment, and quality initiatives were discussed at team meetings. We observed functional teamwork in several areas of the office, with polite and appropriate staff and physician interactions. There were written policies and procedures for several testing process steps. By comparison, in office B, leadership only occasionally mentioned safety and quality, and staff and other physicians did not remark on their commitment. There were no team meetings on quality, but we did observe functional teamwork in some areas of the office. The office was small and staff showed minimal support for each other. There were no written protocols or procedures related to the testing process.
Adoption of technology is the incorporation and appropriate use of technology in an office to advance results management, including the presence of an EHR, digital connections between the office and testing facilities, use of technology to facilitate patient communication, and the presence of forcing functions (built-in safeguards and requirements) in the technology. Using the same offices for examples, office A had no EHR, but they did connect on 1 computer terminal to the main referral hospital laboratory (for order entry) and on another terminal for radiology results retrieval. Laboratory results were returned by means of a dedicated printer digitally connected to the laboratory. They did not electronically communicate with patients or generate patient communications, and there was no electronic management of results management steps. Office B did have an EHR, and although it automatically received test results from the main laboratory and radiology facility, it could not electronically send orders to these facilities. The EHR generated paper communication documents for patients. Except for the return of results, all test results management steps available in the EHR had to be regularly monitored and performed by staff or physicians.
Interactions Between Performance of Test Results Management Steps and Quality
Office A and B can also serve as an example of how the quality of performance of test results management steps and their safety awareness and technology adoption interact. Although the tracking system at office A did have some functional and practical problems, its existence and the office’s commitment to its success demonstrated safety awareness. Even though they were committed to safety, without more advanced technology, their system remained cumbersome and open to frequent errors. In contrast, office B had an EHR, but the office did not use it or any other system to track their test orders. Office B’s staff and physicians expressed the belief that electronic-reported results do not get lost. They ignored, however, that several tasks still depend on individuals (eg, the patient, medical assistant, laboratory and radiology staff, etc) and that problems can occur. Even though they adopted an EHR, they did not possess a safety awareness that allowed them to look for and find weaknesses in their test tracking. Both these offices wanted to provide good results management, but they were limited from achieving the highest quality by the innate complexities of the results management steps.
DISCUSSION
Management of test results is a complex process that influences safety and quality in the primary care setting. Although problems and errors in the process have been previously described,3,5,9 this study is among those few to explore tests results management using multiple methods. At these 4 family medicine offices, we found that an office’s level of safety awareness and appropriate adoption of technology to be important factors in high-quality results management. Achieving such quality does not necessarily require high performance of every factor. Good quality can be achieved in an office without an EHR, but without some use of technology, many tasks require more staff, are more cumbersome, and have higher failure rates.3 Even so, an EHR alone never guarantees high quality.30
The complexity of results management makes clear that the steps, their tasks, and the staff involved are interconnected and that multiple components of the process must be evaluated to assess for quality.31 Despite a desire by all the offices to provide high-quality results management, none of them emerged as an exemplar office. Most of their standardized practices revolved around results management steps that dealt with communication with the testing facility (return of results to the office) and getting the results to the physician. Although these steps are necessary to assure that test results arrive at the office, the lack of standardization around patient notification and follow-up is disturbing. As these steps occur farther down the process (Figure 1⇑), there will be less chance for staff and patients to mitigate or recover from errors32,33 before harm or adverse events occur.9
The work culture of an office is important, and in the hospital setting, the importance of assessing for safety has been well established.34 Recently, safety culture in ambulatory care has also been studied.27–29 Although our study assessed the performance of some common components of a safety culture as they relate to results management (leadership, teamwork, communication, and policy and procedures), we did not fully assess the offices’ safety culture. Our observations and interviews, however, allowed us to assess each office’s use and understanding of many safety culture components, what we called an awareness of safety. This awareness varied among the offices. Even though most of the offices performed most safety awareness factors fairly well (Table 3⇑), office A showed strong leadership in quality control, whereas office D lacked many of the fundamentals of safety awareness, findings that were consistent with the data on the quality of results management. Further research comparing validated ambulatory safety culture surveys in family medicine offices with the actual performance of results management processes would be beneficial in finding additional ways to assess safety and quality.
Technology, specifically the incorporation of the EHR by primary care physicians, has been considered by some to be necessary to ensure high quality and safety in clinical care.30,35–39 Unfortunately, the ability to achieve such quality and safety with the EHR in real practice has fallen short of the promise.30,35,37–39 Our study confirms this finding. Although only office B had a fully implemented EHR, the office was not performing at the highest level of results management quality and safety, in part because the commercial EHR product was not designed primarily for these purposes.37,38 The office’s lack of strong safety awareness then led the staff to a sense of complacency. While enjoying the improved electronic return of results from the main laboratory and radiology facilities, they did not look for the other tasks and steps where errors and problems still occur.
All the offices used technology to some extent for test results management, but those without an EHR found that tracking, follow-up, and documentation required more staff time than often was available, findings consistent with other research focusing on the testing process in family medicine.3 These offices often found that their results management steps were not maintained at a high level when staff were pulled away for other purposes, that training was minimal, and that shortcuts were taken to save time. Dependence on staff for tracking and follow-up often meant dependence on human memory and double-checking,40 areas usually improved by technology.
There are limitations to this study. For financial and logistic reasons, we studied only 4 offices, and they were all in a single geographic region, limiting potential generalizability. Even though we purposefully chose offices with a range of characteristics, that they agreed to participate may indicate they were different from offices that chose not to participate. We collected data on processes, so we do not know whether the different processes we observed were correlated with different clinical outcomes. As an exploratory study, not all factors related to the quality of test results management may have been delineated, and we were unable to assess relationships between office characteristics, such as size and location and the quality of results management. Further research should help answer these important questions.
Family medicine offices today are asking for best practices and guidelines, not only for the best drug to prescribe, but also for the best systems or processes in which to practice.41–43 Unfortunately, these common, everyday processes, including test results management, are often lacking. As a result, even in this small sample of family medicine offices, we found great variation in how these tasks are performed, both by offices and by individuals within an office. We did find that 2 themes are important in assessing a high-quality results management process: an awareness of safety (exemplified by strong leadership, open communication, functional teams, and appropriate policies and procedures), and the adoption of appropriate technology. Although it is probably both impossible and unwise to assume there will ever be only 1 best way to manage test results in primary care, an increased awareness of quality factors and successful examples of results management13,26,44 would go a long way toward improving quality and safety in the office setting.
Footnotes
-
Conflicts of interest: none reported.
-
Funding support: This research is supported by a grant from the Agency for Healthcare Research and Quality: 1 K08 HS013914-01A2.
- Received for publication May 14, 2008.
- Revision received October 8, 2008.
- Accepted for publication October 13, 2008.
- © 2009 Annals of Family Medicine, Inc.