Abstract
PURPOSE Primary care practices use different approaches in their quest for high-quality care. Previous work in the Practice Partner Research Network (PPRNet) found that improved outcomes are associated with strategies to prioritize performance, involve staff, redesign elements of the delivery system, make patients active partners in guideline adherence, and use tools embedded in the electronic medical record. The aim of this study was to examine variations in the adoption of improvements among sites achieving the best outcomes.
METHODS This study used an observational case study design. A practice-level measure of adherence to clinical guidelines was used to identify the highest performing practices in a network of internal and family medicine practices participating in a national demonstration project. We analyzed qualitative and quantitative information derived from project documents, field notes, and evaluation questionnaires to develop and compare case studies.
RESULTS Nine cases are described. All use many of the same improvement strategies. Differences in the way improvements are organized define 3 distinct archetypes: the Technophiles, the Motivated Team, and the Care Enterprise. There is no single approach that explains the superior performance of high-performing practices, though each has adopted variations of PPRNet’s improvement model.
CONCLUSIONS Practices will vary in their path to high-quality care. The archetypes could prove to be a useful guide to other practices selecting an overall quality improvement approach.
INTRODUCTION
Given widespread evidence of problems with quality in health care,1–3 researchers and practitioners alike are looking to map out successful paths to improvement. Research conducted in large medical groups and smaller independent practices finds aspects of organizational culture or motivation to be important.4–7 Prescriptions for quality also frequently call for information technology,1,8–10 yet trials of electronic medical record (EMR) tools have met with mixed success.11–15
In response to the need for more research into effective ways to improve quality in the health care system, the Agency for Healthcare Research and Quality funded 21 Partnerships for Quality demonstration projects.16 Among these was Accelerating the Translation of Research into Practice in a Practice-Based Research Network (the A-TRIP study) conducted by the Practice Partner Research Network (PPRNet). A-TRIP joins researchers, an EMR vendor, and primary care practices in an effort to increase adherence to clinical practice guidelines in 8 clinical areas: heart disease and stroke, diabetes mellitus, cancer screening, adult immunizations, respiratory disease/infectious disease, mental health and substance abuse, nutrition and obesity, and medication prescribing in the elderly. The practices use the same EMR, elected to join a research network focused on quality improvement, and agreed to participate in the A-TRIP demonstration project. They transmit quarterly data extracts from their EMR and receive, in return, quarterly performance reports for the indicators being studied. Additionally, the practices can elect to receive site visits or attend annual network meetings, both of which are designed to assist the practices with improvement efforts.
In an earlier study, PPRNet identified strategies that were significantly related to higher achievement in clinical process and patient outcomes in small practices and grouped them into 5 categories: activities among practice leaders that (1) prioritize performance and (2) involve all staff; and organizational changes to (3) redesign delivery systems, (4) activate patients, and (5) use EMR tools.17 Strategies in the first 2 categories enable organizational change and in the latter 3 define specific changes in the delivery of care. The combined set of strategies were packaged as the PPRNet TRIP (Translation of Research into Practice) improvement model and are promoted in A-TRIP practices to guide improvement efforts. Each practice is encouraged to experiment to find the combination of strategies that works best in their setting.18,19
Feedback from the practices indicated that some wanted a more-organic picture. They wondered whether there were specific combinations of strategies that would be more effective in particular situations. They wanted to know what the most successful practices were doing. The study reported in this article was conducted in response to these questions. The aim was to examine the adoption of the PPRNet TRIP improvement model among sites with the best outcomes. Archetypes have been identified that could suggest a path to improvement.
METHODS
The design was observational case study research including retrospective and prospective data. How or why questions that focus on contemporary events over which a researcher has little control are ideal for case study research.20
The A-TRIP project continuously enrolled practices from October 2002 through September 2005, ultimately including 101 primary care practices. The medical director or owner of each practice consented to participate in the A-TRIP study on behalf of the whole practice. A-TRIP was approved by the Institutional Review Board at the Medical University of South Carolina. A-TRIP process evaluation was approved by the Institutional Review Board at the University of Southern California.
A composite quality score, called the Summary Quality Index (SQUID) was calculated quarterly for each practice participating in the A-TRIP project.21 The practice-level SQUID is the average percentage of adherence to all applicable clinical guideline indicators among adult patients eligible for at least 1 of the indicators.22 Altogether, 31 process and 5 patient outcome indicators were included. The 10 practices showing the greatest adherence to the clinical guidelines as indicated by their SQUID at 2 different time points were selected as cases for further study. The 2 time points represented the first 2 quarters after the intervention midpoint, which allowed for an adequate pool of practices active for a year or more, as well as sufficient time to study the selected case practices before the project’s end. Using 2 points in time for selection allowed us to evaluate trends and ensure that a practice was not selected whose high performance at 1 point in time could be explained by a data anomaly.
We combined qualitative and quantitative methods and structured and flexible approaches to data collection to maximize case completeness and validity.20,23 We reviewed every available source of data from the A-TRIP project and extracted pertinent information. One data set, created in SPSS 11.5 (SPSS Inc, Chicago, Ill), included quantitative information about adoption of the PPRNet TRIP improvement model. The data came from a survey completed by the medical director of A-TRIP practices, as well as ratings based on structured observations documented in site visitors’ field notes. A second data set, created in NVIVO 2.0 (QSR International, Australia), included qualitative information about each case practice gleaned from comments made by clinicians and staff in each practice, on-site visit evaluations, e-mail follow-up between researchers and the practice, listserv messages posted by practice physicians, best practice presentations made by case study sites at the project’s annual meetings, and notes from key informant interviews with members of the case study sites. Data were collected from different participants and provide multiple perspectives. To minimize reporting bias, we did not explicitly state the intent to develop case studies from our observations until we were at the stage of confirming our findings.24,25
A conceptual framework is useful in case study research to guide data collection and analysis.20 In addition to general practice characteristics, we specifically sought information to describe how case practices were adhering to the PPRNet TRIP improvement model. The framework allowed us to make cross-case comparisons.26 Information in the NVIVO data set was coded and written up as similarly structured individual case summaries. These summaries were reviewed by site visitors and discussed by the research team. The within-case analyses were conducted first to maximize validity.24 Cross-case comparisons were then conducted, and additional information was sought if needed. Case-ordered matrices of model adoption were constructed to compare approaches, and qualitative themes were compared among cases to identify case types. These types and their distinguishing features were discussed among the research team with an aim to identify any inconsistencies or alternative explanations. The resulting case practice types, or archetypes, are designed to serve as models for other practices with similar features. In describing the archetypes, differentiating characteristics to which other practices might relate have been emphasized to increase personal meaning or relevance.27 The report was shared with case practice sites and presented at the final project network meeting to validate findings.* Additional details of the study methods can be found in the Supplemental Appendix, which is available online at http://www.annfammed.org/cgi/content/full/5/3/233/DC1.
RESULTS
Selecting the top 10 practices at 2 time points midway into the intervention (January and April 2005), resulted in a pool of 11 medical offices. When we compared these case practices with the 89 other practices in the A-TRIP study using data available for April 2005, we found the case practices had significantly higher SQUID values (using t tests: 54.5% ± 4.9% vs 33.0%±8.0%; P <.001). The patient population may have been slightly more complicated among case practices compared with the other practices. For example, the mean age among patients in case practices was higher (albeit not significantly) than among patients in other practices (52.8 ± 6.8 years vs 48.5 ± 5.4 years; P = .26), and the number of eligible quality measures per patient (of a possible 37) was significantly higher in case practices compared with other practices (13.2 ± 2.4 vs 11.1±1.3; P = .025).
Three of the 11 practices were semi-independent offices within 1 medical group and were combined for qualitative study, yielding 9 cases, or practices, altogether. From these practices, 3 main approaches to improving guideline adherence emerged: practices that relied mostly on high-end use of EMR tools (the Technophiles), practices that focused first on engaging staff and clinicians and experimented with a variety of improvement strategies (the Motivated Team), and practices that resembled motivated teams but also organized improvements with focused care management (the Care Enterprise). Information about each of the practices is summarized in Table 1⇓ by archetype. The practice characteristics that are provided do not vary predictably by archetype.
Common Features
Case practices shared several traits that are likely to have contributed to their high performance. The practices were older and well established and were in no apparent financial distress. The newest practice was started by a physician who had a successful history of opening and expanding medical offices. Additionally, a leader prioritized performance in each practice at baseline, and regardless of whether individuals had reservations about improvements, they were able to work together as a practice to support their leader’s vision of excellence. To guide their efforts, most practices used the A-TRIP practice performance reports, as well as corresponding patient-level reports that function as a registry. The practices were realistic about performance measurement in the understanding that the ultimate goal was providing the best care for each individual patient. The practices also worked in close partnership with their patients. They used care plans and patient education to ensure that patients played an active role in guideline adherence.
Variations in adoption of PPRNet’s improvement model are depicted in Table 2⇓. Technophiles used a smaller variety of strategies than the other archetypes. As a group, the case practices adopted more strategies than their A-TRIP peers.
The Technophiles Archetype
All practices in the A-TRIP study used an EMR. Five of the high-performing case practices were distinguished by their quick adoption and consistent use of EMR tools.
The practices in this archetype developed problem-based templates that guide clinicians and staff to perform and document routine tasks consistently. A template might guide a nurse to record blood pressure and prompt a screening assessment of alcohol use. A template could also guide a clinician through routine management of a particular chronic condition. The template could import the patient’s last laboratory results or blood pressure, temperature, pulse rate, and respiratory rate, and include reminders about care recommended in new guidelines, as well as links to patient education. Improvements were seen in all 5 Technophile practices in both process and control measures: after 36 months, for example, the mean percentage of patients with diabetes who had a glycosylated hemoglobin (HbA1c) test in the past 6 months increased from 66.1% (SD 8.1) to 78.6% (SD 4.7), and the mean percentage of patients whose low-density lipoprotein cholesterol was <100 mg/dL increased from to 43.3% (SD 6.0) to 65.0% (SD 5.9). The mean percentage of patients with hypertension whose last blood pressure measurement was <140/90 mm Hg increased from 57.2% (SD 12.2) to 72.2% (SD 4.8).
The Technophiles used interfaces that electronically load laboratory results into the EMR. They took advantage of features that automate the physicians’ review of test results. They developed personalized patient letters that provided tests results and education. They established protocols and assigned responsibility to run regular recalls for care, notifying patients that visits or tests were needed, and they used their scheduling software and reminders to make sure patients got follow-up care when they needed it for conditions such as diabetes.
The clinicians in this archetype made good use of internal messaging features in the EMR. Messages were sent by physicians to themselves to remind them to do follow-up items. Messages sent between staff and clinicians made care coordination more efficient. The best practices in this archetype also used consistent diagnostic nomenclature, which made it easier for them to query their EMR data using the built-in EMR query functions to identify subgroups of patients needing follow-up care.
At the core of each of the practices in this archetype was a physician who was driven to achieve both good care and efficiency. This physician was dedicated to finding new ways to use the EMR, and tended to speak in terms of the time it takes to get things done. Some have described themselves, and have been described, as compulsive. These physicians were innovators and could modify the EMR quickly and easily to include reminders and links that automate care decisions, coordination, and documentation. These physicians also stimulated practicewide improvement by their role as a change champion. One physician explained at a network meeting that when she wants to implement a change in her practice, she sets up a tool in the EMR so that there is no faster or easier way to do the job. Her innovation made it easy to convince other clinicians and staff to adopt the change. An innovator in another practice customized all the encounter templates for medical conditions that might require aspirin prophylaxis. She was able to design this customization during a practice meeting as others were discussing ways to increase aspirin prescribing. She then explained to her practice team how to use the new codes. The practice saw how it worked and decided to adopt this strategy. Their solution to low prescribing rates for aspirin therapy was implemented in a matter of minutes.
The Motivated Team Archetype
All high-performing case practices described in this article participated in biannual site visits and/or annual network meetings to assist them with their change efforts. Two high-performing case practices are distinguished by the additional effort they put into motivating and enabling their staff to play important roles in their improvement efforts. In the larger practice, this focus on engagement extended to clinicians as well. Multiple clinicians frequently deliver care in different ways and may need to be convinced to make changes to conform to practicewide efforts for quality improvement. The motivation in this practice was enhanced by bonuses from a local pay-for-performance program.
The practices in this archetype used multiple strategies to get everyone involved. A lead physician promoted the importance of new activities in conversations and behavior, which staff feedback indicated was a critical first step. Further, because the doctors are usually viewed as the person in charge, this step gave staff employees permission to move forward. The practice leaders in this archetype shared the A-TRIP performance feedback with their staff and ensured that staff members had training about guidelines so that they were aware of the goals behind new tasks. The leaders held practice meetings, in addition to site visits, to check progress and make plans.
The practice staff worked with the clinicians to select indicators they wish to improve and designed activities to achieve improvement. As an example, nursing staff in one practice checked to determine whether health maintenance items were up-to-date and scheduled patients for services, if needed. Mammography rates in the past 2 years for women aged 40 years and older rose from 77.2% to 88.4% in 12 months and remained high, at 89.5%, after another 2 years. Staff in these practices set the tone for visits and saved their doctors some time by providing information about routine care that was due. One licensed vocational nurse said, “By bringing up health maintenance goals before the doctor comes in, patients are prepared for services or discussion.”
The higher-performing practice in this archetype devoted an additional half-day each quarter to making improvements. The office was closed, and all practice employees gathered for meetings and concentrated activity, such as reviewing patient-level reports (a form of registry generated by the A-TRIP project). The physician-owners of the practice emphasized that improving quality is as important as anything they do. Among their experiments was testing a staff incentive plan using graduated individual retirement account contributions based on varying levels of improvement in the practice’s quality indicators.
Evaluation data indicated that the staff in these practices responded to their leaders’ efforts to include and motivate them. Staff feedback to their leaders, as well as anonymous surveys for the project, confirmed that the staff valued quality, recognized the importance of their expanded role to their practice, and appreciated the additional training and tools. For example, a medical assistant explained that her motivation to take on extra work to deliver higher-quality care comes from the knowledge that her effort was in the best interest of the patients. Another practice’s medical assistant said that from her participation in the A-TRIP project, she learned she can be more helpful. A receptionist said, “I like education and being informed … checklists, specific goals, and deadlines motivate me.”
In comparison with the Technophiles, the practices in this archetype were more likely to express mixed feelings about their EMR software. In the beginning, individuals were pessimistic about adopting particular EMR tools, and many needed EMR skill development.
The Care Enterprise Archetype
As part of their solution to improving care, 2 high-performing practices took a business approach that was influenced by customer service and risk management. They organized special service lines in the form of focused care management clinics, hiring corresponding staff as needed (eg, certified diabetes educators or nurse-practitioners focusing on women’s health). The problem-focused clinics were designed to provide comprehensive, competitive, guideline-adherent care that was convenient for the patient and that ensured the doctor has done everything possible to manage care appropriately. One physician explained the personal value of his approach: “The ability to reduce risk of harm to my patients and simultaneously reduce risk of litigation is an important benefit.”
The care enterprises could be considered a special form of motivated teams. The leading physicians in these practices delegated care management responsibilities to their staff and provided regular supervision, such as weekly care coordination meetings between nurses and physicians. They excelled in redesigning delivery systems to support their special service lines and their general practice. The practices conducted point-of-care tests and used an in-house laboratory so that test results were available during the visit. Standing orders for routine monitoring tests and immunizations for designated populations were used to delegate components of the care management process. One practice abandoned annual physical visits with patients, instead scheduling intake visits with nurses alone and shorter follow-up visits with the physician. The practice arranged additional follow-up in nurse blood pressure or anticoagulation clinics, and in nurse-practitioner women’s health or diabetes foot clinics, as well as physician medical follow-up based on practice protocols. Staff tracked and enforced the condition-specific schedules.
The practices in this archetype also incorporated a population approach to quality management. In one practice, staff were delegated to use EMR queries to identify patients with specific needs. They conducted outreach to patients when guidelines were not met. These efforts contributed to improvements; for instance, the percentage of patients aged 65 years and older or with conditions requiring pneumococcal vaccines who were immunized in the past year rose from 36.9% to 62.5% in the first year and continued to rise to 87.6% after 3 years.
The 2 practices in this archetype implemented changes in their settings quickly. They did not appear to need to negotiate for investment within their team. The practices’ improvement efforts had a greater focus on A-TRIP indicators related to diabetes and heart disease than other areas.
DISCUSSION
Improved information systems are believed to be the key to unlocking the health care system’s potential for higher quality care. The question remains: How can this technology be implemented in the context of day-to-day practice, particularly in busy primary care settings? The A-TRIP project conducted by PPRNet is investigating ways to stimulate improvements in guideline adherence in practices that have already invested in an EMR. It has become eminently clear that the EMR or decision support alone is insufficient to produce high-quality care.14,15,28,29 The EMR is a useful tool in a practice’s improvement efforts, yet its role in improvements depends on other characteristics of the practice.
In this study, we used retrospective and prospective data to analyze the path to high-quality care in the 9 highest achievers midway into an improvement project. We found these could be defined by 3 archetypes.
Not surprisingly, in this practice-based research network that emphasized EMRs, the Technophile archetype describes the largest number of high-achieving practices, supporting the important role ascribed to information tools for achieving quality. The approach to excellence exemplified by this group also fits the suggestion of other researchers that point-of-care reminders and opinion leaders are the best solution for lapse errors (such as failing to deliver care recommended in clinical practice guidelines).9 In these practices, the ability to use the EMR was facilitated by having an in-house computer-loving physician. This physician, usually one of the practice owners, designed modifications and fine-tuned the EMR software to provide the practice with the specific reminders and documentation they needed. This EMR expert was also a change champion and led the practice to skilled use of EMR tools to deliver and document quality care. The Technophiles achieved high performance with less time and energy devoted to organizing people and delivery processes in the practice.
The existence of 2 other archetypes suggests a practice does not have to start out technically focused to be among the best. These 2 archetypes may be more easily emulated examples for late adopters of health information technology. These practices used an EMR, but drove their quest for quality with other approaches. One group was oriented toward making the most of its human capital; the other used teams, along with a service-oriented model of developing focused care management clinics, to organize improvements. The Motivated Team archetype looked at new ways of using existing resources to improve quality. Because staff and clinicians were taking on new roles, improvement strategies addressed communication, motivation, and competence. The Care Enterprise archetype selected clinical areas of focus, brought in staff, if needed, and designed award-winning care systems. Both experimented with multiple changes to roles, delivery systems, and patient activation. Secondary to these approaches, and with varying degrees of comfort, both archetypes tested, accepted, and adopted EMR tools. As did the Technophiles, these practices also had change champions promoting the improvement agenda and encouraging participation; the difference is they encouraged a wider variety of strategies to meet improvement goals. Others have noted the pivotal role of a change champion in improvement efforts.30–32 The presence of an effective champion in each of these archetypes is likely an important component of their success.
The A-TRIP intervention promoted best practices within the network. Not surprisingly, the top performers also ventured to learn from each other and evolved to resemble each other as they adopted the other’s improvement strategies. We believe the Technophiles served as role models to the other archetypes, demonstrating that EMR tools were both efficient and feasible. The Motivated Teams and the Care Enterprises learned about strategies from the Technophiles and grew in their willingness to try them in their own practices. Technophiles decided in turn to add group activities to address planning and motivation when they heard staff from the other archetypes speak enthusiastically of the changes in their practice. Additional study is needed to verify the importance of role models in a network’s improvement process. It appears that a new practice purchasing an EMR and wanting to use it to its best advantage should join a user’s group that focuses on improvement and fosters interaction between experts and novices. In such a forum, new users might access technological expertise, as was offered by our Technophiles.
It is difficult to compare practices across studies because of differing measurement approaches; however, another report suggests that the performance of practices in A-TRIP was better than national averages.33 For illustration, the 2004 National Health-care Quality Report (NHQR) indicated that among patients with diabetes, HbA1c was less than 7% in 37% of patients with diabetes, whereas A-TRIP practices achieved this target in 51% of patients. The subset of A-TRIP practices described in this report achieved better results than their peers across a wide range of areas, from diabetes and hypertension care to mammogram screening and adult vaccinations.
An Organizational Culture Perspective
Although this study provides insights regarding practice behavior from a qualitative perspective, recent interest in identifying organizational culture characteristics as factors influencing the quality and cost of care have prompted evaluation of several quantitative tools for potential use in health care settings.34,35 The competing values framework (CVF) has measured organizational culture in various settings outside health care.36 CVF identifies 4 organizational archetypes derived from Jungian influence37: Clan (focused on teams), hierarchy (invested in coordination and control), adhocracy (concerned with value innovation), and market-driven (motivated by competition). The organization’s dominant characteristics influence its members’ values and assumptions, the way they think, and how they process information. The archetypes of high performance that we found in our study have features in common with the CVF types.
The Technophiles archetype fits with CVFs adhocracy culture. This archetype values innovation and flexibility and develops electronic solutions to the demands of a quality-driven practice. The Motivated Team archetype fits a clan culture. These practices share a passion for cohesion, employee involvement, and commitment to team development. We observed a high degree of semiautonomous behavior in the practices that engaged their staff in cycles of improvement. By empowering the staff to come up with relevant solutions to identified problems, the Motivated Team practices were successful with change. The Care Enterprise archetype has additional features similar to the market-driven and hierarchical cultures. This archetype has strong leadership, organizes itself well, studies the community, and defines and packages services that are needed. Care Enterprise practices valued productivity and arranged for comprehensive convenient care for patients.
There are limitations to case study research. The findings are subjective; they are not meant to be representative or externally valid. In this study, only the best cases were observed, so controls for poor performance are lacking. The analysis is open to additional bias from the selection of data, confounding, and the researchers’ interpretations. The study implies that archetype features have led these case practices to their high achievement, but we need additional study comparing these practices with lower-performing practices to verify whether this finding is indeed true.
Another limitation is that case studies are meant to provide illustrations to which a reader can relate. One might feel we are lacking important details about the practices. It is already known that variations in health care quality cannot be explained on the basis of fee-for-service or managed care, size of the community, or type of insurance.38,39 The problem is believed to reside in how medicine is practiced, and we may have missed or misinterpreted key components. One strength of case study research is that results can be used to generate new hypotheses or insights. We highlight rare observations, such as the high performers, and consider their context.27 The analysis investigates how high performers approached improvements, and why their performance might be high—questions not easily answered in a large-scale outcome experiment.40 We encourage others to test the ideas generated here.
In conclusion, high-performing practices in the A-TRIP study adopted variations of the PPRNet TRIP improvement model. There were many common strategies among them, but they organized change using different approaches. No single approach dictated superior performance. Additional study is needed to assess whether better outcomes are caused by the approach of these archetypes and whether adoption of these approaches would translate to high achievement in other practices.
Acknowledgments
This study would not have been successful without the participation of the dedicated clinicians and staff in our top-performing sites.
Footnotes
-
↵* After the case types were presented to A-TRIP participants at the project’s final network meeting in September 2006, practices identified themselves by archetype. For example, “We are a Motivated Team, but I’d like to become a Technophile.”
-
Conflicts of interest: This study was supported by a grant from the Agency for Healthcare Research and Quality (Grant 1 U18 HS013716) to Practice Partner, the vendor of the EMR software used in PPRNet practices. Dr Feifer participated in this study as a consultant for Practice Partner; a subcontract to the Medical University of South Carolina supported the participation of the other authors in the project. Drs Ornstein and Wessell are consultants for Practice Partner for activities not directly related to the study described in this manuscript.
-
Funding support: This study was supported by a grant from the Agency for Healthcare Research and Quality (Grant 1 U18 HS013716).
A paper on this topic was presented with the title “Strategies used by Successful Practices to Accelerate the Translation of Research into Practice” at the North American Primary Care Research Group 33rd Annual Meeting in Quebec in October 2005.
- Received for publication March 13, 2006.
- Revision received October 20, 2006.
- Accepted for publication November 1, 2006.
- © 2007 Annals of Family Medicine, Inc.