|
|
||||||||

1 Centre for Quality of Care Research (WOK), Care and Public Health Research Institute (CAPHRI), and Department of Primary Care, Maastricht University, The Netherlands
2 Centre for Diagnostics and Consultation, St Jans Hospital, Weert, The Netherlands
3 Centre for Diagnostics and Consultation, Elkerliek Hospital, Helmond, The Netherlands
4 Medical Diagnostic Centre, Canisius-Wilhelmina Hospital, Nijmegen, The Netherlands
5 Department of Methodology and Statistics, Maastricht University, The Netherlands
CORRESPONDING AUTHOR: Wim Verstappen, MD, PhD, Department of Primary Care, Maastricht University, PO Box 616, 6200 MD Maastricht, The Netherlands wim.verstappen{at}hag.unimaas.nl
| ABSTRACT |
|---|
|
|
|---|
METHODS We enrolled 194 primary care physicians from 27 local primary care practice groups in 5 health care regions (5 diagnostic centers). The study was a cluster randomized trial with randomization at the local physician group level. We evaluated an innovative, multifaceted strategy, combining written comparative feedback, group education on national guidelines, and social influence by peers in quality improvement sessions in small groups. The strategy was aimed at 3 specific clinical topics: cardiovascular issues, upper abdominal complaints, and lower abdominal complaints.
The mean number of tests per physician per 6 months at baseline and the physicians region were used as independent variables, and the mean number of tests per physician per 6 months was the dependent variable.
RESULTS The new strategy was executed in 13 primary care groups, whereas 14 groups received feedback only. For all 3 clinical topics, the decrease in mean total number of tests ordered by physicians in the intervention arm was far more substantial (on average 51 fewer tests per physician per half-year) than the decrease in mean number of tests ordered by physicians in the feedback arm (P = .005). Five tests considered to be inappropriate for the clinical problem of upper abdominal complaints decreased in the intervention arm, with physicians in the feedback arm ordering 13 more tests per 6 months (P = .002). Interdoctor variation in test ordering decreased more in the intervention arm.
CONCLUSION Compared with only disseminating comparative feedback reports to primary care physicians, the new strategy of involving peer interaction and social influence improved the physicians test-ordering behavior. To be effective, feedback needs to be integrated in an interactive, educational environment.
Key Words: Quality assurance, health care test-ordering behavior feedback small-group quality improvement quality of health care
| INTRODUCTION |
|---|
|
|
|---|
The literature shows that multifaceted strategies are generally superior to single methods when it comes to influencing behavior.1012 Success rates of specific strategies seem to be strongly influenced by the extent to which they fit in with the local and organizational context and the physicians day-today work routine.13,14 Favorable experience has been gained in particular with small-group education and interactive quality improvement sessions for primary care physicians.15,16 We therefore decided to develop a multifaceted strategy, combining transparent, individual graphic feedback on test-ordering routines, education on clinical guidelines for test ordering, and small-group quality improvement meetings among primary care physicians. At these meetings, test-ordering behavior and changes in routines were discussed, using social influence and peer influence as important motivators for change. Social influence from respected colleagues or opinion leaders seems to have a greater effect on practice routines than do traditional medical education activities focusing on changing professional cognitions or attitudes.1721 We therefore expected our strategy to be useful, because it is closely linked to the everyday setting for many physicians, who tend to work more or less in isolation and have limited contact with peers about topics like test-ordering behavior.
We hypothesized that greater insights into and discussion of the physicians own performance in a safe group of respected colleagues would be a powerful instrument to improve the quality of test ordering. Because classic feedback is increasingly used as a routine quality improvement strategy, and because this simple and cheap strategy might suffice, we further hypothesized that this innovative, multifaceted strategy would have an added value relative to standardized feedback only. In one multicenter randomized trial with a block design, this strategy was indeed found to have a favorable effect on physicians test-ordering behavior.22 A cost analysis of the new strategy and a process evaluation showed that it was a cost-efficient and feasible tool for improvement of physicians test-ordering behavior.23,24 This article determines the effects of this innovative, multifaceted strategy, compared with a classic feedback strategy, to assess the added value of the small-group quality improvement meetings.
| METHODS |
|---|
|
|
|---|
Local teams of primary care physicians collaborating in a specific region are a common feature of Dutch primary care. Every physician working in a solo, 2-person, group, or health center practice in the Netherlands is a member of a local primary care practice group. Continuous medical education (for example, quality improvement meetings about prescribing) is an important activity in most groups. One task of a diagnostic center medical coordinator is to give feedback to these physicians on their test-ordering behavior, and the medical coordinators are considered to be opinion leaders concerning test ordering. From May 1998 until September 1998 the coordinators of the 5 diagnostic centers recruited local primary care groups in their regions to participate in the trial.
Intervention
The new strategy consisted of the following elements: (1) personalized graphic feedback, including a comparison of each physicians own data with those of colleagues; (2) dissemination of and education on national, evidence-based guidelines; and (3) continuous quality improvement meetings in small groups. The improvement strategy concentrated on 3 specific clinical topics (cardiovascular conditions, upper abdominal complaints, and lower abdominal complaints) and the tests used for these clinical problems, because it was believed that the physicians would prefer to discuss specific clinical topics rather than specific tests (Table 1
). During the first half of 1999 each physician received 3 different feedback reports on these 3 clinical problems by mail, together with concise information on the evidence-based clinical guidelines for these specific clinical topics developed by the Dutch College of Primary Care Physicians (an example of a feedback report is available online only as a supplemental figure at: http://www.annfammed.org/cgi/content/full/2/6/569/DC1). Each postal contact was followed by a 90-minute standardized small-group quality improvement meeting about 2 weeks later at which one of the clinical problems was discussed based on the feedback reports and the guidelines (Figure 1
). In these meetings social influence, which was an important vehicle to reach improvement on test ordering, consisted of the following major components. The first was mutual personal feedback by peers, who worked in pairs at the start of the meeting. The second component was interactive group education in which national guidelines were related to the individual physicians actual test-ordering behavior and an effort to reach a group consensus on the optimal test-ordering behavior. The third was the development of individual and group plans for change to stimulate the physicians to put their plans into daily practice. As a critical follow-up, achieving the goals of these plans was discussed at the next meeting. The medical coordinators disseminated the feedback reports and organized and supervised the quality improvement meetings. That the medical coordinators were respected regional opinion leaders concerning test-ordering behavior was an additional important component in the social influence strategy.
|
|
|
Statistical Analysis
Differences on individual physician characteristics were tested for significance with the Pearsons chi-square-test. To evaluate intervention effects, the unit should be the local primary care practice group, because that group was the unit of randomization. A 3-level model was used with the practice group as level 3, physicians as level 2, and numbers of tests as level 1. This model was analyzed using SAS PROC MIXED Release 8.2 (SAS Institute, Cary, NC). Power calculations based on the baseline data showed that each arm needed approximately 85 physicians to detect a 10% difference in mean total numbers of tests with 80% power, and a risk of type 1 error of .05. The region appeared to be an important determinant of the between-group variance and was used as independent variable, together with the baseline numbers of tests. All effects were analyzed with analyses of covariance. This regression equation gives the intervention effect ß from which the follow-up numbers of tests are the dependent variable and the baseline numbers and the region are the independent variables. ß reflects the total change between mean numbers of tests in the intervention arm minus the total change between baseline and follow-up in mean numbers of tests in the feedback arm, adjusted for baseline and region.
Interdoctor variation was calculated by the coefficient of variance, the standard deviation (SD) divided by the mean.
| RESULTS |
|---|
|
|
|---|
Table 2
describes the characteristics of the study population. Mean group size in the intervention arm was 6.9 (SD 2.1) compared with 7.8 (SD 4.2) in the feedback arm. There was a large, but statistically insignificant difference in mean total numbers of tests per physician per 6 months between the 2 arms at baseline; the intervention arm had 478 (SD 309), the feedback arm had 541 (SD 337). An intention-to-treat analysis was not possible for 10 physicians in each arm, including one entire local practice group in the intervention arm. Data for the follow-up measurements for these physicians were lacking because of absence, change of jobs, or practice-related data registration problems. Multilevel analyses showed that the point estimation and standard deviation were the same at the group level as in the analysis of covariance at the individual physician level; therefore, no correction for local practice groups was needed.
|
|
|
| DISCUSSION |
|---|
|
|
|---|
Some methodological comments can be made on our study. It is possible that only motivated, well-functioning groups of physicians participated, and it is therefore questionable whether the strategy would work for all groups. Despite the large numbers of participating physicians, a difference between the 2 study arms was found in baseline performance. This difference is probably due to chance, however, as the number of randomization objects was small (n = 27). Despite the smaller mean number of tests at baseline, the intervention arm succeeded in substantially decreasing the numbers of tests ordered. We did not include a nonintervention control arm; we did not consider doing so a relevant contrasting strategy because feedback is now a regularly used strategy in primary care in the Netherlands. Unfortunately, we could not use clinical data to evaluate the effect, but because the evidence-based guidelines recommend a reduction in the total numbers of tests, the decrease we found can be interpreted as a quality improvement. Moreover, there is empirical evidence that a general reduction in test use in primary care does not lead to more referrals or substitution of care.27,28
We expect that these limitations have had only minor impact on the results, and our results may yield 2 important conclusions. The first is that this new strategy can be a powerful innovative instrument to change physician test-ordering behavior. The strategy gives physicians an opportunity to discuss their test-ordering performance with colleagues on the basis of actual performance data, making the participants feel more committed to the agreements. Our strategy also seems worthwhile because small-group quality improvement meetings can help build a local practice group focusing on quality improvement. Many test-ordering problems that physicians encounter in everyday practice, such as demands for tests by patients and changing guidelines, can be discussed and may be solved in an open and respectful discussion among colleagues. The second conclusion is that merely sending feedback reports to physicians, without other activities, such as peer discussion or other strategies that fit in with everyday practice, does not have much impact on test-ordering behavior. More effort is needed, and feedback reports must fit in with a more ambitious continuous quality improvement program. Further, although our method was applied for test-ordering behavior, it also seems applicable to quality improvement for other issues, such as prescribing and referral behavior, and for other teams of collaborating physicians.
Nationwide implementation of this innovative strategy would be a logical next step and is now being prepared in the Netherlands.
| FOOTNOTES |
|---|
Deceased. Conflicts of interest: none reported
Funding support: The authors gratefully acknowledge the financial contribution to the study provided by the Dutch College for Health Insurances.
Received for publication January 20, 2004. Revision received May 11, 2004. Accepted for publication May 19, 2004.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
C. A. McNulty, M. Thomas, J. Bowen, C. Buckley, A. Charlett, D. Gelb, C. Foy, J. Sloss, and S. Smellie Improving the appropriateness of laboratory submissions for urinalysis from general practice Fam. Pract., August 1, 2008; 25(4): 272 - 278. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Ioannidis, A. Papaioannou, L. Thabane, A. Gafni, A. Hodsman, B. Kvern, D. Johnstone, N. Plumley, A. Baldwin, M. Doupe, et al. Canadian Quality Circle pilot project in osteoporosis: Rationale, methods, and feasibility Can Fam Physician, October 1, 2007; 53(10): 1694 - 1700. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. J. Zolotor, G. D. Randolph, J. K. Johnson, S. Wegner, L. Edwards, C. Powell, and M. H. Esporas Effectiveness of a Practice-Based, Multimodal Quality Improvement Intervention for Gastroenteritis Within a Medicaid Managed Care Network Pediatrics, September 1, 2007; 120(3): e644 - e650. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. C. Stange "Our community of reflective practitioners will continue its quest informed...." Ann. Fam. Med, January 1, 2005; 3(1): 86 - 87. [Full Text] [PDF] |
||||
![]() |
K. C. Stange In This Issue: New Model Finances, Systematic Reviews, Patients and Health Care Ann. Fam. Med, November 1, 2004; 2(6): 530 - 531. [Full Text] [PDF] |
||||
Read all TRACK Comments
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |