Improving Test Ordering in Primary Care: The Added Value of a Small-Group Quality Improvement Strategy Compared With Classic Feedback Only

My compliments for the great achievement of comparing two groups of GP's. However, I have two remarks: 1) The effect on the health of the patient seems to me unmeasureably low compared to the heavy input of the GP's; 2) the financial benefit will be negative if the hours of the GP's are also included.

My conclusions of the study of Verstappen are: it turns out to be possible to compare groups of GP's; the benefits for the patients are too little compared to the financial and personal input of GP's

Roeland Drijver

Competing interests:   None declared

WOK, the Dutch Inter-universitary Center for Quality of Care Research has a remarkable track record in evaluating the effect of interventions designed to influence general practitioners’ behavior, especially with respect to test ordering. Feedback is one of the main methods to reach this goal.

In the past, individual feedback to GPs on their laboratory test prescriptions has been shown to be very influential in a setting where the use of guidelines progressively was accepted by the GPs as a support method for clinical decision making (1). In other studies in different countries, the effect of feedback only proved not or far less effective. On the basis of a large number of complementary studies Richard Grol, the first chairman of Equip, the European Quality improvement group, who also supervised this study, made a plea for multifaceted interventions, in stead of any individual intervention (2).

In this clustered RCT, feedback only was compared with a multifaceted intervention, combining different individual as well as social approaches (3). The combined effect resulted in a decrease of both the number of tests that were prescribed and the inter-doctor variation.

In the ‘feedback only’ group this was not or far less the case. It progressively becomes clear that feedback, including comparison with national guidelines, peer group results or both, may be a decisive part of a behavioral change program, but only a part. To be influential, a number of prerequisites have to be met:

1. The aim of the feedback should be clear and acceptable for everybody concerned. This means that both providers and receivers of the feedback have to agree on the definition of ‘good behavior’ in the situation concerned. If they disagree, the discussion has to be solved before, or the whole exercise is going to be worthless. This may explain the problems when trying to influence the antibiotics prescription behavior of GPs for acute bronchitis or other respiratory diseases. Publishing guidelines only may not be sufficient for the purpose.
2. The data on which the feedback is based have to be trustworthy. In the case of large national databases, the precision of the data can be a problem. Worse, however, is the use of an inappropriate denominator, especially in countries where GPs don’t have listed patients. Some years ago a large intervention of the Belgian Government in order to decrease drug prescription rates by feedback was without any chance because the denominator they used was the number of patient contacts (at that time the only indicator that was available). Everybody realized that the calculation of number of prescriptions per 100 contacts was biased by the large inter-doctor differences in the yearly number of contacts per patient.
3. The provider of the feedback has to be acceptable for the professionals to which the feedback is aimed. In the study by Verstappen e.a., as well as in previous studies of the same group, the feedback was provided by the regional diagnostic centres, with whom the GPs are used to collaborate. In some cases physicians who previously worked as a GP are engaged in the management of such centre. The major objective was quality improvement, not cost decrease. In some countries, the feedback is provided by the government or the social insurance authorities and primarily intended to cut costs. That may not be the best option.
4. Using the influence of local peers by small-group discussions was one of the parts of the intervention by Verstappen e.a.. From some more detailed analyses published in his PhD thesis, this proved to work well and GPs were happy with the system. If GPs do not like to discuss their results in peer groups, this may be an important indicator for the existence of problems earlier in the chain, e.g. disagreement about the methods of the feedback or about the definition of ‘good behaviour’.
5. Depending on the exact topic it may be important to include the patients in the process. GPs don’t like to directly confront their patients with ‘new’ and unpopular messages, such as ‘I am not prescribing you an antibiotic for this cough, as I used to do in the past’. Last year the Belgian National Council on Quality Improvement (among others) launched two programmes to GPs in order to decrease the prescription rates of antibiotics and antihypertensive drugs. They included the production and distribution of guidelines, individual feedback with peer comparison, local small group discussions among peers and financial consequences for the group of GPs at large. The antibiotics program also included a series of TV commercials “Antibiotics don’t help in case of...”. This was not the case for the second program. The antibiotics program was successful, the antihypertensives programme only in a very limited way.
6. Feedback has to be either continuously or recurrent. A successful, but single intervention generally does not last a long time. The highly successful initial interventions in Maastricht in order to optimise the test prescription habits were repeated twice yearly. Also from other studies three to six months effect duration seems to be a maximum.

In the study of Verstappen et al, feedback, especially in combination with other interventions, has proven again to be a strong tool in the hands of those who continuously intend to improve health care quality. It is, however, no panacea and should be used with care.

1. Winkens RAG, Pop P, Bugter-Maessen AMA, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995: 498-502.
2. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change patients’care. Lancet 2003; 362: 1225-1230.
3. Verstappen WHJM, van der Weijden T, Dubois WI, et al. Improving test ordering in primary care: the added value of a small-group quality improvement strategy compared with classic feedback only. Ann Fam Med 2004; 2: 569-575.

Competing interests:   None declared

Verstappen and his colleagues are to be congratulated for their study that assessed the effectiveness of a multifaceted intervention consisting of comparative feedback, education on national guidelines and small-group sessions to improve test ordering. They adopted a contextual approach that explored variables at three levels: regional, group practice and individual. They used a randomized cluster design trial that took into account the non-independence of observations. They assessed the outcome of interest, number of requested tests, using an objective measure from computerized files. By assessing the change in interdoctor variation, they addressed a major concern for those involved in quality of care improvement(1). Most importantly, they provided valuable insight about the contribution of multifaceted strategies in changing health care professional behavior. This is particularly interesting, since evidence about the efficacy of multifaceted strategies is still somewhat controversial(2). Given the performance of the intervention, the authors rightfully discussed the mechanisms by which it might have influenced physicians’ behavior. They hypothesized that the improved outcomes may have resulted from social influence and peer influence. They proposed that interaction that occurred between participants during the small-group sessions “implicitly resulted in an individual or group contract”. This is plausible. However, a theory-based process evaluation along side this trial could have enriched our understanding of how to translate evidence into practice. For example, use of theoretically driven constructs such as social norm (an individual’s perception that relevant individuals are likely to approve or disapprove of the adoption of a given behavior)(3), role belief (the perception by the individual that members of a specific group would perform a given behavior)(4) or perception of control (the individual’s perception of barriers or facilitating factors likely to influence the adoption of the behaviour)(3), could have helped sort out whether, social influence or perception of control, predicted test ordering and had been modified by the intervention. In our own research, both social norm and perception of control influenced primary care physicians’ intention to adopt shared decision making(5). However, comparative feedback, combined with an interactive workshop and a reminder at the point of care, appeared to only influence their perception of control. Social norm was not influenced by our multifaceted intervention, which included comparative feedback coupled with an interactive workshop. Therefore, future trials aimed at changing health care providers’ behaviors would gain from integrating theory-driven measurements. Only then will the knowledge base of family medicine be complete(6).

References: 1. Wennberg JE. Practice variation: implications for our health care system. Manag Care 2004;13(9 Suppl):3-7. 2. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6):iii-iv, 1-72. 3. Ajzen I. Attitudes, personality and behavior: Open University Press; 1988. 4. Triandis HC. Interpersonal Behavior. Monterey, Ca.: Brooks/Cole Publishing Company A Division of Wadsworth Publishing Company, Inc.; 1977. 5. Légaré F, Graham I, O’Connor A, Wells G, Turcot-Lemay L, Aubin M, Baillargeon L, Castel J, Leduc Y, Maziade J. Theory-Based Process Evaluation of a Multifaceted Strategy to Implement the Ottawa Decision Support Framework (ODSF) in clinical practice. In: Wonca, Orlando; 2004 October 13-17; Orlando, Florida; 2004. 6. Stange KC, Miller WL, McWhinney I. Developing the knowledge base of family practice. Fam Med 2001;33(4):286-297.

Competing interests:   None declared