Developing an International Register of Clinical Prediction Rules for Use in Primary Care: A Descriptive Analysis

This paper presents a laudable effort, i.e. a systematic source for evidence on clinical prediction models and rules. As with other reviews, we note three phases of evolution: development, validation, and impact assessment. It is however remarkable that relatively many validation studies were identified, compared e.g. to what we found with the PROGRESS group (http://www.ncbi.nlm.nih.gov/pubmed/23393430).

A number of other reflections, specifically on the naming of "prediction rules":

• Should we say 'model' or 'rule'? The difference may seem trivial, but Reilly et al (http://www.ncbi.nlm.nih.gov/pubmed/16461965) make a clear point on this issue. These authors explain that a prediction model provides probability estimates. These range between 0 and 100%. A prediction rule requires that we define a cut-off point to classify patients as low vs high risk. This cut-off is the decision threshold. It is often difficult to determine, since it requires the weighting of false-positive vs false-negative decisions. If a false-positive decision is of relatively low importance in comparison to a false-negative decision, a low threshold should be used. This is e.g. reasonable for giving antibiotics in case of suspicion of a severe infection. In contrast, a higher cut-off is necessary in case of more invasive tests, such as doing a biopsy in case of suspicion of prostate cancer.
• Prediction rules may also be derived with specific methods, such as classification trees, which define groups of patients with lower to higher risk. This approach is especially popular in trauma research, despite its inferior predictive performance (http://www.ncbi.nlm.nih.gov/pubmed/22026551).
• The transportability between secondary and primary setting may always be a challenge. Systematic differences in prevalence of diagnoses and incidences of outcomes are common. The adaptation of a prediction model may be easier than adaptation of a rule, e.g. the Ottawa Ankle Rule.
• An advantage of a rule may be that it is simpler, and hence easier to remember. In the near future, more and more prediction models will be available through the web, or through specific Apps. This may make that any model can be presented to the user (physician and/or patient), whatever the underlying mathematical complexity.
• Even if we agree on the preference for presenting models that provide continuous predictions between 0 and 100%, the specific naming is open for debate. In the PROGRESS group, some prefer naming as 'Risk prediction model', or RPM, while others prefer 'Clinical prediction model', or CPM. The latter term is also followed in my book on 'Clinical Prediction Models' (Springer 2009, http://www.clinicalpredictionmodels.org/).

In all, the current study is a valuable effort which deserves strong support, also in the context of keeping the registry up to date. Various methodological debates may come up from a more detailed look at the contents of each study in the registry. The naming issue of 'prediction model' or 'prediction rule' is just a start.

Competing interests: I receive royalties from Springer for my book Clinical Prediction Models.

A long overdue 'stock take' of the state of clinical prediction rules in primary care. We often teach that prediction rules are the 'pinnacle' of rationale/evidence-based diagnostic/prognostic clinical tests, as they should (emphasis on should!) help clinicians identify and use clinical features which have been shown to be (or not, as this article shows!) most useful in ruling in/out a condition. Equally though, whenever I ask a clinical audience to name clinical prediction rules, mostly they run out of ideas after the usual suspects of Ottawa rule, Centor criteria.... etc.

Claire Keogh et al's article tells us why - only about half of rules have any validation, and a tiny number have their impact assessed. Perhaps clinicians are right to be wary of them? Maybe, but as a method for improving diagnostic/prognostic ability, this seems a pity. So what should researchers and clinicians do now? Expanding the 'stock take' of the current state of play as the Dublin group suggest will be useful. I suggest three further steps: 1) Lets ask primary care clinicians what clinical prediction rules they actually want (to stop researchers coming up with rules that are never likely to be used), 2) For these priority areas lets see what prediction rules already exist, how further validation could be done using existing data sets (or electronic health record databases), and target these for impact studies, 3) Perhaps time for research funders to consider this rational approach to funding new derivation studies, unless there is a good reason. Is a moratorium too harsh?!

Competing interests: None declared