Methods to Achieve High Interrater Reliability in Data Collection From Primary Care Medical Records

Clare Liddy; Miriam Wiens; William Hogg

doi:10.1370/afm.1195

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article

Compose eLetter

Title *

Author Information

Contributors
First Name and Middle Initial * First or given name, e.g. 'Peter'. Last Name * Your last, or family, name, e.g. 'MacMoody'. Email Address * Your email address, e.g. higgs-boson@gmail.com Occupation * Your role and/or occupation, e.g. 'Orthopedic Surgeon'. Affiliation * Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.

Statement of Competing Interests

Competing interests? *

Yes

Please describe the competing interests

CAPTCHA

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

What code is in the image? *

Enter the characters shown in the image.

Vertical Tabs

Jump to comment:

Quantity versus quality
Andrew Worster

Published on: 20 January 2011

Published on: (20 January 2011)
Page navigation anchor for Quantity versus quality
Quantity versus quality
Andrew Worster, Hamilton, ON
The study by Liddy et al is a welcomed contribution to advancing the methods of retrospective studies. Inter-rater reliability assessment is a complex subject and the comparison of performance of two or more data abstractors is not quite so simple as determining how many variables that they agreed on beyond that expected by chance alone. This approach assumes that all of the variables are dichotomous and none are multilev...
Show More

The study by Liddy et al is a welcomed contribution to advancing the methods of retrospective studies. Inter-rater reliability assessment is a complex subject and the comparison of performance of two or more data abstractors is not quite so simple as determining how many variables that they agreed on beyond that expected by chance alone. This approach assumes that all of the variables are dichotomous and none are multilevel and also that all variables have the same clinical weighting. For example, the chances of all of the abstractors agreeing on a patient's exact age are far less than them all agreeing upon whether the patient smoked. However, if using both of these variables as determinants of cardiovascular disease, an error in recording the age as 67 instead of 76 will likely have far less impact than an error in recording the patient's smoking status. Therefore, a single IRR measurement will only reveal the quantity of the error and not the quality. A solution might be to group variables of similar weighting and those with a similar number of potential responses and measuring the IRRs of each group separately. Readers will be reassured of a study's methodological rigor when the IRR of clinical variables of greatest importance is high.
Competing interests: None
Show Less

Competing Interests: None declared.