Criterion-Referenced Examinations: Implications for the Reporting and Interpretation of Examination Results

March 19, 2013
Annals of Family Medicine
I appreciate the comments of Kenneth D. Royal and James C. Puffer regarding the possible benefits of criterion-based compared to percentile-ranked recertification exams "Criterion-Referenced Examinations: Implications for the Reporting and Interpretation of Examination Results" (March/April 2013 vol. 11 no. 2 185-187). If physicians' capabilities correlated with certainty by how well they performed on criterion-based exams I would agree with them.

But for physicians who have already been certified and who because they have been in practice for several years may have tailored their practices to the availability of specialists and to the demographics of their communities. This means that they may no longer do pediatrics, or gynecology or hospital and nursing home practice. Thus when they take the recertification exam they may no longer have the knowledge base necessary for them to pass the exam. Thus, this group of physicians is seriously disadvantaged when they recertify for their capabilities cannot be accurately assessed by how well or how poorly they perform on the criterion-based that they discussed.

There are many dimensions to what contributes to the totality of physicians' capabilities, dimensions that are not identified on recertification tests. The list is long but some are the ability to form good relationships with patients, the ability to work with the health care team and making timely consultations; participation in hospital activities and state and county medical society affairs also are important attributes of the well-rounded physician.

Considering these shortcomings of recertification it is wrong to penalize physicians with a failing grade after they have passed their initial certification. This punitive approach unfairly stigmatizes them and can have serious repercussions that can harm their reputations and their livelihoods. For hospitals and insurers can use certification as a requirement for privileges. There are some leaders in medicine who would like to use recertification as a requirement for re-licensure. Another problem with the criterion-based tests is their failure to take into account which examinees have taken board review courses. Cramming on test-taking tactics and memorizing material that has a high likelihood of being on the recertification tests gives them a definite advantage over those who haven't taken the review courses. This skews the results and makes them questionable. Criterion-based exams may sound good psychometrically but practically speaking, their value is limited. Disregarding "non-testable" qualities (and there are others) does a serious disservice to physicians.

Those who design the tests have a difficult job for it is filled with academic, practical, and moral dimensions. Which if they are not given equal consideration can harm good doctors. Perhaps simply giving physicians a printout of their tests so they can self-assess their performance and remedy those areas where they are deficient would be the best solution. This would help physicians immensely and make maintenance of certification/recertification something to look forward to and not something to be feared.

Edward Volpintesta MD

Competing interests:   None declared