We continue to receive a number of inquiries from physicians regarding the low pass rate statistics for recertification candidates for the reporting periods from 2010 to 2012. Many physicians fear the examination has changed in some way, putting those attempting to maintain their certification at a distinct disadvantage. Others worry that the passing score criteria for recertifiers are greater than those of initial certifiers. We appreciate the active interest in ongoing assessment results and welcome the opportunity to clarify the source of these statistical changes. In short, the Maintenance of Certification for Family Physicians (MC-FP) examination has not changed. Passing score criteria for those attempting to maintain their certificates are no more stringent now than in recent years. The apparent pass rate discrepancy for this group in 2010, 2011, and 2012 is a sample bias among recertification candidates created by a policy change that extended certification from 7 years to 10 years. We will attempt to explain this statistical artifact in simple terms and address the concerns expressed by many family physicians.
Certification Extension Policy Change
In January 2006, the ABFM announced an enhancement to the MC-FP process that went into effect in January of 2007. The most important enhancement to the process was the opportunity to extend one’s certification by 3 years, resulting in a 10-year certification period. Many physicians that would previously have taken the examination during their 7th year successfully met the new MC-FP requirements and did not sit for the MC-FP examination as a result of earning the 3-year extension of their certificate. This option resulted in a much smaller volume of MC-FP (recertification) candidates testing during the 3-year window from 2010 to 2012.
Recent Pass/Fail Statistics for Initial Certifiers and Recertifiers
Examination pass rate statistics are available on the ABFM website.1 We would like to draw readers’ attention to the summer statistics for years 2008 through 2012. Pass rates for 2008 and 2009 initial certifiers were 82.9% and 86.7%. Pass rates for recertification candidates were 86.8% and 85.7%, respectively. Readers will notice in 2010, 2011, and 2012 that initial certification pass rates remain quite comparable, with pass rates of 84.0%, 84.2%, and 83.2%, respectively. However, it is with the recertification candidates that the pass rate statistics become misleading. In 2010, recertification candidates passed at a rate of 67.0%, with pass rates of 66.3% and 73.9% in 2011 and 2012. These values are considerably below recent historical trends. However, when one notes the volume of candidates sitting for the examination during these years the rest of the story begins to unfold.
Initial certification candidate volume has remained quite stable since 2008, with as few as 2,236 examinees and as many as 2,804. Recertification candidate volume, on the other hand, experienced a steep decline. In 2008 and 2009, recertification candidate volume ranged between 8,356 and 8,453. In 2010, recertification candidate volume dipped to only 1,397 examinees, thus resulting in 6,959 fewer recertifiers than the previous year. The trend continued in 2011 and 2012, as only 1,209 and 1,395 recertification candidates, respectively, sat for the examination. The certification extension reduced the volume of candidates re-testing during the transition period. Many candidates who did re-test in this period were repeating the examination because of previous failure. A larger proportion of previously unsuccessful candidates combined with a smaller number of candidates testing overall created a dramatically lower passing rate percentage statistic.
Table 1 provides a breakdown of pass rates and mean scaled scores for both initial certifiers and recertifiers based on whether it was the examinees first attempt, or a repeat attempt at the MC-FP examination. Results indicate that candidates who successfully pass their first attempt tend to continue to perform well. Candidates repeating the examination because of a previous failure tend to perform less well on subsequent examinations. Significantly fewer recertification candidates sat for the examination during the transition period in which most diplomates earned 3-year extensions to their certificates. A significant proportion of these candidates were attempting due to previous failures. Therefore, pass rate percentage statistics reported in this transition period are not truly comparable to previously reported statistics. Particularly, recertification candidates testing during the 3-year extended period are not representative of recertification candidates in normal, high-volume years.
Examination Performance Based on Attempt Status
Conclusions
On the surface, pass rate statistics for MC-FP candidates may appear alarming for recent years as pass rates are significantly lower than previous years. These candidates will be happy to know that this statistic is misleading, as a sample bias stemming from a policy change that extended certification from 7 years to 10 years is present in the current recertification candidate statistics. This news should be reassuring to physicians that are approaching recertification and beginning study preparations for the November 2013 MC-FP examination.
To further emphasize these points, we have just finished preliminary analysis of the April 2013 examination. The volume of recertification candidates once again resembles the figures cited above in 2008 and 2009, and the characteristics of the sample no longer contain the sample bias of having a significant proportion of recertification candidates testing as a result of a previous failure. Accordingly, the pass rates for candidates taking the examination to maintain their certificate (recertifiers) are very similar to the pass rates for this cohort in 2009. These results were expected and should further assure family physicians that our examinations behave as predicted, since the examinations are standardized and the same passing standard is in effect for the 2013 examination as was the case in 2009.
- © 2013 Annals of Family Medicine, Inc.
