Continuous board certification serves physician Diplomates, the public, and the profession “by providing a system that supports the ongoing commitment of participating physicians to provide safe, high quality, patient centered care.”1 It provides an independent attestation that a physician has met medical specialty standards for professional standing, knowledge, commitment to self-assessment and lifelong learning, and improving the quality of their care.
ABFM has a long history of evaluating and revising its certification activities.2-8 Using surveys and activity evaluations, we continually examine aspects of Diplomate learning related to certification, including preparation for Family Medicine examinations, use of external resources to answer questions, seeking additional information after an activity, and identification of areas for future study (Table 1). Analyses have consistently demonstrated positive reactions, short-term knowledge gains, intent to change, and self-reported practice change from ABFM Knowledge Self-Assessments (KSA), Continuous Knowledge Self-Assessments (CKSA), performance improvement activities, and the Family Medicine Certification Longitudinal Assessment (FMCLA), which was recently approved for permanent use.9 In this article, we briefly describe the goals of the ABFM Strategy to support and promote Diplomate learning, 3 major areas of work, and potential collaborations that can arise from these efforts.
Select Survey Questions Used by the ABFM to Assess Diplomate Learning From Certification Activities
Goals
We seek to optimize the learning value of certification for Diplomates, while simultaneously enhancing ABFM’s processes determining certification status of family physicians. Evidence to date does not suggest that Diplomates select ABFM activities based primarily on identified gaps in their knowledge or performance.10 We will continue to regularly provide feedback to help Diplomates reflect on their learning needs and engage in directed self-learning.11 By supporting both assessment of learning (summative assessment) and assessment for learning (formative assessment), we will continually improve certification activities to enhance relevance, reduce burden, and support clinical mastery across Diplomates’ careers.
The strategy is:
Grounded in theory and evidence about adult learning, including longitudinal, spaced, interactive, multimodal, reflective, repeated, and reinforced formats associated with improvement in physician practice.11-13
Informed by Diplomate feedback, including feedback that has already helped us develop FMCLA9 and improve our KSAs.14
Built on a framework of continuous professional development (CPD).15
Additionally, it will:
Enable development of competencies beyond medical knowledge, including communication skills, team-based care, professionalism, patient-centered care, population health, and models of care delivery.
Provide additional opportunities for customization to individual practice type and scope while recognizing the need for general learning required to be an effective, broad-based, personal physician.
Augment learning opportunities provided by the American Academy of Family Physicians (AAFP) and other education providers.
The Strategy includes the following major components:
Enhancing Longitudinal Assessment
Participating Diplomates found the FMCLA pilot useful for identifying individual knowledge strengths and weaknesses and supporting learning. Nearly 90% reported using FMCLA as part of their ongoing knowledge acquisition. Over 80% reported that FMCLA stimulated patient care changes. The data also indicates a low correlation between Diplomate confidence in their answer to a question and if they were actually correct. Stakes also seem to matter—despite being correct more often, Diplomates reported lower confidence in FMCLA responses than in CKSA, another longitudinal but self-assessment–only activity. These findings are consistent with studies showing that in the absence of data, physicians (like other professionals), are not particularly accurate in self-assessment.16,17
We plan to further enhance FMCLA and CKSA by including testing points in critiques and improving the individual reports showing overall performance over time and within knowledge blueprint categories. We are also piloting the effectiveness of spaced repetition11 of questions, especially those answered incorrectly but with high confidence, in improving knowledge retention and transfer to different clinical scenarios. We will also explore whether longitudinal assessment improves a physician’s ability to self-assess, plan, monitor, and address their own performance.18
Developing Measures of Engagement and Enhancing Measures of Learning
ABFM post-activity completion surveys have provided us with some indicators of diplomate learning and engagement. The FMCLA pilot provided us with other potential indicators. The average time to read and answer a question was 2.3 minutes (most commonly the time was less than 45 seconds). Ninety-four percent of the time, critiques were viewed for less than 2 minutes. In addition to refining these indicators, we plan to explore other measures of Diplomate learning over time.19,20 We hope this information will help us improve our activities and inform individual learning plans.
Connecting Learning Across the Components of Continuing Certification
We envision a future where Diplomates use ABFM certification activities not solely to fulfill requirements, but as an essential part of their CPD.15 Diplomates would select practice-relevant lifelong learning activities based on gaps identified from their ABFM assessments and external performance data. Question feedback and reports from ABFM activities can highlight opportunities for additional learning. KSA, CKSA, journal articles (an ABFM Journal Club is in pilot phase), targeted continuing medical education (CME), and other individual learning efforts can be utilized for deeper learning. Diplomates can apply this new knowledge through performance improvement activities (external or internal to ABFM) and use subsequent data and ABFM assessments to assess their progress. The cycle could continue until Diplomates achieve their goals. We aim to refine Diplomate feedback to facilitate these cycles and explore the impact on processes and outcomes of care.
Moving Forward
ABFM activities are just one part of ongoing learning and development for family physicians. We want ABFM activities to provide Diplomates with a valuable, objective look at their learning needs, enabling them to select targeted educational and improvement activities to address gaps in knowledge and practice performance. We intend to collaborate with educational, quality improvement, and other experts as we pursue this work. We envision the possibility of sharing deidentified information that could inform educational opportunities offered by the AAFP, AAFP state chapters, health systems, and other CME/continuing professional development (CPD) providers. By connecting the assessment role of ABFM with the intrinsic desire to continuously learn and improve that motivates our Diplomates and our Family Medicine partners, we aim to enable enduring learning and practice change that ultimately improves patients’ care outcomes.21
- © 2021 Annals of Family Medicine, Inc.
References
In this issue
Related Articles
Cited By...
- No citing articles found.