Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleFamily Medicine UpdatesF

THE AMERICAN BOARD OF FAMILY MEDICINE STRATEGY TO SUPPORT AND PROMOTE DIPLOMATE LEARNING

David W. Price, Andrew Bazemore, Elizabeth G. Baxley, Keith Stelter, Thomas R. O’Neill, Roger Fain, Michael K. Magill and Warren P. Newton
The Annals of Family Medicine September 2021, 19 (5) 468-470; DOI: https://doi.org/10.1370/afm.2749
David W. Price
University of Colorado Anschutz School of Medicine (corresponding author);
MD, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew Bazemore
University of Minnesota School of Medicine;
MD, MPH, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elizabeth G. Baxley
University of Minnesota School of Medicine;
MD, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Keith Stelter
University of Minnesota School of Medicine;
MD, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thomas R. O’Neill
University of Utah School of Medicine;
PhD, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roger Fain
University of Utah School of Medicine;
MA, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael K. Magill
University of Utah School of Medicine;
MD, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Warren P. Newton
MD, MPH, ABFM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Continuous board certification serves physician Diplomates, the public, and the profession “by providing a system that supports the ongoing commitment of participating physicians to provide safe, high quality, patient centered care.”1 It provides an independent attestation that a physician has met medical specialty standards for professional standing, knowledge, commitment to self-assessment and lifelong learning, and improving the quality of their care.

ABFM has a long history of evaluating and revising its certification activities.2-8 Using surveys and activity evaluations, we continually examine aspects of Diplomate learning related to certification, including preparation for Family Medicine examinations, use of external resources to answer questions, seeking additional information after an activity, and identification of areas for future study (Table 1). Analyses have consistently demonstrated positive reactions, short-term knowledge gains, intent to change, and self-reported practice change from ABFM Knowledge Self-Assessments (KSA), Continuous Knowledge Self-Assessments (CKSA), performance improvement activities, and the Family Medicine Certification Longitudinal Assessment (FMCLA), which was recently approved for permanent use.9 In this article, we briefly describe the goals of the ABFM Strategy to support and promote Diplomate learning, 3 major areas of work, and potential collaborations that can arise from these efforts.

View this table:
  • View inline
  • View popup
Table 1.

Select Survey Questions Used by the ABFM to Assess Diplomate Learning From Certification Activities

Goals

We seek to optimize the learning value of certification for Diplomates, while simultaneously enhancing ABFM’s processes determining certification status of family physicians. Evidence to date does not suggest that Diplomates select ABFM activities based primarily on identified gaps in their knowledge or performance.10 We will continue to regularly provide feedback to help Diplomates reflect on their learning needs and engage in directed self-learning.11 By supporting both assessment of learning (summative assessment) and assessment for learning (formative assessment), we will continually improve certification activities to enhance relevance, reduce burden, and support clinical mastery across Diplomates’ careers.

The strategy is:

  • Grounded in theory and evidence about adult learning, including longitudinal, spaced, interactive, multimodal, reflective, repeated, and reinforced formats associated with improvement in physician practice.11-13

  • Informed by Diplomate feedback, including feedback that has already helped us develop FMCLA9 and improve our KSAs.14

  • Built on a framework of continuous professional development (CPD).15

Additionally, it will:

  • Enable development of competencies beyond medical knowledge, including communication skills, team-based care, professionalism, patient-centered care, population health, and models of care delivery.

  • Provide additional opportunities for customization to individual practice type and scope while recognizing the need for general learning required to be an effective, broad-based, personal physician.

  • Augment learning opportunities provided by the American Academy of Family Physicians (AAFP) and other education providers.

The Strategy includes the following major components:

Enhancing Longitudinal Assessment

Participating Diplomates found the FMCLA pilot useful for identifying individual knowledge strengths and weaknesses and supporting learning. Nearly 90% reported using FMCLA as part of their ongoing knowledge acquisition. Over 80% reported that FMCLA stimulated patient care changes. The data also indicates a low correlation between Diplomate confidence in their answer to a question and if they were actually correct. Stakes also seem to matter—despite being correct more often, Diplomates reported lower confidence in FMCLA responses than in CKSA, another longitudinal but self-assessment–only activity. These findings are consistent with studies showing that in the absence of data, physicians (like other professionals), are not particularly accurate in self-assessment.16,17

We plan to further enhance FMCLA and CKSA by including testing points in critiques and improving the individual reports showing overall performance over time and within knowledge blueprint categories. We are also piloting the effectiveness of spaced repetition11 of questions, especially those answered incorrectly but with high confidence, in improving knowledge retention and transfer to different clinical scenarios. We will also explore whether longitudinal assessment improves a physician’s ability to self-assess, plan, monitor, and address their own performance.18

Developing Measures of Engagement and Enhancing Measures of Learning

ABFM post-activity completion surveys have provided us with some indicators of diplomate learning and engagement. The FMCLA pilot provided us with other potential indicators. The average time to read and answer a question was 2.3 minutes (most commonly the time was less than 45 seconds). Ninety-four percent of the time, critiques were viewed for less than 2 minutes. In addition to refining these indicators, we plan to explore other measures of Diplomate learning over time.19,20 We hope this information will help us improve our activities and inform individual learning plans.

Connecting Learning Across the Components of Continuing Certification

We envision a future where Diplomates use ABFM certification activities not solely to fulfill requirements, but as an essential part of their CPD.15 Diplomates would select practice-relevant lifelong learning activities based on gaps identified from their ABFM assessments and external performance data. Question feedback and reports from ABFM activities can highlight opportunities for additional learning. KSA, CKSA, journal articles (an ABFM Journal Club is in pilot phase), targeted continuing medical education (CME), and other individual learning efforts can be utilized for deeper learning. Diplomates can apply this new knowledge through performance improvement activities (external or internal to ABFM) and use subsequent data and ABFM assessments to assess their progress. The cycle could continue until Diplomates achieve their goals. We aim to refine Diplomate feedback to facilitate these cycles and explore the impact on processes and outcomes of care.

Moving Forward

ABFM activities are just one part of ongoing learning and development for family physicians. We want ABFM activities to provide Diplomates with a valuable, objective look at their learning needs, enabling them to select targeted educational and improvement activities to address gaps in knowledge and practice performance. We intend to collaborate with educational, quality improvement, and other experts as we pursue this work. We envision the possibility of sharing deidentified information that could inform educational opportunities offered by the AAFP, AAFP state chapters, health systems, and other CME/continuing professional development (CPD) providers. By connecting the assessment role of ABFM with the intrinsic desire to continuously learn and improve that motivates our Diplomates and our Family Medicine partners, we aim to enable enduring learning and practice change that ultimately improves patients’ care outcomes.21

  • © 2021 Annals of Family Medicine, Inc.

References

  1. 1.↵
    1. Vision for the Future Commission
    . Continuing board certification vision for the future. Published Feb 12, 2019. Accessed Jul 13, 2021. https://visioninitiative.org/wp-content/uploads/2019/02/Commission_Final_Report_20190212.pdf
  2. 2.↵
    1. Hagen MD,
    2. Ivins DJ,
    3. Puffer JC, et al.
    Maintenance of certification for family physicians (MC-FP) self assessment modules (SAMs): the first year. J Am Board Fam Med. 2006; 19(4): 398-403. doi:10.31222/jabfm.19.4.398
    OpenUrlAbstract/FREE Full Text
  3. 3.
    1. Hagen MD,
    2. Sumner W,
    3. Fu H. et al.
    Diuretic of choice in ABFM hypertension self-assessment module simulations. J Am Board Fam Med. 2012; 25(6): 805-809. doi:10.31222/jabfm.2012.06.110343
    OpenUrlAbstract/FREE Full Text
  4. 4.
    1. Peterson LE,
    2. Blackburn B,
    3. King MR
    .. Completing self-assessment modules during residency is associated with better certification exam results. Family Med. 2014; 46(8): 597-602.
    OpenUrl
  5. 5.
    1. Elward K,
    2. Blackburn B,
    3. Peterson LE,
    4. Greenwald M,
    5. Hagen MD
    . Improving quality of care and guideline adherence for asthma through a group self-assessment module. J Am Board Fam Med. 2014; 27(3): 391-398. doi:10.3122/jabfm.2014.03.130241
    OpenUrlAbstract/FREE Full Text
  6. 6.
    1. O’Neill TR,
    2. Sun L,
    3. Peabody MR,
    4. Royal KD
    . The impact of repeated exposure to items. Teach Learn Med. 2015; 27(4): 404-409. doi:10.1080/10401334.2015.107714131
    OpenUrlCrossRef
  7. 7.
    1. Brooks EM,
    2. Gonzalez M,
    3. Eden AR,
    4. O’Neal J,
    5. Sabo RT,
    6. Etz RS
    . What family physicians really think of Maintenance of Certification Part II activities. J Contin Educ Health Prof. 2017; 37(4): 223-229. doi:10.1097/CEH.0000000000000170
    OpenUrlCrossRef
  8. 8.↵
    1. Newton WP,
    2. Baxley L,
    3. Williamson A,
    4. Rode K
    . Improving performance improvement. J Am Board Fam Med. 2020; 33: 819-821
    OpenUrlFREE Full Text
  9. 9.↵
    1. Newton WP,
    2. O’Neill T,
    3. Baxley E,
    4. Fain R,
    5. Stelter K,
    6. Price DW,
    7. Rode K
    . Family medicine certification longitudinal assessment becomes permanent. J Am Board Fam Med. In press.
  10. 10.↵
    1. Peterson LE,
    2. Blackburn B,
    3. Bazemore A,
    4. O’Neill T,
    5. Phillips RL Jr..
    Do family physicians choose self-assessment activities based on what they know or don’t know? J Contin Educ Health Prof. 2014; 34(3): 164-170. doi:10.1002/chp.21247
    OpenUrlCrossRef
  11. 11.↵
    1. Price DW,
    2. Swanson DB,
    3. Irons MB,
    4. Hawkins RE
    . Longitudinal assessments in continuing specialty certification and lifelong learning. Med Teach. 2018; 40(9): 917–919. doi:10.1080/0142159X.2018.1471202
    OpenUrlCrossRef
  12. 12.
    1. Knowles MS,
    2. Holton EF,
    3. Swanson EF
    . The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development. 6th ed. Elsevier; 2005.
  13. 13.↵
    1. Cervero RM,
    2. Gaines JK
    . The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Contin Educ Health Prof. 2015; 35(2): 131-138.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Baxley EG,
    2. Banik E,
    3. Fain R,
    4. Stelter K,
    5. Price D,
    6. Dewahare A,
    7. Quan M,
    8. Newton WP
    . Evolving certification to meet today’s needs: the ABFM’s KSA revision initiative. Ann Fam Med. 2020; 18: 566-569. doi:10.1370/afm.2619
    OpenUrlFREE Full Text
  15. 15.↵
    1. Rayburn W,
    2. Turco M,
    3. Davis DA
    1. Price D
    . Maintenance of certification, continuing professional development, and performance improvement. In: Rayburn W, Turco M, Davis DA, eds. Continuing Professional Development in Medicine and Health Care: Better Education, Better Patient Outcomes. Wolters Kluwer; 2017.
  16. 16.↵
    1. Davis DA,
    2. Mazmanian PE,
    3. Fordis M,
    4. Van Harrison R,
    5. Thorpe KE,
    6. Perrier L
    . Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006; 296: 1094–1102.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Zell E,
    2. Krizan, Z
    . Do people have insight into their abilities? A metasynthesis. Persp Psychol Sci. 2014: 9(2): 111-125. doi:10.1177/1745691613518075
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. Chick, N
    . Metacognition. Vanderbilt University Center for Teaching. Published 2013. Accessed Jun 30, 2021. https://cft.vanderbilt.edu/guides-sub-pages/metacognition/
  19. 19.↵
    1. Hatala R,
    2. Gutman J,
    3. Lineberry M,
    4. Triola M,
    5. Pusic M
    . How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation. Adv Health Sci Educ Theory Pract. 2019 Mar; 24(1): 45-63. doi:10.1007/s10459-018-9846-x
    OpenUrlCrossRef
  20. 20.↵
    1. Reinstein I,
    2. Hill J,
    3. Cook DA,
    4. Lineberry M,
    5. Pusic MV
    . Multi-level longitudinal learning curve regression models integrated with item difficulty metrics for deliberate practice of visual diagnosis: groundwork for adaptive learning. Adv Health Sci Educ Theory Pract. 2021; 26(3): 881-912. doi:10.1007/s10459-021-10027-0
    OpenUrlCrossRef
  21. 21.↵
    1. Kirkpatrick DL. and
    2. Kirkpatrick JD
    . Transferring Learning to Behavior: Using the Four Levels to Improve Performance. Berrett-Koehler Publishers; 2005.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 19 (5)
The Annals of Family Medicine: 19 (5)
Vol. 19, Issue 5
1 Sep 2021
  • Table of Contents
  • Index by author
  • Back Matter (PDF)
  • Front Matter (PDF)
  • The Issue in Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
THE AMERICAN BOARD OF FAMILY MEDICINE STRATEGY TO SUPPORT AND PROMOTE DIPLOMATE LEARNING
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
15 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
THE AMERICAN BOARD OF FAMILY MEDICINE STRATEGY TO SUPPORT AND PROMOTE DIPLOMATE LEARNING
David W. Price, Andrew Bazemore, Elizabeth G. Baxley, Keith Stelter, Thomas R. O’Neill, Roger Fain, Michael K. Magill, Warren P. Newton
The Annals of Family Medicine Sep 2021, 19 (5) 468-470; DOI: 10.1370/afm.2749

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
THE AMERICAN BOARD OF FAMILY MEDICINE STRATEGY TO SUPPORT AND PROMOTE DIPLOMATE LEARNING
David W. Price, Andrew Bazemore, Elizabeth G. Baxley, Keith Stelter, Thomas R. O’Neill, Roger Fain, Michael K. Magill, Warren P. Newton
The Annals of Family Medicine Sep 2021, 19 (5) 468-470; DOI: 10.1370/afm.2749
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Goals
    • Moving Forward
    • References
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Support for the WHO Resolution on Social Participation
  • Resident Leadership Roles and Selection
  • New Advocacy Ambassadors Program Helps AAFP Members Engage With Their Legislators
Show more Family Medicine Updates

Similar Articles

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine