Competencies for the Use of Artificial Intelligence in Primary Care

INTRODUCTION

The artificial intelligence (AI) revolution is here, and primary care clinicians must adapt.^1-3 This call for change follows the years of unrealized promise of electronic health records (EHRs), an era that has caused primary care clinicians to regard AI with skepticism.⁴ In addition to privacy and liability concerns, critics argue that AI can magnify existing biases, is not generalizable, and degrades over time.^5-8 These shortcomings underscore the need to train primary care clinicians to competently work with AI to advance the Quintuple Aim of better patient outcomes, population health, and health equity, at lower costs while preserving clinician well-being.⁹ To accomplish this goal, our workforce needs additional knowledge and skills so that AI can support the primary care functions of continuity, coordination, timeliness, and comprehensiveness.^3,10,11

Without training, this goal will not be achieved, risking harm instead of the intended benefits. AI-based tools will be deployed without rigorous evaluation and created absent specification for the unique needs of primary care. Patient safety will be compromised, and clinicians will become dissatisfied, leading to higher costs, greater fragmentation, and more burnout. To avoid this predicament, primary care clinicians must understand basic principles and have opportunities to practice with AI, similar to learning how to use a stethoscope or ultrasound. Training in AI is essential for primary care, the United State’s largest health care delivery platform.¹² Because of its coordinating function and whole-person approach, primary care synthesizes data across a fragmented health system. Interpreting these poorly organized data streams is demanding and a source of burnout.¹³ Given this central role,^12,14 groups such as the American Board of Family Medicine, the American Academy of Family Physicians, and the College of Family Physicians of Canada have sought to identify how AI can support primary care and have launched initiatives that bring together AI experts and primary care clinicians to take on these challenges.^15-17

Although training has attempted to facilitate technology-enabled primary care, more work is needed. In the United States, family medicine milestones call for residents to use information technology. Little has been done, however, to operationalize requirements that face important revisions in 2022.¹⁸ Following the expansion of virtual care during the COVID-19 pandemic, professional societies have published competencies to guide telehealth training.^19,20 The American Board of Artificial Intelligence in Medicine (ABAIM) recently launched certification in AI, and Mount Sinai started the Department of AI and Human Health.^21,22 Despite these developments, clinicians are calling for more training because they feel ill-prepared to thrive within this digital future.²³ In the United States and abroad, 3 out of 4 medical students think AI competencies should be taught in medical school.^24,25 Unfortunately, medical education has been slow to adapt. In one national study in Ireland, two-thirds of medical students received no training in AI, and over 40% had never heard of the term “machine learning.”²⁵

To fill this gap and pave the way for future curricula on AI for clinicians, we propose 6 competencies (Table 1) that build on those published by others²⁶ and describe how these competencies vary for different learners (Table 2).²³

First, learners need a foundational understanding of AI, including the types of tasks that are amenable to AI, appropriate areas in which to consider its application, and stages in its progression from development to implementation and regulation. They need to nest this understanding in a broader context, including AI’s impact on the role of physicians, the challenges of making clinical decisions with an abundance of data, and technology’s influence on clinician-patient relationships. This competency will provide learners with the language and background needed to complete higher-order tasks.

Second, learners need to develop critical appraisal skills tailored to the unique features of AI. Similar to new medications, tests, and programs, AI-based tools should undergo testing for accuracy, generalizability, effectiveness, and fairness. Although these concepts are already introduced through evidence-based medicine (EBM) curricula,⁶ appropriate selection and application of AI-based tools requires further understanding of its unique challenges, such as inconsistent performance across populations and performance degradation over time (“calibration drift”). Similarly, some AI approaches (such as neural networks) prioritize accuracy over explainability. This lack of transparency becomes important when AI misclassifies a patient, exposes them to unnecessary harm, and is unable to determine why the error was made. Likewise, awareness of the range of sources AI can draw from allows learners to appreciate what can happen when data are inaccurate, incomplete, or biased.²⁷

Third, learners need to understand how to incorporate these tools into medical decision making. For example, AI-based tools can now use smartphone cameras to make dermatologic diagnoses.^28,29 Training is needed to guide decision making when the patient’s lesion appears benign to the human evaluator after the AI-based tool identifies the lesion as malignant.³⁰ If these tools prove beneficial, their adoption has important implications for equity. Patients in resource-poor communities may lack access to the requisite technology,³¹ which can exacerbate disparities similar to how telehealth use varied during the COVID-19 pandemic.³²

Fourth, learners need the technical skills required to use AI-based tools in a manner that is effective and efficient. Furthermore, the technology needed to use AI-based tools will inevitably fail. When this occurs, clinicians need to know how to react. Otherwise, they will experience frustration, dissatisfaction, and loss of self-efficacy that contributes to burnout.³³

Fifth, learners need to understand how to communicate with patients regarding the use of AI-based tools. This includes explaining how and why the tools are being used, answering questions about privacy and confidentiality, and engaging in shared decision making. They also need to recognize the tools’ impact on clinician-patient relationships. Electronic health records have demonstrated that entering data during visits adversely affects the flow of conversation, attention paid to emotional issues, trust, and patient satisfaction.³⁴ Without adequate training, AI could create similar strains on relationships.

Lastly, the application of any technology comes with unintended consequences. When errors occur, biases are introduced, or disagreements arise, clinicians must understand how to adjust their reasoning and communicate relevant information. These limitations serve as an antidote to overconfidence. Just as learners study the limitations of diagnostic tests, they also need to appreciate how these tools contribute to probabilistic thinking as opposed to diagnostic and prognostic certainty. This domain is cross-cutting, as unintended consequences apply to the 5 competencies above.

When applying these competencies, several caveats need to be taken into consideration. First, training opportunities must be integrated across undergraduate medical education, graduate medical education, and continuing medical education (Table 2). This training is ideal for family medicine residencies extending to 4 years despite an already crowded training space.³⁵ For programs that remain at 3 years, training in AI can be integrated into existing sessions on health informatics or EBM. Second, these competencies will change over time and must be tailored to the local context. For example, we anticipate that AI will become more widespread, with tools progressing from the development to the evaluation, validation, and monitoring phases.³⁶ Furthermore, the specific AI-based tools presented to students may differ based on the prevalence of diseases, the high-priority problems, and the resources available within communities. Thus, learners do not need exposure to the breadth of available AI-based tools but rather to concepts and exemplars that can be applied to a wide range of clinical settings. Third, these competencies serve as point of departure, and more work is needed before integrating them into training. For example, subcompetencies need to be developed for primary, intermediate, and expert users, similar to the process underway to develop competencies for clinical informatics more broadly.^37,38 Ultimately, only a small percentage of primary care clinicians will become expert users, for it is not necessary to teach all clinicians to build machine learning models in the same way that it is not required for all clinicians to know the intricacies of how a magnetic resonance imaging machine works or how to run statistical software. Those who become expert users may benefit from additional training to ensure that their knowledge and skills are consistent with the current evidence base. Nevertheless, all primary care clinicians should know how to appraise and apply AI. While we have focused on family physicians, these competencies can also be a starting point for other primary care team members, including nurse practitioners, physician assistants, nurses, and psychologists. For AI to improve care, all health professionals who participate in the team-based care delivery will need additional training.³⁹

In 1991, Gordon Guyatt introduced the term EBM to highlight the need to integrate evidence into medical decision making.⁴⁰ Incorporating EBM into primary care required customization, with primary care educators emphasizing the importance of patient-oriented evidence, information mastery, and primary care research methods.⁴¹ While these concepts have informed medical decision making in primary care, one systematic review on EBM curricula found that no studies assessed the influence of these curricula on patient outcomes, that there was no validated tool to assess these curricula, and that a lack of EBM teachers is a barrier to broader dissemination of such curricula.^42,43 These findings highlight the need to adapt AI curricula to primary care. For example, new curricula need to be evaluated so that studies can track whether the use of the curricula affects burnout and AI knowledge, skills, and attitudes. Developing these curricula for primary care will require the involvement of clinicians, educators, informaticists, and AI experts. Some professional societies, such as ABAIM and the American Medical Informatics Association (AMIA), have foundational curricula that can be adapted for primary care. The challenges are real, but the potential payoff is substantial. Through thoughtful development of these competencies, the primary care workforce can use AI to ensure that this digital revolution realizes its potential for the benefit of patients, clinicians, health systems, and society.