Abstract
PURPOSE Oral disease has a major impact on the overall health of US children, with dental caries being the most prevalent chronic disease in this age group. Given nationwide shortages of dental professionals, interprofessional clinicians and staff with proper training can influence oral health access. The American Academy of Pediatrics created the Oral Health Knowledge Network (OHKN) in 2018 to bring together pediatric clinicians via monthly virtual sessions to learn from experts, share resources, and network.
METHODS The Center for Integration of Primary Care and Oral Health partnered with the American Academy of Pediatrics to evaluate the OHKN in 2021. The mixed method evaluation included an online survey and qualitative interviews among program participants. They were asked to provide information on their professional role and prior commitment to medical-dental integration as well as feedback on the OHKN learning sessions.
RESULTS Of the 72 program participants invited, 41 (57%) completed the survey questionnaire and 11 took part in the qualitative interviews. Analysis showed that OHKN participation supported both clinicians and nonclinicians in integrating oral health into primary care. The greatest clinical impact was incorporating oral health training for medical professionals (cited by 82% of respondents), while the greatest nonclinical impact was learning new information (cited by 85% of respondents). The qualitative interviews highlighted the participants’ prior commitment to medical-dental integration as well as drivers for their current medical-dental integration work.
CONCLUSIONS Overall, the OHKN had a positive impact on pediatric clinicians and nonclinicians and, as a learning collaborative, successfully educated and motivated health care professionals to improve their patients’ access to oral health through rapid resource sharing as well as clinical practice change.
- oral health
- pediatrics
- integrated health care systems
- learning collaborative
- interdisciplinary research
- primary care
- patient care team
- organizational change
- health services
- access to care
- professional practice
INTRODUCTION
Oral disease is common and affects overall health profoundly. Dental caries is the most prevalent chronic disease among children in the United States despite decades of public and private initiatives.1,2 Oral disease is complex with origins that are biological, psychological, and sociological.3 Experts agree that an improvement in oral health will come about only if it is addressed in an interprofessional collaborative manner that emphasizes medical-dental integration (MDI).4
Currently, approximately 50% of children with Medicaid dental benefits visit a dentist annually.5 This number is lower for children who are Black or Hispanic, or financially challenged.6 Meanwhile, 90% of similar children had a preventive medical visit.7 Clearly, the medical setting is one where oral health promotion and dental disease prevention can be achieved; however, only 26% of pediatric residency program directors are satisfied with their graduating residents’ level of oral health competence, and less than 20% of pediatricians offer the preventive service of fluoride varnish to young children.8,9 Some states, health disciplines, and training programs are more advanced in their efforts than others; however, even in areas where progress is being made, efforts are inconsistent and well-defined results are lacking.10,11
In an attempt to bridge these gaps, the American Academy of Pediatrics (AAP) created the Oral Health Knowledge Network (OHKN) in March 2018. The objective was to increase the number of pediatric clinicians who provide preventive oral health services in practice or in MDI programs. Additionally, the premise of the OHKN learning sessions was to bring together pediatric clinicians interested in addressing oral health clinically with public health, dental, and education experts. The model was loosely based on the classic Project ECHO (Extension for Community Healthcare Outcomes) methodology.12 Participants would learn from and teach each other through their own experiences, while hearing from experts. One-hour virtual meetings occurred monthly, and each meeting had a theme. Participants were encouraged to ask questions, offer suggestions, and share personal experiences. Twenty-four meetings were held between March 1, 2018 and December 2020, with individuals joining the OHKN at different time points. The AAP collaborated with the Arcora Foundation who provided technical support.
The Center for Integration of Primary Care and Oral Health (CIPCOH) partnered with the AAP’s OHKN team to evaluate the program from January to November 2021. Funding for this evaluation was part of CIPCOH’s 5-year Health Resources and Services Administration cooperative agreement award. We report the conduct and findings of this evaluation.
METHODS
We used a mixed methods evaluation having 2 phases: a survey followed by semistructured interviews to explore survey data. Our evaluation received an exempt determination by the institutional review boards of Harvard University (protocol IRB17-0189) and the University of Massachusetts Medical School (protocol H00012069).
Survey
Our study population consisted of 72 participants from the AAP’s OHKN e-mail listserv. The AAP distributed our 25-item questionnaire in January 2021 after the team pilot-tested it with content experts and with non-OHKN AAP members.
Questions asked about participant demographics and practice characteristics; specific OHKN sessions attended and helpfulness ratings for each session; knowledge of MDI before and after attending the sessions; other activities participated in related to oral health; both clinical and nonclinical practice changes as a result of OHKN participation; barriers to attendance and to implementing practice changes; recommendations for improving OHKN sessions; and the likelihood of participating in future OHKN activities.
Participants were surveyed at a single time point (January 2021) rather than before and after each learning session. We collected data with Qualtrics survey software (Qualtrics Experience Management) and performed univariate analyses to compute descriptive statistics (eg, frequencies).
Qualitative Interviews
From 21 survey respondents willing to be interviewed, we selected a purposive mixed sample of 11 respondents (nonclinicians, nonmedical clinicians, and medical clinicians) for interviews. A semistructured interview guide was developed from previous research13,14 with minor modifications based on the 3 cohorts of interviewees (Supplemental Appendix 1, Supplemental Appendix 2, and Supplemental Appendix 3). The interview guides covered topics that included an overview of the participant’s professional role; involvement with and impact and evaluation of the OHKN; factors influencing MDI; barriers to integration activities; and suggestions for future OHKN sessions. Interviewees were also asked whether specific OHKN learning sessions led to clinical practice change(s).
Participants were recruited through e-mail, and telephone interviews lasting up to 60 minutes were conducted in July 2021. One author (T.J.) conducted all 11 interviews, which were audio recorded. Audio files were transcribed verbatim, the transcripts were reviewed for accuracy, and 2 authors (T.J. and C.A.R.) conducted a content analysis.
We developed a coding framework using the interview guides, discussions among the team members, and previous research.13 Once the 2 authors (T.J. and C.A.R.) reached consensus, parent codes and subthemes were collated and summarized. We did not ask quantitative questions during the interviews; however, we calculated the frequencies with which subthemes were mentioned to supplement the analysis.
RESULTS
Survey Findings
Of the 72 total OHKN attendees, 41 completed the survey questionnaire with sufficient information to include their responses (57% response rate). Participants reported having multiple job roles; 30% were clinicians and 70% had nonclinical roles (45% educators, 25% administrators, 18% researchers, and 47% “other” roles) (Table 1). The majority (43%) were from the East or Northeast of the United States, 80% were women, and 45% had been working in the health sector for more than 20 years.
Characteristics of OHKN Survey Respondents (N = 41)
Of the 24 sessions offered, 7 participants did not attend any; the number attended ranged widely from 0 to 24 (mean = 7; SD 7) (Table 2). Of the 34 participants who attended at least 1 session, more than one-half (59%) attended 1 to 8 sessions and 12% attended more than two-thirds of the sessions. When asked, using a 5-point Likert scale, how helpful the OHKN sessions were in integrating oral health into a primary care setting, 81% of participants said they were “extremely” or “very” helpful (Table 3).
Respondents’ Assessment of OHKN Engagement and Impact (N = 41)
Respondents’ Assessment of OHKN Sessions and Integration of Oral Health and Primary Care (N = 41)
The most commonly reported clinical changes prompted by OHKN participation included incorporating oral health training for the medical team (82%) and increasing fluoride varnish applications (73%) (Table 3). Nearly two-thirds (62%) of participants reported making at least 3 clinical practice changes, while more than three-quarters (82%) reported making at least 3 nonclinical practice changes (the most prevalent being engaging with others in MDI [97%], networking [94%], and obtaining further oral health training [88%]) (Tables 2 and 3). The most important motivating factors for participation in the OHKN sessions included networking (77%) and learning new information (85%).
Participants were asked to retrospectively rate their level of MDI knowledge before and after participating in the OHKN sessions. Although one-quarter (26%) of participants rated themselves as “very knowledgeable” and 70% rated themselves as “very” or “moderately” knowledgeable before attending sessions, these values increased after attendance to 50% rating themselves “very knowledgeable” and 95% rating themselves either “very” or “moderately” knowledgeable (Table 2).
The most frequently cited barriers to attending OHKN sessions included scheduling conflicts (67%) and competing priorities (48%) (Table 3). Only 5 respondents (12%) reported no barriers to attendance. Barriers to implementing change(s) in practice after attending the OHKN sessions included workflow (18%) and lack of institutional support (18%). One-quarter (26%) of respondents noted no barriers to implementation.
Participants were asked about any additional activities they participated in related to oral health knowledge. Only 8% noted none (Table 2). The remaining participants listed 2 to 4 activities; the most common activities included attending oral health conferences and attending oral health webinars (89% and 82%, respectively).
Among the 6 recommendations on how the OHKN sessions could be improved, about one-third of respondents each suggested more resource sharing (36%) and more networking events (33%) (Table 3). Nearly one-third (31%) were in favor of the ability for small group discussions. All attendees indicated they were likely to attend future OHKN sessions. When asked whether they were more likely to attend if continuing education credits were provided, only 26% responded “more likely.” When asked whether they would be more likely to attend if sessions were presented in a more traditional Project ECHO format,12 only 13% said “more likely.”
Interview Findings
We interviewed 3 nonclinicians (2 administrators, 1 public health practitioner), 1 medical clinician, and 7 nonmedical clinicians (2 dentists, 5 dental hygienists). These participants came from diverse work settings (eg, universities, private practices, and state agencies), had worked less than 5 to more than 20 years in the health sector, and represented all 4 regions of the United States. Five themes emerged from the interviews; each theme was further subdivided, as described below. The themes and subthemes are outlined in Table 4.
Themes and Subthemes From Qualitative Interviews (N = 11)
Impact of the OHKN
Overall, 91% of participants reported that the OHKN had a clinical impact; the most commonly mentioned impacts were future implementation of clinical outcome measures (50%) and an increase in fluoride varnish applications (36%). All participants indicated that the OHKN had a nonclinical impact; the most prevalent nonclinical impacts were peer-to-peer learning and knowing about other participants’ MDI activities (100% for both). Some cited the OHKN as influencing nonclinicians; one participant reported the OHKN helped her be “another voice in western Maryland” to advocate for MDI legislation. Slightly more than one-half of respondents (55%) reported having difficulty tracing a direct impact of OHKN on their work.
Prior Commitment to MDI
Nearly all interviewed participants (91%) reported that they were involved in MDI before joining the OHKN.
Evaluation of the OHKN
In evaluating the OHKN, participants were asked about strengths and benefits of the network, as well as areas for improvement. The majority reported the greatest strength was sharing information about MDI across the country, including success stories and failures. Furthermore, participants noted the e-mail listserv was very effective in the timely dissemination of new oral health guidelines or protocols. The regularity of meetings allowed for frequent communication, and the high “caliber of presentations” was noteworthy.
Participants suggested multiple improvements to the OHKN. They proposed organizing resources into a central location/website, expanding the network to include individuals from more states and internationally, providing continuing education credits, and incorporating more networking events/ informal discussions.
Drivers of MDI
Participants’ responses pertaining to the factors that drive MDI revealed 12 subthemes. The leading drivers were funding (cited by 100%), relationships (100%), buy-in (91%), finances (91%), and improved workflow/processes (91%) (Table 4). Patient demand was the least commonly mentioned driver (18%).
Another driver of MDI was oral health champions, who provided knowledge, motivated change, and advocated for the oral health of their communities. Box 1 presents a case study of an oral health champion’s story.
Case study: an oral health champion’s impact and experience with the OHKN.
Oral health (OH) champions were commonly reported as a driver for medical-dental integration (MDI) (64%). One champion’s story is highlighted below.
EC is a pediatrician and executive at a dental insurance company. Her interest in OH began in pediatric residency, where she witnessed rampant dental caries among her patients. As a result, she brought fluoride varnish (FV) into her practice in 2007 and has been disseminating OH information to her colleagues and residents ever since.
EC practices part time at an urban hospital that serves about 80% of patients receiving medical insurance assistance. The hospital has a pediatric dentistry department, with which her clinic shares electronic health records. Despite having a colocated dental clinic, only 50% of her patients seek care there, while the other one-half go to external clinics. EC rated her patients’ level of oral disease as fair to poor. When asked to rate her overall success of MDI, she gave herself and her team a 6 out of 10: “Everyone knows that we have a million excuses why we can’t do oral health… Our patient population (often has) many other challenges that… take precedence… (such as) homelessness, food insecurity, all kinds of issues… Mental health has taken center stage, so I think oral health is kind of the neglected younger sibling.”
EC joined the Oral Health Knowledge Network (OHKN) in 2018 because of her desire to join a group with a passion for OH. She credited organized systems and champions, like the OHKN and its members, for initiating and sustaining MDI efforts. She shared that the OHKN has taught her there are many ways to implement MDI with the goal of improving patient care. For example, one challenge EC endured was that despite the dental clinic being 1 floor away, “they might as well be 100 blocks away because patients don’t get there… physical adjacency is not enough.” The dental clinic reported seeing only about 5% of children aged 1 year when their model relied solely on referrals. As a result, her team implemented oral examinations, FV applications, and OH counseling during their 1-year well-child visits within their medical office. Then the next dental visit was scheduled in the dental clinic. Although it is anecdotal, EC reported this change as being successful in increasing access and a great lesson for the team.
Barriers to Oral Health Activities
Participants’ responses revealed 5 barrier themes: (1) overall barriers (eg, policies); (2) barriers to implementing something learned through OHKN; (3) barriers to implementing oral health activities (eg, need for training, reimbursements, buy-in); (4) barriers to maintaining oral health activities (eg, COVID-19 impact); and (5) how barriers were overcome (Table 4).
The most commonly mentioned implementation barrier was buy-in (cited by 91%), whether from leadership, staff, or clinicians. Meanwhile, the most commonly noted barrier to maintaining oral health activities was the COVID-19 pandemic, which led to delays or interruptions in oral health examinations and fluoride varnish applications. Additionally, 64% of participants reported national and state policies, such as limited dental benefits, interfering with oral health activities. Others reported that dental practice/scope of practice acts have created barriers to collocating dental professionals in primary care settings in some states. Several participants shared that the OHKN helped overcome barriers to integration. For example, one participant remarked that clinicians who were struggling to sustain oral health services benefited from technical assistance programs that proactively engaged with clinicians.
DISCUSSION
Overall, the OHKN was successful in bringing together pediatric primary care clinicians and nonclinicians to learn from experts, share resources, and network. Its positive impact is evidenced by the vast majority of respondents (81%) reporting the sessions helped them integrate oral health into primary care extremely or very well. Other important impacts included motivating participants to train clinical colleagues in oral health topics, prompting them to incorporate fluoride varnish into their practices, and networking within this group. Furthermore, the number of participants who reported they were very knowledgeable about MDI doubled with the sessions, and almost everyone felt at least moderately knowledgeable. Interviews confirmed that OHKN had both clinical and nonclinical influences. It should be noted the vast majority of participants did attend at least 1 other oral health event (eg, a conference or webinar) during the participation and did have some prior MDI experience, which may have contributed to the positive outcomes.
Addressing barriers to attending OHKN sessions may be difficult with some being beyond the control of the OHKN organizers (eg, competing priorities, time); however, offering additional times to meet may help with scheduling conflicts. Surprisingly, almost three-quarters of attendees said they would continue to attend without an offer of continuing education credits and with only minor changes to improve the sharing of resources and networking. The results also suggest that attending only approximately one-half of the sessions can still influence change, although we cannot rule out the possibility that change may have been related to oral health work outside of the OHKN and respondents did not clarify which events were the most likely to have influenced change.
Though independent of the barriers just mentioned, barriers to MDI and oral health implementation such as buy-in, changing practice acts, or funding could be areas that the OHKN focuses on in future sessions with specific recommendations. Our evaluation did not focus on potential solutions to address these barriers, such as providing incentives and training teams on workflows and implementation strategies, but that could be an area for future studies. Innovative solutions could stem from involving frontline workers in the decision process and being open to change.
Evidence shows that learning collaboratives and communities of practice have had mixed success. Positive outcomes are influenced by the type and intensity of facilitation, mode and frequency of communication, and governance structure.15 The OHKN has found a good balance of these factors with their monthly virtual meetings that are a combination of guest and participant presentations. The AAP has transitioned the coordination of the OHKN to the National Maternal and Child Oral Health Resource Center, and it is being rebranded as the Oral Health Learning Café. The center can use these lessons to make this learning collaborative even more effective.
Although our evaluation had a number of strengths that support our findings, some limitations must be considered. Our participants represented all regions of the United States and a wide spectrum of professional settings/backgrounds, but our evaluation may be limited in generalizability because of the nature of small samples. We were also able to interview only a few medical clinicians; however, we found that nonmedical clinicians and nonclinicians were critical to implementing workflows that impact MDI, indicating importance of team-based care. Furthermore, participants were surveyed only after they attended the learning sessions and not before, and only at a single time point rather than after each session, which may have introduced recall bias. Self-reported survey data as well as those obtained through interviews can be subject to information bias, often as a result of social desirability. Lastly, the clinical and behavioral changes we measured among participants were likely associated with both their OHKN attendance and their participation in other oral health activities and previous MDI experience. Future studies could evaluate participants’ oral health knowledge as well as assess knowledge and behavioral changes before and after each learning session.
In conclusion, a national learning network such as the OHKN can have a positive impact on clinicians and nonclinicians alike. Our findings show the benefits of learning from colleagues, sharing ideas and resources, and meeting on a regular basis to motivate and implement change. These findings could help the OHKN (future Oral Health Learning Café) be more effective in promoting MDI among pediatric professionals. Others should join this network (as well as other oral health activities/networks) to make improvements in their state that enhance the oral health of patients and populations.
Acknowledgments
The authors would like to thank Kate Sullivan, Lauren Barone, and Lorna Chiasson for their assistance during this evaluation.
Footnotes
Conflicts of interest: authors report none.
Funding support: This work was funded by the US Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA) under cooperative agreement UH1HP29962.
Disclaimer: The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS or the US Government.
- Received for publication June 17, 2022.
- Revision received September 5, 2022.
- Accepted for publication October 12, 2022.
- © 2023 Annals of Family Medicine, Inc.
