Developing an AI Tool to Derive Social Determinants of Health for Primary Care Patients: Qualitative Findings From a Codesign Workshop

Study Sample

Participants were recruited through e-mail invitations circulated to all FHT clinicians, allied health care professionals, clinic staff, and medical residents. The FHT provides care across 5 practice sites in the downtown core of Toronto, Ontario, Canada’s largest city with an estimated population of more than 3 million people.⁹ It is part of a teaching network within the University of Toronto and serves approximately 50,000 patients, many of whom experience social and economic challenges.

Data Collection

We offered 4 workshops, each lasting 50 minutes, at the convenience of participants from May to June 2023; we held 3 in person at practice sites and conducted 1 virtually (Supplemental Appendix). The workshops were facilitated by a postdoctoral fellow embedded in the FHT with experience in primary care EHR data research (S.G.) and an FHT family medicine resident (S.L.).

The workshops began with an overview of the project, including a short tutorial on machine learning and details about ongoing work to test the use of AI to derive SDOH from EHR data. Participants briefly discussed what SDOH data were available and used for patient care. Next, a short, animated example was shown, demonstrating how the AI tool might function in a mock patient visit. This led into a brainstorming exercise that was guided by a sociotechnical systems framework for the application of AI in health care delivery¹⁰ and centered on 3 areas: (1) who are the people who should access and/or act on AI-derived social data, (2) what is the optimal process for integrating the tool into clinical workflow, and (3) what should the technology look like or achieve. Lastly, participants were asked about perceived barriers and facilitators when implementing this AI tool. We used a deliberative dialog technique to facilitate meaningful feedback that could be incorporated into the development of the AI tool.^11,12 A demographic survey captured participants’ personal and practice characteristics (gender, age, language, role, years in practice). Written consent was obtained from each participant before the workshop and a $25 gift card was provided as appreciation for participants’ time.

Data Analysis

A voice-recording device was used to capture audio from in-person workshops and Zoom (Zoom Video Communications, Inc) was used to record the online workshop. The audio data were transcribed using Otter.ai software (Otter.ai Inc), and the transcript was reviewed alongside the audio by one of the workshop facilitators (S.G.) to verify the accuracy and edit as needed. An inductive thematic analysis¹³ was conducted to classify the discussions and ideas generated from the workshops. The data were categorized independently by 2 study team members (S.G., S.L.), then an iterative process was used to ensure alignment, where the analysts would meet to discuss, compare, and reach consensus on codes and themes.

Coding and analysis of the transcribed text was conducted using NVivo 12 version 12.6.0.959 (QSR International). Qualtrics (Silver Lake Technology Management, LLC) was used for the electronic demographic survey.

This study was reviewed and approved by the Unity Health Toronto Research Ethics Board (REB No. 22-036).

How Do Participants Currently Access and Use SDOH Information?

Participants reported having access to some SDOH information in the personal history and/or Cumulative Patient Profile sections of the EHR, which is obtained by asking patients directly. SDOH data are also captured through the Health Equity Questionnaire⁶ and becomes part of the patient record. Participants reported, however, that patient social data were often inconsistent, difficult to search for, and incomplete for most. During patient intake interviews, clinicians generally asked about employment, education, social supports, immigration, and other determinants as appropriate. Most participants felt confident that gender identity and sexual orientation were kept up to date, but there was uncertainty about how frequently other SDOH were updated.

Participants spoke of the importance of having access to SDOH information for providing tailored and respectful patient care. As one clinician articulated, “You might not assume that they have a lot of, maybe, resourcefulness in terms of like … what kind of instructions you give them, how to proceed with the treatment or how to access other resources.”

There was a clear desire to have information about patients’ income, employment, and health insurance to inform prescribing decisions for those who may not have medication benefits (eg, choosing a lower-cost drug) or to provide free supplies, such as dressing changes for wound care. Knowledge of sexual orientation helped provide appropriate education, screening, and resources for sexually transmitted infections to their patients. Information on preferred spoken language was also seen as valuable to determine if interpreter services were required during visits, to prompt clinicians into giving shorter, simpler instructions to patients, or to allocate extra appointment time.

Implementation and Use of AI-Derived SDOH

As previously noted, our brainstorming exercise elicited participants’ preferences in 3 areas related to the implementation and use of an AI tool to derive social data: people, process, and technology. Preferences in these domains, which are based on a sociotechnical systems framework for the application of AI in health care delivery,¹⁰ are detailed below and summarized in Table 2.

Barriers and Facilitators

The barriers and facilitators for implementing and using the AI tool reported by participants are summarized in Table 3. Many concerns centered around biases in the data and AI, as well as maintaining patient confidentiality and trust. There was considerable worry that the AI tool could perpetuate discrimination or create racial biases, which might affect clinical decisions or communication with patients. Participants also mentioned the selection bias that could occur if patients were required to provide explicit consent for use of the AI tool—for instance, patients who might benefit from additional social resources may be less willing to permit the use of AI on their data.

Findings in Context

The codesign workshops described here provided a guiding path for our continued development of an AI tool for social data and eventual integration into the FHT’s practice. This “last mile of implementation” into real-world clinical settings is arguably the most difficult and remains largely unsolved, even with AI systems that are highly accurate.^14,15 Hesitation for implementing AI tools may stem from one of the predominant tensions that emerged from the workshop discussions, where AI was perceived as a time-saving tool that could alleviate clinician workload, but it was also anticipated to create additional burden on the practice. A number of questions still remain and will require further investigation, including workflow specifics, staffing considerations, and the patient verification process.

Participants in this study all mentioned concerns with data quality and the potential for bias; clinicians, as both data creators and users, are acutely aware of the possibilities and limitations of EHR data when used for secondary purposes. Given the current challenges with data quality and algorithms that are unlikely to perform perfectly, the ideal balance may require limiting the AI-derived SDOH data for aggregate, clinic-level purposes (eg, planning, resourcing) rather than using it for direct patient care. Patient self-report of SDOH should ideally continue, not only to support personalized care with individual-based solutions to social needs but also as a true reference standard for any secondary analyses in the future.

The challenges reported by participants are consistent with those in other studies on AI use in primary care, such as system readiness; ethical, legal, and social implications; and balancing the value of adopting AI against the large time and staff commitment.^4,5 Broad strategies to ameliorate these concerns include codesign/cocreation, iterative implementation, and continual evaluation.^4,5 More specifically, participants indicated their preference to start slowly with a small pilot project; to ensure regular communication between staff and researchers; to better define the workflow once a social need was identified; and to understand and address all resourcing implications.

Finally, participants expressed uncertainty about how to talk to patients about the use and outcomes of AI applied to their data and what a feasible consent model might look like. A recent review emphasized the necessity of moving machine learning models from ones merely being interpretable to ones that provide justifiability,¹⁶ for instance, rather than using AI to indicate whether a patient is experiencing a particular issue (eg, poverty), the tool should also provide the reasoning (eg, unemployment record found, key words identified in progress notes). This also highlights the importance of patient and clinician educational resources for AI and digital health, which could facilitate more informed conversations around the use of AI in clinical settings and possibly allay concerns about the technology itself.

Limitations

This study reflects the perspectives of one primary care group in Toronto, Ontario and may not be applicable to all primary care teams, health care settings, or regions. The patient-related barriers and facilitators were reported by participants based on their experiences and conversations with patients, and may not reflect concerns from patients themselves. Lastly, we likely had underrepresentation of social workers and other staff/allied health care professionals who are primarily responsible for addressing aspects of SDOH (eg, housing, food insecurity, transportation), which could affect the design of workflow processes.

Conclusions

The use of AI in health care settings is growing, with many possible applications and purposes. It is critical to engage and codesign with end users throughout the entire process of AI ideation and development. Our study highlights the preferences of one large urban academic family health team on an AI tool to derive social data for their patients. A future study is needed to formally evaluate the implementation of the AI tool once it is ready for deployment.