Developing an AI Tool to Derive Social Determinants of Health for Primary Care Patients: Qualitative Findings From a Codesign Workshop

The integration of Artificial Intelligence (AI) into primary care, particularly for deriving Social Determinants of Health (SDOH), represents a significant advancement in addressing health disparities. The study by Garies et al. explores the codesign process of developing an AI tool aimed at extracting SDOH data from Electronic Health Records (EHRs). While the approach is innovative, and provide several dimensions that warrant closer examination, particularly regarding the implementation, ethical considerations, and the potential impact on patient care.

Codesign Process and Stakeholder Involvement
The study’s emphasis on codesign, involving primary care clinicians in the development of the AI tool, is a commendable approach. Engaging end users in the design process ensures that the tool is tailored to meet the specific needs of those who will use it daily. This participatory design process aligns with best practices in health informatics, where user-centered design is crucial for the successful adoption of new technologies (1). However, the study could have benefited from a more diverse range of participants, including patients and other community stakeholders. This broader involvement would ensure that the tool not only meets clinical needs but also respects patient perspectives and social contexts, which are integral to the accurate interpretation of SDOH (2).

Ethical Considerations and Data Accuracy
The study highlights concern about the accuracy of AI-derived SDOH data and the potential need for validation with patients. This is a critical and a essential step, as the use of AI in healthcare raises significant ethical issues, particularly regarding data quality and the risk of perpetuating biases present in the training data. If not addressed, these biases (potential algorithm biases) could lead to misinterpretations of a patient’s social context, resulting in inappropriate care recommendations (3). The study’s recommendation for a pilot implementation focused on a few key determinants, such as income and housing, is a prudent step. However, this approach should be accompanied by robust mechanisms for patient feedback and data correction to mitigate these risks (4).

Implementation Challenges
The participants in the study have rightly expressed concerns about the additional staff time and resources required to implement the AI tool effectively. This is a significant consideration, as the integration of new technologies into clinical workflows often faces resistance due to the perceived increase in workload (5). The study’s suggestion to use AI output as a “check-in” prompt with patients is a practical solution, but it must be accompanied by adequate training and support for clinicians to manage these new tasks without compromising patient care (6). Furthermore, the study could have explored the potential for AI to streamline other aspects of care, thereby offsetting the increased workload associated with SDOH data management.

Impact on Patient-Clinician Relationships
One of the most compelling aspects of the study is the discussion on how AI-derived SDOH data might influence patient-clinician interactions. The concern that patients may react negatively to AI-generated insights about their social circumstances is valid, particularly if they feel that their privacy is being compromised or if the data are inaccurate (7). To address this, the study suggests that clinicians explain the AI’s role and the data it generates. This transparency is essential for maintaining trust, but it also highlights the need for AI literacy among both clinicians and patients. Educational initiatives should be part of the tool’s rollout to ensure that all parties understand how AI works and its limitations (8).

Final thoughts
The development of an AI tool to derive SDOH data for primary care has the potential to significantly enhance the delivery of personalized and equitable healthcare. However, the study by Garies et al. underscores the importance of a careful and thoughtful implementation process. By involving a diverse range of stakeholders, addressing ethical concerns, and providing adequate support for clinicians, the tool can be effectively integrated into primary care practices. As appropriately develop AI could enhance patient care in healthcare, such considerations will be crucial in ensuring that technological advancements translate into real-world benefits for patients and clinicians alike.

References:

Kushniruk AW, Borycki EM. Human, social, and organizational aspects of health information systems. Hershey, PA: IGI Global; 2008.

Greenhalgh T, Hinton L, Finlay T, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expect. 2019;22(4):785-801.

Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453.

Nelson AI. Data-driven healthcare: Applications and opportunities for AI. Health Aff (Millwood). 2020;39(9):1601-1608.

Rogers EM. Diffusion of innovations. 5th ed. New York: Free Press; 2003.
Kaplan B, Harris-Salamone KD. Health IT success and failure: Recommendations from literature and an AMIA workshop. J Am Med Inform Assoc. 2009;16(3):291-299.

Char DS, Shah NH, Magnus D. Implementing machine learning in health care-addressing ethical challenges. N Engl J Med. 2018;378(11):981-983.
George J, Bernstein J. How AI can improve patient engagement. Harv Bus Rev. 2020;May 22.