An Artificial Intelligence-Based Chatbot to Promote HIV Primary Care Self-Management: a Mixed Method Usability Study

Context: We developed MARVIN, an artificial intelligence-based chatbot to engage people with HIV in their primary care and support their HIV self-management.

Objective: To assess its usability and identify the barriers and facilitators to its acceptance.

Study Design and Analysis: A 4-week pilot study using mixed methods.

Setting: McGill University Health Centre (Montreal, Canada).

Population studied: People with HIV on regular treatment.

Intervention/Instrument & Outcome Measures: Participants were asked to have at least 20 conversations within 3 weeks with MARVIN on predetermined topics and then, to complete the Usability Metric for User Experience-lite (UMUX-lite) and Acceptability E-Scale (AES) surveys. Observed mean scores were compared with predetermined thresholds (68/100 and 24/30, respectively). Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Verbatim transcriptions were deductively coded using the constructs of the Consolidated Framework for Implementation Research. Barriers and facilitators were identified according to the four subconstructs of the Technology Acceptance Model (TAM): perceived ease of use, perceived usefulness, attitude toward use, and behavioral intention to use.

Results: From April to December 2021, 28 participants completed the questionnaires. Their mean age was 40.2 years (SD=11.7), most were male (n=24/28), and over half (n=15/28) preferred to communicate with MARVIN in English. Mean scores for the UMUX-lite and AES were 69.9 and 23.8, both were not significantly below their respective thresholds (p=.76 and p=.42). Nine participants were interviewed. Identified facilitators included user-friendliness, accessibility across devices, confidentiality with a sense of security, and reliability of the information provided. However, lack of topics and functions, limited comprehension, and lack of usage guidance and support were identified as barriers, along with its implementation on only a single platform, Facebook Messenger.

Conclusions: MARVIN is easy to use, useful, and acceptable as a self-management tool for People with HIV. The qualitative results highlight the enhanced accessibility of relevant information and sense of interaction and safety using MARVIN as facilitating its usability and acceptance, while the quality of information provided, and the technology’s adaptability are factors that require further attention.