Abstract
PURPOSE We assessed the feasibility and acceptability of implementing a digital cognitive assessment (DCA) for Alzheimer disease and related dementias (ADRD) screening into primary care. We also assessed the prevalence of positive screens and measured diagnostic and care outcomes after a positive DCA result.
METHODS We conducted a single-arm pragmatic clinical demonstration project in 7 diverse primary care clinics to test implementation of the Linus Health Core Cognitive Evaluation and Digital Clock and Recall DCAs (Linus Health, Inc). Eligible patients were aged ≥65 years. Patients were ineligible if unable to see or hear, not English or Spanish speaking, or if they had a DCA in the past 12 months with an unimpaired or impaired result.
RESULTS There were 16,708 eligible encounters during the 12-month study period (June 2022-May 2023). A total of 1,808 DCAs (10.8%) were completed by 1,722 unique patients; 3,727 (22.3%) declined, and at 9,232 encounters (55.3%) the physicians declined to have the patient complete the DCA or the encounter was deemed out of scope. Among those who completed DCAs, results for 762 (44.3%) were categorized as unimpaired, 628 (36.5%) borderline, 236 (13.7%) impaired, and 96 (5.6%) inconclusive. Among the 236 patients who were categorized as impaired, 2.1% received a new diagnosis of ADRD, and 5.1% received a new diagnosis of mild cognitive impairment within 90 days after the DCA.
CONCLUSIONS One-half of all patients scored impaired or borderline for cognitive impairment. Digital cognitive assessments can be implemented in primary care, have utility for early detection, and could represent the first step in identification of patients who could benefit from ADRD disease-modifying therapeutics, care management, or other interventions to improve patient and family caregiver outcomes.
INTRODUCTION
A missed or late diagnosis of Alzheimer disease or a related dementia (ADRD) can lead to poor health outcomes including worse behavioral and psychological symptoms, mismanagement of chronic comorbidities, increased care utilization, and missed opportunities to participate in advance care planning and ADRD research such as clinical trials.1-5 For family caregivers, a late diagnosis of ADRD increases caregiver stress and burden and decreases opportunities to prepare for caregiving.1 In primary care, a lack of workflows and care processes that support timely detection and diagnosis of ADRD limits older adults’ ability to be assessed for new pharmacologic treatments6 and increases the disparities of ADRD diagnosis and treatment by race and socioeconomic status.5,7
Most Americans with ADRD receive a majority of their health care from primary care physicians (PCPs) and have limited access to specialists for diagnosis and management of ADRD.8 Despite requirements for screening/detection of cognitive impairment during the Medicare annual wellness visit, less than one-third of older adults report receiving an assessment of cognition by their PCP,9,10 and it is estimated that more than one-half of older primary care patients who meet diagnostic criteria for ADRD never receive a diagnosis, with high rates of undetected cognitive impairment in underrepresented minoritized groups and socially vulnerable older adults.3,11,12 When ADRD is detected earlier, PCPs can evaluate for potential causes of cognitive impairment, both reversible and syndromic, and facilitate advance care planning, improve patient and caregiver preparedness related to disease progression and prognosis, connect patients to home and community-based services, and initiate behavioral and pharmacologic treatment.13,14
Research on the risks and benefits of early detection of ADRD in primary care has shown that patients do not experience adverse psychological reactions, such as depression or anxiety, from screening yet benefits are difficult to measure when the screening program is not coupled with an evidence-based diagnostic assessment that is feasible and acceptable to primary care patients, their family members, and clinicians15,16 Interventions to improve the detection and diagnosis of ADRD have focused on improving clinician knowledge and confidence in making a diagnosis and the general public’s knowledge regarding ADRD.13,17-22 Whereas knowledge of ADRD has improved, the effect of these informational interventions on timely ADRD diagnosis remains unknown.17-22 In addition, patients have shown a high level of acceptance of screening for cognitive impairment in primary care.23
With the data showing worse outcomes for patients with late diagnoses and the emergence of monoclonal antibody therapies indicated for those with mild cognitive impairment (MCI) or mild-stage Alzheimer disease,24,25 the importance of testing strategies for early identification of patients in primary care with or at risk of developing ADRD is more critical than ever. Yet, no clinical practice guidelines or standard workflows exist to implement cognitive screening in routine primary care for care, treatment, or research recruitment. Therefore, the development, implementation, and evaluation of pragmatic processes to identify patients with ADRD in the primary care setting is essential.
From June 2022 to May 2023, Indiana University and Indiana University Health comprised 1 of 7 sites globally to participate in the Davos Alzheimer’s Collaborative Healthcare System Preparedness Project Early Detection Program as a clinical demonstration initiative to evaluate and measure the feasibility and acceptability of implementing a digital cognitive assessment (DCA) into routine primary care for older primary care patients.
METHODS
Study Design
The primary goal of this single-arm, pragmatic clinical demonstration project was to evaluate the feasibility, acceptability, and preliminary patient outcomes of integrating a DCA as part of standard workflows in 7 diverse primary care clinics. We also measured the prevalence of screening for borderline cognitive impairment and cognitive impairment and assessed patient outcomes 90 days after screening. The study was approved by the Indiana University Institutional Review Board. A waiver of informed consent was granted for this pragmatic clinical demonstration project.
Participants
Clinics
The 7 participating primary care clinics are part of a large statewide academic health care system and were selected to represent suburban and urban locations and a racially and socioeconomically diverse sample of primary care patients.
Patients
All patients were eligible to complete a DCA if they were aged ≥65 years and presented to one of the participating primary care clinics, regardless of the purpose of the clinic visit. Patients who scored borderline for cognitive impairment and had another encounter in the 12-month project period were eligible for reassessment. Patients were ineligible to complete the DCA if they were unable to see or hear well enough to complete the DCA, did not speak English or Spanish, or had a DCA in the past 12 months with an unimpaired or impaired result.
Description of the Digital Cognitive Assessment Tool
Primary care physicians, clinic staff, members of the Primary Care Patient and Family Advisory Council, and researchers involved in the project participated in a Delphi process to inform the selection of which DCA tool would be used for the project. A detailed description of this process has been described elsewhere.26 The Linus Health Core Cognitive Evaluation (CCE) (Linus Health, Inc) was the DCA selected for this project. The CCE is a tablet-based cognitive assessment consisting of the Digital Clock and Recall (DCR) tool, which includes a 3-word immediate recall, the Digital Clock Drawing Test (DCTclock), delayed recall of the 3 words,27 and the Life and Health Questionnaire, which provides qualitative data to the clinician regarding a patient’s current cognitive status and future dementia risk.28 The DCR portion can be administered independently, is scored the same as the CCE, and has shown strong psychometric properties for identifying individuals with possible MCI and early dementia compared with the Mini Mental State Examination27 and Mini-Cog instrument.29,30 Validation studies that included multimodal models incorporating graphomotor, memory, and voice and speech features captured during the process of completing the DCR showed strong classification of verbal memory impairment (area under the curve 0.83, sensitivity 0.81, specificity 0.80)31 and cognitive impairment (area under the curve 0.89).32
Intervention and Procedures
We used agile implementation science33 methods to design the workflows for implementing the DCA at each clinic and assess the facilitators for and barriers to implementing the DCA as part of routine primary care for older adults. Details of the methods and implementation processes for this project are described elsewhere.26 In brief, before starting the project, the clinics participated in a series of structured, time-bound sessions termed sprints to define the minimally viable solution for implementing the DCA at each clinic and what would be defined as in and out of scope for implementing the DCA at each clinic. The topics of the sprints included (1) the timing of when the DCA would be administered during an encounter/visit, (2) the types of encounters/visits for which the DCA would and would not be administered, (3) specific days of the week or appointment times during the day when the DCA would and would not be administered, and (4) who in the clinic would present the DCA to patients (Table 1 summarizes differences in the ways the clinics placed DCA administration).
Examples of Implementation Differences by Clinics, Which Defined Out-of-Scope Encounters
The placement of DCA administration and what was defined as out of scope for clinics provided insights into the adaptability of the intervention to various clinical settings and informed the potential for broader applicability. For example, as clinics integrated the DCA into their workflows, some adapted their workflow by switching from the Linus Health CCE to the DCR portion to shorten the test by excluding the Life and Health Questionnaire. Clinics also had the opportunity to determine how patients who screened positive on the DCA received follow-up care. Given the pragmatic design of the project, patients had the opportunity to refuse the DCA or not complete it if started, and clinicians had the opportunity to decline offering their patient the DCA at any encounter.
For patients who completed the DCA, scores ranged from 0 to 5, with 0-1 indicating cognitive impairment, 2-3 as borderline for cognitive impairment, and 4-5 indicating no cognitive impairment. The delayed recall subtest contributes 0-3 points to the total score based on the number of words correctly recalled. The DCTclock subtest contributes 0-2 points based on a transformed summary score ranging from 0 to 100, with cutoff scores of <60, 60-74, and ≥75 contributing 0-2 points, respectively. The DCTclock includes 4 Command Clock and Copy Clock composite scales evaluating simple and complex motor skills, drawing efficiency, spatial reasoning, and information processing speed.34 In some instances, DCAs are unable to be scored and are considered inconclusive. The vast majority of these are due to insufficient number of strokes during the DCTclock (ie, no detectable clockface, <2 nonnoise strokes in the drawing).
Before starting the project, PCPs received care pathways to direct follow-up care based on the patients’ DCA results; however, because this was a pragmatic demonstration project, PCPs could proceed with patient care at their discretion. During month 3 of the project, the study introduced a new clinical staff member, the Brain Health Navigator (BHN), to assist the clinics. This registered nurse position was added given the overwhelming feedback from clinicians who wanted additional support for completing the care pathways when a patient was flagged for cognitive impairment. Details regarding this role and training are described in the companion article describing the implementation process.26
Data Collection and Measures
Outcomes
Data regarding cognitive assessment attempts were collected for every eligible patient for each eligible encounter. In addition, data were collected for all missed approaches, ineligible patients, patients who refused to complete the DCA, clinicians who refused to have their patients complete the DCA, and complete and incomplete cognitive assessments. All data regarding approaches were collected by research personnel embedded in the clinics and recorded into a secure Research Electronic Data Capture database (REDCap; Vanderbilt University). Reports produced by the DCA were downloaded from the Linus Health portal and added to the patient’s electronic health record (EHR) by research staff to simulate EHR integration of the DCA.
Feasibility was defined as ≥50% of eligible patients approached completing a DCA and measured by calculating the number of patients eligible to be approached during the project period, the number who were approached or missed, and completion of the DCA with enough data to score the assessment. Acceptability was defined as ≥50% of eligible patients approached agreeing to complete the DCA and measured by the number of clinicians who agreed to have their patient be approached to complete the DCA and the number of patients who agreed to complete it.
Information from the patient’s EHR, including age, biologic sex, race, ethnicity, 9-digit zip code, and comorbidities was collected at the time of approach. Patient and clinician outcomes, including number and percent of patients who screened borderline impaired or impaired, new diagnoses of ADRD or MCI, and physician referrals and orders related to an ADRD diagnostic assessment in the 90 days after completion of the DCA, were assessed. Specifically, we measured any orders for laboratory tests for thyroid-stimulating hormone (TSH) or serum vitamin B12, individually or combined, at any point during the 90 days after the DCA. In addition, any referrals to specialists (geriatrics, neurology, neuropsychology, psychiatry) or the BHN or orders for brain imaging (computed tomography, magnetic resonance imaging, positron emission tomography, magnetic resonance angiography) of the head and neck, brain, or skull at any point during the 90 days after assessment were collected.
Statistical Analyses
Descriptive statistics for patient social and demographic characteristics are presented as No. (%) for categorical variables and mean (SD) for normal continuous variables. These descriptive statistics were categorized by performance on the DCA (unimpaired, borderline, and impaired). Patient characteristics were compared according to completion of a DCA using the χ2 test for categorical variables and 2-sample t test or Wilcoxon rank sum test for continuous variables. We also conducted a logistic regression to determine the association of patient characteristics with completion of a DCA. We used the χ2 test for categorical variables and 1-way analysis of variance or Kruskal-Wallis test to determine if patient characteristics differed between those who had a DCA with a score of 0-1 (impaired) vs 2-3 (borderline) vs 4-5 (unimpaired). In addition, data regarding diagnostic assessment referrals and care 90 days after screening were calculated and compared using the Fisher exact test for patients who screened borderline vs impaired. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc). A P value <.05 was considered statistically significant. Given the descriptive nature of this report, corrections were not applied for multiple comparisons.
RESULTS
During the 1-year project period, there were 17,894 primary care encounters across the 7 clinics (Figure 1). Of those encounters, 16,708 (93%) were for patients who were eligible to be approached for a DCA. For those eligible encounters, 1,808 (10.8%) cognitive screenings were completed. For the eligible encounters in which a patient was approached, 3,727 (22.3%) patients declined to complete the DCA. In addition, during 9,232 (55.3%) eligible encounters, either clinicians declined to have the patient complete the DCA during that clinic visit (for various clinical reasons) or that encounter was deemed out of scope by the predetermined DCA implementation workflow developed and adapted by the clinic.
Feasibility and Acceptability of Digital Cognitive Assessment Implementation
DCA = digital cognitive assessment; PCP = primary care physician.
a For each encounter, the PCP could decline to have the patient complete the DCA, and as different clinics rolled out the DCA screening, some encounters were defined as out of scope on the basis of clinic-specific workflows and perceived system constraints.
b Missed: patient was eligible to complete the DCA but was not approached.
c Incomplete: patient started the DCA but did not complete enough of the test to produce a score.
Because this was a pragmatic study, some patients were approached more than once to complete the DCA if they had >1 encounter during the 12-month study period or had screened borderline on their first assessment. Therefore, the 1,808 assessments represent 1,722 unique patients; 76 completed the DCA 2 times, and 5 completed it 3 times. For these patients, their first completed assessment with a score was used to summarize patient characteristics by DCA result.
A comparison of eligible patients who completed a DCA vs those who did not showed that those who completed a DCA were significantly younger (73.5 [SD 6] years vs 74.7 [7.4] years; P <.001), lived in neighborhoods with lower Area Deprivation Index (51 vs 54; P <.05), and varied by race (P <.001) (Table 2). After controlling for clinic, age, and race, significant differences remained for those who completed a DCA vs those who did not (Table 3).
Demographic Characteristics of Unique Patients Approached to Complete a Digital Cognitive Assessment
Logistic Regression Comparing Patients Who Completed a Digital Cognitive Assessment vs Patients Who Did Not
Among the patients who completed a DCA, just under one-half (762; 44.3%) were unimpaired, 628 (36.5%) were borderline, and 236 (13.7%) were impaired (Table 4). An additional 96 patients (5.6%) completed the DCA, but the data elements needed to score the results, such as audio and video data collected during administration, were not of adequate quality or quantity for analysis and were considered inconclusive.
Patient Demographic Characteristics, by Digital Cognitive Assessment Result
As noted in Table 4, statistically significant sociodemographic and clinical differences were found between patients who screened not impaired, borderline, and impaired. Patients with impaired or borderline results were significantly older (76.2 [7.2] years vs 74.2 [6.2] years vs 72.1 [5.0] years; P <.001), had fewer years of education, were less likely to report their race as White, were more likely to live in neighborhoods with a higher Area Deprivation Index, and were more likely to have a higher Charlson Comorbidity Index score. In addition, patients whose DCA indicated cognitive impairment were more likely to report being a current or former smoker and/or have a diagnosis of hypertension, chronic kidney disease, diabetes, congestive heart failure, or obesity.
All usual standard of care follow-up options for diagnostic assessment, including orders for laboratory work, imaging, and referrals to specialists, were available to PCPs for patients who were screened. In addition, PCPs had the ability to refer patients with a DCA result of impaired or borderline to the BHN for additional assessment. Orders and referrals for testing and specialists were measured within 90 days after the first completed DCA. Among the 236 patients with a DCA result of impaired, 85 (36%) had a laboratory order for a vitamin B12 test, 84 (35.6%) had an order for a TSH test, and 40 (16.9%) had an order for any imaging of the head or neck (Table 5). Twenty-eight (11.9%) were referred to neurology, 8 (3.4%) to geriatrics, 5 (2.1%) to neuropsychology, and 0 to psychiatry. One hundred and forty-eight (62.7%) of the patients who screened as impaired were referred to the BHN. Five (2.1%) patients received a new diagnosis of ADRD, and 12 (5.1%) received a new diagnosis of MCI in their EHR. Six (2.5%) were started on a new prescription for donepezil, galantamine, rivastigmine, tacrine, memantine, or memantine/donepezil combination therapy.
Outcomes Within 90 Days After Digital Cognitive Assessment
Among the 628 patients with a DCA result of borderline, 139 (22.1%) had a laboratory order for a vitamin B12 test, 188 (29.9%) had an order for a TSH test, and 48 (7.6%) had an order for any imaging of the head or neck. Twenty-five (4%) were referred to neurology, 4 (0.6%) to geriatrics, 3 (0.5%) to neuropsychology, and 1 (0.2%) to psychiatry. Two hundred and ninety-nine (47.6%) of the patients who screened as borderline were referred to the BHN. Four (0.6%) patients received a new diagnosis of ADRD, and 9 (1.4%) received a new diagnosis of MCI in their EHR. Three (0.5%) were started on a new prescription for donepezil, galantamine, rivastigmine, tacrine, memantine, or memantine/donepezil combination therapy.
DISCUSSION
Primary care clinicians are at the front lines of identifying people who may have cognitive impairment due to MCI or ADRD. The evidence supporting early detection of ADRD in primary care is growing. For example, there are models of care that show improved patient and caregiver outcomes with early detection,35 and the window of eligibility for new disease-modifying therapies is narrow—and for patients in the earliest stages of cognitive impairment.6,24 In this large, pragmatic clinical demonstration project in 7 diverse primary care clinics, we found that during all primary encounters for patients aged ≥65 years, a cognitive assessment was completed 11% of the time. This measure of feasibility was lower than we hypothesized, especially given the number of encounters for patients aged ≥65 years and who did not have a diagnosis of ADRD. In instances when eligible patients were offered a DCA as part of the visit, they refused only 22% of the time, indicating a willingness to undergo cognitive assessment. Notably, this level of patient refusal for cognitive assessment is lower than reported values from randomized controlled trials in primary care settings that require informed consent to undergo routine cognitive evaluation. In addition, this was a clinical demonstration project of population-level screening, for which all patients were eligible, and the DCA cost was covered by the project. It is unknown how many patients would have declined a DCA if they had a copay or out-of-pocket cost for the DCA, or if the refusal rate would have been lower if a risk-stratification method was used to identity patients with subjective complaints or others who are most at risk of having undetected cognitive impairment.
Clinician engagement was much lower than anticipated, with greater numbers of clinicians declining to have their patient complete a DCA or by establishing workflows that deemed some encounters out of scope for screening. These results imply that at least one-third of eligible patients who are approached are willing to undergo a DCA if it is built into their routine primary care, yet clinicians have hesitation regarding whether the assessment can be incorporated into existing workflows within the scheduled visit.
We used agile science as the methodology for implementation and included input from clinicians, patients, and other clinic stakeholders regarding what screening assessment to use and what adaptations to clinic workflows would be most conducive to implementing the screening. A guiding theory for this approach is that for any implementation to be successful, there must be demand—by the patient, clinician, or system. Given that there were 19.5% of encounters that the clinician declined to have the patient screened, and 35.7% of encounters that the clinics deemed to be out of scope based on the clinic-specific workflows that they established, there appears to have been tension between the potential benefits of early detection for patients and that of a primary care system that is unprepared for widespread implementation of cognitive assessments in routine workflows. Details regarding the barriers to implementation are noted in our companion study,26 yet it is worth noting that the study team was actively engaged with the clinics throughout the 12-month project, which is not consistent with a typical initiative introduced into a primary care setting. This resulted in the team being highly responsive regarding adapting how the screening was implemented and responding to clinicians’ concerns (eg, creating the role of the BHN to assist clinicians with follow-up pathways after a DCA result of borderline or impaired). Completion numbers might therefore have been greater if this initiative were rolled out as standard of care, with payment to the practices, and aligned with primary care quality measures for older adults. Demand for widespread cognitive assessments will likely not increase until detection, as well as accessible evidence-based diagnostic assessments and longitudinal care for those with a diagnosis, are supported by payers and as a measure of quality primary care for older adults.
This study’s finding of 13.7% of patients testing as impaired is slightly greater than estimated rates of ADRD in populations aged ≥65 years. The most recent Alzheimer’s Association Facts and Figures5 report estimates that 11% of people aged ≥65 years have ADRD. Given that our results are based on screening alone and not diagnoses, we believe these values are generally comparable. In addition, 68% of Medicare-age Americans have ≥2 comorbidities36 and take an average of 5 prescriptions per year.37,38 Not surprisingly, patients who were older and who had more cardiometabolic comorbidities were more likely to screen impaired or borderline for cognitive impairment. It is difficult to differentiate from this study how many patients have cognitive impairment as a result of Alzheimer disease or another dementia etiology or other reasons not related to neurodegenerative disease. This was also a motivation for the introduction of the BHN role—to assist PCPs with risk-stratifying patients for follow-up and who might be best served by a referral to a specialist for a more thorough cognitive evaluation or a focus on comorbidity management and deprescribing.
Primary care physicians are ideally positioned to detect cognitive impairment, initiate diagnostic evaluations, and implement ADRD risk-reduction strategies. However, PCPs cannot do it all on their own. Health systems, health system leaders, payers, policy makers, professional societies, and other stakeholders must be involved to address barriers to implementing cognitive assessments and providing follow-up care. Examples of these barriers include lack of guidelines on cognitive assessment tools and workflows that support early detection, no or limited reimbursement for PCPs to assess cognition outside of Medicare annual wellness visits, and insufficient time, skills, and support for diagnostic assessment.
CONCLUSION
Digital cognitive assessments are feasible and able to be implemented as part of routine primary care but require additional resources and adaptations to workflows beyond the existing infrastructure in primary care. Without these embedded changes, clinician and patient demand for cognitive assessments will remain low. Almost one-half of patients scored impaired or borderline for cognitive impairment. Clearly, not all have ADRD, but DCAs have utility for early detection in primary care and could represent the first step of risk identification for patients who could benefit from further assessments, referrals to specialists, disease-modifying therapeutics, or other evidence-based care management interventions to improve patient and family caregiver outcomes.
Acknowledgments
This project was made possible by the support of the Davos Alzheimer’s Collaborative Healthcare System Preparedness Program. Linus Health provided in-kind use of the Core Cognitive Evaluation and Digital and Clock Recall tools as well as technical support for this project.
Footnotes
Annals Early Access article
Conflicts of interest: authors report none.
- Received for publication June 20, 2024.
- Revision received January 14, 2025.
- Accepted for publication January 20, 2025.
- © 2025 Annals of Family Medicine, Inc.