Abstract
PURPOSE This study aimed to assess how agile implementation–driven iterative processes and tailored workflows can facilitate the implementation of a digital cognitive assessment (DCA) tool for patients aged 65 years or older into primary care practices.
METHODS We used agile implementation principles to integrate a DCA tool into routine workflows across 7 primary care clinics. The intervention involved a structured selection process for identifying an appropriate DCA tool, stakeholder engagement through iterative sprints (structured, time-bound cycles), and development of tailored workflows to meet clinic-specific needs. A brain health navigator role was established to support patients with positive or borderline screenings, and assist primary care clinicians with follow-up assessment. We used the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework to evaluate the intervention’s performance over a 12-month period.
RESULTS The intervention engaged 69 (63.8%) of 108 clinicians across the 7 clinics. DCA screening was completed in 1,808 (10.8%) of 16,708 eligible visits. We selected the Linus Health Core Cognitive Evaluation tool as our DCA tool based on stakeholder evaluations. Screening workflows were tailored to each clinic. The brain health navigator received 447 referrals for further assessment of a positive or borderline screening result. Four clinics fully adopted the intervention, achieving a DCA completion rate of at least 20%, and 5 clinics were still routinely using the DCA tool at 12 months.
CONCLUSIONS Agile implementation effectively helped integrate the DCA tool into primary care workflows. Customized workflows, stakeholder engagement, and iterative improvements were crucial for adoption and sustainability. These insights can guide future efforts for early detection and management of cognitive impairment in primary care, ultimately improving patient outcomes and easing the burden on health care professionals.
- dementia screening
- primary health care
- digital cognitive assessment
- quality improvement
- workflow
- implementation science
- health informatics
- cognitive dysfunction
- Alzheimer’s disease
- neuropsychologic tests
- mild cognitive impairment
- practice-based research
- clinical practice patterns
- organizational innovation
INTRODUCTION
Alzheimer disease and related dementias affect approximately 6.7 million adults in the United States1 with cases among underserved populations such as non-Hispanic Black and Hispanic older adults (aged ≥65 years) disproportionately undiagnosed and undertreated.1-3 This gap in care is concerning given the suggested cost-effectiveness of early detection4-6 and advances in anti-amyloid therapies for Alzheimer disease.7-11
Primary care clinicians play a critical role in early detection12 but face barriers13-16 including a lack of validated screening tools embedded in electronic health records, inadequate reimbursement for cognitive assessments, time constraints, and insufficient training and confidence in diagnosing cognitive impairments.12,17-20
Limited awareness regarding the benefits of early intervention and modest therapeutic efficacy21 further impact detection efforts; however, collaborative care12 has improved quality of life for older adults with mild cognitive impairment, even before the advent of newer therapies.22 Real-world evidence on integrating screening into outpatient clinic settings is limited, highlighting the need for scalable solutions in primary care workflows.
Implementation science offers frameworks to bridge the research-to-practice gap by addressing barriers to translating evidence into routine care.23 Agile implementation applies principles from behavioral economics, complex adaptive systems theory, and network science to understand and “nudge” the behaviors of individuals and organizations toward successful implementation.24,25 It emphasizes rapid, iterative development and deployment of interventions according to an agreed-on cross-site minimum standard,26 allowing for the tailoring of interventions to the unique needs of specific clinical settings, which ensures both effectiveness and sustainability.24,25 Agile implementation identifies implementation champions27 locally through engagement of early engagers, a type of clinician persona.28-30 It has been shown to allow generalizability of a common standardized approach in the context of locally customized implementation in health care settings.24,25,28,31-34 This study evaluated the implementation of a digital cognitive assessment (DCA) tool into primary care using agile implementation. The objectives were to identify a tool acceptable to stakeholders, develop clinic-specific screening workflows, and optimize care pathways for diagnostic assessment of positive screenings.
Patient navigation in primary care is an essential approach for addressing barriers to care and bridging gaps caused by health care fragmentation.35 Navigation supports those with complex needs by connecting patients with resources and facilitating care coordination. To address clinician-identified barriers such as limited time, resources, knowledge, and confidence in evaluating patients for cognitive impairment,12,17-20 we also introduced a brain health navigator (BHN) role.
METHODS
Study Overview
We conducted a pragmatic clinical quality improvement project36-38 using agile implementation to integrate DCA into workflows across 7 diverse primary care clinics. Clinics were selected based on panel size of older adults (aged ≥65 years), demographic diversity, and variety of practice type and location. We used Area Deprivation Index scores,39,40 ranging from 1 to 100, to measure neighborhood disadvantage.
We developed and implemented the DCA screening intervention and then monitored outcomes over 12 months. Development had 4 key components: DCA tool selection, primary care leadership and clinician engagement, development of a workflow tailored to each clinic’s specific needs, and establishment of the BHN role. We identified clinicians exhibiting early engager characteristics27 and involved them early in the process.
This study was approved by the Indiana University Institutional Review Board as a quality improvement study (study no. 15325) and an evaluation exempt study (study no. 15281), with a waiver granted for informed consent.
Study Setting
The study was conducted at 7 Indiana University Health primary care clinics, from June 1, 2022, to May 31, 2023, as part of the Davos Alzheimer’s Collaborative Healthcare System Preparedness Early Detection Flagship program (hereafter Davos Flagship program). Their characteristics are shown in Table 1. Of the 7 clinics, 4 were suburban (including 1 value-based clinic) and 3 were urban (including 1 value-based clinic and 1 residency clinic). (A value-based clinic is a health care practice that prioritizes patient outcomes over the volume of services provided. These clinics use team-based care and payment models that reward quality and improved health rather than quantity.)
Baseline Characteristics of Participating Primary Care Clinics
Eligibility Criteria for Screening
Patients aged 65 years or older presenting for any primary care visit were eligible for DCA screening; however, to establish realistic goals, engage clinicians and staff meaningfully, and promote incremental improvement, clinicians and clinics were allowed to restrict eligibility to patients this age making specific types of visits. Through collaborative meetings at each clinic, we incorporated clinician and staff input to develop tailored criteria that aligned with the unique operational characteristics of that clinic. Depending on the clinic, eligible visits were certain types of visits—Annual Medicare Wellness Visits, routine nonurgent follow-up visits, or visits occurring during specific clinic sessions and times—or all visits by patients aged 65 years or older. Patients meeting the eligibility criteria for their clinic were flagged in the electronic health record system, prompting the medical assistant to perform the DCA screening during the visit. When an eligible patient was not screened, we classified the screening as missed, declined by clinician, or declined by patient.
Agile Implementation Process
The agile implementation process involved 8 key steps, detailed below.
1. Identifying Opportunities for Improvement
We formed an interdisciplinary team to participate in the Davos Flagship program consisting of primary care leaders, specialists in Alzheimer disease and related dementias, agile implementation specialists, DCA tool representatives, clinic physicians, clinical staff champions, and research assistants.
2. Selecting a DCA Tool
We designed a sprint (a structured, time-bound cycle), guided by the Delphi method,41 to select the DCA tool from those offered by 3 commercial DCA vendors approved by the Davos Flagship program.42 We defined 3 key stakeholder groups: research (Alzheimer disease and related dementias researchers); clinical (primary care clinicians and administrative leaders); and patient and medical assistant (Patient and Family Advisory Council members and medical assistants). We used discussions with key informants from each group to develop a list of potential aspects of the DCA tool or DCA workflow that would be of particular interest to that group. Before engaging with DCA vendors, stakeholders completed an online questionnaire to rank-order tool selection criteria based on the key informant list. Weighted rankings of the criteria are shown in Table 2.
Criteria for Digital Cognitive Assessment Tool Selection
Each stakeholder group received vendor information in advance and attended a 2-hour session that included 30-minute vendor presentations followed by question-and-answer segments. Stakeholders then assessed each vendor via an online evaluation, and the weighted decision matrix43 was used to score each tool against the defined criteria.
3. Developing an Evaluation Plan
We set process and outcome measures to evaluate the implementation. Tracking and reporting strategies were developed and implemented to support the implementation measurement plan.
4. Mapping Current Processes
A dedicated sprint was performed to gauge clinician engagement levels. We conducted preintervention surveys and interactive sessions to assess clinicians’ perceptions and preparedness, as well as to make inquiries into unique aspects of their practices influencing the screening process. The concept of a Minimum Viable Solution prototype was introduced to define essential non-negotiable elements in the screening process. Comprehensive feedback from all stakeholders and analysis of performance metrics were emphasized. We customized evidence-based solutions for each clinic, adjusting clinician schedules, designating medical assistant representatives, and identifying physical spaces for related activities.
5. Conducting Implementation Sprints
We conducted sprints to iteratively test the identified intervention specifications. These sprints included clinician protocol training sessions designed to familiarize clinicians with the DCA tool and its integration into routine practice, and team member DCA training focused on equipping medical assistants with the necessary skills to administer the DCA and manage workflows. In addition, workflow development was iteratively tailored to each clinic’s unique context, to ensure the processes were practical, efficient, and aligned with clinic operations. Weekly reviews and feedback loops during these sprints allowed for continuous refinement and ensured adherence to the Minimum Viable Solution prototype, thereby enhancing fidelity and scalability of the intervention across diverse primary care settings.
6. Monitoring Performance
Senior academic and health system leaders, representing primary care, neurology, and geriatrics, met monthly throughout the study period to address high-level gaps or barriers. Agile implementation coaches held a weekly huddle with the physician and staff champion at each clinic to respond quickly to challenges and opportunities. We tracked outcomes through run charts and weekly cross-clinic huddles to monitor patient eligibility and screening completion rates. Cumulative clinic and aggregate dashboard reports were used to discuss implementation progress with clinic and cross-clinic leadership.
7. Assessing Organizational Impact
We collaborated with clinic and primary care leaders to evaluate the intervention’s impact and to identify any unintended consequences. Clinic-specific performance reports on operational metrics and quality programs continued to perform at targeted performance levels.
8. Formulating a Standardized Operating Procedure
We developed a blueprint for implementing DCA screening across different settings based on achieved objectives.
Workflow Development
During preliminary discussions with primary care leaders about initiating DCA screening, it became clear that managing patients with borderline or positive screenings required a solution that did not increase clinicians’ workload. This led to conceptualization of the BHN role. The BHN, a registered nurse, conducted additional assessments, educated patients, provided community resources, assisted primary care clinicians with workups for reversible causes, and facilitated referrals. Clinicians and administrators provided input on BHN scheduling, location, case scope, and communication preferences. The additional assessments conducted included the Montreal Cognitive Assessment, the Patient Health Questionnaire, the Generalized Anxiety Disorder assessment, a review of laboratory and imaging results, and calculation of the anticholinergic burden score and the atherosclerotic cardiovascular disease score.
Data Collection and Measures
Baseline data were collected for the 12 months before the intervention, and outcomes were monitored during the 12 months after it was implemented. We gathered data from electronic health records, DCA screenings, and weekly cross-clinic huddles. Demographic data were captured to describe patient characteristics. Screening data included patient eligibility, screening completion, referral completion, and BHN follow-up actions. Huddle and meeting attendance were tracked for clinicians and staff. Electronic health record and screening-related data were entered into a REDCap database (Vanderbilt University and REDCap Consortium) for analysis.
We used the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to evaluate intervention outcomes across these 5 dimensions.44,45 This framework accounts for multilevel factors that influence implementation, and provides insights into the generalizability of the intervention, the conditions under which it was implemented, and the outcomes achieved.44
Analysis
We summarized descriptive statistics for baseline clinic characteristics as proportions. Screening completion rates were calculated for each week and the entire 12-month study period.
For the RE-AIM reach component, we assessed both clinician reach (defined as the percentage of clinicians engaged in the intervention) and patient reach (defined as the percentage of eligible visits in which DCA screening was completed). For the effectiveness component, we assessed selection of the DCA tool, development of a screening workflow, and number of positive screenings resulting in referrals to the BHN. For the adoption component, we determined whether clinics achieved full adoption (defined as a cumulative DCA screening completion rate of ≥20% during the 12-month period) or partial adoption (defined as a rate of <20%). For the implementation component, we assessed implementation fidelity, defined as adherence to the Minimum Viable Solution prototype workflow at the end of the study period. For the maintenance component, we ascertained continued DCA use 6 and 12 months after implementation, measured by the number of clinics still performing screening and the percentage of referrals to the BHN.
We conducted comparative analysis to explore differences in DCA screening completion rates based on clinician-specific factors, such as hours of sprint sessions attended and clinician engagement during implementation sprints. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc). Statistical significance was set at P <.05.
RESULTS
Reach
The DCA screening intervention engaged 69 (63.8%) of 108 clinicians across the 7 primary care clinics. Screening was completed in 1,808 (10.8%) of 16,708 eligible visits.
Demographics of the 1,722 unique patients who completed a DCA screening are detailed in an accompanying article.46 In brief, the patients had a mean (SD) age of 73.5 (6.0) years, 58% were female, and 79% were White. Their median Area Deprivation Index score was 51.
Effectiveness
After evaluating the 3 DCA tools, stakeholders selected the Linus Health Core Cognitive Evaluation tool,47 administered and scored using a tablet device and modeled after the paper-and-pencil version of the Mini-Cog tool.48,49 Although the developed workflows varied by clinic, the Minimum Viable Solution prototype was maintained across clinics, allowing integration of the DCA tool and the BHN into varied clinical settings. Agile implementation permitted clinics to tailor their workflows to either the complete Linus Health Core Cognitive Evaluation or the shorter Linus Health Digital Clock and Recall. The BHN received a total of 447 referrals for further assessment of a positive or borderline screening result. Detailed screening and follow-up results are reported in the accompanying article.46
Adoption
The DCA screening completion rate for unique eligible in-scope patients varied across clinics (Figure 1). Four clinics fully adopted the intervention, achieving a cumulative DCA screening completion rate of at least 20%: clinic 1 (26.5%), clinic 4 (29.3%), clinic 5 (47.6%), and clinic 6 (31.7%). The other 3 clinics showed partial adoption: clinic 2 (4.1%) and clinic 7 (1.4%), which participated from the start, and clinic 3 (12.0%), which joined 8 months into the intervention.
Cumulative Percentage of Unique Eligible Patients Within Scope Who Completed a DCA Screening Between June 1, 2022 and May 31, 2023, Overall and by Clinic
DCA = digital cognitive assessment.
Fully 87% of early engagers reported readiness to conduct screenings for eligible patients, and early engagers had the highest DCA screening completion rates. There was a moderate, though not significant, correlation of completion rate with the average hours of sprint sessions attended per clinician (r = 0.54, P = .22) and with clinicians’ engagement during implementation sprints (r = 0.61, P = .15).
Implementation
All clinics demonstrated fidelity to the Minimum Viable Solution prototype workflow, with consistency of implementation across clinics evident in the run charts. Feedback received during weekly huddles led to iterative adjustments. Table 3 summarizes clinic-specific adaptations.
Clinic-Specific Workflow Adaptations
Maintenance
Twelve months after the intervention began, 5 clinics showed sustained routine use of the DCA tool and the 2 other clinics showed periodic use. Clinicians valued the BHN’s support in managing the diagnostic pathway, with ongoing referrals to the BHN still occurring at 12 months. Availability of the BHN was noted as a contributor to maintaining use of the DCA tool in routine practice.
DISCUSSION
This study evaluated implementation of a DCA tool across diverse primary care clinics using agile implementation. The agile implementation approach enabled successful integration of the DCA tool into routine clinical workflows through structured, iterative, and customizable processes, addressing specific clinic needs.
Key Findings
Workflow Adaptability
Adaptability of the screening workflows to each clinic’s unique environment was crucial. This flexibility allowed clinics to overcome initial barriers such as clinicians’ lack of readiness and technical difficulties.
Clinician Engagement
Targeted engagement sprints proved effective. By identifying clinicians having personas more receptive to agile implementation (early engagers), we tailored strategies to address specific concerns and motivations, leading to improved local adoption across clinics. Future studies should evaluate additional factors related to clinician engagement including revenue, assessing the impact of increased visit complexity or reimbursement of specific Current Procedural Terminology billing codes in primary care.50
BHN Role
The BHN played a key role in managing positive and borderline screenings and reducing related primary care clinician workload. By facilitating additional assessments and ensuring appropriate follow-up, the BHN supported patients and caregivers in the diagnostic pathway and added considerable value for clinicians. Future research should evaluate this role’s effectiveness and reimbursement mechanisms for sustainability.
Patient Reach and Screening Completion
The intervention reached a substantial number of patients, with 1,808 DCA screenings completed during 12 months. Despite an overall screening completion rate of only 10.8%, the progressive increase in rate throughout the period demonstrated growing and sustained integration of the DCA tool into routine clinic operations at several clinics.
Cost-Effectiveness
Beyond the incremental costs of adding DCA screening to our existing primary care workforce, implementation required incremental resources including the BHN, agile implementation coaching and facilitation, and support for data collection and analysis. As part of the Davos Flagship program, DCA expenses were provided in-kind. The growing prevalence of dementia and its associated economic burden51,52 underscores the need to evaluate the cost-effectiveness of screening interventions. Previous studies suggest that dementia screening programs can be cost-effective.4-6 Future research should assess economic sustainability while delivering measurable clinical benefits.
Challenges
Initial Technical Barriers
Initial technical challenges, including DCA tool integration, were resolved through iterative adjustments and feedback mechanisms.
Staffing Shortages
Staffing shortages, particularly vacant medical assistant positions during the study period, impeded initial implementation efforts. Addressing these shortages through strategic workforce planning is essential for sustaining screening workflows and should be considered in future studies. Although the BHN supported patient assessment after a DCA screening, medical assistants and clinicians still faced added burden from additional screening tasks. Strategic evaluation of primary care workforce ratios and reimbursement models at the health system, state, and national levels is necessary to address the broader impact of primary care–based screenings.
Influence of Recommendations
The lack of a US Preventive Services Task Force endorsement for routine cognitive screening in older adults53 presented a challenge to clinician engagement. Despite recommendations from other professional organizations, such as the American Academy of Neurology, which advocates for annual cognitive screenings,54 clinician ambivalence persisted. In addition, patients’ reluctance to report memory concerns54 highlights the need for more robust evidence on the benefits of early cognitive assessment. A key component of guideline-supported screening is its practical implementation in routine care. This study offers insights into workflow integration and adoption of DCA tools through successful implementation.
Implications for Practice
Adoption of an agile implementation framework for implementing interventions in health care settings has demonstrated considerable potential to enhance both effectiveness and sustainability.24,25,28,31-34,55 The agile implementation model highlights the critical importance of adaptability and stakeholder engagement in successful implementation.
Limitations
This study has several limitations. First, the nonrandom selection of clinics and voluntary participation of clinicians may have introduced selection bias, affecting the generalizability of the results. Our study used quality improvement methods, which place greater emphasis on ongoing measurement in a real-world context but entail less emphasis on bias, less control of confounding variables, and less stringent population parameters than traditional research, consistent with other quality improvement methods.36 Second, variations in practice type, staffing, and available physical space for activities could have influenced implementation outcomes. Third, the 12-month study period limits evaluation of the intervention’s long-term sustainability and impact. Future studies should explore randomized designs and conduct cost-effectiveness analyses to ensure scalability and sustainability.
CONCLUSIONS
This study demonstrates successful implementation of a DCA tool in primary care using agile implementation principles. Key factors for adoption and sustainability included tailored DCA screening workflows and the integration of a BHN role, which contributed to a screening completion rate of 10.8%. Addressing staffing shortages and establishing sustainable revenue models are critical for ongoing implementation.
Our findings underscore the value of iterative improvement and stakeholder engagement in implementing health interventions. Future research should extend follow-up periods, expand to a broader range of clinical settings, and evaluate cost-effectiveness and the long-term impact on patient care and the health care system.
Acknowledgments
This Early Detection program was made possible by the support of the Davos Alzheimer’s Collaborative Healthcare System Preparedness program. Linus Health provided in-kind use of the Linus Health Core Cognitive Evaluation and Digital Clock Recall, and technical support for this project.
Footnotes
Annals Early Access article
Conflicts of interest: authors report none.
Funding support: This study was funded by the Davos Alzheimer’s Collaborative Healthcare System Preparedness Program.
Disclaimer: Linus Health did not have any role in the design of the study; the conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation of the manuscript; or the decision to submit the manuscript for publication.
- Received for publication June 20, 2024.
- Revision received January 14, 2025.
- Accepted for publication February 4, 2025.
- © 2025 Annals of Family Medicine, Inc.
