Technology-Enabled and Artificial Intelligence Support for Pre-Visit Planning in Ambulatory Care: Findings From an Environmental Scan
=====================================================================================================================================

* Laura M. Holdsworth
* Chance Park
* Steven M. Asch
* Steven Lin

## Abstract

**PURPOSE** Pre-visit planning (PVP) is believed to improve effectiveness, efficiency, and experience of care, yet numerous implementation barriers exist. There are opportunities for technology-enabled and artificial intelligence (AI) support to augment existing human-driven PVP processes—from appointment reminders and pre-visit questionnaires to pre-visit order sets and care gap closures. This study aimed to explore the current state of PVP, barriers to implementation, evidence of impact, and potential use of non-AI and AI tools to support PVP.

**METHODS** We used an environmental scan approach involving: (1) literature review; (2) key informant interviews with PVP experts in ambulatory care; and (3) a search of the public domain for technology-enabled and AI solutions that support PVP. We then synthesized the findings using a qualitative matrix analysis.

**RESULTS** We found 26 unique PVP implementations in the literature and conducted 16 key informant interviews. Demonstration of impact is typically limited to process outcomes, with improved patient outcomes remaining elusive. Our key informants reported that many PVP barriers are human effort–related and see potential for non-AI and AI technologies to support certain aspects of PVP. We identified 8 examples of commercially available technology-enabled tools that support PVP, some with AI capabilities; however, few of these have been independently evaluated.

**CONCLUSIONS** As health systems transition toward value-based payment models in a world where the coronavirus disease 2019 pandemic has shifted patient care into the virtual space, PVP activities—driven by humans and supported by technology—may become more important and powerful and should be rigorously evaluated.

Key words:
*   ambulatory care
*   artificial intelligence
*   qualitative methods

## INTRODUCTION

Similar to pre-flight checklists for pilots, experts widely recommend pre-visit planning (PVP) before patient encounters to enhance the quality, efficiency, and experience of care.1,2 Pre-visit planning can help optimize care for not-at-goal patients,3 improve interdisciplinary teamwork,4 and reduce no-shows.5 The American Medical Association (AMA) has established 10 steps for PVP before and between visits.2 In practice, clinicians may implement 1 or more steps and there is no direct evidence to distinguish between different methods. Pre-visit planning depends on human labor and many clinicians perceive it as costly or difficult to implement, resulting in patchy adoption of PVP recommendations.

Advances in technology-enabled and artificial intelligence (AI) tools may help drive the implementation of PVP.6,7 AI is defined broadly here as any information technology designed to mimic human cognitive function. Much PVP consists of well-defined data elements assembled on routine time schedules and highly structured communications between clinicians and patients—activities that are well within bounds for modern electronic health records and AI.

While appeal of AI has spawned numerous quality improvement projects, often their development takes place in the absence of rigorous needs assessments, which may yield AI solutions in search of a problem.8 A scoping review found relatively few new AI tools have undergone evaluations in real-world settings.9 Ignoring the complexity of health systems may not only delay realization of AI’s potential in health care but also misdirect investment.8

We sought to conduct just such a needs assessment for PVP. Using qualitative methods, we aimed to explore the current state of PVP, barriers to implementation, evidence of benefits, and the use of non-AI and AI technologies to support PVP.

## METHODS

This study received approval from the Stanford Institutional Review Board, eProtocol#51514.

### Design

We used an environmental scan approach to assess the state of practice of PVP and explore the use of non-AI and AI technologies to support PVP. Environmental scanning has been applied in primary care research, but originates from a business context in which data from a range of sources and methods, formal and informal, is gathered to inform strategic planning and decision making.10 Best practices for environmental scan methods in health services delivery are still emerging.11

Our process followed steps similar to those in public health and health services research reported elsewhere,12 and included: (1) selecting a team with content, skills, and subject interest; (2) defining the scope of the scan and key questions; (3) literature review; (4) key informant interviews; (5) search of AI and other technology-enabled tools; and (6) synthesis of findings from the literature and key informant interviews around our key questions.

The scan approach sought to address the following questions: (1) What is the state of the field for PVP? (2) What are potential solutions for PVP? and (3) Can non-AI and AI technologies be used to support PVP?

### Literature Review

We defined PVP using the AMA’s 10 steps as best practice as they have credibility for our intended audience of primary care clinicians and provided clear practice descriptions that we used to define selection criteria for the literature review.2 First, we carried out a literature review to explore the ways in which PVP has been implemented and its impact. Search terms included: pre-visit/previsit planning, visit preparations, visit plan/planner/planning, care gap, reminder, huddle, and handoff. PubMed and the National Library of Medicine databases were searched. Titles and abstracts were screened by 1 reviewer (C.P.). Full articles were reviewed for inclusion by 3 authors. When there was disagreement, articles were discussed by the group.

Articles were included if they described steps to achieve PVP—that is, an intentionally prepared office visit. Reference lists of included articles were reviewed for additional articles. Articles were imported into NVivo 12 Pro (QSR International) and coded by 2 authors (L.M.H., C.P.) for PVP steps, implementation issues, study design characteristics, and outcomes. Coded data was then displayed in a matrix such that each article comprised a row, themes were in columns, and coded data was presented in each cell.13

### Interviews

We sought primary care experts and PVP implementers to explore PVP and the potential for AI solutions. We carried out semistructured key informant interviews using a snowball sampling strategy in order to identify informants with deep knowledge.14 Initial informants were identified by 1 author (S.L.) and sent an e-mail request for an interview. Key informants were then asked to identify additional informants and send an e-mail to connect the informant with the researcher. Additionally, we invited corresponding authors from primary care–focused articles included in our review that had been published within the past 5 years to participate in an interview by e-mail.

Interview questions were developed following the literature review and loosely based on design-thinking concepts. They covered: what pre-visit planning means to you, perception of the current state of evidence and important outcomes, implementation drivers and barriers, and potential use of technology and AI. Interviews were conducted by telephone by a trained qualitative health services researcher (L.M.H.), recorded with permission, and transcribed for analysis. Interviews were coded using a priori codes from our focal questions and topic guide, while also searching for emergent themes. L.M.H. coded all interviews; C.P. second-coded a subset of 3 interviews to ensure alignment of coding and interpretation of themes. Coded interviews were then displayed in a matrix as described above for articles.

### Search of AI and Other Tools to Support PVP

We had previously performed an environmental scan, through the lens of primary care physicians, of the ways AI will impact primary care.6 With those findings as a starting point, we used a snowball strategy to search the public domain for technology-enabled and AI solutions that support PVP using terms similar to our literature search, plus the following terms: artificial intelligence, AI, machine learning, ML. To limit the scope of our search, we chose to exclude most population health management companies whose primary product was human-powered services enabled by data analytics, rather than the technology platform itself.

### Synthesis

Findings from the matrix analysis for the literature review and interviews were grouped together based on the focal question addressed. Areas of complementarity and divergence in findings between methods were described. We also considered how technology-enabled and AI tools found in our search aligned with findings emerging from these methods.

## RESULTS

We carried out the literature search in June 2019 and included 29 articles representing 26 unique implementations of PVP (Figure 1). We carried out interviews with 16 key informants from 13 unique organizations; 12 informants resulted from using the snowball approach on S.L.’s peer network (12 of 17 approached) and 4 informants identified from articles included in the literature review responded to a request for interview (4 of 12 approached) (Table 1). At the time of interview, 9 (56%) informants occupied senior roles (eg, director, president, vice-president, chief) in their organizations. An additional 3 (19%) informants had previously held senior roles. Interviews took place from August 2019 through February 2020 and lasted on average 37 minutes (range 23-54 minutes).

![Figure 1.](http://www.annfammed.org/https://www.annfammed.org/content/annalsfm/19/5/419/F1.medium.gif)

[Figure 1.](http://www.annfammed.org/content/19/5/419/F1)

Figure 1. 
Flowchart of literature search.

NLM = National Library of Medicine; PVP = pre-visit planning.

View this table:
[Table 1.](http://www.annfammed.org/content/19/5/419/T1)

Table 1. 
Key Informant Characteristics (N = 16)

### State of the Field for PVP

We sought to understand how PVP is defined, how common it is, and assess evidence of benefit or impact on outcomes (Table 2). Pre-visit planning was defined broadly, both by key informants and in the literature, and commonly framed around a goal of enhancing the efficiency of a clinical encounter. While typically focused on preparation for a clinic visit, it was recognized that patients have health care needs outside of visits, such as prescription filling or tests, and that preparation may be necessary across the care continuum:

View this table:
[Table 2.](http://www.annfammed.org/content/19/5/419/T2)

Table 2. 
Definitions, Benefits, and Outcomes of PVP Described by Key Informants

> “Pre-visit planning to me is really preparing to maximize the physical encounter or really wherever the patient is in that spectrum of care.” (Key informant 14)

Similarly, in the literature, most studies of PVP focused on activities in the period between visits, particularly on identifying care gaps, such as screening and vaccinations. Though many articles described processes sufficiently to ascertain their PVP process, others used the term pre-visit planning to indicate merely 1 step of pre-visit planning (eg, pre-visit questionnaires).

While the AMA’s guidelines might be considered a gold standard, no studies described implementation of all 10 practices. Instead, studies described different models of PVP, often implemented within a quality improvement framework and alongside other workflow or clinic process improvements tailored to local needs. This makes identifying a particularly effective model of PVP or which aspects of the process are most effective unlikely.

The perceived benefits of PVP for patients and clinicians aligned closely to the outcomes that informants expected to see from an effective PVP process; Table 2 describes the most important outcomes. Most outcomes were process related, such as quality metrics for screening or vaccination rates,5,15-24 time spent on various activities,25-28 or patient turnaround times.29 Patient outcomes such as A1C values, pain scores, symptom remission, or satisfaction with care have also been studied, and though satisfaction seems more likely to be improved from PVP, changes in clinical outcomes are less common. Identifying significant improvements for physicians has been difficult to demonstrate because the low number of clinicians per practice is typically too small to demonstrate effectiveness.

### Potential Solutions for PVP

In considering solutions for PVP, we identified the key features of PVP and evaluated how they have been implemented. Key informants consistently described PVP as identifying care gaps, such as vaccines needed, routine screening or laboratory tests, or managing medications (eg, opioids). Some also described a cycle of work in support of establishing and maintaining a trusting relationship with a patient so that efficient, effective care could be delivered in a continuous process.

In alignment with informant descriptions, most implementations of PVP in the literature (21 of 26 articles) focused on bridging care gaps (Supplemental Table 1, available at [https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2716/-/DC1](https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2716/-/DC1)). Pre-visit planning strategies most often took place between, rather than immediately before, appointments, and included reviewing charts, identifying and planning for care gaps, and reaching out to patients before a visit with a survey. Less common were activities on the day of an appointment, though key informants indicated that they thought huddling was likely more common than reported in studies, suggesting that some PVP practices may be routine and not described as part of an innovative practice change. Pre-visit planning practices identified in the literature mostly encompassed 2-316,18-22,24,25,28-33 of the AMA’s 10 steps while 5 studies reported just 1 activity (though these were substantive efforts to prepare the patient for a productive visit),15,26,34-37 and 9 studies reported 4 or more steps,5,17,23,27,38,39-42 with 8 being the most steps reported in 1 article.5 Pre-visit planning seems to be used primarily to ensure that health maintenance is up to date, current laboratory data are available, and patients have set priorities for their visit.

Both the literature and the interviews revealed a focus on a high-risk group (eg, patients with diabetes or those taking opioids),5,23,28,30,33-36 and/or quality metrics (eg, pneumococcal vaccination rates) as common strategies for implementing PVP.5,15-24 Most efforts with an evaluative element (14 of 24) described in the literature were implemented within a process improvement setting and utilized quality improvement, plan-do-study-act, or Lean methods. Such approaches seemed to be a facilitating factor for implementation as multiple improvement cycles were often utilized to develop a sustainable workflow.

Pre-visit planning was typically 1 part of an overall process improvement effort or larger complex intervention to improve clinic efficiency or physician satisfaction. In both the literature and interviews, PVP was described as a team process, typically having an MA or nurse review the chart, identify care gaps, and create orders to be signed by the physician. While there may have been some automation, such as an automatic request sent to patients to complete a questionnaire in a patient portal,16,34-36,40 PVP fundamentally involved human labor. Thus, often-described barriers to implementing PVP in both the interviews and literature included: lack of time in a busy environment, staff turnover or lack of staff to support physicians, need to educate patients and clinicians on the benefits of PVP, and difficulty changing established clinic workflows.

> “It’s just battling inertia. It’s just easier to go through your day the way you’ve always gone through your day.” (Key informant 09)

### Technology or AI to Support PVP

Key informants were asked whether they believed there was potential for technology or AI to support PVP. All participants perceived opportunity for aspects of PVP to be automated, such as querying the chart to identify care gaps. Most participants were interested in any solution which would help them save time in the clinic and enhance patient care. Several informants described various ways that technology already supported their work: higher-tech solutions such as a chat bot to check symptoms, lower-tech solutions like electronic sticky notes in the electronic health record, and best practice alerts to indicate overdue health maintenance. Similar examples of technology-enabled solutions were also described in the literature including: an algorithm that runs in the electronic health record and pulls quality deficits or produces a care gap summary,24,28,31 automatically sending questionnaires to patients via a patient portal,16,32,34-36,40 and automated systems to notify physicians of required actions.25,31

Our search of existing technology-enabled solutions identified several tools currently available (Table 3). These tools support components of PVP that are most amenable to automation, including identification of care gaps, appointment reminders, and pre-appointment questionnaires. Some tools leverage machine learning techniques (defined as the use of computer algorithms that improve automatically through experience) in their platforms, while others use traditional non-AI approaches. Some support multiple PVP activities, while others focus on just one. Few of these tools have been independently evaluated in the literature.

View this table:
[Table 3.](http://www.annfammed.org/content/19/5/419/T3)

Table 3. 
Examples of Commercially Available Technology-Enabled Solutions for PVP

Informants expressed both hope and skepticism that AI solutions would emerge in a way that would support PVP. Some informants described AI as becoming the norm in many fields such as marketing and retail, and that such applications in health care were inevitable, and may even be expected. This was balanced against the acknowledgment that workflow change is difficult, especially since physicians have personal preferences, and a sense that AI solutions have been anticipated for years without materializing. One informant commented that any developments needed to be equitable, without simply shifting the burden of work from the health care system to the patient:

> “I am very cautious about asymmetric use of technology in a way that saves one entity’s time at the expense of other humans. If an AI-generated conversation took me a lot of time and yet didn’t actually advance my care in any significant way, then that would feel burdensome to me as the patient.” (Key informant 02)

## DISCUSSION

Pre-visit planning is equated with providing the efficient, high quality care to which physicians aspire. Many different methods and approaches have been studied, but demonstration of impact is typically limited to process outcomes, with improved patient outcomes remaining elusive. While the AMA’s 10 steps for PVP may constitute a gold standard, no examples of perfect PVP implementation were identified. Instead, PVP tends to evolve to meet context-specific needs and resource constraints through process improvement methods. There are multiple barriers to achieving an efficient workflow for PVP, many of which are human effort–related, such as staff shortages or time constraints. This is where the potential of technology or AI solutions may improve the care process.

Interactions between health care clinicians and patients are relatively brief and episodic. Most time spent managing health is between visits. Pre-visit planning increases the chance that a patient visit will run more smoothly, take less time, and result in a higher quality and more satisfying experience for both the patient and the clinician. Artificial intelligence can automate many PVP activities, from appointment reminders and pre-visit questionnaires to pre-visit order sets and care gap closures. As health systems transition toward value-based payment models in a world where the coronavirus disease 2019 pandemic has shifted patient care into the virtual space, these inter-visit care activities—driven by humans and augmentable with AI—will become even more important, and potentially more powerful.45

While there is an emergence of technology-enabled and AI solutions to support certain components of PVP, few of these tools have been independently evaluated in the literature. By exploring the current state of PVP, identifying barriers to implementation and areas for potential automation, we found that there are clear opportunities for technology and AI to augment human-driven processes for PVP. Until the movement toward value-based care is more advanced, the lack of reimbursement for PVP will continue to be a key barrier. Human and technical challenges exist: for patients, convenience, privacy, and data security are chief concerns; for clinicians, electronic health record interoperability and workflow integration are paramount.45 Above all, these technology-enabled or AI innovations should be rigorously evaluated to ensure that they are always supporting—not subverting—the patient-physician relationship,6 which was highly valued by all informants.

### Limitations

We included studies of PVP from primary and specialty care in our literature review as key informants indicated that there were likely generalizable lessons across fields, although important differences, such as reimbursement, may make certain models of PVP more or less practical in primary care. The snowball sampling strategy for key informants may have limited our sample to peers with similar approaches to practice, and thus reinforced common themes; we attempted to mitigate this by interviewing authors of recent works identified in our literature search. Recruitment for interviews stopped when no further informants were forthcoming; we therefore limited our findings to areas where we perceived data saturation and have not presented data on emergent areas that were out of scope of our environmental scan and would benefit from further data collection, eg, PVP implementation strategies.

## CONCLUSION

Pre-visit planning is an essential element of population health management, with great potential to improve effectiveness, efficiency, and experience of care, yet there are numerous implementation barriers, many of which are human effort–related. There exist technology-enabled and AI tools for augmenting current human-driven PVP processes that hold promise of improving the delivery of primary care and they should be rigorously evaluated.

## Acknowledgments

Funding for the work was provided by the Division of Primary Care and Population Health, Stanford School of Medicine.

## Footnotes

*   Conflicts of interest: S.L. is the principal investigator of a research project sponsored by Google to better understand human factors that influence intra-visit care management. All other authors report none.

*   **To read or post commentaries in response to this article, go to [https://www.AnnFamMed.org/content/19/5/419/tab-e-letters](https://www.AnnFamMed.org/content/19/5/419/tab-e-letters)**.

*   **Funding support:** Funding for the work was provided by the Division of Primary Care and Population Health, Stanford School of Medicine.

*   **Previous presentations**: Poster presentation at North American Primary Care Research Group Annual Meeting; November 20-24, 2020; virtual meeting.

*   **Supplemental materials:** Available at [https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2716/-/DC1](https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2716/-/DC1)

*   Received for publication August 25, 2020.
*   Revision received January 15, 2021.
*   Accepted for publication March 15, 2021.


*   © 2021 Annals of Family Medicine, Inc.

## References

1.  1.Sinsky CA, Sinsky TA, Rajcevich E. Putting pre-visit planning into practice. Fam Pract Manag. 2015; 22(6): 34-38.
    
    

2.  2.10 steps to pre-visit planning that can produce big savings. American Medical Association. Published Sep 4, 2015. Accessed Jun 8, 2020. [https://www.ama-assn.org/practice-management/sustainability/10-steps-pre-visit-planning-can-produce-big-savings](https://www.ama-assn.org/practice-management/sustainability/10-steps-pre-visit-planning-can-produce-big-savings)
    
    

3.  3.Huebsch JA, Kottke TE, McGinnis P, et al. A qualitative study of processes used to implement evidence-based care in a primary care practice. Fam Pract. 2015; 32(5): 578-583.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1093/fampra/cmv045&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=26089298&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

4.  4.O’Malley AS, Gourevitch R, Draper K, Bond A, Tirodkar MA. Overcoming challenges to teamwork in patient-centered medical homes: a qualitative study. J Gen Intern Med. 2015; 30(2): 183-192.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-014-3065-9&link_type=DOI) 

5.  5.Rivo J, Page TF, Arrieta A, et al. The impact of comprehensive pre-visit preparation on patient engagement and quality of care in a population of underserved patients with diabetes: evidence from the care management medical home center model. Popul Health Manag. 2016; 19(3): 171-177.
    
    

6.  6.Lin SY, Mahoney MR, Sinsky CA. Ten ways artificial intelligence will transform primary care. J Gen Intern Med. 2019; 34(8): 1626-1630.
    
    

7.  7.Esteva A, Robicquet A, Ramsundar B, et al. A guide to deep learning in healthcare. Nat Med. 2019; 25(1): 24-29.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1038/s41591-018-0316-z&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=30617335&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

8.  8.Smith M, Sattler A, Hong G, Lin S. From code to bedside: implementing artificial intelligence using quality improvement methods. J Gen Intern Med. 2021; 36(4): 1061-1066.
    
    

9.  9.Kueper JK, Terry AL, Zwarenstein M, Lizotte DJ. Artificial intelligence and primary care research: a scoping review. Ann Fam Med. 2020; 18(3): 250-258.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTgvMy8yNTAiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzE5LzUvNDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. 10.Graham P, Evitts T, Thomas-MacLean R. Environmental scans: how useful are they for primary care research? Can Fam Physician. 2008; 54(7): 1022-1023.
    
    [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjk6IjU0LzcvMTAyMiI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvNS80MTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. 11.Charlton P, Doucet S, Azar R, et al. The use of the environmental scan in health services delivery research: a scoping review protocol. BMJ Open. 2019; 9(9): e029805.
    
    

12. 12.Wilburn A, Vanderpool RC, Knight JR. Environmental scanning as a public health tool: Kentucky’s human papillomavirus vaccination project. Prev Chronic Dis. 2016; 13: E109.
    
    

13. 13.Miles MB, Huberman AM, Saldaña J. Qualitative Data Analysis: A Methods Sourcebook. 4th ed. Sage; 2019.
    
    

14. 14.Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed methods implementation research. Adm Policy Ment Health. 2015; 42(5): 533-544.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s10488-013-0528-y&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=24193818&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

15. 15.Baker DW, Persell SD, Kho AN, Thompson JA, Kaiser D. The marginal value of pre-visit paper reminders when added to a multifaceted electronic health record based quality improvement system. J Am Med Inform Assoc. 2011; 18(6): 805-811.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1136/amiajnl-2011-000169&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=21659446&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

16. 16.Bose-Brill S, Feeney M, Prater L, Miles L, Corbett A, Koesters S. Validation of a novel electronic health record patient portal advance care planning delivery system. J Med Internet Res. 2018; 20(6): e208.
    
    

17. 17.Harris JG, Maletta KI, Ren B, Olson JC. Improving pneumococcal vaccination in pediatric rheumatology patients. Pediatrics. 2015; 136(3): e681-e686.
    
    

18. 18.Crandall W, Kappelman MD, Colletti RB, et al. ImproveCareNow: the development of a pediatric inflammatory bowel disease improvement network. Inflamm Bowel Dis. 2011; 17(1): 450-457.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/ibd.21394&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20602466&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 
    
    [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000285258500050&link_type=ISI) 

19. 19.Crandall WV, Margolis PA, Kappelman MD, et al; ImproveCareNow Collaborative. Improved outcomes in a quality improvement collaborative for pediatric inflammatory bowel disease. Pediatrics. 2012; 129(4): e1030-e1041.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTE6IjEyOS80L2UxMDMwIjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOS81LzQxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

20. 20.Garg S, Tsagaris K, Cozmuta R, Lipson A. Improving the combination pneumococcal vaccination rate in systemic lupus erythematosus patients at an adult rheumatology practice. J Rheumatol. 2018; 45(12): 1656-1662.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoianJoZXVtIjtzOjU6InJlc2lkIjtzOjEwOiI0NS8xMi8xNjU2IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOS81LzQxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

21. 21.Hills RL, Kulbok PA, Clark M. Evaluating a quality improvement program for cervical cancer screening at an urban safety net clinic. Health Promot Pract. 2015; 16(5): 631-641.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1177/1524839915587269&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=26032371&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

22. 22.Jean MR, Weaver A, Mastin-Diebold T, et al. Improving a process to obtain hepatitis B serology among patients treated with infliximab at a large urban children’s hospital. BMJ Open Qual. 2017; 6(2): e000092.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYm1qcWlyIjtzOjU6InJlc2lkIjtzOjExOiI2LzIvZTAwMDA5MiI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvNS80MTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

23. 23.Peterson KA, Radosevich DM, O’Connor PJ, et al. Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008; 31(12): 2238-2243.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czoxMDoiMzEvMTIvMjIzOCI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvNS80MTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. 24.Wilkinson C, Champion JD, Sabharwal K. Promoting preventive health screening through the use of a clinical reminder tool: an accountable care organization quality improvement initiative. J Healthc Qual. 2013; 35(5): 7-19.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

25. 25.Contratto E, Romp K, Estrada CA, Agne A, Willett LL. Physician order entry clerical support improves physician satisfaction and productivity. South Med J. 2017; 110(5): 363-368.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

26. 26.Hunt VL, Chaudhry R, Stroebel RJ, North F. Does pre-ordering tests enhance the value of the periodic examination? Study Design – Process implementation with retrospective chart review. BMC Health Serv Res. 2011; 11: 216.
    
    

27. 27.Drayton Jackson M, Bartman T, McGinniss J, Widener P, Dunn AL. Optimizing patient flow in a multidisciplinary haemophilia clinic using quality improvement methodology. Haemophilia. 2019; 25(4): 626-632.
    
    

28. 28.Kawamoto K, Anstrom KJ, Anderson JB, et al. Long-term impact of an electronic health record-enabled, team-based, and scalable population health strategy based on the chronic care model. AMIA Annu Symp Proc. 2017; 2016: 686-695.
    
    

29. 29.Allende-Richter SH, Johnson ST, Maloyan M, Glidden P, Rice K, Epee-Bounya A. A previsit screening checklist improves teamwork and access to preventive services in a medical home serving low-income adolescent and young adult patients. Clin Pediatr (Phila). 2018; 57(7): 835-843.
    
    

30. 30.Cox N, Tak CR, Cochella SE, Leishman E, Gunning K. Impact of pharmacist previsit input to providers on chronic opioid prescribing safety. J Am Board Fam Med. 2018; 31(1): 105-112.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzEvMS8xMDUiO3M6NDoiYXRvbSI7czoyMzoiL2FubmFsc2ZtLzE5LzUvNDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

31. 31.Savarino JR, Kaplan JL, Winter HS, et al. Improving clinical remission rates in pediatric inflammatory bowel disease with previsit planning. BMJ Quality Improvement Reports. 2016; 5: u211063.w4361.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYm1qcWlyIjtzOjU6InJlc2lkIjtzOjE3OiI1LzEvdTIxMTA2My53NDM2MSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvNS80MTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

32. 32.Wald JS, Businger A, Gandhi TK, et al. Implementing practice-linked pre-visit electronic journals in primary care: patient and physician use and satisfaction. J Am Med Inform Assoc. 2010; 17(5): 502-506.
    
    [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1136/jamia.2009.001362&link_type=DOI) 
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=20819852&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

33. 33.Wong R, Carroll W, Muttreja A, Garcia V, Taub E, Fernan A. Improving opioid management and resource utilization in an internal medicine residency clinic: a before-after study over two Plan-Do-Study-Act cycles. Pain Med. 2019; 20(10): 1919-1924.
    
    

34. 34.Grant RW, Uratsu CS, Estacio KR, et al. Pre-Visit prioritization for complex patients with diabetes: randomized trial design and implementation within an integrated health care system. Contemp Clin Trials. 2016; 47: 196-201.
    
    

35. 35.Grant RW, Altschuler A, Uratsu CS, et al. Primary care visit preparation and communication for patients with poorly controlled diabetes: A qualitative study of patients and physicians. Prim Care Diabetes. 2017; 11(2): 148-153.
    
    

36. 36.Vo MT, Uratsu CS, Estacio KR, et al. Prompting patients with poorly controlled diabetes to identify visit priorities before primary care visits: a pragmatic cluster randomized trial. J Gen Intern Med. 2019; 34(6): 831-838.
    
    

37. 37.Grout RW, Cheng ER, Aalsma MC, Downs SM. Let them speak for themselves: improving adolescent self-report rate on pre-visit screening. Acad Pediatr. 2019; 19(5): 581-588.
    
    

38. 38.Ho ET. Improving waiting time and operational clinic flow in a tertiary diabetes center. BMJ Qual Improv Rep. 2014; 5: 2(2).
    
    

39. 39.Cooper AM, Horwitz M, Becker ML. Improving the safety of teratogen prescribing practices in a pediatric rheumatology clinic. Pediatrics. 2019; 143(4): e20180803.
    
    

40. 40.Howard BJ, Sturner R. Use of an online clinical process support system as an aid to identification and management of developmental and mental health problems. Curr Dev Disord Rep. 2017; 4(4): 108-117.
    
    

41. 41.Bentti Vockell AL, Wimberg J, Britto M, Nye A. Using a parent coordinator to support the role of the pediatric nurse practitioner in care coordination. J Pediatr Health Care. 2018; 32(1): 36-42.
    
    

42. 42.Wooldridge AR, Carayon P, Hundt AS, Hoonakker PLT. SEIPS-based process modeling in primary care. Appl Ergon. 2017; 60: 240-254.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom) 

43. 43.Fixen DR, Linnebur SA, Parnes BL, Vejar MV, Vande Griend JP. Development and economic evaluation of a pharmacist-provided chronic care management service in an ambulatory care geriatrics clinic. Am J Health Syst Pharm. 2018; 75(22): 1805-1811.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpocCI7czo1OiJyZXNpZCI7czoxMDoiNzUvMjIvMTgwNSI7czo0OiJhdG9tIjtzOjIzOiIvYW5uYWxzZm0vMTkvNS80MTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

44. 44.Bergman DA, Beck A, Rahm AK. The use of internet-based technology to tailor well-child care encounters. Pediatrics. 2009; 124(1): e37-e43. doi:10.1542/peds.2008-3385.
    
    [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTI0LzEvZTM3IjtzOjQ6ImF0b20iO3M6MjM6Ii9hbm5hbHNmbS8xOS81LzQxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

45. 45.Lin S, Sattler A, Smith M. Retooling primary care in the COVID-19 era. Mayo Clin Proc. 2020; 95(9): 1831-1834.
    
    [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fannalsfm%2F19%2F5%2F419.atom)