Abstract
PURPOSE To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).
METHODS A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.
RESULTS A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.
CONCLUSIONS Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.
- primary health care
- information technology
- medical informatics
- quality of health care
- decision support systems, clinical
INTRODUCTION
Achieving best practice in primary care is a challenge because primary care professionals (PCPs) face a variety of health care issues and cannot always identify and access all the relevant information within the timeframe of the consultation.1,2 Clinical decision support systems (CDSSs) are software designed to be a direct aid to clinical decision making, in which an inference engine matches the features of an individual patient to a computerized clinical knowledge base or a machine learning algorithm and then presents patient-specific assessments or recommendations to the clinician or the patient for a decision.3,4 Clinical decision support systems are intended to improve the quality, safety, and efficiency of care.5-7 In primary care, they have not yet proven effectiveness on clinical outcomes, such as morbidity or mortality.8,9 However, according to a large recent meta-analysis of controlled trials in any settings, CDSSs increase the proportion of patients receiving the desired element of care by 5.8% overall, with a trend toward a worse outcome in the outpatient setting.10
Qualitative evaluations are needed to obtain a comprehensive understanding of the barriers and facilitators to CDSS use, which are key to their implementation success.11,12 For this purpose, several systematic reviews focused on specific types of CDSS (knowledge-based CDSSs,13 clinical reminders14), specific processes of care (drug prescription,15,16 diagnosis17), or specific health issues (antibiotics prescription,18 HIV management19), without any restriction to their context of use. However, health information systems implementation or evaluation models such as the HOT-fit framework (Human, Organization, Technology, Net Benefits20) and others21-25 emphasize the influence of context-specific factors in the use of health information systems. This is even more important for the primary care setting owing to the unique combination of the diversity and complexity of health issues managed (often multiple in the same consultation), its patient-centered care approach, and its particular decision-making context,26 which may generate specific needs for decision support systems.
The objective of the present systematic review was therefore to identify and quantify the barriers and facilitators to the use of CDSSs by PCPs. From a qualitative synthesis based on the HOT-fit framework, we derived a quantitative synthesis assessing the mean impacts of the human, organizational, technological factors, and the net benefits dimension on CDSS use by PCPs.
METHODS
The present study was a mixed-methods systematic review that followed the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) reporting guideline. The study protocol was registered on PROSPERO on July 14, 2020 (CRD42020185199).
Search Strategy
The search strategy was built in cooperation with medical librarians. We searched PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library for relevant studies. We tracked citations from included records to identify additional relevant references. The search was performed on July 5, 2021 without date limitation. The complete search strategy for each database is available in Supplemental Appendix 1.
Eligibility Criteria
We included all qualitative, quantitative, and mixed methods studies and only original articles for which the primary or secondary objectives were to identify barriers and facilitators to the use of CDSSs in primary care. For studies involving various professions or specialties, we only considered those that had at least 50% PCPs in the study sample. We included CDSSs that provided recommendations to PCPs (and possibly to patients) and were intended for use during the consultation.
We excluded publication types that were posters, dissertations or theses, conference proceedings, commentaries, letters, or editorials. We excluded the following decision supports that were not considered as being CDSSs: drug-drug interaction alert systems, risk assessment tools that provided assessments but not recommendations, and clinical decision supports without inference engine. We excluded the following CDSSs: decision aids only used by patients, CDSSs described as concepts or prototypes, and CDSSs evaluated with simulated clinical scenarios.
Selection Process
The selection process was performed using Covidence software (Veritas Innovation Ltd).27 After automatic removal of duplicates, 2 authors (P-Y.M. and C.R.) independently screened titles and abstracts and excluded irrelevant records. They independently screened potentially relevant articles in full text while documenting reasons for exclusion. The concordance () was 0.62 for title and abstract screening and 0.72 for full text screening. The disagreements were resolved by seeking consensus between the 2 authors.
Quality Appraisal
The quality of the included studies was independently appraised by 2 authors (P-Y.M. and C.R.) using the Covidence software. We applied the QuADS tool that has been designed to appraise the methodological and reporting quality of qualitative, quantitative, and mixed-methods studies in systematic reviews, based on 13 common criteria.28 Each criterion is assessed according to the 4 following proposals: no mention at all, very slightly, moderately, complete. The concordance () was 0.39. The disagreements were resolved by seeking consensus between the 2 authors.
Data Extraction
We used a structured data collection form to extract CDSS features (Supplemental Table 1) and methodological features of the included studies. The extraction process was performed independently by 2 authors (P-Y.M. and C.R.); disagreements were resolved by seeking consensus between them. We contacted the main authors of the included studies to obtain data on CDSS features not reported in the published article.
The HOT-fit Framework
The HOT-fit framework describes the interdependent human, organizational, and technological factors related to health information system adoption. These 3 factors are described through 7 dimensions: system use and user satisfaction related to the human factor; environment and structure related to the organizational factor; system, information and service quality related to the technological factor. The framework is complemented by an additional dimension, net benefits, which captured the positive and negative effects of CDSS recommendations on PCPs (Figure 1).20 Each HOT-fit dimension includes several evaluation measures. The HOT-fit framework was chosen as it assesses barriers and facilitators to the use of health information systems from a pragmatic, user-centered approach.
Data Synthesis
The review followed a sequential synthesis design, using a framework synthesis to inform a quantitative synthesis (Figure 2).30-32 First, a convergent integrated approach built up a common set of qualitative data from the included qualitative and quantitative data. It involved data transformation through a narrative interpretation of quantitative data.31 The resulting framework synthesis was based on the HOT-fit framework. Second, we performed a quantitative synthesis by calculating the difference between barriers and facilitators categorized according to the HOT-fit framework.
The framework synthesis was performed using NVivo (QSR International), released in March 2020.33 Barriers and facilitators to the use of CDSSs were coded inductively as concepts and then classified into the appropriate HOT-fit evaluation measures.32 This was done independently by 2 authors (P-Y.M. and E.G.), who reviewed together every coded citation and reached consensus on the concepts and the associated HOT-fit evaluation measures. We extended the HOT-fit framework to include a few concepts we were unable to classify in existing evaluation measures. Due to the various level of detail in the description of the barriers and facilitators to CDSS use in the included studies, certain codes were considered as explanatory elements of higher concepts that we named main barriers and facilitators.
From the framework synthesis results, we first quantified the individual impacts of the 3 HOT-fit factors (human, organization, technology) and the net benefits dimension on the use of each CDSS. In practice, facilitators to CDSS use were quantified as +1 and barriers as −1. For example, if a given CDSS had X barriers and Y facilitators classified in the human factor, this factor’s impact on the use of this CDSS was calculated as Y-X. For each CDSS, barriers and facilitators were considered only once, regardless of the number of their occurrences in the included studies. Then, we calculated the mean impact of each HOT-fit factor and the net benefits dimension on the use of the whole sample of CDSSs included, with a confidence interval (CI).
RESULTS
A total of 48 studies assessing 45 CDSSs were included in the review (Figure 3).34-81 Of these, 29 were mixed-methods studies, 15 were qualitative studies, and 4 were quantitative studies. Included studies were published from 1998 to 2021 (median year: 2016). Three CDSSs (EBMeDS, NHGDoc, and PRIMA-EDS) were assessed in 2 studies.
Description of the CDSSs
Clinical decision support systems were mainly developed (n = 18) and used (n = 17) in the United States. The main users were primary care physicians (n = 37) and nurses (n = 22). Clinical decision support systems were used for preventive care (n = 27), treatment (n = 21), management of chronic disease(s) (n = 13), and/or diagnosis (n = 7). All CDSSs were knowledge based; none integrated a machine learning algorithm. Updates to the knowledge database were made in 11/33 CDSSs for which this information was available. The strength of evidence of CDSS recommendations was provided in 4/19 studies for which the information was available. Twenty-three CDSSs provided a direct link to the source of recommendations. Thirteen CDSSs provided educational materials to help patients with shared decision making. Primary care professionals were trained to the use of 36 CDSSs. Most CDSSs were partially or fully integrated in the electronic health record (EHR) (n = 28). The main features of the included CDSS are described in Table 1. Detailed features are presented in Supplemental Appendix 2.
Quality Assessment
The rationale for the choice of the data collection tool was not mentioned at all in 18 studies and very slightly in 6 studies. The format and content of the data collection tool was estimated as completely appropriate to meet the stated research objectives in 27 studies, and moderately appropriate in 19 studies. The appraisal of the 13 quality criteria of the QuADS tool is presented in Supplemental Table 2. The methodological features of the included studies are presented in Supplemental Table 3.
Framework Synthesis
In total, 186 main barriers and facilitators and 69 explanatory elements were identified. All CDSSs reported technological barriers or facilitators, but 4 CDSSs did not assess the human factor, 4 did not assess the organizational factor, and 3 did not assess the net benefits dimension. Among the 186 main barriers and facilitators, 43 were classified as human factors, 49 as organizational factors, 70 as technological factors, and 24 in the net benefits dimension. The full list is presented in Supplemental Appendix 3, and barriers and facilitators (or their pairs) identified in at least 7 CDSSs are described in Tables 2 and 3 with their explanatory elements. The mean number of main barriers and facilitators identified per CDSS was 28.9; this ranged from 8 to 86 (Supplemental Table 4). We complemented the HOT-fit framework by adding an evaluation measure called “hardware” in the dimension “structure” of the factor “organization” to describe barriers and facilitators related to hardware issues in PCPs facilities.
From the human perspective, PCPs valued CDSSs for which they were trained and perceived to be useful. Conversely, they did not appreciate CDSS that provided recommendations conflicting with their beliefs or expertise. They frequently experienced alert fatigue or information overload.
From the organizational perspective, PCPs appreciated that the CDSS was well integrated into the clinical workflow and physicians appreciated that other professionals eased their workload. Conversely, PCPs expressed difficulties in using CDSSs with patients managed by other specialists because of disagreements between CDSS recommendations and specialists’ prescriptions.
From the technological perspective, PCPs appreciated fully integrated and easy-to-use CDSSs, providing reminders for PCPs and educational materials to patients, and relevant and reliable recommendations. Conversely, they disliked CDSSs that were slow, or targeting only a few health issues. They sometimes questioned the reliability of the recommendations, which they attributed to the quality and completeness of the information collected. PCPs frequently requested CDSS customization features.
From the net benefits perspective, PCPs felt that using CDSSs increased workload during the consultation for 33 of the 45 included CDSSs. Despite this major barrier, PCPs largely agreed on the benefits of CDSSs in terms of their potential to improve quality of care, particularly for preventive care. Primary care clinicians also felt that CDSSs improved the automatic identification of all useful information that they did not systematically recognize at the moment the clinical decision is made, as well as shared medical decision making through the provision of educational material to patients and their adherence to guidelines.
Quantitative Synthesis
Organizational and human HOT-fit factors had an overall negative impact on CDSS use by PCPs. The technological factor had a neutral overall impact, and the net benefits dimension an overall positive impact on CDSS use (Table 4). Individual impacts of the 3 HOT-fit factors and the net benefits dimension on the use of each CDSS are presented in Supplemental Table 5.
DISCUSSION
All barriers and facilitators to CDSS use by PCPs were distributed across the 3 HOT-fit factors and the net benefits dimension, with a predominance of the technological factor, that was the only factor explored in all studies. However, the overall impact of the technological factor on the use of CDSSs by PCPs was neutral, which indicates a balance between technological barriers and facilitators, while the human and organizational factors had overall negative impacts and the net benefit dimension an overall positive impact. The net benefits reported by PCPs support the potential effectiveness of CDSSs in improving quality and safety of care. However, they seem unable to improve care efficiency since they are believed to increase PCP workload.
Comparison With Other Studies
This review is the first to identify and quantify barriers and facilitators to the use of CDSSs specifically in the primary care setting. In 2017, Kilsdonk et al conducted a systematic review and gap analysis of barriers and facilitators to the use of knowledge-based CDSSs in any setting (including hospitals) according to the HOT-fit framework, including CDSS evaluations based on simulated clinical scenarios.13 The quantitative gap analysis revealed the predominance of technological and human factors, and a knowledge gap regarding the organizational factor and the net benefits dimension. However, no conclusion could be drawn on the relative impact of these factors on CDSS use. In the present review, the organizational factor was not less frequently identified than the human factor, and the quantitative synthesis assessed for the first time the relative impact of the HOT-fit factors on CDSS use. In 2013, Moxey et al analyzed both health care providers’ general views on, and use of, CDSSs including computerized guidelines and risk assessment tools, in various settings.15 These systematic reviews identified barriers and facilitators mostly concerning time consumption, workflow, integration in the EHR, user friendliness, and relevance of the recommendations. Moxey et al conducted a subgroup analysis that identified the lack of CDSS integration into the EHR and patient negative opinion as barriers specific to the ambulatory care setting.15 Poor CDSS integration was confirmed as a main technological barrier in our review. The diversity of EHRs developed for primary care may explain the persistence of this barrier over time.82 Patient negative opinion was less frequently reported in our review, presumably because of the increasing acceptance over time of the use of health information systems during consultations.83
Other barriers and facilitators identified in the present review are original as compared with previous reviews.13,15 First, teamwork needs for and benefits of using CDSSs were frequently reported. The importance of teamwork could not be identified previously since it is just emerging in primary care in many health care systems, contrarily to the hospital setting.84,85 Second, PCPs expressed difficulties in using CDSSs with patients co-managed by specialists, due to discrepancies between specialist and CDSS recommendations or to outdated patient information in the EHR. Third, PCPs expected CDSSs covering a large array of conditions in agreement with the diversity of the health issues they manage. Fourth, the contribution of CDSSs to reporting on quality measures was valued by PCPs, in a context of evaluation programs implemented in primary care.
Strengths and Weaknesses
The scope of this review was limited to strictly defined CDSSs providing recommendations to PCPs based on individual patient characteristics, according to a medical decision-making perspective. Two previous systematic reviews13,15 also included decision supports providing individualized assessments (risks) or general recommendations (not personalized to the patient), which presumably increased the heterogeneity of the barriers and facilitators identified. They also have included decision supports used in any care setting (including hospitals), and those evaluated using simulated clinical scenarios. In the present review, only CDSSs used in clinical settings and in the specific context of primary care were included, which further strengthens the external validity of the results. It may, however, not be fully representative of the various CDSSs developed for primary care, as some of them were probably not studied regarding barriers and facilitators to their use.
Frequency-based indicators of the impact of HOT-fit categories on CDSS use were useful for comparison purposes; they were, however, limited by the heterogeneity of CDSS barriers and facilitators and by the difficulty to weight them individually according to their perceived importance by PCPs. In addition, PCPs reported some barriers such as conflicts between CDSS recommendations and their own expertise or lack of computer skills, while the first barrier is more likely related to the use of guidelines86 and the second to the uptake of EHRs.87,88 The classification of barrier and facilitators in the evaluation measures was sometimes subjective because of similarities between, or lack of clear definition of some HOT-fit evaluation measures, as already reported by other authors.13 For instance, the HOT-fit evaluation measure “clinical process” in the dimension “structure” of the organizational factor is close to the evaluation measure “clinical practice” of the net benefits dimension.
Implications
The present systematic review highlights barriers and facilitators to the use of CDSSs related to its feasibility (eg, increased workload), acceptability (eg, conflicts with PCPs expertise or beliefs), meaningfulness (eg, relevance of recommendations), and effectiveness (net benefits dimension). These different forms of evidence refer to the feasibility, appropriateness, meaningfulness, and effectiveness (FAME) evidence-based model,89 which is useful to understand complex interventions such as implementing CDSSs. Based on these findings, we inferred an operational list of 11 intrinsic and contextual CDSS features expected to make them more feasible, acceptable, meaningful, and effective in primary care (Table 5). They are spread across the 3 interdependent human, organizational, and technological factors. Among intrinsic features, the expectation of decision support for preventive care is consistent with the great importance of prevention in primary care practice. The expectation of a large array of conditions covered by CDSSs is explained by the preference of PCPs for a single comprehensive system rather than several CDSSs with limited clinical coverage displaying recommendations in separate windows, each requiring a specific training. Information overload refers to PCPs facing more information than they have the time or cognitive ability to process.90,91 Clinical decision support systems aim at both rationalizing patient management while avoiding overwhelming PCPs with information, which can be achieved by CDSSs providing concise recommendations and prioritizing the most appropriate interventions recommended for each patient.92 In addition, the feature of providing patients with educational material supports shared decision making within a patient-centered approach. Among contextual features, developing CDSSs in close collaboration with PCPs according to a bottom-up approach is needed to improve their perceived usefulness and user-friendliness and the relevance of their recommendations. Providing the rationale for selecting the sources of the CDSS knowledge base is expected to increase CDSS reliability. This seems critical, even more for future non–knowledge-based CDSSs, as health care professionals are exposed to automation bias, which consists of over-relying on automated advice.93 Teamwork for data collection and use of CDSSs may ease physician workload and make EHR patient data more complete and reliable. Several of these intrinsic and contextual features may allow the leading barrier, increased workload, to be overcome. Since the findings of the present review show that human and organizational factors are the most impeding to CDSS use, we recommend that CDSS developers investigate the human and organizational requirements in the early stages of CDSS development and that evaluation studies of CDSS use systematically evaluate these factors. In addition, quantitative studies are particularly needed to assess the weight of the main barriers and facilitators identified through the present framework synthesis.
CONCLUSION
Although benefits reported by PCPs support the potential effectiveness of CDSS use in improving quality and safety of care, they also highlight its lack of efficiency due to increased workload. Our findings emphasize the need for CDSS developers to better address human and organizational issues in addition to technological stakes. Covering theses 3 factors, we inferred core intrinsic and contextual CDSS features expected to improve their usability in primary care.
Acknowledgments
We would like to thank Mathieu Fauvernier and Maxime Bonjour (Service de Biostatistiques des Hospices Civils de Lyon, France; UMR5558, Laboratoire de Biométrie et Biologie Évolutive, Équipe Biostatistique-Santé, Villeurbanne, France) for their comments on the data synthesis method. We would like to thank Colin Sidre and Bastien Blanchon (BIU Santé Médecine, Paris, France) for help with the search strategy. We also thank Philip Robinson (DRS, Hospices Civils de Lyon) for help in manuscript preparation.
Footnotes
Conflicts of interest: authors report none.
Funding support: The study funder was a public health institution (Agence Régionale de Santé Auvergne-Rhône-Alpes) through a grant attributed to Pierre-Yves Meunier.
Disclaimer: The study funder did not interfere in the conduct of this study. All authors confirm the independence of researchers from funders. All authors had full access to all of the data (including tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Trial registration: Protocol registered on PROSPERO (CRD42020185199) (no amendments to information provided in the protocol).
- Received for publication April 4, 2022.
- Revision received September 8, 2022.
- Accepted for publication October 10, 2022.
- © 2023 Annals of Family Medicine, Inc.
REFERENCES
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.
- 23.
- 24.
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 92.↵
- 93.↵