Review articleFactors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis
Introduction
Evidence-based guidelines are developed and implemented to guide and support physician’ clinical decision making in improving quality of care. However, evidence exists that shows that paper-based clinical guidelines are still underutilized in practice [1], [2]. A Clinical Decision Support System (CDSS) can offer physicians patient-specific advice based on guideline recommendations, thereby overcoming obstacles in the use of traditional paper-based guidelines and improve physicians’ adherence to recommendations. [3], [4]. Yet, even though the evidence of CDSSs improving clinical performance and patient outcomes is convincing, the failure rate in introducing CDSS in clinical practices is still over 50 percent [5].
Introducing a CDSS seems fraught with obstacles among which low ease of system use [6], negative end-user attitudes towards the system and negative impact on clinical workflows [5]. Studies evaluating CDSS implementation in clinical care continue to provide insight into factors influencing or issues surrounding CDSS introduction [7], [8]. However, these insights are revealed and discussed from various perspectives [9], among which the perspective of the CDSS technology, the user’s experience with CDSS and cultural and management issues within the healthcare organization. But while evidence on the impact of factors is wide-spread, a synopsis is absent. Building on the knowledge base of evaluation studies of Health Information systems (HIS), Yusof et al. proposed a framework to evaluate HIS while incorporating the concept of fit between Human, Organization and Technology (HOT-fit) [10]. The concept of fit focuses on the alignment between and compatibility of the human, technology and organization. In doing so, the HOT-fit framework provides an excellent framework to report on factors influencing CDSS implementation from these three perspectives.
In a previous conference paper, we already report on a literature study about factors associated with CDSS implementations used by physicians [11]. This paper extents the previous literature review with an in-depth exploration of factors associated with a successful implementation of guideline-based CDSS. We performed a mixed method research synthesis [12], focusing on factors revealed by both quantitative and qualitative evaluation studies of guideline-based CDSS. By combining evidence from quantitative, qualitative and mixed studies we aim to provide a more integrated and differentiated understanding of implementation factors influencing guideline-based CDSS implementation success and, in doing so, illustrate the gaps in the current literature. We extracted these from the studies through a systematic approach and mapped them to the HOT-fit framework to provide a more integrated and differentiated understanding of factors impeding or facilitating CDSS implementation success. This research synthesis ultimately aims to support software teams in development and implementation of guideline-based CDSS.
Section snippets
HOT-fit
The human, organization and technology-fit (HOT-fit) framework aims to assist researchers in conducting thorough evaluation studies of HIS. The HOT-fit framework can be used as a reference model for evaluating the performance, effectiveness and impact of HIS in a rigorous, systematic and continuous manner. The HOT-fit framework is based on earlier models for evaluating information systems (IS); the IS Success Model by DeLone and McLean [13] and Information Technology (IT)-Organization Fit Model
Search results
The literature search generated a total of 3676 publications. After removing the duplicates and reviewing the abstracts, 556 publications were selected for full text review. The measured Cohen's Kappa for the inter-observer agreement between reviewers was 0.754, indicating good agreement [18]. Finally, 35 publications were found eligible for inclusion. Fig. 2 shows the study’s search flow and an overview of the included publications can be found in Supplemental Table S1. Most studies used
Main findings
This literature synthesis provides an in-depth exploration of factors facilitating or impeding implementation of guideline-based CDSS. Previous researchers have advocated that evaluation studies of CDSS should be performed from multiple perspectives [8], [52], [53], [54]. Our mixed studies research synthesis revealed that evaluation studies of guideline-based CDSS have mainly focused on Technological and Human factors, reflected in the high number of factors reported in these domains (210 and
Conflicts of interest
The authors declare that they have no financial or personal relationships that could influence this work.
Authors’ contributions
The authors declare that this manuscript is original, has not been published before and is not currently being considered for publication elsewhere. The manuscript has been approved by all authors. There are no other persons who satisfied the criteria for authorship. We confirm that the order of authors listed in the manuscript has been approved by all authors. The corresponding author will act as the sole contact for the editorial process and we understand that she is responsible for
References (65)
Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality
J. Am. Med. Inform. Assoc.
(2003)An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit)
Int. J. Med. Inform.
(2008)Decision time for clinical decision support systems
Clin. Med. (Lond.)
(2014)A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews
Int. J. Nurs. Stud.
(2009)Evaluating the use of a computerized clinical decision support system for asthma by pediatric pulmonologists
Int. J. Med. Inform.
(2012)Clinical decision support software for management of chronic heart failure: development and evaluation
Comput. Biol. Med.
(2006)Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system
Int. J. Med. Inf.
(2005)- et al.
Impact of four training conditions on physician use of a web-based clinical decision support system
Artif. Intell. Med.
(2013) A network-based system to improve care for schizophrenia: the Medical Informatics Network Tool (MINT)
J. Am. Med. Inform. Assoc.
(2004)A multiple-scenario assessment of the effect of a continuous-care, guideline-based decision support system on clinicians' compliance to clinical guidelines
Int. J. Med. Inform.
(2015)
The effect of incorporating guidelines into a computerized order entry system for diagnostic imaging
J. Am. Coll. Radiol.
Using multi-perspective methodologies to study users' interactions with the prototype front end of a guideline-based decision support system for diabetic foot care
Int. J. Med. Inform.
Integrating usability testing and think-aloud protocol analysis with near-live clinical simulations in evaluating clinical decision support
Int. J. Med. Inform.
A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease
J. Am. Med. Inform. Assoc.
A taxonomic description of computer-based clinical decision support systems
J. Biomed. Inform.
Human factors systems approach to healthcare quality and patient safety
Appl. Ergon.
How can we improve guideline use? A conceptual framework of implementability
Implement. Sci.
Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings
J. Am. Med. Inform. Assoc.
The effectiveness of clinical guideline implementation strategies—a synthesis of systematic review findings
J. Eval. Clin. Pract.
Development and implementation of computerized clinical guidelines: barriers and solutions
Methods Inf. Med.
What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians
Fam. Pract.
Requirements for guidelines systems: implementation challenges and lessons from existing software-engineering efforts
BMC Med. Inform. Decis. Mak.
Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals
J. Med. Syst.
Towards an evaluation framework for information quality management (IQM) practices for health information systems-evaluation criteria for effective IQM practices
J. Eval. Clin. Pract.
Factors known to influence acceptance of clinical decision support systems
Stud. Health Technol. Inform.
Mixed methods research synthesis: definition, framework, and potential
Qual. Quant.
The DeLone and McLean model of information systems success: a ten-year update
J. Manage. Inf. Syst.
The Corporation of the 1990
Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field
J. Health Serv. Res. Policy
The measurement of observer agreement for categorical data
Biometrics
How do medical doctors use a web-based oncology protocol system? A comparison of Australian doctors at different levels of medical training using logfile analysis and an online survey
BMC Med. Inform. Decis. Mak.
Evaluating acceptance and user experience of a guideline-based clinical decision support system execution platform
J. Med. Syst.
Cited by (156)
Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study
2024, International Journal of Medical InformaticsImaging 3.0: A scoping review
2024, Current Problems in Diagnostic RadiologyUsability of an app-based clinical decision support system to monitor psychotropic drug prescribing appropriateness in dementia
2023, International Journal of Medical InformaticsSystematic User-centered Design of a Prototype Clinical Decision Support System for Glaucoma
2023, Ophthalmology ScienceA community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support
2023, Journal of Biomedical InformaticsMultimodal predictions of treatment outcome in major depression: A comparison of data-driven predictors with importance ratings by clinicians
2023, Journal of Affective Disorders