Abstract
This paper reports on a multimethod cross-sectional study of the Ontario electronic consultation (eConsult) service. Utilization and closeout survey data from July 2018 through June 2020 were analyzed using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework. Requesting clinicians submitted 60,474 eConsults, and monthly cases increased from 1,487 in July 2018 to 4,179 in June 2020. The median specialist response time was 1 day. An originally contemplated referral was avoided in 51% of cases. Ontario eConsult showed successful uptake across Ontario, demonstrating continued spread and scale, and offering a template for trailblazers looking to implement digital health innovations in their own jurisdictions.
- digital technology
- eReferral and eConsultation
- heath care quality, access and evaluation
- health services accessibility
- telemedicine
INTRODUCTION
Excessive wait times for specialist care are a serious issue in Canada, resulting in poorer health outcomes for patients and frustrating challenges for clinicians.1 The electronic consultation (eConsult) service offers an opportunity to improve access to specialty advice by allowing primary care clinicians and specialists to communicate electronically over a secure platform. Research has shown that eConsult can improve access to specialty care, reduce the need for in-person visits, and lower health system costs.2 Canada has faced barriers in expanding regional health care innovations or maintaining them beyond the additional pilot phase due to its vast size; fragmented, provincially managed health care system; and challenges in knowledge sharing.3 To overcome these barriers and ensure effective, long-standing spread and scale, innovators must conduct a broad, thorough assessment of the implementation process in order to identify barriers and adapt where needed.
In this paper, we evaluated the first 2 years of usage data from a provincial eConsult service in Ontario, Canada, using an established framework. Our goal is to identify the service’s impact and inform its ongoing spread, scale, and sustainability.
METHODS
This is a multimethod cross-sectional study of Ontario eConsult using a longitudinal and descriptive analysis. The study is set in Ontario, Canada, a province of 14 million residents divided into 5 health regions: North, East, West, Central, and Toronto. Our study included data from all cases submitted to Ontario eConsult from July 1, 2018 through June 30, 2020.
Ontario eConsult is a provincial program built on existing regional services that launched in 2018 (see Supplemental Appendix 1 for a timeline of eConsult’s development and a description of the service).
This study used the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) Framework to guide data collection and analysis.4 Full descriptions of each dimension and applicable metrics for eConsult are presented in Supplemental Table 1.
The project was submitted to the Ottawa Health Science Network Research Ethics Board (OHSN REB) as a Quality Improvement study. The OHSN REB waived its review and the project is REB exempt.
Data Collection and Analysis
Data were drawn from 3 sources: (1) utilization data collected automatically by the eConsult service, (2) registered clinician data, and (3) responses to the mandatory closeout survey (Supplemental Appendix 1). We utilized descriptive statistics to analyze the 2 years of Ontario eConsult data (July 2018 through June 2020). Population data from each Ontario Health region was used to determine the number of eConsults that were sent per 1,000 residents. Data were grouped by requesting clincian health region to observe trends in different regions of the province.
RESULTS
Reach
Requesting clinicians submitted 60,474 eConsults during the study period (23,180 the first year and 37,294 the second) (Figure 1). Monthly rates increased by 181% over the duration, from 1,487 sent in July 2018 to 4,179 in June 2020. There were 4.32 eConsults sent per 1,000 residents in Ontario, with Toronto (7.49), North (6.36) and West (5.08) regions having the highest rates by population (Supplemental Figure 1). The most commonly accessed specialties were dermatology (15%), hematology (9%), endocrinology (7%), obstetrics and gynecology (6%), and neurology (6%) (Supplemental Figure 2).
Effectiveness
Specialists provided an initial response to eConsults in a median of 1.0 day (mean 2.0 days), with 95.9% of initial responses being provided in 7 days or less and 99.8% provided in 30 days or less. Specialists billed a median of 15 minutes per eConsult case resulting in a median cost of $50.00. Requesting clinicians received advice for a new or additional course of action in 55% of cases and confirmation on their original course of action in 40%. In 51% of cases, a referral was originally contemplated but avoided as a result of the eConsult. In 19% of cases, a referral was originally contemplated and was still needed. In 3% of cases, a referral was not originally contemplated, but was initiated as a result of the eConsult. In 21% of cases, a referral was not originally contemplated and was still not needed. Requesting clinicians reported the eConsult response was not useful in only 2% of cases and found there to be no particular benefit to eConsult in only 1% of cases. There were 5,334 (9%) cases without a survey response.
Adoption
Over the study period, 4,498 requesting clinicians sent at least 1 eConsult with 34% (1,508) from the West region, 20% (921) from the Central region, 20% (892) from the East region, 16% (725) from the Toronto region, and 10% (452) from the North region. Of the 4,498 requesting clinicians, 90% (4,068) were primary care clinicians and 8% (363) were specialists, with the role of the remaining 2% unknown. The number of requesting clinicians and active clinicians (ie, those who submitted ≥3 eConsults in a 6-month period) are presented in Figure 2 for 6-month intervals. Requesting clinicians submitted an average total of 13.4 eConsults (range 1 to 335), or roughly 1 to 2 eConsults every 2 months (0.6 per month).
A total of 1,482 specialists across Ontario registered with eConsult, an increase of 887 specialists since the launch of the service. The West region had the most registered specialists (38%) while the North had the least (4%). Among specialists who provide care through the Building Access to Specialists through eConsultation (BASE) model, there were 59 specialty groups supported by 175 specialists in July 2018 compared with 98 specialty groups supported by 297 specialists in June 2020.
Implementation
Ontario eConsult’s implementation followed the 10-step strategy outlined by Liddy and colleagues,5 which discusses key actions such as establishing formal partnerships with sponsoring institutions and delivery partners, leveraging existing infrastructure for the platform, and addressing the needs of local populations (Supplemental Appendix 2).
Maintenance
The College of Family Physicians of Canada and the Royal College of Physicians and Surgeons of Canada have stated their support for eConsult as a standard practice,6,7 and the Government of Ontario has designated Ontario eConsult’s adoption as a year 1 priority for all Ontario Health Teams.8 The Ontario Ministry of Health has shown commitment to Ontario eConsult by establishing it as one of the first initiatives in the Ontario eServices Program, formed in April 2020 with funding commitment to March 2022.9
DISCUSSION
Our findings lend further credence to claims of eConsult’s generalizability, previously demonstrated through replication in sub-regions of the West and East health regions, which showed outcomes closely aligned with the original Ottawa-based service.10,11 For example, median response times were 2 days for South East eConsults,11 and 1.1 days for those in Mississauga Halton,10 while a cross-sectional study of the service overall revealed a median response time of 1.2 days.12 These findings compare well to the average specialist response times of other eConsult services across Canada,13 suggesting that eConsult is highly generalizable and can be expanded without sacrificing its effectiveness.
This study’s chief strength is the size of its data set, which incorporates over 60,000 eConsults submitted across a province with a population of 14 million people. However, it also has limitations. Though conducted across a wide geographic area, the study took place in a single province, reducing its overall generalizability. The East is also underrepresented due to the presence of the Champlain BASE eConsult service, which still operates in the region. Data for this study was limited to that collected through eConsult, which does not allow us to assess later outcomes or patient perspectives directly. We are also unable to clearly quantify eConsult’s impact on overall referral rates, nor have we included qualitative data to gain insight into clinician perspectives in more detail. Lastly, a small percentage of the requesting clinician specialty information in our data set were incorrect, blank, or nondescript, and clinicians could not be classified as requesting vs specialist.
CONCLUSION
During its first 2 years of implementation, Ontario eConsult showed successful uptake across Ontario. Findings demonstrated the service’s reach, effectiveness, and successful adoption, and outlined steps supporting its implementation and maintenance. Results paint an encouraging picture of the service’s continued spread and scale and offer a template for trailblazers looking to implement digital health innovations in their own jurisdictions.
Acknowledgments
The authors wish to thank the primary care clinicians and specialists who use Ontario eConsult, Justin Joschko for his assistance preparing the manuscript for publication, and JJ Hupka, Danielle Claus, and Gilad Epstein for their work on establishing Ontario eConsult.
Footnotes
Conflicts of interest: Dr Liddy and Dr Keely are co-founders of the Champlain BASE eConsult Service, but they have no commercial interest in the service and do not retain any proprietary rights. As Co-executive Directors of the Ontario eConsult Centre of Excellence, they receive salary support from the Ontario Ministry of Health. Dr Keely answers occasional eConsults (less than 1 per month) as a specialist through the service, for which she is reimbursed. Other authors report none.
Funding support: Funding for this project was provided by the Ontario Ministry of Health. The opinions, results, and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by the Ontario Ministry of Health is intended or should be inferred. This work was also supported in part with funding from the Government of Ontario through the Ontario Centre for Learning, Research and Innovation in Long-Term Care hosted at Bruyère. The views expressed herein do not necessarily reflect the views of the Province.
- Received for publication June 9, 2021.
- Revision received October 13, 2021.
- Accepted for publication December 2, 2021.
- © 2022 Annals of Family Medicine, Inc.