Skip to main content

Main menu

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • The Issue in Brief (Plain Language Summaries)
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
  • Contact
    • Feedback
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • The Issue in Brief (Plain Language Summaries)
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
  • Contact
    • Feedback
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleRESEARCH BRIEF

The Provincial Spread and Scale of the Ontario eConsult Service: Evaluation of the First 2 Years

Clare Liddy, Rhea Mitchell, Sheena Guglani, Ariana Mihan, Claire Sethuram, Andrea Miville and Erin Keely
The Annals of Family Medicine May 2022, 20 (3) 262-265; DOI: https://doi.org/10.1370/afm.2812
Clare Liddy
1C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Ontario, Canada
2Department of Family Medicine, University of Ottawa, Ottawa, Ontario, Canada
3eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: cliddy@uottawa.ca
Rhea Mitchell
3eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheena Guglani
1C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Ontario, Canada
3eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ariana Mihan
1C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claire Sethuram
1C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrea Miville
3eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erin Keely
3eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Ontario, Canada
4Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
5Division of Endocrinology/Metabolism, The Ottawa Hospital, Ottawa, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Abstract

This paper reports on a multimethod cross-sectional study of the Ontario electronic consultation (eConsult) service. Utilization and closeout survey data from July 2018 through June 2020 were analyzed using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework. Requesting clinicians submitted 60,474 eConsults, and monthly cases increased from 1,487 in July 2018 to 4,179 in June 2020. The median specialist response time was 1 day. An originally contemplated referral was avoided in 51% of cases. Ontario eConsult showed successful uptake across Ontario, demonstrating continued spread and scale, and offering a template for trailblazers looking to implement digital health innovations in their own jurisdictions.

Key words:
  • digital technology
  • eReferral and eConsultation
  • heath care quality, access and evaluation
  • health services accessibility
  • telemedicine

INTRODUCTION

Excessive wait times for specialist care are a serious issue in Canada, resulting in poorer health outcomes for patients and frustrating challenges for clinicians.1 The electronic consultation (eConsult) service offers an opportunity to improve access to specialty advice by allowing primary care clinicians and specialists to communicate electronically over a secure platform. Research has shown that eConsult can improve access to specialty care, reduce the need for in-person visits, and lower health system costs.2 Canada has faced barriers in expanding regional health care innovations or maintaining them beyond the additional pilot phase due to its vast size; fragmented, provincially managed health care system; and challenges in knowledge sharing.3 To overcome these barriers and ensure effective, long-standing spread and scale, innovators must conduct a broad, thorough assessment of the implementation process in order to identify barriers and adapt where needed.

In this paper, we evaluated the first 2 years of usage data from a provincial eConsult service in Ontario, Canada, using an established framework. Our goal is to identify the service’s impact and inform its ongoing spread, scale, and sustainability.

METHODS

This is a multimethod cross-sectional study of Ontario eConsult using a longitudinal and descriptive analysis. The study is set in Ontario, Canada, a province of 14 million residents divided into 5 health regions: North, East, West, Central, and Toronto. Our study included data from all cases submitted to Ontario eConsult from July 1, 2018 through June 30, 2020.

Ontario eConsult is a provincial program built on existing regional services that launched in 2018 (see Supplemental Appendix 1 for a timeline of eConsult’s development and a description of the service).

This study used the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) Framework to guide data collection and analysis.4 Full descriptions of each dimension and applicable metrics for eConsult are presented in Supplemental Table 1.

The project was submitted to the Ottawa Health Science Network Research Ethics Board (OHSN REB) as a Quality Improvement study. The OHSN REB waived its review and the project is REB exempt.

Data Collection and Analysis

Data were drawn from 3 sources: (1) utilization data collected automatically by the eConsult service, (2) registered clinician data, and (3) responses to the mandatory closeout survey (Supplemental Appendix 1). We utilized descriptive statistics to analyze the 2 years of Ontario eConsult data (July 2018 through June 2020). Population data from each Ontario Health region was used to determine the number of eConsults that were sent per 1,000 residents. Data were grouped by requesting clincian health region to observe trends in different regions of the province.

RESULTS

Reach

Requesting clinicians submitted 60,474 eConsults during the study period (23,180 the first year and 37,294 the second) (Figure 1). Monthly rates increased by 181% over the duration, from 1,487 sent in July 2018 to 4,179 in June 2020. There were 4.32 eConsults sent per 1,000 residents in Ontario, with Toronto (7.49), North (6.36) and West (5.08) regions having the highest rates by population (Supplemental Figure 1). The most commonly accessed specialties were dermatology (15%), hematology (9%), endocrinology (7%), obstetrics and gynecology (6%), and neurology (6%) (Supplemental Figure 2).

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Utilization of the Ontario eConsult service in the first 2 years—number of eConsults sent per month and cumulative total.

eConsult = electronic consultation. Orange line represents the median number of cases submitted during the study time period.

Effectiveness

Specialists provided an initial response to eConsults in a median of 1.0 day (mean 2.0 days), with 95.9% of initial responses being provided in 7 days or less and 99.8% provided in 30 days or less. Specialists billed a median of 15 minutes per eConsult case resulting in a median cost of $50.00. Requesting clinicians received advice for a new or additional course of action in 55% of cases and confirmation on their original course of action in 40%. In 51% of cases, a referral was originally contemplated but avoided as a result of the eConsult. In 19% of cases, a referral was originally contemplated and was still needed. In 3% of cases, a referral was not originally contemplated, but was initiated as a result of the eConsult. In 21% of cases, a referral was not originally contemplated and was still not needed. Requesting clinicians reported the eConsult response was not useful in only 2% of cases and found there to be no particular benefit to eConsult in only 1% of cases. There were 5,334 (9%) cases without a survey response.

Adoption

Over the study period, 4,498 requesting clinicians sent at least 1 eConsult with 34% (1,508) from the West region, 20% (921) from the Central region, 20% (892) from the East region, 16% (725) from the Toronto region, and 10% (452) from the North region. Of the 4,498 requesting clinicians, 90% (4,068) were primary care clinicians and 8% (363) were specialists, with the role of the remaining 2% unknown. The number of requesting clinicians and active clinicians (ie, those who submitted ≥3 eConsults in a 6-month period) are presented in Figure 2 for 6-month intervals. Requesting clinicians submitted an average total of 13.4 eConsults (range 1 to 335), or roughly 1 to 2 eConsults every 2 months (0.6 per month).

Figure 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2.

Requesting clinician adoption over time.

eConsult = electronic consultation.

Note: June 30, 2018 indicates the 6-month period prior to provincial launch of eConsult. Other dates indicate the end of a 6-month period after eConsult was implemented.

A total of 1,482 specialists across Ontario registered with eConsult, an increase of 887 specialists since the launch of the service. The West region had the most registered specialists (38%) while the North had the least (4%). Among specialists who provide care through the Building Access to Specialists through eConsultation (BASE) model, there were 59 specialty groups supported by 175 specialists in July 2018 compared with 98 specialty groups supported by 297 specialists in June 2020.

Implementation

Ontario eConsult’s implementation followed the 10-step strategy outlined by Liddy and colleagues,5 which discusses key actions such as establishing formal partnerships with sponsoring institutions and delivery partners, leveraging existing infrastructure for the platform, and addressing the needs of local populations (Supplemental Appendix 2).

Maintenance

The College of Family Physicians of Canada and the Royal College of Physicians and Surgeons of Canada have stated their support for eConsult as a standard practice,6,7 and the Government of Ontario has designated Ontario eConsult’s adoption as a year 1 priority for all Ontario Health Teams.8 The Ontario Ministry of Health has shown commitment to Ontario eConsult by establishing it as one of the first initiatives in the Ontario eServices Program, formed in April 2020 with funding commitment to March 2022.9

DISCUSSION

Our findings lend further credence to claims of eConsult’s generalizability, previously demonstrated through replication in sub-regions of the West and East health regions, which showed outcomes closely aligned with the original Ottawa-based service.10,11 For example, median response times were 2 days for South East eConsults,11 and 1.1 days for those in Mississauga Halton,10 while a cross-sectional study of the service overall revealed a median response time of 1.2 days.12 These findings compare well to the average specialist response times of other eConsult services across Canada,13 suggesting that eConsult is highly generalizable and can be expanded without sacrificing its effectiveness.

This study’s chief strength is the size of its data set, which incorporates over 60,000 eConsults submitted across a province with a population of 14 million people. However, it also has limitations. Though conducted across a wide geographic area, the study took place in a single province, reducing its overall generalizability. The East is also underrepresented due to the presence of the Champlain BASE eConsult service, which still operates in the region. Data for this study was limited to that collected through eConsult, which does not allow us to assess later outcomes or patient perspectives directly. We are also unable to clearly quantify eConsult’s impact on overall referral rates, nor have we included qualitative data to gain insight into clinician perspectives in more detail. Lastly, a small percentage of the requesting clinician specialty information in our data set were incorrect, blank, or nondescript, and clinicians could not be classified as requesting vs specialist.

CONCLUSION

During its first 2 years of implementation, Ontario eConsult showed successful uptake across Ontario. Findings demonstrated the service’s reach, effectiveness, and successful adoption, and outlined steps supporting its implementation and maintenance. Results paint an encouraging picture of the service’s continued spread and scale and offer a template for trailblazers looking to implement digital health innovations in their own jurisdictions.

Acknowledgments

The authors wish to thank the primary care clinicians and specialists who use Ontario eConsult, Justin Joschko for his assistance preparing the manuscript for publication, and JJ Hupka, Danielle Claus, and Gilad Epstein for their work on establishing Ontario eConsult.

Footnotes

  • Conflicts of interest: Dr Liddy and Dr Keely are co-founders of the Champlain BASE eConsult Service, but they have no commercial interest in the service and do not retain any proprietary rights. As Co-executive Directors of the Ontario eConsult Centre of Excellence, they receive salary support from the Ontario Ministry of Health. Dr Keely answers occasional eConsults (less than 1 per month) as a specialist through the service, for which she is reimbursed. Other authors report none.

  • Read or post commentaries in response to this article.

  • Funding support: Funding for this project was provided by the Ontario Ministry of Health. The opinions, results, and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by the Ontario Ministry of Health is intended or should be inferred. This work was also supported in part with funding from the Government of Ontario through the Ontario Centre for Learning, Research and Innovation in Long-Term Care hosted at Bruyère. The views expressed herein do not necessarily reflect the views of the Province.

  • ↵Supplemental materials

  • Received for publication June 9, 2021.
  • Revision received October 13, 2021.
  • Accepted for publication December 2, 2021.
  • © 2022 Annals of Family Medicine, Inc.

References

  1. 1.↵
    1. Canadian Institute for Health Information
    . Health Care in Canada, 2012: a focus on wait times. Canadian Institute for Health Information. Published 2012. Accessed Nov 23, 2020. https://secure.cihi.ca/free_products/HCIC2012-FullReport-ENweb.pdf
  2. 2.↵
    1. Liddy C,
    2. Moroz I,
    3. Mihan A,
    4. Nawar N,
    5. Keely E.
    A systematic review of asynchronous, provider-to-provider, electronic consultation services to improve access to specialty care available worldwide. Telemed J E Health. 2019; 25(3): 184-198. doi:10.1089/tmj.2018.0005
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. MacNeil M,
    2. Koch M,
    3. Kuspinar A,
    4. Juzwishin D,
    5. Lehoux P,
    6. Stolee P.
    Enabling health technology innovation in Canada: Barriers and facilitators in policy and regulatory processes. Health Policy. 2019; 123(2): 203-214. doi:10.1016/j.healthpol.2018.09.018
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Glasgow RE,
    2. Vogt TM,
    3. Boles SM.
    Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999; 89(9): 1322-1327. doi:10.2105/ajph.89.9.1322
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Liddy C,
    2. Maranger J,
    3. Afkham A,
    4. Keely E.
    Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health. 2013; 19(12): 982-990. doi:10.1089/tmj.2013.0056
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Lemire F.
    Statement of support for eConsult as standard of practice. Published Jun 2018. https://econsultontario.ca/wp-content/uploads/2019/06/CFPC-Support-Letter_eConsult.pdf
  7. 7.↵
    1. Padmos A.
    Statement of support for eConsult as standard of practice. Published Oct 2018. https://econsultontario.ca/wp-content/uploads/2019/06/eConsultSupportStatement_RC.pdf
  8. 8.↵
    1. Ontario Ministry of Health
    . Ontario health teams: guidance for health care providers and organizations. Published 2020. https://health.gov.on.ca/en/pro/programs/connectedcare/oht/docs/guidance_doc_en.pdf
  9. 9.↵
    1. Ontario eConsult
    . The Ontario eServices program. Accessed May 14, 2021. https://econsultontario.ca/the-ontario-eservices-program/
  10. 10.↵
    1. Liddy C,
    2. Moroz I,
    3. Mihan A,
    4. Keely E.
    Assessment of the generalizability of an eConsult service through implementation in a new health region. J Am Board Fam Med. 2019; 32(2): 146-157. doi:10.3122/jabfm.2019.02.180169
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Liddy C,
    2. Moroz I,
    3. Afkham A,
    4. Keely E.
    Evaluating the implementation of the Champlain BASE eConsult service in a new region of Ontario, Canada: a cross-sectional study. Healthc Policy. 2017; 13(2): 79-95. doi:10.12927/hcpol.2017.25320
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Liddy C,
    2. Keely E.
    Using the quadruple aim framework to measure impact of heath technology implementation: a case study of eConsult. J Am Board Fam Med. 2018; 31(3): 445-455. doi:10.3122/jabfm.2018.03.170397
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. Liddy C,
    2. Bello A,
    3. Cook J, et al.
    Supporting the spread and scale-up of electronic consultation across Canada: cross-sectional analysis. BMJ Open. 2019; 9(5): e028888. doi:10.1136/bmjopen-2018-028888
    OpenUrlAbstract/FREE Full Text
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 20 (3)
The Annals of Family Medicine: 20 (3)
Vol. 20, Issue 3
1 May 2022
  • Table of Contents
  • Index by author
  • Plain Language Article Summaries
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The Provincial Spread and Scale of the Ontario eConsult Service: Evaluation of the First 2 Years
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
4 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
The Provincial Spread and Scale of the Ontario eConsult Service: Evaluation of the First 2 Years
Clare Liddy, Rhea Mitchell, Sheena Guglani, Ariana Mihan, Claire Sethuram, Andrea Miville, Erin Keely
The Annals of Family Medicine May 2022, 20 (3) 262-265; DOI: 10.1370/afm.2812

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
The Provincial Spread and Scale of the Ontario eConsult Service: Evaluation of the First 2 Years
Clare Liddy, Rhea Mitchell, Sheena Guglani, Ariana Mihan, Claire Sethuram, Andrea Miville, Erin Keely
The Annals of Family Medicine May 2022, 20 (3) 262-265; DOI: 10.1370/afm.2812
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • Acknowledgments
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Family Physicians Stopping Practice During the COVID-19 Pandemic in Ontario, Canada
  • Communication Gaps Persist Between Primary Care and Specialist Physicians
Show more Research Brief

Similar Articles

Subjects

  • Methods:
    • Quantitative methods
  • Other research types:
    • Health services
  • Core values of primary care:
    • Access
    • Coordination / integration of care
  • Other topics:
    • Health informatics
    • Communication / decision making

Keywords

  • digital technology
  • eReferral and eConsultation
  • heath care quality, access and evaluation
  • health services accessibility
  • telemedicine

Content

  • Current Issue
  • Past Issues
  • Past Issues in Brief
  • Multimedia
  • Articles by Type
  • Articles by Subject
  • Multimedia
  • Supplements
  • Online First
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Media
  • Job Seekers

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2023 Annals of Family Medicine