Evaluation of an Electronic Consultation Service for COVID-19 Care

Design, Participants, and Setting

This study is a mixed methods analysis of eConsult cases submitted by PCPs to the COVID-19 specialist group through 2 Ontario-based eConsult services between March and September 2020. This study received ethics approval from the Ontario Health Science Network Research Ethics Board. Participants in the study were PCPs who were registered with either eConsult service and initiated at least 1 eConsult during the study period to the COVID-19 specialist group.

eConsult Services

Two multispecialty eConsult services currently operate in Ontario under the umbrella of Ontario eConsult. The first service, called Champlain BASE (Building Access to Specialists through eConsultation), was originally implemented as a pilot initiative in 2010 across the Champlain region of eastern Ontario, a region of 1.3 million people spanning 18,000 km², the majority of whom live in the Ottawa metropolitan area.⁶ From its initial implementation, BASE has demonstrated improved access to specialty care, reduced need for face-to-face specialist visits, cost savings, and high physician satisfaction.^7-11 The second service, the Ontario eConsult Service, made access to eConsult available to PCPs across the province starting in 2018. Both services are independent, asynchronous online platforms that facilitate secure communication between a PCP and a specialist. Both are funded through the Ministry of Health and Long-Term Care of Ontario, and are provided at no charge to the PCP. Ten specialists were available to respond to COVID-19 eConsult cases during the study period (1 for the Champlain BASE eConsult service and 9 for the Ontario eConsult Service).

A PCP initiates an eConsult by logging onto a secure web portal and filling out a structured electronic form. The practitioner submits the eConsult case along with any pertinent attachments such as laboratory or imaging data to the COVID-19 specialist. The COVID-19 specialty group is made up of infectious disease physicians with a specific focus on the disease and a public health background. When a case is closed, the PCP is prompted to complete a close-out questionnaire that gathers information on the actions taken after the consultation (Supplemental Table 1).

Before the COVID-19 group was formed, practitioners could submit only patient-specific questions. Given the population health impact of the pandemic, both platforms were adapted to additionally allow non–patient-specific, population-based clinical questions.

Data Collection and Analysis

For the quantitative component of this study, we computed descriptive statistics to examine the overall use of the COVID-19 specialist group from March to September 2020. Data from Ontario eConsult and Champlain BASE were analyzed to determine the number of cases submitted to the COVID-19 specialty group. Time stamps for each eConsult communication were assessed to examine the response interval for specialist responses, and total self-reported time billed was used to determine the amount of time each specialist spent responding to the eConsult.

The qualitative component of this study involved an inductive and deductive content analysis of the PCP eConsult questions from the eConsults that were submitted through Champlain BASE between March and September 2020. At the time of the analysis, case-level data were accessible for Champlain BASE but not for Ontario eConsult. Two investigators (G.E.G., J.S.) independently performed open coding of eConsult cases and grouped similar cases into emergent themes. They discussed their independent assessments using an iterative approach to come to a consensus on a coding framework and to develop a final list of themes. In many cases, a single eConsult case had multiple questions; therefore, each case was not restricted to a single theme. In total, 20 eConsult cases were analyzed until saturation was achieved. Once themes were identified, a single investigator (J.S.) completed a deductive analysis of the remaining eConsult cases. Identified themes from this analysis of cases from Champlain BASE were plotted against a timeline of changes in public health guidelines and measures to assess whether the content of questions changed as public health measures were adapted.

We performed a descriptive analysis of all eConsult close-out questionnaires completed by the PCPs. Specifically, we sought to determine the PCP’s course of action after the eConsult (eg, face-to-face referral avoided, face-to-face referral still needed) and to gain insight into the nature of the advice given to the PCP (eg, new advice given, advice confirmed course of action initially contemplated).

Content of Clinical Questions

The content analysis assessed the clinical questions PCPs asked in the 51 cases submitted through Champlain BASE. It revealed 5 main themes: (1) precautions or workplace accommodations for high-risk or special populations, (2) diagnostic clarification and/or need for COVID-19 testing, (3) guidance on self-isolation and return-to-work criteria, (4) guidance on personal protective equipment and mask exemptions, and (5) management of chronic symptoms of COVID-19 (Table 1). The most common topic pertained to precautions or workplace accommodations for high-risk/special populations (17 cases, 35%), followed by diagnostic clarification and/or need for COVID-19 testing (15 cases, 31%). In total, 11 cases (22%) were population or practice-level questions, whereas the remaining 40 cases (78%) were patient-specific questions.

Notably, question content aligned with events and policies in the province and evolving clinical information about COVID-19 (Figure 1). For example, in March and April, as the world was learning more about symptoms of the infection and testing was restricted to patients with certain symptoms in Ontario, 37% of the eConsult cases were related to diagnostic clarification and the need for COVID-19 testing. In June, once testing in the province was opened to everyone, including asymptomatic individuals, no questions were related to the need for COVID-19 testing. Furthermore, in August, 63% of eConsult cases submitted related to work exemptions for teachers with preexisting health conditions as schools began preparing to reopen in September.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Timeline of emerging themes of COVID-19 eConsults and policy changes in Ontario.

BCG = bacille Calmette-Guerin; eConsult = electronic consultation.

Outcomes of eConsult

PCPs completed the post-eConsult questionnaire for 282 of the 289 cases submitted. Through the consultation, they were able to confirm a course of action that they had initially contemplated in 153 (54%) of the cases and received advice on a new or additional course of action in 111 cases (39%) (Figure 2). In 69 (24%) of cases, PCPs had initially contemplated a face-to-face referral but found it unnecessary after the eConsult, while a referral was needed in only 15 (5%) of cases (Figure 3). Notably, this last finding diverges from eConsult cases submitted to non-COVID specialties during the same period: survey data were available for 32,913 cases, of which 7,657 (23%) required a referral appointment with a specialist.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Outcomes of COVID-19 eConsult cases (N = 282 cases).

BASE = Building Access to Specialists through eConsultation; eConsult = electronic consultation.

^a Response option available only for the Champlain BASE eConsult service.

^b Response option available only for the Ontario eConsult service.

Note: Survey results were unavailable for 7 eConsult cases (all in the Ontario eConsult service).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Referral outcomes of COVID-19 eConsult cases (N = 282 cases).

eConsult = electronic consultation.

^a Response option available only for the Ontario eConsult service.

Note: Survey results were unavailable for 7 eConsult cases (all in the Ontario eConsult service).

Key Lessons

Our findings reveal several key lessons in this context. First, questions related to the pandemic were particularly conducive to resolution through eConsult: only 5% of cases submitted to the COVID-19 specialists required a formal referral, compared with the 29% to 45% seen with previous eConsult studies among other specialties.^7,9-11 This low formal referral rate may have been due in part to specialists adapting their recommendations for in-person visits given the increased risks of exposure during a pandemic. For example, a recent study by Frencher et al¹⁴ looked at eConsults done before and during the COVID-19 pandemic and found a decrease in those recommending face-to-face referrals. The authors suggested this difference was likely due to additional considerations arising during a pandemic, such as the risk of spread and limited personal protective equipment, that prompted specialists to consider limiting in-person visits. It is worth noting, however, that the share of eConsults submitted to non-COVID specialties during the study period that resulted in referral (24%) was more in line with pre-COVID patterns, suggesting that practitioners may have been particularly hesitant to refer COVID cases.

Second, uptake of the service was fast, with case volumes surging in the first 2 months postimplementation (March and April) before decreasing and stabilizing in subsequent months. This pattern was likely due in part to the lack of knowledge and familiarity with COVID-19 among practitioners in the early months following initiation of the lockdown, as research about the symptomatology, modes of transmission, and high-risk populations continued to evolve at a rapid pace.^3,4 As new information about COVID-19 emerged, changes in such policies pertaining to what symptoms were considered indicative of COVID-19,¹⁵ which patients were eligible for testing,^16,17 and the extent of lockdown measures imposed^18,19 all appear reflected in the pattern of questions posed (Figure 1). Regardless of the cause, innovators implementing such a service during a pandemic must be prepared for an initial burst of cases; therefore, it is essential that they have an adequate number of specialists on board and an interface capable of handling the flow.

Third, the content of questions varied with time and often aligned with events in Ontario and evolving clinical information about COVID-19. For example, eConsults related to masks increased in July and August as evidence about their importance evolved and they became mandatory in many settings (Figure 1).^20-22 Similarly, the majority of cases in August were related to work exemptions for teachers with preexisting health conditions as schools were preparing to reopen in September.^23,24 By evaluating clinical questions submitted by PCPs, eConsult services may play a role in providing public health officials with insight into new trends and events in the community as they occur. This ability to closely mirror community events in conjunction with the ease of adapting the service to respond to COVID-19 demonstrates eConsult’s potential as a tool for combatting the pandemic. Likewise, assessing the content of questions allows innovators to adapt the service accordingly. For instance, since the completion of this study, the Champlain BASE and Ontario eConsult services have added other COVID-19 subspecialty groups, including COVID-19 vaccine-allergy and immunology, COVID-19 and pregnancy, COVID-19 and autoimmune diseases, and a COVID-19 public health group, to meet the changing needs of PCPs and patients.

Many regions around the world have had to increasingly rely on virtual care solutions to provide patient care to help limit the spread of COVID-19.^25-27 Although a number of studies have evaluated telehealth during the pandemic,^28,29 few studies have assessed use of an eConsult system.^14,30 A study by Phadke et al³⁰ assessed the impact of the COVID-19 pandemic on the use of an eConsult system in Massachusetts. They assessed the change in volume of traditional ambulatory consults vs eConsults before and after the state of emergency was announced in their region in March 2020. Results from their study showed that there was a significant increase in the relative volume of eConsults providing insight into the increased demand for virtual solutions during a pandemic.

Limitations

Our study has several limitations. First, the qualitative analysis used data from a single health region in Ontario collected over a short period. As mentioned above, at the time of writing, there was no data-sharing agreement yet in place to allow analysis of the eConsult cases completed via Ontario eConsult. Although the Champlain region is diverse, focus on this region alone limits the generalizability of our findings and may not reflect the nature of the questions being asked throughout Ontario. That being said, our results were externally validated by members of our team who are answering eConsult questions from across Ontario and confirm that the identified themes reflect questions from across the province. We have limited follow-up data and are unable to determine whether completed eConsults have resulted in adverse events or eventual face-to-face referrals. Demographic data are not collected through either service, so we were unable to present information on the characteristics of PCPs using the service. Lastly, this study did not examine eConsults from the general infectious disease specialty service. It is plausible that some questions related to COVID-19 were submitted to that group instead.

Conclusions

In conclusion, the COVID specialist group available through the Champlain BASE and Ontario eConsult services was rapidly adopted across the province and demonstrated prompt response times and facilitated avoidance of unnecessary face-to-face referrals. An assessment of clinical question content revealed 5 key themes, providing insight into potential gaps in knowledge among PCPs that could help guide medical education and policy.