Abstract
BACKGROUND The efficacy, effectiveness, and safety of the approved nirmatrelvir/ritonavir regimen for treatment of laboratory-confirmed mild/moderately severe COVID-19 remains unclear.
METHODS We systematically identified randomized controlled trials (RCTs) and real-world studies (RWS; observational studies) of the efficacy/effectiveness and/or safety of the approved nirmatrelvir/ritonavir regimen for COVID-19. We pooled appropriate data (adjusted estimates for RWS) using an inverse variance, random-effects model. We calculated statistical heterogeneity using the I2 statistic. Results are presented as relative risk (RR) with associated 95% CI. We further assessed risk of bias/study quality and conducted trial sequential analysis of the evidence from RCTs.
RESULTS We included 4 RCTs (4,070 persons) and 16 RWS (1,925,047 persons) of adults (aged ≥18 years). One and 3 RCTs were of low and unclear risk of bias, respectively. The RWS were of good quality. Nirmatrelvir/ritonavir significantly decreased COVID-19 hospitalization compared with placebo/no treatment (RR = 0.17; 95% CI, 0.10-0.31; I2 = 77.2%; 2 RCTs, 3,542 persons), but there was no significant difference for decrease of worsening severity (RR = 0.82; 95% CI, 0.66-1.01; I2 = 47.5%; 3 RCTs, 1,824 persons), viral clearance (RR = 1.19; 95% CI, 0.93-1.51; I2 = 82%; 2 RCTs, 528 persons), adverse events (RR = 1.41; 95% CI, 0.92-2.14; I2 = 70.6%; 4 RCTs, 4,070 persons), serious adverse events (RR = 0.82; 95% CI, 0.41-1.62; I2 = 0%; 3 RCTs, 3,806 persons), and all-cause mortality (RR = 0.27; 95% CI, 0.04-1.70; I2 = 49.9%; 3 RCTs, 3,806 persons), although trial sequential analysis suggested that the current total sample sizes for these outcomes were not large enough for conclusions to be drawn. Real-world studies also showed significantly decreased COVID-19 hospitalization (RR = 0.48; 95% CI, 0.37-0.60; I2 = 95.0%; 11 RWS, 1,421,398 persons) and all-cause mortality (RR = 0.24; 95% CI, 0.14-0.34; I2 = 65%; 7 RWS, 286,131 persons) for nirmatrelvir/ritonavir compared with no treatment.
CONCLUSIONS Nirmatrelvir/ritonavir appears to be promising for preventing hospitalization and potentially decreasing all-cause mortality for persons with mild/moderately severe COVID-19, but the evidence is weak. More studies are needed.
- nirmatrelvir/ritonavir
- Paxlovid
- laboratory-confirmed COVID-19
- systematic review
- quantitative methods: meta-analysis
- trial sequential analysis
- efficacy/effectiveness
- safety
INTRODUCTION
The virus responsible for the COVID-19 pandemic, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), remains in circulation. It mutates and keeps infecting large proportions of populations, with attributable substantial disease and economic burden.1-3 The International Health Regulations Emergency Committee of the World Health Organization (WHO) has downgraded the COVID-19 pandemic, and many of the previously instituted public health preventive measures have been relaxed in most jurisdictions. The focus has largely shifted to case management, although vaccination is actively promoted for prevention and amelioration of infection severity, and vaccines are now more broadly available in many jurisdictions.4-6
An orally administered antiviral drug combination (300 mg nirmatrelvir with 100 mg ritonavir) administered twice daily over a period of 5 days has been approved in many jurisdictions for the treatment of nonhospitalized persons aged ≥12 years, weighing ≥40 kg, and presenting with mild/moderately severe disease, ideally within 5 days of symptom onset, and who might be at increased risk of worsening severity, hospitalization, and death.7,8 This drug combination has also been authorized for use in hospitalized patients with COVID-19 but only if presenting within 5 days of symptom onset, severity of illness is judged to be mild/moderate (not requiring supplemental oxygen), and the patient is at a high risk of developing severe illness.7
Nirmatrelvir on its own has been shown to have antiviral potential against all coronaviruses that are known to infect humans; it is a protease inhibitor with demonstrated activity against a viral protease, Mpro, which plays an essential role in viral replication.7,9 Ritonavir has strong inhibitory ability against cytochrome P450 3A4 and pharmacokinetic boosting ability that has been shown to increase the potency of HIV protease inhibitor; when coadministered with nirmatrelvir, it increases the concentration of nirmatrelvir to the target therapeutic range for optimum activity against coronaviruses.7
Approval of nirmatrelvir/ritonavir by the US Food and Drug Administration was via an Emergency Use Authorization, an approval that facilitates the availability and use of a previously unapproved health technology during public health emergencies if there are no adequate, approved, and available alternatives.10 As such, the efficacy and safety of the regimen were not necessarily proven via appropriately conducted and highly powered randomized controlled trials (RCTs) as is typical with approvals of health technologies by the Food and Drug Administration. Evidence from RCTs and real-world studies (RWS; observational studies) is accumulating; however, with mostly conflicting findings.
There are a few published reviews of the efficacy or effectiveness and safety of this drug regimen compared with usual care for persons with mild/moderately severe COVID-19, but they are mostly narrative, and the few that included meta-analysis differed in methodologic approaches, included studies, and assessed outcomes. Even so, the important clinical questions of efficacy and safety remain unanswered, and none of the reviews conducted trial sequential analysis (TSA), a unique cumulative meta-analysis that provides information on the adequacy of the overall sample size of pooled estimates to inform evidence-based clinical practice and guide future reviews on the topic. In view of the gaps in knowledge and accumulating evidence, we systematically summarized published evidence on the efficacy, effectiveness, and safety of nirmatrelvir/ritonavir for COVID-19 and conducted a TSA of the evidence from RCTs.
METHODS
The methods for this review are as reported in our previous systematic review and meta-analysis publications on the use of remdesivir and other antiviral drugs for the treatment of COVID-1911,12; however, for the present review, we made the ad hoc decision to conduct a rapid review and to include real-world evidence and evidence reported in preprint articles (both of which we initially planned to exclude) because of the paucity of evidence from RCTs, which were our initial focus. In brief, we registered a systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO; registration no. CRD42020216817) and conducted a rapid review in accordance with WHO13 as well as Methodological Expectations of Cochrane Intervention Reviews (MECIR) guidelines.14 We report our findings following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)15 and Meta-Analyses Of Observational Studies in Epidemiology (MOOSE) guidelines.16
Literature Search
A health sciences librarian developed a literature search strategy for the MEDLINE bibliographic database, and an independent health sciences librarian reviewed the search strategy using the Peer Review of Electronic Search Strategies (PRESS) checklist.17 The librarian then revised and adapted the revised search strategy (Supplemental Appendix) for Embase, LitCovid,18 the Cochrane COVID-19 Study Register,19 and the WHO COVID-19 Research Database.20 We searched for literature on May 23, 2023. Retrieved citations were managed in EndNote version X9.2 (Clarivate).
Selection Criteria
We imported retrieved citations into a specially designed Microsoft Access 2016 database (Microsoft Corp), and an experienced systematic reviewer (G.N.O.) performed a blind double-screen using a 2-stage sifting approach to review the title/abstract and full-text articles of relevant citations. The reviewer documented the number of ineligible citations at the title/abstract screening stage and the number of and reasons for ineligibility at the full-text article screening stage and resolved any discrepancies between their screening decisions by rescreening the involved citations/articles or involving another reviewer.
Our initial focus was on published full-text articles of RCTs reporting on the efficacy and/or safety of the approved regimen of nirmatrelvir/ritonavir for treatment of laboratory-confirmed (reverse-transcription polymerase chain reaction or antigen test) persons with mild/moderately severe COVID-19 (300 mg nirmatrelvir with 100 mg ritonavir administered twice daily over a period of 5 days). The comparator was placebo or no treatment (usual care). At the preliminary stage of the review, however, during which we informally scoped the available evidence, it became apparent that there was likely a paucity of published RCTs. We therefore made the ad hoc decision to include real-world evidence, which appeared to be substantially published. We also decided to include evidence reported in preprint articles (not yet published) because we observed no differences between data reported in published articles and their associated preprint articles. The efficacy outcomes of interest were viral clearance, clinical progression (worsening symptoms), hospitalization, and all-cause mortality, and the safety outcome was adverse events, including serious adverse events, attributable to treatment. We excluded conference abstracts, commentaries, reviews, letters to the editor, and opinion pieces.
Data Extraction
The same experienced reviewer (G.N.O.) extracted data from the included articles using Microsoft Excel 2016 (Microsoft Corp) and rechecked the extracted data for errors. The extracted data included basic publication and study information, study population characteristics, information regarding interventions and comparators, outcomes assessed, details relevant to risk of bias and study quality assessments, and study results based on an intention-to-treat (strictly as randomized) analysis for the RCTs and appropriately matched and/or multivariable adjusted analysis for RWS. The reviewer assessed the risk of bias in the included RCTs using the Cochrane risk of bias assessment tool for RCTs (RoB, version 2.0.2)21 and reassessed and corrected any errors in their assessment. The reviewer also assessed study quality of the RWS using the US National Heart, Lung, and Blood Institute’s quality assessment tool for observational cohort and cross-sectional studies22 and reassessed and corrected any errors in the assessment; this quality assessment tool comprises 14 assessed criteria to determine study quality. These include clarity of research questions/objectives, appropriateness of selection of study population/participants, justification for sample size, and quality of measurements and analysis of data. A study was judged to be of high quality if it satisfied all assessed parameters, of good quality if it satisfied all but 1 parameter, of moderate quality if it did not satisfy 2 to 4 parameters, and of poor quality if it did not satisfy >4 parameters.
Data Synthesis and Analysis
We synthesized the characteristics and risk of bias/study quality assessments of the included studies in tabular form. For consistency and appropriateness of pooled estimations, we converted reported odds ratios in a few RWS to relative risk (RR) using the methods suggested by Zhang and Yu.23 Given the difficulty of converting hazard ratios (HRs) to RRs, we treated a few reported HRs for most of the follow-up period as RRs because the proportions of the outcomes were small (<10%), and therefore the HR estimates would likely be similar to, and comparable with, the RR estimates.24,25 Where possible (appropriately reported data from ≥2 studies), we conducted meta-analyses of the data with the longest follow-up periods using inverse variance random-effects models implemented in Stata version 13 (StataCorp LLC). We calculated pooled estimates of effects using RR and presented the estimates and associated 95% CIs. We assessed statistical heterogeneity between the pooled estimates using the I2 statistic.26 With an expectation of considerable variation in patient characteristics and usual clinical care across the included studies, we anticipated varying levels of heterogeneity of the pooled effect estimates. We explored heterogeneity if ≥70% and there were clearly defined variables to enable appropriate exploration. Only when there were ≥10 study results contributing to a pooled analysis did we assess publication bias using the Egger test.27 We graded the quality of the evidence for the outcomes from RCTs using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.28
To assess if the required information size was attained for each of the assessed outcomes from RCTs only, we conducted a TSA using TSA software version 0.9.5.5 Beta (Copenhagen Trial Unit Centre for Clinical Intervention Research [www.ctu.dk/tsa]) and following the methods outlined by Wetterslev and colleagues.29 We used a random-effects model with a conventional test boundary of P < .05, an assumed 2-sided test of significance, a power level of 80%, and a 5% type 1 error for information size calculations, adjusted by between-study heterogeneity.
RESULTS
Among a total of 1,104 citations retrieved from the literature search, we included 3 RCTs30-32 and 16 RWS33-48 that met the eligibility criteria (Figure 1). One RCT was conducted in multiple countries,31 1 was conducted in Russia,30 and 1 was conducted in China32 (Table 1). Two RCTs were of open-label type,30,32 and 1 was double-blinded.31 Two of the RCTs were funded by industry sources,30,31 and 1 was funded by a nonindustry source.32 There were 264 participants in each of 2 RCTs30,32 and 2,246 participants in 1,31 with all participants nonhospitalized in 2 RCTs30,31 and hospitalized (possibly for isolation) in 1,32 although all were of mild/moderate COVID-19 severity. After peer review, we included data from a yet-to-be published large multinational RCT (nonhospitalized persons).49 Information regarding inclusion criteria for participation in the included 4 RCTs is presented in Supplemental Table 1. One RCT was judged to be of low risk of bias49 and the rest of unclear risk of bias (Supplemental Table 2). Of the RWS, 9 were conducted in the United States,33-39,44,48 3 in Hong Kong Special Administrative Region, China,45-47 and 1 each in Canada,43 Greece,41 Israel,40 and South Korea42 (Table 2). Sample size for the RWS ranged from 819 to 699,848 persons, and all were outpatients. The RWS were all judged to be of good quality (Supplemental Table 3).
Modified PRISMA flowchart.
PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT = randomized controlled trial.
Summarized Characteristics of Included Randomized Controlled Trials
Summarized Characteristics of Included Real-World Studies
Evidence From Randomized Controlled Trials
Nirmatrelvir/ritonavir significantly decreased COVID-19 hospitalization compared with placebo/no treatment (RR = 0.17; 95% CI, 0.10-0.31; I2 = 77.2%; 2 RCTs, 3,542 persons) (Figure 2), but there was no significant decrease in worsening severity (RR = 0.82; 95% CI, 0.66-1.01; I2 = 47.5%; 3 RCTs, 1,824 persons) (Supplemental Figure 1), viral clearance (RR = 1.19; 95% CI, 0.93-1.51; I2 = 82%; 2 RCTs, 528 persons) (Supplemental Figure 2), adverse events (RR = 1.41; 95% CI, 0.92-2.14; I2 = 70.6%; 4 RCTs, 4,070 persons) (Supplemental Figure 3), serious adverse events (RR = 0.82; 95% CI, 0.41-1.62; I2 = 0%; 3 RCTs, 3,806 persons) (Supplemental Figure 4), and all-cause mortality (RR = 0.27; 95% CI, 0.04-1.70; I2 = 49.9%; 3 RCTs, 3,806 persons) (Supplemental Figure 5). Exploration of heterogeneity in the pooled estimates for viral clearance and adverse events was not possible, given the paucity of studies. Evidence for these outcomes were graded to be of low to moderate certainty (Supplemental Table 4).
Forest plot of nirmatrelvir/ritonavir vs no treatment/placebo for decreasing hospitalization among adults with laboratory-confirmed mild/moderately severe COVID-19 (RCTs).
EPIC-SR = Evaluation of Protease Inhibition for COVID-19 in Standard-Risk Patients; RCT = randomized controlled trial; RR = relative risk.
Using the 4% risk in the control group, an RR decrease of 77% in the treatment group, sample heterogeneity (I2 = 77%), the pooled sample size (n = 3,542 persons), and the estimated study sample size (n = 3,994 persons), the Z curve of the TSA was observed to cross the monitoring boundary without crossing the futility area, thus confirming the significant decrease in COVID-19 hospitalization for nirmatrelvir/ritonavir (Figure 3). Using the 17% risk in the control group, an RR decrease of 40% in the treatment group, and sample heterogeneity (I2 = 48%), the pooled sample size (n = 1,824 persons) was less than the estimated study sample size (n = 16,708), and therefore the observed no significant decrease in worsening COVID-19 severity with nirmatrelvir/ritonavir was inconclusive (Supplemental Figure 6). Similarly, using the 68% risk in the control group, an RR decrease of 19% in the treatment group, and sample heterogeneity (I2 = 82%), the pooled sample size (n = 528 persons) was less than the estimated study sample size (n = 2,030) for viral clearance, and therefore the observed no significant decrease in viral clearance with nirmatrelvir/ritonavir was inconclusive (Supplemental Figure 7). Further, using the 11% risk in the control group, an RR decrease of 41% in the treatment group, and sample heterogeneity (I2 = 71%), the pooled sample size (n = 4,070 persons) was less than the estimated study sample size (n = 11,971) for adverse events, and therefore the observed no significant decrease in adverse events with nirmatrelvir/ritonavir was inconclusive (Supplemental Figure 8). In addition, using the 0.9% risk in the control group, an RR decrease of 40% in the treatment group, and sample heterogeneity (I2 = 0%), the pooled sample size (n = 3,806 persons) was less than the estimated study sample size (n = 17,317) for serious adverse events, and therefore the observed no significant decrease in serious adverse events with nirmatrelvir/ritonavir was also inconclusive (Supplemental Figure 9). Finally, using the 1% risk in the control group, an RR decrease of 73% in the treatment group, and sample heterogeneity (I2 = 50%), the pooled sample size (n = 3,806 persons) was less than the estimated study sample size (n = 13,479) for all-cause mortality, and therefore the observed no significant decrease in all-cause mortality with nirmatrelvir/ritonavir was inconclusive (Supplemental Figure 10).
Trial sequential analysis of nirmatrelvir/ritonavir vs no treatment/placebo for decreasing hospitalization among adults with laboratory-confirmed mild/moderately severe COVID-19 (RCTs).
RCT = randomized controlled trial.
Evidence From Real-World Studies
Nirmatrelvir/ritonavir appeared to significantly decrease hospitalization (RR = 0.48; 95% CI, 0.37-0.60; I2 = 95.0%; 11 RWS, 1,421,398 persons) (Figure 4) and all-cause mortality (RR = 0.24; 95% CI, 0.14-0.34; I2 = 65%; 7 RWS, 286,131 persons) (Figure 5) compared with no treatment but with a high level of heterogeneity of the pooled estimates, particularly for hospitalization, most likely due to differences in disease severity, age distribution, proportions of those with chronic conditions and vaccinations, and differences in usual care (standard treatments) across studies. Exploration of heterogeneity was not possible, given the lack of clarity on the above to inform meta-regression.
Forest plot of nirmatrelvir/ritonavir vs no treatment for decreasing hospitalization among adults with laboratory-confirmed mild/moderately severe COVID-19 (real-world studies).
RR = relative risk.
Forest plot of nirmatrelvir/ritonavir vs no treatment for decreasing all-cause mortality among adults with laboratory-confirmed mild/moderately severe COVID-19 (real-world studies).
RR = relative risk.
DISCUSSION
The evidence from this review suggests small but significant efficacy of the approved nirmatrelvir/ritonavir regimen in decreasing COVID-19 hospitalization and its effectiveness (from RWS) in decreasing hospitalization and all-cause mortality among persons with laboratory-confirmed mild/moderately severe COVID-19. However, these findings should be interpreted with caution given the limited evidence, particularly for RCTs, and the heterogeneous evidence, particularly for RWS. In addition, the findings might not be generalizable.
The reviewed studies mostly used nasopharyngeal swabs for specimen collection, and swabs were collected mostly within 5 days of COVID-19 symptom onset. Diagnostic tests included reverse-transcription polymerase chain reaction assay, nucleic acid amplification methods, and SARS-CoV-2 antigen assay using immunochromatographic analysis. Whereas many of the studies included mostly persons at high risk of hospitalization, study sample size, population demography, proportions of persons with chronic conditions, participant eligibility criteria, and follow-up periods varied considerably. Further, across the studies there were variable proportions of vaccinated persons, number of vaccinations, vaccine types, and usual clinical care, which was of a composite nature. Whereas these observed differences likely had an effect on our findings, the extent is not easily quantifiable.
Viral shedding is influenced by characteristics of the virus strain and host factors including preexisting immunity from previous infection or vaccinations.50 Therefore, considering the varying strains of SARS-CoV-2, even though the Omicron variant was predominantly in circulation during most of the studies, and differences in study participant characteristics, detection of the virus would almost certainly differ across studies. Even so, SARS-CoV-2 viral shedding might precede symptom onset, and considering that symptom onset is self-reported, with likely recall and social desirability biases, it is reasonably possible that some persons might no longer have been shedding a detectable amount of virus by the time of recruitment into a study;51,52 there might therefore be a greater risk of negative results, with the potential for invalid estimations of treatment effect.
A Cochrane review of the efficacy of nirmatrelvir/ritonavir for the treatment of mild/moderately severe COVID-19 was based on a single RCT,53 and the updated review54 was based on 2 RCTs (published during peer review of our present review), both of which are included in the present review. Another systematic review included 2 RCTs, which are also included in this present review, but the authors pooled the estimates from the RCTs together with estimates from RWS55; thus not allowing for the assessment of estimates from RCTs and RWS separately. As such, there is no basis for comparison of our findings with the findings from these reviews, although our present review of the evidence from RCTs was methodologically similar to the Cochrane reviews. Li and colleagues systematically evaluated the effectiveness of nirmatrelvir/ritonavir for the treatment of mild/moderately severe COVID-19 and reported a decreased incidence of all-cause mortality or hospitalization within 30 days for vaccinated persons (RR = 0.53; 95% CI, 0.40-0.70; I2 = 81%), and similar to our present finding, a substantially decreased effectiveness when limited to all-cause mortality (RR = 0.40; 95% CI, 0.19-0.85; I2 = 23%).56 That review, however, included only 7 of the 16 studies included in the present review. Souza and colleagues also systematically evaluated the effectiveness of nirmatrelvir/ritonavir for the treatment of mild/moderately severe COVID-19 and reported a decreased risk of death by 62%, representing an odds ratio of 0.38 (95% CI, 0.30-0.46), and a decreased risk of hospitalization by 56%, representing an odds ratio of 0.44 (95% CI, 0.31-0.64),57 both of which are somewhat comparable to our findings, although we observed a substantially greater decrease in risk of death (76%). However, one of the studies included in the review by Souza and colleagues was on nirmatrelvir alone and not the approved combined nirmatrelvir/ritonavir regimen,58 another was strictly on persons with systemic autoimmune rheumatic disease,59 and a third did not report any adjusted estimates but rather unclear proportions (suppressed results).60 Compared with the above reviews, our present review included the greatest number of studies (both RCTs and RWS), and in addition, we performed a TSA on the assessed outcomes from RCTs, a substantial addition to the evidence base that aids interpretation of the findings and guides further RCTs on the assessed outcomes.
Nevertheless, the present review has some limitations. First, we conducted a rapid review, and as such did not completely adhere to the methodologic expectations of a full systematic review. However, unlike the traditional rapid review for which literature is searched in 1 or 2 databases, we searched for literature in 5 appropriate bibliographic databases, as would have been the case for a full systematic review, and instead of having a reviewer screen citations against the eligibility criteria and extract data from the included studies, we used a highly experienced reviewer for these purposes and further mitigated errors by using a unique self-applied double-blinded approach. Second, there were high levels of heterogeneity in a few pooled estimates, and we could not explore the heterogeneity, given the paucity of studies or lack of clarity on the important variables to inform appropriate meta-regression. As such, our findings might not be entirely precise or accurate and we cannot rule out biases associated with real-world evidence. Notwithstanding, we conducted the present review to the highest standards for a rapid review, and our findings provide new insights to answering previously unanswered clinical questions and might direct future research on the topic.
CONCLUSIONS
The approved nirmatrelvir/ritonavir regimen appears to be promising for preventing hospitalization and potentially all-cause mortality in adults with mild/moderately severe COVID-19. However, the evidence is weak, thus suggesting a need for more RCTs for a stronger evidence base before firm conclusions can be drawn. For now, the approved nirmatrelvir/ritonavir regimen should be treated as experimental and not a definitive antiviral drug treatment for COVID-19.
Acknowledgment
We thank Gail Matheson, MLIS (Neil John Maclean Health Sciences Library, University of Manitoba, Winnipeg, Manitoba, Canada) for performing a peer review of the MEDLINE search strategy.
Footnotes
Conflicts of interest: authors report none.
Author contributions: Conceptualization (G.N.O.), methodology (G.N.O., R.R.), data acquisition (G.N.O., N.A.), formal analysis (G.N.O., R.R.), validation (G.N.O., R.R.), draft of manuscript (G.N.O.), manuscript revisions (G.N.O., N.A., R.R.), final approval for submission (G.N.O., N.A., R.R.), accountability (G.N.O., N.A., R.R.), principal investigator (G.N.O.)
- Received for publication September 22, 2023.
- Revision received March 14, 2024.
- Accepted for publication March 27, 2024.
- © 2024 Annals of Family Medicine, Inc.