Potential Impact of COVID-19 Pandemic on Journal Reviewer Responses

Context: Reviewer recruitment for submissions to peer-refereed journals has become increasingly difficult. The impact of the COVID-19 Pandemic upon responses from invited reviewers is hypothesized to be large.

Objective: To test whether non-responses (including no response or failure to complete) to reviewer invitations have increased after the onset of the COVID-19 Pandemic.

Study Design and Analysis: Retrospective secondary analysis.

Dataset: Records of reviewer responses to the family medicine education journal PRiMER, for all submissions between the initiation of the journal (2016) and May 10, 2023. Analysis was performed via contingency table & chi-square test.

Population Studied: The entire population of invited reviewers for the journal PRiMER.

Instrument: ScholarOne editorial database.

Outcome Measures: Percentages of invited reviews returned (recommendation of Accept, Major or Minor Revision, or Reject), vs. actively declined (Invitation declined or reviewer indicated they were unavailable), vs. no response (either no response to the invitation, or review uncompleted), before onset of the COVID-19 Pandemic in North America (<2020) vs. after onset (2020 – May, 2023).

Results: Responses for 2130 anonymized invited reviewers for 527 new and revised submission to the journal PRiMER were included in the analysis. Completion of reviews declined slightly from pre-pandemic (38.1%, n=498) to the period from 2020 onward (36.2%, n=498), and actively declined submissions decreased slightly (pandemic: 24.1%, n=331 vs. pre-pandemic: 27.9%, n=211). However, invitations with no response and/or completion increased between the pre-pandemic (33.9%, n=256) and pandemic (39.7%, n=546) periods. The differences were significant (p=.021), driven by the 5.8% increase in nonresponses to review invitations.

Conclusions: The years coinciding with the COVID-19 Pandemic saw a marked increase in failure to respond and/or complete review invitations at one journal. Other factors may be involved, including the explosion of new journals and journal submissions (necessitating an increase in the number of invitations received by potential reviewers), as well as many other potential factors. However, these findings lend support to the hypothesis that pandemic-related stress upon the reviewer pool has led to increased difficulty in obtaining timely and complete responses to review invitations. This information should be followed up in other journals and with qualitative data.