Abstract
PURPOSE To explore public knowledge, understanding of public health recommendations, perceptions, and trust in information sources related to COVID-19.
METHODS A cross-sectional survey of central Pennsylvanian adults evaluated self-reported knowledge, and a convergent, mixed methods design was used to assess beliefs about recommendations, intended behaviors, perceptions, and concerns related to infectious disease risk, and trust of information sources.
RESULTS The survey was completed by 5,948 adults. The estimated probability of correct response for the basic knowledge score, weighted with confidence, was 0.79 (95% CI, 0.79-0.80). Knowledge was significantly higher in patients with higher education and nonminority race. While the majority of respondents reported that they believed following CDC recommendations would decrease the spread of COVID-19 in their community and intended to adhere to them, only 65.2% rated social isolation with the highest level of belief and adherence. The most trusted information source was federal public health websites (42.8%). Qualitative responses aligned with quantitative data and described concerns about illness, epidemiologic issues, economic and societal disruptions, and distrust of the executive branch’s messaging. The survey was limited by a lack of minority representation, potential selection bias, and evolving COVID-19 information that may impact generalizability and interpretability.
CONCLUSIONS Knowledge about COVID-19 and intended adherence to behavioral recommendations were high. There was substantial distrust of the executive branch of the federal government, however, and concern about mixed messaging and information overload. These findings highlight the importance of consistent messaging from trusted sources that reaches diverse groups.
INTRODUCTION
Effective communication is critical for successful pandemic management.1-6 Ineffective messaging may result in failure of the public to comply with precautionary measures, propagation of fears and misconceptions, preventable overuse of health services,7,8 and inappropriate or inadequate policy decisions.9 Ineffective communication in the early stages of the current coronavirus disease 2019 (COVID-19) pandemic likely contributed to its rapid spread and high mortality,10 particularly in racial and ethnic minority groups.11 As patients turn to their primary care physicians for guidance and information about the pandemic, it is imperative that clinicians understand the perceptions, understanding, and health beliefs that play a key role in patients’ behavior; and the information sources used, so clinicians may better bridge key knowledge gaps.
A national telephone survey (n = 1,216) found a mixed understanding of COVID-19 recommendations.12 A telephone survey conducted in Chicago (n = 630) found lower COVID-19 knowledge among participants who were older, Black, unmarried, unemployed, retired, had poorer health, or had lower health literacy.12 This paucity of data identifies a critical gap in knowledge. The objective of this study is to address this gap to help clinicians provide better messaging to their patients, and leaders better disseminate evolving mitigation strategies.
METHODS
This cross-sectional online survey study was conducted from March 25-31, 2020, using a convergent mixed methods approach13 to explore public knowledge constructs related to COVID-19.
Survey Design
Because no validated questionnaires about COVID-19 existed, we adapted the European Standard Questionnaire on Risk Perception of an Infectious Disease Outbreak.14 Knowledge questions were based on information published on the US Centers for Disease Control and Prevention (CDC) website.15 The questionnaire was refined based on feedback from 13 individuals, using 2 rounds of cognitive interviewing procedures that utilized the think-aloud technique.16,17 The survey was further refined through pilot testing with a random sample of 1,000 potential participants to ensure adequate knowledge discrimination and qualitative sensibility.
The final survey included 65 to 92 items, depending upon branching logic, and assessed 4 constructs: (1) knowledge and corresponding confidence in that knowledge (15 items each; 30 total); (2) beliefs about and intention to follow CDC recommendations (10 items); (3) perceptions and concerns about COVID-19 and other infections (15 items); and (4) information sources (7 items). There were also 18 demographic questions, which included race and ethnicity categories and terms used by the US Census Bureau. During analysis, 8 of the 15 knowledge items were identified as a basic knowledge set (ie, not testing nuanced medical information).
Following the convergent mixed methods design, we developed 3 open-ended questions to qualitatively assess 3 of the 4 constructs (excluding knowledge): (1) “Please explain what, if anything, prevents you personally from following CDC recommendations about COVID-19”; (2) “What worries you most about the COVID-19 pandemic?”; and (3) “How do you feel about the way information regarding COVID-19 has been delivered to you?” The fourth qualitative question broadly captured additional thoughts participants wanted to contribute: “Is there anything else you would like to share regarding the COVID-19 pandemic?”
Sample
The questionnaire was sent to a convenience sample of adults enrolled in a central Pennsylvanian health care system, the database of which contains 121,573 unique, valid e-mail addresses. This sampling area consisted of primarily Caucasian and rural or suburban residents, some living in smaller urban areas. Potential participants received an e-mail from Penn State Health marketing inviting them to anonymously complete the survey through a link to an online electronic data capture tool, REDCap (Vanderbilt University)17; 1 reminder was sent 48 hours after the initial invitation e-mail.
Institutional Review Board Approval and Reporting Guidelines
The Pennsylvania State University College of Medicine Institutional Review Board approved the study protocol. This study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) and the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.
Statistical Analysis
Each true/false knowledge item was scored as correct or incorrect. Each knowledge item had a corresponding 5-point confidence score (1 = Extremely confident, 5 = Not at all confident [just guessing]). When a knowledge item was completed but the corresponding confidence score was missing, simple imputation was performed by constructing 2-way frequency tables based on both completed knowledge and corresponding completed confidence scores for each item, and imputing with the mode of the confidence score determined from each complete set for each corresponding knowledge item.
The 15 binary knowledge measures were analyzed via a generalized linear mixed-effects model (GLMM) with a logistic link function and a random effect for the participant, modeling the probability of a correct response. Weighted and unweighted analyses were performed; the latter including a weighting variable equal to the inverse of the confidence score. Subgroup comparison analyses for select demographic measures were assessed. The logarithm of the estimated odds ratios from the GLMM were transformed to probability estimates, with 95% CIs. The false-discovery rate was applied to report adjusted P values for statistical comparisons.
To quantify the relationship between perceptions regarding the likelihood and concern for various infectious disease diagnoses, and between the recommendation efficacy beliefs and intended adherence to those recommendations, we first descriptively summarized these data using 5 × 5 frequency tables. A GLMM, with a cumulative logistic link function and a random effect for the participant, was used to perform a bivariate analysis between likelihood of contraction and efficacy belief with concern of diagnosis and intention to follow the recommendation, respectively. Probability estimates and their 95% CIs were calculated, along with the intraclass correlation (ICC) and corresponding 95% CI. When combinations of responses were sparse or skewed, collapsed category responses were explored to analyze the bivariate relationship with symmetry. Even after collapsing, the ICC was still occasionally not reportable due to instability. SAS Version 9.4 (SAS Institute Inc) was used for all statistical analysis.
Qualitative Analysis
Stratified purposive sampling18 was used to select 250 qualitative responses with equal representation from each of the following strata: race (nonminority vs minority), education level (lower vs higher), and sex (male vs female). Conventional content analysis was used to analyze survey responses using NVivo 12 (QSR International).19 Two analysts reviewed the 250-participant surveys to generate a preliminary codebook representing categories and codes that emerged. Data saturation was achieved within approximately the first 50 responses. Codes within each category were then defined and iteratively discussed.
Four analysts coded an additional 20 responses using the constant comparison method.20 The codebook was refined through group discussion. The refined codebook was used by 2 analysts to analyze an additional 40 responses each. A third analyst adjudicated differences in codes. Upon reaching inter-rater reliability of 0.65, the remainder of the responses were coded. All 250 records were coded by 2 coders.
Mixed Methods Integration
Qualitative themes were merged and aligned with conclusions from the quantitative data. Themes are presented in a joint display for each quantitative construct.21
RESULTS
Participation
The e-mail was opened by 53,585 of those who received it (44.1%). Of those who opened the e-mail, 11,675 opened the link (21.8%); 8,072 participants completed portions of the survey (69.1%). Analysis was conducted on the 5,948 consenting respondents who answered at least 1 knowledge and 1 risk-stratification question (73.7% of all who opened the survey, 50.9% of those who opened the survey link, 11.1% of those who opened the e-mail, and 4.8% of the entire list).
Knowledge
Respondents were predominantly female, Caucasian, with nonmedical professions (Table 1). Knowledge results are reported in Table 2. The estimated probability of correct response for the 8-item basic knowledge score, weighted with confidence, was 0.79 (95% CI, 0.79-0.80) and for the full 15-item knowledge assessment, 0.75 (95% CI, 0.74-0.75). The knowledge item that yielded the lowest estimated probability of a correct response pertained to available nonprescription treatments for COVID-19 (Table 2). Respondents with a Bachelor’s degree or higher had 1.64 times higher odds of responding correctly to the 8 basic knowledge items than those without a Bachelor’s degree (95% CI, 1.55-1.73; (Supplemental Table 1, available at https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1). There were no differences in knowledge based on age or sex (Supplemental Table 2, Supplemental Table 3, and Supplemental Table 4, available at https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1).
Beliefs About CDC Recommendations and Intended Behavior
The majority of respondents believed that social isolation would “most certainly” decrease the spread of COVID-19 in their community and reported that they intended to adhere to this recommendation (65.2%; Table 3). The ICC for the relationship between belief about social isolation effectiveness and the intent to follow this recommendation was moderate (0.68, 95% CI, 0.65-0.71). Analysis of other recommended behaviors are reported in Supplemental Table 5 (https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1).
Responses to the question, “What, if anything, prevents you personally from following CDC recommendations about COVID-19?” revealed 2 major themes (Table 3). While most participants reported no or minimal barriers to following CDC recommendations (Theme 1), participants that reported barriers gave reasons of essential life or medical needs (Theme 2).
Perceptions and Concerns Related to COVID-19 and Influenza Risk
Most respondents (93.5%) expressed concern (“slightly concerned,” “concerned,” or “very concerned”) about being diagnosed with COVID-19 compared with 70.0% regarding influenza diagnosis in the next year. Further, 75.6% of respondents felt it was “possible,” “likely,” or “very likely” that they would be diagnosed with COVID-19 in the next year, compared with 64.7% when asked about their likelihood of being diagnosed with influenza. This descriptive contrast may be influenced by whether respondents received vaccinations or have knowledge of vaccination efficacy, which our survey did not address. The ICC for the relationship between the perceived likelihood of COVID-19 infection and concern about being diagnosed with COVID-19 was 0.19 (95% CI, 0.16-0.22; (Supplemental Table 6, https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1)) and 0.11 (95% CI, 0.07-0.15) for the flu, indicating weak relationships (Supplemental Table 7, https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1).
Three themes emerged in response to the question, “What worries you the most about the COVID-19 pandemic?” (Supplemental Table 6). First, respondents most commonly indicated that their worries were related to contracting COVID-19 themselves or that at-risk family members would “get it” (Theme 3). Participants often described concerns about delayed testing impacting abilities to effectively manage the virus, but not about severity of illness. Second, respondents noted concerns related to public health issues (Theme 4), in particular about asymptomatic spread of the virus and community members not practicing social distancing. Third, respondents had anxieties related to economic impact and societal disruptions, ranging from “the economy in general” to the impact of layoffs on workers and small businesses (Theme 5).
Information Sources and Trust
The single most trusted source by respondents for health information was government websites (ie, CDC, National Institutes of Health [NIH], and the World Health Organization [WHO]) (42.8%), followed by television news (27.2%). Three themes emerged related to information sources and trust (Supplemental Table 8, https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1). Respondents reported distrust of information provided by the executive branch of the federal government (Theme 6). They describe concerns about politicizing the response to the pandemic and negative feelings about COVID-19’s media coverage (Theme 7). Respondents felt overwhelmed by “information overload,” frustrated by mixed messaging, and concerned about misinformation. To counter this, participants described a need to “filter” information from multiple sources. When asked about what information was viewed as insufficient, respondents commonly described questions about symptom management (ie, over-the-counter treatment of symptoms), diagnostic testing, viral behavior, and immunity (ie, how long are patients contagious; Theme 8).
DISCUSSION
In this convergent, mixed methods, cross-sectional survey of 5,948 adults from a health system in central Pennsylvania, most respondents had high COVID-19–related knowledge, similar to that reported in an earlier smaller-scale survey.22 These scores suggest that COVID-19 messaging has been generally effective within these populations. Knowledge disparity based on education, however, suggests that messaging to groups with lower education levels needs improvement. Differences in knowledge scores across racial categories were observed, but should be interpreted with caution because much like the geographic region from which this survey was obtained, White, non-Hispanic persons were overrepresented in this survey.
The majority of respondents believe in the effectiveness of current CDC recommendations as a means to prevent the spread of COVID-19 and intend to follow these recommendations. Qualitative data supported the notion that participants’ health beliefs about CDC recommendations supported their intention to follow them. This is reassuring because while knowledge plays an important role in behavior, health beliefs and perceptions affect how individuals receive and respond to information they receive.
In bivariate analysis, only a weak relationship between perceived likelihood of diagnosis with COVID-19 and concern for such a diagnosis was noted, suggesting that participants may be more concerned about the potential harm from being diagnosed with COVID-19, and less about their likelihood of infection. Although individuals cannot control the severity of illness should they contract COVID-19, they can reduce infection risk through preventive measures. Thus, messaging that balances illness severity with prevention efficacy may reduce fear and anxiety related to the disease. The integration of qualitative responses with this data did not entirely support this interpretation, as respondents did not commonly describe specifics about medical concerns, but articulated their concerns more broadly as worry about “getting it” or just becoming ill in general. Additional evaluation to better understand public anxieties about the medical aspects of COVID-19 is warranted.
Most participants (72.5%) felt that it was at least “possible” that they would be diagnosed with COVID-19 over the next year and were at least “slightly concerned” about this, while only 47.0% felt this for influenza, an infection that was frequently compared with COVID-19 in Pennsylvania media. A similar discrepancy was observed in a smaller study that found 87.1% of respondents were worried about COVID-19 yet only 64.1% of were worried about getting influenza.22 The lack of a COVID-19 vaccine at the time of the study may have contributed to the higher risk perception.
Government websites (ie, CDC) were the most frequently cited (42.8%) “single most trusted source” for COVID-19–related information at the time of this study. Qualitatively, participants desired basic, readily available information (eg, how to treat common symptoms of COVID-19 using over-the-counter medications). The qualitative data indicated substantial distrust in the executive branch of the government, but not health agencies like the CDC. Detailed analysis of this distrust goes beyond the scope of this article and will be reported separately; however, at minimum it suggests that information dissemination may be best received by health agencies like the CDC vs the executive branch. Leveraging mobile health technologies may help facilitate real-time information disclosures23 noting that rural or underserved areas with less access to information via technology who have an increased risk of poor outcomes24,25 may require different outreach strategies.
We identified racial disparities in COVID-19 knowledge, as have others,22 which, combined with racial health disparities in COVID-19 outcomes,24,25 warrant larger studies with greater demographic diversity to investigate inequities with messaging. Broad generalizability of our study is limited by lack of racial diversity, minimal urban representation, and because the study region area had not yet directly experienced significant impacts of COVID-19 at the time of the survey. Further, the survey has not been validated. Selection bias may contribute to the relatively high knowledge scores, as those who chose to respond to the survey may be more attentive to COVID-19–related issues. Finally, the rapidly changing knowledge stream about COVID-19 may impact the generalizability of findings.
Our study has several strengths. To our knowledge, at the time of this study, this was the largest and first mixed methods assessment of the COVID-19 pandemic. Our convergent mixed methods design offers a deeper understanding and context to the quantitative analysis. A rigorous data integration in accordance with best practices in mixed methods was performed, strengthening the credibility and validity of our findings.13
In conclusion, this study suggests that knowledge about COVID-19 and adherence to behavioral recommendations are generally high. Lower education groups and minorities appear to be at risk from lesser knowledge, although further exploration is needed. We were surprised to find only a weak relationship between the degree of likelihood for and concern about a COVID-19 diagnosis, suggesting that respondents were focused more on the severity of the disease than the perceived likelihood of contracting it. Less than one-half of the respondents viewed government websites (such as the CDC) as the most trusted source of COVID-19–related information. There was substantial distrust of information provided by the executive branch of government and concern about mixed messaging and information overload, highlighting the importance of consistent messaging from trusted sources that reaches all populations.
Acknowledgments
Without the assistance of the following individuals and groups, the scope and scale of this project would not have been possible. We thank Stacy Beers, Amy Peiffer and the Penn State Health and Penn State College of Medicine Marketing teams, Susan Chobanoff, Neal Thomas, Leslie Parent, Sarah Bronson, Heather Stuckey-Peyrot, and the Penn State Qualitative Mixed Methods Core.
Footnotes
Annals “Online First” article.
Conflicts of interest: authors report none.
To read or post commentaries in response to this article, go to https://www.AnnFamMed.org/content/19/4/293/tab-e-letters.
Author contributions: Concept and design: L.J.VS, B.S., E.L.M., R.P.L., C.L.L., M.T.R., A.E.Z.; Acquisition, analysis, or interpretation of data: All authors; Drafting of the manuscript: L.J.VS., M.T.R., V.M.C., R.P.L.; Critical revision of the manuscript for important intellectual content: All authors; Statistical analysis: L.J.VS, M.T.R., V.M.C.; Obtained funding: L.J.VS., R.P.L., E.L.M., M.T.R., A.E.Z.; Administrative, technical or material support: B.S., E.L.M.; Supervision: L.J.VS., B.S., V.M.C, R.P.L.
Funding support: Research reported in this project was funded by the Huck Institutes of Life Sciences and the Social Science Research Institute of Pennsylvania State University, and the Department of Family and Community Medicine, Pennsylvania State College of Medicine. L.J.VS. has received funding from the National Institute of Health, Canadian Institute of Health Research, John and Wauna Harman Foundation, Society for Critical Care Medicine, the Francis Family Foundation, and Association for Clinical Pastoral Education. E.W. and B.S. receive funding from the Penn State Qualitative and Mixed Methods Core. R.P.L., D.R., and A.E.Z., and E.L.M. receive additional funding for other COVID-19 related studies from the Huck Institutes of Life Sciences and the Social Science Research Institute of Pennsylvania State University. Some authors may have additional funding sources unrelated to this study; this information is available upon request. The REDCap database used in this project was funded by The Penn State Clinical & Translational Research Institute, Pennsylvania State University CTSA, NIH/NCATS Grant Number UL1 TR000127 and UL1 TR002014.
Role of the Funder/Sponsor: DFCM faculty were involved in study design and manuscript production. No other funders were involved in data collection, analysis, interpretation or any aspect of the manuscript production.
Supplemental materials: Available at https://www.AnnFamMed.org/lookup/suppl/doi:10.1370/afm.2674/-/DC1.
- Received for publication May 6, 2020.
- Revision received August 27, 2020.
- Accepted for publication October 29, 2020.
- © 2021 Annals of Family Medicine, Inc.