Abstract
PURPOSE The aim of this study was to evaluate a theory and evidence-based, parent-targeted online intervention, combining microbiological local syndromic surveillance data, symptom information, and home-care advice, to reduce primary care attendance for self-limiting, low-risk pediatric respiratory tract infections (RTIs).
METHODS The effect of this novel intervention on primary care attendance intentions was evaluated in an online experimental study. A representative sample of mothers (n = 806) was randomly assigned to receive the intervention material before (intervention) or after (control) answering questions concerning attendance intentions for an RTI illness scenario and mediating factors. Both groups provided feedback on the material. Group comparisons, linear regression, and path analyses were conducted.
RESULTS Intervention participants reported lower attendance intentions compared with control participants (d = 0.69, 95% CI, 0.55-0.83), an effect that remained when controlling for demographic and clinical characteristics (B = −1.62, 95% CI, −1.97 to −1.30). The path model highlighted that the intervention effect (B = −0.33, 95% CI, −0.40 to −0.26) was mostly indirect and mediated by infection and antibiotic knowledge, symptom severity concerns, and social norm perceptions concerning attendance. Information on when to attend was rated as the most important intervention component 227 times, followed by symptoms rated 186 times. Information on circulating viruses was rated as least important 274 times.
CONCLUSIONS The intervention was effective in reducing primary care attendance intentions by increasing knowledge, lowering attendance motivation, and reducing the need for additional resources. The contribution of individual intervention components and effects on behavioral outcomes requires further testing.
INTRODUCTION
Respiratory tract infections (RTIs) are one of the main reasons for primary care consultation for children.1 These consultations contribute to primary care clinicians’ increasing workload and cost the National Health Service £31.5M per year.2,3 Primary care consultations for RTIs often lead to unnecessary antibiotic prescribing,4,5 promoting antimicrobial resistance.6–9 Providing parents with actionable information can reduce the number of unnecessary consultations by up to 40%,10 can have a significant positive impact on resources, and can promote antimicrobial stewardship.11,12
When children have an RTI, parents want to know how to manage symptoms at home and which symptoms require medical attention.13 Consulting for a child with an RTI is perceived to be the safest course of action, and to be the socially acceptable norm if parents have any doubt about a medical threat to their child.14 Qualitative research investigating the value of an intervention combining real-time community-level information on locally prevalent RTIs with information about typical symptoms and duration found, that while most parents thought such an intervention was useful, and they anticipated using the information to inform lay diagnoses, they had mixed views about whether it would influence decisions to consult primary care physicians.15
The current research is an experimental evaluation of a novel online intervention informed by prior research, following recommendations for evaluating complex digital behavior change interventions.16,17 This study aimed to investigate the effect of the intervention on primary care attendance intentions and mediating factors, and parents’ perception of the intervention material.
METHODS
Intervention Description
The intervention was developed based on a behavioral analysis of the target behavior: consulting with a primary care health professional. Using the capability, opportunity, motivation-behavior model,18 potential pathways of influence were identified as the basis to develop the intervention and to predict mechanisms of action. The intervention (Figure 1 and Supplemental Appendix 1, available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/) was designed to change parents’ capability, opportunity, and motivation to consult primary care for low-risk, self-limiting RTIs in children, by providing microbiological locally enhanced syndromic surveillance data, symptom duration information, and home-care advice. The following behavior change techniques19 were used in the intervention: providing information, instructions on how to perform home-care behaviors, encouraging social comparison, and referring to credible information sources.
Information included in the intervention was drawn from official influenza statistics,20 the Caring for Children With Cough website,21 and national guidance.22,23 To increase relevance, comprehension, and recall, the intervention contained information tailored to parents as a target group, repeated key points, categorized information in a structured way, kept instructions brief, simple, and specific, and used supporting visualizations.24–27 To receive feedback from the target group,28 a parent advisory group was involved in development of the intervention (Supplemental Appendix 2, available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/).
Study Design and Procedure
The study used an experimental design that randomly allocated participants, in equal numbers, to the intervention or control group. Research participants viewed the intervention and completed the survey online. They accessed the survey remotely via a hyperlink and they were automatically randomized based on computer-generated sequences (Supplemental Appendix 4, available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/). Following a scenario approach adopted by Godinho et al29 and McGlone et al,30 participants were asked to imagine that their youngest child was experiencing symptoms consistent with a rhinovirus infection (Supplemental Appendix 3 available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/). The intervention group was presented with the intervention material in a series of static images (Supplemental Appendix 1) before answering questions concerning primary care attendance intentions and factors influencing their decision (Table 1).29,31–44 Control participants completed the same questions before being shown the intervention material. All participants provided feedback on the material, demographic information (child age and sex; parent age, employment, education, and ethnicity; and number of children), and clinical characteristics (pediatric primary care visits over the past 12 months, chronic health issues of the child) potentially associated with primary care attendance.21 Participants were asked to view the intervention material as they would if their child was suffering the described illness to simulate real-life usage instead of examining all information in-depth. Scales, used to assess psychological processes involved in behavior change according to the capability, opportunity, motivation-behavior model,18 and feedback on the intervention material were based on previous research and validated scales (Table 1, and Supplemental Appendix 3). Details of how the survey was amended to increase validity following testing with a parent advisory group are presented elsewhere (Supplemental Appendix 2). The survey required an average reading level (Flesch Reading Ease = 60, Flesch-Kincaid Formula = 7.7, Fog Index = 9.7, SMOG Grading = 7.6).45 Ethical approval was obtained from University College London’s research ethics committee (Project ID Number: CEHP/2013/508). The study protocol is available on the Open Science framework.46
Participant Recruitment
A representative sample, by age, socioeconomic status, and geographic location, of mothers living in England was recruited from January 24 through February 9, 2017 by a market research company online panel using a quota sample. The quota targets were relevant for mothers aged 18 to 65 years, and based on source data used for UK population studies, including participants until all quotas were filled (Supplemental Appendix 4). Participants did not receive financial compensation but they were offered points by the market research company, which can be accrued over time and redeemed for vouchers or for entering prize drawings. To maximize statistical power while minimizing recruitment costs, participation was limited to mothers, as they are more likely (than fathers) to take the child to see a doctor.21 To be eligible, mothers had to have at least 1 child aged 3 months to 12 years; children aged 3 months or younger with fever should be assessed in hospital and home care is not appropriate.22,23 A sample size of 800 participants was estimated to ensure 80% power to detect a statistically significant difference at α = 0.05, assuming a small effect size of d = 0.2 for the main outcome, intention to attend primary care. Previous research by Godinho et al28 identified this size of effect for an intervention designed to increase vaccination intentions.
Analysis
The data were analyzed with the statistics program IBM SPSS Statistics 25 (International Business Machines Corp) and its extension module, Amos, using an alpha level of P <.05 for significance tests. Descriptive statistics were calculated for demographic and clinical sample characteristics. Following assignment of numerical scores in the opposite direction for reverse coded survey items, scales were computed by summing items as outlined in Table 1. Their reliability in the current sample was assessed with Cronbach’s α and distributions examined with histograms and Shapiro-Wilk tests. Scales had high internal consistencies (Table 1) and the main outcome variable, attendance intentions, was left-skewed (S-W = 0.87, P <.001; skewness = −0.62, SE = 0.09). Bootstrapping was used for following statistical procedures, drawing multiple samples (n = 800+) randomly from the original sample, to allow for robust inferences despite the nonnormality distribution.47,48 Compared with normalizing transformations, bootstrapping has the advantage that measurement units remain unchanged, simplifying output interpretations.49,50 Content analysis was used to develop coding frameworks for the open-ended questions, summarizing responses in quantifiable categories.51 Two coders (A.S., A.U.), with a background in health psychology and experience in infection research, independently coded responses without knowing participants’ experimental allocation, discussing discrepancies to reach agreement. Clinical colleagues were available for further discussion, but as responses were typically short comments instead of detailed elaborations, this was not required. Correspondingly, inter-rater reliabilities were high.
The effect size and CI was calculated comparing the intervention group with the control group. Following assumption tests (DW = 2.104) and dummy-coding categorical variables, linear regression analysis was used to test the intervention effect while controlling for demographic and clinical characteristics. The relationship between variables, hypothesized based on prior research and theoretical assumptions concerning the intervention behavior change pathway, was examined specifying a path model. The a priori hypothesis was that the intervention would have an indirect effect (mediated through capability, opportunity, and motivational factors) on intentions to visit primary care for the specified illness scenario.18 The model specification ensured the pathway analysis preconditions of model linearity, causal closure, unitary of variables, and a maximum of 1 curved arrow per path.52 There were no missing data for model parameters and the sample size of 806 sufficed to calculate valid estimates. The model was assessed by the χ2 test statistic, the Comparative Fit Index (CFI), and the Root mean square error of approximation (RMSEA). Considering that a nonsignificant χ2 value, CFI >0.90, and RMSEA <0.06 indicate a good fit,53 the model, as specified according to assumed intervention behavior change pathway, was not an acceptable fit for the data (χ2 = 320.396, df = 12, P <.001; CFI = 0.785; RMSEA = 0.179). To identify a model that better represented the data, post hoc model fitting was carried out by examining the modification indexes that pointed to possible model misspecifications.54 The model adjustment significantly improved its fit on both indices (CFI = 0.99, RMSEA = 0.02) and resulted in a significant χ2 test result (χ2 = 6.445, df = 5, P = .265). To assess participants’ perception of the material, feedback scales and open-ended questions were examined, and drag and drop responses to the intervention component ratings sorted according to importance, generating a diverging stacked bar chart.
RESULTS
Of 2,451 individuals responding to the survey invitation, 806 were included in the final sample (32.9%). Excluded participants either belonged to a sampling quota that was already full (n = 911), dropped out before randomization (n = 341), or partway through the survey (n = 363), or did not fulfill data quality requirements (n = 30) according to algorithms set to identify straight-liners (choosing only 1 answer option for each question and completing the survey in less than half the median completion time).55 Exclusions were carried out by the market research company commissioned to recruit the representative sample (without knowlegde of the research aim and with no interest in the outcome) before the authors conducted the analysis according to the protocol.46 A detailed description of participant enrollment and inclusion is given in Supplemental Appendix 4, available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/. Most participants (70%) were primary caregivers and had attended primary care with their youngest child at least once in the past year (76%). A minority of participants (10%) reported that their child had chronic health issues, most frequently asthma (n = 13), allergies (n = 7), or eczema (n = 3). Table 2 provides an overview of sample demographics and clinical characteristics.
Intervention Effects on Primary Care Attendance Intentions
Scores on the 2-item intention to attend primary care scale ranged from 2 to 10 in the sample (possible range 2 to 10). The mean intention to attend score was 7.29 (SD = 2.56), with the intervention group reporting lower attendance intentions (mean = 6.45, SD = 2.57) than the control group (mean = 8.12, SD = 2.28). The average difference of 1.67 between the intervention and control groups represents a medium-sized effect (d = 0.69, 95% CI, 0.55–0.83). When divided by 2 to convert back to the original 5-point scale (strongly disagree to strongly agree) used to rate both intentions to attend items, the intervention was associated with an almost 1-point decrease (0.84) in attendance intentions.
Consistent with this analysis, participants in the intervention group had lower attendance intentions than those in the control group when adjusting for demographic and clinical characteristics associated with attendance intentions (R2 = 0.17, F20,785 = 8.175, P <.001; Table 3): Asian participants had higher intentions and black participants lower intentions than white participants; participants from the greater London area had higher intentions compared with Northern England; and participants with older children had lower intentions than those with younger children. The intervention effect was still significant (f2 = 0.08) although small when adjusting for those characteristics, with a difference of 0.04 between the adjusted (B = −1.66, 95% CI, −1.99 to −1.32) and the unadjusted intervention regression coefficient (B = −1.62, 95% CI, −1.97 to −1.30).
Mediating Effects of Capability, Motivation, and Opportunity
The a priori hypothesis was that the intervention effect on primary care attendance would be indirect and completely mediated through assessed capability, opportunity, and motivational factors. This model was not an acceptable fit for the data (χ2 = 320.396, df = 12, P <.001, CFI = 0.79, RMSEA = 0.18). Post hoc model fitting resulted in the adjusted model (χ2 = 6.445, df = 5, P = 0. 265, CFI = 0.99, RMSEA = 0.02) outlined in Figure 1, which accounted for 56.9% of the variance in primary care attendance intentions. According to the refit model, most of the intervention’s effect on attendance intentions (B = −0.33, SE = 0.03; 95% CI, −0.40 to −0.26) was still mediated by participants’ capabilities, motivation, and opportunities (B = −0.231, SE = 0.025; 95% CI, −0.279 to −0.180). The intervention specifically increased infection and antibiotic knowledge, reduced concerns about symptom severity, and counteracted the perception of attendance as an accepted and expected social norm (Table 4). Those factors were directly associated with lower attendance intentions. Further direct effects included (1) a small but significant negative effect of the intervention on attendance intentions and number of resources consulted,(2) a negative effect of information sufficiency on attendance seen as a social norm, (3) a negative effect of infection and antibiotic knowledge on attendance seen as a social norm but a positive effect on number of mentioned resources, and (4) a negative effect of confidence in home-care capabilities on worries. There was no direct intervention effect on perceived sufficiency of available information or on number or confidence in self-care capabilities.
Feedback on the Intervention Material
Overall, participants regarded the intervention positively in their open-ended comments, typically highlighting its information as valuable, useful, and useable, eg: “Excellent resource for parents. Really puts people’s minds at rest of the signs and symptoms to look out for, and in an easy format to find the exact information you are looking for.” Ratings of intervention components (Figure 2) highlighted that information about primary care attendance and symptoms were perceived to be most important, and information on locally circulating viruses was the least important.
Participants positively assessed the intervention’s content and presentation, rating the intervention materials to be generally clear (median = 30, range 5 to 35, interquartile range (IQR) is 9 = 34–29), credible (median = 17, range 3 to 21, IQR is 7 = 20–13), cognitively challenging and stimulating (median = 25, range 5 to 35, IQR is 8 = 29–21), novel (median = 12, range 3 to 21, IQR is 3 = 14–11), and emotionally arousing (median = 33, range 7 to 49, IQR is 10 = 38–28). Two participants voiced concerns for the child’s safety if not seen by a doctor and thought the intervention was “irresponsible” for deterring attendance.
DISCUSSION
The parent-targeted online intervention, combining real-time information on locally circulating RTIs with symptom information and advice, was associated with lower intentions to attend primary care for a child with a hypothetical RTI. The intervention effect on intentions was mostly indirect and mediated by increased infection and antibiotic knowledge, reduced symptom severity concerns, and less perceived social pressure to attend. The intervention material was generally well received, particularly the information about when to attend primary care, and symptoms.
Strengths and Limitations
The current research was a theory-informed experimental evaluation of a novel, evidence-based intervention. Such initial intervention tests and agile development approaches are important when developing electronic health interventions particularly for complex interventions that promise considerable improvements of current practice, but require substantial financial investments.28,56 Considering behavioral science theory for the intervention development and the research design, advances the growing field of Internet-based behavior change interventions, which often lack sound theoretical grounding.57
Using intentions as a proxy-measure of behavior and a hypothetical illness scenario instead of examining responses to real illness episodes is a potential limitation of this study and should be considered when interpreting the findings. Evidence from a meta-analysis suggests that intentions are likely to be good predictors for single action, specific, nonhabitual behaviors, such as primary care attendance.58 Outcomes in response to real-life illnesses instead of hypothetical scenarios may, however, show different results.
The survey itself was designed to assess all constructs hypothesized to affect attendance intentions. Ease of readability was average, and the survey was tested before data collection by a parent advisory group for understandability and burdensomeness. That some participants would find the survey too lengthy or struggle to understand the intervention material or survey questions cannot be ruled out, especially for those with different sociodemographic profiles than the mostly white, educated parent advisory group members. Furthermore, by limiting the current research to English mothers, we are unable to say if the intervention would be as effective for reducing intentions to attend primary care among fathers or in other countries. The quota sampling approach ensured a representative but nonrandom sample of English mothers. The re-specification of the original path model was exploratory in nature.52
Relation to Prior Research
The intervention had an effect on factors shown to be important influences on parents’ decisions to consult primary care for pediatric RTIs,13,14,59,60 and was associated with reduced primary care attendance intentions. The intervention increased, for example, knowledge of infections and antibiotic usage, which is important as widespread misperceptions about antibiotic use still exist.21,61 The observed attendance intention patterns correspond with research on parents’ RTI consulation behavior that found non-white ethnicities will consult more often.62 The evidence is not conclusive, but could be due to a combination of different cultural practices and higher anxiety levels, for example, from operating in a second language and in unfamiliar contexts in the case of migrant communities. Parents who are part of minority or discriminated-against groups may feel more pressure to consult, to avoid accusations of failing to attend when necessary and of neglect.14,63
As a whole, the intervention was well received and the information provided was rated as credible, cognitively stimulating, and emotionally engaging. This is an important finding, given that electronic health research has found that ease of use, understanding, credibility, and visual appeal are important factors for encouraging intervention engagement.64–66 The positive assessments of the intervention’s usefulness is supported by prior qualitative research suggesting that parents were interested in accessing information online about locally prevalent RTIs, symptoms, and home-care strategies, before consulting primary care to inform lay diagnoses and make consulation decisions.15 The information would be particularly helpful for first-time parents and parents of young children. Some parents were interested in regularly checking the website to monitor circulating viruses. The intervention could also be used in an effort to prevent infections by heightening awareness of circulating viruses, and increasing the promotion of hand hygiene and other infection-prevention techniques. Accessing the information with primary care clinicians during consultations was also seen as acceptable and could help publicize the intervention.
A key finding was that the novel intervention component, providing information on locally circulating viruses, was seen as less important by participants than symptom and home-care information. It might be useful to add further details on how to interpret and use the dynamic surveillance information, especially for consultation decision making. The static information components, preferred by participants, more directly addressed factors previously identified as influencing consultation behavior.14 This study provides evidence that interventions including such information can change primary care attendance intentions. Ultimately, what users like and engage with is not necessarily linked to the effectiveness of the intervention components in changing behaviors.67 The effects of individual intervention components and their interactions remain to be determined.
Impact and Future Research
Increasing smartphone use means that web-based resources are accessible to people from all socioeconomic backgrounds, and they are particularly suited for interventions that have dynamic elements requiring frequent updates. Paper-based versions of such interventions would be less practical. Even a small size effect of the intervention on parents’ attendance behavior would be impactful if the intervention was rolled out widely, as pediatric RTIs are one of the main reasons for primary care consultations with children.1 Future research should evaluate intervention effects on observed behavioral outcomes in real-world settings, and also examine long-term effects and cost effectiveness. As the current online study was limited to participants with Internet access, the impact of Internet use proficiency should be examined.28 Factorial experimental design studies could help to untangle the complex network of relationships observed between intervention components, mediating factors, and primary care attendance to further refine the intervention.68,69 Since the intervention was effective in increasing knowledge and reducing concerns in mothers, similar interventions aimed at other audiences such as school personnel and clinicians could also be developed to improve the care for pediatric RTIs. Expanding the application of the intervention to other contexts, target groups, and countries requires a careful analysis of the specific setting, appropriate adaptions, and further testing, as effective behavior change is highly context and target-group specific.18
CONCLUSION
The theory-and evidence-based online intervention was effective at reducing primary care attendance intentions for self-limiting, pediatric RTIs. Information on locally circulating viruses, a novel intervention component, was rated as being less important compared with information on when to attend primary care and on symptoms and their management. The effectiveness of individual intervention components in changing parents’ behavior requires further research.
Acknowledgments
The authors wish to thank Arpita Upadhyaya for her help double-coding of the open-ended survey answers.
Footnotes
Conflicts of interest: authors report none.
To read or post commentaries in response to this article, see it online at http://www.AnnFamMed.org/content/17/3/239.
Funding support: This research is supported by the National Institute for Health Research, Health Protection Research Unit in Evaluation of Interventions at the University of Bristol, in partnership with Public Health England. The views expressed are those of the authors and not necessarily those of the National Health Service, the NIHR, the Department of Health and Social Care or Public Health England. Joanna Kesten is partly funded by NIHR HPRU in Evaluation of Interventions at the University of Bristol in partnership with Public Health England and NIHR Collaboration for Leadership in Applied Health Research and Care West at University Hospitals Bristol NHS Foundation Trust.
Prior presentations: Portions of this study have been presented as a poster at the South West Regional Conference of the Society for Academic Primary Care held March 23-24, 2017; Oxford, UK, and as a paper at the PHE Research and Applied Epidemiology Scientific Conference held March 21-22, 2017; Coventry, UK, and at the 3rd Digital Behavior Change Conference held February 22-23, 2017; London, UK.
Supplementary materials: Available at http://www.AnnFamMed.org/content/17/3/239/suppl/DC1/.
- Received for publication September 1, 2018.
- Revision received January 21, 2019.
- Accepted for publication February 12, 2019.
- © 2019 Annals of Family Medicine, Inc.