Combining Web-Based and Mail Surveys Improves Response Rates: A PBRN Study From PRIME Net

Philip J. Kroth; Laurie McPherson; Robert Leverence; Wilson Pace; Elvan Daniels; Robert L. Rhyne; Robert L. Williams

doi:10.1370/afm.944

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article

Compose eLetter

Title *

Author Information

Contributors
First Name and Middle Initial * First or given name, e.g. 'Peter'. Last Name * Your last, or family, name, e.g. 'MacMoody'. Email Address * Your email address, e.g. higgs-boson@gmail.com Occupation * Your role and/or occupation, e.g. 'Orthopedic Surgeon'. Affiliation * Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.

Statement of Competing Interests

Competing interests? *

Yes

Please describe the competing interests

CAPTCHA

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

What code is in the image? *

Enter the characters shown in the image.

Vertical Tabs

Jump to comment:

Excellent points
Philip J Kroth

Published on: 15 May 2009
The Importance of Mailed Surveys
Hector P Rodriguez

Published on: 14 May 2009
E-mail Survey Methods Versus Traditional Paper Survey Methods
Joshua Fogel, PhD

Published on: 14 May 2009
Quantity, Quality, and Economy: Personalization and Prioritization of Contact Methods
Zsolt J Nagykaldi, PhD

Published on: 13 May 2009

Published on: (15 May 2009)
Page navigation anchor for Excellent points
Excellent points
Philip J Kroth, Albuquerque, United States
Thanks for the excellent thoughts and comments on our article. Of particular note, Hector makes an excellent point that the delivery of mailed (paper) questionnaires is more robust than email, e.g., if just one character is incorrect in the email address, the email system cannot deliver as opposed to the "snail mail" system that often delivers to the correct addressee even with one or more errors. The postal mail system...
Show More

Thanks for the excellent thoughts and comments on our article. Of particular note, Hector makes an excellent point that the delivery of mailed (paper) questionnaires is more robust than email, e.g., if just one character is incorrect in the email address, the email system cannot deliver as opposed to the "snail mail" system that often delivers to the correct addressee even with one or more errors. The postal mail system also has a fairly efficient automated forwarding system (not to mention the informal forwarding that occurs when folks at the incorrect or old address forward on the mail to the correct or new address.) It would be an interesting study to send paper surveys to those with "bounced" emailed questionnaires to see how many would have gotten through.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (14 May 2009)
Page navigation anchor for The Importance of Mailed Surveys
The Importance of Mailed Surveys
Hector P Rodriguez, Seattle, WA, USA
Kroth et. al.’s response rate analysis of a PBRN clinician survey provides important evidence that quality improvement initiatives should continue to use mailed surveys for some respondents to achieve adequate response rates. Clearly, the cost per usable response is higher for mailed surveys compared to web-based surveys because of printing, postage and processing costs. However, given that certain clinicians may have...
Show More

Kroth et. al.’s response rate analysis of a PBRN clinician survey provides important evidence that quality improvement initiatives should continue to use mailed surveys for some respondents to achieve adequate response rates. Clearly, the cost per usable response is higher for mailed surveys compared to web-based surveys because of printing, postage and processing costs. However, given that certain clinicians may have a higher propensity to respond only to mailed surveys, e.g., older clinicians or clinicians who do not trust the anonymity of web-based surveys, mailed surveys will likely improve the representativeness of survey respondents. It would be very interesting to assess differences in clinician demographics or practice characteristics of electronic responders vs. mail responders to provide additional evidence of the benefits of multimodal follow-up. This might inform the most cost- effective initial mode and follow0up approach for clinician surveys.
The ordering of the survey modes is an important point worth additional discussion. Sending mailed surveys would increase total costs, but may alter the overall eligible sample. For example, over 18% of initial emails were either invalid because they were sent to non-PCPs or the email “bounced back” because of an incorrect address. Clinicians who were removed from the sample because of incorrect email address information may have been eligible for the survey, i.e., they are still practicing clinicians within the network, and may have completed the survey if the instrument was mailed. It is unclear the extent to which this happened in the study. This phenomenon likely can only be assessed in a randomized trial of different survey mode and follow-up approaches. A larger sample that is more representative of the underlying population may be worth the additional cost of mailing surveys first. Future work should clarify the benefits of initial mailed surveys vs. initial email surveys with different follow-up modes.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (14 May 2009)
Page navigation anchor for E-mail Survey Methods Versus Traditional Paper Survey Methods
E-mail Survey Methods Versus Traditional Paper Survey Methods
Joshua Fogel, PhD, Brooklyn, NY, USA
Technology is often exciting to incorporate in both clinical practice and research. The enthusiastic embrace of technology sometimes results in abandonment of the more traditional approaches. Kroth et al. report in this study the experience of surveys of physicians as part of a practice-based research network (1). They found that although many physicians did participate in the initial waves of e-mail survey requests...
Show More

Technology is often exciting to incorporate in both clinical practice and research. The enthusiastic embrace of technology sometimes results in abandonment of the more traditional approaches. Kroth et al. report in this study the experience of surveys of physicians as part of a practice-based research network (1). They found that although many physicians did participate in the initial waves of e-mail survey requests, there was still benefit for a traditional paper survey approach. There were 4 e-mail survey requests sent at one to two week intervals before a paper survey was mailed by postal mail. E-mail survey requests were highly endorsed and completed by three- quarters of participants. However, almost one-quarter of surveys were completed by the paper approach.
I have conducted numerous research surveys using face-to- face, traditional postal mail, and e-mail approaches. My experience has been that approaching individuals face-to- face (e.g., clinic waiting room) is the most effective of the three above approaches for obtaining participation and completion of the survey. However, not all times can this be done, especially when one wants to survey physicians instead of patients or consumers.
One particular study from my personal research is worth sharing. In this study, young adult participants were initially approached face-to-face to complete a paper survey taking approximately 10 minutes. Response rate was very high with 87% of those approached completing the survey. As part of this survey, there was a 3-month follow-up where participants were asked to provide their e-mail address and would be contacted by e-mail. This follow-up survey was expected to take much less time than the initial survey. It inquired about behaviors performed over the previous 3 months. The time to complete the follow-up survey was expected to be less than 1 minute. In the e-mail sent to the participants, this brief follow-up time was mentioned. Less than half of the participants complete the e-mail follow-up survey even after repeated e-mail reminders.
There is another published study about physicians responding to e-mail and paper surveys (2). In that study, the authors concluded that an initial paper survey mailed by interoffice mail followed by an e-mail survey follow-up resulted in greater participation rates than an initial e-mail survey followed by a follow-up paper survey mailed by interoffice mail.
E-mail survey techniques are worth considering as part of any survey approach. However, as Kroth et al. (1) suggest, it is not yet time to abandon paper survey methods.
References
1.Kroth PJ, McPherson L, Leverence R, Pace W, Daniels E, Rhyne RL, Williams RL; For The Prime Net Consortium. Combining web-based and mail surveys improves response rates: a PBRN study from PRIME Net. Ann Fam Med. 2009;7(3):245-248.
2.Beebe TJ, Locke GR 3rd, Barnes SA, Davern ME, Anderson KJ. Mixing web and mail methods in a survey of physicians. Health Serv Res. 2007;42(3 Pt 1):1219-1234.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (13 May 2009)
Page navigation anchor for Quantity, Quality, and Economy: Personalization and Prioritization of Contact Methods
Quantity, Quality, and Economy: Personalization and Prioritization of Contact Methods
Zsolt J Nagykaldi, PhD, Oklahoma City, OK
I was delighted to see this important study on survey research methodology in this issue of the Annals. Since primary care practices are unique, it has become increasingly important to contextualize not just interventions, but also the research methods we use to measure the impact or effectiveness of interventions. Relative to survey research in primary care settings and beyond quantitative concerns (e.g. response rate),...
Show More

I was delighted to see this important study on survey research methodology in this issue of the Annals. Since primary care practices are unique, it has become increasingly important to contextualize not just interventions, but also the research methods we use to measure the impact or effectiveness of interventions. Relative to survey research in primary care settings and beyond quantitative concerns (e.g. response rate), there is also a concern for the quality and depth of feedback from the inquiry. Rich and meaningful information can be obtained only with instruments and approaches that fit the particular context.
When we add economy into this mix, it is not surprising that surveying PBRN practices requires a substantial effort, multiple communication options, and proper timing / sequencing. In this context, existing relationships with practices and information about these relationships (e.g. a rich network database on practices) becomes important.
Building on the work by Kroth et al, one could propose an advanced system that populates practice contact methods and sequences based on the communication priorities of practices or clinicians (e.g. web survey, e- mail, fax, letter), monitors responses, and adjusts strategies according to incoming feedback and the characteristics of those who have not responded yet. The sequence may be initiated with the most efficacious method (e.g. a series of web-based solicitations), followed by the next in line (e.g. fax), targeting only those who have not yet responded. Once successful patterns are discovered in a particular PBRN, future survey delivery could be initiated based on this pattern and adjusted on the fly.
In the case of typical quantitative survey research, as preferences change over time and more practices become comfortable with electronic communication, paper-based surveys may be phased out in favor of electronic data collection.
Competing interests: None declared
Show Less

Competing Interests: None declared.