Data Collection Outcomes Comparing Paper Forms With PDA Forms in an Office-Based Patient Survey

James M. Galliher; Thomas V. Stewart; Paramod K. Pathak; James J. Werner; L. Miriam Dickinson; John M. Hickner

doi:10.1370/afm.762

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article

Compose eLetter

Title *

Author Information

Contributors
First Name and Middle Initial * First or given name, e.g. 'Peter'. Last Name * Your last, or family, name, e.g. 'MacMoody'. Email Address * Your email address, e.g. higgs-boson@gmail.com Occupation * Your role and/or occupation, e.g. 'Orthopedic Surgeon'. Affiliation * Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.

Statement of Competing Interests

Competing interests? *

Yes

Please describe the competing interests

Vertical Tabs

Jump to comment:

Re: Response to Comments by Mark Cameron
James M. Galliher

Published on: 03 March 2009
Questioning thoroughness of technical selection
Mark R Cameron

Published on: 30 January 2009
PDAs versus Paper for Survey Data Collection
Fred Tudiver

Published on: 18 March 2008
PDAs and Point of Care Data Collection
Wilson D Pace

Published on: 14 March 2008

Published on: (3 March 2009)
Page navigation anchor for Re: Response to Comments by Mark Cameron
Re: Response to Comments by Mark Cameron
James M. Galliher, Leawood, KS, USA
The study investigators selected the software that we did (Pendragon Forms) at the time (2002) because they had been used by one of the authors previously and had experienced no technical problems in uploading data to the vendor's server and downloading those data to one's PC for analysis. So based on familiarity, experience, and expertise of this co-investigator on the team, we selected this software. We were no doubt pr...
Show More

The study investigators selected the software that we did (Pendragon Forms) at the time (2002) because they had been used by one of the authors previously and had experienced no technical problems in uploading data to the vendor's server and downloading those data to one's PC for analysis. So based on familiarity, experience, and expertise of this co-investigator on the team, we selected this software. We were no doubt premature if not shortsighted to assume that our previous experience using Pendragon Forms for data collection via the handheld computers and subsequent data trasmission and downloading would prove to be without problems.
Again, the major "technical" problem was the firewall issue, experienced by the practices attempting to upload data and by our organization in downloading the study data. As Mr. Cameron suggests, we might have conducted a more thorough due-diligence and had a back-up plan, although there is no guarantee that the use of another vendor or software would have solved this major technical problem at the time of the study.
We did develop a back-up plan as described in the manuscript -- we had the study practices mail the handheld computers to the study team after data collection at each site was completed. We then downloaded data from the handhald computers to our PCs for analysis. While this solution had its own set of issues (e.g., some of the handheld computers were lost or stolen), the handheld computers that did make the journey safely provided study data used to compare to the data collected via paper forms.
We may also have been premature in suggesting that the choice of tablet computers might be a better solution to electronic data collection in practice settings over handheld computers. I say premature because we have not conducted a study directly comparing these two types of hardware for gathering the same study data.
Despite the problems encountered with the handheld computers, they did result in more complete data collection for the surveys that were returned when compared to data collection of the same data using paper forms. However, there were overall fewer electronic survey forms returned. The fewer number returned may have been due at least in part to the selection of either the handheld computers v. some other type of hardware (e.g., tablets) and/or to Pendragon Forms as the data collection forms used in the study.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (30 January 2009)
Page navigation anchor for Questioning thoroughness of technical selection
Questioning thoroughness of technical selection
Mark R Cameron, Vancouver, Canada
I don't generally respond to articles given my commercial perspective on the subject, i.e. I generally respect and value impartial research. However, it strikes me that the technical challenges outlined in this article, and the somewhat hasty claim that tablets running web-based surveys might be better than PDA surveys, appear to be based on a very limited viewpoint of what is actually out there.
The mobile form...
Show More

I don't generally respond to articles given my commercial perspective on the subject, i.e. I generally respect and value impartial research. However, it strikes me that the technical challenges outlined in this article, and the somewhat hasty claim that tablets running web-based surveys might be better than PDA surveys, appear to be based on a very limited viewpoint of what is actually out there.
The mobile forms tool chosen for this study, Pendragon Forms, is not a survey tool. I did not see any reference to why the particular software and hardware was chosen, and it comes across as being a very minor issue - - when in fact it is a crucial factor in determining success of such a project.
I would like to better understand why the tools were chosen and what criteria were used to rule out alternatives. As it currently stands I cannot view this article as a credible comparison of the two modes being compared.
Competing interests: Provider of mobile survey software for PDAs
Show Less

Competing Interests: None declared.
Published on: (18 March 2008)
Page navigation anchor for PDAs versus Paper for Survey Data Collection
PDAs versus Paper for Survey Data Collection
Fred Tudiver, Johnson City, TN, USA
The paper by Galliger et al describes a study that compared a PDA versus paper as data collection methods for survey data. However, this study was unique in that the plan was to remotely upload the PDA-collected data via the internet and mail the surveys. Interestingly, all but one of the 22 practices in the study were forced to mail the PDAs themselves due to institutional firewalls! This is a great example of how securi...
Show More

The paper by Galliger et al describes a study that compared a PDA versus paper as data collection methods for survey data. However, this study was unique in that the plan was to remotely upload the PDA-collected data via the internet and mail the surveys. Interestingly, all but one of the 22 practices in the study were forced to mail the PDAs themselves due to institutional firewalls! This is a great example of how security can get in the way of good research, yet, of course we need the security.
Anyway, the study was nicely done, and it came up with a couple of surprises and a couple of non-surprises. The surprises: The first one is that only 3% of the PDA collected data had errors of omission versus 35% of those gathered on paper. I would have predicted a smaller difference between the two. Perhaps this occurred as a result of the investigators designing a forced choice method for the PDA collected data, where the responder could not continue the survey without entering a response. The second surprise was that the overall return rate for paper was better than for PDAs (94% vs. 82% respectively). I would have guessed the opposite, but as mentioned above, most of the subjects had to mail the full PDAs back to the research center. This is certainly a bit more of a chore than just uploading the data, or tossing a survey into an envelope.
The non-surprises: The first is that half of the 44 recruited physicians did not enroll in the study due to lack of resources for writing an IRB application, even though they all belonged to the AAFP National Research Network! This is a sad commentary on how complex doing research is (especially in non-academic practices), yet we cannot turn the clock back of course. The second non-surprise is that several PDAs were lost or stolen and that “technical difficulties” occurred resulting in lower return rates than expected (the inability to upload from most of the study sites). We have found this in a PDA study at our center where simple things like spent batteries with resulting lost data (and no backup) weakened the power of the study results.
Competing interests: None declared
Show Less

Competing Interests: None declared.
Published on: (14 March 2008)
Page navigation anchor for PDAs and Point of Care Data Collection
PDAs and Point of Care Data Collection
Wilson D Pace, Denver, CO
The article by Galliher et. al. adds to the growing literature on the problems and benefits of electronic point of care data collection. The article quotes from an article I helped write concerning the area in general. The referenced article (Pace,W and Staton, E) was primarily concerned with physician point of care data collection when talking about the use of PDAs versus pen tablets or other computers. The current paper...
Show More

The article by Galliher et. al. adds to the growing literature on the problems and benefits of electronic point of care data collection. The article quotes from an article I helped write concerning the area in general. The referenced article (Pace,W and Staton, E) was primarily concerned with physician point of care data collection when talking about the use of PDAs versus pen tablets or other computers. The current paper was attempting to use PDAs for patient level data collection with an elderly population (people in routine need of a pneumococcal vaccination.) This would not be the ideal group on which to test a PDA data collection method as the screen real estate is very small and familiarity with PDAs low in that population. For this population I believe the above referenced paper would point to a pen-tablet as the preferred electronic data collection method.
That being said, the CHECH Network and Ardith Olsen have described very effective use of PDAs with adolescent data collection for clinical use. They did physically attach the PDAs to a clip board so they would not walk off and had little to no problem with lost PDAs.
The main point of the Pace/Staton Annals article was to carefully match your data collection methods with the intent and population from which data would be collected. Perhaps this was not done as well as it could have been in the current project.
Of note- the data reliabilty for those surveys that were completed was much better using electronic methods- as would be expected.
Competing interests: None declared
Show Less

Competing Interests: None declared.