Research ArticleOriginal Research

Electronic Data Collection Options for Practice-Based Research Networks

Wilson D. Pace and Elizabeth W. Staton
The Annals of Family Medicine May 2005, 3 (suppl 1) S21-S29; DOI: https://doi.org/10.1370/afm.270

Abstract

PURPOSE We wanted to describe the potential benefits and problems associated with selected electronic methods of collecting data within practice-based research networks (PBRNs).

METHODS We considered a literature review, discussions with PBRN researchers, industry information, and personal experience. This article presents examples of selected PBRNs’ use of electronic data collection.

RESULTS Collecting research data in the geographically dispersed PBRN environment requires considerable coordination to ensure completeness, accuracy, and timely transmission of the data, as well as a limited burden on the participants. Electronic data collection, particularly at the point of care, offers some potential solutions. Electronic systems allow use of transparent decision algorithms and improved data entry and data integrity. These systems may improve data transfer to the central office as well as tracking systems for monitoring study progress. PBRNs have available to them a wide variety of electronic data collection options, including notebook computers, tablet PCs, personal digital assistants (PDAs), and browser-based systems that operate independent of or over the Internet. Tablet PCs appear particularly advantageous for direct patient data collection in an office environment. PDAs work well for collecting defined data elements at the point of care. Internet-based systems work well for data collection that can be completed after the patient visit, as most primary care offices do not support Internet connectivity in examination rooms.

CONCLUSIONS When planning to collect data electronically, it is important to match the electronic data collection method to the study design. Focusing an inappropriate electronic data collection method onto users can interfere with accurate data gathering and may also anger PBRN members.

INTRODUCTION

Practice-based research networks (PBRNs) strive to collect high-quality data in clinical environments in geographically dispersed institutions. A number of PBRN researchers have turned to electronic methods of data collection to improve the quality of data and the collection process while decreasing cost and eliminating secondary data entry. Given the required investment in hardware, software, and training, PBRN researchers must carefully consider both the pros and cons of adopting electronic data collection methods. This article explores the potential benefits and limitations of electronic data collection within PBRNs. Table 1⇓ lists the tools we refer to and the terms we use.

View this table:
Table 1.

Electronic Data Collection Tools and Terms

The information presented here is derived mainly from our own experiences and discussions with leaders of PBRNs in the United States. We also examined a convenience sample of literature about PBRN studies using electronic data collection methods, that is, articles we could identify on PBRN studies in which the data were collected by some electronic means.

WHY COLLECT DATA ELECTRONICALLY?

Primary care PBRNs have traditionally asked practicing clinicians and office staff to collect research data while seeing patients.1–4 The Ambulatory Sentinel Practice Network (ASPN) popularized the “card study,” in which clinicians carried a small card with them and completed a short set of questions for selected patients. Variations of this method have been widely used by PBRNs.5–9 The card study moves data collection into the office environment, yet it presents considerable challenges to ensuring data integrity. Missing or hard-to-interpret responses are common and can be labor-intensive to correct. Cross-sectional studies of this type, furthermore, can answer only a limited set of research questions. Improvements in data collection methods that support expanded research designs are crucial for PBRNs to become the laboratory that drives advances in primary care practice.

PBRNs have experimented with electronic data collection for a number of years. The International Primary Care Network (IPCN) collected data on otitis media in 4 countries from 131 family physicians and general practitioners using early personal data assistants (PDAs).10 The technology was new and connectivity proved difficult11; nonetheless, the future of handheld data collection looked promising. The Dartmouth Primary Care Cooperative Research Network (Dartmouth COOP) developed and operates a patient Web site that collects information for community, practice, personal, and research use.12,13 Steve Ornstein of the Practice Partners Network and Henk Lamberts of the University of Amsterdam and the Transhis project established groups of practices that use a single electronic health record (EHR).14 Lamberts not only extracted data from the EHR for research purposes, but added project-specific research questions to the EHR for periods of time. The introduction of enhanced technology during the past decade has heightened researchers’ expectations of electronic data collection.

ELECTRONIC DATA COLLECTION OPTIONS

The directors and staff of PBRNs have available to them an array of options for electronically collecting data. Various methods of electronic data collection have strengths and weaknesses; therefore, it is important to match the method to the study design. PBRN staff should be familiar with the full gamut of options available. It is also important to not force inappropriate technology on a network or study design. If the appropriate electronic method is not available, a PBRN should revert to paper methods instead of pushing an inappropriate technology for the study. To understand what may be an appropriate electronic data collection method, one must understand the benefits and limitations of the available technologies. In this sections, we describe a number of technologies that could be used for data collection across a PBRN.

Working With Existing Data Sources

The search by PBRNs for a so-called painless or free means of collecting additional data to supplement specific point-of-care data has typically centered on mining existing data. Adding demographic, diagnostic, or service data to information collected at the point of care can reduce the burden on clinicians and practices, and enhance available information. Collecting these data typically requires patient consent (a difficult, time-consuming task for which most clinicians do not have the time and are not trained to perform).15 With the advent of the Health Insurance Portability and Accountability Act (HIPAA), the use of existing data to support, enrich, or replace specific research-driven data collection activities has become more difficult.16 Models that develop business partner agreements between the practice and the PBRN offer hope in this regard.

Full EHRs offer promise for obtaining electronic data with less effort than that required by traditional paper methods. Unfortunately, data in EHRs are rarely collected according to a study protocol and are highly variable in scope and meaning, even when templates are used. Most EHRs, furthermore, cannot be easily modified for the collection of additional, research-specific data. With a few notable exceptions, PBRNs are still awaiting the potential of EHRs to revolutionize their data collection capabilities.

Thick Client (Centralized) Systems

Data management can be effectively distributed to multiple users at one time with use of networked computers that are running a central application called a thick client. Applications of this nature can guide research assistants through telephone data collection from network members or patients. Error checking for missing and incorrect entries, pick lists (lists from which the user selects a single item or multiple items), and forced data entry (a feature that prevents the user from skipping questions) can be used to improve the reliability of the data collected and speed the collection process. These systems are not efficient options for simultaneous data collection across multiple practices because of the need to either load a copy of the database locally (such as on a notebook computer that is operated off the network) or have a continuous linkage to a central database via a network. A 2001 survey conducted by the Pediatric Practice Research Group (PPRG)17 found just 40% of responding PBRN members had networked computers in their offices, further narrowing the scope of practices that can incorporate thick client systems.

Notebook Computers

Notebook computers offer portability for data entry or collection, combined with the full capabilities of a desktop computer. Examples of the effective use of notebook data collection methods include chart reviews and extractions, and on-site data collection from patient, clinician, or staff interviews. Notebook computers can run a local database, which is easy to program and provides an effective system for studies in which research assistants collect data on-site.

Internet Browser-Based Systems

Internet browser-based systems have the advantage of being available from any location with access to the Internet. Most Americans and all medical personnel are familiar with browsing the Internet; thus, most users are comfortable with these systems. Browser-based systems on workstations typically do not require installation of software, and program updates need be made at only 1 location to be immediately available to all users.

Error Checking With Internet Browser-Based Systems

Traditional Internet-based systems are excellent for managing complex data sets. Using a desktop computer, users can easily handle either text entries or closed-ended questions. Internet-based systems can offer complex error checking, either centrally or at the browser. The central server can perform error checking, but the data must be submitted first. This process can frustrate users. The longer and more complex a single form becomes, moreover, the more likely it is to contain errors. With central error checking, correcting an error requires reloading the form on the user’s workstation, preferably with the errors highlighted. Anyone who has struggled with a complex Internet form, submitting it over and over, knows how frustrating this can be for the user.

A second method of error checking is immediate checking, a method that usually requires advanced programming, typically Sun Java scripting. Java scripting, which is coding that is transmitted with the form to perform actions locally, can speed error checking. The data are checked as the user moves from field to field (much as it is with a traditional thick client system). Unfortunately, Java scripting can make a program less compatible across browsers (eg, Microsoft Internet Explorer, Netscape Navigator, Mozilla) or with older versions of a single browser. This incompatibility is, however, becoming less of a concern as older computers are phased out. Carefully designed screens can improve the reliability of collected data, obviating the need for Java scripting.

Security for Internet Browser-Based Systems

If sensitive data will be transmitted through Internet browser-based systems, a PBRN must consider how to protect the data from access by unauthorized users. The most effective way to secure data is encryption, which is the translation of data into a secret code. To read an encrypted file, users must have access to a secret key or password that enables them to decrypt it. The easiest way to encrypt sensitive data is to use Secure Sockets Layer (SSL), a protocol for transmitting private documents via the Internet. This protocol uses a private key to encrypt data that are transferred over the SSL connection. By convention, uniform resource locators (URLs) that require an SSL connection start with https instead of with http.18

Additionally, a virtual private network (VPN) can be established that not only encrypts the data, but specifies computer-to-computer access to the data to decrease the chance of interception. Establishing a VPN requires users to load nonapplication-specific software onto their computers and activate it with each use, thus complicating basic Internet use. The increase in security achieved with a VPN comes with a price: a recent study by Ariza et al17 found that insufficient staff training and time (among other factors) were obstacles to expanding computer use within PBRNs, indicating that data collection interfaces must be kept simple unless staff can be adequately trained to use new software.

Access to the Internet in PBRN Practices

The Kentucky Ambulatory Network (KAN) recently reported that all but 1 practice responding to an information technology survey had Internet access; however, 43% had only dial-up service (which provides only low-speed transfer of data).19 In the PPRG survey, 87% of respondents had Internet access (including e-mail–only access), but just 20% of those had it on all computers, and only 65% of those with Internet access had Web-browsing capabilities.17 Limited Internet access, specifically Web access, means that PBRNs must carefully consider which studies should take advantage of Internet browser-based systems. Presumably, these percentages are increasing with time.

Internet-based data collection works best when studies permit data entry after the patient visit as opposed to real-time data collection in the examination room. As an alternative, a paper form can be used for initial collection of data, which are then transferred to an Internet form. This approach shifts the data entry activity to the practice. Although this shift improves data turnaround and may help reduce occurrences of missing data, it adds an additional burden on practices. Internet-based data collection systems seem to be ideal for clinician or staff surveys, particularly if all of a network’s clinicians have access to e-mail. A Wisconsin Research Network (WReN) study found that response rates to Internet-based surveys were higher than those to paper-based ones,20 but the Colorado Research Network (CaReNet) found the opposite, as have others.21 Schleyer and Forrest22 provide a thorough discussion of how to design e-mail surveys to obtain quality results, including a comparison of the costs with those of traditional paper-based surveys.

Personal Digital Assistants

One of the hallmarks of PBRNs is point-of-care data collection. With the heavy penetration of the personal digital assistants (PDA) into clinical care, these devices appear to be the best current option for electronic point-of-care data collection. Ariza and colleagues of PPRG17 found 63% of their survey respondents were willing to consider use of handheld touch screen devices.

As PDA systems increase in speed and storage capacity, their ability to provide extensive “just in time” information is impressive. It is logical that PDAs are being heavily explored as a means of capturing electronic data within PBRNs. Point-of-care studies in PBRNs typically have cross-sectional designs that require collection of limited amounts of data. These studies can be done fairly easily with PDA data collection. The experience of CaReNet and other PBRNs indicates that properly designed PDA data collection instruments can often be completed more rapidly than the equivalent paper form. Forced data entry ensures complete data collection, and range checks ensure that the data are logical (although not necessarily correct). Complex algorithms for data collection can be difficult to follow on paper, but branch points and question skipping are relatively easy to program into PDA systems. Thus, skip patterns that appear overwhelming on paper are virtually transparent to PDA users.

Tools for developing software for PDAs lag considerably behind those for more robust computer systems. Further complicating the issue is that few development tools cross the 2 operating systems, Palm OS (Palm-Source, Sunnyvale, Calif) and Windows Pocket PC (Microsoft, Redmond, Wash), well. If a PBRN relies on members to supply their own PDAs, data collection instruments will likely need to be developed for both systems. Additionally, clinicians who use their PDA extensively to facilitate patient care may have little memory available on the device for additional programs. Newer development software may not operate on older versions of PDA operating systems. Newly introduced tools, such as Microsoft .NET, offer hope for powerful, cross-platform development tools.

The small screen size of PDAs along with tedious text entry also must be taken into consideration when developing data collection systems.23 Text fields are difficult to implement as character recognition or on-screen keyboards are relatively slow and prone to errors. Pick lists and numbers work well. Nonetheless, care should be given to the number of selections available. Most PDA screens support approximately 12 lines of viewable text. Single-item pick lists (often displayed as a drop-down list) are easy to program, but if the list is long, searching for the correct answer is not as easy with a PDA as it is with more robust computers. If the question involves selection of multiple items and more than 8 to 10 items are offered, then some will not be displayed on the screen and will require scrolling to be selected. It is difficult to guarantee that users will scroll down and view items that are off the screen, especially in light of the time constraints of point-of-care data collection. When the list includes easily recognized data elements, such as months, users quickly understand that additional information must be available off the screen. But long lists of items that are project specific, such as a list of services provided during a visit, will not immediately prompt the user to scroll for hidden items. Lastly, the small screen size typically means that questions are displayed one at a time in a sequential fashion. This sequence works well when the data collection can be logically ordered, but can be frustrating when data entry is less predicable, such as when recording selected items from a patient history. Thus, developers need to be cognizant of the strengths and limitations of PDAs, and use them only for studies in which these computers enhance data collection.

Synchronizing PDAs across a large network can be challenging. Fire wall administrators may block ports necessary to synchronize data between the central server and the user’s PDA. Newer development systems that are moving to XML (extensible markup language) for data transfer may help eliminate some of these problems. Before a PBRN invests in PDA development and synchronization software, however, a critical first step is testing synchronization with network administrators across a network.

Tablet PCs

The final option we describe for electronically collecting research data is tablet PCs, tablet-sized computers. Tablet PCs are thin, lightweight computers that allow users to enter data on a touch screen. Several models are available; most weigh less than 2 lb and are about the size of a 500-page spiral notebook (9 in × 11 in × 1 in). Tablet PCs have reasonably sized hard drives (typically 6 to 20 GB) and run full operating systems, typically Microsoft’s Windows XP Tablet PC. Tablet PCs cost between $800 and $3,500, depending on the operating system and features.

Tablet PCs offer portability and the ease of using a touch screen system for navigation. With an extended array of development tools and the capability of storing a large volume of data, tablet PCs overcome many of the limitations of typical PDAs. Because of the higher price and larger size of tablet PCs, physicians are unlikely to carry them for intermittent point-of-care data collection, such as during a typical card study, unless they are in use in the practice. Tablet PCs are well suited for delivering multimedia messages to patients or facilitating data collection directly from patients. Interactive models that allow the patient to use the device and then deliver it to the clinician for further data collection are also possible.

PBRN EXPERIENCE WITH ELECTRONIC DATA COLLECTION: SOME EXAMPLES

As the previous section indicates, a number of technologies have the potential to facilitate electronic data collection in PBRNs. Table 2⇓ highlights issues in matching these technologies to the study design. In this section, we provide examples of various PBRNs’ experiences with electronic data collection.

View this table:
Table 2.

Technologic Strengths and Limitations

Data Mining

Data mining has been a part of PBRN research for decades. ASPN used data mining primarily to create age-sex and morbidity profiles of member practices. Even these data elements, contained within billing data sets, were difficult for some practices to supply electronically. Outside the PBRN arena, efforts of Irish general practitioners to collect morbidity data from computerized systems met with major difficulties due in part to problems with extracting data from practice software systems and the need for a high level of dedicated staff and resources to implement such a project.24 Today, virtually all practices have these data in electronic form, although retrieving them in a uniform manner across a large PBRN can still be a difficult undertaking.

Data mining using clinical data within an EHR is finally becoming a reality. Successful ventures to date have all come from PBRNs that use a single EHR system. The Practice Partner Research Network (PPRNet) has created a longitudinal patient database composed of data from the Practice Partner Patient Record from participating physicians throughout the United States. The network has focused on translational activities through data synthesis and on providing practices and clinicians with feedback to improve care.

Perhaps the most advanced data system within the United States that supports a PBRN is the Regenstrief Medical Record System used by Indiana University Medical Group–Primary Care (IUMG–PC), a single-practice organization that contains IUMG–ResNet, an urban primary care PBRN.25 This system has been in operation since 1972 and contains hundreds of millions of discrete coded observations, although many of these data originate from specialist or hospital care.26 The Regenstrief Medical Record System has been a rich source of clinical data, resulting in hundreds of articles in the past 4 years alone. It can be programmed to provide study reminders and has powerful tools for supporting subject recruitment in PBRN offices. The system can extract existing clinical data and link it to data collected at the point of care, thus lessening the burden on the clinician and the practice. With access to data extending back for many years, powerful epidemiologic studies from primary care practices are emerging.25

Henk Lamberts and Inge Hofmans-Okkes27 created an episode-oriented EHR system, called Transhis, that is used by a network of general practitioners in the Netherlands. In this system, the application of the International Classification for Primary Care (ICPC) is the guiding principle to structure episode-oriented epidemiology.28 Transhis contains more than 300,000 patient-years of data.14 The system has been modified to collect specific research data based on selected diagnoses for time-limited studies. The Transition Project is aimed at the further development of episode-oriented epidemiology in general practice, both in the Nether-lands and elsewhere. The system used in this project highlights a data model that supports primary care research with point-of-care physician coding.29

PBRNs in which all members share the same EHR have successfully used their clinical data systems for research. The challenge facing most PBRNs is to develop approaches that will allow them to aggregate data across disparate EHR systems. Efforts to standardize national data should facilitate data sharing across systems, but a PBRN’s activities entail more than data mining, and mixing clinical and research data collection within an EHR system is still rare.

Web Forms and Databases

For many years, the Dartmouth COOP has incrementally developed and tested a patient-centered, Web-based information system specifically designed to support a more productive interaction between the patient or caregiver and the practice team (see http://www.howsyourhealth.org).13 How’s Your Health (HYH) is a free, Web-based survey providing patients with tested, evidence-based health information and an action form designed to help patients take better care of themselves and work more closely with their physicians.

HYH was developed to link consumers with tools that allow them to become more actively involved in preventing and managing their health care problems, and to provide physician offices with resources to help them operate more efficiently and encourage consumers to participate in their own health care. In this project, patients enter information through a Web interface or through a handheld device in a clinician’s office. Using these data, the system then provides patient-specific education and recommendations for the patient and clinician, as well as aggregated data for research. The system blends various data entry options and user views into a single database. This novel population-based research approach bridges community health and practice-based research.

Several PBRNs operate clinical databases that are also used for research.30–32 These systems use a paper interface with the clinician- and practice-level data entry. We will not discuss them further here.

Web-based collection of information related to medical errors has been successfully implemented in a number of networks, including CaReNet,33 the American Academy of Family Physicians National Research Network,34 and many nonresearch institution-based systems. Clinicians and staff appear able to remember the details of an error sufficiently well to delay their report until they have time and access to the Internet. An early comparison of Web with paper reports indicated that clinicians were comfortable providing error information over the Internet, and the reports appeared more detailed when they were submitted electronically than when they were handwritten.35

Tablet PCs for Data Collection

CaReNet successfully programmed tablet PCs to administer separate sections of a national survey—the Primary Care Network Survey (PRINS)—to the front office staff, the patient, and the clinician. This approach to data collection allows linked data collection from the patient, clinician, or office staff member, while maintaining an anonymous format, if desired. Patients helped steer the data collection process to clinicians and staff after completing their portion of the survey.

An instructional demonstration guided users through the use of the tablet PC. After patients completed the PRINS survey, exit interviews were conducted to evaluate their experience with the tablet PC. Patients had favorable reactions to the technology. Overall, patients were able and willing to use tablet PCs for data collection within busy primary care offices.36 Increasing patient involvement in practice-based research may be possible through the use of this technology, which allows patient-directed data collection at a single point in time or longitudinally. Since the PRINS study concluded, the capabilities of reasonably priced tablet PCs have improved, expanding their ability to support data collection from patients and staff.

Multicomponent Data Collection

The Oklahoma Physicians Resource/Research Network (OKPRN) and a number of collaborators developed the Influenza-Like Illness Surveillance and Messaging System (ILI-SMS), a surveillance system for the reporting of influenza-like illness and other acute syndromes. The ILI-SMS regularly sends public health messages to clinicians, and the clinicians, in return, send daily reports to the Oklahoma State Department of Health on cases of influenza-like illness during influenza season. This system is designed for operation by nurses, although front office staff or clinicians can also use it.

Four interfaces have been or will be developed for the ILI-SMS: (1) a Web interface for data entry and retrieval using an Internet-connected computer, (2) a hard-wired PDA interface that transmits and retrieves data through the PDAs “hot sync” function, (3) a Bluetooth PDA interface that transmits and retrieves data wirelessly through the clinic’s wireless local area network, and (4) a wireless PDA interface for PDA and mobile phone devices that transmits and retrieves information over a cellular network. Prompts, flow sheets, and audits can be printed directly from the computer or from the PDA via an infrared printer port, or through the wireless local area network in the case of Blue-tooth-enabled PDAs.

The ILI-SMS has been tested in 29 OKPRN practices. Of these, 27 reported surveillance data on a daily basis more than 90% of the time during a 2-month period. The 549 reports captured 10,892 patient encounters, including 529 cases of influenza-like illness and 29 hospitalizations of patients with such illness.

Clinicians expressed great satisfaction with the feedback they received from the Oklahoma State Department of Health, as they were able to follow the spread of influenza across the state, anticipating the need for additional appointment slots. They indicated that the burden of reporting influenza-like illness was minimal compared with the benefit of the information derived from it. These results suggest that offering multiple options for data collection, feedback, or both within a single project improves acceptability across users and locations.

SHOULD A PBRN PURSUE ELECTRONIC DATA COLLECTION?

Obviously, not every PBRN is ready to pursue electronic data collection. Young PBRNs that are still recruiting initial members, that are focusing on their first studies, or that have limited personnel may wish to direct their resources and energy toward other infrastructure. When considering electronic data collection, PBRN directors should ask themselves the following questions:

  • Do the studies we wish to pursue lend themselves to electronic data collection?

  • Do we have the technologic expertise to implement electronic data collection?

  • Can we support the infrastructure and personnel costs associated with electronic data collection?

Table 3⇓ lists some of the specific issues that PBRN directors should consider before developing electronic data collection methods.

View this table:
Table 3.

Selected Technical Issues to Consider When Developing an Electronic Data Collection System

CONCLUSION

In this article, we have discussed a number of options for collecting data from PBRN members. PBRNs should not assume that any one electronic data collection method will meet all their needs. Administrative costs, the burden on practices and clinicians, and issues of training and data quality moreover must all be weighed in a decision about whether to collect data electronically. PBRNs are rapidly expanding and experimenting with options for electronically collecting and communicating data. While some networks have successfully developed a primary approach for electronic data collection, all will need to match their data collection options with their project requirements. Some networks have begun sharing resources and knowledge to help advance the use of technology and PBRN methods. As PBRNs expand their efforts at translational research, the line between quality improvement and research will continue to blur, as will the distinction between clinically oriented and research-oriented data systems.

PBRNs are well positioned to serve as the primary laboratories to study and improve the delivery of primary care. A critical ingredient in this effort is the improved capacity to collect high-quality data using electronic methods. These methods help networks conduct research effectively and efficiently, and make it possible to collect data longitudinally as well as to conduct studies that are national in scope. Supporting a wide variety of data collection formats is difficult for any single PBRN, and collaborative efforts hold great promise for this effort. Recent support by the Agency for Healthcare Research and Quality initiatives to promote connectivity of health data across a state should help PBRN research activities through enhanced data connectivity. Two National Institutes of Health pilot projects to support multi-PBRN research projects, including electronic data collection, are likely to further speed innovation. We hope that these efforts will lead to greater collaboration and the development of a national infrastructure to support primary care PBRN research.

Footnotes

  • Conflicts of interest: none reported

  • Received for publication July 12, 2004.
  • Revision received November 1, 2004.
  • Accepted for publication November 29, 2004.

REFERENCES

  1. ↵
    Anonymous. An exploratory report of chest pain in primary care: a report from ASPN. J Am Board Fam Pract. 1990;3:143–150.
  2. Culpepper L. Common methods and issues. In: Methods for Practice-Based Research Networks: Challenges and Opportunities. Proceedings from the Practice-Based Research Networks Methods Conference. Convened by the AAFP Task Force, Plan to Enhance Family Practice Research. November 29 - December 1, 2001. San Antonio, Texas. Leawood, Kan: American Academy of Family Physicians; 2003:19–27.
  3. Vinson DC, Lutz LJ. The effect of parental expectations on treatment of children with a cough: a report from ASPN. J Fam Pract. 1993;37:23–27.
    OpenUrlPubMed
  4. ↵
    Miller RS, Iverson DC, Fried RA, Green LA, Nutting PA. Carpal tunnel syndrome in primary care: a report from ASPN. Ambulatory Sentinel Practice Network. J Fam Pract. 1994;38:337–344.
    OpenUrlPubMed
  5. ↵
    Parnes BL, Main DS, Dickinson LM, et al. Clinical decisions regarding HbA1c results in primary care: a report from CaReNet and HPRN. Diabetes Care. 2004;27:13–16.
    OpenUrlAbstract/FREE Full Text
  6. Sinusas K. Deliveries by family physicians in Connecticut: results of a practice-based research network. Arch Fam Med. 2000;9:434–438.
    OpenUrlPubMed
  7. Madlon-Kay DJ. Evaluation and management of newborn jaundice by midwest family physicians. J Fam Pract. 1998;47:461–464.
    OpenUrlPubMed
  8. Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Arch Pediatr Adolesc Med. 2000;154:499–506.
    OpenUrlCrossRefPubMed
  9. ↵
    Flaherty EG, Sege R, Mattson CL, Binns HJ. Assessment of suspicion of abuse in the primary care setting. Ambul Pediatr. 2002;2:120–126.
    OpenUrlCrossRefPubMed
  10. ↵
    Froom J, Culpepper L, Green LA, et al. A cross-national study of acute otitis media: risk factors, severity, and treatment at initial visit. Report from the International Primary Care Network (IPCN) and the Ambulatory Sentinel Practice Network (ASPN). J Am Board Fam Pract. 2001;14:406–417.
    OpenUrlAbstract
  11. ↵
    Green LA, Fryer GEJ, Froom P, Culpepper L, Froom J. Opportunities, challenges, and lessons of international research in practice-based research networks: the case of an international study of acute otitis media. Ann Fam Med. 2004;2:429–433.
    OpenUrlAbstract/FREE Full Text
  12. ↵
    Wasson JH, West HO, Benjamin R. How’s Your Health, America? Americans’ Concerns About Health and What Can Be Done to Improve It. Lebanon, NH: FNX Corporation; 2002.
  13. ↵
    FNX Corp and Trustees of Dartmouth College. How’s your health? [How’s Your Health Web site] 1997. Available at: http://www.howsy-ourhealth.org. Accessed July 8, 2004.
  14. ↵
    Okkes IM, Groen A, Oskam SK, Lamberts H. Advantages of long observation in episode-oriented electronic patient records in family practice. Methods Inform Med. 2001;40:229–235.
    OpenUrlPubMed
  15. ↵
    Tu JV, Willison DJ, Silver FL, et al. Impracticability of informed consent in the Registry of the Canadian Stroke Network. N Engl J Med. 2004;350:1414–1421.
    OpenUrlCrossRefPubMed
  16. ↵
    Pace WD, Staton EW, Holcomb S. Practice-based research network studies in the age of HIPAA. Ann Fam Med. 2005;3(Suppl 1):S38–S45.
    OpenUrlAbstract/FREE Full Text
  17. ↵
    Ariza AJ, Binns HJ, Christoffel KK, for the Pediatric Practice Research Group. Evaluating computer capabilities in a primary care practice-based research network. Ann Fam Med. 2004;2:418–420.
    OpenUrlAbstract/FREE Full Text
  18. ↵
    Webopedia: online computer dictionary for computer and Internet terms and definitions. Available at: http://www.webopedia.com/. Accessed October 27, 2004.
  19. ↵
    Andrews JE, Pearce KA, Sydney C, Ireson C, Love M. Current state of information technology use in a US primary care practice-based research network. Inform Prim Care. 2004;12:11–18.
    OpenUrlCrossRefPubMed
  20. ↵
    Temte JL, Anderson AL. Rapid assessment of agents of biological terrorism: defining the differential diagnosis of inhalational anthrax using electronic communication in a practice-based research network. Ann Fam Med. 2004;2:434–437.
    OpenUrlAbstract/FREE Full Text
  21. ↵
    Raziano DB, Jayadevappa R, Valenzula D, Weiner M, Lavizzo-Mourey R. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist. 2001;41:799–804.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    Schleyer TK, Forrest JL. Methods for the design and administration of Web-based surveys. J Am Med Inform Assoc. 2000;7:416–425.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    Albers M, Kim L. Information design for the small-screen interface: an overview of Web design issues for personal digital assistants. Tech Commun. 2002;49:45–60.
    OpenUrl
  24. ↵
    O’Mahony B, O’Neill Y, Lynch C, Fennessy M, Lanigan AM, O’Reilly O. Morbidity data collection in general practice: experience in the South Eastern Health Board 1998-1999. Ir Med J. 2001;94:299–300, 302.
    OpenUrlPubMed
  25. ↵
    Tierney WM, Brunt M, Kesterson J, Zhou X-H, L’Italien G, Lapuerta P. Quantifying risk of adverse clinical events with one set of vital signs among primary care patients with hypertension. Ann Fam Med. 2004;2:209–217.
    OpenUrlAbstract/FREE Full Text
  26. ↵
    McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999;54:225–253.
    OpenUrlCrossRefPubMed
  27. ↵
    Lamberts H, Hofmans-Okkes I. The generic patient record: an alliance between patient documentation and medical informatics. Methods Inf Med. 1996;35:5–7.
    OpenUrlPubMed
  28. ↵
    Lamberts H, Hofmans-Okkes I. The core of computer based patient records in family practice: episodes of care classified with ICPC. Int J Biomed Comput. 1996;42:35–41.
    OpenUrlPubMed
  29. ↵
    World Health Organization. International Classification of Primary Care, 2nd ed (ICPC-2). Available at: http://www.who.int/classifications/icd/adaptations/icpc2/en/. Accessed October 27, 2004.
  30. ↵
    Minnesota Academy of Family Physicians. MAFP Research Network. Available at: http://www.mafp.org/mafprn/mafprn.asp. Accessed October 27, 2004.
  31. Michigan Great Lakes Research Into Practice Network (GRIN). PREP: Prompting and reminding at encounters for prevention. Available at: http://sitemaker.umich.edu/prep. Accessed October 27, 2004.
  32. ↵
    Rothemich SF, Woolf SH, Johnson RE, Marsland DW. Evaluating the administration of an office-based health survey in a primary care practice. J Clin Epidemiol. 2000;53:1002–1012.
    OpenUrlPubMed
  33. ↵
    Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall J. Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ann Fam Med. 2004;2:327–332.
    OpenUrlAbstract/FREE Full Text
  34. ↵
    Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Safety Health Care. 2002;11:233–238.
  35. ↵
    Makeham MA, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Aust. 2002;177:68–72.
    OpenUrlPubMed
  36. ↵
    Main DS, Quintela J, Araya-Guerra R, Holcomb S, Pace WD. Exploring patient reactions to pen-tablet computers: a report from CaReNet. Ann Fam Med. 2004;2:421–424.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Subjects