Card Studies for Observational Research in Practice

INTRODUCTION

Observational and survey research continues to be an important source of information about primary care. Among the numerous methods available for gathering data about clinicians, patients, and their care, one method has persisted for more than 30 years and is considered by some to be the hallmark of practice-based research: the card study. A card study is a field-tested method for gathering data in the location where patients receive care by those who provide the care.

The card study method is designed to gather data at the point of care about such observable phenomena as disease incidence/prevalence, practice patterns, or clinical behaviors. Pioneered in the United States by the Ambulatory Sentinel Practice Network (ASPN) 30 years ago, card studies use short questionnaires that participating clinicians complete for patients with a specific condition.¹

The name card study derives from a weekly return card introduced by the Sentinel Stations in the Netherlands and later modified by ASPN. The pocket-sized card, which was designed to take fewer than 60 seconds to complete, allowed clinicians to carry it from room to room as they saw eligible patients. Although a card study is essentially a survey, it is designed to be completed as patients are seen by those who provide the care. Card studies have undergone significant development by numerous researchers during the past 30 years and represent a wide array of data collection tools and methods.^2–6

Despite their simplicity, card studies have contributed substantially to new standards of care for such diverse conditions as headache, miscarriage, and otitis media.^7–12 For example, the publication of ASPN’s card study on spontaneous abortion contributed to the evidence that uncomplicated spontaneous abortion could be managed safely without surgery. Using data from this study, Green et al found that card studies are as accurate as medical record review for identifying patients with specific conditions.¹³ In fact, the card study reporting method identified more patients with spontaneous abortion than were found on medical record review, probably because the reporting physician had knowledge of spontaneous abortions treated in other medical settings not recorded in the ambulatory chart.

The SNOCAP practice-based research networks (PBRNs) at the University of Colorado School of Medicine have considerable experience designing and conducting card studies. SNOCAP includes PBRNs focused on rural health care, urban and underserved primary care practices, and small, independent practices. Members range from single-physician offices to community health centers. SNOCAP card studies have examined a range of questions (Table 1⇓).

We describe research questions that can be addressed with a card study, the 2 main types of card studies, the resources required, human subjects protection, implementation matter, and future considerations for card study research. Throughout this article we draw on SNOCAP’s experiences to illustrate important concepts.

CONDITION-SPECIFIC AND PREVALENCE CARD STUDIES

Two basic types of card study methodologies—condition-specific cards and prevalence cards—are derived from different research questions. In condition-specific card studies, the respondent completes a card for each patient with a specified condition. This method is ideal for studying specific diseases or conditions, particularly when describing conditions that are less common or when highly detailed data collection on every patient would place undue burden on clinicians. Typically, data collection occurs during a 2- to 4-week period with each clinician completing approximately 1 to 10 cards. A variation is to conduct the card study in the practice until a predetermined number of cards are completed. This approach will assure adequate responses from each practice, but may add extra study time to smaller practices.

For example, SNOCAP conducted a card study about the competing demands when caring for patients with diabetes, in which the clinician completed a card for each patient with diabetes seen in the office during a 2- to 4-week period.¹⁶ In another study we attached a card to every hemoglobin A_1c (glycated hemoglobin) result sent to the practice from the laboratory during a single week.^14,15 Clinicians completed the card when they reviewed their laboratory reports. Parchman and Abel had clinicians complete a card for each patient seeking care for with a skin infection over a 10-month period.²⁶

The condition-specific method is limited by a number of characteristics. First, it requires the respondent to remember to complete the card when he/she sees a patient with the condition. Second, the method does not necessarily obtain data on the denominator for the study, which means the condition prevalence within the practice or network cannot be determined. Third, although these cards are often designed to be completed during or immediately after the patient-clinician encounter, respondents sometimes wait until the end of the day to fill out their cards, which can introduce recall bias.

Frequently there is no published literature on the prevalence of health conditions in primary care. Most literature on prevalence relies on public health surveys or data from tertiary care centers. The second main type of card study, the prevalence card study, is usually used for conditions or phenomena that are more common or to determine the condition’s prevalence in primary care. This method involves completing a card for each patient seen during a specified period (eg, a half-day or full-day clinic session). By collecting data on all patients seen, it is possible to determine the denominator and a rate of occurrence for the condition or phenomenon.

Prevalence surveys typically begin with a short section on patient demographics and one or more qualifying questions. If the patient does not have the specific condition or phenomenon, data collection is complete at that point. If the patient does have the condition, the clinician completes the remainder of the card. Typically the clinician must complete only a few questions for each patient.

For example, in our study about missing information, the clinician completed a card for every patient seen during 1 clinic session.²² The clinician answered 1 question about whether there was any missing information about the patient (laboratory studies, radiographs, consultations, etc). If the clinician answered no, the card was complete. If the answer was yes (in this case about 12%), the clinician answered several more questions. Another SNOCAP study examined direct-to-consumer advertising.²¹ During 1 day of patient care, the clinician completed a card for every patient seen. If the patient requested a medication during the visit, the clinician completed the entire card; otherwise, the clinician completed just the basic patient demographics. The use of a qualifying question maximizes the information about common primary care issues but does not require the clinician to answer unnecessary questions. Ralston et al determined the prevalence of tobacco usage by having clinicians answer questions for every patient seen during a 2-week period.²⁷ In that study, the investigators spread out data collection among clinicians to adjust for potential seasonal variation in patient visits. Prevalence studies can be biased if the clinical phenomena being studied are overrepresented in some practices or if they differ by the day of the week or time of year.

RESEARCH QUESTIONS AND CARD STUDIES

Some questions cannot be answered using a card study, and some clinical situations happen so infrequently it is not feasible to collect information about them using this method. Questions to consider when deciding whether a card study is the appropriate approach include:

1. What is the estimated prevalence of the condition? How will the season affect data or completion rates? If too few patients have the condition, you may have to stretch data collection over many months at the risk of practice burnout.

2. Do the potential study sites see the appropriate patient population? Do not ask about pregnancy if your practices deliver few babies.

3. Can clinicians answer questions about the condition at time of encounter or immediately thereafter? If they need to look up information, they likely will wait until later to answer.

4. Are the inclusion/exclusion criteria simple enough to make determinations? Will clinicians know the information for each patient? It is best to reduce the need to look elsewhere for the necessary patient data.

5. Can the card be completed in 1 minute or less? Can respondents complete the card without assistance? The goal is to minimize the work for people completing the card.

IMPLEMENTING THE CARD STUDY IN PRACTICE

It is important to develop a thorough implementation plan and discuss it with each practice. Each card study should include a written implementation plan that outlines the roles and responsibilities of each participating practice, clinician, and staff and describes the appropriate flow and allocation of forms. How will the card get into the clinician’s hands? Which patients or charts should have a card? Where should completed cards be placed? A checklist for the practice contact is useful. If possible, we try to discuss the project at a practice staff meeting and encourage them to tell us how implementation will work best in their practice. In large practices or those with less research experience, we may have one of our research assistants facilitate the start of data collection.

The level of card study complexity is an important factor that determines the level of support a practice will require. Understanding how a particular card study fits into the practice flow is essential for a successful study and requires frequent communication with the practice. We encourage each practice to designate a staff person as the primary contact or so-called champion for our PBRNs to facilitate the process. This person is usually the same clinician or staff person for each study but may change based on the study topic.

For a prevalence survey, we recommend having a research staff member present first thing in the morning on the first day. Alternatively, a telephone call first thing in the morning can get the study off to a successful start. If a study requires a single day of data collection in each practice, it may require several weeks or months to complete data collection across the network.

For a study with linked data, each practice will have multiple packets of clinician-specific cards, which may require more assistance and training from research staff. Occasionally a research assistant maybe required to be present when the linkage is essential or particularly complex. As the practices become more accustomed to conducting card studies, they require less on-site assistance.

ANALYZING CARD STUDY DATA

After data cleaning, descriptive statistics (frequency distributions, means, standard deviation, rates) are generated for all variables of interest in the data set. To the extent possible, participants are compared with the target population by computing 95% confidence intervals on means and proportions for sociodemographic and clinical variables. Preliminary bivariate analyses are used to examine associations between potential independent variables (covariates, other predictors) and the outcome(s). Multivariate analyses generally come next, adjusting for covariates identified in bivariate analyses as statistically associated with the outcome (generally at P <.20) or considered clinically important.

Most PBRN data structures are hierarchical, with patients nested (clustered) within clinicians and clinicians nested within practices. Patients within practices may be more similar to each other than patients in different practices. Traditional statistical methods, such as logistic or linear regression, assume independence of observations. In the case of hierarchical (nested, clustered) data, however, these assumptions are often not met. It is important to choose multilevel analytic methods that adjust for clustering in PBRN studies.²⁸

Another consideration in the design of PBRN studies concerns determining the number of subjects needed to address the research question. Again, it is important to determine the extent to which patients within practices are more similar than patients in different practices on the outcome variable of interest. This can be accomplished by using the intraclass correlation coefficient (ICC) from data collected or reported from other studies to adjust for clustering in estimating sample size needed to conduct the study.²⁸

TIMELINES AND COSTS OF CARD STUDIES

Although card studies are known to be quick and cheap, they actually require months of planning and a considerable budget. In our experience, most card studies require 24 months from study idea to publication, and we find that card studies require about $10,000 to $25,000 to complete. Items to consider when developing a budget for a card study include the following: salary support for the principal investigator, project coordinator, and biostatistician/analyst; copying, envelopes, other office supplies; postage (both for sending out practice packets and for practices to return completed cards); ballot boxes to collect completed cards; long-distance fees, telephone lines, and fax lines for communicating with sites; mileage for visiting practices; incentive or compensation to practices for participation, including food for lunches, etc; and travel and meeting funds to present results at conferences or to present results to participating sites.

DISCUSSION

Card studies are still an excellent research tool for PBRNs. Card studies are relatively easy on practices and clinicians. They are relevant to clinicians, are relatively quick to develop and deploy, and can engage practicing clinicians in research without the need for clinicians to dramatically change their practice style. Card studies have a reasonable cost and a standardized analytical approach, and they are flexible and can be customized to desired research question and practice environment. Card studies can fit within the new and changing practice environment to test innovations in care management and practice redesign. Finally, card studies are great for generating pilot data. Although card studies have been around for more than 30 years, they represent a proven method for conducting excellent research in PBRNs.