Cumulative Incidence of False-Positive Results in Repeated, Multimodal Cancer Screening

We thank Drs. Loud, Han, Gierach, and Greene for their comments. We agree with the statement that our findings likely underestimate false positive rates in general clinical practice. As noted in our article, we did not include mammography in our cumulative risk curves because this was not one of the modalities under evaluation in the PLCO randomized controlled cancer screening trial, and we did not want to extrapolate beyond available data in the study. Participants were censored after receipt of a first false-positive exam primarily for methodological reasons. That is, we considered receipt of a first false-positive test to be the endpoint of interest to minimize the need for unverifiable assumptions within our survival analyses.

We also agree that additional high-quality research is needed to better elucidate the short- and long-term harms of false positive screening tests. Although we were able to present both cumulative incidence and absolute rates of invasive procedures associated with false positive tests, other harms, including the associated economic and healthcare system burdens generated by cascading diagnostic follow-up, as well as the less quantifiable effects on participants’ views of their own health and mental well-being, warrant further investigation.

We share the view that premature adoption of new screening modalities into clinical practice should be curtailed. The PLCO trial was conceptualized for this very reason: the screening modalities under consideration in the trial were being applied to the general population without solid evidence in hand of their efficacy in reducing cancer mortality. Loud et al also suggest stratification of our results by participants’ baseline cancer risk, because “individuals at increased cancer risk should have lower FP rates than those in the general population, due to higher disease prevalence.” Although targeting screening to a population with an increased baseline prevalence of disease should improve the positive predictive value of the test, the specificity (and thus, the false positive rate) is actually independent of the underlying prevalence of disease.(1) This is illustrated by the formula for the false positive rate, which does not include prevalence within it:

FPR = 1 – specificity, where specificity is the probability of testing negative for the disease given that the person does not have the disease

Thus, the false positive rate is only dependent on the test’s ability to properly identify people that do not have the disease; technically, it is not affected by the proportion of individuals that do have the disease. As a result, when changing the baseline characteristics of the participants undergoing screening to target those at elevated risk, the associated false positive rate is difficult to predict: it may remain stable, decrease, or even trend in the wrong direction. A conceptual example of this would be that smokers might actually have higher rates of false positive results on chest CT or x-ray than nonsmokers, because exposure to tobacco also brings with it higher rates of chronic obstructive pulmonary disease, lung infections and resulting residual scarring, etc. These lesions may mimic findings suspicious for cancer on radiographic exams, so although the detection rate for cancer could climb, one might also see a rise in false positives.

In point of fact, this very scenario was borne out in an early analysis of the PLCO data not included in our article. In this investigation, we estimated individual cumulative false positive risk curves for CXR by smoking status, and found the risk was approximately 3 percentage points higher in smokers than nonsmoking participants after 3 tests (unpublished data).

We would like to correct one statement made by Loud et al: we calculated cumulative incidence curves for both the overall multi-modality screening program as well as for each individual modality. Information on the discrete cumulative false positive and diagnostic procedure rates for chest x-ray, PSA, digital rectal exam, CA-125, transvaginal ultrasound, or flexible sigmoidoscopy can be found in Figures 2B, 3B, and in the online supplemental materials (http://www.annfammed.org/cgi/data/7/3/212/DC1/1). We agree that this information should be helpful for practitioners during informed decision-making discussions with patients. However, we maintain that having an estimate of the overall potential burden that might be expected to result from a multimodality screening program is also of practical use to practitioners, the public, and guidelines-generating organizations. We believe this is the first study to use randomized data to provide a concrete “big-picture” view of the real consequences of the exact scenario Loud et al describe: “Subsets of providers recommending particular tests that apply to their own specialties, without regard for screening tests beyond their purview.” It is precisely this type of behavior that generates a proliferation of testing with little consideration as to the overall utility of the program or the true balance of benefits and harms to a given individual. It is our hope that primary care physicians might therefore lead the way among providers in discouraging such activities, given the demonstrated downsides of this haphazard approach.

1. Brawley OB and Kramer BS. Cancer screening in theory and in practice. Journal of Clinical Oncology. 2005; 23: 293-300.

Competing interests:   None declared

Croswell et al. have appropriately focused attention on the cumulative impact of false positive (FP) tests when applying routine, periodic, general population screening for multiple cancers. We suggest that their striking FP rates actually underestimate the true rates in clinical practice: mammography – widely applied – was not considered, and their analysis censored patients at first FP event, thereby excluding any subsequent FP results likely experienced with continued screening.

Beyond the high prevalence of FP results, we need to define the actual harms that follow, including potential adverse effects on screening behaviors. Limited current evidence suggests that FP results cause increased anxiety, health distress and fear of death (1), reduced adherence to subsequent screening recommendations (2,3), and increased health care costs due to unnecessary diagnostic evaluations (4). A closer approximation of actual FP-related harms might be reached by analyzing the “level 3” PLCO outcomes as a better surrogate for serious injury. Undergoing a surgical procedure (e.g., laparoscopy) without detecting cancer is a major adverse outcome. These important issues need to be better understood.

Meanwhile, we need a safe and effective strategy for reducing FP cancer screening test rates. Perhaps the most important strategy: do not perform unvalidated screening tests; of the 4 tests studied by PLCO, only one has evidence of clinical utility. Targeting screening to populations at increased cancer risk—identified using validated cancer risk prediction models (5)—is another promising strategy. We therefore encourage the authors to stratify their results by participants’ baseline cancer risk. Individuals at increased cancer risk should have lower FP rates than those in the general population, due to higher disease prevalence. Among those at higher risk, however, an increased understanding of the responses to FP results and the clinical implications of these responses is critical. The harm is arguably greater if a FP result leads a high-risk patient to abandon screening.

Although analytically creative (by addressing aggregate FP screening rates at the health system level), Croswell et al.’s report is limited by having analyzed all screening tests as a group. Utilization of these tests is often driven by subsets of providers recommending particular tests that apply to their own specialties, without regard for screening tests beyond their purview. To better inform clinical care, the authors might consider separately reporting the cumulative FP rates for each test. This information, we would argue, is more likely to beneficially influence clinician-patient decision-making than the combined analysis.

References
1. McGovern PM, Gross CR, Krueger RA, Engelhard DA, Cordes JE, Chruch TR. False-positive cancer screens and health-related quality of life. Can Nurs2004;27(5):347-352.
2. Brewer NT, Salz T, Lillie SE. Systematic Review: The long-term effects of false-positive mammograms. Ann Int Med 2007;146:502-510.
3. Taylor KL, Shelby R, Gelmann E, McGuire C. Quality of life and trial adherence among participants in the prostate, lung, colorectal and ovarian cancer screening trial. JNCI 2004;96(14):1083-1094.
4. Lafata JE, Simpkins J, Lamerato L, Poisson L, Divine G, Johnson CC. The economic impact of false-positive cancer screens. CEB&P2004;13(12):2125-2132.
5. Evans DG, Howell A. Breast cancer risk-assessment models. Breast Cancer Res 2007;9(5):213-

Competing interests:   None declared

We thank Dr. Taplin for his interest and insights regarding our article on cumulative false positives in multi-modality screening programs. We agree that one of the important contributions of the article is the analytic approach, which should be applicable to a broad range of medical screening programs. He correctly points out that the tests being evaluated by the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial (a randomized trial examining the effects of 6 different screening modalities on cancer mortality) include several tests not universally recommended for mass screening (PSA, CA-125, chest x-ray, and transvaginal ultrasound), and as such, believes the reported false positive probabilities may not be reflective of current clinical practice.

We very much appreciate the need to acquire an accurate estimate of the overall false positive burden (and associated morbidity) generated by cancer screening programs in common community use. The difficulty, of course, lies in the general paucity of available high-quality evidence for cancer screening modalities. The PLCO trial provided a rare opportunity to evaluate the burdens of a multimodality screening program in a large, long-term, randomized, controlled setting. Given a dearth of ongoing multiple-modality clinical trials of cancer screening, attempts to generate a similar probability curve for “currently recommended tests” would necessitate the use of modeling or other potentially inaccurate assumptions/imputations, an issue Dr. Taplin raises himself in his commentary.

Furthermore, it is not clear that one can, in point of fact, crisply define what constitutes “currently recommended tests and actual experience in community practice.” Not all guidelines organizations agree upon which screening tests ought to be promoted to the general public. For example, the U.S. Preventive Services Task Force does not currently recommend against prostate cancer screening; rather, it concludes that “the evidence is insufficient to assess the balance of benefits and harms of prostate cancer screening in men younger than age 75 years.”1 Nevertheless both PSA and DRE are actively recommended by some professional societies--most notably, the American Urological Association revised its guidelines in April of this year to state that PSA and digital rectal exam “should be offered [annually] to asymptomatic men 40 years of age or older who have a life expectancy of at least 10 years.”2 Whether or not one agrees with this recommendation, the majority of men in the United States have been screened with one or both of these tests. For CA-125 and transvaginal ultrasound, although the American College of Obstetrics and Gynecology recommends against routine use of these tests3, they, the Ovarian Cancer National Alliance, and the UK National Health Service formally recognize key “early detection symptoms” (abdominal pain, bloating, back pain, fatigue, etc.) “important to be checked”3-5 that are non-specific enough to be likely to trigger the application of one or both of these tests in a potentially broad segment of the population. Finally, the USPSTF does not currently recommend against screening with chest x-ray, but rather concludes that “the evidence is insufficient to recommend for or against screening….”6 Routine admission chest x-rays are the norm in many hospital settings. This modality is an extremely common pre-operative screening tool in community practice (as one of the article’s authors--a 35-year old non-smoking female—personally experienced).

Given the wide disparity in screening recommendations and utilizations currently perpetuated within the U.S. healthcare system, as well as the relative paucity of cancer screening trials to derive information from, we believe that this study has both methodologic and practical implications. It represents a reasonable current “best estimate” of the magnitude of the false positive burden that could be expected within a multi-modality, multi-year program of screening. While the actual estimated percentage will indeed vary depending on the modalities employed and local medical practice, the fundamental point highlighted by this study—that the burden rapidly becomes quite substantial—will remain unchanged.

1. U.S. Preventive Services Task Force. Screening for Prostate Cancer. Ann Intern Med. 2009; 149:185-191.
2. American Urological Association. Policy Statement: Early Detection of Prostate Cancer (Revised April 2009). Accessed online at: http://www.auanet.org/content/guidelines-and-quality-care/policy- statements/e/early-detection-of-prostate-cancer.cfm on May 28, 2009.
3. American College of Obstetrics and Gynecology. Patient Education Pamphlet APO96: Cancer of the Ovary. Accessed online at: http://www.acog.org/publications/patient_education/bp096.cfm on May 28, 2009.
4. Ovarian Cancer National Alliance. Symptoms. Web page. Accessed at: http://www.ovariancancer.org/index.cfm?fuseaction=Page.viewPage&pageId=769&parentID=764&nodeID=1 on May 28, 2009.
5. U.K. National Health Service. NHS Choices: Your Health, Your Choice. Symptoms of Ovarian Cancer. Accessed at: http://www.nhs.uk/Livewell/cancer/Pages/Ovariancancer.aspx on May 28, 2009.
6. U.S. Preventive Services Task Force. Lung Cancer Screening. Ann Intern Med. 2004;140: 738–739.

Competing interests:   None declared