Abstract
PURPOSE In specialist care, fecal calprotectin (FCal) is a commonly used noninvasive diagnostic test for ruling out inflammatory bowel disease (IBD) in children with chronic gastrointestinal symptoms. The aim of this study was to evaluate the diagnostic accuracy of FCal for IBD in symptomatic children in primary care.
METHODS We studied 2 prospective cohorts of children with chronic diarrhea, recurrent abdominal pain, or both: children initially seen in primary care (primary care cohort) and children referred to specialist care (referred cohort). FCal (index test) was measured at baseline and compared with 1 of the 2 reference standards for IBD: endoscopic assessment or 1-year follow-up. Physicians were blinded to FCal results, and values greater than 50 μg/g feces were considered positive. We determined specificity in the primary care cohort and sensitivity in the referred cohort.
RESULTS None of the 114 children in the primary care cohort ultimately received a diagnosis of IBD. The specificity of FCal in the primary care cohort was 0.87 (95% CI, 0.80–0.92). Among the 90 children in the referred cohort, 17 (19%) ultimately received a diagnosis of IBD. The sensitivity of FCal in the referred cohort was 0.99 (95% CI, 0.81–1.00).
CONCLUSIONS The findings of this study suggest that a positive FCal result in children with chronic gastrointestinal symptoms seen in primary care is not likely to be indicative of IBD. A negative FCal result is likely to be a true negative, which safely rules out IBD in children in whom a primary care physician considers referral to specialist care.
- inflammatory bowel disease
- calprotectin
- child
- sensitivity and specificity
- primary health care
- practice-based research
INTRODUCTION
Primary care physicians frequently manage recurrent abdominal pain or diarrhea in children. These symptoms account for approximately 2% to 5% of all childhood consultations.1–3 Although they are typically functional in origin, it is essential that organic disease be ruled out. Inflammatory bowel disease (IBD), that is, Crohn disease and ulcerative colitis, is an organic disease that primary care physicians should not miss. Delay in diagnosing IBD, and the resultant delay in receipt of appropriate treatment, may prolong suffering and can lead to complications such as anemia, irreversible growth failure, and delayed sexual maturation.4,5
According to guidelines, primary care physicians should refer children with chronic diarrhea, recurrent abdominal pain, or both for specialist care if red flags are present.6,7 The red flags are nonspecific and discriminate poorly between functional and organic gastrointestinal diseases,8–10 often leading to referral and extensive diagnostic testing. For children with functional disorders, referral or extensive testing may delay appropriate interventions and further decrease well-being.11,12
Calprotectin is a calcium-binding protein released from neutrophils during intestinal inflammation that can be easily measured in feces.13,14 In specialist care, evidence shows it to be a useful, simple, noninvasive test that can rule out IBD in children with gastrointestinal symptoms.15–17 The diagnostic accuracy of fecal calprotectin (FCal) has not been assessed in children evaluated in primary care, however.10,18,19 Primary and specialist care often have different populations, case mixes, and disease severity, which can affect the pretest probability of IBD and the sensitivity and specificity of the FCal test. The diagnostic accuracy of FCal in the primary care setting should therefore be clarified before this test is recommended for routine use in primary care. We set out to study the diagnostic accuracy of FCal for identifying IBD in children with chronic gastrointestinal symptoms in primary care.
METHODS
Study Design
This was a prospective cohort study with a delayed-type cross-sectional design.20 Children in the Netherlands with chronic gastrointestinal symptoms were included from July 2011 to July 2013 and had 12 months of follow-up. We studied 2 cohorts: (1) the primary care cohort consisted of consecutive children who were seen by any of 64 primary care physicians (38 practices); (2) the referred cohort consisted of consecutive children who were referred for diagnostic workup by primary care physicians and general pediatricians to any of 4 general hospitals or 3 academic centers, as well as children selected from the primary care cohort based on the presence of at least 1 red flag (Figure 1).
The medical ethics committee of the University Medical Centre Groningen approved the study. Written informed consent was provided by the parents of all children and by all children aged 12 years or older. The study design has been described in more detail elsewhere.21
Participants
Children aged 4 to 18 years who sought care for chronic diarrhea, recurrent abdominal pain, or both were eligible. Chronic diarrhea was defined as soft to watery stool (score of 5, 6, or 7 on the Bristol stool chart22) for at least 2 weeks or at least 2 episodes in the past 6 months. Recurrent abdominal pain was defined as at least 2 episodes of abdominal pain or discomfort in the past 6 months. Children were excluded if they had a previous diagnosis of chronic organic gastrointestinal disease; an evaluation with endoscopy or FCal for gastrointestinal symptoms in 6 months before this study; or difficulty in understanding questionnaires. Furthermore, we excluded children with long-term use (>3 months) of antibiotics, nonsteroidal anti-inflammatory drugs, or oral corticosteroids in the past 6 months, as well as those aged younger than 4 years, because previous studies have demonstrated elevated calprotectin concentrations in these groups.23,24
Baseline Evaluation
All participating physicians assessed children for the presence of 10 red flags suggestive of IBD using a structured evaluation form. These red flags consisted of 6 alarm symptoms and 4 blood markers (Table 1).25–27 Feces were tested for pathogens—Salmonella enterica, Campylobacter jejuni, Shigella spp/enteroinvasive Escherichia coli (EIEC), and Shiga toxin–producing Escherichia coli (STEC)—and for parasites—Giardia lamblia, Cryptosporidium spp, Dientamoeba fragilis, and Entamoeba histolytica— using real-time polymerase chain reaction.
Patient Flow
All children in the primary care cohort were evaluated by their primary care physician. Children were selected for further diagnostic workup based on the presence of at least 1 red flag. Children in the referred cohort were evaluated by a pediatric gastroenterologist, who decided whether the child required endoscopic evaluation. All children were followed up for 1 year, during which time the attending physician was free to refer a child for further diagnostic workup. At 1 year, the physician did a structured physical examination of all children with persisting gastrointestinal symptoms who had not received a diagnosis of IBD. Children with at least 1 red flag at this time were seen by a pediatric gastroenterologist for further evaluation. The attending physicians of children lost to follow-up were contacted after 1 year to provide updated information on persisting symptoms and additional diagnoses.
Confirmatory Diagnosis
IBD was diagnosed by esophagogastroduodenoscopy, ileocolonoscopy, and histopathologic examination according to the Porto criteria.6 A negative endoscopy was defined as the absence of endoscopic and histopathologic findings of IBD. Children were considered not to have IBD if, after 12 months, the attending physician found no red flags or the pediatric gastroenterologist decided that red flags were not related to IBD.
Index Test
We used as the index test an FCal test with a lowest sensitivity of 19.5 μg/g feces and a cutoff point of greater than 50 μg/g feces according to the manufacturer. Stool samples collected at baseline were stored at −80°C and analyzed by a commercially available quantitative enzyme-linked immunosorbent assay (PhiCal test, Calpro AS) at the end of the data collection period (September 2014) in the department of clinical chemistry at Erasmus Medical Center. All physicians, pathologists, researchers, and patients were blinded to the outcome of the FCal test and were not allowed to apply for another FCal test during the data collection period. The laboratory staff evaluating FCal were blinded to the children’s clinical characteristics and diagnoses.
Statistical Analysis
Very few children seen in primary care with chronic gastrointestinal symptoms have IBD. When introducing new tests in these children, it is important that false-positive outcomes are minimized to avoid referrals for endoscopies. We therefore estimated specificity with adequate precision of FCal measurements in the primary care cohort. Given an expected specificity of 93%,18,28–30 a 95% CI and precision of 5%, an IBD prevalence of 5%, and a loss to follow-up of 10%, we needed to include 118 children in the primary care cohort.
The likelihood of IBD increases in children in whom the primary care physician considers a referral. In this group, it is important not to miss a case of IBD, and a low false-negative rate is preferred. We therefore estimated sensitivity with adequate precision of FCal measurement in the referred cohort. Given an expected sensitivity of 95%,18,28,30 a 95% CI and precision of ±10%, an IBD prevalence of 20%, and a loss to follow-up of 10%, we needed to include 100 children in the referred cohort.21
For secondary outcomes, we calculated specificity, posttest probability, and area under the receiver operating characteristic curve (AUC) with the 95% CI in the referred cohort. We also determined the effect of applying different FCal cutoff values (>50 μg/g, >100 μg/g, >250 μg/g)16,31 on the test characteristics, number of referrals, and missed diagnoses of IBD.
We assumed that children included in specialist care were comparable to those with at least 1 red flag from the primary care cohort. To test this assumption, we compared the characteristics of these groups. In order to evaluate spectrum bias (whereby the test setting affects the test performance), we performed subgroup analyses in the referred cohort, comparing children who were referred by a primary care physician with those referred by a general pediatrician. To evaluate the likelihood of differential verification bias,32 we separately calculated the test characteristics for both endoscopy and 12-month follow-up.
We conducted a missing value analysis to rule out missing not at random (MNAR) as a possible explanation for missing data from the variables of FCal and diagnosis. The missing data were replaced based on a multiple imputation procedure (conditional specification, predictive mean matching,33 20 iterations, and 20 data sets). The patient characteristics, all symptoms, all diagnostic tests, setting, endoscopy performed, and whether IBD was diagnosed were included as predictors. We used the Rubin rule to calculate the pooled AUC.34 Results of the analyses on the nonimputed and imputed data sets were compared to assess the effect of multiple imputations on diagnostic accuracy. Statistical analyses were performed with IBM SPSS for Windows, version 20.0 (IBM Corp).
RESULTS
Participants
The primary care cohort had 114 children, and the referred cohort had 90 children (24 with red flags from primary care plus 66 from specialist care) (Figure 1). Table 2 shows that the 66 children from specialist care more often had weight loss, extra-intestinal symptoms, and decreased hemoglobin levels compared with the 24 children with red flags from primary care. The 25 children who were referred to the pediatric gastroenterologist by a general pediatrician showed a higher IBD prevalence than the 65 children who were referred by their primary care physician (12 vs 5 cases).
Diagnoses
None of the children in the primary care cohort received a diagnosis of IBD (Table 3). The final diagnosis was based on 12 months’ follow-up in 111 children and endoscopy in 2 children. Of the 90 children in the referred cohort, 29 (32%) underwent endoscopic evaluation, and 17 (19%) ultimately received a diagnosis of IBD.
Fecal Calprotectin
The median intervals from stool collection to endoscopy were 4 days and 8 days for children with IBD and without IBD, respectively; however, 11 of the 27 children (2 had missing samples) who underwent endoscopy experienced a delay of more than 1 month. Two children in the primary care cohort had similar calprotectin values in samples collected at baseline and shortly before endoscopy. In total, values were missing for 11 children (9.6%) in the primary care cohort and 5 (5.5%) in the referred cohort.
The contingency tables and test characteristics for FCal by follow-up or endoscopy are presented in Figure 2 for the nonimputed data set. The outcomes were comparable for all parameters in the imputed and nonimputed data sets. In the imputed data set, the specificity of FCal for IBD in the primary care cohort was 0.87 (95% CI, 0.80–0.92). Six of the 24 children with red flags and 9 of the 90 children without red flags had a positive calprotectin value. In the referred cohort, the sensitivity was 0.99 (95% CI, 0.81–1.00). The AUC of FCal in referred children cohort was 0.98 (95% CI, 0.96–1.00).
In subgroup analyses, the sensitivities of the test in children referred to specialist care by their primary care physician and by a general pediatrician were 0.98 (95% CI, 0.58–1.00) and 1.00 (95% CI, 0.76–1.00), respectively. An increase in the cutoff value for FCal from 50 μg/g to 250 μg/g feces in the referred cohort would lead to an extra 14% reduction in referrals for diagnostic workup for IBD, but would also increase the percentage of missed IBD diagnoses from 0% to 18% (Table 4).
DISCUSSION
This is the first study to evaluate the test characteristics of FCal as a marker for IBD in children seen for chronic gastrointestinal symptoms in primary care. None of the children in the primary care cohort ultimately received a diagnosis of IBD, suggesting that children with chronic gastrointestinal symptoms should not be referred directly for evaluation of IBD. Current guidelines recommend referral for diagnostic workup based on the presence of red flags. In our primary care cohort, referrals based on red flags would have resulted in a higher false-positive referral rate for IBD compared with referrals based on FCal exceeding 50 μg/g feces. To date, however, there is insufficient evidence to recommend FCal as a tool to guide decisions about referral for diagnostic workup of all children with chronic gastrointestinal symptoms seen in primary care.
FCal showed high sensitivity (0.99; 95% CI, 0.81–1.00) in the referred cohort. Therefore, a negative FCal result may safely rule out IBD and thereby reduce the number of referrals for evaluation of IBD in children whom the primary care physician considers a referral. Nevertheless, the 95% CIs of the false-negative rates are large because of the relatively small numbers of children with IBD included in our study and should be interpreted with caution. In this study we focused on IBD, but from the point of view of the primary care physician, it is important to determine whether the symptoms are related to any organic disease. In addition to 17 children with IBD, 3 children were found to have other organic diseases (1 had celiac disease, 1 had reflux esophagitis, and 1 had a solitary rectal ulcer). Children with celiac disease and reflux esophagitis had normal FCal levels and would have been missed if a referral had been solely based on the results of this test.
Cutoff points might need to be higher in primary care to maintain a high negative predictive value.35 Although we found that an increase of the threshold from 50 μg/g to 250 μg/g feces reduced referrals by 14% (with a drop from 32% to 18% referred), this threshold also led to false-negative results and missed cases of IBD (with an increase from 0% to 18% cases missed) in the referred cohort. A pragmatic approach may be to monitor children with an initial calprotectin value between 50 μg/g and 250 μg/g feces. Children whose symptoms persist and whose calprotectin values remain high can still be referred later. A similar approach has been suggested for adults in primary care, where it was suggested that patients with irritable bowel syndrome and an initial FCal value between 50 μg/g and 150 μg/g feces who have persistent symptoms without treatment should be retested after 3 months.31
The specificity of 0.87 that we identified in the primary care cohort is higher than that reported in studies performed in specialist care, where the pooled specificity ranged between 0.68 and 0.76.15–17 We expected lower specificities because the patient mix was thought to be more diverse in primary care and because calprotectin concentrations increase in conditions such as gastroenteritis.36 Moreover, in the referred cohort, the specificity of FCal was lower in children who underwent endoscopy (0.67) than in those who received clinical follow-up (0.87). The higher specificity might be explained by higher numbers of children with functional disease in the primary care cohort and in children with clinical follow-up in the referred cohort. Consequently, the test setting (primary vs specialist care) might affect the specificity of FCal (spectrum bias).
We found a sensitivity of 0.99 in the referred cohort, which is comparable to that reported in other studies performed in specialist care (IBD prevalence of about 60%), where the pooled sensitivity ranged between 0.92 and 0.98.15–17 The reported sensitivity of FCal in this study might be an overestimation of that in children in whom a primary care physician considers a referral for diagnostic workup. In the subgroup of children referred by a primary care physician (IBD prevalence of 8%), the sensitivity was comparable to that in children referred by a general pediatrician (IBD prevalence of 48%) (0.98 vs 1.00). We therefore assume that spectrum bias did not substantially affect the sensitivity of FCal. Lack of spectrum bias might be explained by the fact that the intestinal inflammation caused by IBD is usually severe enough to increase the calprotectin value to more than 50 μg/g feces, even in the early stages of the disease. A meta-analysis showed that sensitivity remains stable over a range of prevalences and is not substantially influenced by spectrum bias.37 A normal FCal level thus could be used to prevent a referral of children with functional symptoms to specialist care.
Ideally, a study of FCal would include consecutive children with symptoms suggestive of IBD initially evaluated in primary care. This design would be extremely time consuming and costly, however. In order to reflect daily, real-world practice, we included a cohort of children in whom the primary care physician considered a referral for diagnostic workup for chronic gastrointestinal symptoms. We assumed that the children first seen in specialist care were comparable to the selected children with red flags seen in the primary care cohort. We are confident that this assumption is valid as there were only a few differences in characteristics between these groups. Children referred by a primary care physician had a lower probability of IBD (8%) than those referred by a general pediatrician (48%), however. These findings are consistent with what one might expect in the Dutch health care system, where children can consult a pediatrician only after obtaining a referral from their primary care physician, and a pediatric gastroenterologist can be consulted only after a referral from a primary care physician or general pediatrician. Comparable health care systems exist in the United Kingdom, Scandinavia, Canada, New Zealand, and Australia.38
Not all patients received the same reference standard test, which may cause differential verification bias.32 As it is unethical to perform endoscopy in children with a low likelihood of organic gastrointestinal disease, these children received follow-up evaluations over 1 year. An important aspect for deciding whether the verification leads to biased estimates of accuracy is the length of the follow-up period. We are confident that we identified all children with IBD, because of the extremely low probability that a child without red flags or indication for endoscopy during 1 year of follow-up has the disease.39 We did not identify new cases of IBD at the end of the follow-up period, even among children who developed red flags. Only 2 children ultimately received a diagnosis of IBD within the follow-up period (both at 6 months). The use of follow-up in children in whom endoscopy is not considered ethical does not correspond very well with the ideal situation that arises when the diagnosis of all cases is determined by endoscopy. Following children during 1 year is the best option given the reality of clinical care, however.20
To recommend a test in a new setting, the diagnostic value of that test needs to be investigated in that setting.40 We found that, in selected children in whom a primary care physician considers a referral, FCal has satisfactory discriminatory power between children with and without IBD. Of greater clinical relevance, however, is whether FCal can add to the diagnostic information that is readily available from a thorough history and physical examination.41 Moreover, the added value of commonly used blood markers should be compared with the added value of FCal. Further research is therefore needed to determine whether FCal should be incorporated into the routine diagnostic evaluation of pediatric patients with chronic gastrointestinal symptoms and red flags in primary care. In addition, research should be performed to evaluate the cost-effectiveness of FCal in primary care.
Acknowledgment
We would like to thank the primary care physicians and pediatricians for recruiting patients for this study, and we thank all participants for their contribution. The analytical support of Sascha Smit, Erasmus, MC, is highly appreciated. The editorial assistance by Dr Robert Sykes is appreciated.
Footnotes
Conflicts of interest: authors report none.
Funding support: This study was funded by a grant from the Netherlands Fonds NutsOhra (grant number 0802-099) and the University Medical Centre Groningen doelmatigheidsonderzoek (grant number 689 302).
Disclaimer: The funders did not influence the design, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.
Previous presentations: The results of this study were presented at the European Society for Paediatric Gastroenterology, Hepatology and Nutrition (ESPGHAN), May 6–9, 2015, Amsterdam, the Netherlands, and at NHG-wetenschapsdag, June 19, 2015, Rotterdam, the Netherlands.
- Received for publication October 5, 2015.
- Revision received March 7, 2016.
- Accepted for publication March 16, 2016.
- © 2016 Annals of Family Medicine, Inc.