Abstract
PURPOSE We aimed to validate a family history screening questionnaire in an Australian primary care population designed to identify people at increased risk for breast, ovarian, colorectal, and prostate cancer; melanoma; ischemic heart disease; and type 2 diabetes.
METHODS We prospectively validated the questionnaire in 6 general practices in Perth, Western Australia among 526 patients aged 20 to 50 years who responded to a single invitation from their general practice. They completed the 15-item questionnaire before a reference standard 3-generation pedigree was obtained by a genetic counselor blinded to the questionnaire responses. We calculated diagnostic performance statistics for the questionnaire using the pedigree as the reference standard.
RESULTS A combination of 9 questions had the following diagnostic performance, expressed as value (95% CI), to identify increased risk of any of the 7 conditions: area under the receiver operating characteristic curve 84.6% (81.2%–88.1%), 95% sensitivity (92%–98%), and 54% specificity (48%–60%). The combination of questions to detect increased risk had sensitivity of 92% (84%–99%) and 96% (93%–99%) for the 5 and 6 conditions applicable only to men and women, respectively. The specificity was 63% (28%–52%) for men and 49% (42%–56%) for women. The positive predictive values were 67% (56%–78%) and 68% (63%–73%), and the false-positive rates were 9% (0.5%–17%) and 9% (3%–15%) for men and women, respectively.
CONCLUSIONS This simple family history screening questionnaire shows good performance for identifying primary care patients at increased disease risk because of their family history. It could be used in primary care as part of a systematic approach to tailored disease prevention.
INTRODUCTION
Taking a family history is traditionally regarded as a routine part of obtaining the medical history, but it is not used in a systematic way in clinical practice.1,2 Internationally, there is growing recognition that a family medical history can support tailored disease prevention, which may be more effective than existing approaches.3 The risk of many common and serious diseases (such as various cancers, heart disease, and diabetes) is increased in the presence of a family history of the disorder, representing not only shared genetic factors but also environmental and behavioral exposures.4–6 Despite noteworthy advances in understanding the human genome’s contribution to disease and in whole-genome sequencing, clinically useful DNA-based tests to predict common disease risk have not yet been identified. In the meantime, the family medical history remains the most relevant genetic risk tool for use in clinical practice.7,8
There are effective interventions for primary and secondary prevention of these common diseases, ranging from disease surveillance to drug treatments and lifestyle management. Some evidence suggests that having knowledge of a family history of a specific condition is associated with improved uptake of a range of disease-preventive activities for breast, colorectal, and skin cancer, such as cancer screening and reduced sun exposure9; therefore, identifying people with a family history of disease could act as an additional motivator for them to change or modify their lifestyle, or participate in disease screening. Recording a patient’s family history to assess disease risk ideally requires a 3-generation pedigree, but constructing one can take up to 30 minutes,10 which is unrealistic in most clinical settings. Not all patients require such a detailed assessment, however. Simple, self-completed family history screening questionnaires (FHQs) could provide an answer, assuming they are accurate. Several already exist, but most are disease specific, and few have been formally tested to determine their screening characteristics.11
We report the development and validation of an FHQ designed to assess risk of multiple common diseases in Australian primary care. The aim of this study was to test its diagnostic accuracy against a reference standard of a detailed 3-generation pedigree.
METHODS
Development of the FHQ
The FHQ, including specific wording of items, was informed by our systematic review of existing FHQs.11 It was designed as a brief questionnaire for patient completion that aimed to classify individuals as either at population risk or potentially increased risk for several common chronic diseases. Those who screened positive on the FHQ would, in clinical practice, require a more detailed assessment of their family history. The selection of conditions for inclusion in the FHQ was based on the Royal Australian College of General Practitioners’ (RACGP) Guidelines for Preventive Activities in General Practice (6th edition)12 and the National Breast and Ovarian Cancer Centre (NBOCC) guidelines for familial breast and ovarian cancer.13 The FHQ included items about family history and ethnicity associated with increased risk of specific conditions. A condition was included if there were clear family history criteria that determined increased risk (as indicated by the guidelines) and effective interventions to manage those at increased risk. We therefore included breast, ovarian, prostate, and colorectal cancer; melanoma; ischemic heart disease; and type 2 diabetes. The FHQ contained 15 questions, each with a binary response, including 3 relating to ethnicity (Figure 1). The format and precise wording of items/questions was pilot tested, resulting in minor modifications, before the validation study commenced.
Participants
Participants from 6 general practices in metropolitan Perth, Western Australia, were eligible if they had attended the practice at least once in the previous 12 months and were aged 20 to 50 years, to represent a population most likely to benefit from early identification of familial disease risk. They were included even if they had a personal history of 1 or more but not all of the conditions in the FHQ. Electronic searches of the general practice records identified an eligible population from which random samples were taken and invited into the study by letter from their general practitioner. No reminders were sent in order to mimic a potential approach to implement the FHQ in practice.
Reference Standard
The reference standard was a 3-generation pedigree, including information about ethnicity, disease status of each relative, and age at diagnosis, collected by a qualified genetic counselor (G.R.). Disease risk for each condition in the FHQ was determined from national risk assessment criteria and classified as either population risk or increased risk.12,13 Where there was uncertainty about risk assessment because of a more complex family history, a clinical geneticist (D.R.) reviewed the pedigree. Collection of the pedigree and risk assessment was conducted blinded to the responses in the FHQ.
Procedures
Recruited participants attended an appointment with a research genetic counselor either at their general practice or in the university department. They completed the FHQ as part of this appointment before the pedigree was collected by the genetic counselor. A psychological impact questionnaire containing measures of anxiety, risk perception, and health behaviors was completed at baseline and at 1 week and 3 months after completing the FHQ (Reid et al, unpublished data, 2013). This study was approved by the University of Western Australia Human Research Ethics Committee (Ref RA/4/1/1410).
Statistical Analyses
Data from participants were excluded from specific analyses if they had a personal history of the condition in question. Also, men were excluded from breast and ovarian cancer analyses, and women from prostate cancer analyses.
We performed univariate analyses using the Fisher exact test to identify questions associated with increased risk of specific conditions. Questions were taken forward into multiple logistic regression models if the univariate association was significant at the 10% level. The sensitivity and specificity of each question, and combinations of related questions, to identify increased risk for each condition were calculated. We calculated associated 95% confidence intervals using standard approaches including the Wilson method to account for small sample sizes in some of the cells. Receiver operating characteristic (ROC) curves using multiple logistic regression analysis were generated to test combinations of questions that identified increased risk of any condition using SPSS version 18 (SPSS Inc). Values are reported below as percent (95% CI).
Sample Size
On the basis of published estimates, we expected that 40% of the population would be at increased risk for 1 of the studied conditions.14,15 A sample of 500 allowed us to estimate sensitivity of 90% and specificity 80% with 95% confidence intervals of 85.8% to 94.2% and 75.5% to 84.5%, respectively.
RESULTS
Participants
We sent 11,899 invitation letters between May 2008 and December 2010; 530 were returned to sender, whereas 604 produced expressions of interest (adjusted response rate = 5.3%). Of these responders, 19 individuals were ineligible because of age, 17 withdrew, and 42 were lost to follow-up.
A total of 526 participants formally consented and completed the FHQ and reference standard pedigree assessment. Their mean age was 40.1 years (SD 7.5); 77% were female, 74% were married or cohabiting, and 57% had a university degree or equivalent. Most (96%) reported themselves to be either Oceanic or of European ethnicity; 3 (0.6%) participants identified themselves as Aboriginal or Torres Strait Islander, 3 Chinese, 3 Indian, 4 (0.8%) Jewish, 4 South East Asian, and 1 (0.2%) Maori. We were unable to obtain data on the demographics of nonresponders because of ethical restrictions; available data for 5 of the 6 recruiting practices showed that 42.9% of all patients were female. Table 1 shows the prevalence of increased risk for each condition; 51.5% of participants were at increased risk for at least 1 condition.
FHQ Performance
Tables 2 and 3 report the estimates of sensitivity and specificity to detect increased risk for conditions applicable to both sexes and for conditions that are sex-specific, respectively. The generic family history question (question 1) performed poorly; the high sensitivity was countered by low specificity for all conditions. There were too few participants from the relevant ethnic backgrounds to assess the performance of the ethnicity-related questions. Performance of the FHQ for identifying increased risk of specific conditions is detailed below.
Heart Disease
In univariate analyses, questions 2, 1, 4, 6, and 5b were associated with increased risk of heart disease, presented in decreasing order of strength of association. These questions were therefore included in subsequent multivariate analyses. Question 2 was the most closely associated predictor for increased risk of heart disease, with sensitivity of 88% (67%–85%) and specificity of 81% (77%–85%), and questions 1, 4, 5b, and 6 were no longer significantly predictive after accounting for question 2. In this population with a prevalence of increased risk of heart disease of 16.6%, question 2 had a positive predictive value (PPV) of 48% (40%–56%) and false-positive rate (FPR) of 3% (1%–5%).
Type 2 Diabetes
In univariate analyses questions 4, 2, 5b, 5a, 9, 5c, and 1 were associated with increased risk of type 2 diabetes, presented in order of strength of association. These questions were therefore included in subsequent multivariate analyses. Question 4 was the most closely associated predictor for increased risk of diabetes, with sensitivity of 92% (86%–97%) and specificity of 81% (77%–85%); the other questions were no longer significantly predictive after accounting for question 4. In this population with a prevalence of increased risk of type 2 diabetes of 23%, question 4 had a PPV of 59% (52%–66%) and FPR of 3% (1%–5%).
Melanoma
In univariate analyses, questions 8 and 14 were associated with increased risk of melanoma and were included in subsequent multivariate analyses. Question 8 was the most closely associated predictor of increased risk of melanoma with a sensitivity of 94% (85%–99%) and a specificity of 84% (80%–87%); question 14 was no longer predictive after accounting for question 8. In this population with a prevalence of increased risk of melanoma of 10%, question 8 had a PPV of 41% (32%–50%) and an FPR of 1% (0.01%–2%).
Colorectal Cancer
In univariate analyses, questions 13, 12, 14, 4, and 15 were associated with increased risk of colorectal cancer, presented in order of strength of association, and were included in subsequent multivariate analyses. Questions 13 alone performed well, with sensitivity of 53% (35%–70%) and specificity of 96% (94%–98%), as did question 12, with sensitivity of 44% (27%–62%) and specificity of 95% (93%–97%). By combining questions 12 and 13 (through a yes response to either), we obtained a higher sensitivity of 76% (59%–89%) and specificity of 92% (90%–95%). In this population with a prevalence of increased risk of colorectal cancer of 6.5%, the combination of questions 12 and 13 had a PPV of 41% (29%–54%) and FPR of 2% (1%–3%).
Prostate Cancer
Only question 6 was significantly associated with increased risk of prostate cancer, with 83% sensitivity (36%–99%) and 98% specificity (94%–99%). In this population with a 5% prevalence of increased risk of prostate cancer in men, question 6 had a PPV of 71% (29%–96%) and FPR of 1% (0.1%–5%).
Ovarian Cancer
In univariate analyses, questions 7, 10, and 14 were associated with increased risk of ovarian cancer, presented in order of strength of association, and were included in subsequent multivariate analyses. Questions 7 alone performed well, with sensitivity of 46% (28%–66%) and specificity of 98% (96%–99%), as did question 10, with sensitivity of 43% (24%–63%) and specificity of 90% (87%–93%). By combining questions 7 and 10 (through a yes response to either), we obtained a higher sensitivity of 79% (59%–92%) with little loss of specificity at 88% (84%–91%). In this population with a prevalence of increased risk of ovarian cancer of 6.9%, the combination of questions 7 and 10 had a PPV of 33% (22%–46%) and FPR of 2% (0.7%–4%).
Breast Cancer
In univariate analyses, questions 10, 9, 7, 14, and 1 were associated with increased risk of breast cancer, presented in order of strength of association, and were included in subsequent multivariate analyses. Table 4 shows the performance characteristics of individual and combinations of these questions. The combination of questions 7, 9, and 10 (yes response to any) was most significantly predictive. In this population with a prevalence of increased risk of breast cancer of 9.3%, these 3 questions combined had a PPV of 41% (31%–52%) and FPR of 1% (0.1%–2%).
Any Study Condition
We were interested in the accuracy of the FHQ to identify patients who were potentially at increased risk for any condition who, in practice, would thearefore require a more detailed assessment of their risk. Figure 2 presents the ROC curve for the combination of 9 questions (questions 2, 4, 6, 7, 8, 9, 10, 12, and 13) to detect increased risk of any condition. The area under curve was 84.6% (81.1%–88.1%), based on 241 out of 478 participants being at increased risk. Use of this set of questions provided 95% sensitivity (92%–98%) and 54% specificity (48%–60%) to identify a patient at increased risk of any condition. The combination had sensitivity of 92% (84%–99%) and 96% (93%–99%) for the 5 and 6 conditions applicable to men and women, respectively. The specificity was 63% (28%–52%) for men and 49% (42%–56%) for women. The PPVs were 67% (56%–78%) and 68% (63%–73%) and the FPRs were 9% (0.5%–17%) and 9% (3%–15%) for men and women, respectively. Figure 3 presents the final short-version FHQ containing the 9 questions.
DISCUSSION
This is the first study to validate an FHQ against a reference standard pedigree for use in Australian primary care. We have shown that a set of 9 simple questions can accurately screen for increased risk of 7 common, potentially preventable, serious conditions. This tool could be applied to identify populations in primary care who could benefit from more detailed assessment and discussion of their disease risk and its management.
This study took place in 6 general practices representing inner and outer metropolitan Perth with a range of socioeconomic backgrounds based on SEIFA (Socio-Economic Indexes for Areas) deciles of relative disadvantage.16 The study was conducted and is reported according to STARD (Standards for Reporting of Diagnostic Accuracy) criteria for studies of diagnostic accuracy. One of the strengths of our study was the ability to identify combinations of questions that perform better than single items to ascertain increased risk. No previous FHQ has been validated in this way.
The most important study limitation is the poor accrual rate of 5%. This is a common problem for screening studies in primary care where an initial letter of invitation from the practice is deemed the most ethically acceptable approach. It also reflects a commonly used approach in clinical practice to invite patients to attend for disease screening, whereby a reminder letter would not necessarily be considered routine. We were recruiting middle-aged adults who likely have competing work and family commitments and for whom invitation to attend for a detailed discussion of their family history may not have seemed relevant. Our sample was therefore biased toward well-educated white women in their 40s. A similar recruitment bias was observed in the US Family Healthware Impact Trial of electronic family history assessment in primary care.17 Our sample was probably more likely to have concerns about their family medical history and therefore chose to enter the study. It is possible that similar response biases would operate in clinical practice, so our sample may in fact be reasonably representative of patients who would complete an FHQ in general practice if recruited in the same way.
The prevalence of increased risk of any condition in our study population was 51.5%. This value should be compared with the Family Healthware Trial prevalence of increased risk of 82%, even though it covered a narrower range of cancers than our study.17 This difference may reflect a greater recruitment bias in their population or less stringent disease risk criteria. The effect of higher prevalence of increased risk in our sample, compared with the general population, would be to overestimate the PPV for items in our FHQ, but such overestimation would not alter the estimates of sensitivity, specificity, or area under the curve. We had too few participants of relevant ethnic backgrounds to measure the accuracy of the ethnicity-specific items in our FHQ. A separate validation study in the United Kingdom of our FHQ, but with a more restricted set of conditions, found very similar estimates of diagnostic accuracy, suggesting our findings are internally and externally valid.18
Previous systematic reviews of family history collection tools found that few have been developed specifically for primary care, and those that have tend to be disease specific, predominantly relating to cancer.11,19 No brief tool has been developed to cover a range of conditions in primary care that has been validated to the same extent as ours. Qureshi et al20 validated a longer, more complex FHQ in primary care and found reasonable agreement with a structured genetic interview to identify increased risk (κ = 0.52). The National Institutes of Health State-of-the-Science conference on family history and improving health identified the need to build further evidence on methods to systematically collect and use family history information in primary care.3 Our study provides important new evidence on such a tool. We specifically chose to include only conditions for which there was clear evidence for interventions aimed at those identified at increased risk. Of note, the traditionally used open-ended question about diseases running in the family (question 1 in our original FHQ) performed poorly as a screening question. We recognize that, in clinical practice, the family history is used to cover a much wider range of conditions than those covered by our FHQ, and includes exploring the psychosocial context of our patients. This use of the family history should continue in primary care alongside use of tools such as the FHQ to implement evidence about tailored disease prevention.
Two key trials have recently reported evidence about the clinical utility of family history risk assessment in primary care. The Family Healthware Trial found significant improvements in self-reported physical activity and fruit and vegetable intake after receipt of tailored preventive recommendations based on their family history.21 It failed to improve cancer screening behaviors, but this negative result was probably due to a ceiling effect given that only 5 participants were eligible for additional screening.22 The ADDFAM Trial (Added Value of Family History in CVD Risk Assessment) in the United Kingdom found an important benefit of systematically adding family history screening to cardiovascular risk assessment using a family history collection tool.23 This intervention led to a 41% relative increase in identification of people at increased cardiovascular disease risk, with associated significant improvements in smoking status. Applying the FHQ systematically in the context of a risk assessment clinical service in primary care was acceptable with a 98% completion rate. This finding suggests that response rates to an FHQ can be dramatically improved in the context of close integration with routine clinical services in primary care.
The family history remains the most useful genetic risk assessment tool for use in primary care. We have demonstrated the accuracy of a short, simple family history screening tool in primary care. In practice, patients who screen positive would require a further assessment of their risk, including obtaining a more detailed pedigree. This process would identify those who were accurately identified at increased risk, and enable the development of a tailored disease prevention management plan, incorporating additional disease screening, lifestyle advice, and, for some, referral to a clinical genetics service.
The challenge now is how best to implement the systematic application of the FHQ in clinical practice.24 Further work is required to understand the feasibility of its implementation into routine practice including, for example, electronic versions of the tool that can integrate with the electronic health record. More active engagement approaches are needed to increase the reach of the tool that do not rely simply on patients responding to a letter. Future pragmatic trials should apply the methods of implementation science, which evaluate relevance, feasibility, and impact on care. In this way, we can identify strategies to promote adoption of the FHQ into routine practice if it is proven to be a cost-effective intervention for tailored disease prevention.25
Acknowledgments
We are grateful to the participants and practices for their involvement in this research.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This research was funded by the National Health and Medical Research Council. A.T.P. was supported by the NIHR Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London, UK.
-
Disclaimer: The views expressed are those of the authors and not necessarily those of the UK NHS, the NIHR, NMHRC or the Department of Health.
-
Presentations: Some of the results reported in this article were presented at the ISONG Annual Scientific Meeting, International Congress of Human Genetics, Montreal, Quebec, Canada, October 2011.
- Received for publication March 17, 2013.
- Revision received August 27, 2013.
- Accepted for publication October 25, 2013.
- © 2014 Annals of Family Medicine, Inc.