Abstract
Background
Patients with lung or colorectal cancer often present late and have a poor prognosis. Identifying diagnostic indicators to optimally assess the risk of these cancers in primary care would support early identification and timely referral for patients at increased risk.
Aim
To obtain consensus regarding potential diagnostic indicators that are important for assessing the risk of lung or colorectal cancer in primary care consulters presenting with lung or abdominal symptoms.
Design and setting
A Delphi study was conducted with 28 participants from primary and secondary care and academic settings in the UK and Europe.
Method
Indicators were obtained from systematic reviews, recent primary studies and consultation with experts prior to the Delphi study being conducted. Over three rounds, participants rated each diagnostic indicator in terms of its importance, ranked them in order of importance, and rated each item as crucial or not crucial to assess during a GP consultation.
Results
The final round resulted in 25 items remaining for each type of cancer, including established cancer symptoms such as rectal bleeding for colorectal cancer and haemoptysis for lung cancer, but also less frequently used indicators such as patients’ concerns about cancer.
Conclusion
This study highlights the items clinicians feel would be most crucial to include in the clinical assessment of primary care patients, a number of which have rarely been noted in the previous literature. Their importance in assessing the risk of lung or colorectal cancer will be tested as part of a large prospective cohort study (CANDID).
Keywords: Delphi technique, neoplasms, primary health care, referral and consultation
INTRODUCTION
Lung and colorectal cancer are two of the most common cancers seen in UK primary care. Colorectal cancer is the second most commonly diagnosed cancer in the UK, with almost 40 000 new cases per year documented in 2009.1 Lung cancer deaths in England and Wales represent 22% of the total mortality from cancer.2 Patients in the UK have been reported to present later and do worse than in other countries,3 raising the issues of early identification and referral in primary care, prompt diagnosis, and effective treatment. However, most symptoms that may indicate cancer are common in primary care and are associated with a very low cancer risk. Referrals based on these symptoms can lead to unnecessary anxiety and investigations for patients at low risk, and unnecessary use of secondary care services.
One way of expressing a patient’s risk of cancer is to use a positive predictive value (PPV), which is the proportion of people with a particular diagnostic indicator who go on to develop cancer.4 There is a lack of evidence for the predictive values of lung or colorectal cancer symptoms, signs, and test results derived from high-quality, prospective primary care cohorts.5–7 In consulting populations the proportion with cancer is lower in primary care than in secondary care, and consequently the PPVs for this population are lower. Recent systematic reviews have identified few symptoms, signs, and test results in primary care that have a PPV of above 5%,4,8 meaning that many have a relatively low risk of being cancer-related.
Clinical prediction rules (CPRs), derived from prospective data collection and consisting of the combination of symptoms, signs, and test results most strongly associated with cancer risk, may be the most robust and reliable way to inform decisions regarding further investigations and onward referral. Existing CPRs for assessing cancer risk have often been developed in secondary care populations,9 meaning they may be less accurate in primary care, and others10,11 have been derived using only routinely collected data, which may not provide valid, standardised information in the way that a study designed specifically to collect information on diagnostic indicators would do. A large, prospective primary care-based cohort study (CANcer DIagnosis Decision rules, CANDID) has therefore been designed to derive prediction rules to support the early diagnosis of lung cancer and colorectal cancer in primary care. This Delphi study represents the first phase of this project, and aims to obtain consensus regarding potential diagnostic indicators that are important for assessing risk of lung and colorectal cancer in primary care consulters presenting with symptoms of possible oncological significance, with a particular emphasis on factors not already supported by evidence from previous research.
How this fits in
It is known that patients with lung or colorectal cancer often present late and have a poor prognosis. Developing clinical prediction rules for primary care may help ensure prompt and appropriate referral. This Delphi study has generated a list of diagnostic indicators that primary care clinicians feel are crucial to assess in a consultation with a patient presenting with symptoms that may be indicative of lung or colorectal cancer. These items will be tested as part of a large, prospective primary care-based cohort study, which will help identify the most important factors for clinicians to focus on when presented with a patient with lung or colorectal symptoms.
METHOD
A modified Delphi study12 was designed to achieve consensus regarding key diagnostic indicators in the identification of primary care patients at risk of lung or colorectal cancer. The Delphi is described as modified because participants were presented with a list of items to consider, rather than generating their own items,12 although there was an option to do this within the study. The Delphi process was anonymised, with panel members unaware of the identity of other members, and responses kept anonymous.
Identification of Delphi panel members
A panel of national and international cancer experts, researchers, and clinicians from primary and secondary care were invited to participate. Participants were identified through the West Midlands North primary care research network, as authors of relevant research publications identified from a scoping search, or via personal networks of the CANDID study team. Seventy-three potential participants were initially identified, with the aim to recruit approximately 20 panel members. There are no set guidelines for deciding on the optimum number of Delphi participants as this is likely to change depending on the purpose of the Delphi.13
Design of the Delphi study
All study materials were sent to participants via email. Participants were informed that their responses would be anonymised. The Delphi rounds were constructed using web-based survey software (Survey Monkey), which participants accessed via a weblink included in the email. The study was conducted between August and November 2012.
An overview of data collection throughout the study can be found in Figure 1. The first round of the Delphi contained the consent form, and participants could only access the survey if they consented to take part. Participants could also choose whether they wanted to answer questions relating to both types of cancer or only lung or colorectal cancer. Items to be included in the first round of the Delphi were identified from a scoping search of the literature to locate previously conducted systematic reviews and key primary studies that had identified factors predictive of lung or colorectal cancer in primary care (for example). 4,8,14,15 Other items were identified by members of the study team and through discussion with expert clinicians.
Figure 1.
Flowchart of how information was collected over the three Delphi rounds. CRC = colorectal cancer. LC = lung cancer. RF = risk factor. Sy = symptom. Si = sign. T = test.
Round 1 involved participants rating the identified diagnostic indicators on a 5-point Likert scale, ranging from ‘very unimportant’ to ‘very important’. The indicators were split into ‘key risk factors’, ‘symptoms’, ‘signs’ and ‘tests’ for lung and colorectal cancer. Participants could also add up to three additional indicators that they felt were missing from the current lists, and were asked to give a reason why they thought that item was important. Participants were also asked to provide information on their occupation (clinical or non-clinical) and the number of years of experience they had, to allow a description of the key characteristics of the panel. Any indicators which received a mean score of <3.0 were removed in the second round, but all new indicators were included. The cut-off point of 3.0 was set to include most items deemed important at this early stage.
Round 2 involved participants ranking each of the indicators in terms of how important they felt they were, with a score of 1 given to the most important indicator and the highest number to the least important indicator in each list. Summated scores generated a ranking within each list of key risk factors, symptoms, signs, and tests. The number of indicators to take forward to the final round was decided by the study team through discussions with a GP, who provided independent clinical input when deciding on the optimal cutoff for removing indicators from each section. To allow final decisions regarding items that should be maintained in the clinical assessment, participants in the final round were asked which of the remaining indicators were crucial or not crucial to assess in a 10-minute GP consultation with patients presenting with either lung or abdominal symptoms suggestive of cancer. Participants could reintroduce a limited number of crucial items that had been removed over the previous rounds, providing a reason for reintroduction of the item.
The rounds were sent out to participants every 4 weeks, with a reminder email being sent out 2 weeks after the initial email. All participants were sent an anonymised summary feedback report at the end of every round to inform them of study progress.
RESULTS
Characteristics of the Delphi panel
Of the 73 people originally invited to participate, 28 responded to round 1, 19 to round 2 and 19 to round 3. The participants described themselves mostly as GPs working in a clinical capacity (n = 25); two described themselves as non-clinical, and one as a specialist. Across the panel there was a mean of 22.7 years of experience (range 5–42 years).
Results of round 1
The results of each round are presented in Table 1 and Table 2 for colorectal cancer and lung cancer, respectively, which also show which items were removed and added. In round 1, few items were removed for either cancer as almost all items received a mean score of ≥3.0, and a number of items were added. Some of these added items were specific to the type of cancer, such as melaena as a sign for colorectal cancer and Horner’s syndrome as a sign for lung cancer. Others were more generic, including jaundice and patients’ concerns regarding their symptoms for either type of cancer.
Table 1.
Inclusion and exclusion of diagnostic indicators at each Delphi round for colorectal cancer
| Diagnostic indicator | Round 1 (mean score for importance) | Round 2 (mean rank score) | Round 3 (crucial) |
|---|---|---|---|
| Key risk factor | |||
| Familial polyposis colia | 4.92 | 2.44 | Y |
| First-degree relative with colorectal cancer <50 yearsa | 4.85 | 3.61 | Y |
| Agea | 4.54 | 3.17 | Y |
| More than 10 polypsa | 4.38 | 6.67 | Y |
| First-degree relative with polyps <50 yearsa | 4.27 | 5.94 | Y |
| Second attendance with the same symptoma | 4.23 | 9.28 | Y |
| Inflammatory bowel diseasea | 4.15 | 6.72 | Y |
| Benign polyps | 3.73 | 10.28 | Nb |
| Smoking status | 3.35 | 10.83 | Nb |
| First-degree relative with other type of cancer | 3.42 | 11.89b | |
| History of endometrial cancer | 3.23 | 12.56b | |
| Last consultation with a GP >6 months ago | 3.15 | 13.56b | |
| Alcohol intake | 3.00 | 13.78b | |
| Socioeconomic statusb | 2.92b | ||
| Ethnicityb | 2.88b | ||
| Diabetes mellitusb | 2.73b | ||
| Sexb | 2.65b | ||
| Occupational historyb | 2.46b | ||
| Progression of symptomsa,c | 5.33 | Y | |
| Patient thinks they have colorectal cancera,c | 10.94 | Y | |
| Last consultation with a GP >5 years agoc | 11.00 | ||
| Patient attends with an adult family memberc | 15.00 | ||
|
| |||
| Symptom | |||
| Rectal bleeding: blood mixed with stoola | 4.73 | 1.88 | Y |
| Bowel symptoms: change in bowel habita | 4.73 | 3.24 | Y |
| Unintentional loss of weight reported by patienta | 4.73 | 4.82 | Y |
| Rectal bleeding: typea,d | 4.50 | 4.06 | Y |
| Symptom durationa,e | 4.46 | 8.35 | Y |
| Bowel symptoms: tenesmusa | 4.08 | 8.76 | Y |
| Bowel symptoms: urgency | 3.88 | 9.71 | Nb |
| Bowel symptoms: incomplete emptyinga | 3.88 | 9.29 | Y |
| Bowel symptoms: diarrhoeaa | 3.81 | 8.24 | Y |
| Fatigue | 3.77 | 15.35b | |
| Loss of appetite | 3.73 | 13.35b | |
| Abdominal pain | 3.69 | 11.24b | |
| Bowel symptoms: nocturnal symptoms | 3.62 | 11.47b | |
| Bowel symptoms: constipation | 3.42 | 14.94b | |
| Distension | 3.42 | 14.88b | |
| Discomfort | 3.27 | 16.65b | |
| Peri-anal symptoms | 3.23 | 14.47b | |
| Bloating | 3.12 | 16.53b | |
| Nauseab | 2.88b | ||
| Jaundicea,c | 8.24 | Y | |
| Mucus in stoolc | 17.12b | ||
| Wet windc | 18.41b | ||
|
| |||
| Sign | |||
| Rectal massa | 4.96 | 1.88 | Y |
| Abdominal massa | 4.85 | 3.12 | Y |
| Cachexiaa | 4.81 | 4.53 | Y |
| Objective loss of weighta | 4.65 | 4.59 | Y |
| Rectal examination: blood on glovea | 4.27 | 5.29 | Y |
| Ascites | 4.54 | 6.00b | |
| Hepatomegaly | 4.31 | 6.12b | |
| Pale conjunctivae | 3.73 | 8.29b | |
| Lymphadenopathy | 3.73 | 7.76b | |
| Melaenac | 7.41b | ||
|
| |||
| Test | |||
| Iron deficiency anaemiaa | 4.77 | 1.12 | Y |
| Anaemia | 4.15 | 2.94 | Nb |
| Positive faecal occult blood testa | 3.96 | 3.53 | Y |
| Raised erythrocyte sedimentation rate | 3.73 | 4.18b | |
| Screened for colorectal cancer in the last 2 years | 3.54 | 4.76b | |
| High white cell countb | 2.54b | ||
| Raised glucose levelb | 2.35b | ||
| Disturbed liver function testsc | 5.06b | ||
| Hypercalcaemiac | 6.41b | ||
Included in round 3.
Excluded in round 2.
Added by participants in round 1.
Refers to characteristics of the rectal bleeding experienced (for example, dark versus bright red blood, whether the blood was noticed in the toilet pan or on the toilet paper.
Refers to whether the symptom was present for a long period of time as opposed to only a short period.
Table 2.
Inclusion and exclusion of diagnostic indicators at each Delphi round for lung cancer
| Diagnostic indicator | Round 1 (mean score for importance) | Round 2 (mean rank score) | Round 3 (crucial) |
|---|---|---|---|
| Key risk factor | |||
| Smoking statusa | 5.00 | 1.13 | Y |
| Occupational exposurea | 4.54 | 4.50 | Y |
| Agea | 4.33 | 3.31 | Y |
| Second attendance with the same symptoma | 4.25 | 8.25 | Y |
| History of chronic obstructive pulmonary diseasea | 4.13 | 6.81 | Y |
| Second-hand smoke exposurea | 4.04 | 7.00 | Y |
| Socioeconomic status | 3.71 | 7.19 | Nb |
| Family history (first-degree relative)a | 3.58 | 9.56 | Y |
| Last consultation with a GP >6 months ago | 3.46 | 12.88b | |
| History of idiopathic pulmonary fibrosis | 3.42 | 12.75b | |
| History of tuberculosis | 3.33 | 13.81b | |
| Sex | 3.25 | 11.19b | |
| Alcohol intake | 3.17 | 11.75b | |
| Ethnicityb | 2.79b | ||
| Last consultation with a GP >5 years agoc | 10.38 | Nb | |
| Patient thinks they may have lung cancera,c | 11.13 | Y | |
| Patient attends with an adult family memberc | 15.75b | ||
| Localisation of painc | 12.63b | ||
| Reduced exercise tolerancea,c | Y | ||
|
| |||
| Symptom | |||
| Haemoptysisa | 4.96 | 1.25 | Y |
| Unintentional loss of weight reported by patienta | 4.79 | 4.44 | Y |
| Symptom durationa,d | 4.25 | 6.19 | Y |
| Cougha | 4.17 | 3.00 | Y |
| Hoarsenessa | 4.13 | 6.06 | Y |
| Dyspnoeaa | 4.00 | 4.38 | Y |
| Chest paina | 3.79 | 6.69 | Y |
| Bone pain | 3.92 | 9.88b | |
| Loss of appetite | 3.88 | 7.88b | |
| Back pain | 3.88 | 10.56b | |
| Fatigue | 3.88 | 8.81b | |
| Shoulder pain | 3.75 | 8.88b | |
|
| |||
| Sign | |||
| Cachexiaa | 4.71 | 3.94 | Y |
| Pleural massa | 4.67 | ||
| Pleural effusiona | 4.67 | 3.69 | Y |
| Superior vena cava obstructiona | 4.58 | 3.31 | Y |
| Objective weight lossa | 4.54 | 4.5 | Y |
| Lymphadenopathya | 4.25 | 5.94 | Y |
| Stridora | 4.21 | 6.00 | Y |
| Finger clubbing | 4.00 | 7.31b | |
| Horner's syndromea,c | 6.50 | Y | |
|
| |||
| Test | |||
| Abnormal chest X-raya | 4.79 | 1.06 | Y |
| Abnormal sputum cytology | 4.13 | 3.75 | Nb |
| Anaemia | 3.71 | 3.69 | Nb |
| Raised erythrocyte sedimentation rate | 3.75 | 4.63b | |
| Raised C-reactive protein | 3.5 | 5.56b | |
| Abnormal spirometry | 3.33 | 6.06b | |
| Thrombocytosis | 3.25 | 7.31b | |
| Hypercalcaemiac | 6.81b | ||
| Disturbed liver function testsc | 6.13b | ||
Included in round 3.
Excluded in round 2.
Added by participants in round 1.
Refers to whether the symptom was present for a long period of time as opposed to only a short period.
Results of round 2
Based on the summated rank scores described in the Method section, abnormal chest X-ray, smoking status, and haemoptysis were ranked most highly for lung cancer and iron deficiency anaemia, blood mixed with stool and rectal mass ranked most highly for colorectal cancer. Based on the importance given to each item, the number of items in each section (risk factors, symptoms, signs, and tests) was reduced by approximately 50%, resulting in a total number of 29 potential diagnostic indicators for colorectal cancer and 28 for lung cancer (Tables 1 and 2). The rank scores given to the items were very similar across the items for the second round, meaning that it was difficult to judge where to place a cut-off based on the rank scores alone.
Results of round 3
Similar to round 1, few items were removed for either type of cancer in this final round, with nearly all indicators being regarded as crucial. Participants could reintroduce previously removed items, but as none of these were suggested for reinclusion more than once, no items were re-entered. The final items for lung cancer included nine risk factors, seven symptoms, eight signs, and one test, while the final list of items for colorectal cancer was made up of nine risk factors, nine symptoms, five signs, and two tests (Figure 1).
DISCUSSION
Summary
This Delphi study generated consensus among cancer experts and primary care clinicians regarding the diagnostic indicators that are most important to assess during a GP consultation to more reliably estimate a patient’s risk of lung or colorectal cancer. The study identified a list of 25 potential indicators each for lung cancer and colorectal cancer, which will provide a guide to the design of web-based data collection forms to be completed by GPs participating in the main CANDID study.
Strengths and limitations
As this study included a mostly primary care-based panel, the authors are confident that the results reflect what GPs perceive as important in everyday clinical practice. This is a strength as the CPRs will be designed and tested in a primary care-based study. Limited data collection on the characteristics of participants means that a full assessment concerning generalisability cannot be made. The consensus represented by the varied clinical experience and academic knowledge of the panel is supported by the literature and also includes novel items that may potentially lead to an improvement in the early identification of cancer.
Although the items were initially chosen from the published literature, further suggestions were made by the study team members and Delphi panel members, and the choice of cut-offs for each round was, although a priori defined, also based on subjective clinical judgement. The use of an independent GP to provide input regarding the cut points after round 2 had the advantage of weighing towards use in practice, although this may have introduced subjectivity. The final round therefore allowed reintroduction of crucial items and consensus regarding the final outcome.
Comparison with existing literature
Consensus was reached on the classic alarm symptoms by which lung and colorectal cancer present, such as haemoptysis and rectal bleeding, although the number of acknowledged symptoms highlights the vagueness and variety of clinical presentations. Some of the crucial symptoms that were eventually decided on were subtypes of classic symptoms (for example, change in bowel habit) into more specific types (for example, diarrhoea as opposed to constipation), and may reflect clinical experience concerning the incidence of the more generic symptoms in primary care populations and their low predictive value for cancer. These findings are consistent with results from other primary care-based studies.8,10 Patients who present with classic alarm symptoms may have a lower risk of mortality than those who do not,16 possibly because these symptoms are more easily recognisable as potential cancer symptoms. There may be some inconsistency with patients’ ideas concerning key symptoms, with one study of symptoms for colorectal cancer17 finding rectal bleeding, change in bowel habit, and weight loss being commonly reported symptoms, along with pain, fatigue, and ‘general indisposition’ or feeling of being unwell. Pain and fatigue have been reported as warning signs for cancer,18 but with very low PPVs.
Well-known cancer-specific risk factors were classified by the panel as crucial (for example, familial polyposis coli for colorectal cancer and chronic obstructive pulmonary disease for lung cancer). With more generic risk factors, some appeared to be judged as crucial for both cancers (for example, age), others were seen as crucial for one cancer but not the other (for example, smoking for lung cancer only) and others were not judged to be crucial for either type of cancer (for example, socioeconomic status). This is likely to be reflective of the make-up of the panel who were mostly GPs with extended clinical experience. A qualitative study of GPs19 found this experience to be a key factor in judging the possibility of cancer in a patient, and also highlighted the importance of interpersonal awareness and how subtle changes in the way a patient talks or behaves may be indicative of a more serious condition.
Symptom duration was present in the first round of the Delphi as an aid to judging cancer probability, although this may represent a high probability of cancer if the duration is short (6 weeks for example) or a low probability if it is long (years). It has been noted that time as a diagnostic tool should be used when the benefits of the delay outweigh any potential harm to the patient,20 and that it is useful when confronted with vague, common symptoms which characterise many presentations of possible cancer. The importance of ‘second attendance with the same symptom’ for lung and colorectal cancer and the introduction by participants of ‘progression of symptoms’ for colorectal cancer represents further use of time as a diagnostic tool within primary care. The absence of this item for lung cancer may reflect the more progressive course of this illness. An additional crucial factor introduced by participants was that of patients’ concerns regarding risk of cancer, representing a patient-centred approach in primary care. Fear of cancer has been found to be prevalent in presenting patients21 and GPs, through a fear of missing a diagnosis.20 These items have not been previously investigated in diagnostic studies, possibly because they are difficult to measure. The inclusion of these more subjective indicators could again be a result of the make-up of the panel.
Implications for research and practice
The findings of this Delphi consensus study will inform a large primary care-based prospective cohort study (CANDID) by highlighting the items that clinicians feel are most important to include in the clinical assessment of primary care consulters. This study has also helped to highlight a number of potential indicators that have rarely been noted in the literature. Their importance in assessing the risk of lung and colorectal cancers will be tested as part of the CANDID study.
Acknowledgments
The authors thank Dr Tope Roberts for her clinical input in each of the Delphi rounds; Dr Irena Zwierska for her advice and guidance on Ethical and R&D approval; the West Midlands North Primary Care Research Network for their help with recruiting potential panel members; the wider CANDID team and collaborators for their advice and input; and finally the panel members themselves for taking the time to complete the Delphi.
Funding
This paper presents independent research funded by the National Institute for Health Research (NIHR) School for Primary Care Research (Grant Reference no. 121c). The NIHR School for Primary Care Research is a partnership between the Universities of Bristol, Birmingham, Keele, Manchester, Nottingham, Oxford, Southampton and University College London. Role of the sponsor(s): The School for Primary Care Research is supported by the NIHR. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Ethical approval
Received from Keele University’s ethical review panel in March 2012.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: bjgp.org.uk/letters
REFERENCES
- 1.Office for National Statistics Colorectal Cancer England, 2009. Part of Bowel cancer in England, 2009 Release. http://www.ons.gov.uk/ons/rel/cancer-unit/bowel-cancer-in-england/2009/sum-colorectal.html (accessed 20 Jun 2014)
- 2.Office of National Statistics Statistical bulletin: Cancer incidence and mortality in the UK, 2007–2009. Part of Cancer incidence and mortality in the United Kingdom, 2007–2009 Release. http://www.ons.gov.uk/ons/rel/cancer-unit/cancer-incidence-and-mortality/2007-2009/stb-cancer-incidence-and-mortality.html (accessed 20 Jun 2014)
- 3.Hubbard R, Baldwin D. Diagnosing lung cancer earlier in the UK. Thorax. 2010;65(9):756–758. doi: 10.1136/thx.2010.145961. [DOI] [PubMed] [Google Scholar]
- 4.Shapley M, Mansell G, Jordan JL, Jordan KP. Positive predictive values of ≥5% in primary care for cancer: systematic review. Br J Gen Pract. 2010 doi: 10.3399/bjgp10X515412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hamilton W, Sharp D. Diagnosis of lung cancer in primary care: a structured review. Fam Pract. 2004;21(6):605–611. doi: 10.1093/fampra/cmh605. [DOI] [PubMed] [Google Scholar]
- 6.Jellema P, van der Windt DAWM, Bruinvels DJ, et al. Value of symptoms and additional diagnostic tests for colorectal cancer in primary care: systematic review and meta-analysis. BMJ. 2010;340:c1269. doi: 10.1136/bmj.c1269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Adelstein B-A, Macaskill P, Chan SF, et al. Most bowel cancer symptoms do not indicate colorectal cancer and polyps: a systematic review. BMC Gastroenterol. 2011;11:65. doi: 10.1186/1471-230X-11-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Astin M, Griffin T, Neal RD, et al. The diagnostic value of symptoms for colorectal cancer in primary care: a systematic review. Br J Gen Pract. 2011 doi: 10.3399/bjgp11X572427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Selvachandran SN, Hodder RJ, Ballal MS, et al. Prediction of colorectal cancer by a patient consultation questionnaire and scoring system: a prospective study. Lancet. 2007;360(9329):278–283. doi: 10.1016/s0140-6736(02)09549-1. [DOI] [PubMed] [Google Scholar]
- 10.Hamilton W. The CAPER studies: Five case-control studies aimed at identifying and quantifying the risk of cancer in symptomatic primary care patients. Br J Cancer. 2009;101(S2):S80–S86. doi: 10.1038/sj.bjc.6605396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hippisley-Cox J, Coupland C. Identifying patients with suspected colorectal cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract. 2012 doi: 10.3399/bjgp12X616346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.McKenna HP. The Delphi technique: a worthwhile approach for nursing? J Adv Nurs. 1994;19(6):1221–1225. doi: 10.1111/j.1365-2648.1994.tb01207.x. [DOI] [PubMed] [Google Scholar]
- 13.Steurer J. The Delphi method: an efficient procedure to generate knowledge. Skeletal Radiol. 2011;40(8):959–961. doi: 10.1007/s00256-011-1145-z. [DOI] [PubMed] [Google Scholar]
- 14.Heintze C, Matysiak-Klose D, Krohn T, et al. Diagnostic work-up of rectal bleeding in general practice. Br J Gen Pract. 2005;55(510):14–19. [PMC free article] [PubMed] [Google Scholar]
- 15.Hamilton W, Peters TJ, Round A, Sharp D. What are the clinical features of lung cancer before the diagnosis is made? A population based case-control study. Thorax. 2005;60(12):1059–1065. doi: 10.1136/thx.2005.045880. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dregan A, Møller H, Charlton J, Gulliford MC. Are alarm symptoms predictive of cancer survival? Population-based cohort study. Br J Gen Pract. 2013 doi: 10.3399/bjgp13X675197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pedersen AF, Hansen RP, Vedsted P. Patient delay in colorectal cancer patients: associations with rectal bleeding and thoughts about cancer. PLOS One. 2013;8(7):e69700. doi: 10.1371/journal.pone.0069700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Ingebrigsten SG, Scheel BI, Hart B, et al. Frequency of ‘warning signs of cancer’ in Norwegian general practice, with prospective recording of subsequent cancer. Fam Pract. 2013;30:153–160. doi: 10.1093/fampra/cms065. [DOI] [PubMed] [Google Scholar]
- 19.Johansen M-L, Holtedahl KA, Rudebeck CE. How does the thought of cancer arise in a general practice consultation? Interviews with GPs. Scand J Prim Health Care. 2012;30:135–140. doi: 10.3109/02813432.2012.688701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Almond SC, Summerton N. Test of time. BMJ. 2009;339:43–45. doi: 10.1136/bmj.b1878. [DOI] [PubMed] [Google Scholar]
- 21.Nylenna M. Fear of cancer among patients in general practice. Scand J Prim Health Care. 1984;2(1):24–26. doi: 10.3109/02813438409017697. [DOI] [PubMed] [Google Scholar]