Skip to main content
NCBI home page
Asian Pacific Journal of Cancer Prevention : APJCP logoLink to Asian Pacific Journal of Cancer Prevention : APJCP
letter
. 2019;20(7):1935–1937. doi: 10.31557/APJCP.2019.20.7.1935

Extraordinary Claims Don’t always Require Extraordinary Evidence, but They Do Require Good Quality Evidence

David Hawkes 1,2,3,4,*
PMCID: PMC6745220  PMID: 31350947

Dear Editor

It was with some confusion that, I read the recent article by Ried et al., (2017). The authors appear to be claiming that their methodology for detecting circulating tumour cells (CTC) is sensitive and also that the levels of CTC are highly predictive of risk of malignancy across a range of cancer diagnoses.

The entire methodology for this study is problematic. The diagnosis (diagnostic criteria) of cancer is poorly explained. It appears that cancer diagnosis for prostate cancer in men with extremely high PSA (330 – 1970 ng/ml, normal range <6.5 ng/ml) based on Ga68-PMSA-PET alone (patients M9, M12, M13 and others) without conformational biopsy and histological results. As this study was undertaken as a clinical trial it could be reasonably expected that claims of CTC being able to detect cancer would be based on a thorough, best practice based diagnostic process rather than a single imaging test which is known to give false positive results for benign foci (Keidar et al., 2018).

There is no description of blinding in this study. Cytology is not a quantitative process and as such is open to bias on behalf of the scientist (Branca and Longatto-Filho, 2015). It is also noted that no pathologists were involved in this study which relies heavily on cytology and cytopathology criteria of cancers.

The assay used does not appear to have been verified or used with any independent controls, there is no description of how the raw data was analysed, nor is there any statistical analysis.

There is no evidence on the specificity of this assay which is critical in light of the authors’ claim of 100% sensitive for cancer. Specificity for cancer detection is critical as a false diagnosis of cancer could cause physical, financial, and/or emotional harm (Hubbard et al., 2011). Additionally a false cancer diagnosis can actually be associated with increased risk of developing cancer (Henderson et al., 2015). To further explore the authors’ claim of 100% sensitivity it appears, from Table 1, that there was also a 100% concordance with cancer staging based on the CTC/ml values which is discordant with other studies (Krebs et al., 2011). Even within this manuscript there are examples of this type of discordance, with patient F1 having liver metastasis (which would classify this as Stage IV according to the authors) detected when her CTC count was only 3.5 per ml ( a low Stage II/III according to Table 1).

In addition to the diagnostic component of the study there is not even basic clinical information on the nutritional treatments used such as dose, mode of treatment (oral, i.v., etc), duration of treatment, or even basic safety monitoring such as liver function test results (e.g. Green tea extract is known to cause liver damage and even death (Seeff et al., 2013). This raises serious concerns as to how the safety of these treatments, which were given as part of a clinical trial, were monitored.

This study also lacks controls. There are no negative controls (e.g. no CTC detected at baseline), baseline results are never repeated prior to intervention, no placebo treatment in the asymptomatic but CTC positive cohort, and as such it is difficult to understand the predictive value of either the CTC result or the nutritional treatment.

In conclusion, this paper makes very bold claims such as “this suggests that CTC screening to be a more reliable measure for the detection of early prostate cancer than standard PSA.” Whilst it is certainly acknowledged that PSA measurement is not an ideal marker, to claim that CTC screening is better based on such a small number of samples for which very little evidence is presented on the validity of the prostate cancer diagnosis (no biopsy/histology evidence is presented for any prostate cancer case) is problematic, and possible ethically dubious as a negative result for CTC could drive a patient down pathway where they do not access evidence-based medical care for symptoms associated with cancer.

Funding Statement

There was no funding associated with this manuscript.

Acknowledgements

The author would like to acknowledge the input of all the cytologists and cytopathologists who provided their expert opinion.

References

  1. Branca M, Longatto-Filho A. Recommendations on quality control and quality assurance in cervical cytology. Acta Cytol. 2015;59:361–9. doi: 10.1159/000441515. [DOI] [PubMed] [Google Scholar]
  2. Henderson LM, Hubbard RA, Sprague BL, et al. Increased risk of developing breast cancer after a false-positive screening mammogram. Cancer Epidemiol Biomarkers Prev. 2015;24:1882–9. doi: 10.1158/1055-9965.EPI-15-0623. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Hubbard RA, Kerlikowske K, Flowers CI, et al. Cumulative probability of false-positive recall or biopsy recommendation after 10 years of screening mammography: a cohort study. Ann Intern Med. 2011;155:481–92. doi: 10.1059/0003-4819-155-8-201110180-00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Keidar Z, Gill R, Goshen E, et al. 68Ga-PSMA PET/CT in prostate cancer patients - patterns of disease, benign findings and pitfalls. Cancer imaging : the official publication of the Int Cancer Imaging Soc. 2018;18:39. doi: 10.1186/s40644-018-0175-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Krebs MG, Sloane R, Priest L, et al. Evaluation and prognostic significance of circulating tumor cells in patients with non-small-cell lung cancer. J Clin Oncol. 2011;29:1556–63. doi: 10.1200/JCO.2010.28.7045. [DOI] [PubMed] [Google Scholar]
  6. Ried K, Eng P, Sali A. Screening for circulating tumour cells allows early detection of cancer and monitoring of treatment effectiveness: An observational study. Asian Pac J Cancer Prev. 2017;18:2275–85. doi: 10.22034/APJCP.2017.18.8.2275. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Seeff L, Stickel F, Navarro VJ. Drug-Induced Liver Disease. (Third Edition) Academic Press; 2013. Chapter 35 - Hepatotoxicity of Herbals and Dietary Supplements; pp. 631–57. [Google Scholar]
Asian Pac J Cancer Prev. 2019;20(7):1935–1937.

Authors’ Reply

Karin Ried 1,*, Peter Eng 1, Avni Avni 1

Dear Editor

The author of the letter states his confusion about our study design and the quality of the methodology for CTC testing in our study.

Firstly, it is important to note that our article summarises the findings of an observational study. This was not a randomized controlled clinical trial or an experimental intervention study. Any diagnosis of cancer was made outside of our study by independent experts and by standard validated testing methods, such as scans and biopsies of tumour tissues. Any treatment of patients was administered and individualized by qualified integrative medical doctors, who followed standard good clinical practice and monitored tolerability and safety of any treatments.

Secondly, the ISET-CTC (Isolation by Size of Epithelial Tumour – Circulating Tumour Cell) test methodology was applied in this trial as a screening test, not as a diagnostic test.There is ample evidence that Circulating Tumour Cells (CTC) are associated with early carcinogenesis or malignant potential (Ilie et al., 2014).

CTC provide a biomarker for cancer prognosis and treatment effectiveness, by which an increase in CTC count is associated with cancer progression, and a decreased in CTC count is associated with cancer containment or remission (Cristofanilli et al., 2004; Hofman et al., 2011). Several technologies have been developed to identify CTC, including the Isolation by Size of Epithelial Tumour (ISET)-CTC testing, developed by Rarecells, France (www.rarecells.com), based on hematological and oncological principles, to apply filtration of blood to isolate CTC and analysis by standard cytology. The ISET-CTC test has been proven superior to other cell-surface marker-based CTC tests, and has been validated in more than 80 peer-reviewed articles over the last 20 years by several independent groups worldwide. Articles can be downloaded at: https://www.rarecells.com/oncology.

Comprehensive assays to assess sensitivity and specificity of the ISET-CTC test had been done by Rarecells and other research groups in the past 20 years, (Laget et al., 2017; Paterlini-Bréchot, 2014) and was outside the scope of our study.

In our study, we followed the established published ISET-CTC protocol for high quality CTC isolation and analysis. Cytological criteria used in ISET-CTC analysis are standard criteria used by cytologists and cytopathologists world-wide. Cytologists involved in our study are highly trained in Australia, and participate regularly in the Quality Assurance Programs of the Royal College of Pathologists Australasia (RCPA-QAP).

In our study, trained experienced cytologists undertook high quality cytological analyses of ISET-CTC test blood samples independent of and blinded to the clinical presentation of the patient. CTC count was verified by two independent cytologists / researchers, and any discrepancies were resolved by discussion. After cytological CTC analysis, results were matched with the clinical picture of the patient for reporting purposes.

All cancer patients (group 1) who participated in the study had been diagnosed with cancer with standard validated diagnostic methods prior to participating in the study and the ISET-CTC screening test. In the article, we compared cancer stage and CTC number in this group of cancer patients. The concept of number of CTC correlating to cancer status is not new, (Cristofanilli et al., 2004; Ilie et al., 2014) and was verified in our study.In a sub-group of asymptomatic patients (study group 2), CTC were detected with the ISET-CTC screening test. For cancer diagnosis, standard diagnostic tools were employed, including scans (PET, PSMA-PET, MRI), as well as biopsies, if required.It has been shown in a large study of more than 18,800 men, that routine screening for prostate cancer by PSA blood test and digital rectal examination failed to detect prostate cancer in a large proportion (85.5%) of screened men with normal test results (Thompson et al., 2004). Three-quarters (78%) of these men without abnormal test results had high grade (Gleason≥7) tumours, Thompson et al., (2004) highlighting the need for more sensitive tests.

In our study, we describe that the ISET-CTC screening test is a more sensitive alternative and non-invasive screening test for the early detection of prostate cancer, as follow-up of men with normal PSA levels and normal digital rectal examination, but positive CTC count showed an indication of early prostate cancer by standard diagnostic PSMA-PET scan. The PSMA-PET scan for prostate cancer is a highly sensitive test, which can detect tumours of 2.4 mm in size, and is a reasonable reliable indicator of underlying malignancy (Lapidus et al., 2000; Mottaghy et al., 2016). A proportion of the male participants with positive CTC count and a positive PSMA-PET scan also undertook prostate biopsies to confirm the diagnosis. Since publication of our article, further analysis of CTC cells of those men with a positive PSMA-PET scan by immunocytochemistry and prostate-specific markers confirmed CTC to be of prostate origin.

We have since presented this research at relevant cancer conferences. Please note that not all patients in group 2 (asymptomatic) had a positive CTC test result, therefore served as a negative control. Follow-up repeat tests were done on a selected group of patients, as outlined in Figure 1.Thirdly, in regards to the author’s concerns re safety of nutritional supplements suggested to patients with a positive CTC count, we would like to reiterate that our study was not a clinical trial, but an observational study of clinic patients.

Patients were in the care of highly qualified integrative medical doctors (PE, AS) with extensive experience in nutritional and herbal medicine, who were familiar with the benefits and safety of their use. Regular routine investigations such as liver function tests were carried out as part of standard good clinical practice. All integrative nutritional therapies listed in Table 4 were taken as oral supplements, as described in the text, and recommended by their treating doctor, who closely monitored tolerability and safety. Description of dosages for individual supplement was outside the scope of the article. In regards to the author’s concern specifically for green tea, we’d like to refer to the following review articles on the safety and risk of liver injury from green tea (Teschke et al., 2014; Hu et al., 2014). Both reviews concluded that green tea extract (GTE) in therapeutic doses can be considered highly tolerable and safe, if not hepato-protective. The maximum-tolerated dose in humans is reported to be 9.9 g/day of GTE, equivalent to 24 cups of green tea (Teschke et al., 2014). In addition, Teschke’s review13 of 88 cases concluded that a direct causality for liver injury by green tea consumption could not be drawn, as concomitant intake of medication or other herbs were potential confounding factors, as well as administration, dosing and type of green tea extract consumed. The review of 159 human intervention studies by Hu et al., (2018) reported no adverse events in 63% of the studies, minor adverse events such as nausea by a small proportion, and slightly elevated liver values could be expected in about 5% when a large bolus of GTE was administered at a fasting state (Yu et al., 2017).

Funding Statement

There was no funding associated with this manuscript.

Acknowledgements

The authors would like to acknowledge the input from the cytologists and medical specialists who provided their expert opinion.

References

  1. Cristofanilli M, Budd GT, Ellis MJ, et al. Circulating tumor cells, disease progression, and survival in metastatic breast cancer. N Engl J Med. 2004;351:781–91. doi: 10.1056/NEJMoa040766. [DOI] [PubMed] [Google Scholar]
  2. Hofman V, Ilie MI, Long E, et al. Detection of circulating tumor cells as a prognostic factor in patients undergoing radical surgery for non-small-cell lung carcinoma: comparison of the efficacy of the CellSearch Assay and the isolation by size of epithelial tumor cell method. Int J Cancer. 2011;129:1651–60. doi: 10.1002/ijc.25819. [DOI] [PubMed] [Google Scholar]
  3. Hofman V, Long E, Ilie M, et al. Morphological analysis of circulating tumour cells in patients undergoing surgery for non-small cell lung carcinoma using the isolation by size of epithelial tumour cell (ISET) method. Cytopathology. 2012;23:30–8. doi: 10.1111/j.1365-2303.2010.00835.x. [DOI] [PubMed] [Google Scholar]
  4. Hofman VJ, Ilie MI, Bonnetaud C, et al. Cytopathologic detection of circulating tumor cells using the isolation by size of epithelial tumor cell method: promises and pitfalls. Am J Clin Pathol. 2011;135:146–56. doi: 10.1309/AJCP9X8OZBEIQVVI. [DOI] [PubMed] [Google Scholar]
  5. Hu J, Webster D, Cao J, Shao A. The safety of green tea and green tea extract consumption in adults–results of a systematic review. Regul Toxicol Pharmacol. 2018;95:412–33. doi: 10.1016/j.yrtph.2018.03.019. [DOI] [PubMed] [Google Scholar]
  6. Ilie M, Hofman V, Long-Mira E, et al. “Sentinel” circulating tumor cells allow early diagnosis of lung cancer in patients with chronic obstructive pulmonary disease. PLoS One. 2014;9:e111597. doi: 10.1371/journal.pone.0111597. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Laget S, Broncy L, Hormigos K, et al. Technical insights into highly sensitive isolation and molecular characterization of fixed and live circulating tumor cells for early detection of tumor invasion. PLoS One. 2017;12:e0169427. doi: 10.1371/journal.pone.0169427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Lapidus RG, Tiffany CW, Isaacs JT, Slusher BS. Prostate-specific membrane antigen (PSMA) enzyme activity is elevated in prostate cancer cells. Prostate. 2000;45:350–4. doi: 10.1002/1097-0045(20001201)45:4<350::aid-pros10>3.0.co;2-u. [DOI] [PubMed] [Google Scholar]
  9. Mottaghy FM, Behrendt FF, Verburg FA. 68-Ga-PSMA-HBED-CC Pet/CT: where molecular imaging has an edge over morphological imaging. Eur J Nucl Med Mol Imaging. 2016;43:394–6. doi: 10.1007/s00259-015-3212-5. [DOI] [PubMed] [Google Scholar]
  10. Paterlini-Bréchot P. Circulating tumor cells: Who is the killer? Cancer Microenvironment. 2014;7:161–76. doi: 10.1007/s12307-014-0164-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Peer-reviewed articles using ISET-CTC technology. [ https://www.rarecells.com/oncology ]
  12. Teschke R, Zhang L, Melzer L, Schulze J, Eickhoff A. Green tea extract and the risk of drug-induced liver injury. Expert Opin Drug Metab Toxicol. 2014;10:1663–76. doi: 10.1517/17425255.2014.971011. [DOI] [PubMed] [Google Scholar]
  13. Thompson IM, Pauler DK, Goodman PJ, et al. Prevalence of prostate cancer among men with a prostate-specific antigen level≤ 40 ng per milliliter. N Eng J Med. 2004;350:2239–46. doi: 10.1056/NEJMoa031918. [DOI] [PubMed] [Google Scholar]
  14. Vona G, Sabile A, Louha M, et al. Isolation by size of epithelial tumor cells: a new method for the immunomorphological and molecular characterization of circulatingtumor cells. Am J Pathol. 2000;156:57–63. doi: 10.1016/S0002-9440(10)64706-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Yu Z, Samavat H, Dostal AM, et al. Effect of green tea supplements on liver enzyme elevation: results from a randomized intervention study in the United States. Cancer Prev Res. 2017;10:571–9. doi: 10.1158/1940-6207.CAPR-17-0160. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Asian Pacific Journal of Cancer Prevention : APJCP are provided here courtesy of West Asia Organization for Cancer Prevention

RESOURCES