The continuing problem of poor transparency of reporting and use of inappropriate methods for RT-qPCR

Abstract

Attendance at this year’s European Calcified Tissue Society’s (ECTS) Congress reveals that the methods used to obtain qPCR results continue to be significantly flawed and that and their reporting remain inadequate.

Applications for real-time quantitative PCR (qPCR)-based methods continue to increase across all areas of the life sciences and have become routine tools used to evaluate anything from the micro RNA content of exosomes to preparing cDNA libraries for strand-specific sequencing. An important application of reverse transcription (RT)-qPCR is the assessment of differential expression patterns characteristic of diseases and infection, as well as evaluating their prognostic usefulness and using them as an indicator of treatment efficacy. The most recent meeting of the European Calcified Tissue Society (ECTS) provided a snapshot of current practices in a medically important area of biomedical research typified by the need to evaluate RNA derived from difficult to obtain tissue and to associate gene expression signatures with a wide range of conditions that range from osteoporosis to impaired skeletal muscle function.

Unfortunately, it is clear that despite the publication of the MIQE guidelines eight years ago [1], the awareness of the need to report detailed and useful experimental protocols is woefully inadequate. A survey of participants revealed that whilst 72% and 68% respectively, of individuals carrying out RT-qPCR experiments thought the technique was simple and reliable, only 6% were aware of the guidelines (Table 1). Regrettably, this also applied to those describing themselves as “expert” users, with a disappointing 13% awareness. Most disheartening was that none of the novice users had heard of the existence of the guidelines.

Table 1.

Random participants at the ECTS meeting in Salzburg (May 2017) were asked whether they used RT-qPCR in their research and those that replied in the affirmative (n = 53) were asked additional questions.

	Overall	%	Novice	%	Competent	%	Expert	%
Total	53	100%	7	11%	38	49%	8	8%
"RT-qPCR is simple"	38	72%	6	86%	28	74%	4	50%
"RT-qPCR is reliable"	36	68%	6	86%	26	68%	4	50%

Test for	Overall	%	Novice	%	Competent	%	Expert	%
RNA integrity	11	21%	1	14%	8	21%	2	25%
RNA purity	2	4%	0	0%	1	3%	1	13%
PCR specificity	26	49%	0	0%	18	47%	8	100%
PCR efficiency	18	34%	0	0%	12	32%	6	75%
Awareness of MIQE guidelines	3	6%	0	0%	2	5%	1	13%

This was reflected in the additional answers provided, with RNA integrity and purity rarely assessed and PCR specificity and efficiency neglected by novice and competent users especially. These results are confirmed by a survey of fifteen recent publications in this field, which demonstrates quite clearly that there has been little improvement in the transparency of reporting of qPCR protocols since we published our first evaluation of around 2000 peer-reviewed papers [2] and is consistent with several surveys carried out since (Table 2).

Table 2.

Analysis of 15 publications selected at random from Pubmed searches using the terms “RT-PCR” and “musculoskeletal” or “osteoporosis” or “bone and hematopoiesis” or “calcified tissue”.

Reference	RNA integrity	RT replicates	RT conditions	PCR conditions	PCR efficiency	Analysis	No of RG	RG	RG validated
[7]	no	no	no	yes	no	ΔΔCq	1	β−Actin	no
[8]	no	no	no	no	no	not reported	1	GAPDH	no
[9]	no	no	no	no	no	not reported	not reported	not reported	no
[10]	no	no	no	no	no	ΔΔCq	1	GUS β	no
[11]	yes (mean RIN = 5.7; range, 2.4–8.4)	no	no	yes	no	ΔΔCq	1	ribosomal protein, large, P0	yes
[12]	no	no	partial	yes	no	ΔΔCq	1	GAPDH	no
[13]	no	no	no	no	no	ΔΔCq	1	GAPDH	no
[14]	no	no	no	no	no	ΔΔCq	1	HPRT	no
[15]	no	no	no	yes	no	ΔΔCq	1	GAPDH	no
[16]	no	no	no	no	no	ΔΔCq	1	GAPDH or B2M	no
[17]	no	no	no	no	no	geNorm	3	not reported	yes
[18]	no	no	partial	no	yes	not reported	3	β−Actin, GFAPDH, LDHA	yes
[19]	no	no	no	yes	no	ΔΔCq	1	β−Actin	no
[20]	yes (mean RIN = 7.9; range, 7.3–8.7)	no	no	yes	no	ΔΔCq	1	β−Actin	no
[21]	no	no	partial	no	no	ΔΔCq	1	YWHAZ	no

A surprising issue that continues to dog qPCR-based publications is that the published primer sequences are often wrong. For example, a recent publication looking at the impact of dendritic cell interactions with bone grafts used GAPDH as a reference gene. However, the published primer sequences for the 19 base pair forward and reverse primers have two mismatches each with the database reference sequence (XM_017321385.1) [3]. Furthermore, those primers also amplify a pseudogene (XM_001476707.5), making their use to quantify a single reference gene rather unconvincing. The fact that the amplicon has a secondary structure at the reverse primer binding site is also not ideal. In addition, primers targeting one of the main genes of interest amplify both it (bone gamma-carboxyglutamate protein, Bglap NM_007541.3) as well as two closely related targets (Bglap2 (NM_001032298.3 and Bglap3 NM_001305449.1)).

Most worryingly, qPCR data analysis continues to be confounded by the near universal use of single, unvalidated reference genes which are used to calculate ΔΔCq values despite no attempts having been made to calculate the efficiencies of the various qPCR assays. This is despite the clear directive in the original publication that in order to be valid, the amplification efficiencies of the target and reference genes must be approximately equal and detailed instructions on how to ensure that this is the case [4]. This would be less of an issue if the reported differences in mRNA abundance were huge, but they are typically in the region of 1.5–8-fold, suggesting that many of the results may be a result of technical noise. In a certain percentage of papers the results are meaningless, because not only are single, unvalidated reference genes used to report expression profiles, but published evidence suggests that the reference genes themselves are regulated in the conditions under investigation. For example, GAPDH is widely used as a reference gene in osteosarcomas, yet it is apparently upregulated at both RNA and protein levels compared with healthy controls [5].

It is obvious that this situation is not going to improve until journal editors, in particular, begin taking this egregious, I am tempted to say scandalous, situation seriously and start to appreciate first, that the majority of peer-reviewed publications utilising qPCR-based methods are seriously flawed due to inappropriate methods being used and second, that results are frequently not reproducible due to lack of relevant information supplied in the Materials and Methods section. It took 40 years from the first report of cell line contamination and misidentification [6] for statements about cell line validation to become submission prerequisites for most of the major journals. Some, including BDQ, require authors to submit detailed information with regards to their qPCR and digital PCR protocols, but most do not. Let us hope that it does not take another forty years before this situation is remedied.

Handled by Jim Huggett