Use of Pearson and Spearman correlation testing in Indian anesthesia journals: An audit

Abstract

Background and Aims:

Correct usage and interpretation of biostatistical tests is imperative. Aim of the present article was to evaluate the use of “correlation test” for biostatistical analysis in two leading Indian journals of anesthesia and sensitize the readers regarding its correct usage.

Material and Methods:

A prospective analysis was done for all original articles using the correlation test (Pearson or Spearman) that were published in “Indian Journal of Anaesthesia” (IJA) or “Journal of Anaesthesiology and Clinical Pharmacology” (JOACP) in the years 2019 and 2020.

Results:

Amongst all included original studies, correlation test were used in 6% (JOACP) and 6.5% (IJA) respectively (averaged for the years 2019 and 2020). Correlation test was usedinappropriately) for evaluating an aim of prediction/agreement/comparison, rather than association, in 25% and 10% instances each (JOACP and IJA). In both JOACP and IJA, there were high rates of using and interpreting results without citing 95% confidence intervals (CIs) of correlation coefficient (88% and 90%, respectively), P value for significance of the association (50% and 90%, respectively), or coefficient of discrimination (88% and 70%, respectively). In majority of the instances, test to ascertain presence of mandatory prerequisites such as normal distribution of data could not be found (62% and 90%, respectively).

Conclusion:

The complete potential of correlation test in exploring research questions is probably underappreciated. Further, even when used, its application and interpretation are prone to errors. We hope that the present analysis and narrative is a well-timed appropriate step in bridging the gaps in existing knowledge regarding use of correlation test in national anesthesia literature.

Introduction

Correct usage and interpretation of biostatistical tests is frequently emphasized in clinical research.[1] Thus, reviews on basic biostatistical tools are also to be found with ease.[2] An enhanced and correct understanding of biostatistics would help to not only improve the quality of clinical research, but also aid in analyzing published studies for better clinical decisions. The need to understand biostatistics has fast become greater than ever due to the advancements in statistical analyses consequent to computerization, and the clinical use of evidence-based medical practice that relies solely on research.

Additionally, the potential for research opportunities and intellectual stimulation have been cited as important reasons that promote the choice of anesthesia as a career.[3] Thus, enhancing skills and knowledge related to biostatistical tools would likely aid in satisfaction and associated growth of our specialty as a professional career. Coupled with the perceived need to expand our skills related to biostatistical tools, there is a continuously expanding horizon for the type of biostatistical tests being used in published literature. The tests now range from the most basic descriptive analysis to advanced and complex procedures.[1,4,5] “Correlation” testing has been classified as one of the basic/elementary tests of biostatistical analysis. Although there could be several types of correlation tests, the Pearson and Spearman correlation tests are one of the commonest ones.

The aim of the present article is to evaluate the usage and interpretation of the Pearson and Spearman correlation tests for biostatistical analysis in two leading Indian journals of anesthesia, linked with a simultaneous discussion to increase awareness regarding their correct application.

Material and Methods

The first part of this study aimed at evaluating usage and interpretation of correlation tests in two leading Indian journals of anesthesia specialty, “Indian Journal of Anaesthesia” (IJA) and “Journal of Anaesthesiology and Clinical Pharmacology” (JOACP). We chose JOACP and IJA since both journals are published in India, indexed with PubMed, and remain popular among Indian researchers as credible national journals for topics pertaining to anesthesiology and related fields. Further, both journals are official publications of national societies related to the subject; while JOACP is the official publication of the Research Society of Anaesthesiology and Clinical Pharmacology (RSACP), IJA is for the Indian Society of Anaesthesiology. The societies hold regular annual conferences and have official websites as well. In a nutshell, both journals have a well-established credibility record and would be fair reflections of research being conducted in the specialty in India.

All original research articles published in JOACP and IJA in the years 2019 and 2020 were identified by the “type of publication” indicated in the table of contents. This did not include editorials, commentaries, reviews, meta-analysis, case reports, letters to editor, and others. Each of the original articles published in both journals (2019 and 2020) was then assessed to identify those that used correlation testing. Those mentioning correlation as an aim, or in statistical tests section, or presenting correlation coefficient as a result in the text/tables/figures were then scrutinized for detailed parameters of correlation testing.

All original articles were scrutinized by two independent researchers. In case of any differences in the interpretation, a third dedicated researcher resolved and took the final decision for data entry. All researchers first attended an online tutorial on “correlation testing” taken by one of the senior researchers along with guidance by a biostatistician. The senior researcher has over 100 publications alongwith personal experience in conducting correlation testing.

Observations

The detailed parameters related to correlation testing that were individually noted, so as to determine its usage pattern/interpretation included those related to (1) correct choice of test: study design, whether the aim merited a correlation test or was a prediction/causative/agreement theory misused with the test, choice between the types of correlation test used (Pearson or Spearman correlation test); (2) use and interpretation of the statistical output besides the “correlation coefficient (r)” obtained in a correlation test, that is, 95% confidence interval (CI) of the correlation coefficient (r), coefficient of determination (R²), P value, as well as the scatter plot; and (3) ensuring that the essential prerequisites for applying the correlation test were checked for: normal distribution of data, absence of outliers, and lack of multiple/repeated observations. Additionally, the percentage of original articles using correlation testing was also calculated. The article uses results obtained from the present audit to link with a simultaneous discussion regarding the appropriate choice, usage/interpretation, and essential prerequisites of correlation testing (Pearson and Spearman).

Results

A total of 13 original articles from IJA, and nine from JOACP published over the 2-year period mentioned correlation as an aim or cited its usage in the section of statistical analysis. From among these, however, 3/13 and 1/9, respectively, did not eventually show application of Pearson or Spearman correlation test in the results section. After elimination of these four articles, a total of 18 remaining ones were analyzed for the audit. The details of parameters related to use or interpretation of correlation test, and its essential prerequisites were assessed in all these 18 articles [Table 1], both individually for the years 2019 and 2020, as well as averaged for the years. Additionally, a total figure for both journals together is also presented [Table 1].

Table 1.

Details of parameters used for correlation testing

	JOACP (2019) (n=4)	JOACP (2020) (n=4)	JOACP* (n=8)	IJA (2019) (n=8)	IJA (2020) (n=2)	IJA* (n=10)	Total# (n=18)
Type of study design	Cross sectional (1), prospective (3)	Prospective (3), retrospective (1)	Prospective (6), retrospective (1), cross sectional (1)	Retrospective (4), prospective (4)	Prospective (2)	Prospective (6), retrospective (4)
Did the aim merit correlation test?	4 (100)	2 (50)	6 (75)	7 (88)	2 (100)	9 (90)	15 (83)
Pearson: Spearman: not specified	3:1:0 (75:25:0)	1:2:1 (25:50:25)	4:3:1 (50:38:12)	2:4:2 (25:50:25)	2:0:0 (100:0:0)	4:4:2 (40:40:20)	8:7:3 (44:39:17)
95% CI of the correlation coefficient	0 (0)	1 (25)	1 (12)	1 (12)	0 (0)	1 (10)	2 (11)
Coefficient of determination	0 (0)	1 (25)	1 (12)	2 (25)	1 (50)	3 (30)	4 (22)
P	2 (50)	2 (50)	4 (50)	1 (12)	0 (0)	1 (10)	5 (28)
Scatter plots	1 (25)	2 (50)	3 (38)	4 (50)	1 (50)	5 (50)	8 (44)
Normal distribution of data analyzed	1 (25)	2 (50)	3 (38)	1 (12)	0 (0)	1 (10)	4 (22)
Outliers mentioned	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Multiple observations used	2 (50)	1 (25)	3 (38)	1 (12)	0 (0)	1 (10)	4 (22)

CI=confidence interval, IJA=Indian Journal of Anaesthesia, JOACP=Journal of Anaesthesiology and Clinical Pharmacology. Values are numbers (percentage). ^*Average of years 2019 and 2020. ^#Total for the two journals over the 2-year period

The averaged figures for the 2 years were taken as representative of the individual journal and showed the following results.

The type of study design using correlation test included prospective, retrospective, or cross sectional in both journals [Table 1].

The correlation test was correctly used with the aim of evaluating an association or relationship between quantitative variables in 75% and 90% articles in JOACP and IJA, respectively, with an overall tally of 83%. The 25% articles with inappropriate use of correlation testing in JOACP used it for prediction or agreement (n = 1 each) and in IJA for a comparison between two groups (10%; n = 1). The Pearson correlation test, compared to the Spearman correlation test, was used in slightly greater percentage of studies in JOACP (50% vs. 38%) and equally in IJA (40% each), with an overall tally of 44% versus 39%. Even though the indications of using Pearson and Spearman tests are distinct, in 17% studies, it was not specified which of the tests were used. The 95% CI of the correlation coefficient was mentioned in 12% and 10% of the correlation studies in JOACP and IJA, respectively (11% overall). The coefficient of discrimination was presented in 12% and 30% of the correlation studies in JOACP and IJA, respectively (22% overall). The P value for testing of correlation coefficient was mentioned in 50% and 10% of the studies in JOACP and IJA, respectively (28% overall). Scatter plots were used to depict the relationship obtained in correlation testing for 38% articles in JOACP and a slightly higher number in IJA (50%) (44% overall) [Table 1]. The normal distribution of data was assessed, and results depicted the same in 38% and 10% of the studies for JOACP and IJA, respectively. None of the studies in either journal mentioned about outliers in the results section. The correlation test was applied on parameters obtained by multiple observations in 38% and 10% of the studies in JOACP and IJA, respectively. Total numbers of original articles published per year in JOACP were 65 (2019) and 60 (2020) and in IJA were 80 (2019) and 99 (2020). Out of these, the papers which tested for correlation were four each in 2019 (6%) and 2020 (7%) in JOACP and eight (10%) and two (2%) in 2019 and 2020, respectively, in IJA.

Discussion

We could identify only limited earlier attempts among Indian researchers to analyze the current usage patterns of various biostatistical testing in published literature, with slightly greater focus on the issue from developed parts of the world.[6,7,8]

An evaluation of usage of the correlation test (Spearman and Pearson) during the current audit showed interesting patterns. The use of correlation test was rather limited in both JOACP and IJA (6% and 6.5%, respectively), even though correlation test is considered a “basic elementary test.”[9] The various categories of biostatistical methods as classified by Windish et al.[9] include the descriptive, elementary, multivariable, advanced regression analysis, and “other” more advanced ones such as those related to machine learning.

We evaluated whether the test was correctly chosen for the analysis in terms of the study design to which it was applied, whether the research aim was evaluation of an “association” between variables, and lastly if the reason for using Pearson versus Spearman correlation test was explained. In majority of the studies in both journals, the correlation test was correctly chosen to assess an association/relationship between variables (75% and 90% for JOACP and IJA, respectively). Both association and relation are terms frequently used as synonyms, and their presence implies a concurrence of two variables more often than explained by chance.[10,11] However, the remaining 17% studies, in both journals considered together, used correlation testing inappropriately by using it for finding out cause, predictive value, or agreement between variables. Causation (and hence, prediction in clinical practice) remains distinctly different from association (or the analogous terms including relation). While cause is a definite association, an association may not be sufficient to attribute a “causal” relationship. Indeed, in biostatistical reviews, it is often noted that one of the common misinterpretations during correlation testing is to infer the association or relationship as a cause, as witnessed in our audit also.[12] The worry is that since causal studies such as randomized controlled trials are more labor intensive, phrases such as association/relation/risk factors may be interpreted as proof of prediction or causation for clinical application.

With regards to the choice made between using Pearson or Spearman correlation test, 17% of all included studies failed to specify as to which of the two was being used and why. This is despite a clear distinction between the indication for using the two tests. The Spearman test is applied when the relationship between the two variables is not linear, i.e., there is absence of bivariate normal distribution,[13] or when data is collected on a scale that is not truly interval in nature (e.g., data obtained from Likert scale administration). Unlike the Pearson correlation coefficient, the Spearman coefficient is calculated using the ranks rather than actual values themselves.

The parameters we analyzed to assess whether appropriate interpretation of the statistical output was done included use of 95% CI of r, R², the P value, as well as the scatter plot. While awareness regarding correlation coefficient (r) was evident from its presentation in all articles, the rest appeared sort of neglected. A total of 89% of the included studies did not mention the 95% CI of r, 78% failed to present R², and 72% did not mention the P value. Further, the scatter plots which represent the graphical output of the test to elaborate upon the results were not depicted in 54% of the included studies. The interpretation of the correlation test may be incorrect unless a holistic view of the entire statistical output parameters is taken. To explain the same, we will now take up a working example.

Let us assume that the clinical question being explored is “does there exist an association between variables A and B,” and the statistical output after correlation test produces the following parameters (with their values): r = correlation coefficient (with its 95% CI) = 0.451 (95% CI: 0.260–0.609); R² = coefficient of discrimination = 0.203; and P value: <0.001. The correlation coefficient (r) helps in judging not only the strength, but also direction of the association, and can vary between −1 and +1. A value of 0 implies lack of any association and 1 indicates a perfectly linear relationship. Intermediate values are interpreted to suggest weak, moderate, or high degree of association/correlation.[10] The precise cut-off values to define such a grading of magnitude of correlation are arbitrary and hence inconsistent,[12] but values <0.1 and >0.9 represent negligible and very strong associations, respectively, while >0.65 may be used to represent a good association. Thus, our result of r = 0.451 implies an association that is present, but neither negligible nor very strong and is only moderate at best, failing to even reach a good association level. The presence of an association stills needs to be further defined by directionality. An increase in one variable could result in rise or fall of the other variable. A positive coefficient (i.e., +x) shows simultaneous increase or decrease in both variables, while a negative one (i.e., −x) implies opposite directions of change in the variables. Our working example with a positive r = +0.451 implies that an increase in variable A would result in a rise in variable B and the reverse (a simultaneous decrease in both).

It is as important to observe the 95% CI of r, since this helps to infer whether the calculated coefficient is likely to also apply to the population from where the sample was drawn (and therefore reflect validity of the results).[14] A 95% CI range implies that if the experiment would be repeated 100 times, the results would fall within this range 95/100 times. Thus, a narrow interval is meaningful and desirable for a consistent and strong association, with greater validation of the results. Our working example shows the 95% CI to be = 0.260–0.609, implying that the r would lie between 0.260 and 0.609 in 95% of repetitions of the experiment, a relatively narrow and desirable CI. Despite a strong association with high values of r (>0.6 at least), if the 95% CI is a very wide range, the results may be inconsistent and difficult to validate. The P value in correlation test helps to determine whether the association is “by chance.” It tests the null hypothesis that the correlation coefficient is 0 or, in other words, that there is no association between the variables being studied. If the P value is calculated to be < 0.05, that is, the value is statistically significant, it implies a rejection of null hypothesis (the correlation coefficient is not 0, and there, indeed, exists an association between the variables). Our working example demonstrates a highly significant association between the variables A and B with a P value of < 0.001. This is despite the fact that the strength of association was only moderate (r = 0.451). Indeed, a relatively small correlation coefficient, as with our example, could become statistically significant in a large sample size, even though this does not indicate clinical significance. This should always be kept in mind while interpreting the correlation results.

Lastly, the role of R² (coefficient of determination) in correlation test should not be undermined or confused with the r itself. It is the squared value of correlation coefficient (r), and its interpretation is slightly different. R² shows the proportion of variance in one variable accounted or explained by the other associated variable. The relationship seen in the working example used by us between variable A and variable B has an r = 0.451 and its squared value (R²) = 0.203. This R² of 0.203 means that 20.3% variability in variable A is explainable by its relationship or association with variable B and the remaining 79.7% variability is due to other factors. Another distinction between r and R² is that since the latter is a squared value, it would always be positive and hence cannot guide regarding direction of the association.

The graphical output for correlation test is the scatter plot. Figure 1 is a scatter plot showing the results of correlation test between the hypothesized variable A and variable B in our working example. A scatter plot has two axes (X and Y) and shows several data points, each having a distinct X and Y coordinate [Figure 1]. The collection of all the data points plotted can be interpreted as a trend using the “line of best fit,” a line drawn through the center of the data points. A line of best fit that rises uphill as traced from left to right along the X axis depicts a positive relationship [Figure 1], while a downhill slope suggests a negative relationship [Figure 2]. Lack of any such trend shows absence of association between the two variables. In such cases, a line of best fit can be horizontal, vertical, or absent. A visual estimation of the strength of correlation is also possible. If data points are snugly fitted with each other along the line of best fit, a stronger association with higher coefficient is suggested. Loosely distributed data points suggest a weak association: Figure 2 shows weaker association than Figure 1.

Figure 1.

Scatter plot reflecting a positive correlation coefficient

Figure 2.

Scatter plot reflecting a negative correlation coefficient

After understanding the utility and relevance of each of the statistical output parameters including the graphical analog, the lacunae in appropriate usage/interpretation of correlation tests evidenced by the current audit can be better appreciated.

Lastly, we also tried to audit whether the included studies had ascertained presence of essential prerequisites for applying correlation test. These essential features for correlation testing includes the following: both variables be continuous (quantitative), jointly normally distributed (when choosing Pearson correlation), outliers should be absent (for Pearson test), and each observed x–y pair be measured independently of the other (i.e., no multiple observations from the same subject). In the present audit, while the correlation test was correctly applied to quantitative data in all studies, there are several instances to suggest an inappropriate application: tests to ascertain normal distribution were conducted in only 22% of instances, the clarification for using Spearman rather than Pearson test was not offered in any, there was no mention of presence or absence of outliers, and the test was applied to multiple observations in 22% of included studies.

The relevance of these prerequisites is demonstrated through Figure 3. One of the essential prerequisites for applying Pearson test is that the variables are jointly normally distributed and the association between them is linear. Data distribution on a scatter plot makes for easy understanding of linear or nonlinear association [Figure 3]. A linear association is a specific type of monotonic relationship. Monotonic association implies an increase in magnitude of one variable will be accompanied by an increase (positive correlation) or decrease (negative correlation) in the other. Thus, Figure 3a and b depicts linear associations between variables A and B.

Figure 3.

Correlation coefficients under different associations: (a) linear perfect positive correlation (r = +1); (b) correlation coefficients with an outlier; (c) correlation between two normally distributed variables; (d) a quadratic correlation with r nearly equal to zero; (e) the relation is neither linear nor monotonic; (f) an exponential association

A closer look at Figure 3b also explains why the presence of outliers can affect the results of the Pearson coefficient falsely. In Figure 3b, an outlier data set is visible as an entry relatively separated from rest of the collection (the right-most data point). After ensuring that such a skewed data point is not due to a typographical error while entering the data, possible ways to deal with it include using Spearman rather than Pearson test, or using transformation of data, and analyzing the effect on the results when it is deleted. The variation in Pearson and Spearman coefficients can be seen when applied to data set with an outlier [Figure 3b]. While the Pearson coefficient was 0.753, applying the more robust Spearman test for nonlinear association resulted in a value of 1.0.

Similarly, Figure 3d–f demonstrates that when there is an association between the two variables that is nonlinear in nature, the coefficients may be misleading and fail to reveal the nature of the relationship. Hence, applicability of Pearson correlation to linear association is an essential requirement, and scatter plots are an excellent way to show the linearity or its absence.

We feel that the instances of inappropriate use of Pearson/Spearman correlation testing seen in the present audit are the result of confusions or lack of awareness regarding its indication, interpretation of related statistical output parameters, as well as the essential prerequisites that must be met by the data before being subjected to correlation testing. This is the lacuna that our present audit, amalgamated with presentation of conceptual knowledge regarding correlation testing per se, aims to overcome. Our results need to be interpreted within the domain of methodology adopted by us. The present article has dealt only with the correlation between quantitative variables using the Pearson or Spearman correlation tests. Other tests for evaluating relationship, such as regression, are not included in the current study. Tests of correlation and regression are not mutually replaceable, since both have specific purpose and individual assumptions that should be met.[15] Also, for the association between categorical variables, it is the Chi-square test that needs to be applied and interpreted. Further, there are other types of correlation tests besides the Pearson and Spearman that are not the focus of our article. These tests could be the focus of future audits to better understand and use the same in anesthesia research.

A limitation of the present audit is its application to only two journals of a single specialty. This may create a selection bias. However, we feel this audit is only a preliminary step in the right direction and could generate enthusiasm to evaluate wider Indian databases/journals and thus build consensus at a national level for education policy building. It is also possible that the articles that may have been rejected and not published in the audited issues fared differently regarding application of the correlation test. This could be interpreted as a publication bias that may alter the results regarding actual knowledge/awareness of correlation testing among researchers. Based on our current findings from studies in leading national anesthesia journals, it appears that despite being an elementary biostatistical test and the search for associations being common in clinical practice, use of Pearson and Spearman correlation test is limited and often inappropriate. The complete potential of such correlation testing in exploring research questions is probably underappreciated.

We hope that the present analysis and narrative is a well-timed appropriate step in bridging the gaps in existing knowledge regarding use of correlation test in anesthesia literature.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.