How to Clearly and Accurately Report Odds Ratio and Hazard Ratio in Diagnostic Research Studies?

Take-home points

• • The odds ratio (OR) shows the association between test results and the presence or absence of a disease in diagnostic research studies and can be obtained from both case-control and diagnostic cohort designs.

• • The OR approximates the relative risk (RR), which cannot be obtained from the case-control design, when the disease prevalence in the population of interest is very low.

• • The hazard ratio (HR) shows the association between test results and the events occurring over time and contextualizes the RR of the events in a time-to-event analysis.

• • Reporting OR and HR should include a clear definition of events, a specification of the reference category for categorical variables, and a description of the one-unit amount for continuous variables.

The odds ratio (OR) is a statistic commonly used to show the strength of association between test results (such as imaging findings) and the presence or absence of a disease in diagnostic research studies. Similarly, the hazard ratio (HR) is frequently used in diagnostic research studies to show the association between test results and events occurring over time [1]. Although these statistics are widely used in radiology research, it is not rare to see them described or interpreted unclearly or imprecisely in diagnostic research manuscripts, particularly at the peer review stage before publication. Therefore, this article aims to promote a clearer and more accurate reporting of OR and HR in diagnostic research studies.

How to Calculate and Report OR

OR is the ratio between the odds values. Odds are defined as the probability that the event will occur divided by the probability that the event will not occur as follows:

$$\text{Odds}=\frac{\text{Probability}}{\left(1-\text{probability}\right)}$$

Therefore, odds and probability are different ways of expressing the same concept—how likely it is that an event will occur?

The OR can be obtained from both case-control and diagnostic cohort (or more generally referred to as cross-sectional) designs (Fig. 1). The diagnostic cohort design selects all participants together from the population of interest, whereas the case-control design selects participants based on the column variable (i.e., D+ vs. D-) in the diagnostic cross-table, as shown in Figure 1 [2,3]. Taking the diagnosis of lung cancer (D+ vs. D-) according to the findings on chest computed tomography (CT) (F+ vs. F-) as an example, the diagnostic cohort design defines the population of interest by establishing certain eligibility criteria (for example, adults aged 55 years and older with X pack-year smoking history) and recruits all patients who meet the criteria. Some of the recruited patients may have lung cancer according to the natural prevalence, and patients will have F+ and F- on chest CT according to the natural distribution of the CT findings. Therefore, all numbers (A, B, C, and D) in the diagnostic cross-table shown in Figure 1 are naturally determined. In contrast, the case-control design separately collects a certain number of participants with (case) and without (control) lung cancer, and the case-to-control ratio is arbitrarily chosen by the researchers (determined by M and N in Fig. 1). In retrospective diagnostic research studies using convenience sampling, even though the study samples are not precisely collected in a case-control manner, they may be similar to case-control samples because of the various participant selection steps involved [4].

Fig. 1. Diagnostic cross-tables showing the case-control and diagnostic cohort designs.

D+ = patients with the disease, D- = patients without the disease, F+ = patients with a finding in the test, F- = patients without a finding in the test

In the cross-tables shown in Figure 1, the investigators want to know whether and to what degree the disease is more common in patients with F+ (i.e., whether F+ would indicate a diagnosis of the disease), for which the tabulated data need to be analyzed in the horizontal direction. Odds and OR can be calculated from the diagnostic cohort design as follows:

$$\text{Odds of D}+\mathrm{\hspace{1em}}\text{in F}+\mathrm{\hspace{1em}}\left(\text{from the diagnostic cohort design}\right)\mathrm{\hspace{1em}}=\mathrm{\hspace{1em}}\frac{A/(A+B)}{1-A/(A+B)}\hspace{1em}=\hspace{1em}A/B$$

$$\text{Odds of D}+\mathrm{\hspace{1em}}\text{in F}-\mathrm{\hspace{1em}}\left(\text{from the diagnostic cohort design}\right)\mathrm{\hspace{1em}}=\mathrm{\hspace{1em}}\frac{C/(C+D)}{1-C/(C+D)}\hspace{1em}=\hspace{1em}C/D$$

$$\text{OR}\mathrm{\hspace{1em}}\left(\text{from the diagnostic cohort design}\right)=\mathrm{\hspace{1em}}\frac{\text{Odds of D}+\mathrm{\hspace{1em}}\text{in F}+}{\text{Odds of D}+\mathrm{\hspace{1em}}\text{in F}-}=\hspace{1em}\frac{A/B}{C/D}$$

However, in the case-control design, the corresponding odds values are not meaningful because it selects participants based on the column variable (D+ vs. D-), and the odds values depend partly on the arbitrary ratio of D+ and D- participants investigators chose to include in the study. Nevertheless, the OR is still valid in a case-control design, even though the individual odds values are not valid, as demonstrated by the calculations below.

$$\text{OR}\mathrm{\hspace{1em}}\left(\text{from the case-control design}\right)=\mathrm{\hspace{1em}}\frac{\text{Odds of D+ in F+}}{\text{Odds of D+ in F-}}\hspace{1em}=\hspace{1em}\frac{a/b}{c/d}\hspace{1em}=\hspace{1em}\frac{AM/BN}{CM/DN}\hspace{1em}=\hspace{1em}\frac{A/B}{C/D}$$

As M and N are dropped out of the calculations, the OR calculated using the case-control design is the same as that obtained using the diagnostic cohort design. Furthermore, the OR can be calculated even if we do not know M and N.

Some items should be described transparently when reporting the OR in a research manuscript. First, a clear mention of how the odds value was defined (i.e., the odds of what) is needed to calculate the OR. In the above example, the odds of D+ was used. Second, the reference category for OR calculation must be specified for categorical variables. In the example above, the reference category is F-. If these designations are switched, that is, by using either the odds of D- instead of D+ or setting the reference category as F+ instead of F-, the OR will have a reciprocal value. Therefore, describing these items clearly will prevent confusion. Additionally, as the OR can also be calculated for a continuous variable, the one-unit amount for OR should be specified if the OR is reported for a continuous variable. For example, Yang et al. [5] reported an OR value of 2.04 for patient age in predicting malignancy in patients with hemophagocytic lymphohistiocytosis. Their results specifically mention that the OR was for every increase of 10 years. Without this explanation, readers might mistake it as an OR of 2.04 for every 1-year increase, which would erroneously make the OR for a 10-year increase 1248.25 (2.04¹⁰). Also, caution is required, when imaging or laboratory measurements have extensive or small ranges of values. For such cases, the one-unit amount for calculating OR may have been redefined for practical estimation of OR; for example, the OR per 1 × 10² or 1 × 10^-2 increase in value instead of one original unit value.

When 34 original research studies reporting OR published in the Korean Journal of Radiology in 2020–2021 [5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38] were evaluated against a rigorous standard (i.e., complete transparency in reporting), 91.2% (31/34) clearly mentioned how they defined the odds, 62.1% (18/29) unmistakably described the reference category for OR calculation for categorical variables, and 42.9% (9/21) specified the one-unit amount for OR for continuous variables. Therefore, there is room for improvement in reporting.

How to Interpret OR

An OR value greater than 1 in the example above indicates that the disease is more likely when the test result is F+ than F-. The greater the OR value, the more likely the patient with F+ has the disease. In this case, how can the exact OR value, for instance, an OR of 5, be interpreted? One should avoid being tempted to think that F+ patients are five times more likely to have the disease than F- patients, as such an interpretation may be incorrect. This interpretation is valid for a relative risk (RR) of 5. The RR is the ratio of probability values and can be obtained from the diagnostic cohort design as follows:

Probability of D+ in F+ (from the diagnostic cohort design) = A/(A + B)

Probability of D+ in F- (from the diagnostic cohort design) = C/(C + D)

$$\text{RR\hspace{1em}}\left(\text{from the diagnostic cohort design}\right)=\frac{\text{Probability of D+ in F+}}{\text{Probability of D+ in F-}}\mathrm{\hspace{1em}}=\mathrm{\hspace{1em}}\frac{\frac{A}{A+B}}{\frac{C}{C+D}}$$

The RR cannot be directly calculated with data from a case-control study because, unlike the OR calculation, M and N do not drop out of the RR calculation, as shown in the following equation:

$$\text{Invalid RR}\mathrm{\hspace{1em}}\left(\text{from the case-control design}\right)=\mathrm{\hspace{1em}}\frac{\frac{a}{a+b}}{\frac{c}{c+d}}\hspace{1em}=\hspace{1em}\frac{\frac{AM}{AM+BN}}{\frac{CM}{CM+DN}}\hspace{1em}\ne \hspace{1em}\frac{\frac{A}{A+B}}{\frac{C}{C+D}}$$

When the disease prevalence is very low in the population of interest in the study, the OR, which can also be calculated from a case-control design, approximates the true RR. If the disease prevalence is very low, A must be much smaller than B, and C must be much smaller than D. In this case, the RR equation can be simplified by ignoring A and C in the respective denominators as shown below as an approximation.

$$\text{RR}\mathrm{\hspace{1em}}=\mathrm{\hspace{1em}}\frac{\frac{A}{A+B}}{\frac{C}{C+D}}\hspace{1em}\approx \hspace{1em}\frac{\frac{A}{B}}{\frac{C}{D}}\hspace{1em}=\hspace{1em}\text{OR}$$

It is appropriate to interpret the OR obtained from a case-control study as an approximation of RR in a setting like this. However, the assumption of very low prevalence often does not hold in diagnostic research studies.

How to Report HR

The hazard is the slope of a survival curve, which is the rate of developing events in a time period [1]. The HR is the ratio of hazards of two survival curves and essentially describes the RR of the events occurring in a survival analysis (more generally referred to as time-to-event analysis) over time [1]. Similar to the reporting of OR, a few items should be explicitly described when reporting HR in research studies. First, a clear definition of events for survival analysis should be provided. Second, the reference category for HR calculation must be clarified for categorical variables. Finally, if HR is reported for a continuous variable, the one-unit amount for HR should be specified.

When 28 original research studies reporting HR published in the Korean Journal of Radiology in 2020–2021 [11,15,32,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63] were evaluated against a rigorous standard, 96.4% (27/28) explicitly defined the events for HR, 48.1% (13/27) unmistakably described the reference category for HR calculation for categorical variables, and only 50% (10/20) clearly described the one-unit amount for HR for continuous variables. Therefore, further improvements are required.

Examples of Clear, Accurate Reporting of OR and HR

Figure 2 shows slightly different styles to clearly and accurately report OR or HR [19,41,45]. Examples can be found in other published articles [15,28,33,37,38,52,59,62].

Fig. 2. Examples of transparent, accurate reporting of odds ratio and hazard ratio.

A-C. Red rectangles indicate areas of note. (A) Modified from Kim et al. Korean J Radiol 2021;22:1640-1649, with permission of the Korean Society of Radiology [19]. (B) Modified from Kim et al. Korean J Radiol 2022;23:172-179, with permission of the Korean Society of Radiology [41]. (C) Modified from Kwon et al. Korean J Radiol 2021;22:1985-1995, with permission of the Korean Society of Radiology [45].

Conclusion

Further efforts to report OR and HR more clearly and accurately in research manuscripts, as explained in this article, would facilitate a more effective delivery of scientific information.

Availability of Data and Material

Data sharing does not apply to this article as no datasets were generated or analyzed during the current study.