Abstract
Two procedures are proposed to assess sensitivity to a binary unobserved confounder for a binary outcome when the causal effect is expressed as a (log) odds ratio, as commonly arises in standard logistic modelling, particularly in case-control studies. The methods are based on graphical tools that visualize the extent to which an unobserved confounder could attenuate, nullify, or even reverse the estimated causal effect. The second procedure relies on a single sensitivity parameter, yielding a naturally bounded assessment that can be translated into an objective measure. Connections with Cornfield’s conditions on relative risks are presented, thereby enlarging the circumstances where the proposed procedures can be applied and establishing a link that allows extensions to polytomous confounders.
Keywords: Causal inference, Odds ratios, Sensitivity analysis, Logistic regression
1. Introduction
Causal conclusions drawn from observational studies necessarily rely on the unverifiable assumption that there are no unobserved confounders. In the context of a binary treatment—smoking versus non-smoking—and a binary outcome—the development of lung cancer—Cornfield and co-authors (Cornfield, Haenszel, Hammond, Lilienfeld, Shimkin, and Wynder, 1959) introduced a procedure to quantify the strength an unobserved binary confounder, such as a genetic factor, would need in order to fully explain away the observed relative risk. Since then, a range of methodological tools, known as sensitivity analysis, have been developed to challenge causal conclusions drawn from non-randomized experiments. These methods have been extended to accommodate continuous outcomes, see Cinelli and Hazlett (2020), time-to-event or other outcome variables, see Lin, Psaty, and Kronmal (1998) or Lee, Jeong, Lee, and Lee (2024), but also exclusion restrictions and other unverifiable assumptions.
In this work, we propose a set of procedures to assess sensitivity to a binary unobserved confounder for a binary outcome when the causal effect is expressed as a (log) odds ratio, as commonly arises in standard logistic modelling, particularly in case–control studies. This work complements that of Gasparin, Scarpa, and Stanghellini (2025), where a similar problem is addressed under the assumption of a continuous unobserved confounder. Two methods are proposed, both based on graphical tools that visualize the extent to which an unobserved confounder could attenuate, nullify, or even reverse the estimated causal effect. The second procedure relies on a single sensitivity parameter, yielding a naturally bounded assessment that can be translated into an objective measure.
In the context of a binary outcome, causal effects are typically measured using the relative risk or the risk difference, see Rosenbaum and Rubin (1983). Lee (2011) presents the bounding formula for the bias induced by a confounder with many levels. Ding and VanderWeele (2016) extend Cornfield’s conditions to cover confounder of any nature. Due to a strong parallelism between the two contexts, a link with the existing literature is established that could possibly pave the way to further developments.
We begin by setting out the notation and deriving formulae for a number of relevant parameters in Section 2. In Sections 3 and 4 we set out our proposals for two plot-based sensitivity analyses and link these approaches to the existing literature in Section 5. Section 6 shows an application of the methods on data from the Magistrate’s court data in England and Wales. We conclude with a discussion in Section 7.
2. Notation and First Derivations
Let X, U and Y be binary random variables such that:
| (1) |
| (2) |
Addition of covariates is straightforward and is omitted for brevity (see Section 6). If U is not measured, it is said to be an unobserved confounder that modifies the effect of X on Y. Notice that the derivations do not change if instead we postulate U to be a mediator, as in our application, lying on the pathway between X and Y. In this case, (2) is substituted by
| (3) |
where, by standard results, , the log of the odds ratio of the two-by-two table of . Let
be the relative risk of U for varying Y in the distribution of . Then, by standard probability results:
| (4) |
where with we denote the log of the odds ratio of the two-by-two contingency table of . Notice further that, as
where , (4) may be written in alternative way
| (5) |
where
As expected, swapping of the label of U does not change . The term d represents the distortion induced on when a binary mediator/confounder is omitted. Let
with (see Stanghellini and Doretti (2019) for details)
The parametric expression of can be derived as a consequence, as:
and, as ,
| (6) |
It is worth noting that only the for varying x matters. Let:
| (7) |
where
| (8) |
and
| (9) |
where
| (10) |
Therefore,
| (11) |
where . Sensitivity analysis seeks to identify the constellations of parameter values—specifically , , , (or, equivalently, ), such that and to assess how plausible, relative to the substantive context under investigation, such constellations are.
Closely related is the study of sign reversal. For a fixed value of , this involves determining which constellations of the remaining parameters lead taking the opposite sign, irrespective of the labelling of U.
Without loss of generality, we assume . We impose for now (see Section 7 for discussion), thereby assuming that the conditional effect of X on Y does not vary within levels of U. If so, (10) implies that whenever and do not share the same sign. We then notice that
-
•
if and only if , for all values of the other parameters;
for the only if, notice that there cannot be a constellation of values such that even if . Furthermore, if and only if . Notice that for a sign reversal to happen, , thereby confirming the finding of Raggi, Stanghellini, and Doretti (2023) (see also Cox and Wermuth (2003) for a more general context).
3. Our Proposal
Given , we try and determine for which constellation of the other parameters (, , , ) takes on values that are either zero or below zero. In this situation, and for , and have to share the same sign. This is because, if is positive (negative), for d to be positive, has to be positive (negative), see (11), and therefore has to be larger (smaller) than . Some algebra shows that this is possible only if is positive (negative).
One possible strategy is to fix to a particular positive value, consider and as free parameters and use (11) to calculate . Notice, however, that while is a function of and only, is also a function of , which is the object of the sensitivity analysis. However, for fixed values of , depends on both and and is therefore is free to vary.
This allows us to proceed as follows:
-
1.
fix to a particular positive value;
-
2.
use and as free parameters;
-
3.
plot the contour level of ;
-
4.
determine for which values of and d is above, equal or below ;
-
5.
interpret those values in the light of the notion that: only when d equals then and ; when d is larger than , then is negative; when d is smaller than then is positive and .
In Figure 1, the contour plot for at different values of is presented. As it will become clear in next Section 4, for no sign reversal is possible. Researchers may judge how plausible the combinations of and are for a sign reversal to be possible. To account for sampling variability, should be taken as the the lower limit of a confidence interval, with p reasonably small.
Figure 1.

Contour plot of function , , for different positive values of . In black the contour line for the . Along this line, and . Values above 0 give rise to a negative value of and therefore to a sign reversal. Only large values of can be combined with reasonable positive values of to form a d large enough that, when summed with a negative value of lead to a positive value of . If is small, no sign reversal is possible for the range of values of and . Notice that for a fixed value of , the largest value of such that for all possible values of should be seen as the lower limit for beyond which a sign reversal may happen (or, equivalently, the upper limit for that makes a sign reversal not possible).
4. Another Proposal: Linear Function
Previous derivations do not lead to a naturally bounded area where the can be negative and therefore assessment on how likely is a sign reversal is largely subjective. In what follows, we propose a way to sidestep this issue. From (6), it follows that:
| (12) |
Let and . We denote with . Then,
or
| (13) |
from which it follows:
-
1.
if , for all and ;
-
2.
if , can be equal to also if .
Condition 1 is coherent with the collapsibility condition, as if U as no impact on Y, i.e. , (Greenland and Pearl (2011)); Condition 2 may happen in a particular parametric constellation such that
see (9). To avoid the trivial condition of absence of counfoundedness, we impose . Therefore,
| (14) |
This corresponds to a set of parallel lines with slope , which is always positive, and intercept . Once again, in what follows, we shall impose that . With no loss of generality we further impose . In order for and to be plausible values (i.e., between 0 and 1), the intercept has to be between −1 and 1. The sign of the intercept depends on the value of and
| (15) |
with the first inequality trivially satisfied for . If , then (14) becomes the equation of the diagonal. In this setting, this may happen by parametric cancellation (), a possibility that we rule out because it corresponds to a set of 0 Lebesgue measure in the parameter space. We therefore consider the diagonal as the limiting behavior of the model with L tending to 1 from above.
We now impose that . In this case, values of are plausible for the intercept. We fix , with . Then (14) becomes
| (16) |
For a given choice of ϵ, we then draw (16) in the square with vertices , where the x-axis represents and the y-axis represents , which, as probabilities, are bounded between 0 and 1. Points above (16) should correspond to models with . In fact they cannot belong to models with , as in this case both the intercept and the slope will be smaller than the ones in (16). Points between (16) and the diagonal lie on lines with the intercept and/or the slope smaller than the ones in (16). In order to satisfy the constraint (15), for fixed, they correspond to models with . If , the slope is larger than 1 and lines never cross the diagonal. If then the slope is smaller than 1. However, in this case, the intercept has to be negative and, again, the corresponding lines never cross the diagonal.
To summarise, for a given :
-
•
Points above the line (16) belong either to a line with higher intercept or steeper slope or both. In all cases, they correspond to models with ;
-
•
Points between (16) and the diagonal correspond to models with ;
-
•
Points below the diagonal correspond to models with .
For each choice of ϵ, it therefore makes sense to consider the area above the (16) as an indication of how likely a sign reversal is. The area of the triangle is naturally bounded by 1 and can be easily plotted against (omitted for brevity). Notice that this approach links the sensitivity analysis to just one parameter. As a large ϵ leads to a small value of the intercept of (16), making a sign reversal more likely, we suggest to take as a good starting point the ϵ so that equals the smallest integer larger than .
To establish a connection with the previous derivations, the contour levels for may be superimposed, as in Figure 2. For a fixed , the equation
intersects (16) in at most two points (see also Daniel, Zhang, and Farewell (2021)). Focusing on the corresponding to the contour level with a single point on (16) (which is therefore the tangent), this value represents the maximum value of such that a sign reversal is not possible and allows us to consider how likely this is, taking into account context-specific information.
Figure 2.

Plot showing the combinations of values of and leading to a sign reversal for . For smaller than no sign reversal is possible. For the area above the black line is really small, making the sign reversal highly unlikely. The contour plot of tangent the line provides information on the highest value of that, for combined with the value , makes the sign reversal not possible. Larger values of may lead to a sign reversal.
5. Links with Works on Relative Risks
Let
with and analogously defined. Let
We then can write:
where and . Once again, with no loss of generality, we impose . Cornfield’s first condition (Cornfield et al., 1959) that can be easily derived.
If we now denote with:
the relative risk of Y against U, conditional on , constant w.r. to x (i.e., ), then the ratio between the marginal relative risk and the conditional relative risk can be written as (see Schlesselman (1978))
Due to the parallelism between this formula and (13), an analogous line of reasoning can be followed. After imposing , and denoting by , we have:
which corresponds to a set of parallel lines with slope r (always positive) and intercept . For and to be bounded between 0 and 1, we need to impose
| (17) |
with the first inequality trivially satisfied for . Notice that the second inequality entails Cornfield’s second condition, as if , then and . For a fixed value of Λ, say , with , the line therefore becomes
| (18) |
Points above (18) in the square with vertices , where the x-axis represents and the y-axis represents , lie on lines corresponding to models with (and therefore ). Points between (18) and the diagonal correspond to models with . Once again, for a given choice of , the area above the line gives a measure of how likely the sign reversal is. Superimposing the contour plot of the function
gives an idea of how plausible the values of are within that area. We think that being able to combine the odds ratio of the marginal distribution of with relative risks is one of the added values of the proposed investigation.
6. Application
We use data from the Magistrate’s court in England covering Weapon’s offences registered between 01/01/2018 and 31/12/2020 to explore how likely it is, for the significantly positive coefficient of ethnicity in the logistic regression for immediate custody (incarceration) as a response, to become non-significant if a mediator is adjusted for. We posit that the potential mediator is whether an offender had previous convictions and that, controlling for this information, the ethnicity becomes irrelevant for the Magistrate’s decision of incarceration. Notice that, in this case, the mediator is a variable we observe as the aim is to explain how to interpret the plots. The total sample size for this analysis is 18322. For a simulation based sensitivity analysis see Pina-Sánchez, Morales, Guilfoyle, Veiga, and Geneletti (2024).
A number of simplifying assumptions are made. A more nuanced analysis of the associations between ethnicity and custodial sentences, with previous conviction as one of a number of potential mediators for different offence types using the same data set are given in Pina-Sánchez, Morales, Guilfoyle, Veiga, and Geneletti (2025). Let the outcome if the offender receives a custodial sentence and 0 if they do not. We binarise a number of variables: In particular, Ethnicity, denoted by if the offender is not White and 0 if they are White; the binary mediator, denoted by U, taking value 1 if the offender has previous convictions or not. We only adjust for sex of the offender (binary) and age at the time of offence (numeric) and run three regressions,
| (19) |
| (20) |
| (21) |
where Z represents the two covariates sex and age. Equation (19) is the termed the marginal regression, equation (20) is akin to (2) and (21) is akin to (3), with the addition of Z.
Table 1 presents the outcome of standard Maximum Likelihood estimating procedure. The term in (20) is non significant (not reported in the table). We note that the coefficient of ethnicity in all the regressions is positive and significant and, while it decreases in the regression where previous convictions are adjusted for, it neither becomes non-significant nor changes sign. The marginal OR for ethnicity exp() = 1.46, the conditional OR for ethnicity exp() = 1.42 and the OR associated with the mediator previous convictions is exp() = 2.23. We take , which corresponds to the lower limit of the 95% confidence interval, and discuss the two types of plots proposed.
Table 1.
Maximum likelihood estimates of the three logistic regressions.
| Parameter | Estimate | 95% CI | OR | 95% CI | p-val |
| −3.55 | (−3.88,−3.22) | 0.03 | (0.02,0.04) | * | |
| 0.38 | (0.26,0.5) | 1.46 | (1.29,1.65) | * | |
| 0.004 | (0,0.008) | 1.004 | (0.999,1.01) | ||
| 0.78 | (0.49,1.07) | 2.18 | (1.61,2.94) | * | |
| −3.91 | (−4.24,−3.58) | 0.02 | (0.01,0.03) | * | |
| 0.35 | (0.23,0.47) | 1.41 | (1.25,1.60) | * | |
| 0.005 | (0.001,0.009) | 1.005 | (1.001,1.01) | * | |
| 0.73 | (0.44,1.02) | 2.08 | (1.54,2.82) | * | |
| 0.8 | (0.68,0.92) | 2.22 | (1.98,2.49) | * | |
| −0.58 | (−0.72,−0.44) | 1.79 | (0.47,0.64) | * | |
| 0.18 | (0.1,0.26) | 1.19 | (1.11,1.28) | * | |
| −0.005 | (−0.007,−0.003) | 1.005 | (1.003,1.008) | * | |
| 0.24 | (0.12,0.36) | 1.27 | (1.13,1.43) | * |
Figure 3 shows the plots of against with the contours of derived for a range of values of . If we look at the 0 contour line, this represents models with so that contours above it indicate a sign reversal. We see that as increases in value, the size of , and therefore of , for which a 0 value of , or indeed a sign reversal, is possible becomes smaller. The first panel in the central row corresponds to the estimated value of as in Table 1. According to this, a sign reversal is possible for . This implies that the odds for a non-White offender to have previous convictions should be at least 4.5 larger than the same odds for White offender. In order to bring the odds ratio to a reasonable value (say about 2, corresponding to a equal to 0.69), has to be at least equal to 2. This implies that in both strata formed by X, the odds of a person with previous convictions to be incarcerated have to be 7.4 times higher than the same odds for a person without previous convictions. While this value is not supported in our particular application (see Table 1), in a situation where the mediator is unobserved, there is a non-negligible possibility of a sign reversal.
Figure 3.

Contour plot of with respect to and for the estimated log odds ratio coming from the Magistrate’s court data.
The second set of plots in Figure 4 tackles the same question from a different angle. The dashed white line represents whereas the solid black line represents line in eq. (14). The area above the solid line can be seen as a measure of how likely a sign reversal is. Once again, the first plot of the central row corresponds to the as estimated from the data. Here the area corresponds to 22% of the whole area. When looking at the contour levels of , we see that has to be larger than 1.5 for a sign reversal to become possible and the same considerations of above apply.
Figure 4.

Linear plot for the estimated log odds ratio coming from the Magistrate’s court data. Contour plot of is overimposed.
In conclusion, in a similar situation where the potential mediator or confounder was unobserved, an with standard error of 0.06, would make a sign reversal unlikely, but not impossible. This is borne out by the directly estimated value of , which is 0.35.
7. Discussion
We have presented two graphical tools to perform sensitivity analysis when the causal effect is measured on a (log) odds ratio scale, the second one based on just one sensitivity parameter and leading to a naturally bounded measure. Notice that equation (13) is also in Arah, Chiba, and Greenland (2008) where, however, a different line of reasoning is taken. We are not aware of sensitivity based on (11). Extensions to are rather straightforward, provided that , with a positive (negative) making a sign reversal more (less) likely. Outside this range of values, not only do the derivations become clumsy, but the interpretation of a sign reversal becomes cumbersome, as the sign of the effects of one variable vary within the levels of the other. The parallelism between the proposed methods and the existing ones on relative risks allows us to explore extensions to U having more than two levels. A real data example is presented to show how to critically interpret the proposed plots. Straightforward removal of the assumption of a binary treatment is strongly recommended for subject matter conclusions, which are beyond the scope of this paper.
We finally point out that the paper by Cornfield and co-authors constituted a seminal contribution, not only by introducing an objective approach to assessing the robustness of causal conclusions to departures from underlying assumptions, but also by critically comparing a large body of evidence, elucidating and discussing important differences across studies in outcome measures, variable definitions, study populations, sampling designs, measured covariates, diagnostic instruments and exposure to other etiological factors. For these reasons, the paper is regarded as a landmark in applied research.
Funding
Sara Geneletti was supported in part by the Secondary Data Analysis Initiative of the Economic and Social Research Council (Grant Ref: ES/W00738X/1). Elena Stanghellini was supported in part by Fondo Ricerca di Ateneo Università degli studi di Perugia, edizione 2022 (WP 4.1, “Rationalists”).
Note
This work was undertaken in the Office for National Statistics Secure Research Service using data from ONS and other owners and does not imply the endorsement of the ONS or other data owners.
Routines in R to construct the plots can be downloaded at https://github.com/sggi-76/sensitivity_OR.
Acknowledgments
The authors would like to acknowledge Prof. Pina-Sanchez who provided invaluable substantive advice and insight.
Funding Statement
Sara Geneletti was supported in part by the Secondary Data Analysis Initiative of the Economic and Social Research Council (Grant Ref: ES/W00738X/1). Elena Stanghellini was supported in part by Fondo Ricerca di Ateneo Università degli studi di Perugia, edizione 2022 (WP 4.1, “Rationalists”).
References
- Arah Onyebuchi A., Chiba Yasutaka, Greenland Sander. Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. https://pubmed.ncbi.nlm.nih.gov/18652982/ Annals of Epidemiology. 2008;18(8):637–644. doi: 10.1016/j.annepidem.2008.04.008. [DOI] [PubMed] [Google Scholar]
- Cinelli Carlos, Hazlett Chad. Making sense of sensitivity: Extending omitted variable bias. Journal of the Royal Statistical Society, Series B, Statistical Methodology. 2020;82(1):39–67. doi: 10.1111/rssb.12348. [DOI] [Google Scholar]
- Cornfield Jerome, Haenszel William, Hammond E. Cuyler, Lilienfeld Abraham M., Shimkin Michael B., Wynder Ernst L.. Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute. 1959;22(1):173–203. [PubMed] [Google Scholar]
- Cox David R., Wermuth Nanny. A general condition for avoiding effect reversal after marginalization. Journal of the Royal Statistical Society, Series B, Statistical Methodology. 2003;65(4):937–941. [Google Scholar]
- Daniel Rhian, Zhang Jingjing, Farewell Daniel. Making apples from oranges: Comparing noncollapsible effect estimators and their standard errors after adjustment for different covariate sets. Biometrical Journal. 2021;63(3):528–557. doi: 10.1002/bimj.201900297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ding Peng, VanderWeele Tyler J.. Sensitivity analysis without assumptions. Epidemiology. 2016;27(3):368–377. doi: 10.1097/EDE.0000000000000457. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gasparin Matteo, Scarpa Bruno, Stanghellini Elena. Omitting continuous covariates in binary regression models: Implications for sensitivity and mediation analysis. Statistica Neerlandica. 2025;79(1):e12369. [Google Scholar]
- Greenland Sander, Pearl Judea. Adjustments and their consequences—collapsibility analysis using graphical models. International Statistical Review. 2011;79(3):401–426. [Google Scholar]
- Lee Seungjae, Jeong Boram, Lee Donghwan, Lee Woojoo. Sensitivity analysis for effects of multiple exposures in the presence of unmeasured confounding: Non-Gaussian and time-to-event outcomes. Statistics in Medicine. 2024;43(30):5996–6025. doi: 10.1002/sim.10293. [DOI] [PubMed] [Google Scholar]
- Lee Wen-Chung. Bounding the bias of unmeasured factors with confounding and effect-modifying potentials. Statistics in Medicine. 2011;30(9):1007–1017. doi: 10.1002/sim.4151. [DOI] [PubMed] [Google Scholar]
- Lin D. Y., Psaty B. M., Kronmal R. A.. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics. 1998;54(3):948–963. doi: 10.2307/2533848. [DOI] [PubMed] [Google Scholar]
- Pina-Sánchez Jose Morales Ana Guilfoyle Eoin Veiga Ana Geneletti Sara Journal of Legal Research Methodology2024
- Pina-Sánchez Jose, Morales Ana, Guilfoyle Eoin, Veiga Ana, Geneletti Sara. Testing the interrelationship between area deprivation and ethnic disparities in sentencing. https://spssi.onlinelibrary.wiley.com/doi/abs/10.1111/asap.12446 Analyses of Social Issues and Public Policy. 2025;25(1):e12446. doi: 10.1111/asap.12446. [DOI] [Google Scholar]
- Raggi Martina, Stanghellini Elena, Doretti Marco. Path analysis for binary random variables. Sociological Methods & Research. 2023;52(4):1883–1915. doi: 10.1177/00491241211031260. [DOI] [Google Scholar]
- Rosenbaum Paul, Rubin Donald B.. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society, Series B, Methodological. 1983;45(2):212–218. [Google Scholar]
- Schlesselman James J.. Assessing effects of confounding variables. American Journal of Epidemiology. 1978;108(1):3–8. [PubMed] [Google Scholar]
- Stanghellini Elena, Doretti Marco. On marginal and conditional parameters in logistic regression models. Biometrika. 2019;106(3):732–739. [Google Scholar]
