Skip to main content
Philosophical transactions. Series A, Mathematical, physical, and engineering sciences logoLink to Philosophical transactions. Series A, Mathematical, physical, and engineering sciences
. 2024 Jan 29;382(2268):20230015. doi: 10.1098/rsta.2023.0015

Hidden variables, free choice, context-independence and all that

Ehtibar N Dzhafarov 1,
PMCID: PMC10822708  PMID: 38281720

Abstract

This paper provides a systematic account of the hidden variable models (HVMs) formulated to describe systems of random variables with mutually exclusive contexts. Any such system can be described either by a model with free choice but generally context-dependent mapping of the hidden variables into observable ones, or by a model with context-independent mapping but generally compromised free choice. These two types of HVMs are equivalent, one can always be translated into another. They are also unfalsifiable, applicable to all possible systems. These facts, the equivalence and unfalsifiability, imply that freedom of choice and context-independent mapping are no assumptions at all, and they tell us nothing about freedom of choice or physical influences exerted by contexts as these notions would be understood in science and philosophy. The conjunction of these two notions, however, defines a falsifiable HVM that describes non-contextuality when applied to systems with no disturbance or to consistifications of arbitrary systems. This HVM is most adequately captured by the term ‘context-irrelevance’, meaning that no distribution in the model changes with context.

This article is part of the theme issue ‘Quantum contextuality, causality and freedom of choice’.

Keywords: contextuality, context-independent mapping, context-irrelevance, coupling, free choice, hidden-variable

1. Introduction

Hidden variable models (HVMs) are arguably the main reason why contextuality and its non-locality version have acquired prominence in the foundations of quantum mechanics (QM). Ever since it was accepted that results of a measurement, such as that of a spin, are almost always random variables (with the exception of repeated sharp measurements), physicists have been interested in the possibility of ‘explaining’ such random variables as deterministic functions of some underlying sources of variability, even if as yet unknown to us, ‘hidden’. This possibility is often presented as a belief famously held by Albert Einstein, and then famously ruled out by Bell’s and Kochen–Specker’s theorems juxtaposed with QM predictions [1,2].

However, even before any detailed analysis, there is a good reason to doubt that HVMs can play an explanatory role. The reason is that the existence of a random variable of which several jointly distributed random variables are deterministic functions is ensured trivially: the properties of being jointly distributed and being functions of a single random variable are one and the same property. Conversely, variables that are not jointly distributed, as they are predicated on mutually exclusive conditions, cannot be functions of a single random variable. This means that one must have as many hidden variables as there are mutually exclusive contexts, even if they all have the same distribution. This is not to say that HVMs cannot be meaningfully constructed and interpreted. This only means that one should be careful not to attach deep physical or otherwise substantive connotations to purely mathematical and universally satisfiable representations. This is a point elaborated throughout the paper.

Here, I will synthesise some of my recent published work to provide a comprehensive and rigorous account of HVMs. The most restrictive HVM, one introduced by Bell and describing non-contextual systems with no disturbance, is known not to hold for many systems of random variables. When this happens, the constraints imposed on an HVM have to be relaxed, and this can be done in two ways: either by allowing for a dependence of the measurement outcome distributions on contexts or by allowing for an interdependence between the hidden variables and the choices of settings for the measurements. In [3], I proved the equivalence of these two options. In this paper, I present an improved and more rigorous proof. I will argue that such assumptions as freedom of choice and context-independent mapping (of hidden variables into observable ones) are merely metaphorical depictions of some basic representations of jointly distributed random variables. Next, I discuss the problem of separating disturbance (or signalling) from contextuality in the situations in which Bell’s HVM does not hold. While this is the central issue for the Contextuality-by-Default (CbD) theory [47], the difference between disturbance and contextuality is not apparent in the formulations of the HVMs. However, one can effectively separate disturbance from contextuality by using the consistified systems introduced in Dzhafarov [8,9]. Any system of random variables can be reformulated as an equivalent, in a well-defined sense, system that has no disturbance (is consistently connected, in the CbD terminology). The equivalence of the HVMs with context-dependent mapping and the HVMs with violations of free choice holds for these consistified systems too, but now any such HVM indicates pure contextuality. At the conclusion of the paper, I will discuss two assumptions that one could suspect to be required for the development presented, and show that, once again, they are not assumptions at all, because they are trivially satisfied in the language of random variables.

2. Conceptual and terminological set-up

A system of random variables is a double-indexed set

R={Rqc:cC,qQ,qc}, 2.1

where Q is a set of contents, C is a set of contexts, and qc means that content q is measured in context c. A content q in Rqc can be viewed as a question that the random variable Rqc answers (e.g. ‘is the spin along axis q up?’, answered ‘yes/no’) or as a choice of measurements (spin along axis q) whose outcomes (up/down) are represented by Rqc. The context c in Rqc indicates conditions under which Rqc is recorded, such as the set of all other measurements made together with Rqc and the spatial and temporal relations among them. The matrix below provides an example of a system of random variables:

2. 2.2

The subsystem of all random variables within a given context c is called a bunch (of random variables),

Rc={Rqc:qQc}, 2.3

where

Qc={qQ:qc}. 2.4

For instance, the bunch R2 in the system R0 is {R32,R42}. Any bunch Rc is a random variable, which means that all components Rqc of Rc are jointly distributed (are measurable functions on the same probability space). However, no two random variables from different bunches have a joint distribution, they are stochastically unrelated (are measurable functions on distinct probability spaces). Indeed, consider what a joint distribution of Rqc and Rqc with cc could look like (X and Y being any measurable sets):

2. 2.5

The question marks cannot be all replaced with zeros because they must sum to one. At the same time, any non-zero joint probability would indicate that Rqc and Rqc co-occur, which would contradict the fact that c and c are mutually exclusive contexts.

3. Hidden variable models

(a) . (Excessively) general HVM

Let us begin with the most general possible HVM, denoted HVMGen:

Rc=α(Qc,Λc(c),c). 3.1

The function α returns as its value an indexed set, and the dependence of α on Qc should be understood as its indexing, matching the indexing of Rc. Thus, for system R0 in (2.2), Q2={3,4}, and the HVMGen representation for R2=(R32,R42) is

α(Q2,Λ2(2),2)=(Projq=3α(Q2,Λ2(2),2),Projq=4α(Q2,Λ2(2),2))=(α(3,Λ2(2),2),α(4,Λ2(2),2)), 3.2

where ProjqV stands for the q-indexed component of the indexed set V.1

One can present this model graphically as

(a) . 3.3

The arrows ab in this and subsequent diagrams (where b is a random variable and a is a random variable or a parameter) should be read as ‘different values of a may result in different distributions of b.’

HVMGen is not a falsifiable model, it can be applied to any system of random variables. This can be demonstrated by simply putting Λc(c)=Rc, with the stand-alone c in α becoming a dummy argument, and Qc extracted from Λc(c) as its indexing set.

(b) . Context-independent mapping without free choice

The argument just presented shows that the direct dependence of the distribution of Rc on c can be eliminated:

Rc=β(Qc,Λc(c)), 3.4

or graphically,

(b) . 3.5

Although not obvious at first glance, this HVM would traditionally be interpreted as a model with a context-independent mapping of Λc(c) into Rc (no arrow from c to Rc) but with generally compromised freedom of choice (the distribution of Λc may depend on c).

I will denote this model HVM+CIMFC, using the self-evident abbreviations. We have established that HVMGen can always be reduced to HVM+CIMFC. Using again as an example system R0 in (2.2), the HVM+CIMFC representation for R2=(R32,R42) is

β(Q2,Λ2(2))=(β(3,Λ2(2)),β(4,Λ2(2))). 3.6

Freedom of choice in the QM literature is usually discussed in terms of the relationship between one’s choice of c and the hidden variable Λc(c). This means that c is treated as a random variable (which is a dubious viewpoint, see [3]), and freedom of choice means that c and Λc are stochastically independent. In my representation of HVMs, c is always a deterministic parameter, which, with respect to the traditional view, simply means that all random variables in the model are conditioned on fixed values of c. Any restriction of freedom of choice in the traditional sense then translates into a dependence of the distribution of Λc on c. As a special case, this also applies to the possibility that c is a function of the hidden variable, c=f(Λc), which may possibly be interpreted as a depiction of superdeterminism: in an HVM+CIMFC, one simply replaces this function with Λc(c), defined by f(Λc(c))=c.

(c) . Free choice without context-independent mapping

It is further possible to transform HVM+CIMFC into a model that is, in a sense, its reverse. Given (3.4), one can form an arbitrary coupling of Λc for all contexts c,2

Γ:={Λc(c):cC}, 3.7

and then create, for every cC, a distributional copy Γc of Γ, so that these copies are pairwise stochastically unrelated. Then

Λc(c)=ProjcΓc 3.8

and

Rc=β(Qc,ProjcΓc)=γ(Qc,Γc,c), 3.9

where the variables Γc (as indicated by the lack of c as their argument) have one and the same distribution for all cC. Note that we cannot eliminate the index c in Γc, because Rc=γ(Qc,Γ,c) would make all Rc jointly distributed.

The traditional interpretation of the HVM described by (3.9) would be that the freedom of choice is not compromised here, but context-independence is generally violated. Using our graphical representation,

(c) . 3.10

I will denote this model HVMCIM+FC. We have established that HVM+CIMFC implies (can be translated into) HVMCIM+FC. Using our example of system R0 in (2.2), the HVMCIM+FC representation for R2=(R32,R42) is

γ(Q2,Γ2,2)=(γ(3,Γ2,2),γ(4,Γ2,2)). 3.11

(d) . Free choice with context-independent mapping

Both HVM+CIMFC, and HVMCIM+FC can be viewed as deviations from their special case

Rc=δ(Qc,Γc), 3.12

or, graphically,

(d) . 3.13

where the random variables Γc for all cC are identically distributed and pairwise stochastically unrelated. This model can be denoted HVM+CIM+FC, as it satisfies both freedom of choice and context-independence in the mapping of Γc into Rc. In our example of system R0 in (2.2), the HVM+CIM+FC representation for R2=(R32,R42) is

δ(Q2,Γ2)=(δ(3,Γ2),δ(4,Γ2)). 3.14

Unlike the previous two HVMs, this one is a true model, as it is falsifiable. The latter is demonstrated, for instance, by relating predictions of QM to the Bell-type [1] and Kochen–Specker-type theorems (in addition to the original references [1,2] see, e.g. [1012]). The Bell-type theorems establish necessary and sufficient conditions for a system of random variables to be described by HVM+CIM+FC, which can then be shown to fail for some QM systems. In the Kochen–Specker-type theorems one constructs systems of random variables in accordance with QM, and then demonstrate that they cannot be described by HVM+CIM+FC.

(e) . Equivalence theorem and its consequences

Combining the implications in §§3b,c,

HVMGenHVM+CIMFCHVMCIM+FC, 3.15

and observing that HVMCIM+FC is a special case of HVMGen, we obtain the following statement.

Theorem 3.1. —

The models HVMGen, HVM+CIMFC and HVMCIM+FC are pairwise equivalent:

graphic file with name rsta20230015f07.jpg 3.16

Let us consider two consequences of this theorem. One of them is that when HVM+CIM+FC is not applicable to a system, one can arbitrarily choose between describing the system in the language of HVM+CIMFC or in the language of HVMCIM+FC. In particular, one can always use one and the same measure for the degree of deviation of these two HVMs from HVM+CIM+FC:

(e) . 3.17

A special case of this corollary, for a particular system of random variables, is presented in [13].

The second consequence of the theorem is that HVM+CIMFC and HVMCIM+FC are both unfalsifiable, either of them can describe any system of random variables. This follows from the demonstration, at the end of §3(a), that HVMGen is unfalsifiable, in fact, even in the form of HVM+CIMFC. This, in combination with the inter-translatability of HVM+CIMFC and HVMCIM+FC, should make one skeptical about interpreting the dependence of the distribution of Λc on c in terms of freedom of choice, in any substantive meaning of these words, and interpreting an arrow from c to Rc as a physical influence exerted by the context. Their complete equivalence and empirical emptiness (universal applicability) suggest the view that HVM+CIMFC and HVMCIM+FC are purely mathematical descriptions of the joint distributions within bunches of random variables and of the differences between them.

This view does not change if one constrains or even completely specifies all distributions and functions in the formulation of HVM+CIMFC or HVMCIM+FC, making them thereby predictive and falsifiable. The inter-translatability of the two types of models holds irrespective of their falsifiability. Moreover, a completely specified HVM can always be thought of as a corresponding unconstrained HVM after it has been applied to the system predicted by the completely specified HVM. Clearly, the ontological interpretation of a model (say, HVM+CIMFC) does not depend on whether it has been applied to a particular system of random variables, because this does not change the facts that (A) it could have been applied to any other system, and (B) it can be translated into an HVM of a completely different nature (in this case, HVMCIM+FC).

This is not to say that the notions of freedom of choice and context-(in)dependent mapping may not be assigned substantive meanings and be propitiously used in physical or other scientific theories. One should, however, distinguish HVMs per se from scientific theories that predict specific systems of random variables and therefore HVM representations thereof. My only point here is that these substantive meanings belong to the parts of theories extraneous to the HVMs to which the theories lead. In other words, these meanings cannot be derived from the HVMs themselves, from the fact that a system can be described by HVM+CIMFC or HVMCIM+FC (or even HVMGen, combining the two)—because any system can, and by any of them. The language of HVMs as understood in this paper (and in most discussions of the HVMs in the foundations of physics, beginning with Bell’s work) is simply too crude to capture certain substantive notions and distinctions. We will see below that it is sometimes too crude even to depict the difference between much more clear-cut notions of contextuality and signaling. A simple analogy may help to understand this. Any real-valued random variable R can be generated by applying an appropriate transformation f to a variable U uniformly distributed between 0 and 1. As one observes values of R, it is possible that there is a computer program that de facto computes them by first generating values of U and then applying to them the function f. If this is known from some extraneous source of knowledge, then we have a valid naturalistic interpretation of the model R=f(U), which then acquires a privileged status over other representations of R (such as R=g(E), for an exponentially distributed E). However, such an interpretation cannot be derived from the fact that R is representable as f(U)—because this representation is mathematically guaranteed, and moreover, can be replaced with other representations (referring, e.g. to the same R=g(E)).

The terms freedom of choice and context-(in)dependent mapping may still be conveniently used as labels for HVM components, provided one does not impute to them their colloquial, physical, or philosophical connotations. Moreover, the conjunction of these two notions does have a substantive meaning, because HVM+CIM+FC is a falsifiable model which de facto does not apply to some QM systems of random variables. In [3], I argued that the notions in question should only be used in conjunction: ‘one cannot accept local causality without free choice, because denying free choice is equivalent to denying local causality’ (local causality being the specific form of context-independent mapping used by Bell in the discussion published in [14]). While the present paper only strengthens this assertion, I would like to add here that one can very well decide to abandon the terms freedom of choice and context-(in)dependent mapping altogether, and use instead a simpler way to characterise HVM+CIM+FC. Namely, this is the model in which context c is irrelevant for determining any distributions involved (which includes the distribution of the hidden variable Λc and the distribution of the observable bunch Rc). Therefore, HVM+CIM+FC can be referred to as the model satisfying the assumption of context-irrelevance.

4. Contextuality in consistently connected systems

We have managed so far to discuss HVMs without involving the notion of (non)contextuality. It is now time to involve it. The traditional definition of this notion simply coincides with that of HVM+CIM+FC: a system of random variables is non-contextual (or, for distributed systems, local) if it is described by this HVM, and a system that cannot be so described is contextual. One consequence of this definition is that a non-contextual system must be consistently connected. The latter is a CbD term for what is usually called in QM non-disturbance or non-signaling: in a consistently connected system, any two random variables sharing a content, Rqc and Rqc, have the same distribution. Inconsistent connectedness (disturbance, signalling) therefore makes a system contextual. This definition makes the class of contextual systems too large and heterogeneous, and CbD offers a more analytic approach, presented in the next section. For now, however, let us confine consideration to consistently connected systems.3

The main consequence of R being described by HVM+CIM+FC is as follows. With reference to (3.12), construct the random variable S defined by

S=δ(Q,Γ), 4.1

where Γ has the same distribution as Γc in (3.12). The variable S is called a reduced coupling of the system R [15]. Its (jointly distributed) elements are indexed by the elements of Q, and for any cC, we have

Rc=dProjQcS, 4.2

where =d indicates equality of distributions. Thus, for our system R0 in (2.2), the reduced coupling has the form S={S1,S2,S3,S4}, and the condition (4.2) means that in the matrix

4. 4.3

the rows are distributed as the corresponding rows in (2.2).

It is clear that the implication HVM+CIM+FCS can be reversed, whence we have the following criterion: system R is described by HVM+CIM+FC if and only if it has a reduced coupling (4.1) subject to (4.2). For some simple systems, this has been semi-formally derived as the ‘joint distribution criterion’ by Fine [12], based on the idea of Suppes & Zanotti [16]. Note that the use of the language of random variables makes this criterion obtain essentially automatically.

For these and other simple systems (notably for the important class of the so-called cyclic systems [17]) other criteria have been derived, primarily in the form of inequalities involving expected values of the products of the random variables within different bunches. These additional criteria should be viewed as mere shortcuts, because in all cases when they are available and in many cases when they are not, the existence or non-existence of a reduced coupling (4.2)–(4.3) can be established directly, by means of linear programming.

This is a good place to note that some authors, having correctly observed that Bell-type inequalities require a system of jointly distributed variables, as in (4.3), and having also correctly observed that in a system of observable probabilities different bunches are not jointly distributed, have then erroneously concluded that the Bell-type theorems were wrong [1820]. In fact, the only problem with these theorems, from the earliest ones in the 1960s all the way to the present, is that they are usually proved less than rigorously, with unacknowledged abuse of notation. When viewed as theorems about reduced couplings, their proofs are correct. The corrected proofs do not require that different bunches be jointly distributed. They only require that a system can be described by HVM+CIM+FC, the model that does preserve stochastic unrelatedness of different bunches.

5. Contextuality in inconsistently connected systems

CbD offers a generalised notion of (non)contextuality, one that applies to all systems of random variables, including inconsistently connected ones (those with disturbance, or signalling).4 Given a system R in (2.1), its (complete) coupling is defined as a random variable

S={Sqc:cC,qQ,qc}, 5.1

such that, for every cC,

Sc=dRc, 5.2

where

Sc:={Sqc:qQc}. 5.3

Note that calling S a random variable implies that, unlike in the system R, all components of S are jointly distributed. A system R is non-contextual if it has a coupling S in which, for every content qQ and any two contexts c,c such that qc and qc, the probability

p[Sqc=Sqc], 5.4

is maximal possible. The maximum is computed for fixed distributions of Sqc, Sqc (which coincide with the distributions of Rqc, Rqc, respectively). If such a coupling does not exist, R is contextual. If R is consistently connected, then the maximal probability for any event Sqc=Sqc equals 1, and the definition reduces to the existence of the reduced coupling introduced in the previous section.

To illustrate this for our example (2.2), a coupling for R0 is a random variable

5. 5.5

whose rows are distributed as the corresponding rows in (2.2). R0 is non-contextual if and only if its coupling S can be chosen so that the probabilities of the events

S11=S13,S21=S23,S31=S32,S31=S33,S32=S33,S42=S43, 5.6

are all maximal possible. In particular, if R0 is consistently connected, then it is non-contextual if and only if all these probabilities in some coupling S equal 1. In such a coupling, the variables S11 and S13 can both be renamed into S1, the variables S21 and S23 can be renamed into S2, etc. We thus obtain the reduced coupling {S1,S2,S3,S4} subject to (4.3).

6. Consistified systems

What is an HVM representation of contextuality in the case when a system may be inconsistently connected? Clearly, Bell’s HVM+CIM+FC cannot be used, so one should choose between the two equivalent options: HVMCIM+FC and HVM+CIMFC. The problem here is that these representations do not allow us to separate inconsistent connectedness from contextuality. It may seem therefore that unlike the traditional theory of contextuality, CbD cannot use HVMs as a useful descriptive tool.

However, this difficulty can be easily remedied if one replaces a system under consideration with its consistified equivalent [8,9]. A consistified equivalent R of a system R is a consistently connected system that depicts the same empirical or theoretical situation and is contextual if and only if R is contextual. Specifically, given R in (2.1), R is defined as

R={Rξπ:πC,ξQ,ξπ}, 6.1

where

C={π:π=(,c),cC}{π:π=(q,),qQ}, 6.2
Q={ξ:ξ=(q,c),qQ,cC,qc} 6.3
andξπξ=(q,c)Q&[π=(,c) or π=(q,)]. 6.4

For any context π=(,c), the bunch in this context is defined as

Rπ=R(,c)=dRc. 6.5

To define the bunch for a context π=(q,), we need an auxiliary notion. For a given qQ, define a random variable

Tq={Tqc:cC,qc}, 6.6

such that for any two components Tqc,Tqc in Tq,

Tqc=dRqc, 6.7

and the probability

p[Tqc=Tqc], 6.8

is maximal possible. Let us assume, for simplicity, that such Tq exists and is unique for all qQ.5 Then, for any context π=(q,), the bunch in this context is defined as

Rπ=R(q,)=dTq. 6.9

This completes the construction of R.

Clearly, a consistified system is (strongly) consistently connected: for any ξ=(q,c) it contains two distributional copies of Rqc, in the contexts (,c) and (q,). It should also be clear, by comparing the CbD definition of (non)contextuality with the traditional definition applied to the consistified equivalent of a system, that the system and its equivalent are always contextual or non-contextual together. For a more rigorous argument, see [8].

For our example R0 in (2.2), the consistified equivalent is (omitting commas and brackets to save space)

6. 6.10

where the bunches in the first three rows are distributional copies of the corresponding rows in R0, the distributions of the two variables in each column are identical, and in each of the last four rows the probability of the pairwise equality of its elements is maximal possible.

7. Equivalence theorem for consistified systems

The main reason why the notion of a consistified systems is useful is the fact that the inconsistent connectedness of a system R is eliminated in R (more precisely, translated into the structure of its (q,)-bunches) while its contextuality status is preserved. One can ascertain therefore whether R is describable by HVM+CIM+FC as one can with any other (strongly) consistently connected system. If it is not, then R should be described by either of HVMCIM+FC and HVM+CIMFC, and this time there can be no confusion as to whether they depict inconsistent connectedness or contextuality—it is definitely the latter. However, the applicability of and deviations from HVM+CIM+FC acquire a specific form in the case of consistified systems.

It should be clear from the construction of R that the indexing sets Q(,c) of different (,c)-bunches are disjoint, and that the union of these indexing sets is the entire Q (consisting of all ξ=(q,c) such that qc). This means that we can use the same function to represent all (,c)-bunches,

R(,c)=f(Q(,c),X(,c)(c)), 7.1

where X(,c)(c) for different cC is a set of stochastically unrelated random variables whose distributions may vary with c. By forming an arbitrary coupling X of X(,c)(c) for all cC, we can rewrite this as

R(,c)=s(Q(,c),Proj(,c)X(,c))=t(Q(,c),X(,c),(,c)), 7.2

where X(,c) are stochastically unrelated distributional copies of X. Since Q(,c) uniquely determines (,c), the function can be rewritten as

R(,c)=u(Q(,c),X(,c)). 7.3

By the same argument, for all (q,)-bunches we have

R(q,)=v(Q(q,),Y(q,)). 7.4

The last two formulae represent the HVMGen for consistified systems.

It can be easily shown that one can simplify this HVM by either making the two functions u and v one and the same function or making all X(,c) and X(q,) variables identically distributed. For the latter option, create an arbitrary coupling Φ=(X,Y) and make its distributional copies Φ(,c) and Φ(q,) for all contexts of R. Then

R(,c)=u(Q(,c),Proj1Φ(,c))=ϕ1(Q(,c),Φ(,c)) 7.5

and

R(q,)=v(Q(q,),Proj2Φ(q,))=ϕ2(Q(q,),Φ(q,)). 7.6

This is the form of the HVMCIM+FC for consistified systems: the distribution of the hidden variables is the same for all contexts, but the observable variables depend on the type of the context, (,c)-type or (q,)-type. Thus, the HVMCIM+FC representation for R(,2)=(R322,R422) and R(3,)=(R313,R323,R333) in system R0 in (6.10) are, respectively:

ϕ1(Q(,2),Φ(,2))=(ϕ1((3,2),Φ(,2)),ϕ1((4,2),Φ(,2))) 7.7

and

ϕ2(Q(3,),Φ(3,))=(ϕ2((3,1),Φ(3,)),ϕ2((3,2),Φ(3,)),ϕ2((3,3),Φ(3,))). 7.8

The form of HVM+CIMFC for consistified system obtains by creating arbitrary couplings

Ψ1={R(,c):cC}andΨ2={R(q,):cC}, 7.9

and forming their distributional copies for all (,c)-bunches and (q,)-bunches. Note that both Ψ1 and Ψ2 are indexed by all (q,c)Q. Then

R(,c)=ProjQ(,c)Ψ1(,c)=ψ(Q(,c),Ψ1(,c)) 7.10

and

R(q,)=ProjQ(q,)Ψ2(q,)=ψ(Q(q,),Ψ2(q,)). 7.11

For our example with R(,2)=(R322,R422) and R(3,)=(R313,R323,R333) in (6.10), the HVM+CIMFC representation is

ψ(Q(,2),Ψ1(,2))=(ψ((3,2),Ψ1(,2)),ψ((3,4),Ψ1(,2))) 7.12

and

ψ(Q(3,),Ψ2(3,))=(ψ((3,1),Ψ2(3,)),ψ((3,2),Ψ2(3,)),ψ((3,3),Ψ2(3,))). 7.13

The falsifiable HVM+CIM+FC, describing non-contextual R (hence also, non-contextual R in the CbD sense), is obtained by making HVM+CIMFC and HVMCIM+FC coincide:

R(,c)=ψ(Q(,c),Ψ(,c)) 7.14

and

R(q,)=ψ(Q(q,),Ψ(q,)). 7.15

Using again our example (6.10), the HVM+CIM+FC representation for R(,2)=(R322,R422) and R(3,)=(R313,R323,R333) is

ψ(Q(,2),Ψ(,2))=(ψ((3,2),Ψ(,2)),ψ((3,4),Ψ(,2))) 7.16

and

ψ(Q(3,),Ψ(3,))=(ψ((3,1),Ψ(3,)),ψ((3,2),Ψ(3,)),ψ((3,3),Ψ(3,))). 7.17

8. Hidden assumptions about hidden variables

The literature on HVMs and contextuality contains many attempts to explicate various assumptions underlying HVM+CIM+FC. We have seen that freedom of choice and context-independent mapping, taken separately, are not assumptions, as they are universally satisfiable. We have also seen that their conjunction is restrictive, but that it is conceptually simpler to replace it with a single assumption, one that I dubbed context-irrelevance. I will now briefly discuss two additional propositions that are sometimes presented as assumptions.

Outcome determinism is the assumption that hidden variables and parameters of the situation (contents and contexts) uniquely determine the observable outcomes. Some researchers find this assumption challengeable [21]. Did we not tacitly introduce this assumption somewhere in the course of the development above? The answer is no: once one consistently describes HVMs in the language of random variables, rather than events and their probabilities, outcome determinism is satisfied automatically. Unless one imposes constraints on the possible distributions of Λc(c), either of the two unfalsifiable HVMs we have discussed, say, HVM+CIMFC, can be constructed for any system of random variables. The very fact that the components of Rc are jointly distributed means that there is a random variable of which all these components are measurable functions. This yields the representation (3.4).

Factorisability is another assumption that is often presented as central for HVM+CIM+FC [22]. Its meaning is that, using HVMCIM+FC for definiteness,

p[γ(Qc,Γc,c)=G|Γc=g]=qQcp[ProjqQcγ(Qc,Γc,c)=ProjqQc(G)|Γc=g], 8.1

where G is a set of values indexed by Qc and g is a specific value of Γc. Did we not have to use this assumption? Within our conceptual framework, we did not. Once outcome determinism is accepted as trivially satisfied, factorisability has to be accepted too. Indeed, all probabilities in this equation equal 0 or 1, and the left-hand side probability is 1 if and only if all the right-hand side probabilities are 1.

9. Conclusion

Let us summarise.

  • 1.

    The propositions that are usually presented as the assumption of free choice and the assumption of context-independent mapping in constructing HVMs, when taken separately, are not in fact assumptions. Rather they are two inter-translatable and universally satisfiable ways of describing joint distributions of random variables in a system. Because of their equivalence and their substantive emptiness these notions are mere technical labels in HVMs: one should not take them as saying anything about freedom of choice or physical influences exerted by contexts in the sense in which these notions would be discussed in science or philosophy.

  • 2.

    The conjunction of free choice and context-independent mapping is a falsifiable (and de facto inapplicable to some systems) model. However, rather than being a conjunction of two assumptions (as they were viewed, e.g. in the historic discussion [14]), it is a single assumption in precisely the same sense in which a single sentence can consist of two parts neither of which is a sentence. One can avoid using the terminology of free choice and context-independent mapping altogether, even as technical labels, by interpreting HVM+CIM+FC as an HVM with context-irrelevance: no distributions in this model may depend on context.

  • 3.

    The positions just formulated are obtained almost automatically if one systematically and carefully uses the language of random variables in discussing HVMs. This also allows one to avoid the necessity of certain additional assumptions, such as outcome determinism and factorisability. To utilise the advantages of this language one has to pay meticulous attention to the distinction between jointly distributed variables and stochastically unrelated ones. ‘Hidden variables’ are nothing more than a tool for representing jointly distributed variables as measurable functions defined on the same probability space—which is true essentially by definition. The variables from different contexts, however, cannot be presented as functions of a single source of randomness, even in the HVMs with context-irrelevance: the hidden variables in these models must still be indexed by contexts.

Conflict of interest declaration

The author declares no competing interests.

Acknowledgements

I am grateful to Philippe Grangier, Howard Wiseman and Matthew Jones for thought-provoking discussions. I am indebted to Jerald D. Balakrishnan for critically reading an earlier draft of this paper and suggesting many improvements. I am grateful to Samson Abramsky, Adán Cabello and Paweł Kurzyński for co-organizing with me the meeting on contextuality (QCQMB 2022) at which some of the issues presented in this paper have been the subject of lively discussions.

Footnotes

1

The notation Λc(c) may appear excessive, but it is not. The superscript c merely indicates that the random variables in different contexts are different and stochastically unrelated (this applies to both Λc and Rc). The superscript c therefore is universal and ineliminable. The dependence of Λc on c as an argument means that the distribution of the hidden variable may be different in different contexts. This may or may not be the case in other HVMs.

2

This coupling is a random variable whose components are indexed by c and distributed as the corresponding Λc.

3

In fact, HVM+CIM+FC predicts a more restricted form of consistent connectedness, termed in CbD strong consistent connectedness: for any contexts c,c and any set of contents IQcQc, we should have identically distributed {Rqc:qI} and {Rqc:qI}. A system that is consistently connected but not strongly so is always contextual in CbD.

4

More precisely, the current, second version of CbD, applies to arbitrary systems of dichotomous random variables [5,6]. However, this constraint is not relevant to the present discussion. Moreover, the discussion below could easily be generalised to a class of approaches that include CbD as a special case [8].

5

Reference [8] provides an outline of how the discussion should be modified if this is not the case. If there is more than or less than one Tq for some of the qQ, one should consider a class of consistified systems R, each with one possible combination of the realisations of Tq. This class is deemed non-contextual if and only if at least one of its elements is non-contextual (and this happens if and only if the original system R is non-contextual in the CbD sense). In particular, if the class of Tq is empty for some q, then the class of R is empty, and it should be deemed contextual.

Data accessibility

This article has no additional data.

Declaration of AI use

No AI-assisted technologies were used in creating this article.

Conflict of interest declaration

I declare I have no competing interests.

Funding

This research was partially supported by the Foundational Questions Institute grant FQxI-MGA-2201.

References

  • 1.Bell JS. 1966. On the problem of hidden variables in quantum mechanics. Rev. Mod. Phys. 38, 447. ( 10.1103/RevModPhys.38.447) [DOI] [Google Scholar]
  • 2.Kochen S, Specker EP. 1967. The problem of hidden variables in quantum mechanics. J. Math. Mech. 17, 59-87. ( 10.1512/iumj.1968.17.17004) [DOI] [Google Scholar]
  • 3.Dzhafarov EN. 2022. Context-independent mapping and free choice are equivalent: a general proof. J. Phys. A: Math Theor. 55, 305304. Corrigendum Note in J. Phys. A: Math Theor. 55, 4095020, 2022. ( 10.1088/1751-8121/ac7d20) [DOI] [Google Scholar]
  • 4.Dzhafarov EN. 2022. Contents, contexts, and basics of contextuality. In From electrons to elephants and elections, the frontiers collection (eds Shyam Wuppuluri, Ian Stewart). pp. 259–286. Cham, Switzerland: Springer.
  • 5.Dzhafarov EN, Cervantes VH, Kujala JV. 2017. Contextuality in canonical systems of random variables. Phil. Trans. R. Soc. A 375, 20160389. ( 10.1098/rsta.2016.0389) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kujala JV, Dzhafarov EN. 2021. Contextuality and dichotomizations of random variables. Found. Phys. 52, 13. ( 10.1007/s10701-021-00527-9) [DOI] [Google Scholar]
  • 7.Dzhafarov EN, Cervantes VH, Kujala JV. 2023. Contextuality and informational redundancy. Entropy 25, 6. ( 10.3390/e25010006) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Dzhafarov EN, Kujala JV. 2023. Contextuality with disturbance and without: neither can violate substantive requirements the other satisfies. Entropy 25, 581. ( 10.3390/e25040581) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Dzhafarov EN. 2022. The Contextuality-by-Default view of the Sheaf-Theoretic approach to contextuality. In Samson Abramsky on logic and structure in computer science and beyond, in series outstanding contributions to logic (eds A Palmigiano, M Sadrzadeh). Springer Nature.
  • 10.Clauser JF, Horne MA, Shimony A, Holt RA. 1969. Proposed experiment to test local hidden-variable theories. Phys. Rev. Lett. 23, 880. ( 10.1103/PhysRevLett.23.880) [DOI] [Google Scholar]
  • 11.Cabello A. 2008. Experimentally testable state-independent quantum contextuality. Phys. Rev. Lett. 101, 210401. ( 10.1103/PhysRevLett.101.210401) [DOI] [PubMed] [Google Scholar]
  • 12.Fine A. 1982. Joint distributions, quantum correlations, and commuting observables. J. Math. Phys. 23, 1306-1310. ( 10.1063/1.525514) [DOI] [Google Scholar]
  • 13.Blasiak P, Pothos EM, Yearsley JM, Gallus C, Borsuk E. 2021. Violations of locality and free choice are equivalent resources in Bell experiments. Proc. Natl Acad. Sci. USA 118, e2020569118. ( 10.1073/pnas.2020569118) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Bell JS, Shimony A, Horne MA, Clauser JF. 1985. An exchange on local beables. Dialectica 39, 85-110. ( 10.1111/j.1746-8361.1985.tb01249.x) [DOI] [Google Scholar]
  • 15.Dzhafarov EN, Kujala JV. 2016. Probability, random variables, and selectivity. In The new handbook of mathematical psychology (eds W Batchelder, H Colonius, EN Dzhafarov, J Myung), vol. 1, pp. 85–150. Cambridge, UK: Cambridge University Press.
  • 16.Suppes P, Zanotti M. 1981. When are probabilistic explanations possible? Synthese 48, 191-199. ( 10.1007/BF01063886) [DOI] [Google Scholar]
  • 17.Araújo M, Quintino MT, Budroni C, Cunha MT, Cabello A. 2013. All noncontextuality inequalities for the n-cycle scenario. Phys. Rev. A 88, 022118. ( 10.1103/PhysRevA.88.022118) [DOI] [Google Scholar]
  • 18.Avis D, Fischer P, Hilbert A, Khrennikov A. 2009. Single, Complete, probability spaces consistent with EPR-Bohm–Bell experimental data. In Foundations Probability and Physics-5, AIP Conf. Proc., vol. 1101, pp. 294–301.
  • 19.Khrennikov A. 2015. CHSH inequality: quantum probabilities as classical conditional probabilities. Found. Phys. 45, 711-725. ( 10.1007/s10701-014-9851-8) [DOI] [Google Scholar]
  • 20.Kupczynski M. 2021. Contextuality-by-Default description of Bell tests: contextuality as the rule and not as an exception. Entropy 23, 1104. ( 10.3390/e23091104) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Grangier P. 2021. Contextual inferences, nonlocality, and the incompleteness of quantum mechanics. Entropy 23, 1660. ( 10.3390/e23121660) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Abramsky S, Brandenburger A. 2011. The sheaf-theoretic structure of non-locality and contextuality. New J. Phys. 13, 113 036-113 075. ( 10.1088/1367-2630/13/11/113036) [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

This article has no additional data.


Articles from Philosophical transactions. Series A, Mathematical, physical, and engineering sciences are provided here courtesy of The Royal Society

RESOURCES