Pragmatic Hypotheses in the Evolution of Science

Abstract

This paper introduces pragmatic hypotheses and relates this concept to the spiral of scientific evolution. Previous works determined a characterization of logically consistent statistical hypothesis tests and showed that the modal operators obtained from this test can be represented in the hexagon of oppositions. However, despite the importance of precise hypothesis in science, they cannot be accepted by logically consistent tests. Here, we show that this dilemma can be overcome by the use of pragmatic versions of precise hypotheses. These pragmatic versions allow a level of imprecision in the hypothesis that is small relative to other experimental conditions. The introduction of pragmatic hypotheses allows the evolution of scientific theories based on statistical hypothesis testing to be interpreted using the narratological structure of hexagonal spirals, as defined by Pierre Gallais.

1. Introduction

Standard hypothesis tests can lead to immediate logical incoherence, which makes their conclusions hard to interpret. This incoherence occurs because such tests have only two possible outcomes. Indeed, Izbicki and Esteves [1] shows that there exists no two-valued test that satisfies desirable statistical properties and is also logically coherent.

In order to overcome such an impossibility result, Esteves et al. [2] propose agnostic hypothesis tests, which have three possible outputs: (A) accept the hypothesis, say H, (E) reject H, or (Y) remain agnostic about H. These tests can be made logically coherent while preserving desirable statistical properties. For instance, both conditions are satisfied by the Generalized Full Bayesian Significance Test (GFBST). Furthermore, Stern et al. [3] show that the GFBST’s modal operators and their respective negations can be represented by vertices of the hexagon of oppositions [4,5,6,7,8,9], which is depicted in Figure 1.

Figure 1.

Hexagons of opposition for statistical modalities.

This paper complements the above static representation with an analysis of the GFBST in the dynamic evolution of scientific theories. The analysis is based on the metaphor of evolutive hexagonal spirals [10,11], in which the logical modalities associated to scientific theories change over time, as in Figure 2. Our key point in this paradigm is reconciling two apparently contradictory facts. On the one hand, precise or sharp hypotheses, that is, hypotheses that have a priori zero probability are central in scientific theories [12,13]. On the other hand, the GFBST never accepts (A) precise hypotheses. These observations lead to the apparent paradox that, if the GFBST were used to test scientific theories, then the acceptance step in the spiral of scientific theories would be forfeited.

Figure 2.

Gallais’ evolutionary spiral.

We overcome this paradox by proposing the concept of a “pragmatic hypothesis” associated to a precise hypothesis. Although precise hypotheses are commonly obtained from mathematical theories used in areas of science and technology [12,13], the associated pragmatic hypothesis is an imprecise hypothesis that is sufficiently good from the practical purpose of an end-user of the theories. For instance, Newtonian theory assumes a gravitational force of magnitude given by the equation $F=G{m}_1{m}_2{d}^{-2}$, where the gravitational constant G has a precise value. However, the current Committee on Data for Science and Technology (CODATA) value for the gravitational constant is $G=6.67408\left(31\right)\times {10}^{-11}{\mathrm{m}}^3{\mathrm{kg}}^{-1}{s}^{-2}$, which includes a standard deviation for the last significant digits, $408\pm 31$. Thus, it may be reasonable for a given end-user to assume that the theoretical form of the last equation is exact, but that, pragmatically, the constant G can only be known up to a chosen precision. As a result, one might wish to test an imprecise hypothesis associated to the scientific hypothesis of interest [14,15].

This article advocates for the conceptual distinction between a precise scientific theory and an associated pragmatic hypotheses. The alternate use of precise and pragmatic versions of corresponding statistical hypotheses enables the GFBST to (pragmatically) accept scientific hypotheses. Moreover, this alternate use allows the GFBST to track the evolution of scientific theories, as interpreted in the context of Gallais’ hexagonal spirals.

Our main goal in this paper is to formalize testing procedures for a theory taking into consideration the level of precision that is appropriate for a given end-user. To handle this problem, we consider the end-user’s predictions about an experiment of his interest. The variation in these predictions can be explained by a combination of the level of imprecision in the theory and by properties of the end-user’s experiment. For instance, the latter source of variation is influenced by properties of the equipment, including the precision, accuracy, and resolution of measuring devices [16,17], and also error bounds for fundamental constants and calibration factors [18,19,20,21,22,23,24,25,26]. We propose to choose a pragmatic hypothesis in such a way that the imprecision in the end-user’s predictions is mostly due to his experimental conditions and not due to the level of imprecision in the theory that he uses.

In order to develop this argument, Section 2 first adapts Gallais’s metaphor of hexagonal spirals to the evolution of science. Next, Section 3 proposes three methods of decomposing the variability in an end-user’s predictions into the level of precision of the theory he uses and his experimental conditions. Section 3.1 and Section 3.2 use these decompositions in order to build pragmatic hypothesis. They build pragmatic hypotheses for simple hypotheses and then prove that there exists a single way of extending this construction to composite hypotheses while preserving logical coherence in simultaneous hypothesis testing. This methodology is illustrated in Section 4. All proofs are found in Appendix A.

2. Gallais’ Hexagonal Spirals and the Evolution of Science

Following a well-established tradition in structural semantics and narratology [27,28], Gallais and Pollina [10] propose that many classical medieval tales follow the same organizational pattern. More precisely, these narratives exhibit an underlying “intellectual structure” and are organized according to an underlying archetypal format or prototypical pattern. This pattern includes both static and a dynamical aspects. From a static perspective, the logical structure of the narrative is such that each arch is represented by a vertex of the “hexagon of oppositions” [4]. The static hexagon of oppositions is depicted in Figure 1 and represents in each vertex a modal operator among necessity (□), possibility (⋄), contingency ($\Delta$), and their negations (¬). These modal operators are structured according to three axes of opposition, $(===)$; a triangle of contrariety, $(---)$; another triangle of sub-contrariety, $(\cdots )$; and several edges of subalteration $(⟶$). From a dynamical perspective, the temporal evolution of the narrative follows a spiral (Figure 2) that unwinds (se déroule) around concentric and expanding hexagons of opposition [10,11].

Because the evolution of science can also be conceived as following a spiral pattern [29], its analysis can benefit from the structure in the works by the authors of [10,11]. From a static perspective, the logical modalities induced by agnostic hypothesis tests [3] can be represented in the hexagon of oppositions. From a dynamic perspective, scientific theories evolve as a spiral which unwinds around the following states:

• $A_1$- Extant thesis: This vertex represents a standing paradigm, an accepted theory using well-known formalisms and familiar concepts, relying on accredited experimental means and methods, etc. In fact, the concepts of a current paradigm may become so familiar and look so natural that they become part of a reified ontology. That is, there is a perceived correspondence between concepts of the theory and “dinge-an-sich” (things-in-themselves) as seen in nature [29,30].

• $U_1$- Analysis: This vertex represents the moment when some hypotheses of the standing theory are put in question. At this moment, possible alternatives to the standing hypotheses may still be only vaguely defined.

• $E_1$- Antithesis: This vertex represents the moment when some laws of the standing theory have to be rejected. Such a rejection of old laws may put in question the entire world-view of the current paradigm, opening the way for revolutionary ideas, as described in the next vertex.

• $O_2$- Apothesis/ Prosthesis: This vertex is the locus of revolutionary freedom. Alternative models are considered, and specific (precise) forms investigated. There is intellectual freedom to set aside and dispose of (apothesis) old preconceptions, prejudices and stereotypes, and also to explore and investigate new paths, to put together (prosthesis) and try out new concepts and ideas.

• $Y_2$- Synthesis: It is at this vertex that new laws are formulated; this is the point of Eureka moment(s). A selection of old and and new concepts seem to click into place, fitting together in the form of new laws, laws that are able to explain new phenomena and incorporate objects of an expanded reality.

• $I_2$- Enthesis: At this vertex new laws, concepts and methods must enter and be integrated into a consistent and coherent system. At this stage many tasks are performed in order to combine novel and traditional pieces or to accommodate original and conventional components into an well-integrated framework. Finally, new experimental means and methods are developed and perfected, allowing the new laws to be corroborated.

• $A_2$- New Thesis: At this vertex, the new theory is accepted as the standard paradigm that succeeds the preceding one ($A_1$). Acceptance occurs after careful determination of fundamental constants and calibration factors (including their known precision), metrological and instrumentational error bounds, etc. At later stages of maturity, equivalent theoretical frameworks may be developed using alternative formalisms and ontologies. For example, analytical mechanics offers variational alternatives that are (almost) equivalent to the classical formulation of Newtonian mechanics [31]. Usually, these alternative worldviews reinforce the trust and confidence on the underlying laws. Nevertheless, the existence of such alternative perspectives may also foster exploratory efforts and investigative works in the next cycle in evolution.

Table 1 applies this spiral structure to the evolution of the theories of orbital astronomy and chemical affinity. The evolution of orbital astronomy has been widely studied [32]. The evolution of chemical affinity is presented in greater detail in Stern [29], Stern and Nakano [33].

Table 1.

Evolution of orbital astronomy and chemical affinity.

Vertex	Orbital Astronomy	Chemical Affinity
$I_1$- Enthesis/ $A_1$- Thesis	Ptolemaic/Copernican cycles and epicycles	Geoffroy affinity table and highest rank substitution
$U_1$- Analysis	Circular or oval orbits?	Ordinal or numeric affinity?
$E_1$- Antithesis	Non-circular orbits	Non-ordinal affinity
$O_2$- Apothesis /Prosthesis	Elliptic planetary orbits, focal centering of sun	Integer affinity values, for arithmetic recombination
$Y_2$- Synthesis	Kepler laws!	Morveau rules and tables!
$I_2$- Enthesis $A_2$- Thesis	Vortex physics theories, Keplerian astronomy	Affinity + stoichiometry substitution reactions
$U_2$- Analysis	Tangential or radial forces?	Total or partial reaction?
$E_2$- Antithesis	Non-tangential forces	Non-total substitutions
$O_3$- Apothesis/Prosthesis	Radial attraction forces, inverse square of distance	Reversible reactions, equilibrium conditions
$Y_3$- Synthesis	Newton laws!	Mass-Action kinetics!
$I_3$- Enthesis/ $A_3$- Thesis	Newtonian mechanics & variational equivalents	Thermodynamic theories for reaction networks

The above spiral structure highlights that a statistical methodology should be able to obtain each of the six modalities in the hexagon of oppositions. Before an acceptance vertex (A) in the hexagon is reached by the spiral of scientific evolution, theoretically precise or sharp hypotheses must be formulated. However, a logically coherent hypothesis test, such as the GFBST, can choose solely between rejecting or remaining agnostic (i.e., corroborating) such sharp hypotheses. Once the evolving theory becomes (part of) a well-established paradigm, the GFBST can be used with the goal of accepting non-sharp hypotheses in the context of the same paradigm, a context that includes fundamental constants and calibration factors (and their respective uncertainties), metrological error bounds, specified accuracies of scientific intrumentation, etc. The non-sharp versions of sharp hypotheses used in such tests are called pragmatic, and their formulation is developed in the following sections.

3. Pragmatic Hypotheses

In order to derive pragmatic hypotheses from precise ones, it is necessary to define an idealized future experiment. Let $\theta$ be an unknown parameter of interest, which is used to express scientific hypotheses and that takes values in the parameter space, $\Theta$. A scientific hypothesis takes the form $H_0:\theta \in {\Theta }_0$, where ${\Theta }_0\subset \Theta$. Whenever there is no ambiguity, $H_0$ and ${\Theta }_0$ are used interchangeably. Also, the determination of $\theta$ is useful for predicting an idealized future experiment, $\mathbf{Z}$, which takes values in $\mathcal{Z}$. The uncertainty about Z depends on $\theta$ by means of $P_{{\theta }^{*}}$, the probability measure over $\mathcal{Z}$ when it is known that $\theta ={\theta }^{*}$, ${\theta }^{*}\in \Theta$.

Often, it is sufficient for an end-user to determine a pragmatic hypothesis, that is, that the parameter lies in a set of plausible values, which is larger than the null hypothesis. This set can be chosen in such a way that the variation over predictions about a future experiment is mostly due to experimental conditions rather than to the imprecision in the value of the parameter. This section formally develops a methodology for determining these pragmatic hypotheses.

In order to compare two parameter values, we use a “predictive dissimilarity”, $d_{\mathbf{Z}}$, which is a function, $d_{\mathbf{Z}}:\Theta \times \Theta \to {\mathbb{R}}^{+}$, such that $d_{\mathbf{Z}}({\theta }_0,{\theta }^{*})$ measures how much the predictions made for $\mathbf{Z}$ based on ${\theta }^{*}$ diverge from the ones made based on ${\theta }_0$. We define and compare three possible choices for such a dissimilarity.

Definition 1.

The Kullback–Leibler predictive dissimilarity, ${KL}_{\mathbf{Z}}$, is

$$\begin{array}{cc}\hfill {KL}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =KL({\mathbb{P}}_{{\theta }^{*}},{\mathbb{P}}_{{\theta }_0})={\int }_{\mathcal{Z}}log\left(\frac{d{\mathbb{P}}_{{\theta }^{*}}}{d{\mathbb{P}}_{{\theta }_0}}\right)d{\mathbb{P}}_{{\theta }^{*}},\hfill \end{array}$$

that is, ${KL}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})$ is the relative entropy between ${\mathbb{P}}_{{\theta }^{*}}$ and ${\mathbb{P}}_{{\theta }_0}$.

Example 1

(Gaussian with known variance.). Let $\mathbf{Z}=(Z_1\dots ,Z_d)\sim N(\theta ,{\Sigma }_0)$ be a random vector with a multivariate Gaussian distribution:

$$\begin{array}{cc}\hfill \frac{d{\mathbb{P}}_{\theta }\left(\mathbf{z}\right)}{d\mathbf{z}}& =\parallel 2\pi {\Sigma }_0{\parallel }^{-0.5}exp\left(-0.5{(\mathbf{z}-\theta )}^t{\Sigma }_0^{-1}d(\mathbf{z}-\theta )\right)\hfill \\ \hfill {KL}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& ={\int }_{{\mathbb{R}}^d}log\left(\frac{d{\mathbb{P}}_{{\theta }^{*}}\left(\mathbf{z}\right)}{d{\mathbb{P}}_{{\theta }_0}\left(\mathbf{z}\right)}\right)d{\mathbb{P}}_{{\theta }^{*}}\left(\mathbf{z}\right)=0.5{({\theta }_0-{\theta }^{*})}^t{\Sigma }_0^{-1}({\theta }_0-{\theta }^{*}),\hfill \end{array}$$

When $d=1$ and ${\Sigma }_0={\sigma }_0^2$,

$$\begin{array}{cc}\hfill K{L}_{\mathbf{z}}({\theta }_0,{\theta }^{*})& =\frac{{({\theta }_0-{\theta }^{*})}^2}{2{\sigma }_0^2}\hfill \end{array}$$ (1)

The KL dissimilarity evaluates the distance between the predictive probability distributions for the future experiment under two parameter values, ${\theta }_0$ and ${\theta }^{*}$. Although the KL dissimilarity is general, it can be challenging to interpret. In particular, it can be hard to establish the quality of the predictions for $\mathbf{Z}$ based on ${\theta }^{*}$ when $\mathbf{Z}$ is actually generated from ${\theta }_0$ and $K{L}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})\le ϵ$. A more interpretable dissimilarity is obtained by taking $d_{\mathbf{Z}}({\theta }_0,{\theta }^{*})$ to measure how far are the best predictions for $\mathbf{Z}$ based on ${\theta }^{*}$ and ${\theta }_0$. In this case, if one makes a prediction for $\mathbf{Z}$ based on ${\theta }^{*}$, ${\mathbf{z}}_{*}$, and $\mathbf{Z}$ was actually generated using ${\theta }_0$, then $d_{\mathbf{Z}}({\theta }_0,{\theta }^{*})\le ϵ$ guarantees that ${\mathbf{z}}_{*}$ will be at most $ϵ$ apart from the best possible prediction. Such a dissimilarity is discussed in the following definition.

Definition 2

(Best prediction dissimilarity—BP.). Let $\widehat{\mathbf{Z}}:\Theta \to \mathcal{Z}$ be such that $\widehat{\mathbf{Z}}\left({\theta }_0\right)$ is the best prediction for $\mathbf{Z}$ given that $\theta ={\theta }_0$. For example, one can take

$$\begin{array}{cc}\hfill \widehat{\mathbf{Z}}\left({\theta }_0\right)& =arg\underset{\mathbf{z}\in \mathcal{Z}}{min}{\delta }_{\mathbf{Z},{\theta }_0}\left(\mathbf{z}\right),\hfill \end{array}$$

where ${\delta }_{\mathbf{Z},{\theta }_0}:\mathcal{Z}\to \mathbb{R}$ is such that ${\delta }_{\mathbf{Z},{\theta }_0}\left(\mathbf{z}\right)$ measures how bad $\mathbf{z}$ predicts $\mathbf{Z}$ when $\theta ={\theta }_0$. The “best prediction dissimilarity”, ${BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})$, measures how badly $\widehat{\mathbf{Z}}\left({\theta }^{*}\right)$ predicts $\mathbf{Z}$ relatively to $\widehat{\mathbf{Z}}\left({\theta }_0\right)$ when $\theta ={\theta }_0$. Formally,

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =g\left(\frac{{\delta }_{\mathbf{Z},{\theta }_0}\left(\widehat{\mathbf{Z}}\left({\theta }^{*}\right)\right)-{\delta }_{\mathbf{Z},{\theta }_0}\left(\widehat{\mathbf{Z}}\left({\theta }_0\right)\right)}{{\delta }_{\mathbf{Z},{\theta }_0}\left(\widehat{\mathbf{Z}}\left({\theta }_0\right)\right)}\right),\hfill \end{array}$$

where $g:\mathbb{R}⟶\mathbb{R}$ is a motononic function. The choice of g in a particular setting aims at improving the interpretation of the best prediction dissimilarity criterion.

Example 2

(BP under quadratic form.). Let $\mathcal{Z}={\mathbb{R}}^d$, ${\mu }_{\mathbf{Z},\theta }=\mathbf{E}\left[\mathbf{Z}\right|\theta ]$, ${\Sigma }_{\mathbf{Z},\theta }=\mathbb{V}\left[\mathbf{Z}\right|\theta ]$ and $\mathbf{S}$ be a positive definite matrix. Define the quadratic form induced by $\mathbf{S}$ to be ${\parallel \mathbf{z}\parallel }_{\mathbf{S}}^2={\mathbf{z}}^T\mathbf{S}\mathbf{z}$ and

$$\begin{array}{c}\hfill {\delta }_{\mathbf{Z},{\theta }_0}\left(\mathbf{z}\right)=\mathbf{E}\left[{\parallel \mathbf{Z}-\mathbf{z}\parallel }_{\mathbf{S}}^2|\theta ={\theta }_0\right]\end{array}$$

The optimal prediction under ${\theta }^{*}$ is $\widehat{\mathbf{Z}}\left({\theta }^{*}\right)={\mu }_{\mathbf{Z},{\theta }^{*}}$. It follows that

$$\begin{array}{cc}\hfill {\delta }_{\mathbf{Z},{\theta }_0}\left(\widehat{\mathbf{Z}}\left({\theta }^{*}\right)\right)& =\mathbf{E}\left[\parallel \mathbf{Z}-{\mu }_{\mathbf{Z},{\theta }^{*}}{\parallel }_{\mathbf{S}}^2|\theta ={\theta }_0\right]\hfill \\ \hfill & =\parallel {\mu }_{\mathbf{Z},{\theta }_0}-{\mu }_{\mathbf{Z},{\theta }^{*}}{\parallel }_{\mathbf{S}}^2+\mathbf{E}\left[\parallel \mathbf{Z}-{\mu }_{\mathbf{Z},{\theta }_0}{\parallel }_{\mathbf{S}}^2|\theta ={\theta }_0\right]\hfill \end{array}$$

In particular, ${\delta }_{\mathbf{Z},{\theta }_0}\left(\widehat{\mathbf{Z}}\left({\theta }_0\right)\right)=\mathbf{E}\left[\parallel \mathbf{Z}-{\mu }_{\mathbf{Z},{\theta }_0}{\parallel }_{\mathbf{S}}^2|\theta ={\theta }_0\right]$. Therefore,

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =g\left(\frac{\parallel {\mu }_{\mathbf{Z},{\theta }_0}-{\mu }_{\mathbf{Z},{\theta }^{*}}{\parallel }_{\mathbf{S}}^2}{\mathbf{E}\left[\parallel \mathbf{Z}-{\mu }_{\mathbf{Z},{\theta }_0}{\parallel }_{\mathbf{S}}^2|\theta ={\theta }_0\right]}\right)\hfill \end{array}$$ (2)

In this example, BP${}_{\mathbf{Z}}$ can be put in the same scale as $\mathbf{Z}$ by taking $g\left(x\right)=\sqrt{x}$. Also, two choices of $\mathbf{S}$ are of particular interest. When $\mathbf{S}=\mathbb{V}{\left[\mathbf{Z}\right|\theta ={\theta }_0]}^{-1}$, Equation (2) simplifies to

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =g\left(d^{-1}{\parallel {\mu }_{\mathbf{Z},{\theta }_0}-{\mu }_{\mathbf{Z},{\theta }^{*}}\parallel }_{{\Sigma }_{\mathbf{Z},{\theta }_0}^{-1}}^2\right)\hfill \end{array}$$ (3)

Similarly, when $\mathbf{S}$ is the identity matrix, Equation (2) simplifies to

$$\begin{array}{c}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})=g\left(\frac{\parallel \mathbf{E}\left[\mathbf{Z}\right|\theta ={\theta }_0]-\mathbf{E}\left[\mathbf{Z}\right|\theta ={\theta }^{*}]{\parallel }_2^2}{tr\left(\mathbb{V}\left[\mathbf{Z}\right|\theta ={\theta }_0]\right)}\right)\end{array}$$ (4)

Equation (4) admits an intuitive interpretation. The larger the value of $tr\left(\mathbb{V}\left[\mathbf{Z}\right|\theta ={\theta }_0]\right)$, the more $\mathbf{Z}$ is dispersed and the harder it is to predict its value. Also, $\parallel \mathbf{E}\left[\mathbf{Z}\right|\theta ={\theta }_0]-\mathbf{E}\left[\mathbf{Z}\right|\theta ={\theta }^{*}]{\parallel }_2^2$ measures how far apart are the best prediction for $\mathbf{Z}$ under $\theta ={\theta }_0$ and $\theta ={\theta }^{*}$. That is, ${\mathrm{BP}}_Z({\theta }_0,{\theta }^{*})$ captures that, if one predicts $\mathbf{Z}$ assuming that $\theta ={\theta }^{*}$ when it is actually $\theta ={\theta }_0$, then the error with respect to the best prediction is increased as a function of the distance between the predictions over the dispersion of $\mathbf{Z}$.

Example 3

(Gaussian with known variance.). Consider Example 1 and let ${\delta }_{\mathbf{Z},{\theta }_0}\left(\mathbf{z}\right)$ be as in Example 2. It follows from Equation (4) that when $\mathbf{S}$ is the identity matrix,

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =g\left(\frac{\parallel {\theta }_0-{\theta }^{*}{\parallel }_2^2}{tr\left({\Sigma }_0\right)}\right)\hfill \end{array}$$ (5)

Similarly, it follows from Equation (3) that when $\mathbf{S}={\Sigma }_0^{-1}$,

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =g\left(d^{-1}{({\theta }_0-{\theta }^{*})}^t{\Sigma }_0^{-1}({\theta }_0-{\theta }^{*})\right)\hfill \end{array}$$ (6)

Conclude from Equation (6) that, if $\mathbf{S}={\Sigma }_0^{-1}$ and $g\left(x\right)=x$, then ${BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})=2d^{-1}{KL}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})$. Also, when $d=1$, ${\Sigma }_0={\sigma }_0^2$ and $g\left(x\right)=\sqrt{x}$, both Equations (5) and (6) simplify to

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& ={\sigma }_0^{-1}|{\theta }_0-{\theta }^{*}|\hfill \end{array}$$ (7)

In some situations, $\mathbf{Z}$ is the average of m independent observations distributed as $N(\theta ,{\Sigma }_0)$. In this case, $\mathbf{Z}\sim N(\theta ,m^{-1}{\Sigma }_0)$. It follows from Equation (5) that $B{P}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})=g\left(\frac{m\parallel {\theta }_0-{\theta }^{*}{\parallel }_2^2}{tr\left({\Sigma }_0\right)}\right)$ when $\mathbf{S}$ is the identity, and ${BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})=g\left(m{d}^{-1}{({\theta }_0-{\theta }^{*})}^t{\Sigma }_0^{-1}({\theta }_0-{\theta }^{*})\right)$ when $\mathbf{S}={\Sigma }_0^{-1}$.

Although ${\mathrm{BP}}_{\mathbf{Z}}$ is more interpretable then ${\mathrm{KL}}_{\mathbf{Z}}$, it also relies on more tuning variables, such as $\delta$, $\widehat{Z}$, and g. A balance between these features is obtained by a third predictive dissimilarity, which evaluates how easy it is to recover the value of $\theta$ between ${\theta }_0$ or ${\theta }^{*}$ based on $\mathbf{Z}$.

Definition 3

(Classification distance—CD.). Let ${\widehat{\theta }}_{{\theta }_0,{\theta }^{*}}:\mathcal{Z}\to \Theta$ be such that

$$\begin{array}{cc}\hfill {\widehat{\theta }}_{{\theta }_0,{\theta }^{*}}\left(\mathbf{z}\right)& =arg\underset{\theta \in \{{\theta }_0,{\theta }^{*}\}}{max}{f}_{\mathbf{Z}}\left(\mathbf{z}\right|\theta )\hfill \end{array}$$

${\widehat{\theta }}_{{\theta }_0,{\theta }^{*}}$ assigns to each possible outcome of the future experiment $\mathbf{z}$, in which the values of θ, ${\theta }_0$, or ${\theta }^{*}$ make the experimental result more likely. The classification distance between ${\theta }_0$, ${\theta }^{*}$, and $CD({\theta }_0,{\theta }^{*})$ is defined as

$$\begin{array}{cc}\hfill CD({\theta }_0,{\theta }^{*})& =0.5\mathbb{P}\left({\widehat{\theta }}_{{\theta }_0,{\theta }^{*}}\left(\mathbf{Z}\right)={\theta }_0|{\theta }_0\right)+0.5\mathbb{P}\left({\widehat{\theta }}_{{\theta }_0,{\theta }^{*}}\left(\mathbf{Z}\right)={\theta }^{*}|{\theta }^{*}\right)-0.5\hfill \end{array}$$

$CD({\theta }_0,{\theta }^{*})+0.5$ is the best Bayes utility in an hypothesis test of ${\theta }_0$ against ${\theta }^{*}$ using a uniform prior for θ and the 0/1 utility [15]. By subtracting $0.5$ from this quantity, $CD({\theta }_0,{\theta }^{*})$ varies between 0 and $0.5$ and is a distance. Also,

$$\begin{array}{cc}\hfill CD({\theta }_0,{\theta }^{*})& =0.5TV({\mathbb{P}}_{{\theta }_0},{\mathbb{P}}_{{\theta }^{*}})=0.25{\parallel {\mathbb{P}}_{{\theta }_0}-{\mathbb{P}}_{{\theta }^{*}}\parallel }_1,\hfill \end{array}$$

where $TV({\mathbb{P}}_{{\theta }_0},{\mathbb{P}}_{{\theta }^{*}})={sup}_A|{\mathbb{P}}_{{\theta }_0}\left(A\right)-{\mathbb{P}}_{{\theta }^{*}}\left(A\right)|$ and $\parallel {\mathbb{P}}_{{\theta }_0}-{\mathbb{P}}_{{\theta }^{*}}{\parallel }_1={\int }_{\mathcal{Z}}|{\mathbb{P}}_{{\theta }_0}\left(z\right)-{\mathbb{P}}_{{\theta }^{*}}\left(z\right)|dz$ is the ${\mathcal{L}}_1$-distance between probability measures.

Example 4

(Gaussian with known variance.). Consider Examples 1 and 3, when $d=1$, ${\Sigma }_0={\sigma }_0^2$, obtain

$$\begin{array}{cc}\hfill {CD}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =\Phi \left(\frac{|{\theta }_0-{\theta }^{*}|}{2{\sigma }_0}\right)-\frac{1}{2}\hfill \end{array}$$ (8)

Note that, in this case, $CD$ would be the same as $BP$ if, instead of taking $g\left(x\right)=\sqrt{x}$, one chose $g\left(x\right)=\Phi \left(0.5\sqrt{x}\right)-0.5$.

Although analytical expressions for CD are generally not available, it is possible to approximate it via numerical integration methods.

The choice between predictive dissimilarity functions depends on the type of guarantee the end-user wishes to obtain. In particular, $BP$, $KL$, and $CD$ is not an exhaustive list of dissimilarities. However, some of their properties can be useful in obtaining a choice. For instance, although $BP$ yields a metric on the parameter space, $KL$ and $CD$ are dissimilarities between probability functions. That is, although $BP$ will generate pragmatic hypothesis that have parameter values numerically close to a given ${\theta }_0$, $KL$ and $CD$ will yield pragmatic hypothesis that have parameter values lead to similar prediction about $\mathbf{Z}$. Also, although $KL$ evaluates similarity between predictions from an information theoretic perspective, $CD$ evaluates them from a perspective of hypothesis tests.

3.1. Singleton Hypotheses

We start by defining the pragmatic hypothesis associated to a singleton hypothesis. A singleton hypothesis is one in which the parameter assumes a single value, such as $H_0:\theta ={\theta }_0$. In this case, the pragmatic hypothesis associated to $H_0$ is the set of points whose dissimilarity to ${\theta }_0$ is at most $ϵ$, as formalized below.

Definition 4

(Pragmatic hypothesis for a singleton.). Let $H_0:\theta ={\theta }_0$, $d_{\mathbf{Z}}$ be a predictive dissimilarity function and $ϵ>0$. The pragmatic hypothesis for $H_0$, $Pg(\left\{{\theta }_0\right\},d_{\mathbf{Z}},ϵ)$, is

$$\begin{array}{cc}\hfill Pg(\left\{{\theta }_0\right\},d_{\mathbf{Z}},ϵ)& =\{{\theta }^{*}\in \Theta :d_{\mathbf{Z}}({\theta }_0,{\theta }^{*})\le ϵ\}\hfill \end{array}$$

Note that for ${ϵ}_1<{ϵ}_2$, $Pg(\left\{{\theta }_0\right\},d_{\mathbf{Z}},{ϵ}_1)\subseteq Pg(\left\{{\theta }_0\right\},d_{\mathbf{Z}},{ϵ}_2)$.

Example 5

(Gaussian with known variance). Consider Examples 1 and 3 when $d=1$, ${\Sigma }_0={\sigma }_0^2$ and $g\left(x\right)=\sqrt{x}$. It follows from Equations (1), (7) and (8) that

$$\begin{array}{cc}\hfill Pg(\left\{{\theta }_0\right\},B{P}_{\mathbf{Z}},ϵ)& =\left[{\theta }_0-ϵ{\sigma }_0,{\theta }_0+ϵ{\sigma }_0\right]\hfill \\ \hfill Pg(\left\{{\theta }_0\right\},K{L}_{\mathbf{Z}},ϵ)& =\left[{\theta }_0-\sqrt{2ϵ}{\sigma }_0,{\theta }_0+\sqrt{2ϵ}{\sigma }_0\right]\hfill \\ \hfill Pg(\left\{{\theta }_0\right\},C{D}_{\mathbf{Z}},ϵ)& =\left[{\theta }_0-2{\Phi }^{-1}(0.5+ϵ){\sigma }_0,{\theta }_0+2{\Phi }^{-1}(0.5+ϵ){\sigma }_0\right]\hfill \end{array}$$

Note that the size of each of the pragmatic hypothesis is proportional to ${\sigma }_0$. This occurs because each predictive dissimilarity functions makes the prediction error due to the unknown parameter value small with respect to that due to the data variability, ${\sigma }_0^2$.

3.2. Composite Hypotheses

Next, we consider pragmatic hypotheses for general hypotheses $H_0:\theta \in {\Theta }_0$, where ${\Theta }_0\subset \Theta$.

Definition 5.

For each hypothesis ${\Theta }_0\subseteq \Theta$, predictive dissimilarity $d_{\mathbf{Z}}$ and $ϵ>0$, $Pg({\Theta }_0,d_{\mathbf{Z}},ϵ)$ is the pragmatic hypothesis associated to ${\Theta }_0$ induced by $d_{\mathbf{Z}}$ and ϵ. Whenever $d_{\mathbf{Z}}$ and ϵ are clear or not relevant to the result, we write $Pg\left({\Theta }_0\right)$ instead of $Pg({\Theta }_0,d_{\mathbf{Z}},ϵ)$.

In order to construct these pragmatic hypotheses, we use logically coherent agnostic hypothesis tests. For each hypothesis, an agnostic hypothesis test can either reject it (1), accept it (0), or remain agnostic (1/2) [34]. Esteves et al. [2] shows that an agnostic hypothesis test is logically coherent if and only if it is based on a region estimator. Such tests are presented in Definition 7 and illustrated in Figure 3.

Definition 6.

Let $\mathcal{X}$ denote the sample space of the data used to test a hypothesis. A region estimator is a function, $R:\mathcal{X}⟶\mathcal{P}(\Theta )$, where $\mathcal{P}(\Theta )$ is the power set of Θ.

Definition 7

(Agnostic test based on a region estimator.). The agnostic test based on the region estimator R for testing $H_0$, ${\varphi }_{H_0}^R$, such that

$$\begin{array}{cc}\hfill {\varphi }_{H_0}^R\left(x\right)& =\left\{\begin{array}{cc}0\hfill & ,ifR\left(x\right)\subseteq H_0\hfill \\ 1\hfill & ,ifR\left(x\right)\subseteq H_0^c\hfill \\ \frac{1}{2}\hfill & ,otherwise.\hfill \end{array}\right.\hfill \end{array}$$

Figure 3.

$\varphi \left(x\right)$ is an agnostic test based on the region estimator $R\left(x\right)$ for testing $H_0$.

Besides the logical conditions on the hypothesis test, one might also impose logical restraints on how pragmatic hypotheses are constructed. For instance, let A and B be two hypothesis, such that B logically entails A, that is, $B\subseteq A$. If a logically coherent test accepts B, then it also accepts A. This property is called monotonocity [1,35,36,37]. One might also impose that $Pg$, such that if a logically coherent hypothesis test accepts $Pg\left(B\right)$, then it should also accept $Pg\left(A\right)$. Similarly, let ${\left(A_i\right)}_{i\in I}$ be a collection of hypothesis which cover A, that is, $A\subseteq {\cup }_{i\in I}{A}_i$. If a logically coherent hypothesis test rejects every $A_i$, then it rejects A. This property is called union consonance. One might also impose that $Pg$ is such that, if a logically coherent hypothesis test rejects $Pg\left(A_i\right)$ for every i, then it should also reject $Pg\left(A\right)$. The above conditions define the logical coherence of a procedure for constructing pragmatic hypotheses.

Definition 8.

A procedure for constructing pragmatic hypothesis, $Pg$, is logically coherent if, for every logically coherent hypothesis test ϕ and sample point x:

1. If ${\varphi }_{Pg\left(B\right)}\left(x\right)=0$ for some $B\subseteq A$, then ${\varphi }_{Pg\left(A\right)}\left(x\right)=0$.

2. If ${\varphi }_{Pg\left(A_i\right)}\left(x\right)=1$ for every $i\in I$ and $A\subseteq {\cup }_{i\in I}{A}_i$, then ${\varphi }_{Pg\left(A\right)}\left(x\right)=1$.

In order to motivate the above definition, consider that the frequencies of $AA$, $AB$, and $BB$ in a given population are ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$, respectively. Note that $B:=\{0.25,0.5,0.25\}$ is a subset of $A=\{(p^2,2p(1-p),{(1-p)}^2):p\in [0,1]\}$, which denotes the Hardy–Weinberg equilibrium. That is, if the frequencies $AA$, $AB$, and $BB$ are, respectively, 0.25, 0.5, and 0.25, then the population follows the Hardy–Weinberg equilibrium. As a result, if one pragmatically accepts that the population satisfies the specified proportions, then one might also wish to pragmatically accept that the population follows the Hardy–Weinberg. Similarly, if one pragmatically rejects for every $p\in [0,1]$ that the frequencies of $AA$, $AB$, and $BB$ are, respectively, $p^2$, $2p(1-p)$, and ${(1-p)}^2$, then one might also wish to pragmatically reject that the population follows the Hardy–Weinberg equilibrium. These conditions are assured in Definition 8.

In a logically coherent procedure for constructing pragmatic hypotheses, the pragmatic hypothesis associated to a composite hypothesis is completely determined by the pragmatic hypotheses associated to simple hypotheses. This result is presented in Theorem 1.

Theorem 1.

A procedure for constructing pragmatic hypothesis, $Pg$, is logically coherent if and only if, for every hypothesis, ${\Theta }_0$, $Pg\left({\Theta }_0\right)={\bigcup }_{\theta \in {\Theta }_0}Pg\left(\left\{\theta \right\}\right)$.

Using Theorem 1, it is possible to determine a logically coherent procedure for constructing pragmatic hypotheses by determining only the pragmatic hypothesis associated to simple hypothesis, such as in Section 3.1. Theorem 1 is illustrated in Section 4. One can also obtain the following general relation between predictive dissimilarities.

Lemma 1.

$Pg({\Theta }_0,K{L}_{\mathbf{Z}},8{ϵ}^2)\subseteq Pg({\Theta }_0,C{D}_{\mathbf{Z}},ϵ)$.

Besides being logically coherent, it is often desirable in statistics [38,39] and in science [12,13] for a procedure to be invariant to reparametrization, so as to ensure that the procedure reaches the same conclusions whatever the coordinate system is used to specify both the sample and the parameter spaces. For instance, the pragmatic hypothesis that is obtained using the International metric system should be compatible to the one that is obtained using the English metric system. Invariance to reparametrization is formally presented in Definition 10.

Definition 9.

${\left({\mathbb{P}}_{{\theta }^{*}}^{*}\right)}_{{\theta }^{*}\in {\Theta }^{*}}$ is a reparameterization of ${\left({\mathbb{P}}_{\theta }\right)}_{\theta \in \Theta }$ if there exists a bijective function, $f:\Theta \to {\Theta }^{*}$, such that for every $\theta \in \Theta$, ${\mathbb{P}}_{\theta }={\mathbb{P}}_{f\left(\theta \right)}^{*}$.

Definition 10.

Let ${\left({\mathbb{P}}_{{\theta }^{*}}^{*}\right)}_{{\theta }^{*}\in {\Theta }^{*}}$ be a reparametrization of ${\left({\mathbb{P}}_{\theta }\right)}_{\theta \in \Theta }$ by a bijective function, $f:\Theta \to {\Theta }^{*}$. Also, let $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ be predictive dissimilarity functions. The functions $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ are invariant to the reparametrization if for every logically coherent procedure for constructing pragmatic hypotheses, $Pg$,

$$\begin{array}{cc}\hfill f\left[Pg({\Theta }_0,d_{\mathbf{Z}},ϵ)\right]& =Pg(f\left[{\Theta }_0\right],d_{\mathbf{Z}}^{*},ϵ),\hfill \end{array}$$

Definition 10 states that, if ${\Theta }_0$ is an hypothesis and invariance to reparametrization holds, then the pragmatic hypothesis obtained in a reparametrization of ${\Theta }_0$, say $Pg\left(f\left[{\Theta }_0\right]\right)$, is the same as the transformed pragmatic hypothesis associated to ${\Theta }_0$, $f\left[Pg\left({\Theta }_0\right)\right]$. Theorem 2 presents a sufficient condition for obtaining invariance to reparametrization.

Theorem 2.

Let ${\left({\mathbb{P}}_{{\theta }^{*}}^{*}\right)}_{{\theta }^{*}\in {\Theta }^{*}}$ be a reparameterization of ${\left({\mathbb{P}}_{\theta }\right)}_{\theta \in \Theta }$ given by a bijective function, f. If $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ satisfy $d_{\mathbf{Z}}({\theta }_0,\theta )=d_{\mathbf{Z}}^{*}(f\left({\theta }_0\right),f\left(\theta \right))$, then $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ are invariant to this reparametrization.

Corollary 1.

If $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ are the same choice between KL, BP, or CD, then $d_{\mathbf{Z}}$ and $d_{\mathbf{Z}}^{*}$ are invariant to every reparametrization.

The procedures for constructing pragmatic hypotheses induced by $KL$ and $CD$ also satisfy an additional property given by Theorem 3.

Theorem 3.

Let ${\mathbf{Z}}_m=(Z_1,\dots ,Z_m)$, where $Z_i$’s are i.i.d. $F_{\theta }$ and ${\left(F_{\theta }\right)}_{\theta \in \Theta }$ is identifiable [40,41]. Also, let $K{L}_m$ and $C{D}_m$ be the dissimilarities calculated using ${\mathbf{Z}}_m$. If $Pg$ is logically coherent, then, for every ${\Theta }_0\subseteq \Theta$ and $ϵ>0$,

• (i) ${\left(Pg({\Theta }_0,K{L}_m,ϵ)\right)}_{m\ge 1}$ and ${\left(Pg({\Theta }_0,C{D}_m,ϵ)\right)}_{m\ge 1}$ are non-increasing sequences of sets
• (ii) $Pg({\Theta }_0,K{L}_m,ϵ)\stackrel{m\to \infty }{\to }{\Theta }_0$ and $Pg({\Theta }_0,C{D}_m,ϵ)\stackrel{m\to \infty }{\to }{\Theta }_0$.

Theorem 3 states that the sequence of pragmatic hypotheses for ${\Theta }_0$ induced by $d_{{\mathbf{Z}}_m}$ is non-increasing if the dissimilarity is evaluated by either KL or CD. The greater the number of observable quantities ${\mathbf{Z}}_m$, the easier it is to distinguish two parameter values ${\theta }_0$ and ${\theta }^{*}$, and therefore the smaller the amount of parameters that are taken as close to ${\theta }_0$. Also, as the sample size goes to infinity, the pragmatic hypothesis associated to ${\Theta }_0$ converges to to ${\Theta }_0$. In other words, for each ${\theta }_0\in {\Theta }_0$, no other parameter value can predict infinitely many observable quantities with a precision sufficiently close to that of ${\theta }_0$.

4. Applications

In the following, pragmatic hypotheses for standard statistical problems are derived. Coscrato et al. [42] provide additional examples and methods for obtaining pragmatic hypotheses.

Example 6

(Gaussian with unknown variance.). Consider the setting from Example 5, but with ${\sigma }^2$ unknown and $0<{\sigma }^2\le M^2$. In this case, the parameter is $\theta =(\mu ,{\sigma }^2)$. Consider the composite hypothesis $H_0:\left\{{\mu }_0\right\}\times (0,M^2]$, which is often written as $H_0:\mu ={\mu }_0$. In this case, let ${\theta }_0=({\mu }_0,{\sigma }_0^2)$ and ${\Theta }_0=\left\{{\mu }_0\right\}\times (0,M^2]$. Proceeding as in Example 5, it follows that

$$\begin{array}{ccc}\hfill Pg(\left\{{\theta }_0\right\},B{P}_{\mathbf{Z}},ϵ)& =[{\mu }_0-ϵ{\sigma }_0,{\mu }_0+ϵ{\sigma }_0]\times (0,M^2]\hfill & \\ \hfill Pg({\Theta }_0,B{P}_{\mathbf{Z}},ϵ)& =[{\mu }_0-ϵM,{\mu }_0+ϵM]\times (0,M^2]\hfill & Theorem 1\end{array}$$

The rectangular shape of these pragmatic hypotheses seems to be unreasonable, as, for instance, whether a point $(\mu ,{\sigma }^2)$ is close to $({\mu }_0,{\sigma }_0^2)$ does not depend on ${\sigma }_0^2$. This is a consequence of the choice of δ in Example 5.

Figure 4 presents the pragmatic hypotheses for $H_0:\mu =0,{\sigma }^2=1$ and $H_0:\mu =0$ when $ϵ=0.1$ and $M^2=2$, and using the KL and CD dissimilarities. Contrary to BP, the hypotheses obtained from these dissimilarities do not have a rectangular shape. In particular, the triangular shape of the pragmatic hypotheses for $H_0:\mu =0$ is such that the closer ${\sigma }^2$ is to 0, the smaller the range of values for μ that are included in the pragmatic hypothesis. This behavior might be desirable, as when ${\sigma }^2$ is small, there is little uncertainty about the value of $\mathbf{Z}$, and consequently a narrow interval of values of μ can predict Z with precision ϵ.

Example 7

(Hardy–Weinberg equilibrium). Let $\mathbf{Z}\sim Multinomial(m,\mathit{\theta })$, where $\mathit{\theta }=({\theta }_1,{\theta }_2,{\theta }_3)$, ${\theta }_i\ge 0$, and ${\sum }_{i=1}^3{\theta }_i=1$. The Hardy–Weinberg (HW) hypothesis [43], $H_0$, which is depicted in the red curve in Figure 5 satisfies

$$\begin{array}{ccc}\hfill H_0:\theta \in {\Theta }_0,& \hfill & \hfill {\Theta }_0=\left\{\left(p^2,2p(1-p),{(1-p)}^2\right):0\le p\le 1\right\}\end{array}$$

If ${\mathit{\theta }}_0^p=(p^2,2p(1-p),{(1-p)}^2)$, ${\delta }_{\mathbf{Z}}\left(\mathbf{z}\right)={\mathit{E}[\parallel \mathbf{Z}-\mathbf{z}\parallel }_2^2|\mathit{\theta }={\mathit{\theta }}_0^p]$ and $g\left(x\right)=\sqrt{x}$, then it follows from Example 2 that

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\mathit{\theta }}_0^p,{\mathit{\theta }}^{*})& ={\left(m\times \frac{{({\theta }_1-p^2)}^2+{({\theta }_2-2p(1-p))}^2+{({\theta }_3-{(1-p)}^2)}^2}{p^2(1-p^2)+2p(1-p)(1-2p(1-p))+{(1-p)}^2(1-{(1-p)}^2)}\right)}^{0.5}\hfill \end{array}$$

The pragmatic hypotheses that are obtained using $KL$, $BP$, and $CD$ for the HW hypothesis are depicted in Figure 5. The choice between BP or KL and CD has a large impact over the shape of the pragmatic hypotheses. Although, for BP, the width of the pragmatic hypothesis is approximately uniform along the HW curve, the width of the pragmatic hypotheses obtained using $KL$ and $CD$ is smaller towards the edges of the HW curve. This behavior could be expected, as towards the edges of the HW curve, $\mathbf{Z}$ has the smallest variability. The figure also depicts the challenge in calibrating $KL$. Although the pragmatic hypotheses for $BP$ and $CD$ have similar sizes when using $ϵ=0.1$, this result was obtained for $KL$ while using $ϵ=0.01$.

The pragmatic hypotheses in Figure 5 are further tested using data from Brentani et al. [44], which is presented in Table 2. This study had the goal of verifying association between the APOE-ϵ4 gene and Alzheimer disease. The lower panels of Figure 5 present the 80% HPD regions for the distribution of this gene in each of the eight groups observed in the study. Additionally, they present two simulated datasets, 9 and 10. Groups 9 and 10 were generated by populations that were, respectively, not under and under the HW equilibrium. Group 9 and 10 fall, respectively, outside and inside of the pragmatic hypothesis.

Example 8

(Bioequivalence). Assume that $\mathbf{Z}=(X,Y)\sim N(({\mu }_1,{\mu }_2),{\sigma }^2{\mathbb{I}}_2)$, with σ known. We derive the pragmatic hypothesis for $H_0:{\mu }_1={\mu }_2$, that is, for $\{({\mu }_1,{\mu }_2)\in {\mathbb{R}}^2:{\mu }_1={\mu }_2\}$. Such a test might be used in a bioequivalence study, where X and Y are the concentrations of an active ingredient in a generic (test) drug medication and in the brand name (reference) medication [45], respectively. As $H_0$ is composite, it helps to derive the pragmatic hypothesis of its constituents.

In order to do so, let ${\theta }_0=({\mu }_0,{\mu }_0)$, ${\mu }_0\in \mathbb{R}$, ${\theta }^{*}=({\mu }_1^{*},{\mu }_2^{*})$, and $H^{{\theta }_0}:\theta ={\theta }_0$. If ${\delta }_{\mathbf{Z},{\theta }^{*}}\left(\mathbf{z}\right)=\mathbf{E}\left[{(X-{\mathbf{z}}_1)}^2+{(Y-{\mathbf{z}}_2)}^2|\theta ={\theta }^{*}\right]$ and $g\left(x\right)=\sqrt{x}$, then

$$\begin{array}{cc}\hfill {BP}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})& =\sqrt{\frac{{({\mu }_1^{*}-{\mu }_0)}^2+{({\mu }_2^{*}-{\mu }_0)}^2}{2{\sigma }^2}}\hfill \end{array}$$

Hence, $Pg(\left\{{\theta }_0\right\},B{P}_{\mathbf{Z}},ϵ)=\left\{({\mu }_1^{*},{\mu }_2^{*}):{({\mu }_1^{*}-{\mu }_0)}^2+{({\mu }_2^{*}-{\mu }_0)}^2\le 2{ϵ}^2{\sigma }^2\right\}$, which is a circle with center $({\mu }_0,{\mu }_0)$ and radius $\sqrt{2}ϵ\sigma$, as depicted on the left panel of Figure 6. In this case, the pragmatic hypothesis is the Tier 1 Equivalence Test hypothesis suggested by the US Food and Drug Administration [45]. The pragmatic hypothesis for $H_0:{\mu }_1={\mu }_2$ is obtained by taking the union of the pragmatic hypotheses associated to its constituents, as illustrated in the right panel of Figure 6. Specifically,

$$\begin{array}{c}\hfill Pg(H_0,B{P}_{\mathbf{Z}},ϵ)=\left\{({\mu }_1^{*},{\mu }_2^{*}):|{\mu }_2^{*}-{\mu }_1^{*}|\le ϵ\sigma \right\}\end{array}$$

The pragmatic hypothesis for $H_0$ using KL is obtained similarly. Note that

$$\begin{array}{c}\hfill {KL}_{\mathbf{Z}}({\theta }_0,{\theta }^{*})=0.5{BP}_{\mathbf{Z}}^2({\theta }_0,{\theta }^{*})\end{array}$$

Therefore, $Pg(\left\{{\theta }_0\right\},K{L}_{\mathbf{Z}},ϵ)=\left\{({\mu }_1^{*},{\mu }_2^{*}):{({\mu }_1^{*}-{\mu }_0)}^2+{({\mu }_2^{*}-{\mu }_0)}^2\le 2ϵ{\sigma }^2\right\}$ and

$$\begin{array}{cc}\hfill Pg(H_0,K{L}_{\mathbf{Z}},ϵ)& =Pg(H_0,K{L}_{\mathbf{Z}},0.5{ϵ}^2)\hfill \end{array}$$

The pragmatic hypothesis for $H_0$ that is obtained using CD has no analytic expression. However, by observing that $N(\mu ,{\sigma }^2)=\mu +\sigma N(0,1)$, it is possible to show that there exists a monotonically increasing function, $h:\mathbb{R}⟶\mathbb{R}$, such that

$$\begin{array}{cc}\hfill Pg(H_0,C{D}_{\mathbf{Z}},0.5{ϵ}^2)& =\left\{({\mu }_1^{*},{\mu }_2^{*}):|{\mu }_2^{*}-{\mu }_1^{*}|\le h\left(ϵ\right)\sigma \right\}\hfill \end{array}$$

That is, the pragmatic hypothesis associated to $H_0$ have the same shape as in the right panel of Figure 6. They differ solely on how many standard deviations correspond to the width of the pragmatic hypothesis.

Figure 4.

Pragmatic hypotheses in Example 6 for $H_0:\mu =0$ with KL (upper), CD (lower), $ϵ=0.1$, and $M^2=2$. $H_0$ is represented by a red line in all figures.

Figure 5.

Pragmatic hypotheses obtained for the HW equilibrium, depicted in red, using $m=20$, $ϵ=0.1$ for BP and CD and $ϵ=0.01$ for KL. The blue regions indicate the pragmatic hypothesis for HW and $p=\frac{1}{3}$ (top) and for HW (bottom). The lower, middle, and right panels were obtained, respectively, with BP, KL, and CD. The green regions in the right panels represents 80% HPD regions for the genotype distribution of each of the eight groups collected by Brentani et al. [44] and two simulated datasets.

Table 2.

Genotype counts for the eight groups in Brentani et al. [44]. Also, the decision of the GFBST agnostic hypothesis test [2] for testing in each group the pragmatic Hardy–Weinberg equilibrium hypothesis with $m=20$. The decisions are the same for $KL$, $BP$, and $CD$.

	AA	AD	DD	Decision
1	4	18	94	Agnostic
2	6	53	74	Accept
3	57	118	100	Agnostic
4	58	97	48	Agnostic
5	120	361	194	Agnostic
6	206	309	142	Accept
7	110	148	44	Accept
8	34	22	12	Agnostic
9	198	282	520	Reject
10	641	314	45	Accept

Figure 6.

Pragmatic hypotheses using BP in Example 8 when σ is known.

5. Final Remarks

The spiral structure studied in the work by the authors of [10] can be used to describe scientific evolution. However, in order for the analogy to be complete, it is necessary to indicate what types of scientific theories or hypotheses are effectively tested in the acceptance vertex of the hexagon of oppositions. We defend that these are pragmatic hypotheses, which are sufficiently precise for the end-user of the theory.

In order to make this statement formal, we introduce three methods for constructing a pragmatic hypothesis associated to a precise hypothesis. These methods are based on three predictive dissimilarity functions: KL, BP and CD. Each of these methods have different advantages. For instance, the scale of BP and CD is more interpretable than KL, making it easier to determine whether the former are large or small. On the other hand, BP relies on the definition of more functions than KL and CD, such as ${\delta }_{\mathbf{Z},{\theta }_0}\left(\mathbf{z}\right)$ in Definition 2. If these function are chosen inadequately, then the shape of the resultant pragmatic hypothesis might be counterintuitive or meaningless. Finally, CD often does not have an analytic expression. It relies on numerical integration over the sample space, which can be taxing in high dimensions.

The applications at Section 4 present adequate choices of metrics and bounds (defining pragmatic hypotheses) for some given theoretical and experimental setups. Nevertheless, the authors did not propose a general or automated recipe for making these choices, nor do they think this to be a feasible goal. In future research, the authors intend to explore a variety of application cases, some using historical data of important experiments, and discuss possible choices of metrics and bounds for each case. The authors hope that, in time, the accumulation of such examples will provide useful guidelines for the good use of methods developed in this paper, in the same way that the statistical literature provides useful guidelines for choosing good statistical models for practical applications.

Appendix A. Proofs

Proof of Lemma 1.

Let $\theta \in Pg({\Theta }_0,K{L}_{\mathbf{Z}},8{ϵ}^2)$. It follows from Theorem 1 that there exist ${\theta }_0\in {\Theta }_0$, such that $K{L}_{\mathbf{Z}}({\theta }_0,\theta )\le 0.5{ϵ}^2$. Conclude from Pinsker’s inequality that $C{D}_{\mathbf{Z}}({\theta }_0,\theta )\le \sqrt{8^{-1}K{L}_{\mathbf{Z}}({\theta }_0,\theta )}=ϵ$, that is, $\theta \in Pg({\Theta }_0,C{D}_{\mathbf{Z}},ϵ)$. □

Proof of Theorem 1.

Let $Pg$ be logically coherent. Pick an arbitrary ${\theta }_0\in {\Theta }_0$ and note that, if $R\left(x\right)\equiv Pg\left(\left\{{\theta }_0\right\}\right)$, then ${\varphi }_{Pg\left(\left\{{\theta }_0\right\}\right)}^R\left(x\right)=0$. Since $Pg$ is logically coherent, conclude that ${\varphi }_{Pg\left({\Theta }_0\right)}^R\left(x\right)\equiv 0$, that is, $Pg\left(\left\{{\theta }_0\right\}\right)\subseteq Pg\left({\Theta }_0\right)$. Since ${\theta }_0\in {\Theta }_0$ was arbitrary, conclude that

$$\begin{array}{cc}\hfill \bigcup _{{\theta }_0\in {\Theta }_0}Pg\left(\left\{{\theta }_0\right\}\right)& \subseteq Pg\left({\Theta }_0\right)\hfill \end{array}$$ (A1)

Next, let $R\left(x\right)\equiv {\bigcap }_{{\theta }_0\in {\Theta }_0}Pg{\left(\left\{{\theta }_0\right\}\right)}^c$. For every ${\theta }_0\in {\Theta }_0$, ${\varphi }_{Pg\left(\left\{{\theta }_0\right\}\right)}^R\left(x\right)=1$. Since $Pg$ is logically coherent, ${\varphi }_{Pg\left({\Theta }_0\right)}^R\equiv 1$, that is, $Pg\left({\Theta }_0\right)\subseteq R^c\equiv {\bigcup }_{{\theta }_0\in {\Theta }_0}Pg\left(\left\{{\theta }_0\right\}\right)$. Conclude that

$$\begin{array}{cc}\hfill Pg\left({\Theta }_0\right)& \subseteq \bigcup _{{\theta }_0\in {\Theta }_0}Pg\left(\left\{{\theta }_0\right\}\right)\hfill \end{array}$$ (A2)

It follows from Equations (A1) and (A2) that $Pg\left({\Theta }_0\right)={\bigcup }_{{\theta }_0\in {\Theta }_0}Pg\left(\left\{{\theta }_0\right\}\right)$. It also follows from direct calculation that, if $Pg\left({\Theta }_0\right)={\bigcup }_{{\theta }_0\in {\Theta }_0}Pg\left(\left\{{\theta }_0\right\}\right)$, then $Pg$ is logically coherent. □

Proof of Theorem 2.

Let ${\Theta }_0\subseteq \Theta$

$$\begin{array}{cc}\hfill Pg(f\left[{\Theta }_0\right],d_{\mathbf{Z}}^{*},ϵ)& =\{{\theta }^{*}\in {\Theta }^{*}:\exists {\theta }_0^{*}\in f\left[{\Theta }_0\right]\mathrm{s}.\mathrm{t}.d_{\mathbf{Z}}^{*}({\theta }^{*},{\theta }_0^{*})\le ϵ\}\hfill \\ \hfill & =\{{\theta }^{*}\in {\Theta }^{*}:\exists {\theta }_0^{*}\in f\left[{\Theta }_0\right]\mathrm{s}.\mathrm{t}.d_{\mathbf{Z}}(f^{-1}\left({\theta }^{*}\right),f^{-1}\left({\theta }_0^{*}\right))\le ϵ\}\hfill \\ \hfill & =f\left[\{\theta \in \Theta :\exists {\theta }_0\in {\Theta }_0\mathrm{s}.\mathrm{t}.d_{\mathbf{Z}}(\theta ,{\theta }_0)\le ϵ\}\right]\hfill \\ \hfill & =f\left[Pg({\Theta }_0,d_{\mathbf{Z}},ϵ)\right]\hfill \end{array}$$

 □

Proof of Theorem 3.

Since the $Z_i$’s are i.i.d., ${\mathrm{KL}}_m({\theta }_0,{\theta }^{*})=m{\mathrm{KL}}_{Z_1}({\theta }_0,{\theta }^{*})$. It follows that

$$\begin{array}{cc}\hfill Pg({\Theta }_0,K{L}_m,ϵ)& =\bigcup _{{\theta }_0\in {\Theta }_0}Pg(\left\{{\theta }_0\right\},{\mathrm{KL}}_m,ϵ)\hfill \\ \hfill & =\bigcup _{{\theta }_0\in {\Theta }_0}Pg(\left\{{\theta }_0\right\},m{\mathrm{KL}}_{Z_1},ϵ)=\bigcup _{{\theta }_0\in {\Theta }_0}\left\{{\theta }^{*}\in \Theta :K{L}_{Z_1}({\theta }_0,{\theta }^{*})\le m^{-1}ϵ\right\}\hfill \end{array}$$

Thus, ${\left(Pg({\Theta }_0,K{L}_m,ϵ)\right)}_{m\ge 1}$ is a non-increasing sequence of sets. It follows that

$$\begin{array}{cc}\hfill \underset{m\to \infty }{lim}Pg({\Theta }_0,K{L}_m,ϵ)& =\bigcap _{m\ge 1}\bigcup _{{\theta }_0\in {\Theta }_0}\left\{{\theta }^{*}\in \Theta :{\mathrm{KL}}_{Z_1}({\theta }_0,{\theta }^{*})\le m^{-1}ϵ\right\}\hfill \\ \hfill & =\bigcup _{{\theta }_0\in {\Theta }_0}\bigcap _{m\ge 1}\left\{{\theta }^{*}\in \Theta :{\mathrm{KL}}_{Z_1}({\theta }_0,{\theta }^{*})\le m^{-1}ϵ\right\}\hfill \\ \hfill & =\bigcup _{{\theta }_0\in {\Theta }_0}\left\{{\theta }^{*}\in \Theta :{\mathrm{KL}}_{Z_1}({\theta }_0,{\theta }^{*})=0\right\}\hfill \\ \hfill & =\bigcup _{{\theta }_0\in {\Theta }_0}\left\{{\theta }_0\right\}={\Theta }_0\hfill \end{array}$$

where the next-to-last equality follows from the assumption that ${\left(F_{\theta }\right)}_{\theta \in \Theta }$ is identifiable. The proofs for the $CD$ divergence follows from the fact that $\mathrm{TV}({\mathbb{P}}_{{\theta }_0},{\mathbb{P}}_{{\theta }^{*}})\le \sqrt{\mathrm{KL}({\mathbb{P}}_{{\theta }_0},{\mathbb{P}}_{{\theta }^{*}})}$. □

Author Contributions

All authors contributed equally to this paper.

Funding

This work was partially supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) grants PQ 306943-2017-4, 301206-2011-2, and 301892-2015-6; and Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) grants 2019/11321-9, 2017/03363-8, 2014/25302-2, CEPID-2013/07375-0, and CEPID-2014/50279-4. This study was also financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Finance Code 001.

Conflicts of Interest

The authors declare no conflict of interest.