An interchange property for the rooted phylogenetic subnet diversity on phylogenetic networks

Abstract

Faith’s Phylogenetic Diversity (PD) on rooted phylogenetic trees satisfies the so-called strong exchange property that guarantees that, for every two sets of leaves of different cardinalities, a leaf can always be moved from the larger set to the smaller set in such a way that the sum of the PD values does not decrease. This strong exchange property entails a simple polynomial-time greedy solution to the PD optimization problem on rooted phylogenetic trees. In this paper we obtain an exchange property for the rooted Phylogenetic Subnet Diversity (rPSD) on rooted phylogenetic networks, which involves a more complicated exchange of leaves. We derive from it a polynomial-time greedy solution to the rPSD optimization problem on rooted semibinary level-2 phylogenetic networks.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00285-024-02142-4.

Introduction

Over the last few centuries, human activity has caused the destruction of natural habitats at an unprecedented pace, resulting in a major episode of biodiversity extinction (Kolbert 2014). Urgent action is required to combat extinction and preserve biodiversity, but there are challenges, including a lack of funding and uncertainties about conservation strategies. Consequently, there has been an increasing need to provide criteria for defining priorities and proposing variables that allow quantification of biodiversity.

The traditional approach to assessing biodiversity based on species counts, species richness, and number of endemic species has limitations. For instance, this type of data is so heterogeneous that it can be difficult to compare across different sites and times (Gaston 1996). The approach based on lists of threatened species also has its drawbacks: for example, changes in the composition of these lists may represent changes in knowledge of species status rather than changes in the status itself (Possingham et al. 2002). Finally, measures of biodiversity based solely on species have been criticized for treating all species as equal, without regard to their functional roles in the ecosystem or their evolutionary history (Faith 1992).

A feature of species that may influence their biodiversity value is their evolutionary distinctness. A species with few close living evolutionary relatives is considered more worthy of protection than a species with many close genetically and phenotypically similar relatives (McNeely et al. 1990). At the beginning of the 1990s, the qualitative value afforded to evolutionarily distinct species was replaced by quantitative measures of phylogenetic distinctness. One of the first published measures of biodiversity based on phylogenetic information was Faith’s phylogenetic diversity, PD (Faith 1992). The PD value of a set of species placed in the leaves of a phylogenetic tree is defined as the total weight (i.e., the sum of the branch lengths) of the spanning tree connecting the root and these leaves. In its original formulation, the branch lengths represented the number of changes in phenotypic characters, and PD measured the diversity of phenotypic characters in a set of species. In the current usual interpretation of phylogenetic trees, branch lengths represent evolutionary time, which is assumed to be positively correlated with character variation.

Since its introduction, PD has been widely studied and applied (Pellens and Grandcolas 2016). One of its most useful properties, both from the formal and the applicability point of view, is the possibility of efficiently finding and characterizing all subsets of species in a phylogenetic tree of a given size with maximal PD value by means of a very simple greedy algorithm (Pardi and Goldman 2005; Steel 2005); for instance, for a recent application to the analysis of SARS-CoV-2 phylogeny, see Zhukova et al. (2021). The basis of this result is the so-called strong exchange property stating that for every pair of sets of leaves $X,X^{\prime }$ with $|X|>|X^{\prime }|$, we can always move a leaf from X to $X^{\prime }$ without decreasing the sum of the PD values.

Faith’s PD is defined on evolutionary histories modelled by means of phylogenetic trees. But phylogenetic trees can only cope with speciation events due to mutations, where each species other than the universal common ancestor has only one parent in the evolutionary history (its parent in the tree). It is clearly understood now that other speciation events, which cannot be properly represented by means of single arcs in a tree, play an important role in evolution (Doolittle 1999). These are reticulate events, like genetic recombinations, hybridizations, or lateral gene transfers, where a species is the result of the interaction between several parent species. This has lead to the introduction of phylogenetic networks as models of phylogenetic histories that allow to include these reticulate events (Huson et al. 2010). Faith’s PD has been extended to split networks1(Spillner et al. 2008) and to rooted phylogenetic networks (Wicke and Fischer 2018; Bordewich et al. 2022); as a matter of fact, several generalizations to rooted phylogenetic networks have been proposed, the most natural of which is the rooted Phylogenetic Subnet Diversity, rPSD, introduced by Wicke and Fischer (2018) and renamed AllPaths-PD by Bordewich et al. (2022).

It has been proved that the PD optimization problem can be solved efficiently on circular split networks2 using integer programming (Chernomor et al. 2016; Spillner et al. 2008), as well as (for rPSD) on the simplest class of non-tree rooted phylogenetic networks, the so-called galled trees, by reducing it to sets of linear size of minimum-cost flow problems (Bordewich et al. 2009, 2022). It is also known that these optimization problems are in general NP-hard on rooted phylogenetic networks (Bordewich et al. 2022) and on split networks (Chernomor et al. 2016).

In this paper we focus on the extension of the greedy optimization algorithm for PD on phylogenetic trees to rPSD on rooted phylogenetic networks. As we have mentioned, the greedy algorithm on phylogenetic trees is a consequence of the strong exchange property for PD that guarantees that, given two sets of leaves of different cardinalities, we can always move some element from the larger set to the smaller one without lowering the sum of the PD values. It is easy to check that this strong exchange property for rPSD is no longer valid even on galled trees (Bordewich et al. 2022). So, our first main contribution is its generalization to rPSD through a more involved exchange of leaves than simply moving one leaf from one set to another.

Our exchange property then allows us to strengthen the result of Bordewich et al. on galled trees, by proving that every rPSD-optimal set of m leaves in a galled tree is always obtained from an rPSD-optimal set of $m-1$ leaves by either optimally adding a leaf or optimally replacing a leaf by a pair of leaves. It also allows us to give polynomial time greedy solutions for the rPSD problem on semibinary level-2 networks and semi-3-ary level-1 networks, the next complexity level of rooted phylogenetic networks (see §2.1 for the definitions). On the negative side, we have not been able to deduce from it a greedy algorithm for semibinary level-3 or semi-4-ary level-1 networks and the problem for these more general classes remains open.

This paper is organized as follows. In Sect. 2.1 we define the concepts necessary to understand this work, including a generalization of the Phylogenetic Diversity due to Wicke and Fischer (2018), together with its properties and an example. Section 3 contains the main result of this manuscript, Theorem 1, and Sect. 4 exposes some of its applications to galled trees and to semi-d-ary level-k networks, for particular instances of d and k. We end in Sect. 5 with some concluding remarks. The proof of Theorem 1 together with two required lemmas can be found in the Appendix and proofs of additional results can be found in the Supplementary Material.

Preliminaries

Phylogenetic networks

Let $\mathrm{\Sigma }$ be a finite set of labels. By a phylogenetic network on $\mathrm{\Sigma }$ we understand a rooted directed acyclic simple graph where each node of in-degree $⩾2$ has out-degree exactly 1 and whose leaves (i.e., its nodes of out-degree 0) are bijectively labeled by $\mathrm{\Sigma }$ (Huson et al. 2010). A phylogenetic tree is simply a phylogenetic network without nodes of in-degree $⩾2$. Let us point out here that, although the usual definition of phylogenetic tree and network forbids, for reconstructibility reasons, the existence of elementary nodes, that is, of nodes of in-degree $⩽1$ and out-degree 1, we shall allow their existence in order to simplify some statements and proofs.

Let N be a phylogenetic network. We shall denote its root (i.e., its only node of in-degree 0) by r and its sets of nodes and arcs by V(N) and E(N), respectively, and we shall always identify its leaves with their corresponding labels. Given two nodes u, v in N, we say that v is a child of u, and also that u is a parent of v, when $(u,v)\in E(N)$. A node in N is of tree type, or a tree node, when its in-degree is $⩽1$, and a reticulation when its in-degree is $⩾2$ (and hence, its out-degree is 1). We shall say that N is semi-d-ary when all its reticulations have in-degree $⩽d$, and that N is binary when it is semibinary and all its internal tree nodes have out-degree 2.

We shall denote a (directed) path in N from a node u to a node v by $u⇝v$. The intermediate nodes of a path $u⇝v$ are the nodes involved in it other than u and v. For every $u,v\in V(N)$, we say that v is a descendant of u, and also that u is an ancestor of v, when there exists a path $u⇝v$, and that v is a descendant of an arc $e=(u^{\prime },u)$ when it is a descendant of its end u. In particular, every node is an ancestor, and a descendant, of itself. If v is a descendant of u and $u\ne v$, we shall say that it is a proper descendant of u. A set of nodes $V_0\subseteq V(N)$ is independent when no node in it is a proper descendant of any other node in it.

For every $v\in V(N)$, its cluster $C_N(v)\subseteq \mathrm{\Sigma }$ (or simply C(v) when N is clear from the context), is the set of (labels of) the descendant leaves of v, and the subnetwork of N rooted at v is the subgraph $N_v$ of N induced by the set of all descendants of v. $N_v$ is a phylogenetic network on C(v) with root v.

For every $X\subseteq V(N)$, we shall denote the set of all nodes in N that are ancestors of nodes in X by $\uparrow X$. Given an arc $e=(u,u^{\prime })\in E(N)$, we shall make the abuse of notation of writing $e\in \uparrow X$ to mean that e has some descendant in X, that is, that $u^{\prime }\in \uparrow X$.

A subgraph of a phylogenetic network N is biconnected when it is connected (as an undirected graph) and it remains connected after removing any node from it together with all arcs incident to this node. Every node and every arc in N are biconnected subgraphs. A biconnected component of N is a maximal biconnected subgraph, and we shall call a biconnected component with more than 2 nodes a blob. Every blob $\mathcal{B}$ has one, and only one, node that is an ancestor of all its nodes; we call it its split node. Every node in a blob $\mathcal{B}$ with no child inside $\mathcal{B}$ is a reticulation (should it be of tree type, removing its parent would disconnect $\mathcal{B}$); we call such reticulations the exit reticulations of $\mathcal{B}$, and the rest of its reticulations, internal. Every node in $\mathcal{B}$ has some descendant exit reticulation.

A phylogenetic network is level-k (Jansson and Sung 2006) when every biconnected component contains at most k reticulations. Thus, a level-0 network is a phylogenetic tree. A semibinary level-1 network is also called a galled tree (Gusfield et al. 2004); the phylogenetic network in Fig. 1 is a galled tree.

Fig. 1.

A weighted phylogenetic network. The tree nodes are represented by circles, the reticulation by a square, and the arcs’ labels represent their weights

A phylogenetic network N is weighted when it is endowed with a weight mapping $w:E(N)\to {\mathbb{R}}_{⩾0}$. The total weight of a subgraph of a weighted phylogenetic network is the sum of the weights of all arcs in the subgraph. In particular, the weight of a path is the sum of the weights of its arcs. All phylogenetic networks (and trees) appearing from now on in this paper are assumed to be weighted, usually without any further notice.

The rooted phylogenetic diversity on phylogenetic trees

Given a finite set $\mathrm{\Sigma }$, we shall denote henceforth its set of subsets by $\mathcal{P}(\mathrm{\Sigma })$ and, for every $k⩾0$, the set of all its subsets of cardinality k by ${\mathcal{P}}_k(\mathrm{\Sigma })$.

Given a weighted phylogenetic tree T on $\mathrm{\Sigma }$, Faith’s rooted Phylogenetic Diversity (Faith 1992) is the set function ${\text{PD}}_T:\mathcal{P}(\mathrm{\Sigma })\to {\mathbb{R}}_{⩾0}$ sending each $X\subseteq \mathrm{\Sigma }$ to the total weight of the subtree induced by the ancestors of nodes in X:

$$\begin{array}{c}\hfill {\text{PD}}_T(X)=\sum _{e\in \uparrow X}w(e).\end{array}$$

This function ${\text{PD}}_T$ on phylogenetic trees satisfies the following strong exchange property, introduced by Steel (2005) for unrooted phylogenetic trees: for every phylogenetic tree T on $\mathrm{\Sigma }$ and for every $X,X^{\prime }\subseteq \mathrm{\Sigma }$ such that $|X^{\prime }|<|X|$, there exists some $x\in X\backslash X^{\prime }$ such that

$$\begin{array}{c}\hfill {\text{PD}}_T(X)+{\text{PD}}_T(X^{\prime })⩽{\text{PD}}_T(X^{\prime }\cup \{x\})+{\text{PD}}_T(X\backslash \{x\}).\end{array}$$

For a proof of this fact in the rooted case, see (Steel 2016, §6.4.1).

This strong exchange property for ${\text{PD}}_T$ is the key ingredient in the proof that the simple Algorithm 1 given below produces, for every $k⩾1$, the family ${\mathcal{M}}_k$ of all ${\text{PD}}_T$-optimal subsets of $\mathrm{\Sigma }$ of cardinality k, that is, of all sets of k leaves with maximum ${\text{PD}}_T$ value. For this proof in the unrooted case, see Steel (2005); the proof in the rooted case is similar: cf. §6.4.1 in Steel (2016). In particular, given a phylogenetic tree T on $\mathrm{\Sigma }$, this algorithm provides a polynomial solution to the problem of finding the maximum ${\text{PD}}_T$ value among all members of ${\mathcal{P}}_k(\mathrm{\Sigma })$, and a member of ${\mathcal{P}}_k(\mathrm{\Sigma })$ reaching this maximum.

Algorithm 1.

Greedy for phylogenetic trees

The rooted phylogenetic subnet diversity

Wicke and Fischer (2018) proposed several generalizations of Faith’s rooted Phylogenetic Diversity function to phylogenetic networks. One of them, and possibly the most straightforward, is the rooted Phylogenetic Subnet Diversity: the set function ${\text{rPSD}}_N:\mathcal{P}(\mathrm{\Sigma })\to {\mathbb{R}}_{⩾0}$ sending each $X\subseteq \mathrm{\Sigma }$ to the total weight of the subgraph induced by the ancestors of nodes in X:

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)=\sum _{e\in \uparrow X}w(e).\end{array}$$

It is clear that if N is a phylogenetic tree, then ${\text{rPSD}}_N={\text{PD}}_N$. When N is clear from the context, we shall omit the subscript N and simply write $\text{rPSD}$.

Example 1

On the phylogenetic network N depicted in Fig. 1,

$$\begin{array}{c}\hfill \begin{array}{c}\text{rPSD}(x_1)=5,\text{rPSD}(x_2)=6,\text{rPSD}(x_3)=5,\text{rPSD}(x_4)=4,\hfill \\ \text{rPSD}(\{x_1,x_2\})=8,\text{rPSD}(\{x_1,x_3\})=9,\text{rPSD}(\{x_1,x_4\})=9,\hfill \\ \text{rPSD}(\{x_2,x_3\})=8,\text{rPSD}(\{x_2,x_4\})=10,\text{rPSD}(\{x_3,x_4\})=9,\hfill \\ \text{rPSD}(\{x_1,x_2,x_3\})=10,\text{rPSD}(\{x_1,x_2,x_4\})=12,\text{rPSD}(\{x_1,x_3,x_4\})=13,\hfill \\ \text{rPSD}(\{x_2,x_3,x_4\})=12,\text{rPSD}(\{x_1,x_2,x_3,x_4\})=14.\hfill \end{array}\end{array}$$

For every phylogenetic network N on $\mathrm{\Sigma }$, $\text{rPSD}$ is:

• (i)Monotone nondecreasing: For every $X\subseteq Y\subseteq \mathrm{\Sigma }$, $\text{rPSD}(X)⩽\text{rPSD}(Y)$.
• (ii)
  Subadditive: For every $X,Y\subseteq \mathrm{\Sigma }$,

  $$\begin{array}{c}\hfill \text{rPSD}(X\cup Y)⩽\text{rPSD}(X)+\text{rPSD}(Y).\end{array}$$
• (iii)
  Submodular: For every $X\subseteq Y\subseteq \mathrm{\Sigma }$ and for every $a\in \mathrm{\Sigma }\backslash Y$,

  $$\begin{array}{c}\hfill \text{rPSD}(Y\cup \{a\})-\text{rPSD}(Y)⩽\text{rPSD}(X\cup \{a\})-\text{rPSD}(X).\end{array}$$

(i) and (ii) are clear. As to (iii), it is proved by Bordewich et al. (2022).

On the negative side, $\text{rPSD}$ need not satisfy the strong exchange property, even for the simplest non-tree networks N. Indeed, consider again the binary galled tree N depicted in Fig. 1. Take $X=\{x_1,x_3,x_4\}$ and $X^{\prime }=\{x_2,x_4\}$. Then

$$\begin{array}{c}\hfill \begin{array}{c}\text{rPSD}(\{x_1,x_3,x_4\})+\text{rPSD}(\{x_2,x_4\})=23,\hfill \\ \text{rPSD}(\{x_3,x_4\})+\text{rPSD}(\{x_1,x_2,x_4\})=\text{rPSD}(\{x_1,x_4\})+\text{rPSD}(\{x_2,x_3,x_4\})=21.\hfill \end{array}\end{array}$$

Therefore, there is no $x\in X\backslash X^{\prime }$ such that

$$\begin{array}{c}\hfill \text{rPSD}(X)+\text{rPSD}(X^{\prime })⩽\text{rPSD}(X\backslash \{x\})+\text{rPSD}(X^{\prime }\cup \{x\}).\end{array}$$

As a consequence, an $\text{rPSD}$-optimal set of cardinality k of a phylogenetic network N need not contain any $\text{rPSD}$-optimal set of cardinality $k-1$. Consider again the galled tree depicted in Fig. 1. Its only set of two labels with largest $\text{rPSD}$ value is $\{x_2,x_4\}$ and its only set of three labels with largest $\text{rPSD}$ value is $\{x_1,x_3,x_4\}$.

So, Algorithm 1 cannot be used to produce $\text{rPSD}$-optimal sets of a given cardinality as it stands. Actually, Bordewich et al. (2022) prove that, given a phylogenetic network N on $\mathrm{\Sigma }$ and an integer k, the problem of finding the maximum ${\text{rPSD}}_N$ value on ${\mathcal{P}}_k(\mathrm{\Sigma })$ is NP-hard. On the positive side, these authors also prove that this problem can be solved in polynomial time on binary galled trees.

A general exchange property

Let $\mathrm{\Sigma }$ be a finite set and $W:\mathcal{P}(\mathrm{\Sigma })\to {\mathbb{R}}_{⩾0}$ a function. Given $X,X^{\prime }\subseteq \mathrm{\Sigma }$ such that $|X^{\prime }|<|X|$, a W-improving pair for $X,X^{\prime }$ is a pair of sets (A, B), with $A\subseteq X\backslash X^{\prime }$, $B\subseteq X^{\prime }\backslash X$, and $|B|<|A|$, such that

$$\begin{array}{c}\hfill W(X)+W(X^{\prime })⩽W((X\backslash A)\cup B)+W((X^{\prime }\backslash B)\cup A).\end{array}$$

To simplify the notation, given $X\subseteq \mathrm{\Sigma }$, $S\subseteq X$ and $T\subseteq \mathrm{\Sigma }\backslash X$, we shall denote henceforth $(X\backslash S)\cup T$ by ${\tau }_{S,T}(X)$.

Given a set

$$\begin{array}{c}\hfill \mathcal{S}\subseteq \{(A,B)\in \mathcal{P}{(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },|B|<|A|\},\end{array}$$

we shall say that $W:\mathcal{P}(\mathrm{\Sigma })\to {\mathbb{R}}_{⩾0}$ satisfies the exchange property with respect to $\mathcal{S}$ when every pair of sets $X,X^{\prime }\subseteq \mathrm{\Sigma }$ with $|X^{\prime }|<|X|$ has a W-improving pair in $\mathcal{S}$. So, Steel’s strong exchange property for phylogenetic trees mentioned in §2.2 says that, for every phylogenetic tree T on $\mathrm{\Sigma }$, ${\text{PD}}_T:\mathcal{P}(\mathrm{\Sigma })\to {\mathbb{R}}_{⩾0}$ satisfies the exchange property with respect to

$$\begin{array}{c}\hfill {\mathcal{S}}_0(\mathrm{\Sigma })=\{(\{x\},\mathrm{\varnothing }):x\in \mathrm{\Sigma }\}.\end{array}$$

As we have seen, this is no longer true for $\text{rPSD}$ on galled trees. The main result in this paper, Theorem 1, says that $\text{rPSD}$ satisfies, on every semi-d-ary level-k phylogenetic network on $\mathrm{\Sigma }$, the exchange property with respect to a larger family of pairs of subsets ${\mathcal{S}}_{k,d}(\mathrm{\Sigma })$ whose description only depends on k and d. These families are, when $k=1$,

$$\begin{array}{c}\hfill {\mathcal{S}}_{1,d}(\mathrm{\Sigma })={\mathcal{S}}_0(\mathrm{\Sigma })\cup \{(A,B)\in \mathcal{P}{(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },1⩽|B|<|A|⩽d\}\end{array}$$

and, when $k⩾2$,

$$\begin{array}{cc}\hfill {\mathcal{S}}_{k,d}(\mathrm{\Sigma })& ={\mathcal{S}}_0(\mathrm{\Sigma })\hfill \\ \hfill & \cup \{(A,B)\in \mathcal{P}{(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },1⩽|B|<|A|<dk,\hfill \\ \hfill & |A|-|B|⩽(d-1)k\}.\hfill \end{array}$$

From now on, when it is unnecessary to explicit the set of labels $\mathrm{\Sigma }$, we shall omit it from the notation of these families.

Given k and d, the cardinalities of these families of sets are polynomial in $|\mathrm{\Sigma }|=n$: $|{\mathcal{S}}_0|=n$ and

$$\begin{array}{cc}\hfill |{\mathcal{S}}_{1,d}|& =n+\sum _{j=2}^d\sum _{i=1}^{j-1}\left(\begin{array}{c}n\\ j\end{array}\right)\left(\begin{array}{c}n-j\\ i\end{array}\right),\hfill \\ \hfill |{\mathcal{S}}_{k,d}|& =n+\sum _{j=2}^{dk-1}\sum _{i=j-(d-1)k}^{j-1}\left(\begin{array}{c}n\\ j\end{array}\right)\left(\begin{array}{c}n-j\\ i\end{array}\right)\text{when}k⩾2.\hfill \end{array}$$

As we announced above, the main result in this section is the following theorem. Since its proof is quite long and technical, in order not to lose the thread of the manuscript we postpone it until Appendix A at the end of the paper.

Theorem 1

If N is a semi-d-ary level-k phylogenetic network, ${\text{rPSD}}_N$ satisfies the exchange property with respect to ${\mathcal{S}}_{k,d}$.

The family ${\mathcal{S}}_{k,d}$ cannot be improved, because there are semi-d-ary level-k phylogenetic networks N and pairs of sets of leaves $X,X^{\prime }$ with $|X^{\prime }|<|X|$ having no ${\text{rPSD}}_N$-improving pair (A, B) with $|A|-|B|<(d-1)k$. The next example describes one such network for $d=2$; it is straightforward to generalize it to the semi-d-ary setting for any $d⩾2$

Example 2

Consider the binary level-k phylogenetic network N on $\mathrm{\Sigma }=\{y,x_1,\dots ,x_k\}$ depicted in Fig. 2. Assume that all its arcs e have weight $w(e)>0$.

Fig. 2.

The network N in Example 2. The grey arcs form the set $E_0$

Let $X=\{x_1,\dots ,x_k\}$ and $X^{\prime }=\{y\}$. Let us check that, for every (A, B) such that $A\subseteq X$, $B\subseteq X^{\prime }$, and $|B|<|A|$,

$$\begin{array}{c}\hfill \text{rPSD}(X)+\text{rPSD}(X^{\prime })⩾\text{rPSD}({\tau }_{A,B}(X))+\text{rPSD}({\tau }_{B,A}(X^{\prime }))\end{array}$$

and that the equality holds only when $(A,B)=(X,X^{\prime })$. This will imply that the only $\text{rPSD}$-improving pair for $X,X^{\prime }$ in ${\mathcal{S}}_{k,2}$ is $(X,X^{\prime })$ itself.

Let:

• $E_0$ be the arcs in $\uparrow \{v_1,\dots ,v_k\}$; that is, $(r,a_1)$ and those beginning in $\{a_1,\dots ,a_{k-1}\}$.

• $E_1=E(N)\backslash (E_0\cup {\{e_i\}}_{i=1,\dots ,k})$; that is, the arcs ending in $\{H_1,H_2,\dots ,H_k,y\}$.

Then,

$$\begin{array}{c}\hfill \text{rPSD}(X)=\sum _{i=1}^{k}w(e_i)+\sum _{e\in E_0}w(e),\text{rPSD}(X^{\prime })=\sum _{e\in E_0\cup E_1}w(e).\end{array}$$

Now, on the one hand, if $B=\mathrm{\varnothing }$ and $A\ne \mathrm{\varnothing }$

$$\begin{array}{cc}& \text{rPSD}({\tau }_{A,\mathrm{\varnothing }}(X))=\text{rPSD}(X\backslash A)=\sum _{x_i\notin A}w(e_i)+\sum _{e\in E_0\cap \uparrow (X\backslash A)}w(e)\hfill \\ \hfill & \text{rPSD}({\tau }_{\mathrm{\varnothing },A}(X^{\prime }))=\text{rPSD}(X^{\prime }\cup A)=\text{rPSD}(X^{\prime })+\sum _{x_i\in A}w(e_i)\hfill \end{array}$$

and then

$$\begin{array}{cc}& \text{rPSD}(X)+\text{rPSD}(X^{\prime })-(\text{rPSD}({\tau }_{A,\mathrm{\varnothing }}(X))+\text{rPSD}({\tau }_{\mathrm{\varnothing },A}(X^{\prime })))\hfill \\ \hfill & =\sum _{e\in E_0}w(e)-\sum _{e\in E_0\cap \uparrow (X\backslash A)}w(e)>0\hfill \end{array}$$

because for every $x_i\in A$ the arc $(a_i,v_i)$ (or $(a_{k-1},v_k)$ if $i=k$) does not belong to $\uparrow (X\backslash A)$ and therefore $E_0\cap \uparrow (X\backslash A)⊊E_0$.

On the other hand, if $B=X^{\prime }=\{y\}$,

$$\begin{array}{cc}& \text{rPSD}({\tau }_{A,\{y\}}(X))=\text{rPSD}((X\backslash A)\cup \{y\})=\sum _{x_i\notin A}w(e_i)+\text{rPSD}(X^{\prime }),\hfill \\ \hfill & \text{rPSD}({\tau }_{\{y\},A}(X^{\prime }))=\text{rPSD}(A)=\sum _{x_i\in A}w(e_i)+\sum _{e\in E_0\cap \uparrow A}w(e)\hfill \end{array}$$

and then

$$\begin{array}{cc}& \text{rPSD}(X)+\text{rPSD}(X^{\prime })-(\text{rPSD}({\tau }_{A,\{y\}}(X))+\text{rPSD}({\tau }_{\{y\},A}(X^{\prime })))\hfill \\ \hfill & =\sum _{e\in E_0}w(e)-\sum _{e\in E_0\cap \uparrow A}w(e)⩾0,\hfill \end{array}$$

where, arguing as above, the inequality is an equality exactly when $A=X$.

We close this section with a refinement of Theorem 1 for level-1 networks. The proof is similar, and we provide it in Section 2 of the Supplementary file.

Corollary 1

If N is a semi-d-ary level-1 phylogenetic network on $\mathrm{\Sigma }$, ${\text{rPSD}}_N$ satisfies the exchange property with respect to

$$\begin{array}{c}\hfill {\mathcal{S}}_d={\mathcal{S}}_0\cup \{(A,\{b\})\in \mathcal{P}{(\mathrm{\Sigma })}^2:b\notin A,1<|A|⩽d\}\end{array}$$

Moreover, if $X,X^{\prime }$ have an improving pair $(A,\{b\})\in {\mathcal{S}}_d$, then there exists a blob in N with exit reticulation H and split node v such that $X\cap C(H)=\mathrm{\varnothing }$, $b\in C(H)$, and $A\subseteq C(v)$.

Applications

In this section we apply Theorem 1 to the study of ${\text{rPSD}}_N$-optimal subsets for low values of the level of N and the in-degree of its reticulations. Throughout this section, let N be a phylogenetic network on a set $\mathrm{\Sigma }$ of cardinality n and $\text{rPSD}={\text{rPSD}}_N$. We shall use the following notation:

• For every m, let ${\text{Opt}}_m$ be the family of $\text{rPSD}$-optimal subsets of $\mathrm{\Sigma }$ of cardinality m:

  $$\begin{array}{c}\hfill {\text{Opt}}_m=\{Z\in {\mathcal{P}}_m(\mathrm{\Sigma }):\text{rPSD}(Z)=max(\text{rPSD}({\mathcal{P}}_m(\mathrm{\Sigma })))\}.\end{array}$$

• An optimal sequence of N is a sequence $Y={(Y_m)}_{0⩽m⩽n}$ with each $Y_m\in {\text{Opt}}_m$.

• For every $k⩾1$ and $d⩾2$, for every $1⩽j⩽(d-1)k$, and for every $X\in \mathcal{P}(\mathrm{\Sigma })$,

  • ${\tau }_{k,d,j}(X)$ is the family of subsets of $\mathrm{\Sigma }$ of cardinality $|X|+j$ of the form ${\tau }_{B,A}(X)$ (this is, $(X\backslash B)\cup A)$ with $(A,B)\in {\mathcal{S}}_{k,d}$, $B\subseteq X$, $A\subseteq \mathrm{\Sigma }\backslash X$, and $|A|-|B|=j$.
  • $\text{Opt-}{\tau }_{k,d,j}(X)$ are the members of ${\tau }_{k,d,j}(X)$ with largest $\text{rPSD}$ value.

  and, analogously,

  • ${\tau }_{k,d,j}^{-1}(X)$ is the family of subsets of $\mathrm{\Sigma }$ of cardinality $|X|-j$ of the form ${\tau }_{A,B}(X)$ (this is, $(X\backslash A)\cup B)$ with $(A,B)\in {\mathcal{S}}_{k,d}$, $A\subseteq X$, $B\subseteq \mathrm{\Sigma }\backslash X$, and $|A|-|B|=j$.
  • $\text{Opt-}{\tau }_{k,d,j}^{-1}(X)$ are the members of ${\tau }_{k,d,j}^{-1}(X)$ with largest $\text{rPSD}$ value.

  Notice that $X^{\prime }\in {\tau }_{k,d,j}(X)$ if, and only if, $X\in {\tau }_{k,d,j}^{-1}(X^{\prime })$.
• Finally, for every $k⩾1$ and $d⩾2$, for every $1⩽j⩽(d-1)k$, we describe the family of subsets of $\mathrm{\Sigma }$ of cardinality $m+j$ (resp. $m-j$) of the form ${\tau }_{B,A}(Y)$ (resp. ${\tau }_{A,B}(Y)$) with $(A,B)\in {\mathcal{S}}_{k,d}$, $|A|-|B|=j$, with largest $\text{rPSD}$ value obtained from each $Y\in {\text{Opt}}_m$:

  • $\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_m)={\bigcup }_{Y\in {\text{Opt}}_m}\text{Opt-}{\tau }_{k,d,j}(Y)$.
  • $\text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_m)={\bigcup }_{Y\in {\text{Opt}}_m}\text{Opt-}{\tau }_{k,d,j}^{-1}(Y)$.

  The aim of this section will be to relate each $\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_m)$ with ${\text{Opt}}_{m+j}$ and $\text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_m)$ with ${\text{Opt}}_{m-j}$, providing the key ingredient of the greedy algorithm.

We begin with galled trees. As we have already mentioned, it was proved in Bordewich et al. (2022, Cor 4.6) that the optimization problem for $\text{rPSD}$ can be solved in polynomial time on galled trees. The next proposition strengthens this result by providing a recursive construction of the $\text{rPSD}$-optimal sets for these networks.

Proposition 1

Let N be a galled tree. Then, for every $m=1,\dots ,n$,

$$\begin{array}{c}\hfill {\text{Opt}}_m=\text{Opt-}{\tau }_{1,2,1}({\text{Opt}}_{m-1}).\end{array}$$

Proof

Let $Y_m\in {\text{Opt}}_m$ and $Y_{m-1}\in {\text{Opt}}_{m-1}$. By Theorem 1, there exists some $(A,B)\in {\mathcal{S}}_{1,2}$, with $A\subseteq Y_m\backslash Y_{m-1}$ and $B\subseteq Y_{m-1}\backslash Y_m$, such that

$$\begin{array}{c}\hfill \text{rPSD}(Y_m)+\text{rPSD}(Y_{m-1})⩽\text{rPSD}({\tau }_{A,B}(Y_m))+\text{rPSD}({\tau }_{B,A}(Y_{m-1})).\end{array}$$ 1

Since $|A|-|B|=1$, we have that ${\tau }_{A,B}(Y_m)\in {\mathcal{P}}_{m-1}(\mathrm{\Sigma })$ and ${\tau }_{B,A}(Y_{m-1})\in {\mathcal{P}}_m(\mathrm{\Sigma })$, and then, being $Y_{m-1}$ and $Y_m$ optimal in ${\mathcal{P}}_{m-1}(\mathrm{\Sigma })$ and ${\mathcal{P}}_m(\mathrm{\Sigma })$, respectively,

$$\begin{array}{c}\hfill \text{rPSD}({\tau }_{A,B}(Y_m))⩽\text{rPSD}(Y_{m-1}),\text{rPSD}({\tau }_{B,A}(Y_{m-1}))⩽\text{rPSD}(Y_m).\end{array}$$ 2

Combining these inequalities with (1) we obtain

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \text{rPSD}(Y_m)+\text{rPSD}(Y_{m-1})& ⩽\text{rPSD}({\tau }_{A,B}(Y_m))+\text{rPSD}({\tau }_{B,A}(Y_{m-1}))\hfill \\ \hfill & ⩽\text{rPSD}(Y_{m-1})+\text{rPSD}(Y_m).\hfill \end{array}\end{array}$$

Then, the inequalities (2) must be equalities, from which we deduce that:

• ${\tau }_{A,B}(Y_m)\in {\text{Opt}}_{m-1}$, and thus $Y_m={\tau }_{B,A}({\tau }_{A,B}(Y_m))\in \text{Opt-}{\tau }_{1,2,1}({\text{Opt}}_{m-1})$.

• ${\tau }_{B,A}(Y_{m-1})\in {\text{Opt}}_m$, and thus $\text{Opt-}{\tau }_{1,2,1}(Y_{m-1})\subseteq {\text{Opt}}_m$.

Since the choice of the optimal sets $Y_m,Y_{m-1}$ was arbitrary, we conclude that

$$\begin{array}{c}\hfill {\text{Opt}}_m\subseteq \text{Opt-}{\tau }_{1,2,1}({\text{Opt}}_{m-1})\text{and Opt-}{\tau }_{1,2,1}({\text{Opt}}_{m-1})\subseteq {\text{Opt}}_m\end{array}$$

as stated. $\square$

Remark 1

Notice that along the proof of the previous proposition we have proved that, in a galled tree, for every $Y_m\in {\text{Opt}}_m$ and $Y_{m-1}\in {\text{Opt}}_{m-1}$, there exists some pair $(A,B)\in {\mathcal{S}}_{1,2}$, with $A\subseteq Y_m\backslash Y_{m-1}$ and $B\subseteq Y_{m-1}\backslash Y_m$, such that ${\tau }_{A,B}(Y_m)\in {\text{Opt}}_{m-1}$ and ${\tau }_{B,A}(Y_{m-1})\in {\text{Opt}}_m$.

Proposition 1 implies that, on a galled tree, the members of ${\text{Opt}}_m$ are those obtained from members of ${\text{Opt}}_{m-1}$ by either optimally adding a leaf or optimally replacing a leaf by two leaves. This result yields the simple greedy polynomial time Algorithm 2 computing the family of optimal sets ${\text{Opt}}_m$ in increasing order of m that extends the greedy Algorithm 1 for phylogenetic trees.

Algorithm 2.

Greedy for galled trees

Remark 2

Proposition 1 also implies that, on a galled tree, the members of each ${\text{Opt}}_m$ are obtained from members of ${\text{Opt}}_{m+1}$ by removing a leaf or replacing a pair of leaves by a leaf in such a way that the value of $\text{rPSD}$ decreases the least.

To move up in the complexity ladder of phylogenetic networks, it is convenient to introduce a notation that allows a more compact description of the arguments of the type used in the previous proposition. Given a semi-d-ary level-k phylogenetic network N and an optimal sequence $Y={(Y_p)}_{0⩽p⩽n}$ of it, we shall write, for every $0⩽q<p⩽n$ and for every $j⩾1$,

$$\begin{array}{c}\hfill (p,q)\prec {·}^Y(p-j,q+j)\end{array}$$

to mean that there exists an $\text{rPSD}$-improving pair $(A,B)\in {\mathcal{S}}_{k,d}$ for $Y_p$ and $Y_q$ such that $|A|-|B|=j$. When we need to emphasize an improving pair (A, B), we shall write “$(p,q)\prec {·}^Y(p-j,q+j)$ by an improving pair (A, B)”. In addition, we shall write $(p,q)\prec {·}_j^Y\{p^{\prime },q^{\prime }\}$ to mean that $(p,q)\prec {·}^Y(p-j,q+j)$ and $\{p-j,q+j\}=\{p^{\prime },q^{\prime }\}$.

Remark 3

By Theorem 1, given any optimal sequence Y of a semi-d-ary level-k phylogenetic network and $0⩽q<p$, there always exists some $1⩽j⩽(d-1)k$ such that $(p,q)\prec {·}^Y(p-j,q+j)$.

The proof of the next lemma, which we leave to the reader, is similar to that of Proposition 1; actually, that proposition is a direct consequence of this lemma for $j=1$.

Lemma 1

Let N be a phylogenetic network and Y an optimal sequence of N. If $(p,q)\prec {·}^Y(p-j,q+j)$ and $\text{rPSD}(Y_{p-j})+\text{rPSD}(Y_{q+j})⩽\text{rPSD}(Y_p)+\text{rPSD}(Y_q)$, then $Y_p\in \text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_{p-j})$ and $Y_q\in \text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_{q+j})$.

In particular, if $p-q=j$ and $(p,q)\prec {·}^Y(q,p)$, then $Y_p\in \text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_q)$ and $Y_q\in \text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_p)$.

Corollary 2

Let N be a phylogenetic network and Y an optimal sequence of N. If there exists a closed $\prec {·}^Y$-chain of length $m⩾1$

$$\begin{array}{cc}& (p_1,q_1)\prec {·}_1^Y\{p_2,q_2\}\text{by an improving pair}(A_1,B_1)\hfill \\ \hfill & (p_2,q_2)\prec {·}_2^Y\{p_3,q_3\}\text{by an improving pair}(A_2,B_2)\hfill \\ \hfill & ⋮\hfill \\ \hfill & (p_m,q_m)\prec {·}_m^Y\{p_1,q_1\}\text{by an improving pair}(A_m,B_m)\hfill \end{array}$$

then, for each $i=1,\dots m$,

$$\begin{array}{c}\hfill Y_{p_i}\in \text{Opt-}{\tau }_{k,d,j_i}({\text{Opt}}_{p_i-j_i})\text{and}{Y}_{q_i}\in \text{Opt-}{\tau }_{k,d,j_i}^{-1}({\text{Opt}}_{q_i+j_i}).\end{array}$$

Proof

The closed chain ensures that all the inequalities in

$$\begin{array}{cc}& \text{rPSD}(Y_{p_1})+\text{rPSD}(Y_{q_1})⩽\text{rPSD}({\tau }_{A_1,B_1}(Y_{p_1}))+\text{rPSD}({\tau }_{B_1,A_1}(Y_{q_1}))\hfill \\ \hfill & ⩽\text{rPSD}(Y_{p_2})+\text{rPSD}(Y_{q_2})⩽\text{rPSD}({\tau }_{A_2,B_2}(Y_{p_2}))+\text{rPSD}({\tau }_{B_2,A_2}(Y_{q_2}))\hfill \\ \hfill & ⩽\text{rPSD}(Y_{p_3})+\text{rPSD}(Y_{q_3})⩽\text{rPSD}({\tau }_{A_3,B_3}(Y_{p_3}))+\text{rPSD}({\tau }_{B_3,A_3}(Y_{q_3}))\hfill \\ \hfill & ⋮\hfill \\ \hfill & ⩽\text{rPSD}(Y_{p_m})+\text{rPSD}(Y_{q_m})⩽\text{rPSD}({\tau }_{A_m,B_m}(Y_{p_m}))+\text{rPSD}({\tau }_{B_m,A_m}(Y_{q_m}))\hfill \\ \hfill & ⩽\text{rPSD}(Y_{p_1})+\text{rPSD}(Y_{q_1}),\hfill \end{array}$$

are equalities, and the result follows from applying the Lemma 1 to each $(p_i,q_i)\prec {·}^Y(p_i-j_i,q_i+j_i).$ $\square$

It is time to move one step up in the complexity ladder of phylogenetic networks. Recall that

$$\begin{array}{c}\hfill {\mathcal{S}}_{2,2}={\mathcal{S}}_{1,3}={\mathcal{S}}_0{\cup \{(A,B)\in \mathcal{P}(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },1⩽|B|<|A|⩽3)\}\end{array}$$

and in particular, for every $j=1,2$, $\text{Opt-}{\tau }_{1,3,j}=\text{Opt-}{\tau }_{2,2,j}$.

Proposition 2

If N is a semibinary level-2 or a semi-3-ary level-1 network, then:

1. ${\text{Opt}}_m\subseteq \text{Opt-}{\tau }_{2,2,1}({\text{Opt}}_{m-1})\cup \text{Opt-}{\tau }_{2,2,2}({\text{Opt}}_{m-2})$ for every $m=1,\dots ,n$.

2. ${\text{Opt}}_m\subseteq \text{Opt-}{\tau }_{2,2,1}^{-1}({\text{Opt}}_{m+1})\cup \text{Opt-}{\tau }_{2,2,2}^{-1}({\text{Opt}}_{m+2})$ for every $m=1,\dots ,n-1$.

Proof

Let Y be an optimal sequence of N and fix $1⩽m⩽n$. Then, by Theorem 1,

$$\begin{array}{c}\hfill (m,m-1)\prec {·}^Y(m-j_1,m-1+j_1)\end{array}$$ 3

for some $j_1=1$ or $j_1=2$.

1. If $j_1=1$, Eqn. (3) says that $(m,m-1)\prec {·}^Y(m-1,m)$, and hence, by Corollary 2,

   $$\begin{array}{c}\hfill Y_m\in \text{Opt-}{\tau }_{2,2,1}({\text{Opt}}_{m-1})\text{and}{Y}_{m-1}\in \text{Opt-}{\tau }_{2,2,1}^{-1}({\text{Opt}}_m).\end{array}$$
2. If $j_1=2$, Eqn. (3) says that $(m,m-1)\prec {·}^Y(m-2,m+1)$. Applying Theorem 1 again,

   $$\begin{array}{c}\hfill (m+1,m-2)\prec {·}^Y(m+1-j_2,m-2+j_2),\end{array}$$

   for some $j_2=1$ or $j_2=2$. In both cases, $\{m+1-j_2,m-2+j_2\}=\{m-1,m\}$, thus closing the $\prec ·$-chain initiated with (3). Then, by Corollary 2,

   $$\begin{array}{c}\hfill Y_m\in \text{Opt-}{\tau }_{2,2,2}({\text{Opt}}_{m-2})\text{and}{Y}_{m-1}\in \text{Opt-}{\tau }_{2,2,2}^{-1}({\text{Opt}}_{m+1}).\end{array}$$

Thus, in both cases we have that

$$\begin{array}{c}\hfill \begin{array}{c}{Y}_m\in \text{Opt-}{\tau }_{2,2,1}({\text{Opt}}_{m-1})\cup \text{Opt-}{\tau }_{2,2,2}({\text{Opt}}_{m-2}),\hfill \\ Y_{m-1}\in \text{Opt-}{\tau }_{2,2,1}^{-1}({\text{Opt}}_m)\cup \text{Opt-}{\tau }_{2,2,2}^{-1}({\text{Opt}}_{m+1}),\hfill \end{array}\end{array}$$

which, by the arbitrary choice of Y and m, concludes the proof. $\square$

Point (a) in the last proposition tells us that if N is semibinary level-2 or semi-3-ary level-1, all members of each ${\text{Opt}}_m$ are obtained either from members of ${\text{Opt}}_{m-1}$ by optimally adding a leaf, optimally replacing a leaf by a pair of leaves, or optimally replacing a pair of leaves by a triple of leaves (this possibility need not be considered in the semi-3-ary level-1 case by Corollary 1), or from members of ${\text{Opt}}_{m-2}$ by optimally replacing a leaf by a triple of leaves. This proves the correctness of the polynomial time greedy Algorithm 3 to compute the family of optimal sets ${\text{Opt}}_m$ for such a network N in increasing order of m (as we have mentioned, if N is semi-3-ary level-1, the sets ${\mathcal{M}}^{(4)}$ in the loop need not be computed).

Algorithm 3.

Greedy for semibinary level-2 or semi-3-ary level-1 networks

Example 3

Consider the phylogenetic networks in Fig. 3. On the left, a semi-3-ary level-1 network and on the right a semibinary level-2 network obtained by blowing up the reticulations in the left-hand side network into a pair of in-degree 2 connected reticulations. In both networks, we have the following optimal sets of leaves:

$$\begin{array}{c}\hfill \begin{array}{cc}{\text{Opt}}_1:\{z_0\}\hfill & {\text{Opt}}_5:\{x_{00},x_{01},x_{02},x_{11},x_{12}\}\hfill \\ {\text{Opt}}_2:\{z_0,z_1\}\hfill & {\text{Opt}}_6:\{x_{00},x_{01},x_{02},x_{10},x_{11},x_{12}\}\hfill \\ {\text{Opt}}_3:\{x_{11},x_{12},z_0\}\hfill & {\text{Opt}}_7:\{x_{00},x_{01},x_{02},x_{10},x_{11},x_{12},z_1\}\hfill \\ {\text{Opt}}_4:\{x_{00},x_{01},x_{02},z_1\}\hfill \end{array}\end{array}$$

Then, in both networks,

$$\begin{array}{c}\hfill \{x_{00},x_{01},x_{02},z_1\}\in {\text{Opt}}_4\backslash \text{Opt-}{\tau }_{2,2,1}({\text{Opt}}_3),\{x_{11},x_{12},z_0\}\in {\text{Opt}}_3\backslash \text{Opt-}{\tau }_{2,2,1}^{-1}({\text{Opt}}_4).\end{array}$$

Fig. 3.

The networks in Example 3

Now, if we move one more step further in the complexity ladder, the structure of the optimal sets is no longer so simple.

Proposition 3

If N is a semibinary level-3 or a semi-4-ary level-1 network, then, for every $m=1,\dots ,n$, at least one of the following assertions is true:

1. ${\text{Opt}}_m\subseteq {\bigcup }_{j=1}^3\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_{m-j})$ and ${\text{Opt}}_{m-1}\subseteq {\bigcup }_{j=1}^3\text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_{m-1+j})$.

2. ${\text{Opt}}_{m+1}=\text{Opt-}{\tau }_{k,d,3}({\text{Opt}}_{m-2})$,

where (k, d) is (3, 2) or (1, 4), depending on the type of network.

Proof

To begin with, notice that

$$\begin{array}{cc}\hfill {\mathcal{S}}_{3,2}& ={\mathcal{S}}_0{\cup \{(A,B)\in \mathcal{P}(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },1⩽|B|<|A|<6,|A|-|B|⩽3\}\hfill \\ \hfill {\mathcal{S}}_{1,4}& ={\mathcal{S}}_0{\cup \{(A,B)\in \mathcal{P}(\mathrm{\Sigma })}^2:A\cap B=\mathrm{\varnothing },1⩽|B|<|A|⩽4\}\hfill \end{array}$$

and therefore ${\mathcal{S}}_{1,4}\subseteq {\mathcal{S}}_{3,2}$. To simplify the notation, we shall abbreviate $\text{Opt-}{\tau }_{k,d,j}$ by simply $\text{Opt-}{\tau }_j$. Observe that j can only go from 1 to 3.

Let Y be an optimal sequence of N and fix $1<m⩽n$. To ease the task of the reader, we sketch the flow of the proof in Fig. 4; all implications leading to (a) or (b) are due to Cor. 2.

Fig. 4.

Sketch of the proof of Proposition 3

By Theorem 1,

$$\begin{array}{c}\hfill (m,m-1)\prec {·}^Y(m-j_1,m-1+j_1)\end{array}$$ 4

for some $j_1\in \{1,2,3\}$.

1. If $j_1=1$, then $(m,m-1)\prec {·}^Y(m-1,m)$ and we conclude as in (1) in the proof of Proposition 2 that $Y_m\in \text{Opt-}{\tau }_1({\text{Opt}}_{m-1})$ and $Y_{m-1}\in \text{Opt-}{\tau }_1^{-1}({\text{Opt}}_m)$.

2. If $j_1=2$, then $(m,m-1)\prec {·}^Y(m-2,m+1)$. Applying Theorem 1 again,

   $$\begin{array}{c}\hfill (m+1,m-2)\prec {·}^Y(m+1-j_2,m-2+j_2),\end{array}$$

   for some $j_2\in \{1,2,3\}$.

   1. If $j_2=1$ or $j_2=2$, $(m+1,m-2)\prec {·}_{j_2}^Y\{m,m-1\}$ and we conclude as in (2) in the proof of Proposition 2 that $Y_m\in \text{Opt-}{\tau }_2({\text{Opt}}_{m-2})$ and $Y_{m-1}\in \text{Opt-}{\tau }_2^{-1}({\text{Opt}}_{m+1})$.
   2. When $j_2=3$, we have $(m+1,m-2)\prec {·}^Y(m-2,m+1)$ and we can only deduce that $Y_{m+1}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-2})$ and $Y_{m-2}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+1})$.
3. If $j_1=3$, then $(m,m-1)\prec {·}^Y(m-3,m+2)$. Applying Theorem 1 again,

   $$\begin{array}{c}\hfill (m+2,m-3)\prec {·}^Y(m+2-j_2,m-3+j_2),\end{array}$$

   for some $j_2\in \{1,2,3\}$.

   1. If $j_2=1$, then $(m+2,m-3)\prec {·}^Y(m+1,m-2)$. Applying Theorem 1, we have

      $$\begin{array}{c}\hfill (m+1,m-2)\prec {·}^Y(m+1-j_3,m-2+j_3)\end{array}$$

      for some $j_3\in \{1,2,3\}$.

      1. If $j_3=1$ or $j_3=2$, then $(m+1,m-2)\prec {·}^Y\{m,m-1\}$, closing the $\prec ·$-chain initiated with (4). Then, by Corollary 2, $Y_m\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-3})$ and $Y_{m-1}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+2})$.
      2. If $j_3=3$, then $(m+1,m-2)\prec {·}^Y(m-2,m+1)$ as in (2.b) and we only have that $Y_{m+1}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-2})$ and $Y_{m-2}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+1})$.
   2. If $j_2=2$ or $j_2=3$, then $(m+2,m-3)\prec {·}^Y\{m,m-1\}$, closing the $\prec ·$-chain initiated with (4). Then, by Corollary 2, $Y_m\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-3})$ and $Y_{m-1}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+2})$.

Summarizing, we only have two possibilities:

• On the one hand, in the cases (1), (2.a), (3.a.i), and (3.b),

  $$\begin{array}{c}\hfill Y_m\in \bigcup _{j=1}^3\text{Opt-}{\tau }_j({\text{Opt}}_{m-j})\text{and}{Y}_{m-1}\in \bigcup _{j=1}^3\text{Opt-}{\tau }_j^{-1}({\text{Opt}}_{m-1+j}).\end{array}$$
• On the other hand, in the cases (2.b) and (3.a.ii),

  $$\begin{array}{c}\hfill Y_{m+1}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-2})\text{and Opt-}{\tau }_3(Y_{m-2})\subseteq {\text{Opt}}_{m+1}.\end{array}$$

By the arbitrary choice of Y and m, this concludes the proof. $\square$

A similar result holds for (k, d) such that $(d-1)k=4$. We give its proof in Section 3 of the Supplementary file.

Proposition 4

If N is a semi-5-ary level-1 or a semi-3-ary level-2 network, then, for every $m=1,\dots ,n$, at least one of the following assertions is true:

1. ${\text{Opt}}_m\subseteq {\bigcup }_{j=1}^4\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_{m-j})$ and ${\text{Opt}}_{m-1}\subseteq {\bigcup }_{j=1}^4\text{Opt-}{\tau }_{k,d,j}^{-1}({\text{Opt}}_{m-1+j})$.

2. ${\text{Opt}}_{m+1}=\text{Opt-}{\tau }_{k,d,3}({\text{Opt}}_{m-2})$,

where $(k,d)=(2,3)$ or (1, 5), depending on the type of network.

So, while we could give a greedy optimization algorithm for semibinary level-2 networks or semi-3-ary level-1 networks, an analogous argument fails for more complex networks. The reason why Propositions 3 and 4 are not sufficient to provide such a greedy algorithm is that we would require their assertion (a) —or a similar expression— to be true for all m. In the occurrence of any m where only assertion (b) holds, we do not have enough information about ${\text{Opt}}_m$ to be able to ensure that it can be obtained from previous optimal sets.

Remark 4

A close analysis of the proof of Proposition 3, using Corollary 2 in its full strength, shows that we actually have a more general result: for every optimal sequence Y of N and for every $1<m⩽n$, at least one of the following conditions holds (the labels correspond to the cases in the proof):

1. $Y_m\in \text{Opt-}{\tau }_1({\text{Opt}}_{m-1})$ and $Y_{m-1}\in \text{Opt-}{\tau }_1^{-1}({\text{Opt}}_m)$.

2. $Y_m\in \text{Opt-}{\tau }_2({\text{Opt}}_{m-2})$, $Y_{m-1}\in \text{Opt-}{\tau }_2^{-1}({\text{Opt}}_{m+1})$, and

   • $Y_{m+1}\in \text{Opt-}{\tau }_1({\text{Opt}}_m)$ and $Y_{m-2}\in \text{Opt-}{\tau }_1^{-1}{\text{Opt}}_{m-1}$, or
   • $Y_{m+1}\in \text{Opt-}{\tau }_2({\text{Opt}}_{m-1})$ and $Y_{m-2}\in \text{Opt-}{\tau }_2^{-1}{\text{Opt}}_m$.

3. $Y_{m+1}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-2})$ and $Y_{m-2}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+1})$.

4. $Y_m\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-3})$, $Y_{m-1}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+2})$, $Y_{m+2}\in \text{Opt-}{\tau }_1({\text{Opt}}_{m+1})$, $Y_{m-3}\in \text{Opt-}{\tau }_1^{-1}{\text{Opt}}_{m-2}$, and

   • $Y_{m+1}\in \text{Opt-}{\tau }_1({\text{Opt}}_m)$ and $Y_{m-2}\in \text{Opt-}{\tau }_1^{-1}{\text{Opt}}_{m-1}$, or
   • $Y_{m+1}\in \text{Opt-}{\tau }_2({\text{Opt}}_{m-1})$ and $Y_{m-2}\in \text{Opt-}{\tau }_2^{-1}{\text{Opt}}_m$.

5. $Y_{m+1}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-2})$ and $Y_{m-2}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+1})$.

6. $Y_m\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-3})$, $Y_{m-1}\in \text{Opt-}{\tau }_3^{-1}({\text{Opt}}_{m+2})$, and

   • $Y_{m+2}\in \text{Opt-}{\tau }_2({\text{Opt}}_m)$ and $Y_{m-3}\in \text{Opt-}{\tau }_2^{-1}{\text{Opt}}_{m-1}$, or
   • $Y_{m+2}\in \text{Opt-}{\tau }_3({\text{Opt}}_{m-1})$ and $Y_{m-3}\in \text{Opt-}{\tau }_3^{-1}{\text{Opt}}_m$.

Unfortunately, the extra information obtained in this way is still not enough to prove the correctness of a greedy $\text{rPSD}$-optimization algorithm for the networks considered in that proposition. A similar situation appears in the context of Proposition 4.

But we must point out that we have not been able to find any semibinary level-3 or any semi-4-ary level-1 network for which ${\text{Opt}}_m⊈{\bigcup }_{j=1}^3\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_{m-j})$ for some m. Similarly, we have not been able to find any semi-5-ary level-1 or any semi-3-ary level-2 network for which ${\text{Opt}}_m⊈{\bigcup }_{j=1}^4\text{Opt-}{\tau }_{k,d,j}({\text{Opt}}_{m-j})$ for some m. So, it might be possible that the greedy algorithm also works in these cases, since we have not discovered a counterexample that disproves its correctness for these types of networks. In Section 4 of the Supplementary file we provide several examples that illustrate our search for a counterexample. More examples can be found in the second author’s PhD Thesis (Riera 2023).

Conclusions

PD on phylogenetic trees satisfies the strong exchange property that guarantees that, for every two sets of leaves of different cardinalities, a leaf can always be moved from the larger set to the smaller one without decreasing the sum of the PD values. But rPSD does not longer satisfy this exchange property even for galled trees. In this paper we have generalized this exchange property to rPSD on phylogenetic networks of bounded level and reticulations’ in-degree, showing that a similar results holds if we allow more involved exchanges of leaves’ subsets. Our final goal was to use this generalized exchange property to find a polynomial time greedy algorithm for the optimization of rPSD on phylogenetic networks of bounded level and in-degree of reticulations. We have ultimately failed in this goal. We have indeed shown that the generalized exchange property entails such a greedy algorithm for semibinary level-2 networks and semi-3-ary level-1 networks (and sheds new light on the structure of the families of rPSD-optimal sets ${\text{Opt}}_m$ on galled trees) but it cannot be used, as it stands, to obtain such an algorithm on more complex networks. However, we have not been able to find examples of semibinary level-3 networks or semi-4-ary level-1 networks where the greedy algorithm fails: it is simply that the generalized exchange property alone seems not to be enough to prove its correctness.

Finally, it is important to point out that just like the $\text{rPSD}$ optimization problem itself, testing counterexamples is computationally expensive, too. While the greedy algorithm runs in polynomial time, finding whether ${\text{Opt}}_m$ can be obtained from some ${\text{Opt}}_{m-j}$ or not still requires calculating ${\text{Opt}}_m$ by brute force, and testing whether the exchange property holds for a certain subset of ${\mathcal{S}}_{k,d}$ where $|A|-|B|<j$ also requires testing all subsets $X,X^{\prime }\subseteq \mathrm{\Sigma }$. All these operations are exponential, hence trying even slightly larger examples can dramatically increase the runtime of the test.

Supplementary Information

Below is the link to the electronic supplementary material.

Appendix A: Proof of Theorem 1

We begin by stating two auxiliary lemmas. From now on, we shall call a semi-d-ary k-blob any blob with k reticulations, all of them of in-degree $⩽d$. Given such a semi-d-ary k-blob $\mathcal{B}$ and a non-empty subset ${\mathcal{E}}_1$ of its exit reticulations, the first lemma provides a sharp upper bound for the cardinality of any independent set of nodes V of $\mathcal{B}$ whose members have no descendant exit reticulation outside ${\mathcal{E}}_1$. This bound will entail the bound for the cardinality of A in the definition of ${\mathcal{S}}_{k,d}$. We give the proof of this lemma in Section 1 of the Supplementary file.

Lemma 2

Let $\mathcal{B}$ be a semi-d-ary k-blob with l exit reticulations and ${\mathcal{E}}_1$ a non-empty subset of its exit reticulations of cardinality $l_1⩾1$. Then, for every independent set of nodes V of $\mathcal{B}$ without descendant exit reticulations outside ${\mathcal{E}}_1$, $|V|⩽d{l}_1+(d-1)(k-l).$

The constructions explained in the proof of this lemma easily show that the bound it provides is sharp, in the sense that, for every $d,k,l,l_1$ with $d⩾2$ and $k⩾l⩾l_1⩾1$, there are semi-d-ary k-blobs with l exit reticulations and subsets ${\mathcal{E}}_1$ of $l_1$ exit reticulations containing an independent set of nodes V without descendant exit reticulations outside ${\mathcal{E}}_1$ of cardinality $d{l}_1+(d-1)(k-l)$: cf. Fig. 5.

Fig. 5.

A semibinary k-blob with l exit reticulations, a subset ${\mathcal{E}}_1$ of $l_1>0$ exit reticulations, and an independent set of nodes (represented by filled circles) without descendant exit reticulations outside ${\mathcal{E}}_1$ reaching the upper bound in Lemma 2 for $d=2$

Remark 5

By Lemma 2, if $\mathcal{B}$ is a semi-d-ary blob without internal reticulations, if ${\mathcal{E}}_1$ is a subset of its exit reticulations of cardinality $l_1$, and if V is an independent set of nodes in $\mathcal{B}$ without descendant exit reticulations outside ${\mathcal{E}}_1$, then $|V|⩽l_{1}d$. A close analysis of the proof of that lemma easily shows that the upper bound $|V|=d{l}_1$ is achieved when all the reticulations in ${\mathcal{E}}_1$ have in-degree d and the set V contains, for every $H\in {\mathcal{E}}_1$, exactly d nodes whose only reticulate descendant is H. Of course, such sets do not always exist: for instance, when $\mathcal{B}$ contains a node that is a parent of two different exit reticulations.

The second auxiliary lemma extracts a key technical step in the proof of Theorem 1. This lemma provides an analog of the exchange property for sets of ancestors of multisets of nodes of a blob. More precisely, we prove that if $X,X^{\prime }$ are multisets of nodes of a semi-d-ary k-blob with $|X|>|X^{\prime }|$ and satisfying some extra conditions (those under which we shall apply the lemma in the proof of the main theorem) then there exist a subset A of X disjoint with $X^{\prime }$ and a submultiset B of $X^{\prime }$ disjoint with X whose cardinalities satisfy the restrictions defining the family ${\mathcal{S}}_{k,d}$ and such that if we replace X and $X^{\prime }$ by $(X\backslash A)\cup B$ and $(X^{\prime }\backslash B)\cup A$, the set of nodes that are simultaneously ancestors of nodes in both sets does not decrease.

We use in this lemma some standard notation for multisets X: $m_X(v)$ denotes the multiplicity of an element v in X; $\text{Supp}X$ denotes the support of X, that is, the set of elements v such that $m_X(v)>0$; we say that X is a set when all its multiplicities are $⩽1$, and then we identify it with its support; a submultiset Y of X is full when $m_Y(y)=m_X(y)$ for every $y\in \text{Supp}Y\subseteq \text{Supp}X$; and the cardinality of X is $|X|={\sum }_{v\in \text{Supp}X}{m}_X(v)$. We shall also use the notation ${\tau }_{S,T}(X)=(X\backslash S)\cup T$ when X, S, T are multisets with $S\subseteq X$ and $\text{Supp}T\subseteq \mathrm{\Sigma }\backslash \text{Supp}X$.

This lemma also uses some basic properties of $\uparrow$-notation. Some simple results in this regard are that, for any two sets A, B, $\uparrow (A\cup B)=\uparrow A\cup \uparrow B$ and $\uparrow A\backslash \uparrow B\subseteq \uparrow (A\backslash B)$, and that if $A\subseteq B$, then $\uparrow A\subseteq \uparrow B$. Moreover, given a multiset A, we define $\uparrow A$ as $\uparrow \text{Supp}A$, without taking into account the multiplicities of the elements of A.

Lemma 3

Let $\mathcal{B}$ be a semi-d-ary k-blob and $X,X^{\prime }$ two multisets of nodes of $\mathcal{B}$ with $|X^{\prime }|<|X|$ and satisfying the following two further conditions:

• (i)For each $v\in V(\mathcal{B})$, if $m_{X^{\prime }}(v)<m_X(v)$, then $m_X(v)=1$ and $m_{X^{\prime }}(v)=0$.
• (ii)Each exit reticulation of $\mathcal{B}$ belongs to X or $X^{\prime }$.

Then

$$\begin{array}{c}\hfill \uparrow X\cap \uparrow X^{\prime }\subseteq \uparrow {\tau }_{A,B}(X)\cap \uparrow {\tau }_{B,A}(X^{\prime })\end{array}$$ 5

for some set $A\subseteq \text{Supp}X\backslash \text{Supp}{X}^{\prime }$ and some full submultiset B of $X^{\prime }$ with $\text{Supp}B\subseteq \text{Supp}{X}^{\prime }\backslash \text{Supp}X$ such that $B=\mathrm{\varnothing }$ and $|A|=1$, or $0<|B|<|A|=d$, or $0<|B|<|A|<dk$ and $|A|-|B|⩽(d-1)k$.

Proof

First, we introduce some notation.

• Let $\mathcal{E}$ be the set of exit reticulations of $\mathcal{B}$, and let

  $$\begin{array}{c}\hfill \begin{array}{c}{\mathcal{E}}_X=\mathcal{E}\cap (\text{Supp}X\backslash \text{Supp}{X}^{\prime }),{\mathcal{E}}_{X^{\prime }}=\mathcal{E}\cap (\text{Supp}{X}^{\prime }\backslash \text{Supp}X),\\ [1ex]{\mathcal{E}}_{X,X^{\prime }}=\mathcal{E}\cap (\text{Supp}X\cap \text{Supp}{X}^{\prime }).\end{array}\end{array}$$

  Let $l_X=|{\mathcal{E}}_X|$, $l_{X^{\prime }}=|{\mathcal{E}}_{X^{\prime }}|$ and $l_{X,X^{\prime }}=|{\mathcal{E}}_{X,X^{\prime }}|$. By (ii), $l_X+l_{X^{\prime }}+l_{X,X^{\prime }}=|\mathcal{E}|$.

• For each $H\in \mathcal{E}$, let ${\uparrow }_{\mathit{only}}H$ be the set $\uparrow H\backslash \uparrow (\mathcal{E}\backslash \{H\})$ of nodes whose only descendant exit reticulation is H. Since every node in $V(\mathcal{B})$ has some descendant exit reticulation, ${\uparrow }_{\mathit{only}}H=V(\mathcal{B})\backslash \uparrow (\mathcal{E}\backslash \{H\})$. Observe that ${\uparrow }_{\mathit{only}}{H\cap \uparrow }_{\mathit{only}}{H}^{\prime }=\mathrm{\varnothing }$ if $H\ne H^{\prime }$.

• Let $\widehat{X}$ be the set $\text{Supp}X\backslash \text{Supp}{X}^{\prime }$. By (i), $\widehat{X}=\{v\in V(\mathcal{B}):m_{X^{\prime }}(v)<m_X(v)\}$.

• Let ${\widehat{X}}^{\prime }$ be the full submultiset of $X^{\prime }$ supported on $\text{Supp}{X}^{\prime }\backslash \text{Supp}X$.

The inequality $|X|>|X^{\prime }|$ implies that $|\widehat{X}|>|{\widehat{X}}^{\prime }|$, too. Indeed:

$$\begin{array}{cc}\hfill 0& <|X|-|X^{\prime }|=\sum _{v\in V(\mathcal{B})}(m_X(v)-m_{X^{\prime }}(v))\hfill \\ \hfill & =\sum _{\begin{array}{c}v\in V(\mathcal{B})\\ m_X(v)>m_{X^{\prime }}(v)\end{array}}(m_X(v)-m_{X^{\prime }}(v))-\sum _{\begin{array}{c}v\in V(\mathcal{B})\\ m_{X^{\prime }}(v)>m_X(v)\end{array}}(m_{X^{\prime }}(v)-m_X(v))\hfill \\ \hfill & =|\widehat{X}|-\sum _{\begin{array}{c}v\in V(\mathcal{B})\\ m_{X^{\prime }}(v)>m_X(v)\end{array}}(m_{X^{\prime }}(v)-m_X(v))⩽|\widehat{X}|-\sum _{v\in {\widehat{X}}^{\prime }}(m_{X^{\prime }}(v)-m_X(v))\hfill \\ \hfill & =|\widehat{X}|-\sum _{v\in {\widehat{X}}^{\prime }}{m}_{X^{\prime }}(v)=|\widehat{X}|-|{\widehat{X}}^{\prime }|.\hfill \end{array}$$ 6

We shall consider three cases; in all of them we shall choose a subset $A\subseteq \widehat{X}$ and a full submultiset $B\subseteq {\widehat{X}}^{\prime }$ satisfying the requirements in the statement and we shall prove that they satisfy Eqn. (5).

(a) If there exists some $x\in \widehat{X}$ with a proper descendant in X, then $x\in \uparrow (X\backslash \{x\})$ and hence $\uparrow X=\uparrow (X\backslash \{x\})$. In this case, taking $A=\{x\}$ and $B=\mathrm{\varnothing }$ we have that

$$\begin{array}{c}\hfill \uparrow X\cap \uparrow X^{\prime }=\uparrow (X\backslash \{x\})\cap \uparrow X^{\prime }\subseteq \uparrow (X\backslash \{x\})\cap \uparrow (X^{\prime }\cup \{x\}).\end{array}$$

(b) Assume that no $x\in \widehat{X}$ has any proper descendant in X and that ${\mathcal{E}}_{X^{\prime }}=\mathrm{\varnothing }$. This implies that $\mathcal{E}={\mathcal{E}}_X\cup {\mathcal{E}}_{X,X^{\prime }}\subseteq X$ and that $\widehat{X}={\mathcal{E}}_X$, as any $x\in \widehat{X}\backslash {\mathcal{E}}_X$ would have some proper descendant in $\mathcal{E}\subseteq X$.

In this case, there exists an $H_0\in {\mathcal{E}}_X$ such that ${\widehat{X}}^{\prime }\subseteq \uparrow (\mathcal{E}\backslash \{H_0\})$. Indeed, assume that for every $H\in {\mathcal{E}}_X$ there existed some node $x_H^{\prime }\in {\widehat{X}}^{\prime }$ without any descendant in $\mathcal{E}\backslash \{H\}$. Then, each $x_H^{\prime }$ would belong to ${\uparrow }_{\mathit{only}}H$. Since the sets ${\uparrow }_{\mathit{only}}H$ are pairwise disjoint, the nodes $x_H^{\prime }$ would be pairwise different, forming a subset of $\text{Supp}{\widehat{X}}^{\prime }$ of cardinality $|{\mathcal{E}}_X|=|\widehat{X}|$, which cannot exist because $|{\widehat{X}}^{\prime }|<|\widehat{X}|$.

Take then $A=\{H_0\}$ and $B=\mathrm{\varnothing }$. If can prove that $\uparrow X^{\prime }\subseteq \uparrow (X\backslash \{H_0\})$, then we will have

$$\begin{array}{c}\hfill \uparrow X\cap \uparrow X^{\prime }\subseteq \uparrow X^{\prime }=\uparrow (X\backslash \{H_0\})\cap \uparrow X^{\prime }\subseteq \uparrow (X\backslash \{H_0\})\cap \uparrow (X^{\prime }\cup \{H_0\}).\end{array}$$

So, let $v\in \uparrow X^{\prime }$. There are two possibilities:

• If v has some descendant in ${\widehat{X}}^{\prime }$, then the latter will have a descendant in $\mathcal{E}\backslash \{H_0\}\subseteq X\backslash \{H_0\}$, which will also be a descendant of v.

• If v has no descendant in ${\widehat{X}}^{\prime }$, then

  $$\begin{array}{cc}\hfill v\in \uparrow X^{\prime }\backslash \uparrow {\widehat{X}}^{\prime }\subseteq \uparrow (X^{\prime }\backslash {\widehat{X}}^{\prime })=& \uparrow (X\cap X^{\prime })=\uparrow (X\backslash \widehat{X})\hfill \\ \hfill =& \uparrow (X\backslash {\mathcal{E}}_X)\subseteq \uparrow (X\backslash \{H_0\}).\hfill \end{array}$$

(c) Assume finally that ${\mathcal{E}}_{X^{\prime }}\ne \mathrm{\varnothing }$ and that no $x\in \widehat{X}$ has any proper descendant in X. This last condition implies that the set of nodes $\widehat{X}\backslash {\mathcal{E}}_X$ is independent and all their descendant exit reticulations belong to ${\mathcal{E}}_{X^{\prime }}$. Then, by Lemma 2 we have that

$$\begin{array}{cc}& |\widehat{X}|=|\widehat{X}\backslash {\mathcal{E}}_X|+|{\mathcal{E}}_X|⩽(d-1)(k-l_X-l_{X^{\prime }}-l_{X,X^{\prime }})+d{l}_{X^{\prime }}+l_X\hfill \\ \hfill & =(d-1)k-(d-2)l_X+l_{X^{\prime }}-(d-1)l_{X,X^{\prime }}\hfill \\ \hfill & ⩽(d-1)k+l_{X^{\prime }}(\text{because}d⩾2)\hfill \\ \hfill & ⩽(d-1)k+min\{k,|{\widehat{X}}^{\prime }|\}(\text{because}{l}_{X^{\prime }}⩽k\text{and}{l}_{X^{\prime }}⩽|\text{Supp}{\widehat{X}}^{\prime }|⩽|{\widehat{X}}^{\prime }|).\hfill \end{array}$$ 7

In particular,

$$\begin{array}{c}\hfill |\widehat{X}|⩽dk\text{and}|\widehat{X}|-|{\widehat{X}}^{\prime }|⩽(d-1)k.\end{array}$$ 8

Now, on the one hand, if $|\widehat{X}|<dk$, take $A=\widehat{X}$ and $B={\widehat{X}}^{\prime }$. By Eqns. (6) and (8), they satisfy the required conditions in the statement, and

$$\begin{array}{cc}& \text{Supp}{\tau }_{B,A}(X^{\prime })=\text{Supp}((X^{\prime }\backslash {\widehat{X}}^{\prime })\cup \widehat{X})=\text{Supp}X,\hfill \\ \hfill & \text{Supp}{\tau }_{A,B}(X)=\text{Supp}((X\backslash \widehat{X})\cup {\widehat{X}}^{\prime })=\text{Supp}{X}^{\prime },\hfill \end{array}$$

which implies $\uparrow X^{\prime }\cap \uparrow X=\uparrow {\tau }_{A,B}(X)\cap \uparrow {\tau }_{B,A}(X^{\prime })$.

On the other hand, if $|\widehat{X}|=dk$, then all inequalities in the sequence (7) as well as the inequality $l_{X^{\prime }}⩽k$ are equalities. The equality $l_{X^{\prime }}=k$ implies that the blob $\mathcal{B}$ has no reticulation other than those in ${\mathcal{E}}_{X^{\prime }}$. Moreover, since the first inequality in (7) is an equality, $\widehat{X}$ reaches the maximum number of possible independent nodes in $\uparrow {\mathcal{E}}_{X^{\prime }}=\uparrow \mathcal{E}$. Then, as noted in Remark 5, it must happen for each $H\in \mathcal{E}$ that ${deg}_{\mathit{in}}(H)=d$ and $|\widehat{X}\cap \uparrow H|=|\widehat{X}\cap {\uparrow }_{\mathit{only}}H|=d$.

Now, since $k=|{\mathcal{E}}_{X^{\prime }}|⩽|{\widehat{X}}^{\prime }|<|\widehat{X}|=dk$, there must exist some $H_0\in {\mathcal{E}}_{X^{\prime }}$ with $m_{X^{\prime }}(H_0)<d$. Take $A=\widehat{X}{\cap \uparrow }_{\mathit{only}}{H}_0=\widehat{X}\cap \uparrow H_0$ and B the multiset with $\text{Supp}B=\{H_0\}$ and $m_B(H_0)=m_{X^{\prime }}(H_0)$. We have that $0<|B|<|A|=d$ and hence the pair (A, B) satisfies the requirements in the statement. As to Eqn. (5), notice that

$$\begin{array}{cc}\hfill \uparrow X^{\prime }\subseteq & \uparrow (X^{\prime }\backslash \{H_0\})\cup \uparrow A\cup \{H_0\}\hfill \\ \hfill & \cup \{x^{\prime }\in X^{\prime }\mid x^{\prime }\text{intermediate in some path}A⇝H_0\}.\hfill \end{array}$$

Now, $H_0\notin \uparrow X$ and, by assumption, the elements of A have no proper descendant in X, which implies

$$\begin{array}{c}\hfill (\{H_0\}\cup \{x^{\prime }\in X^{\prime }\mid x^{\prime }\text{intermediate in some path}A⇝H_0\})\cap \uparrow X=\mathrm{\varnothing }.\end{array}$$

Moreover, since $A\subseteq \uparrow H_0$, we have that $\uparrow X\subseteq \uparrow ((X\backslash A)\cup \{H_0\})$. Therefore

$$\begin{array}{c}\hfill \uparrow X^{\prime }\cap \uparrow X\subseteq (\uparrow (X^{\prime }\backslash \{H_0\})\cup \uparrow A)\cap \uparrow X\subseteq \uparrow ((X^{\prime }\backslash \{H_0\})\cup A)\cap \uparrow ((X\backslash A)\cup \{H_0\})\end{array}$$

as we wanted to prove. $\square$

Theorem 1

If N is a semi-d-ary level-k phylogenetic network, ${\text{rPSD}}_N$ satisfies the exchange property with respect to ${\mathcal{S}}_{k,d}$.

Proof

The case $k=0$ is Steel’s strong exchange property for phylogenetic trees (Steel 2016, §6.4.1). So, we shall focus on the case $k⩾1$.

Without any loss of generality, we can assume that every tree node in N is at most bifurcating, in the sense that the out-degree of each tree node is at most 2 (recall that we do not forbid out-degree 1 tree nodes in our networks). Indeed, let first $N^{\prime }$ be the phylogenetic network obtained from N as follows: for every node v that is the split node of more than one blob and for each such blob rooted at v, add a new split node $v_i$ to the blob and a new arc $(v,v_i)$ with weight 0. $N^{\prime }$ is still semi-d-ary and level-k, no node in it is the split node of more than one blob, and ${\text{rPSD}}_N(Z)={\text{rPSD}}_{N^{\prime }}(Z)$ for every $Z\subseteq \mathrm{\Sigma }$. Now, let $N^{\prime \prime }$ be the phylogenetic network obtained from $N^{\prime }$ as follows: for every tree node v with $k⩾3$ children $v_1,\dots ,v_k$, replace in N the subgraph supported on $\{v,v_1,\dots ,v_k\}$ by a bifurcating tree with root v and leaves $v_1,\dots ,v_k$ and all its arcs except those ending in $v_1,\dots ,v_k$ of weight 0: the arc ending in each $v_i$ inherits the original weight of $(v,v_i)$; if any node $v_i$ had any entering arcs other than $(v,v_i)$, we keep them with their weights. Since v was the split node of at most one blob, no blob increases its level from $N^{\prime }$ to $N^{\prime \prime }$, and therefore $N^{\prime \prime }$ is still semi-d-ary and level-k, and ${\text{rPSD}}_{N^{\prime \prime }}(Z)={\text{rPSD}}_{N^{\prime }}(Z)={\text{rPSD}}_N(Z)$ for every $Z\subseteq \mathrm{\Sigma }$.

So, in the rest of this proof we shall suppose that N is at-most-bifurcating and in particular that no node in N is the split node of more than one blob.

We shall proceed by induction on the number $\alpha$ of arcs of the network. A phylogenetic network with $\alpha =0$ is a phylogenetic tree consisting of a single leaf, where the stated exchange property trivially holds. Now, let N be an at-most-bifurcating semi-d-ary level-k phylogenetic network with $\alpha ⩾1$ arcs, and let us suppose that the thesis in the statement is true for all at-most-bifurcating semi-d-ary level-k phylogenetic networks with less than $\alpha$ arcs.

Let $X,X^{\prime }\subseteq \mathrm{\Sigma }$ with $|X^{\prime }|<|X|$. If $|X|=1$ the exchange property is trivially satisfied taking $A=X$ and $B=X^{\prime }=\mathrm{\varnothing }$, so we assume from now on that $|X|⩾2$. Now consider the tree of blobs T of N (Gusfield et al. 2007), obtained by collapsing each blob in N into its split node. Then, T is a phylogenetic tree with the same root r as N, $V(T)\subseteq V(N)$, and, for every $v\in V(T)$, its cluster in T and in N are the same; let us denote it by C(v). Since $|X^{\prime }|<|X|$ and $|X|⩾2$, the set of nodes v in T such that $|X^{\prime }\cap C(v)|<|X\cap C(v)|$ and $1<|X\cap C(v)|$ is nonempty: it contains the root r.

We shall consider four cases.

(a) Assume that T contains some node $v_0\ne r$ such that $|X\cap C(v_0)|>|X^{\prime }\cap C(v_0)|$ and $|X\cap C(v_0)|>1$. Since $v_0\in V(T)$, $v_0$ is in N a tree node such that the arc $e_0=(v_1,v_0)$ ending in it does not belong to any blob, which implies that it is a cut arc. Let $N_0=N_{v_0}$ and let $N_1$ be the network obtained from N by removing $N_{v_0}$ and the arc $e_0$ and, if $v_1$ is a reticulation node, appending to it a dummy leaf child (not labelled in $\mathrm{\Sigma }$) through an arc of weight 0; cf. Figure 6. By the induction hypothesis, $N_0$ satisfies the thesis in the statement.

Fig. 6.

The network N in case (a)

Now, for every $Z\subseteq \mathrm{\Sigma }$, if $Z\cap C(v_0)=\mathrm{\varnothing }$, then ${\text{rPSD}}_N(Z)={\text{rPSD}}_{N_1}(Z)$, and if $Z\cap C(v_0)\ne \mathrm{\varnothing }$, then

$$\begin{array}{c}\hfill {\displaystyle {\text{rPSD}}_N(Z)={\text{rPSD}}_{N_0}(Z)+{\text{rPSD}}_{N_1}(Z)+w(e_0)+\sum _{e\in \uparrow v_1\backslash \uparrow (Z\backslash C(v_0))}w(e).}\end{array}$$

(Throughout this proof, given a network $N^{\prime }$ with set of leaves ${\mathrm{\Sigma }}^{\prime }$ and a set Z, we write ${\text{rPSD}}_{N^{\prime }}(Z)$ to denote actually ${\text{rPSD}}_{N^{\prime }}(Z\cap {\mathrm{\Sigma }}^{\prime })$. So, for instance, ${\text{rPSD}}_{N_0}(Z)$ and ${\text{rPSD}}_{N_1}(Z)$ in the expressions above actually mean ${\text{rPSD}}_{N_0}(Z\cap C(v_0))$ and ${\text{rPSD}}_{N_1}(Z\backslash C(v_0))$, respectively.)

Since $|X\cap C(v_0)|>|X^{\prime }\cap C(v_0)|$, by the induction hypothesis there exist $A\subseteq (X\backslash X^{\prime })\cap C(v_0)$ and $B\subseteq (X^{\prime }\backslash X)\cap C(v_0)$ such that $(A,B)\in {\mathcal{S}}_{k,d}(C(v_0))\subseteq {\mathcal{S}}_{k,d}$ and

$$\begin{array}{c}\hfill {\text{rPSD}}_{N_0}(X)-{\text{rPSD}}_{N_0}({\tau }_{A,B}(X))⩽{\text{rPSD}}_{N_0}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{N_0}(X^{\prime }).\end{array}$$ 9

Since $A,B\subseteq C(v_0)$, ${\tau }_{A,B}(X)\backslash C(v_0)=X\backslash C(v_0)$ and ${\tau }_{B,A}(X^{\prime })\backslash C(v_0)=X^{\prime }\backslash C(v_0)$, and thus, in particular,

$$\begin{array}{c}\hfill {\text{rPSD}}_{N_1}(X)={\text{rPSD}}_{N_1}({\tau }_{A,B}(X)),{\text{rPSD}}_{N_1}(X^{\prime })={\text{rPSD}}_{N_1}({\tau }_{B,A}(X^{\prime })).\end{array}$$

Notice also that ${\tau }_{B,A}(X^{\prime })\cap C(v_0)\ne \mathrm{\varnothing }$ because $A\ne \mathrm{\varnothing }$.

Assume first that $B\ne \mathrm{\varnothing }$, so that $X^{\prime }\cap C(v_0)\ne \mathrm{\varnothing }$ and ${\tau }_{A,B}(X)\cap C(v_0)\ne \mathrm{\varnothing }$. Then,

By the same argument, using that $X^{\prime }\cap C(v_0)\ne \mathrm{\varnothing }$ and ${\tau }_{B,A}(X^{\prime })\cap C(v_0)\ne \mathrm{\varnothing }$, we also have that

$$\begin{array}{cc}& {\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime })={\text{rPSD}}_{N_0}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{N_0}(X^{\prime }).\hfill \end{array}$$

Therefore, by Eqn. (9),

$$\begin{array}{cc}& {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))={\text{rPSD}}_{N_0}(X)-{\text{rPSD}}_{N_0}({\tau }_{A,B}(X))\hfill \\ \hfill & ⩽{\text{rPSD}}_{N_0}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{N_0}(X^{\prime })={\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\hfill \end{array}$$

Assume now that $B=\mathrm{\varnothing }$. Then, by the definition of ${\mathcal{S}}_{k,d}$, the set A must be a singleton and then ${\tau }_{A,B}(X)\cap C(v_0)=(X\backslash A)\cap C(v_0)\ne \mathrm{\varnothing }$, because, by assumption, $|X\cap C(v_0)|>1$. Then, arguing as above, we have that

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))={\text{rPSD}}_{N_0}(X)-{\text{rPSD}}_{N_0}({\tau }_{A,B}(X)).\end{array}$$

Similarly, if $X^{\prime }\cap C(v_0)\ne \mathrm{\varnothing }$,

$$\begin{array}{c}\hfill {\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime })={\text{rPSD}}_{N_0}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{N_0}(X^{\prime }),\end{array}$$

while if $X^{\prime }\cap C(v_0)=\mathrm{\varnothing }$ (and using that ${\tau }_{B,A}(X^{\prime })\backslash C(v_0)=X^{\prime }\backslash C(v_0)$),

In either case, by Eqn. (9) we have again

$$\begin{array}{cc}& {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))={\text{rPSD}}_{N_0}(X)-{\text{rPSD}}_{N_0}({\tau }_{A,B}(X))\hfill \\ \hfill & ⩽{\text{rPSD}}_{N_0}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{N_0}(X^{\prime })⩽{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\hfill \end{array}$$

(b) Assume now that the only node v in T such that $|X\cap C(v)|>|X^{\prime }\cap C(v)|$ and $|X\cap C(v)|>1$ is the root r, and that r is not the split node of any blob in N. Then, each child v of r in N is also its child in T and thus, if $|X\cap C(v)|>|X^{\prime }\cap C(v)|$, then $|X\cap C(v)|=1$. But since $|X|>|X^{\prime }|$, r must have some child $v_1$ such that $|X\cap C(v_1)|>|X^{\prime }\cap C(v_1)|$ and hence such that $|X\cap C(v_1)|=1$ and $X^{\prime }\cap C(v_1)=\mathrm{\varnothing }$; and then, since $|X|⩾2$, r must have a second child $v_2$ and $X\cap C(v_2)\ne \mathrm{\varnothing }$. For each $i=1,2$, let $e_i=(r,v_i)$ and let $N_i$ be the subnetwork of N rooted at $v_i$. The sets of leaves $C(v_1),C(v_2)$ of $N_1,N_2$ are disjoint and therefore, for each $Z\subseteq \mathrm{\Sigma }$,

$$\begin{array}{c}\hfill {\text{rPSD}}_N(Z)={\text{rPSD}}_{N_1}(Z)+{\text{rPSD}}_{N_2}(Z)+{\chi }_{N_1}(Z)w(e_1)+{\chi }_{N_2}(Z)w(e_2)\end{array}$$

where, for each $i=1,2$, ${\chi }_{N_i}(Z)=1$ if $Z\cap C(v_i)\ne \mathrm{\varnothing }$ and ${\chi }_{N_i}(Z)=0$ otherwise.

Let $X\cap C(v_1)=\{x\}$ and take $A=\{x\}$ and $B=\mathrm{\varnothing }$. Then, $(A,B)\in {\mathcal{S}}_0$ and ${\tau }_{A,B}(X)\cap C(v_1)=\mathrm{\varnothing }$, ${\tau }_{B,A}(X^{\prime })\cap C(v_1)=\{x\}$, ${\tau }_{A,B}(X)\cap C(v_2)=X\cap C(v_2)$, and ${\tau }_{B,A}(X^{\prime })\cap C(v_2)=X^{\prime }\cap C(v_2)$. Therefore,

$$\begin{array}{cc}& {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))\hfill \\ \hfill & ={\text{rPSD}}_{N_1}(X)+{\text{rPSD}}_{N_2}(X)+{\chi }_{N_1}(X)w(e_1)+{\chi }_{N_2}(X)w(e_2)-{\text{rPSD}}_{N_1}({\tau }_{A,B}(X))\hfill \\ \hfill & -{\text{rPSD}}_{N_2}({\tau }_{A,B}(X))-{\chi }_{N_1}({\tau }_{A,B}(X))w(e_1)-{\chi }_{N_2}({\tau }_{A,B}(X))w(e_2)\hfill \\ \hfill & ={\text{rPSD}}_{N_1}(\{x\})+{\text{rPSD}}_{N_2}(X)+w(e_1)+w(e_2)-0-{\text{rPSD}}_{N_2}(X)-0·w(e_1)-w(e_2)\hfill \\ \hfill & ={\text{rPSD}}_{N_1}(\{x\})+w(e_1)\hfill \end{array}$$

and, similarly,

$$\begin{array}{c}\hfill \begin{array}{c}{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime })\hfill \\ ={\text{rPSD}}_{N_1}(\{x\})+{\text{rPSD}}_{N_2}(X^{\prime })+w(e_1)+{\chi }_{N_2}(X^{\prime })w(e_2)\hfill \\ -0-{\text{rPSD}}_{N_2}(X^{\prime })-0·w(e_1)-{\chi }_{N_2}(X^{\prime })w(e_2)\hfill \\ ={\text{rPSD}}_{N_1}(\{x\})+w(e_1).\hfill \end{array}\end{array}$$

Hence, in this case,

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))={\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\end{array}$$

(c) Assume finally that the only node v in T such that $|X\cap C(v)|>|X^{\prime }\cap C(v)|$ and $|X\cap C(v)|>1$ is the root r, and that r is the split node of a (single) blob $\mathcal{B}$. we distinguish two subcases.

(c.1) If $\mathcal{B}$ contains some exit reticulation H with no descendant in $X\cup X^{\prime }$, and if $v_1,\dots ,v_{d^{\prime }}$ are the parents of H, then let $\widehat{N}$ be the phylogenetic network obtained from N by removing the subnetwork $N_H$, adding new leaves $h_1,\dots ,h_{d^{\prime }}$ with dummy labels outside $\mathrm{\Sigma }$, and replacing each arc $(v_i,H)$ by an arc $(v_i,h_i)$ with weight 0; cf. Figure 7. $\widehat{N}$ is still at-most-bifurcating, semi-d-ary, and level-k and it has less than $\alpha$ arcs (we have removed the arcs in $N_H$). Therefore, by the induction hypothesis, it satisfies the thesis in the statement. Let $\widehat{\mathrm{\Sigma }}$ be its set of labels. Then, since, by assumption, $X,X^{\prime }\subseteq \widehat{\mathrm{\Sigma }}\cap \mathrm{\Sigma }$, there exist $A\subseteq X\backslash X^{\prime }$ and $B\subseteq X^{\prime }\backslash X$ such that $(A,B)\in {\mathcal{S}}_{k,d}(\widehat{\mathrm{\Sigma }}\cap \mathrm{\Sigma })\subseteq {\mathcal{S}}_{k,d}$ and

$$\begin{array}{c}\hfill {\text{rPSD}}_{\widehat{N}}(X)-{\text{rPSD}}_{\widehat{N}}({\tau }_{A,B}(X))⩽{\text{rPSD}}_{\widehat{N}}({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_{\widehat{N}}(X^{\prime }).\end{array}$$

Since ${\text{rPSD}}_{\widehat{N}}(Z)={\text{rPSD}}_N(Z)$ for every $Z\subseteq \widehat{\mathrm{\Sigma }}\cap \mathrm{\Sigma }$, we conclude that

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))⩽{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\end{array}$$

(c.2) Finally, assume that all the exit reticulations of the blob $\mathcal{B}$ rooted at r have descendants in X or $X^{\prime }$. Let ${\mathcal{B}}^{\ast }$ be the set of nodes of $\mathcal{B}$ that have a child outside of $\mathcal{B}$; if $v\in {\mathcal{B}}^{\ast }$, we shall denote its child outside of $\mathcal{B}$ by $\overline{v}$. Notice that:

• $r\notin {\mathcal{B}}^{\ast }$ (its two children must belong to the blob);

• the exit reticulations of $\mathcal{B}$ belong to ${\mathcal{B}}^{\ast }$;

• since reticulations have out-degree 1, the internal reticulations of $\mathcal{B}$ do not belong to ${\mathcal{B}}^{\ast }$;

• $\overline{v}\in V(T)\backslash \{r\}$ for every $v\in {\mathcal{B}}^{\ast }$, and thus, by the current assumption, if $|X\cap C(\overline{v})|>|X^{\prime }\cap C(\overline{v})|$ then $|X\cap C(\overline{v})|=1$.

For each $v\in {\mathcal{B}}^{\ast }$ let ${\overline{N}}_v$ be the subnetwork of N rooted at v consisting of $N_{\overline{v}}$, v and the arc $(v,\overline{v})$.

Fig. 7.

The networks N and $\widehat{N}$ in case (c.1)

For each $Z\subseteq \mathrm{\Sigma }$, we shall denote by ${\mathcal{B}}_Z^{\ast }$ the multiset of nodes of ${\mathcal{B}}^{\ast }$ supported on

$$\begin{array}{c}\hfill \text{Supp}{\mathcal{B}}_Z^{\ast }=\{v\in {\mathcal{B}}^{\ast }:Z\cap C(\overline{v})\ne \mathrm{\varnothing }\}\end{array}$$

and with multiplicities $m_{{\mathcal{B}}_Z^{\ast }}(v)=|Z\cap C(\overline{v})|$. Since the subnetworks ${\overline{N}}_v$, with $v\in {\mathcal{B}}^{\ast }$, have pairwise disjoint sets of leaves and the union of their sets of leaves is $\mathrm{\Sigma }$, we have that $|{\mathcal{B}}_Z^{\ast }|=|Z|$ and

$$\begin{array}{c}\hfill {\text{rPSD}}_N(Z)=\sum _{v\in \text{Supp}{\mathcal{B}}_Z^{\ast }}{\text{rPSD}}_{{\overline{N}}_v}(Z)+\sum _{e\in \uparrow {\mathcal{B}}_Z^{\ast }}w(e).\end{array}$$ 10

So, $|{\mathcal{B}}_{X^{\prime }}^{\ast }|=|X^{\prime }|<|X|=|{\mathcal{B}}_X^{\ast }|$; by the current assumption, every exit reticulation belongs to ${\mathcal{B}}_X^{\ast }\cup {\mathcal{B}}_{X^{\prime }}^{\ast }$; and if $m_{{\mathcal{B}}_{X^{\prime }}^{\ast }}(v)=|X^{\prime }\cap C(\overline{v})|<m_{{\mathcal{B}}_X^{\ast }}(v)=|X\cap C(\overline{v})|$, then $m_{{\mathcal{B}}_X^{\ast }}(v)=1$. Therefore, the multisets ${\mathcal{B}}_X^{\ast }$, ${\mathcal{B}}_{X^{\prime }}^{\ast }$ satisfy the hypotheses of Lemma 3, which implies the existence of a set ${\mathcal{B}}_A$ and a multiset ${\mathcal{B}}_B$ of nodes of $\mathcal{B}$ such that:

1. ${\mathcal{B}}_A\subseteq \text{Supp}{\mathcal{B}}_X^{\ast }\backslash \text{Supp}{\mathcal{B}}_{X^{\prime }}^{\ast }$; thus, if $v\in {\mathcal{B}}_A$, $|X\cap C(\overline{v})|=1$ and $|X^{\prime }\cap C(\overline{v})|=0$.

2. $\text{Supp}{\mathcal{B}}_B\subseteq \text{Supp}{\mathcal{B}}_{X^{\prime }}^{\ast }\backslash \text{Supp}{\mathcal{B}}_X^{\ast }$ and, for every $v\in \text{Supp}{\mathcal{B}}_B$, $m_{{\mathcal{B}}_B}(v)=m_{{\mathcal{B}}_{\prime }^{\ast }}(v)=|X^{\prime }\cap C(\overline{v})|$.

3. ${\mathcal{B}}_B=\mathrm{\varnothing }$ and $|{\mathcal{B}}_A|=1$, or $0<|{\mathcal{B}}_B|<|{\mathcal{B}}_A|=d$, or $0<|{\mathcal{B}}_B|<|{\mathcal{B}}_A|<dk$ and $|{\mathcal{B}}_A|-|{\mathcal{B}}_B|⩽(d-1)k$.

4. $\uparrow {\mathcal{B}}_X^{\ast }\cap \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }\subseteq \uparrow {\tau }_{{\mathcal{B}}_A,{\mathcal{B}}_B}({\mathcal{B}}_X^{\ast })\cap \uparrow {\tau }_{{\mathcal{B}}_B,{\mathcal{B}}_A}({\mathcal{B}}_{X^{\prime }}^{\ast })$.

Let

$$\begin{array}{c}\hfill A=\bigcup _{v\in {\mathcal{B}}_A}(X\cap C(\overline{v})),B=\bigcup _{v\in \text{Supp}{\mathcal{B}}_B}(X^{\prime }\cap C(\overline{v})).\end{array}$$

Then, $A\subseteq X\backslash X^{\prime }$ and $B\subseteq X^{\prime }\backslash X$ with $|A|=|{\mathcal{B}}_A|$ and $|B|=|{\mathcal{B}}_B|$. In particular, by property (3), $(A,B)\in {\mathcal{S}}_{k,d}$. We shall prove that

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))⩽{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\end{array}$$

Before doing so, let us point out some facts that we shall use. First, notice that ${\mathcal{B}}_A={\mathcal{B}}_A^{\ast }$ and ${\mathcal{B}}_B={\mathcal{B}}_B^{\ast }$, because for every $v\in {\mathcal{B}}^{\ast }$

$$\begin{array}{cc}& m_{{\mathcal{B}}_A^{\ast }}(v)=|A\cap C(\overline{v})|=\left\{\begin{array}{c}1\text{if}v\in A\hfill \\ 0\text{if}v\notin A\hfill \end{array}\right\}=m_{{\mathcal{B}}_A}(v)\hfill \\ \hfill & m_{{\mathcal{B}}_B^{\ast }}(v)=|B\cap C(\overline{v})|=|X^{\prime }\cap C(\overline{v})|\hfill \\ \hfill & (\text{because the clusters}C(\overline{v})\text{are pairwise disjoint})\hfill \\ \hfill & =m_{{\mathcal{B}}_{X^{\prime }}^{\ast }}(v)=m_{{\mathcal{B}}_B}(v).\text{(by definition)}\hfill \end{array}$$

Moreover

$$\begin{array}{cc}& {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }={\tau }_{{\mathcal{B}}_A,{\mathcal{B}}_B}({\mathcal{B}}_X^{\ast })\text{and}\text{Supp}{\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }=((\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A)\cup \text{Supp}{\mathcal{B}}_B,\hfill \end{array}$$ 11

$$\begin{array}{cc}& {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }={\tau }_{{\mathcal{B}}_B,{\mathcal{B}}_A}({\mathcal{B}}_{X^{\prime }}^{\ast })\text{and}\text{Supp}{\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }=(\text{Supp}{\mathcal{B}}_{X^{\prime }}^{\ast }\backslash \text{Supp}{\mathcal{B}}_B)\cup {\mathcal{B}}_A.\hfill \end{array}$$ 12

Indeed, as to Eqn. (11), for every $v\in {\mathcal{B}}^{\ast }$

$$\begin{array}{cc}& m_{{\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}(v)=|((X\backslash A)\cup B)\cap C(\overline{v})|=|X\cap C(\overline{v})|-|A\cap C(\overline{v})|+|B\cap C(\overline{v})|\hfill \\ \hfill & =m_{{\mathcal{B}}_X^{\ast }}(v)-m_{{\mathcal{B}}_A}(v)+m_{{\mathcal{B}}_B}(v)=m_{{\mathcal{B}}_X^{\ast }\backslash {\mathcal{B}}_A}(v)+m_{{\mathcal{B}}_B}(v)=m_{({\mathcal{B}}_X^{\ast }\backslash {\mathcal{B}}_A)\cup {\mathcal{B}}_B}(v)\hfill \end{array}$$

and in particular

$$\begin{array}{c}\hfill \text{Supp}{\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }=\text{Supp}(({\mathcal{B}}_X^{\ast }\backslash {\mathcal{B}}_A)\cup {\mathcal{B}}_B)=((\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A)\cup \text{Supp}{\mathcal{B}}_B^{\ast }\end{array}$$

because $m_{{\mathcal{B}}_A}(v)=m_{{\mathcal{B}}_X^{\ast }}(v)$ for every $v\in {\mathcal{B}}_A$.

A similar argument, using that, for every $v\in \text{Supp}{\mathcal{B}}_B$, $m_{{\mathcal{B}}_B}(v)=m_{{\mathcal{B}}_{X^{\prime }}^{\ast }}(v)=|B\cap C(\overline{v})|=|X^{\prime }\cap C(\overline{v})|$ and that ${\mathcal{B}}_A\cap \text{Supp}{\mathcal{B}}_{X^{\prime }}^{\ast }=\mathrm{\varnothing }$, proves Eqn. (12).

We can proceed now to prove the desired inequality

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))⩽{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime }).\end{array}$$

By Eqn. (10),

$$\begin{array}{cc}& {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))\hfill \\ \hfill & =\sum _{v\in \text{Supp}{\mathcal{B}}_X^{\ast }}{\text{rPSD}}_{{\overline{N}}_v}(X)-\sum _{v\in \text{Supp}{\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}{\text{rPSD}}_{{\overline{N}}_v}({\tau }_{A,B}(X))+\sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }}w(e)-\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}w(e)\hfill \end{array}$$ 13

where

$$\begin{array}{cc}& \sum _{v\in \text{Supp}{\mathcal{B}}_X^{\ast }}{\text{rPSD}}_{{\overline{N}}_v}(X)=\sum _{v\in (\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(X)+\sum _{v\in {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(X)\hfill \\ \hfill & =\sum _{v\in (\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}((X\backslash A))+\sum _{v\in {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(A)\hfill \end{array}$$ 14

because if $v\in (\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A$, then $A\cap C(\overline{v})=\mathrm{\varnothing }$ and if $v\in {\mathcal{B}}_A$, then $X\cap C(\overline{v})=A\cap C(\overline{v})$; and

$$\begin{array}{cc}& \sum _{v\in \text{Supp}{\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}{\text{rPSD}}_{{\overline{N}}_v}({\tau }_{A,B}(X))\hfill \\ \hfill & =\sum _{v\in (\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(((X\backslash A)\cup B))+\sum _{v\in \text{Supp}{\mathcal{B}}_B}{\text{rPSD}}_{{\overline{N}}_v}(((X\backslash A)\cup B))\hfill \\ \hfill & (\text{by}11)\hfill \\ \hfill & =\sum _{v\in (\text{Supp}{\mathcal{B}}_X^{\ast })\backslash {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}((X\backslash A))+\sum _{v\in \text{Supp}{\mathcal{B}}_B}{\text{rPSD}}_{{\overline{N}}_v}(B)\hfill \end{array}$$ 15

because if $v\in \text{Supp}{\mathcal{B}}_X^{\ast }$, then $B\cap C(\overline{v})=\mathrm{\varnothing }$, and if $v\in \text{Supp}{\mathcal{B}}_B$, then $X\cap C(\overline{v})=\mathrm{\varnothing }$.

Therefore, combining Eqns. (13) to (15), we obtain

$$\begin{array}{cc}& {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))=\hfill \\ \hfill & =\sum _{v\in {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(A)-\sum _{v\in \text{Supp}{\mathcal{B}}_B}{\text{rPSD}}_{{\overline{N}}_v}(B)+\sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }}w(e)-\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}w(e).\hfill \end{array}$$

A similar argument proves that

$$\begin{array}{cc}& {\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime })\hfill \\ \hfill & =\sum _{v\in {\mathcal{B}}_A}{\text{rPSD}}_{{\overline{N}}_v}(A)-\sum _{v\in \mathcal{\text{Supp}}{B}_B}{\text{rPSD}}_{{\overline{N}}_v}(B)+\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }}w(e)-\sum _{e\in \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }}w(e).\hfill \end{array}$$

Thus,

$$\begin{array}{c}\hfill {\text{rPSD}}_N(X)-{\text{rPSD}}_N({\tau }_{A,B}(X))⩽{\text{rPSD}}_N({\tau }_{B,A}(X^{\prime }))-{\text{rPSD}}_N(X^{\prime })\end{array}$$

if, and only if,

$$\begin{array}{c}\hfill \sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }}w(e)⩽\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }}w(e).\end{array}$$

Finally, this last inequality holds because

$$\begin{array}{cc}& \sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }}w(e)=\sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }\cup \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_X^{\ast }\cap \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }}w(e)\hfill \\ \hfill & ⩽\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }\cup \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }\cap \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }}w(e)(\ast )\hfill \\ \hfill & =\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }}w(e)+\sum _{e\in \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }}w(e)\hfill \end{array}$$

where step $(\ast )$ is due to

$$\begin{array}{cc}& \uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }\cup \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }=\uparrow ({\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }\cup {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast })\hfill \\ \hfill & =\uparrow ({\tau }_{{\mathcal{B}}_A,{\mathcal{B}}_B}({\mathcal{B}}_X^{\ast })\cup {\tau }_{{\mathcal{B}}_B,{\mathcal{B}}_A}({\mathcal{B}}_{X^{\prime }}^{\ast }))(\text{by}(11)\text{and}(12))\hfill \\ \hfill & =\uparrow (((({\mathcal{B}}_X^{\ast }\backslash {\mathcal{B}}_A)\cup {\mathcal{B}}_B)\cup (({\mathcal{B}}_{X^{\prime }}^{\ast }\backslash {\mathcal{B}}_B)\cup {\mathcal{B}}_A)))\hfill \\ \hfill & =\uparrow ({\mathcal{B}}_X^{\ast }\cup {\mathcal{B}}_{X^{\prime }}^{\ast })=\uparrow {\mathcal{B}}_X^{\ast }\cup \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }\hfill \end{array}$$

and, by property (4) of ${\mathcal{B}}_A$ and ${\mathcal{B}}_B$ (and, again, (11) and (12)),

$$\begin{array}{c}\hfill \uparrow {\mathcal{B}}_X^{\ast }\cap \uparrow {\mathcal{B}}_{X^{\prime }}^{\ast }\subseteq \uparrow {\tau }_{{\mathcal{B}}_A,{\mathcal{B}}_B}({\mathcal{B}}_X^{\ast })\cap \uparrow {\tau }_{{\mathcal{B}}_B,{\mathcal{B}}_A}({\mathcal{B}}_{X^{\prime }}^{\ast })=\uparrow {\mathcal{B}}_{{\tau }_{A,B}(X)}^{\ast }\cup \uparrow {\mathcal{B}}_{{\tau }_{B,A}(X^{\prime })}^{\ast }.\end{array}$$

This completes the proof of case (c.2). $\square$

Funding

Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.

Declarations

Conflict of interest

The authors of this article declare that they have no financial Conflict of interest with the content of this article.