Dynamics of ranking

Abstract

Virtually anything can be and is ranked; people, institutions, countries, words, genes. Rankings reduce complex systems to ordered lists, reflecting the ability of their elements to perform relevant functions, and are being used from socioeconomic policy to knowledge extraction. A century of research has found regularities when temporal rank data is aggregated. Far less is known, however, about how rankings change in time. Here we explore the dynamics of 30 rankings in natural, social, economic, and infrastructural systems, comprising millions of elements and timescales from minutes to centuries. We find that the flux of new elements determines the stability of a ranking: for high flux only the top of the list is stable, otherwise top and bottom are equally stable. We show that two basic mechanisms — displacement and replacement of elements — capture empirical ranking dynamics. The model uncovers two regimes of behavior; fast and large rank changes, or slow diffusion. Our results indicate that the balance between robustness and adaptability in ranked systems might be governed by simple random processes irrespective of system details.

Ranking lists are relevant to various areas of nature and society, however their evolution with the elements changing rank in time remained unexplored. The authors uncover a mechanism of ranking dynamics induced by the flux governing the arrival of new elements in the list, for improved predictability of ranking models.

Introduction

Rankings are everywhere. From country development indices, academic indicators, and candidate poll numbers to music charts and sports scoreboards, rankings are key to how humans measure and make sense of the world^1,2. The ubiquity of rankings stems from the generality of their definition: reducing the (often high-dimensional) complexity of a system to a few or even a single measurable quantity of interest^3,4, dubbed score, leads to an ordered list where elements are ranked, typically from highest to lowest score. Rankings are, in this sense, a proxy of relevance or fitness to perform a function in the system. Rankings are being used to identify the most accomplished individuals or institutions, and to find the essential pieces of knowledge or infrastructure in society¹. Since rankings often determine who gets access to resources (education, jobs, and funds), they play a role in the formation of social hierarchies^5,6 and the potential rise of systematic inequality⁷.

The statistical properties of ranking lists have caught the attention of natural and social scientists for more than a century. A heavy-tailed decay of score with rank, commonly known as Zipf’s law^8,9, has been systematically observed in the ranking of cities by population^10,11, words and phrases by frequency of use^12–17, companies by size^18–20, and many features of the Internet²¹. Zipf’s law appears even in the score-rank distributions of natural systems, such as earthquakes^22,23, DNA sequences²⁴, and metabolic networks²⁵. Rankings have also proven useful when analyzing productivity and impact in science and the arts^7,26–29, in human urban mobility^30–32, epidemic spreading by influentials³³, and the development pathways of entire countries³⁴. Recently, studies of language use^17,35, sports performance³⁶, and many biological and socioeconomic rankings³⁷ have strengthened the notion of universality suggested by Zipf’s law: despite microscopic differences in elements, scores, and types of interaction, the aggregate, macroscopic properties of ranking lists are remarkably similar throughout nature and society.

The similarity of score-rank distributions across systems raises the question of the existence of simple generative mechanisms behind them. While mechanisms of proportional growth³⁸, cumulative advantage³⁹, and preferential attachment⁴⁰ are often used to explain the heavy-tailed distributions of ranking lists at single points of time^41,42, they fail to reproduce the way elements actually move in rank⁴³, such as the sudden changes in city size throughout history^44,45. Here, we report on the existence of generic features of rank dynamics over a wide array of systems, from individuals to countries, and spatio-temporal scales, from minutes to centuries. By measuring the flux of elements across ranking lists^46–48, we identify a continuum ranging from systems where highly ranked elements are more stable than the rest, to systems where the least relevant elements are also stable. We show that simple mechanisms relying on fluxes generated by displacement and replacement of elements can account for all observed patterns of rank stability. A model based on these ingredients uncovers two regimes in rank dynamics, a fast regime driven by long jumps in rank space, and a slow one driven by diffusion.

Results

We gather 30 ranking lists in natural, social, economic, and infrastructural systems. Data include human and animal groups, languages, countries and cities, universities, companies, transportation systems, online platforms, and sports, with no selection criteria other than having enough information for analysis (for data details see Supplementary Information [SI] Section S2 and Table S1). Elements in each list are ranked by a measurable score that changes in time: scientists by citations, businesses by revenue, regions by a number of earthquakes, players by points, etc. Size and temporal scales in the data vary widely, from the number of people in 636 station entrances of the London Underground every 15min during a week in 2012⁴⁹, to the written frequency of 124k English words every year since the 17th century⁵⁰. Following an element’s rank through time reveals systematic patterns (Fig. 1). For example, in the Academic Ranking of World Universities (ARWU)⁵¹, published yearly since 2003, institutions like Harvard and Stanford maintain a high score, while institutions down the list change rank frequently (Fig. 1a).

Fig. 1. Ranking lists in nature and society show generic patterns in their dynamics.

a Yearly top ranking of universities worldwide according to ARWU score⁵¹. Elements in the system change rank as their scores evolve in time. b (Left) Rank turnover o_t at time t for studied systems, defined as the number N_t of elements ever seen in the ranking list up to t relative to list size N₀ (see SI Fig. S5). (Right) Correlation between mean turnover rate $\dot{o}$ and mean flux F (average probability that an element enters or leaves the list). Ranking lists form a continuum from the most open systems ($F,\dot{o}~1$) to the less open ($F,\dot{o}~0$; for values see SI Table S2). The area between dashed lines has linear scales to show closed systems with $F=\dot{o}=0$. (c) Time series of rank R_t/N₀ occupied by elements across the ranking list in selected systems (all datasets in SI Fig. S2). In the least open systems available, the top and bottom of the ranking list are stable. In open systems, only the top is stable. d Rank change C (average probability that element at rank R changes between t − 1 and t) across ranking lists (see SI Fig. S6), for F ≥ 0.01 (top) and F < 0.01 (bottom). The stable top and bottom ranks of less open systems mean C is roughly symmetric. In open systems, C increases with rank R.

Ranking lists typically have a fixed size N₀ (e.g., the Top 100 universities⁵¹, the Fortune 500 companies⁵²), so elements may enter or leave the list at any of the T observations t = 0, …, T − 1, allowing us to measure the flux of elements across rank boundaries^42,46,47 (for the observed values of N₀, T see SI Table S1). We introduce two time-dependent measures of flux: the rank turnover o_t = N_t/N₀, representing the number N_t of elements ever seen in the ranking list until time t relative to the list size N₀, and the rank flux F_t, representing the probability that an element enters or leaves the ranking list at time t. Rank turnover is a monotonic increasing function indicating how fast new elements reach the list (Fig. 1b left; all datasets in SI Fig. S5). In turn, flux shows striking stationarity in time despite differences in temporal scales and potential shocks to the system (SI Fig. S3). By averaging over time, the mean turnover rate $\dot{o}=\left(o_{T-1}-o_0\right)/\left(T-1\right)$ and the mean flux F = 〈F_t〉 turn out to be highly correlated quantities that uncover a continuum of ranking lists (Fig. 1b right; values in SI Table S2). In one extreme, the most open systems ($F,\dot{o}~1$) have elements that constantly enter and leave the list. Less open systems ($F,\dot{o}~0$) have a progressively lower turnout of constituents. Five out of 30 ranking lists are completely closed ($F=\dot{o}=0$), meaning no single new element is recorded during the observation window.

The measures of rank turnover and flux reveal regularities in the stability of ranking processes^43,53. We follow the time series of the rank R_t occupied by a given element at time t⁴⁴ (Fig. 1c; all datasets in SI Fig. S2). In most systems, highly ranked elements like Harvard University and the English word ‘the’ never change position, showcasing the correspondence between rank stability and notions of relevance like academic prestige^7,27, grammatical function^17,50, and underlying network structure⁵³. As we go down the ranking list of open systems, rank trajectories increasingly fluctuate in time. In the least open systems where turnover and flux are low, however, low ranked elements are also stable. In the ranking of British cities by population, for example, both Birmingham and Nairn remain the most and least populated local authority areas throughout the 20th century⁵⁴. These findings uncover a more fine-grained sense of rank stability: most systems have a stable top ranking, but only the least open systems feature stable bottom ranks as well. The rank change C, measured as the average probability that element at rank R changes between times t − 1 and t, varies between an approximately monotonic increasing function of R for open systems to a symmetric shape as systems become less open (Fig. 1d; all datasets in SI Fig. S6).

Since the stability patterns of an empirical ranking list (as measured by rank change C) can be systematically connected to the number of elements flowing into and out of the list, we build a model of rank dynamics based solely on simple generative mechanisms of flux (Fig. 2). Without assuming system-specific features of elements or their interactions, there are at least two ways to implement flux in rank space. Smooth (but arbitrarily large) changes in the score of an element might make it larger or smaller than other scores, causing elements to move across ranks (the way some scientists gather more citations than others²⁷, or how population size fluctuates due to historical events⁴⁴). Regardless of the score, elements might also disappear from the list and be replaced by new elements: young athletes enter competitions while old ones retire³⁶; new words replace anachronisms due to cultural shifts⁵⁰. We implement random mechanisms of displacement and replacement in a simple model by considering a synthetic ranking list of length N₀ embedded within a larger system of size N ≥ N₀. At each time step of length Δt = 1/N, a randomly chosen element moves to a randomly selected rank with probability τ, displacing others. At the same time, a randomly chosen element gets replaced by a new one with probability ν, leaving other ranks untouched. The dynamics involve all N elements, but to mimic real ranking lists, we only consider the top N₀ ranks when comparing with empirical data (Fig. 2a; model details in SI Section S4).

Fig. 2. Model of rank dynamics reproduces features of real-world ranking lists.

a Model of rank dynamics in a system of N elements and ranking list size N₀. At time t, a random element is moved to a random rank with probability τ. A random element is also replaced by a new element with probability ν. b Probability P_x,t that element in rank r = R/N moves to x = X/N after time t (uppercase/lowercase symbols are integer/normalized ranks). Elements not replaced diffuse around x = r (with probability D_x,t) or perform Lévy walks⁵⁵ (with probability L_t). Eq. (2) recovers simulation results, shown here for τ = 0.1, ν = 0.2, N = 100, and N₀ = 80 at times t = 1, 5 (left/right plots), averaged over 10⁵ realizations. c Time series of rank flux F_t over observation period T for data (lines), and mean flux F from the fitted model (dashes) (all datasets in SI Fig. S3; for fitting see SI Section S5). d Probability P_x,t for t = 1 and varying r (left) and rank change C (right), shown for selected datasets (lines) and fitted model (dashes; τ and ν in the plot) (empirical P_x,t is passed through a Savitzky–Golay smoothing filter; see SI Figs. S6–S9 and SI Table S2). As systems become more open, we lose symmetry in the rank dependence of both C and the height of the diffusion peaks of P_x,t (signaled by curved arrows). Data and model have similar qualitative behavior in all rank measures (for a systematic comparison see SI Fig. S19).

We solve the model analytically by introducing the displacement probability P_x,t that an element with rank r = R/N gets displaced to rank x = X/N after a time t (Fig. 2b; uppercase/lowercase symbols denote integer/normalized ranks). Since for small Δx = 1/N the probability that at time t an element has not yet been replaced is e^−νt, we have

$$P_{x,t}=e^{-\nu t}\left(L_t+D_{x,t}\right).$$ 1

Here, L_t = (1 − e^−τt)/N is the (rank-independent) probability that up until time t an element gets selected and jumps to any other rank. The length of jumps is uniformly distributed, so they can be thought of as a Lévy random walk with step length exponent 0⁵⁵ (full derivation in SI Section S4). The probability D_x,t = D(x, t)Δx that the element in rank r gets displaced to rank x after a time t (due to Lévy walks of other elements) follows approximately the diffusion-like equation

$$\frac{\partial D}{\partial t}=\alpha x\left(1-x\right)\frac{{\partial }^{2}D}{\partial x^2},$$ 2

where α = τ/N. Since ∑_xD_x,t = e^−τt, both D_x,t and D(x, t) are not conserved in time. Instead of a standard diffusion equation, Eq. (2) is equivalent to the Wright-Fisher equation of random genetic drift in allele populations^56,57. The solution D(x, t) of Eq. (2) is well approximated by a decaying Gaussian distribution with mean r and standard deviation $\sqrt{2\alpha r\left(1-r\right)t}$, i.e., a diffusion kernel (Fig. 2b). Overall, local displacement makes elements slowly diffuse around their initial rank, while Lévy walks and the replacement dynamics reduces exponentially the probability that old elements remain in the ranking list.

An explicit expression for the displacement probability P_x,t allows us to derive the mean flux

$$F=1-e^{-\nu }\left[p+\left(1-p\right)e^{-\tau }\right],$$ 3

and the mean turnover rate

$$\dot{o}=\nu \frac{\nu +\tau }{\nu +p\tau },$$ 4

where p = N₀/N is the length of the ranking list relative to system size (see SI Section S4). In order to fit the model to each empirical ranking list, we obtain N₀ from the data and approximate N = N_T−1 as the number of distinct elements ever seen in the list during the observation period T, thus fixing p (values for all datasets in SI Tables S1–S2). The remaining free parameters τ and ν (regulating the mechanisms of displacement and replacement) come from numerically solving Eqs. (3)–(4) with F and $\dot{o}$ fixed by the data (Figs. 1b and 2c; for model fitting see SI Section S5). The approximations in Eqs. (3)–(4) introduce a small bias in the estimation of τ (SI Fig. S18). Despite this bias, the simple generative mechanisms of flux in the model are enough to recover the behavior of ranking lists as quantified by P_x,t and C (Fig. 2d and SI Fig. S19): When rank flux is low, both the top and bottom of the list are similarly stable and rank dynamics is mostly driven by an interplay between Lévy walks and diffusion. As systems become more open, however, this symmetry gets broken due to a growing flux of elements at the bottom of the ranking list (see SI Fig. S4). Regardless of whether we rank people or animals, words or countries, the pattern of stability across a ranking list is accurately emulated by random mechanisms of flux that disregard the microscopic details of the individual system.

The characterization of flux in ranking lists with mechanisms of displacement and replacement of elements reveals regimes of dynamical behavior that are not apparent from the data alone (Fig. 3). By rescaling the fitted parameter values of the model as

$${\tau }_r=\frac{\tau }{p\left(1-p\right)\dot{o}}\text{and}{\nu }_r=\frac{\nu -p\dot{o}}{\dot{o}},$$ 5

most open ranking lists ($F,\dot{o}>0$) are predicted to follow the universal curve

$${\tau }_r{\nu }_r=1,$$ 6

which suggests that ranking dynamics are regulated by a single effective parameter [Fig. 3a; derivation in SI Section S5; for a discussion of the role of fluctuations on the validity of Eq. (6) see SI Figs. S18–S20]. Even if, potentially, displacement and replacement could appear in any relative quantity, adjusting the model to observations of rank flux and turnover (Fig. 1b) leads to an inverse relationship between parameters regulating their generative mechanisms. Real-world ranking lists lie in a spectrum where their dynamics is either mainly driven by score changes that displace elements in rank (high τ_r and low ν_r, like for GitHub software repositories⁵⁸ ranked by daily popularity), or by birth-death processes triggering element replacement (low τ_r and high ν_r, like for Fortune 500 companies⁵² ranked by yearly revenue). While the symmetry (or lack thereof) in rank change C may seemingly imply two distinct classes of systems (see Figs. 1d and 2d), Eq. (6) reveals the existence of a continuum of open ranking lists, which can be captured by a single model with a single effective parameter.

Fig. 3. Model uncovers regimes of dynamical behavior in open ranking lists.

a Rescaled model parameters τ_r and ν_r in open ranking lists, obtained from fitted parameters τ and ν, relative ranking list size p, and mean turnover rate $\dot{o}$ [see Eq. (5) and SI Section S5; only systems with $\dot{o}>10^{-3}$ are shown]. Values collapse onto the universal curve τ_rν_r = 1, so an inverse relationship between displacement and replacement is enough to emulate empirical rank dynamics (asterisks denote datasets that are farther away from the universal curve than bootstrapped model simulations; see SI Fig. S20). b (Top) Rate of element replacement ν/kℓ when subsampling data every k observations of length ℓ (see SI Table S1 and SI Section S5). Online social systems have the largest rates, followed by sports and languages (SI Table S2). (Bottom) Parameters τ_r and ν_r for T_eff = ⌈T/k⌉ subsampled observations (all datasets in SI Fig. S22). By subsampling ranking dynamics, systems move downwards along the universal curve while keeping a constant replacement rate. c–e Average probability that an element changes rank by Lévy walk (W_levy), diffusion (W_diff), or is replaced (W_repl) between consecutive observations in the data. Probabilities are shown both for selected datasets (dots), and for the model moving along the curve τ_rν_r = 1 with the same p and $\dot{o}$ as the data (lines) (for rest of systems see SI Fig. S17). The simulated system in (e) is the model itself for given values of τ, ν, and p (shown in plot). The model reveals a crossover in real-world ranking lists between a regime dominated by Lévy walks (b) to one driven by diffusion (c). Although not seen in data, the model also predicts a third regime driven by replacement (d).

Data on empirical ranking lists is constrained by the average time length ℓ between recorded observations, which varies from minutes to years depending on the source and intended use of the rankings (ℓ for all datasets is listed in SI Table S1). We explore such scoping effect by subsampling data every k observations (for details see SI Section S5 and SI Figs. S21–S22). Longer times between snapshots of the ranking list lead to an increase in rank flux, turnover, and fitted parameters, such that the rate of element replacement ν/kℓ stays roughly constant (Fig. 3b top). A conserved replacement probability per unit time, robust to changes in sampling rate, is yet another measure of rank stability: online social systems exchange elements frequently (e.g., the ranking lists by daily popularity of both GitHub software repositories and of online readers of the British newspaper The Guardian⁵⁹), followed by sports, while languages are the most stable (values for k = 1 in SI Table S2).

The universal curve in Eq. (6) displays three regimes in the dynamics of open ranking lists, as measured by the average probabilities that, between consecutive observations in the data, an element performs either a Lévy walk [W_levy = e^−ν(1 − e^−τ)], changes rank by diffusion [W_diff = e^−νe^−τ], or is replaced [W_repl = 1 − e^−ν], with W_levy + W_diff + W_repl = 1. In systems with the largest rank flux and turnover (GitHub repositories and The Guardian readers), elements tend to change rank via long jumps, following a Lévy walk, where W_levy > W_diff, W_repl (Fig. 3c). Here, long-range rank changes take elements in and out of a short ranking list within a big system (low p), thus generating large mean flux F (see SI Table S2). Most datasets, like the yearly rankings of scientists by citations in American Physical Society journals^27,60 and of countries by economic complexity^34,61, belong instead to a diffusion regime with W_diff > W_levy, W_repl (Fig. 3d). In this regime, local, diffusive rank dynamics is the result of elements smoothly changing their scores and overcoming their neighbors in rank space. Under subsampling, ranking lists move downwards along the universal curve, going from a state with a certain number of Lévy walks to one more driven by diffusion (Fig. 3b bottom; all datasets in SI Fig. S22).

The model also predicts a third regime dominated by replacement (W_repl > W_levy, W_diff; Fig. 3e), where elements are more likely to disappear than change rank. Such ranking lists replace most constituents from one observation to the next, forming a highly fluctuating regime that we do not observe in empirical data. To showcase the crossover between regimes, we simulate the model along the universal curve of Eq. (6) while keeping p and $\dot{o}$ fixed in Eq. (5) (lines in Fig. 3c–e). These curves show how close systems are to a change of regime, i.e., from one dominated by Lévy walks to one driven by diffusion. When a ranking list is close to a regime boundary, external shocks (amounting to variations in parameters τ and ν) may change the main mechanism behind rank dynamics, thus affecting the overall stability of the system.

Discussion

Ranking lists reduce the elements of high-dimensional complex systems into ordered values of a summary statistic, allowing us to compare seemingly disparate phenomena in nature and society^2,42. The diversity of their components (people, animals, words, institutions, and countries) stands in contrast with the statistical regularity of score-rank distributions when aggregated over time^9,37. By exploring the flux of elements of 30 ranking lists in natural, social, economic, and infrastructural systems, we present evidence of generic temporal patterns of rank dynamics. While open systems (large flux) keep the same elements only in top ranks, less open systems (lower flux) also have stable bottom ranks, forming a continuum of ranking lists explained by a single class of models. The model reveals two regimes of dynamical behavior for systems with nonzero flux. Real-world ranking lists are driven either by Lévy random walks⁵⁵ that change the rank of elements abruptly or by a more local, diffusive movement similar to genetic drift^56,57, both alongside a relatively low rate of element replacement independent from the frequency at which the ranking list is measured.

Our results suggest that, even though score distributions differ across systems depending on what type of elements and interactions they have (SI Fig. S1), ranking lists have similar stability features. What are the underlying properties of the system that enhance this similarity? An extension of our model explicitly considering the links between score and rank may help further understand the experimental evidence in this area, like the recent observation that the stability of crowdsourced rankings depends on the magnitude difference between quality scores⁶². It is also interesting to consider the observed deviations from the predictions of our model, even at the level of ranks. Rank flux for languages is not constant but decreases over time (SI Fig. S3), arguably due to the long observation period (over three centuries; see SI Table S1), variations in word sampling across decades, or even cognitive distortions at the societal scale⁶³. The rank-dependence of flux for very open systems (SI Fig. S4) and the slow decay of inertia with long times we observe in most datasets (SI Fig. S7) might be better reproduced by a non-uniform sampling of elements in the mechanisms of displacement and replacement of the model. Finally, deviations in the data (indicating a departure from the assumptions of randomness and stationarity built into our model) could be used to detect shocks to the system larger than expected statistical fluctuations, such as the sudden increase in the rank flux of Fortune 500 companies during financial crises (SI Fig. S3).

A more nuanced understanding of the generic features of ranking dynamics might help us limit resource exhaustion in competitive environments, such as information overload in online social platforms and prestige biases in scientific publishing⁶⁴, via better algorithmic rating tools⁶⁵. The observation of a systematic interplay between “slower” and “faster” ranking dynamics⁶⁶ (see SI Section S6) can be refined by exploring the relationship between ranking lists associated with the same system, or by incorporating networked interactions that lead to macroscopic ordering^44,67, which may provide a deeper understanding of network centrality measures based on ranking⁶⁸. Given that rankings often mediate access to resources via policy, similar mechanisms to those explored here may play a role in finding better ways to avoid social and economic disparity. In general, a better understanding of rank dynamics is promising for regulating systems by adjusting their temporal heterogeneity.

Author contributions

G.I., C.P., C.G., and A.-L.B. designed the study. G.I. performed data analysis and model fitting. G.I. and C.P. derived analytical results and performed numerical simulations. G.I., C.P., C.G., and A.-L.B. wrote the paper.

Peer review

Peer review information

Nature Communications thank the anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Data availability

For data availability, see SI Section S2. Non-public data is available from the authors upon reasonable request.

Code availability

Code to reproduce the results of the paper is publicly available at https://github.com/iniguezg/Farranks⁶⁹.

Competing interests

A.-L.B. is the founder of Foodome, ScipherMedicine, and Datapolis, companies that explore the role of networks in health and urban environments. G.I. is the founder of Predify, a data science consulting startup in Mexico. C.P. and C.G. declare no competing interest.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-022-29256-x.