Abstract
In game theory, Parrondo's paradox describes the possibility of achieving winning outcomes by alternating between losing strategies. The framework had been conceptualized from a physical phenomenon termed flashing Brownian ratchets, but has since been useful in understanding a broad range of phenomena in the physical and life sciences, including the behavior of ecological systems and evolutionary trends. A minimal representation of the paradox is that of a pair of games played in random order; unfortunately, closed‐form solutions general in all parameters remain elusive. Here, we present explicit solutions for capital statistics and outcome conditions for a generalized game pair. The methodology is general and can be applied to the development of analytical methods across ratchet‐type models, and of Parrondo's paradox in general, which have wide‐ranging applications across physical and biological systems.
Keywords: analytical methods, Brownian ratchets, game theory, generalized solutions, noise, non‐linear dynamics, Parrondo's paradox
The game‐theoretic Parrondo's paradox refers to the attainment of winning outcomes by alternating between losing strategies. Closed‐form solutions are presented for capital statistics in a stochastically mixed game pair with drawing outcomes, and in a further generalization with independently tunable game branches; the solutions accommodate arbitrary symmetry modulus M. The approach can be adapted to other ratchet‐type models.
1. Introduction
Particles can exhibit bulk drift when exposed to asymmetric periodic potentials, even when the space‐averaged force is zero or in an opposing direction[ 1 , 2 , 3 ]; underlying the unexpected motion is an agitation‐ratcheting mechanism, formed from alternating diffusive and localizing temporal phases as the potential oscillates. This has accordingly been termed the flashing Brownian ratchet, in similarity to the Brownian ratchet‐and‐pawl system of Smoluchowski and Feynman,[ 4 , 5 ] and has been applied successfully in the control of particle ensembles.[ 6 ] An abstraction of the phenomenon is Parrondo's paradox, in which switching between two game‐theoretic losing strategies in a random or deterministic order can result in winning outcomes.[ 7 ]
Molecular motors and enzyme transport had been analyzed through Brownian ratchet models[ 8 , 9 , 10 ]; more recently, the broader class of Parrondo's paradox has been used to describe a large range of phenomena, including stability in mixed chaotic systems,[ 11 , 12 ] unexpected drifts in granular flow[ 13 ] and switched diffusion processes,[ 14 ] and entropic behaviour in information thermodynamics.[ 15 , 16 ] Quantum paradoxical systems have also been studied, some suggesting implications on quantum information processing,[ 17 ] and algorithms exploiting the paradox have been devised for engineering optimization.[ 18 , 19 ] In biophysics, evolutionary processes,[ 20 , 21 , 22 , 23 , 24 , 25 ] population biology,[ 26 ] ecological dynamics,[ 27 , 28 , 29 ] cellular machinery,[ 30 , 31 , 32 ] and social behavior[ 33 , 34 , 35 ] have all been linked to the paradox, and a degree of universality across scales is suspected.[ 36 ] The seminal results in genetics include the possibility of proliferation and fixation of lower‐fitness autosomal alleles amidst antagonistic selection and epistasis,[ 37 ] and selection for random phase variation[ 23 , 38 ] and phenotypic switching[ 24 , 25 ] in microorganisms, despite the strategy being likely locally maladaptive; in ecological systems, environmental fluctuations have been shown to enable the persistence of rare species in invaded habitats.[ 39 ] The framework of Parrondo's games has even seen applications in the modeling and control of cancer tumours, to varying degrees of approximation.[ 40 , 41 , 42 ]
Minimally, the classical Parrondo's paradox can be represented by a pair of games[ 43 ]—the first a coin‐flipping game, analogous to a linear potential supporting diffusive spread in the flashing Brownian ratchet, and the second having two branches, one played when the capital is a multiple of a modulus and the other played otherwise, analogous to an asymmetric periodic potential supporting localization of particle density. Stochastically mixing the games can produce winning outcomes (positive capital drift) despite both being losing individually. Despite the significant research interest, current descriptions of the properties of the paradox is incomplete.[ 17 , 36 , 43 , 44 ] Perturbative and asymptotic analyses have been presented,[ 45 , 46 ] alongside simulational studies,[ 47 ] but explicit solutions for capital statistics for arbitrary modulus have not been reported; known solutions for natural generalizations of the games also remain sparse.
In this paper, we present explicit closed‐form solutions for capital mean and variance in Parrondo's games generalized to include drawing outcomes; solutions for a further generalization into ‐branch games are also derived, for arbitrary probability configuration and . The boundaries of winning, drawing, and losing parameter spaces are similarly found in closed‐form. These results are exact in steady‐state, and accurate to for rounds in transient conditions; extensive verification against simulations is provided. This work essentially gives a complete solution to single‐agent capital‐dependent Parrondo's games, extensible to the multitude of variants modelable on discrete‐time Markov chains.
2. Generalized Game Pair
A generalized capital‐dependent Parrondo's game structure with drawing outcomes is first considered. In game , winning, drawing, and losing outcomes, respectively, occur with probability , , and in each round; in game , they respectively occur with probability , , and when capital is divisible by and , , and otherwise. Stochastic mixing is controlled by parameter , reflecting probabilities and of playing game and , respectively, at each round. Each winning and losing outcome results in a unit capital increment () and decrement (), respectively, and each draw results in unchanged capital (). Lose–win ratios are , , and .
2.1. Conditions
The conditions for winning, fair, and losing outcomes can be deduced by considering the probability of reaching zero capital. With mixed transition probabilities and mixed lose–win ratios for and , the conditions are remarkably simple
(1) |
where for stochastically mixed games. For pure game , , reducing , and ; whereas for pure game , , reducing , and . A derivation of this result is given in Appendix A1.
2.2. Capital Statistics
We now use an explicit Markov chain formulation, seeking analytical solutions for expected capital and capital variance at round . Capital states satisfying for state are considered. The distribution between these states at a game round can be found from that of the previous as where is the transition matrix, defined by
(2) |
where is the Kronecker delta. The corresponding payoff matrix is of similar form as but with for and , and is defined, where denotes the Hadamard product. The stationary distribution is characterized by the eigenvector equation , which can be converted to a coupled recurrence system and solved with a normalization constraint (see Appendix A2) to yield the solution
(3) |
for . The average outcome probabilities , , and , are
(4) |
for . The steady‐state per‐round change in expected capital and the capital mean are therefore
(5a) |
(5b) |
Equivalently, where is a column vector of ones; an alternative derivation via the capital distribution is given in Appendix A3. Next, obtaining the capital variance necessitates computing the fundamental matrix of the Markov chain,[ 45 , 48 ] , where is the limiting matrix comprising as rows and identity applies. A similar solving method can be used; the derivation and full solution for is detailed in Appendix A4. The steady‐state per‐round change in capital variance and capital variance are
(6a) |
(6b) |
where is the covariance in capital change between game rounds and . This yields
(7) |
where we denote as the modular successor and predecessor of index , and is given in Equation (3). Substituting into Equation (6) completes the closed‐form solution for . In summary, we have analytical solutions for parameter space boundaries in Equation (1), and capital statistics and in Equations (5) and (6). We note and are constructed from exact steady‐state and , and therefore neglect initial transients in the game; accuracy is when transients are present.
2.3. Validation
Monte Carlo simulations of the game structure were run to validate these analytical results. A comparison of and from Equations (5) and (6) against simulation results for differing is presented in Figure 1. The strong Parrondo effect arises when but and , and the weak effect arises so long as and . Phase plots of paradoxical regions and their boundaries are shown in Figure 1, and an example of and exhibiting the strong paradox is shown in Figure 2a. Lastly, fair boundaries () as predicted by Equation (1) are shown in Figure 2b,c, alongside simulation results. Excellent agreement across all theoretical and simulation results is observed. We provide implementation details of the simulation package in Appendix C.
3. Generalized M‐Branch Game Pair
A further generalization is considered, in which game comprises branches , with branch played when . Each is associated with winning, drawing, and losing probabilities , , and , respectively, and lose‐win ratio . Game remains unchanged. As before, , , and .
3.1. Conditions
Again, winning, fair, and losing conditions may first be derived. capital states satisfying for state are considered. By analyzing the probability of winning through consecutive states, a simple condition of identical form can be deduced
(8) |
where for stochastically mixed games and are the mixed lose‐win ratios. For pure game , , so a reduced is obtained, consistent with Equation (1) for the two‐branch structure; for pure game , , and is obtained. It is noted, furthermore, that under the conditions for and , the two‐branch results and are indeed recovered. The derivation of this result is detailed in Appendix B1.
3.2. Capital Statistics
Now we seek solutions for and . The transition matrix is generalized to
(9) |
The payoff matrix can again be obtained from with for and , and likewise . The eigenvector equation characterizing the stationary distribution between states can likewise be converted to a coupled recurrence system, but the recurrence now involves non‐constant coefficients, so the usual method of characteristic polynomials cannot be used. Instead, a tracking method can be used on the recursion tree to yield the solution
(10) |
for , and where counting functions
(11a) |
(11b) |
and is the non‐negative solution set to the Diophantine equation , parametrizable as , comprises all order permutations of ones and twos, and accumulator . The derivation of this solution is detailed in Appendix B2.
The average outcome probabilities are
(12) |
for . Substituting and into Equation (5) yields and for this generalized ‐branch structure. Obtaining likewise necessitates computing the fundamental matrix . A similar procedure as that for the two‐branch structure can be used, but again, as the recurrence is non‐constant, a tracking method has to be employed. The derivation and full solution for is given in Appendix B3. Expanding and simplifying gives
(13) |
Substituting into Equation (6) completes the closed‐form solution for . In summary, we have closed‐form solutions for parameter space boundaries in Equation (8), and capital statistics and in Equations (5), (6), (12), and (13). Again, since and are constructed from exact steady‐state and and neglect transients, an accuracy of is expected.
3.3. Validation
Monte Carlo simulations were likewise run to validate these theoretical results. Contour plots of and as computed from Equations (12) and (13) across pi parameter space for are illustrated in Figure 3a, for which simulation results match to excellent accuracy. A comparison of and from theory and simulations across – parameter space is also given in Figure 3b. The theoretical capital statistics and phase boundaries match simulations extremely well. Implementation details of the simulations are provided in Appendix C.
4. General Outlook
This paper has presented much‐sought explicit closed‐form solutions for capital statistics and parameter space boundaries for a family of generalized Parrondo's games, of which the canonical game pair[ 7 ] is a special case. The results are exact in steady‐state conditions, and of accuracy when initial transients are present; the effect of transients is dependent on initial conditions and game configuration, and will manifest as a small offset in capital statistics against steady‐state predictions. Importantly, the solutions accommodate arbitrary modulus and probability configurations; this work hence reports a fully general closed‐form solution for single‐agent capital‐dependent games. In comparison to usual Markov‐chain propagation calculations in predicting capital statistics, these solutions enable computational cost improvements in the treatment of related models (see Appendix D).
Recent developments in physics of living systems[ 20 , 21 , 36 ] and noise‐induced phenomena[ 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 ] have renewed interest in stochastic ratchet‐type mechanisms and the broader Parrondo framework—the methodology developed here is general to the multitude of Parrondo‐type model variants, so long as modelable as discrete‐time Markov chains (of arbitrary order), and can enable a paradigm shift from the current reliance on numerical simulations toward analytical methods. In particular, the theoretical treatment of ratcheting eco‐evolutionary and biomechanical models may benefit from the presented results. It may be of interest, in the future, to extend a similar analysis to history‐dependent game structures or multi‐agent games on arbitrary graph topologies.
Conflict of Interest
The authors declare no conflict of interest.
Supporting information
Acknowledgements
This project was supported by the Singapore University of Technology and Design (Grant No. SRG SCI 2019 142). The authors thank the three anonymous reviewers for aiding them in improving the paper.
Koh J. M., Cheong K. H., Generalized Solutions of Parrondo's Games. Adv. Sci. 2020, 7, 2001126 10.1002/advs.202001126
References
- 1. Rousselet J., Salome L., Ajdari A., Prostt J., Nature 1994, 370, 446. [DOI] [PubMed] [Google Scholar]
- 2. Parrondo J. M. R., Harmer G. P., Abbott D., Phys. Rev. Lett. 2000, 85, 5226. [DOI] [PubMed] [Google Scholar]
- 3. Ethier S. N., Lee J., R. Soc. Open Sci. 2018, 5, 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. von Smoluchowski M., Phys. Z. 1912, XIII, 1069. [Google Scholar]
- 5. Feynman R. P., Leighton R. B., Sands M., The Feynman Lectures on Physics, Vol. 1, Addison‐Wesley, Reading, MA: 1963. [Google Scholar]
- 6. Cao F. J., Dinis L., Parrondo J. M., Phys. Rev. Lett. 2004, 93, 040603. [DOI] [PubMed] [Google Scholar]
- 7. Harmer G. P., Abbott D., Nature 1999, 402, 864. [Google Scholar]
- 8. Astumian R. D., Bier M., Phys. Rev. Lett. 1994, 72, 1766. [DOI] [PubMed] [Google Scholar]
- 9. Doering C. R., Il Nuovo Cimento D 1995, 17, 685. [Google Scholar]
- 10. Magnasco M. O., Phys. Rev. Lett. 1993, 71, 1477. [DOI] [PubMed] [Google Scholar]
- 11. Danca M.‐F., Romera M. Fečkan, M., Int. J. Bifurc. Chaos 2014, 24, 1450008. [Google Scholar]
- 12. Chau N. P., Phys. Lett. A 1997, 234, 193. [Google Scholar]
- 13. Rosato A., Strandburg K. J., Prinz F., Swendsen R. H., Phys. Rev. Lett. 1987, 58, 1038. [DOI] [PubMed] [Google Scholar]
- 14. Pinsky R., Scheutzow M., Ann. Inst. Henri Poincare, B 1992, 28, 519. [Google Scholar]
- 15. Harmer G. P., Abbott D., Taylor P. G., Pearce C. E. M., Parrondo J. M. R., AIP Conf. Proc. 2000, 502, 544. [Google Scholar]
- 16. Cheong K. H., Saakian D. B., Zadourian R., Phys. Rev. E 2017, 96, 062303. [DOI] [PubMed] [Google Scholar]
- 17. Lai J. W., Cheong K. H., Nonlinear Dyn. 2020, 100, 849. [Google Scholar]
- 18. Koh J. M., Cheong K. H., J Electron Spectrosc. Relat. Phenom. 2018, 227, 31. [Google Scholar]
- 19. Cheong K. H., Koh J. M., Ultramicroscopy 2019, 202, 100. [DOI] [PubMed] [Google Scholar]
- 20. Tan Z.‐X., Koh J. M., Koonin E. V., Cheong K. H., Adv. Sci. 2020, 7, 1901559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Cheong K. H., Koh J. M., Jones M. C., Proc. Natl. Acad. Sci. U. S. A. 2018, 115, E5258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Cheong K. H., Tan Z. X., Xie N.‐g., Jones M. C., Sci. Rep. 2016, 6, 34889. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Wolf D. M., Vazirani V. V., Arkin A. P., J. Theor. Biol. 2005, 234, 2 227. [DOI] [PubMed] [Google Scholar]
- 24. Acar M., Mettetal J. T., van Oudenaarden A., Nat. Genet. 2008, 40, 471. [DOI] [PubMed] [Google Scholar]
- 25. Kussell E., Leibler S., Science 2005, 309, 2075. [DOI] [PubMed] [Google Scholar]
- 26. Tan Z.‐X., Cheong K. H., Nonlinear Dyn. 2019, 98, 1. [Google Scholar]
- 27. Koh J. M., Xie N.‐g., Cheong K. H., Nonlinear Dyn. 2018, 94, 1467. [Google Scholar]
- 28. Cheong K. H., Tan Z. X., Ling Y. H., Commun. Nonlinear Sci. Numer. Simul. 2018, 60, 107. [Google Scholar]
- 29. Tan Z. X., Cheong K. H., eLife 2017, 6, e21673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Ait‐Haddou R., Herzog W., Cell Biochem. Biophys. 2003, 38, 191. [DOI] [PubMed] [Google Scholar]
- 31. Kulish O., Wright A. D., Terentjev E. M., Sci. Rep. 2016, 6, 28180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Roth J. S., Zhang Y., Bao P., Cheetham M. R., Han X., Evans S. D., Appl. Phys. Lett. 2015, 106, 183703. [Google Scholar]
- 33. Koh J. M., Cheong K. H., Nonlinear Dyn. 2019, 96, 257. [Google Scholar]
- 34. Ye Y., Hang X. R., Koh J. M., Miszczak J. A., Cheong K. H., Xie N. G., Nonlinear Dyn. 2019, 98, 1821. [Google Scholar]
- 35. Ye Y., Hang X. R., Koh J. M., Miszczak J. A., Cheong K. H., Xie N.‐g., Chaos Soliton. Fract. 2020, 130, 109464. [Google Scholar]
- 36. Cheong K. H., Koh J. M., Jones M. C., BioEssays 2019, 41, 1900027. [Google Scholar]
- 37. Reed F. A., Genetics 2007, 176, 1923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Wolf D. M., Vazirani V. V., Arkin A. P., J. Theor. Biol. 2005, 234, 2 255. [DOI] [PubMed] [Google Scholar]
- 39. Levine J. M., Rees M., Am. Nat. 2004, 164, 350. [DOI] [PubMed] [Google Scholar]
- 40. Danca M.‐F., Lai D., Int. J. Bifurcation Chaos 2012, 22, 1250258. [Google Scholar]
- 41. Danca M.‐F., Romera M., Pastor G., Montoya F., Nonlinear Dyn. 2012, 67, 2317. [Google Scholar]
- 42. Gatenby R. A., Maini P. K., Nature 2003, 421, 321. [DOI] [PubMed] [Google Scholar]
- 43. Harmer G. P., Abbott D., Fluct. Noise Lett. 2002, 2, R71. [Google Scholar]
- 44. Ethier S., Lee J., Electron. J. Probab. 2012, 17, 1. [Google Scholar]
- 45. Ethier S., Lee J., Electron. J. Probab. 2009, 14, 1827. [Google Scholar]
- 46. Pyke R., in Mathematical Statistics and Applications: Festschrift for Constance Van Eeden, Vol. 42, Institute of Mathematical Statistics, Beachwood, Ohio: 2003, pp. 185–216. [Google Scholar]
- 47. Shu J.‐J., Wang Q.‐W., Sci. Rep. 2014, 4, 4244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Kemeny J. G., Snell J. L., Markov Chains, Springer‐Verlag, New York: 1976. [Google Scholar]
- 49. Saakian D. B., Yakushkina T., Koonin E. V., Phys. Rev. E 2019, 99, 022407. [DOI] [PubMed] [Google Scholar]
- 50. Yakushkina T., Saakian D. B., Bratus A., Hu C.‐K., J. Phys. Soc. Jpn. 2015, 84, 064802. [Google Scholar]
- 51. Deng X., Liu Q., Deng Y., Appl. Math. Comput. 2016, 273, 868. [Google Scholar]
- 52. Han Y., Deng Y., Int. J. Intell. Sys. 2019, 34, 690. [Google Scholar]
- 53. Buceta J., Lindenberg K., Parrondo J. M. R., Phys. Rev. Lett. 2001, 88, 024103. [DOI] [PubMed] [Google Scholar]
- 54. Sagués F., Sancho J. M., García‐Ojalvo J., Rev. Mod. Phys. 2007, 79, 829. [Google Scholar]
- 55. Lo W.‐C., Zheng L., Nie Q., R. Soc. Open Sci. 2016, 3, 160485. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Lei J., Lo W.‐C., Nie Q., Ann. Math. Sci. Appl. 2016, 1, 427. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.