Skip to main content
NCBI home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2020 Apr 18;12075:56–83. doi: 10.1007/978-3-030-44914-8_3

On the Versatility of Open Logical Relations

Continuity, Automatic Differentiation, and a Containment Theorem

Gilles Barthe 9,12, Raphaëlle Crubillé 12, Ugo Dal Lago 10,11,✉, Francesco Gavazzo 10,11,12
Editor: Peter Müller1
PMCID: PMC7702265

Abstract

Logical relations are one among the most powerful techniques in the theory of programming languages, and have been used extensively for proving properties of a variety of higher-order calculi. However, there are properties that cannot be immediately proved by means of logical relations, for instance program continuity and differentiability in higher-order languages extended with real-valued functions. Informally, the problem stems from the fact that these properties are naturally expressed on terms of non-ground type (or, equivalently, on open terms of base type), and there is no apparent good definition for a base case (i.e. for closed terms of ground types). To overcome this issue, we study a generalization of the concept of a logical relation, called open logical relation, and prove that it can be fruitfully applied in several contexts in which the property of interest is about expressions of first-order type. Our setting is a simply-typed λ-calculus enriched with real numbers and real-valued first-order functions from a given set, such as the one of continuous or differentiable functions. We first prove a containment theorem stating that for any collection of real-valued first-order functions including projection functions and closed under function composition, any well-typed term of first-order type denotes a function belonging to that collection. Then, we show by way of open logical relations the correctness of the core of a recently published algorithm for forward automatic differentiation. Finally, we define a refinement-based type system for local continuity in an extension of our calculus with conditionals, and prove the soundness of the type system using open logical relations.

Keywords: Lambda Calculus, Logical Relations, Continuity Analysis, Automatic Differentiation

Footnotes

The Second and Fourth Authors are supported by the ANR project 16CE250011 REPAS, the ERC Consolidator Grant DIAPASoN – DLV-818616, and the MIUR PRIN 201784YSZ5 ASPRA.

Contributor Information

Peter Müller, Email: peter.mueller@inf.ethz.ch.

Ugo Dal Lago, Email: ugo.dallago@unibo.it.

References

  • 1.Abadi, M., Plotkin, G.D.: A simple differentiable programming language. PACMPL 4(POPL), 38:1–38:28 (2020)
  • 2.Ahmed, A.J.: Step-indexed syntactic logical relations for recursive and quantified types. In: Proc. of ESOP 2006. pp. 69–83 (2006)
  • 3.Appel, A.W., McAllester, D.A.: An indexed model of recursive types for foundational proof-carrying code. ACM Trans. Program. Lang. Syst. 23(5), 657–683 (2001)
  • 4.Appel, A.W., Mellies, P.A., Richards, C.D., Vouillon, J.: A very modal model of a modern, major, general type system. In: ACM SIGPLAN Notices. vol. 42, pp. 109–122. ACM (2007)
  • 5.Baillot, P., Dal Lago, U.: Higher-order interpretations and program complexity. In: Proc. of CSL 2012. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2012)
  • 6.Barendregt, H.P.: The lambda calculus: its syntax and semantics. North-Holland (1984)
  • 7.Barthe, G., Crubillé, R., Dal Lago, U., Gavazzo, F.: On the versatility of open logical relations: Continuity, automatic differentiation, and a containment theorem (long version) (2019), available at https://arxiv.org/abs/2002.08489
  • 8.Bartholomew-Biggs, M., Brown, S., Christianson, B., Dixon, L.: Automatic differentiation of algorithms. Journal of Computational and Applied Mathematics 124(1), 171–190 (2000), numerical Analysis 2000. Vol. IV: Optimization and Nonlinear Equations
  • 9.Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research 18, 153:1–153:43 (2017)
  • 10.Benton, N., Hofmann, M., Nigam, V.: Abstract effects and proof-relevant logical relations. In: Proc. of POPL 2014. pp. 619–632 (2014)
  • 11.Biernacki, D., Piróg, M., Polesiuk, P., Sieczkowski, F.: Handle with care: relational interpretation of algebraic effects and handlers. PACMPL 2(POPL), 8:1–8:30 (2018)
  • 12.Birkedal, L., Jaber, G., Sieczkowski, F., Thamsborg, J.: A kripke logical relation for effect-based program transformations. Inf. Comput. 249, 160–189 (2016)
  • 13.Birkedal, L., Sieczkowski, F., Thamsborg, J.: A concurrent logical relation. In: Proc. of CSL 2012. pp. 107–121 (2012)
  • 14.Bizjak, A., Birkedal, L.: Step-indexed logical relations for probability. In: Proc. of FoSSaCS 2015. pp. 279–294 (2015)
  • 15.Bowman, W.J., Ahmed, A.: Noninterference for free. In: Proc. of ICFP 2015. pp. 101–113 (2015)
  • 16.Brunel, A., Mazza, D., Pagani, M.: Backpropagation in the simply typed lambda-calculus with linear negation. PACMPL 4(POPL), 64:1–64:27 (2020)
  • 17.Brunel, A., Terui, K.: Church => scott = ptime: an application of resource sensitive realizability. In: Proc. of DICE 2010. pp. 31–46 (2010)
  • 18.Chaudhuri, S., Gulwani, S., Lublinerman, R.: Continuity analysis of programs. In: Proc. of POPL 2010. pp. 57–70 (2010)
  • 19.Chaudhuri, S., Gulwani, S., Lublinerman, R.: Continuity and robustness of programs. Commun. ACM 55(8), 107–115 (2012)
  • 20.Chaudhuri, S., Gulwani, S., Lublinerman, R., NavidPour, S.: Proving programs robust. In: Proc. of SIGSOFT/FSE 2011. pp. 102–112 (2011)
  • 21.Clifford: Preliminary Sketch of Biquaternions. Proceedings of the London Mathematical Society s1-4(1), 381–395 (11 1871)
  • 22.Cook, S.A., Kapron, B.M.: Characterizations of the basic feasible functionals of finite type (extended abstract). In: 30th Annual Symposium on Foundations of Computer Science, Research Triangle Park, North Carolina, USA, 30 October - 1 November 1989. pp. 154–159 (1989)
  • 23.Crary, K., Harper, R.: Syntactic logical relations for polymorphic and recursive types. Electr. Notes Theor. Comput. Sci. 172, 259–299 (2007)
  • 24.Crole, R.L.: Categories for Types. Cambridge mathematical textbooks, Cambridge University Press (1993)
  • 25.Dreyer, D., Neis, G., Birkedal, L.: The impact of higher-order state and control effects on local relational reasoning. J. Funct. Program. 22(4–5), 477–528 (2012)
  • 26.Edalat, A.: The domain of differentiable functions. Electr. Notes Theor. Comput. Sci. 40, 144 (2000)
  • 27.Edalat, A., Lieutier, A.: Domain theory and differential calculus (functions of one variable). In: Proc. of LICS 2002. pp. 277–286 (2002)
  • 28.Elliott, C.: The simple essence of automatic differentiation. PACMPL 2(ICFP), 70:1–70:29 (2018)
  • 29.Escardó, M.H., Ho, W.K.: Operational domain theory and topology of sequential programming languages. Inf. Comput. 207(3), 411–437 (2009)
  • 30.Fiore, M.P.: Semantic analysis of normalisation by evaluation for typed lambda calculus. In: Proc. of PPDP 2002. pp. 26–37 (2002)
  • 31.Freeman, T., Pfenning, F.: Refinement types for ml. In: Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation. pp. 268–277. PLDI ’91 (1991)
  • 32.Gianantonio, P.D., Edalat, A.: A language for differentiable functions. In: Proc. of FOSSACS 2013. pp. 337–352 (2013)
  • 33.Girard, J.Y.: Une extension de l’interpretation de gödel a l’analyse, et son application a l’elimination des coupures dans l’analyse et la theorie des types. In: Studies in Logic and the Foundations of Mathematics, vol. 63, pp. 63–92. Elsevier (1971)
  • 34.Goubault-Larrecq, J., Lasota, S., Nowak, D.: Logical relations for monadic types. In: International Workshop on Computer Science Logic. pp. 553–568. Springer (2002)
  • 35.Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, second edn. (2008)
  • 36.Hofmann, M.: Logical relations and nondeterminism. In: Software, Services, and Systems - Essays Dedicated to Martin Wirsing on the Occasion of His Retirement from the Chair of Programming and Software Engineering. pp. 62–74 (2015)
  • 37.Huot, M., Staton, S., Vákár, M.: Correctness of automatic differentiation via diffeologies and categorical gluing (2020), to appear in Proc. of ESOP 2020 (long version available at http://arxiv.org/abs/2001.02209)
  • 38.Jaber, G.: Syteci: automating contextual equivalence for higher-order programs with references. PACMPL 4(POPL), 59:1–59:28 (2020)
  • 39.Jung, A., Tiuryn, J.: A new characterization of lambda definability. In: Proc. of TLCA 1993. pp. 245–257 (1993)
  • 40.Kapron, B.M., Cook, S.A.: A new characterization of type-2 feasibility. SIAM J. Comput. 25(1), 117–132 (1996)
  • 41.Lafont, Y.: Logiques, catégories & machines: implantation de langages de programmation guidée par la logique catégorique. Institut national de recherche en informatique et en automatique (1988)
  • 42.Manzyuk, O., Pearlmutter, B.A., Radul, A.A., Rush, D.R., Siskind, J.M.: Perturbation confusion in forward automatic differentiation of higher-order functions. J. Funct. Program. 29, e12 (2019)
  • 43.Mitchell, J.C.: Foundations for programming languages. Foundation of computing series, MIT Press (1996)
  • 44.Owens, S., Myreen, M.O., Kumar, R., Tan, Y.K.: Functional big-step semantics. In: Proc. of ESOP 2016. pp. 589–615 (2016)
  • 45.Pearlmutter, B.A., Siskind, J.M.: Lazy multivariate higher-order forward-mode AD. In: Proc. of POPL 2007. pp. 155–160 (2007)
  • 46.Pearlmutter, B.A., Siskind, J.M.: Reverse-mode AD in a functional framework: Lambda the ultimate backpropagator. ACM Trans. Program. Lang. Syst. 30(2), 7:1–7:36 (2008)
  • 47.Pitts, A.M., Stark, I.D.B.: Observable properties of higher order functions that dynamically create local names, or what’s new? In: Proc. of MFCS 1993. pp. 122–141 (1993)
  • 48.Plotkin, G.: Lambda-definability and logical relations. Edinburgh University (1973)
  • 49.Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Neurocomputing: Foundations of research. chap. Learning Representations by Back-propagating Errors, pp. 696–699. MIT Press (1988)
  • 50.Shaikhha, A., Fitzgibbon, A., Vytiniotis, D., Peyton Jones, S.: Efficient differentiable programming in a functional array-processing language. PACMPL 3(ICFP), 97:1–97:30 (2019)
  • 51.Siskind, J.M., Pearlmutter, B.A.: Nesting forward-mode AD in a functional framework. Higher-Order and Symbolic Computation 21(4), 361–376 (2008)
  • 52.Spivak, M.: Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus. Avalon Publishing (1971)
  • 53.Staton, S., Yang, H., Wood, F.D., Heunen, C., Kammar, O.: Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints. In: Proc. of LICS 2016. pp. 525–534 (2016)
  • 54.Tait, W.W.: Intensional interpretations of functionals of finite type i. Journal of Symbolic Logic 32(2), 198–212 (1967)
  • 55.Tait, W.W.: A realizability interpretation of the theory of species. In: Logic Colloquium. pp. 240–251. Springer, Berlin, Heidelberg (1975)
  • 56.Turon, A.J., Thamsborg, J., Ahmed, A., Birkedal, L., Dreyer, D.: Logical relations for fine-grained concurrency. In: Proc. of POPL 2013. pp. 343–356 (2013)
  • 57.Vuillemin, J.: Exact real computer arithmetic with continued fractions. IEEE Trans. Comput. 39(8), 1087–1105 (1990)
  • 58.Weihrauch, K.: Computable Analysis: An Introduction. Texts in Theoretical Computer Science. An EATCS Series, Springer, Berlin Heidelberg (2000)
  • 59.Zhao, J., Zhang, Q., Zdancewic, S.: Relational parametricity for a polymorphic linear lambda calculus. In: Proc. of APLAS 2010. pp. 344–359 (2010)

Articles from Programming Languages and Systems are provided here courtesy of Nature Publishing Group

RESOURCES