Skip to main content
NCBI home page
Journal of the Experimental Analysis of Behavior logoLink to Journal of the Experimental Analysis of Behavior
. 1981 May;35(3):367–396. doi: 10.1901/jeab.1981.35-367

How to maximize reward rate on two variable-interval paradigms

Alasdair I Houston, John McNamara
PMCID: PMC1333091  PMID: 16812223

Abstract

Without assuming any constraints on behavior, we derive the policy that maximizes overall reward rate on two variable-interval paradigms. The first paradigm is concurrent variable time-variable time with changeover delay. It is shown that for nearly all parameter values, a switch to the schedule with the longer interval should be followed immediately by a switch back to the schedule with the shorter interval. The matching law does not hold at the optimum and does not uniquely specify the obtained reward rate. The second paradigm is discrete trial concurrent variable interval-variable interval. For given schedule parameters, the optimal policy involves a cycle of a fixed number of choices of the schedule with the shorter interval followed by one choice of the schedule with the longer interval. Molecular maximization sometimes results in optimal behavior.

Keywords: optimization, matching, immediate maximization, reward rate, variable-interval schedule

Full text

PDF
367

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Allison T. S., Lloyd K. E. Concurrent schedules of reinforcement: effects of gradual and abrupt increases in changeover delay. J Exp Anal Behav. 1971 Jul;16(1):67–73. doi: 10.1901/jeab.1971.16-67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Baum W. M. On two types of deviation from the matching law: bias and undermatching. J Exp Anal Behav. 1974 Jul;22(1):231–242. doi: 10.1901/jeab.1974.22-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baum W. M., Rachlin H. C. Choice as time allocation. J Exp Anal Behav. 1969 Nov;12(6):861–874. doi: 10.1901/jeab.1969.12-861. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brownstein A. J., Pliskoff S. S. Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement. J Exp Anal Behav. 1968 Nov;11(6):683–688. doi: 10.1901/jeab.1968.11-683. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Charnov E. L. Optimal foraging, the marginal value theorem. Theor Popul Biol. 1976 Apr;9(2):129–136. doi: 10.1016/0040-5809(76)90040-x. [DOI] [PubMed] [Google Scholar]
  6. FLESHLER M., HOFFMAN H. S. A progression for generating variable-interval schedules. J Exp Anal Behav. 1962 Oct;5:529–530. doi: 10.1901/jeab.1962.5-529. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. HERRNSTEIN R. J. Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav. 1961 Jul;4:267–272. doi: 10.1901/jeab.1961.4-267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Herrnstein R. J. On the law of effect. J Exp Anal Behav. 1970 Mar;13(2):243–266. doi: 10.1901/jeab.1970.13-243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Heyman G. M. A Markov model description of changeover probabilities on concurrent variable-interval schedules. J Exp Anal Behav. 1979 Jan;31(1):41–51. doi: 10.1901/jeab.1979.31-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. McDowell J. J., Kessel R. A multivariate rate equation for variable-interval performance. J Exp Anal Behav. 1979 Mar;31(2):267–283. doi: 10.1901/jeab.1979.31-267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. McNamara J., Houston A. The application of statistical decision theory to animal behaviour. J Theor Biol. 1980 Aug 21;85(4):673–690. doi: 10.1016/0022-5193(80)90265-9. [DOI] [PubMed] [Google Scholar]
  12. Nevin J. A. Interval reinforcement of choice behavior in discrete trials. J Exp Anal Behav. 1969 Nov;12(6):875–885. doi: 10.1901/jeab.1969.12-875. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Pliskoff S. S. Effects of symmetrical and asymmetrical changeover delays on concurrent performances. J Exp Anal Behav. 1971 Sep;16(2):249–256. doi: 10.1901/jeab.1971.16-249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Rachlin H. A molar theory of reinforcement schedules. J Exp Anal Behav. 1978 Nov;30(3):345–360. doi: 10.1901/jeab.1978.30-345. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Shull R. L., Pliskoff S. S. Changeover delay and concurrent schedules: some effects on relative performance measures. J Exp Anal Behav. 1967 Nov;10(6):517–527. doi: 10.1901/jeab.1967.10-517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Silberberg A., Fantino E. Choice, rate of reinforcement, and the changeover delay. J Exp Anal Behav. 1970 Mar;13(2):187–197. doi: 10.1901/jeab.1970.13-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Staddon J. E. On Herrnstein's equation and related forms. J Exp Anal Behav. 1977 Sep;28(2):163–170. doi: 10.1901/jeab.1977.28-163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Staddon J. E. Spaced responding and choice: a preliminary analysis. J Exp Anal Behav. 1968 Nov;11(6):669–682. doi: 10.1901/jeab.1968.11-669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Todorov J. C. Concurrent performances: effect of punishment contingent on the switching response. J Exp Anal Behav. 1971 Jul;16(1):51–62. doi: 10.1901/jeab.1971.16-51. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior

RESOURCES