Skip to main content
Journal of the Experimental Analysis of Behavior logoLink to Journal of the Experimental Analysis of Behavior
. 1982 Sep;38(2):217–232. doi: 10.1901/jeab.1982.38-217

Incentive theory: II. Models for choice

Peter R Killeen
PMCID: PMC1347819  PMID: 16812299

Abstract

Incentive theory is extended to account for concurrent chained schedules of reinforcement. The basic model consists of additive contributions from the primary and secondary effects of reinforcers, which serve to direct the behavior activated by reinforcement. The activation is proportional to the rate of reinforcement and interacts multiplicatively with the directive effects. The two free parameters are q, the slope of the delay of reinforcement gradient, whose value is constant across many experiments, and b, a bias parameter. The model is shown to provide an excellent description of all results from studies that have varied the terminal-link schedules, and of many of the results from studies that have varied initial-link schedules. The model is extended to diverse modifications of the terminal links, such as varied amount of reinforcement, varied signaling of the terminal-link schedules, and segmentation of the terminal-link schedules. It is demonstrated that incentive theory provides an accurate and integrated account of many of the phenomena of choice.

Keywords: concurrent chained schedules, mathematical models of choice

Full text

PDF
217

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Ainslie G. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol Bull. 1975 Jul;82(4):463–496. doi: 10.1037/h0076860. [DOI] [PubMed] [Google Scholar]
  2. Catania A. C. Freedom and knowledge: an experimental analysis of preference in pigeons. J Exp Anal Behav. 1975 Jul;24(1):89–106. doi: 10.1901/jeab.1975.24-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Catania A. C., Sagvolden T. Preference for free choice over forced choice in pigeons. J Exp Anal Behav. 1980 Jul;34(1):77–86. doi: 10.1901/jeab.1980.34-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chung S. H. Effects of delayed reinforcement in a concurrent situation. J Exp Anal Behav. 1965 Nov;8(6):439–444. doi: 10.1901/jeab.1965.8-439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chung S. H., Herrnstein R. J. Choice and delay of reinforcement. J Exp Anal Behav. 1967 Jan;10(1):67–74. doi: 10.1901/jeab.1967.10-67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cicerone R. A. Preference for mixed versus constant delay of reinforcement. J Exp Anal Behav. 1976 Mar;25(2):257–261. doi: 10.1901/jeab.1976.25-257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Davison M. C., Hunter I. W. Concurrent schedules: undermatching and control by previous experimental conditions. J Exp Anal Behav. 1979 Sep;32(2):233–244. doi: 10.1901/jeab.1979.32-233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Davison M. C. Preference for fixed-interval schedules: effects of unequal initial links. J Exp Anal Behav. 1976 May;25(3):371–376. doi: 10.1901/jeab.1976.25-371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Davison M. C. Preference for mixed-interval versus fixed-interval schedules. J Exp Anal Behav. 1969 Mar;12(2):247–252. doi: 10.1901/jeab.1969.12-247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davison M. C., Temple W. Preference for fixed-interval schedules: an alternative model. J Exp Anal Behav. 1973 Nov;20(3):393–403. doi: 10.1901/jeab.1973.20-393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dinsmoor J. A., Mulvaney D. E., Jwaideh A. R. Conditioned reinforcement as a function of duration of stimulus. J Exp Anal Behav. 1981 Jul;36(1):41–49. doi: 10.1901/jeab.1981.36-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Donahoe J. W. Some implications of a relational principle of reinforcement. J Exp Anal Behav. 1977 Mar;27(2):341–350. doi: 10.1901/jeab.1977.27-341. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Duncan B., Fantino E. Choice for periodic schedules of reinforcement. J Exp Anal Behav. 1970 Jul;14(1):73–86. doi: 10.1901/jeab.1970.14-73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Duncan B., Fantino E. The psychological distance to reward. J Exp Anal Behav. 1972 Jul;18(1):23–34. doi: 10.1901/jeab.1972.18-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Fantino E. Choice and rate of reinforcement. J Exp Anal Behav. 1969 Sep;12(5):723–730. doi: 10.1901/jeab.1969.12-723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Fantino E., Moore J. Uncertainty reduction, conditioned reinforcement, and observing. J Exp Anal Behav. 1980 Jan;33(1):3–13. doi: 10.1901/jeab.1980.33-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fantino E. Preference for mixed- versus fixed-ratio schedules. J Exp Anal Behav. 1967 Jan;10(1):35–43. doi: 10.1901/jeab.1967.10-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Findley J. D. Preference and Switching under Concurrent Scheduling. J Exp Anal Behav. 1958 Apr;1(2):123–144. doi: 10.1901/jeab.1958.1-123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gentry G. D., Marr M. J. Choice and reinforcement delay. J Exp Anal Behav. 1980 Jan;33(1):27–37. doi: 10.1901/jeab.1980.33-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Green L., Rachlin H. Pigeons' preferences for stimulus information: effects of amount of information. J Exp Anal Behav. 1977 Mar;27(2):255–263. doi: 10.1901/jeab.1977.27-255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Green L., Snyderman M. Choice between rewards differing in amount and delay: Toward a choice model of self control. J Exp Anal Behav. 1980 Sep;34(2):135–147. doi: 10.1901/jeab.1980.34-135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. HERRNSTEIN R. J. SECONDARY REINFORCEMENT AND RATE OF PRIMARY REINFORCEMENT. J Exp Anal Behav. 1964 Jan;7:27–36. doi: 10.1901/jeab.1964.7-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hursh S. R., Fantino E. An appraisal of preference for multiple versus mixed schedules. J Exp Anal Behav. 1974 Jul;22(1):31–38. doi: 10.1901/jeab.1974.22-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hursh S. R., Fantino E. Relative delay of reinforcement and choice. J Exp Anal Behav. 1973 May;19(3):437–450. doi: 10.1901/jeab.1973.19-437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ito M., Asaki K. Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement. J Exp Anal Behav. 1982 May;37(3):383–392. doi: 10.1901/jeab.1982.37-383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Johnson J. H., Williams T. A. An exploratory study of the interaction between personality and psychopathology. J Clin Psychol. 1978 Apr;34(2):371–379. doi: 10.1002/1097-4679(197804)34:2<371::aid-jclp2270340222>3.0.co;2-m. [DOI] [PubMed] [Google Scholar]
  27. Kaplan P. S., Hearst E. Bridging temporal gaps between CS and US in autoshaping: insertion of other stimuli before, during, and after CS. J Exp Psychol Anim Behav Process. 1982 Apr;8(2):187–203. [PubMed] [Google Scholar]
  28. Killeen P. R., Hanson S. J., Osborne S. R. Arousal: its genesis and manifestation as response rate. Psychol Rev. 1978 Nov;85(6):571–581. [PubMed] [Google Scholar]
  29. Killeen P. On the measurement of reinforcement frequency in the study of preference. J Exp Anal Behav. 1968 May;11(3):263–269. doi: 10.1901/jeab.1968.11-263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Lendenmann K. W., Myers D. L., Fantino E. Effects of reinforcer duration on responding in two-link chained interval schedules. J Exp Anal Behav. 1982 Mar;37(2):217–222. doi: 10.1901/jeab.1982.37-217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Macewen D. The effects of terminal-link fixed-interval and variable-interval schedules on responding under concurrent chained schedules. J Exp Anal Behav. 1972 Sep;18(2):253–261. doi: 10.1901/jeab.1972.18-253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Marcattilio A. J., Richards R. W. Preference for signaled versus unsignaled reinforcement delay in concurrent-chain schedules. J Exp Anal Behav. 1981 Sep;36(2):221–229. doi: 10.1901/jeab.1981.36-221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. McMillan J. C. Average uncertainty as a determinant of observing behavior. J Exp Anal Behav. 1974 Sep;22(2):401–408. doi: 10.1901/jeab.1974.22-401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Moore J. Choice and number of reinforcers. J Exp Anal Behav. 1979 Jul;32(1):51–63. doi: 10.1901/jeab.1979.32-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Nevin J. A., Mandell C. Conditioned reinforcement and choice. J Exp Anal Behav. 1978 Jan;29(1):135–148. doi: 10.1901/jeab.1978.29-135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Rachlin H., Green L. Commitment, choice and self-control. J Exp Anal Behav. 1972 Jan;17(1):15–22. doi: 10.1901/jeab.1972.17-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rescorla R. A. Effect of a stimulus intervening between CS and US in autoshaping. J Exp Psychol Anim Behav Process. 1982 Apr;8(2):131–141. [PubMed] [Google Scholar]
  38. Reynolds G. S. Potency of Conditioned Reinforcers Based on Food and on Food and Punishment. Science. 1963 Mar 1;139(3557):838–839. doi: 10.1126/science.139.3557.838. [DOI] [PubMed] [Google Scholar]
  39. Salmon D. P., D'Amato M. R. Note on delay-interval illumination effects on retention in monkeys (Cebus apella). J Exp Anal Behav. 1981 Nov;36(3):381–385. doi: 10.1901/jeab.1981.36-381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Shull R. L., Spear D. J., Bryson A. E. Delay or rate of food delivery as determiners of response rate. J Exp Anal Behav. 1981 Mar;35(2):129–143. doi: 10.1901/jeab.1981.35-129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Squires N., Fantino E. A model for choice in simple concurrent and concurrent-chains schedules. J Exp Anal Behav. 1971 Jan;15(1):27–38. doi: 10.1901/jeab.1971.15-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. WYCKOFF L. B., Jr The role of observing responses in discrimination learning. Psychol Rev. 1952 Nov;59(6):431–442. doi: 10.1037/h0053932. [DOI] [PubMed] [Google Scholar]
  43. Wardlaw G. R., Davison M. C. Preference for fixed-interval schedules: effects of initial-link length. J Exp Anal Behav. 1974 Mar;21(2):331–340. doi: 10.1901/jeab.1974.21-331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Wilkie D. M., Summers R. J., Spetch M. L. Effect of delay-interval stimuli on delayed symbolic matching to sample in the pigeon. J Exp Anal Behav. 1981 Mar;35(2):153–160. doi: 10.1901/jeab.1981.35-153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Williams B. A., Fantino E. Effects on choice of reinforcement delay and conditioned reinforcement. J Exp Anal Behav. 1978 Jan;29(1):77–86. doi: 10.1901/jeab.1978.29-77. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior

RESOURCES