Skip to main content
Journal of the Experimental Analysis of Behavior logoLink to Journal of the Experimental Analysis of Behavior
. 2000 Jul;74(1):1–24. doi: 10.1901/jeab.2000.74-1

Choice in a variable environment: every reinforcer counts.

M Davison 1, W M Baum 1
PMCID: PMC1284782  PMID: 10966094

Abstract

Six pigeons were trained in sessions composed of seven components, each arranged with a different concurrent-schedule reinforcer ratio. These components occurred in an irregular order with equal frequency, separated by 10-s blackouts. No signals differentiated the different reinforcer ratios. Conditions lasted 50 sessions, and data were collected from the last 35 sessions. In Part 1, the arranged overall reinforcer rate was 2.22 reinforcers per minute. Over conditions, number of reinforcers per component was varied from 4 to 12. In Part 2, the overall reinforcer rate was six per minute, with both 4 and 12 reinforcers per component. Within components, log response-allocation ratios adjusted rapidly as more reinforcers were delivered in the component, and the slope of the choice relation (sensitivity) leveled off at moderately high levels after only about eight reinforcers. When the carryover from previous components was taken into account, the number of reinforcers in the components appeared to have no systematic effect on the speed at which behavior changed after a component started. Consequently, sensitivity values at each reinforcer delivery were superimposable. However, adjustment to changing reinforcer ratios was faster, and reached greater sensitivity values, when overall reinforcer rate was higher. Within a component, each successive reinforcer from the same alternative ("confirming") had a smaller effect than the one before, but single reinforcers from the other alternative ("disconfirming") always had a large effect. Choice in the prior component carried over into the next component, and its effects could be discerned even after five or six reinforcement and nonreinforcement is suggested.

Full Text

The Full Text of this article is available as a PDF (420.5 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Alsop B., Elliffe D. Concurrent-schedule performance: Effects of relative and overall reinforcer rate. J Exp Anal Behav. 1988 Jan;49(1):21–36. doi: 10.1901/jeab.1988.49-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bailey J. T., Mazur J. E. Choice behavior in transition: development of preference for the higher probability of reinforcement. J Exp Anal Behav. 1990 May;53(3):409–422. doi: 10.1901/jeab.1990.53-409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baum W. M. Matching, undermatching, and overmatching in studies of choice. J Exp Anal Behav. 1979 Sep;32(2):269–281. doi: 10.1901/jeab.1979.32-269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baum W. M. On two types of deviation from the matching law: bias and undermatching. J Exp Anal Behav. 1974 Jul;22(1):231–242. doi: 10.1901/jeab.1974.22-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Baum W., Schwendiman J., Bell K. Choice, contingency discrimination, and foraging theory. J Exp Anal Behav. 1999 May;71(3):355–373. doi: 10.1901/jeab.1999.71-355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Belke T. W., Heyman G. M. Increasing and signaling background reinforcement: effect on the foreground response-reinforcer relation. J Exp Anal Behav. 1994 Jan;61(1):65–81. doi: 10.1901/jeab.1994.61-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Davis D. G., Staddon J. E., Machado A., Palmer R. G. The process of recurrent choice. Psychol Rev. 1993 Apr;100(2):320–341. doi: 10.1037/0033-295x.100.2.320. [DOI] [PubMed] [Google Scholar]
  8. Davison M. C., Hunter I. W. Concurrent schedules: undermatching and control by previous experimental conditions. J Exp Anal Behav. 1979 Sep;32(2):233–244. doi: 10.1901/jeab.1979.32-233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Davison M., Jones B. M. A quantitative analysis of extreme choice. J Exp Anal Behav. 1995 Sep;64(2):147–162. doi: 10.1901/jeab.1995.64-147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davison M., Jones B. Performance On Concurrent Variable-interval Extinction Schedules. J Exp Anal Behav. 1998 Jan;69(1):49–57. doi: 10.1901/jeab.1998.69-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Davison M., Nevin J. Stimuli, reinforcers, and behavior: an integration. J Exp Anal Behav. 1999 May;71(3):439–482. doi: 10.1901/jeab.1999.71-439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Elliffe D., Alsop B. Concurrent choice: Effects of overall reinforcer rate and the temporal distribution of reinforcers. J Exp Anal Behav. 1996 Mar;65(2):445–463. doi: 10.1901/jeab.1996.65-445. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. HERRNSTEIN R. J. Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav. 1961 Jul;4:267–272. doi: 10.1901/jeab.1961.4-267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Herrnstein R. J. On the law of effect. J Exp Anal Behav. 1970 Mar;13(2):243–266. doi: 10.1901/jeab.1970.13-243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hunter I., Davison M. Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules. J Exp Anal Behav. 1985 Jan;43(1):43–59. doi: 10.1901/jeab.1985.43-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Killeen P. R. Stability criteria. J Exp Anal Behav. 1978 Jan;29(1):17–25. doi: 10.1901/jeab.1978.29-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lobb B., Davison M. C. Performance in concurrent interval schedules: a systematic replication. J Exp Anal Behav. 1975 Sep;24(2):191–197. doi: 10.1901/jeab.1975.24-191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Mazur J. E. Choice behavior in transition: development of preference with ratio and interval schedules. J Exp Psychol Anim Behav Process. 1992 Oct;18(4):364–378. doi: 10.1037//0097-7403.18.4.364. [DOI] [PubMed] [Google Scholar]
  19. Schofield G., Davison M. Nonstable concurrent choice in pigeons. J Exp Anal Behav. 1997 Sep;68(2):219–232. doi: 10.1901/jeab.1997.68-219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Taylor R., Davison M. Sensitivity to reinforcement in concurrent arithmetic and exponential schedules. J Exp Anal Behav. 1983 Jan;39(1):191–198. doi: 10.1901/jeab.1983.39-191. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior

RESOURCES