Skip to main content

Some NLM-NCBI services and products are experiencing heavy traffic, which may affect performance and availability. We apologize for the inconvenience and appreciate your patience. For assistance, please contact our Help Desk at info@ncbi.nlm.nih.gov.

Journal of Applied Behavior Analysis logoLink to Journal of Applied Behavior Analysis
. 1977 Spring;10(1):127–131. doi: 10.1901/jaba.1977.10-127

A probability-based formula for calculating interobserver agreement1

Ann R Yelton 1,2,3,4, Beth G Wildman 1,2,3,4, Marilyn T Erickson 1,2,3,4
PMCID: PMC1311159  PMID: 16795541

Abstract

Estimates of observer agreement are necessary to assess the acceptability of interval data. A common method for assessing observer agreement, per cent agreement, includes several major weaknesses and varies as a function of the frequency of behavior recorded and the inclusion or exclusion of agreements on nonoccurrences. Also, agreements that might be expected to occur by chance are not taken into account. An alternative method for assessing observer agreement that determines the exact probability that the obtained number of agreements or better would have occurred by chance is presented and explained. Agreements on both occurrences and nonoccurrences of behavior are considered in the calculation of this probability.

Keywords: observer agreement, probability-based formula, recording and measurement techniques, time sampling, reliability

Full text

PDF
127

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Werry J. S., Quay H. C. Observing the classroom behavior of elementary school children. Except Child. 1969 Feb;35(6):461–470. [PubMed] [Google Scholar]

Articles from Journal of Applied Behavior Analysis are provided here courtesy of Society for the Experimental Analysis of Behavior

RESOURCES