Abstract
Visual analysis is the dominant method of analysis for single-case time series. The literature assumes that visual analysts will be conservative judges. We show that previous research into visual analysis has not adequately examined false alarm and miss rates or the effect of serial dependence. In order to measure false alarm and miss rates while varying serial dependence, amount of random variability, and effect size, 37 students undertaking a postgraduate course in single-case design and analysis were required to assess the presence of an intervention effect in each of 27 AB charts constructed using a first-order autoregressive model. Three levels of effect size and three levels of variability, representative of values found in published charts, were combined with autocorrelation coefficients of 0, 0.3 and 0.6 in a factorial design. False alarm rates were surprisingly high (16% to 84%). Positive autocorrelation and increased random variation both significantly increased the false alarm rates and interacted in a nonlinear fashion. Miss rates were relatively low (0% to 22%) and were not significantly affected by the design parameters. Thus, visual analysts were not conservative, and serial dependence did influence judgment.
Keywords: visual inference, data analysis, single-subject design
Full text
PDF










Images in this article
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Chapman L. J., Chapman J. P. Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. J Abnorm Psychol. 1969 Jun;74(3):271–280. doi: 10.1037/h0027592. [DOI] [PubMed] [Google Scholar]
- Deprospero A., Cohen S. Inconsistent visual analyses of intrasubject data. J Appl Behav Anal. 1979 Winter;12(4):573–579. doi: 10.1901/jaba.1979.12-573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Furlong M. J., Wampold B. E. Intervention effects and relative variation as dimensions in experts' use of visual inference. J Appl Behav Anal. 1982 Fall;15(3):415–421. doi: 10.1901/jaba.1982.15-415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hojem M. A., Ottenbacher K. J. Empirical investigation of visual-inspection versus trend-line analysis of single-subject data. Phys Ther. 1988 Jun;68(6):983–988. doi: 10.1093/ptj/68.6.983. [DOI] [PubMed] [Google Scholar]
- Jones R. R., Weinrott M. R., Vaught R. S. Effects of serial dependency on the agreement between visual and statistical inference. J Appl Behav Anal. 1978 Summer;11(2):277–283. doi: 10.1901/jaba.1978.11-277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ottenbacher K. J. Reliability and accuracy of visually analyzing graphed data from single-subject designs. Am J Occup Ther. 1986 Jul;40(7):464–469. doi: 10.5014/ajot.40.7.464. [DOI] [PubMed] [Google Scholar]
- Rosenthal R., Kleid J. J., Cohen M. V. Abnormal mitral valve motion associated with ventricular septal defect following acute myocardial infarction. Am Heart J. 1979 Nov;98(5):638–641. doi: 10.1016/0002-8703(79)90291-6. [DOI] [PubMed] [Google Scholar]