Skip to main content
PMC home page
Journal of Neurology, Neurosurgery, and Psychiatry logoLink to Journal of Neurology, Neurosurgery, and Psychiatry
. 2003 Apr;74(4):447–451. doi: 10.1136/jnnp.74.4.447

Reliability of the variables in a new set of models that predict outcome after stroke

N Weir 1, C Counsell 1, M McDowall 1, A Gunkel 1, M Dennis 1
PMCID: PMC1738385  PMID: 12640061

Abstract

Objectives: To provide valid predictions of outcome, the variables included in a prognostic model must be capable of reliable collection. The authors have recently reported a set of simple but rigorously developed models that predict outcome after stroke. The aim of this study was to establish the inter-rater reliability of the variables included in the models.

Methods: Inter-rater agreement was measured prospectively (between two clinicians; 92 patients) and retrospectively (between two auditors; 200 patients) and the validity of the data collected retrospectively was estimated by comparing them with data collected prospectively (195 patients). In the prospective study inter-rater agreement for urinary incontinence and for the variables of three other previously published models was also measured. The median difference (md) between ages and κ statistics for other variables was calculated.

Results: For the model variables, prospective agreement ranged from good to excellent (age: md 0 years; living alone before the stroke κ 0.84; pre-stroke functional independence κ 0.67; normal verbal Glasgow Coma Scale score κ 0.79; ability to lift both arms against gravity κ 0.97; ability to walk unaided κ 0.91) while retrospective agreement (age: md 0 years; κ 0.55–0.92) and agreement between prospective and retrospective observers (age: md 0 years; κ 0.49–0.78) was acceptable but less good. Prospective agreement was excellent for urinary incontinence (κ 0.87) and variable for the other models (κ 0.23–0.81)

Conclusion: The variables included in these new simple models of outcome after stroke are capable of reliable collection, comparable to or better than that of the other predictive variables considered. When collected retrospectively, the model variables are likely to remain reliable and reasonably valid.

Full Text

The Full Text of this article is available as a PDF (87.4 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Allen C. M. Predicting the outcome of acute stroke: a prognostic score. J Neurol Neurosurg Psychiatry. 1984 May;47(5):475–480. doi: 10.1136/jnnp.47.5.475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Baird A. E., Dashe J., Connor A., Burzynski C., Schlaug G., Warach S. Comparison of retrospective and prospective measurements of the national institutes of health stroke scale. Cerebrovasc Dis. 2000 Jan-Feb;10(1):80–81. doi: 10.1159/000016031. [DOI] [PubMed] [Google Scholar]
  3. Brennan P., Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ. 1992 Jun 6;304(6840):1491–1494. doi: 10.1136/bmj.304.6840.1491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bushnell C. D., Johnston D. C., Goldstein L. B. Retrospective assessment of initial stroke severity: comparison of the NIH Stroke Scale and the Canadian Neurological Scale. Stroke. 2001 Mar;32(3):656–660. doi: 10.1161/01.str.32.3.656. [DOI] [PubMed] [Google Scholar]
  5. Caplan R. A., Posner K. L., Cheney F. W. Effect of outcome on physician judgments of appropriateness of care. JAMA. 1991 Apr 17;265(15):1957–1960. [PubMed] [Google Scholar]
  6. Counsell C., Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis. 2001;12(3):159–170. doi: 10.1159/000047699. [DOI] [PubMed] [Google Scholar]
  7. Counsell Carl, Dennis Martin, McDowall Michael, Warlow Charles. Predicting outcome after acute and subacute stroke: development and validation of new prognostic models. Stroke. 2002 Apr;33(4):1041–1047. doi: 10.1161/hs0402.105909. [DOI] [PubMed] [Google Scholar]
  8. D'Olhaberriague L., Litvan I., Mitsias P., Mansbach H. H. A reappraisal of reliability and validity studies in stroke. Stroke. 1996 Dec;27(12):2331–2336. doi: 10.1161/01.str.27.12.2331. [DOI] [PubMed] [Google Scholar]
  9. Davenport R. J., Dennis M. S., Warlow C. P. Improving the recording of the clinical assessment of stroke patients using a clerking pro forma. Age Ageing. 1995 Jan;24(1):43–48. doi: 10.1093/ageing/24.1.43. [DOI] [PubMed] [Google Scholar]
  10. Dewey H. M., Donnan G. A., Freeman E. J., Sharples C. M., Macdonell R. A., McNeil J. J., Thrift A. G. Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovasc Dis. 1999 Nov-Dec;9(6):323–327. doi: 10.1159/000016006. [DOI] [PubMed] [Google Scholar]
  11. Frithz G., Werner I. Studies on cerebrovascular strokes. II. Clinical findings and short-term prognosis in a stroke material. Acta Med Scand. 1976;199(1-2):133–140. [PubMed] [Google Scholar]
  12. Gelmers H. J., Gorter K., de Weerdt C. J., Wiezer H. J. Assessment of interobserver variability in a Dutch multicenter study on acute ischemic stroke. Stroke. 1988 Jun;19(6):709–711. doi: 10.1161/01.str.19.6.709. [DOI] [PubMed] [Google Scholar]
  13. Gladman J. R., Harwood D. M., Barer D. H. Predicting the outcome of acute stroke: prospective evaluation of five multivariate models and comparison with simple methods. J Neurol Neurosurg Psychiatry. 1992 May;55(5):347–351. doi: 10.1136/jnnp.55.5.347. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Goldstein L. B., Chilukuri V. Retrospective assessment of initial stroke severity with the Canadian Neurological Scale. Stroke. 1997 Jun;28(6):1181–1184. doi: 10.1161/01.str.28.6.1181. [DOI] [PubMed] [Google Scholar]
  15. Kalra L., Crome P. The role of prognostic scores in targeting stroke rehabilitation in elderly patients. J Am Geriatr Soc. 1993 Apr;41(4):396–400. doi: 10.1111/j.1532-5415.1993.tb06947.x. [DOI] [PubMed] [Google Scholar]
  16. Kasner S. E., Chalela J. A., Luciano J. M., Cucchiara B. L., Raps E. C., McGarvey M. L., Conroy M. B., Localio A. R. Reliability and validity of estimating the NIH stroke scale score from medical records. Stroke. 1999 Aug;30(8):1534–1537. doi: 10.1161/01.str.30.8.1534. [DOI] [PubMed] [Google Scholar]
  17. Kwakkel G., Wagenaar R. C., Kollen B. J., Lankhorst G. J. Predicting disability in stroke--a critical review of the literature. Age Ageing. 1996 Nov;25(6):479–489. doi: 10.1093/ageing/25.6.479. [DOI] [PubMed] [Google Scholar]
  18. Laupacis A., Sekar N., Stiell I. G. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997 Feb 12;277(6):488–494. [PubMed] [Google Scholar]
  19. Lindley R. I., Warlow C. P., Wardlaw J. M., Dennis M. S., Slattery J., Sandercock P. A. Interobserver reliability of a clinical classification of acute cerebral infarction. Stroke. 1993 Dec;24(12):1801–1804. doi: 10.1161/01.str.24.12.1801. [DOI] [PubMed] [Google Scholar]
  20. Osborne C. E., Thompson H. C. Criteria for evaluation of ambulatory child health care by chart audit: development and testing of a methodology. Final report of the Joint Committee on Quality Assurance of Ambulatory Health Care for Children and Youth. Pediatrics. 1975 Oct;56(4 Pt 2 Suppl):625–692. [PubMed] [Google Scholar]
  21. Prescott R. J., Garraway W. M., Akhtar A. J. Predicting functional outcome following acute stroke using a standard clinical examination. Stroke. 1982 Sep-Oct;13(5):641–647. doi: 10.1161/01.str.13.5.641. [DOI] [PubMed] [Google Scholar]
  22. Romm F. J., Putnam S. M. The validity of the medical record. Med Care. 1981 Mar;19(3):310–315. doi: 10.1097/00005650-198103000-00006. [DOI] [PubMed] [Google Scholar]
  23. Wade D. T., Hewer R. L. Outlook after an acute stroke: urinary incontinence and loss of consciousness compared in 532 patients. Q J Med. 1985 Sep;56(221):601–608. [PubMed] [Google Scholar]
  24. Weir N., Dennis M. S., Scottish Stroke Outcomes Study Group Towards a national system for monitoring the quality of hospital-based stroke services. Stroke. 2001 Jun;32(6):1415–1421. doi: 10.1161/01.str.32.6.1415. [DOI] [PubMed] [Google Scholar]
  25. Williams L. S., Yilmaz E. Y., Lopez-Yunez A. M. Retrospective assessment of initial stroke severity with the NIH Stroke Scale. Stroke. 2000 Apr;31(4):858–862. doi: 10.1161/01.str.31.4.858. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Neurology, Neurosurgery, and Psychiatry are provided here courtesy of BMJ Publishing Group

RESOURCES