Skip to main content
. Author manuscript; available in PMC: 2015 Feb 11.
Published in final edited form as: J Nerv Ment Dis. 2013 Oct;201(10):860–871. doi: 10.1097/NMD.0b013e3182a5c184

TABLE 3.

Scores and IRR From Pretesting of the GAP-REACH Checklist and the GAP-REACH Score Assessing the Use of REC-Related Factors in a Random Sample of Psychiatric Research Articles Published in 2002

Coder 1 Coder 2
Item na % na % κ 95% Confidence Limit
Overall REC use
  1. At least one REC term used in the title or abstract 4 13.3 4 13.3 1.0 1.0–1.0
  2. At least one REC term used in the article text 16 53.3 17 56.7 0.93 0.80–1.0
  3. Definition/conceptualization of REC term(s) provided 3 10.0 3 10.0 1.0 1.0–1.0
Introduction/Background
  4. Rationale for study question or design discussed in terms of REC factors 4 13.3 4 13.3 1.0 1.0–1.0
Methods
  5. REC included in sampling procedure 3 10.0 4 13.3 0.52 0.04–0.99
  6. Sample described in terms of REC characteristics 13 43.3 12 40.0 0.93 0.80–1.0
  7. Method described for assessing REC characteristics of participants 4 13.3 7 23.3 0.67 0.34–1.0
  8. Language proficiency of participants specified 7 23.3 8 26.7 0.92 0.76–1.0
  9. Relevance of REC characteristics of interviewers and participants discussed 4 13.3 4 13.3 1.0 1.0–1.0
  10. Relevance of language characteristics of interviewers and participants discussed 5 16.7 5 16.7 1.0 1.0–1.0
  11. Language match between participants and instruments is appropriately described 5 16.7 5 16.7 1.0 1.0–1.0
  12. Measurement equivalence of instruments described for all REC groups in the study 6 20.0 5 16.7 0.89 0.69–1.0
Results
  13. Effect of REC factors on study outcome(s) tested 10 33.3 7 23.3 0.78 0.55–1.0
  14. REC factors included in data analysis 7 23.3 6 20.0 0.90 0.71–1.0
Discussion
  15. REC emphasized in data interpretation 2 6.7 3 10.0 0.78 0.37–1.0
  16. Study limitations discussed in REC terms 2 6.7 2 6.7 1.0 1.0–1.0
Sum of items 95 96 0.91 0.86–0.95.
Average GAP-REACH score (30 articles) 19.8 20.0

Data are presented as number and percentage of articles (n = 30) meeting each item.

Coder 1: item 8: yes (n = 3), NA (n = 4); item 9: yes (n = 0), NA (n = 4); item 10: yes (n = 1), NA (n = 4); item 11: yes (n = 1), NA (n = 4); item 12: yes (n = 2), NA (n = 4); item 13: yes (n = 5), NA (n = 5).

Coder 2: item 8: yes (n = 4), NA (n = 4); item 9: yes (n = 0), NA (n = 4); item 10: yes (n = 1), NA (n = 4); item 11: yes (n = 1), NA (n = 4); item 12: yes (n = 1), NA (n = 4); item 13: yes (n = 5), NA (n = 2).

NA indicates not applicable.

a

Includes articles coded yes for items 1 to 7 and 14 to 16 and for the GAP-REACH score. For items 8 to 13, includes articles coded yes plus articles coded not applicable.