Table 2.

Interrater agreement, internal consistency, and reliability with interpretations [46-49].

Rating category	Fleiss κ	Cronbach α	Guttman λ6	ICC(2,k)a (95% CI)
Engagement	0.62 (substantial)	.93 (excellent)	0.93 (high)	0.92 (0.88-0.95) (excellent)
Functionality	0.52 (moderate)	.90 (excellent)	0.89 (high)	0.89 (0.82-0.93) (good)
Aesthetics	0.49 (moderate)	.96 (excellent)	0.97 (high)	0.94 (0.88-0.97) (good)
Information	0.78 (substantial)	.96 (excellent)	0.96 (high)	0.96 (0.94-0.98) (excellent)
MARSb score	0.74 (substantial)	.95 (excellent)	0.94 (high)	0.94 (0.91-0.97) (excellent)
Subjective score	0.45 (moderate)	.96 (excellent)	0.96 (high)	0.95 (0.92-0.97) (excellent)

^aICC: intraclass correlation.

^bMARS: Mobile Application Rating Scale.