Table 5. Test-retest and convergent and divergent validity of the Hypo-METRICS app scales and questions.

	Test-retest1	Sleep quality (PROMIS)	Depression (PHQ-9)	Anxiety (GAD-7)	Vitality (SF-36)	Cognitive functioning (PDQ-20)	Fear of hypoglycemia (HFS-II)	Diabetes distress (PAID-20)8		Financial situation (DIDP question)8		HbA1C (mmol/mol)8	Diabetes duration8
Morning
Sleep quality	0.83	-0.68	-0.62	-0.51	0.52	-0.46	-0.29*	-0.35*		-0.18ns		-0.11ns	0.07ns
Overall mood	0.81	-0.56	-0.68	-0.60	0.55	-0.53	-0.28ns	-0.41		-0.21ns		-0.12ns	0.01ns
Negative affect	0.83	-0.47	-0.63	-0.70	0.50	-0.52	-0.41	-0.50		-0.11ns		-0.19ns	0.04ns
Energy level	0.86	-0.51	-0.68	-0.54	0.65	-0.53	-0.34*	-0.40		-0.20s		-0.11ns	-0.03ns
Cognitive functioning	0.88	-0.45	-0.65	-0.59	0.55	-0.57	-0.21ns	-0.39*		-0.18ns		-0.10ns	0.06ns
Fear of hypoglycemia	0.94	-0.33*	-0.36*	-0.40	0.27ns	-0.28*	-0.60	-0.51		-0.15ns		-0.04ns	0.16ns
Fear of hyperglycemia	0.93	-0.34*	-0.38*	-0.36*	0.36*	-0.37*	-0.53	-0.482	-0.49 3	-0.18ns		-0.12ns	0.05ns
Afternoon
Overall mood	0.82	-0.54	-0.60	-0.55	0.51	-0.46	-0.25ns	-0.39*		-0.17ns		-0.22ns	-0.02ns
Negative affect	0.86	-0.47	-0.62	-0.66	0.49	-0.51	-0.41	-0.53		-0.06ns		-0.14ns	0.03ns
Energy level	0.85	-0.49	-0.58	-0.46	0.60	-0.53	-0.35*	-0.34*		-0.16ns		-0.16ns	0.03ns
Cognitive functioning	0.85	-0.42	-0.58	-0.53	0.51	-0.54	-0.18ns	-0.37*		-0.13ns		-0.09ns	0.07ns
Fear of hypoglycemia	0.93	-0.38*	-0.40	-0.43	0.29*	-0.30*	-0.57	-0.51		-0.13ns		-0.04ns	0.21ns
Fear of hyperglycemia	0.91	-0.35*	-0.40	-0.36*	0.37*	-0.39*	-0.53	-0.492	-0.49 3	-0.17ns		-0.11ns	0.07ns
Evening
Overall mood	0.76	-0.46	-0.61	-0.58	0.53	-0.51	-0.25ns	-0.40*		-0.25ns		-0.17ns	0.03ns
Negative affect	0.85	-0.45	-0.62	-0.70	0.50	-0.53	-0.44	-0.55		-0.13ns		-0.19ns	0.06ns
Energy level	0.83	-0.41	-0.57	-0.48	0.61	-0.57	-0.36*	-0.38*		-0.24ns		-0.13ns	0.05ns
Cognitive functioning	0.88	-0.36*	-0.55	-0.56	0.54	-0.61	-0.23ns	-0.34ns		-0.20ns		-0.06ns	0.07ns
Memory	0.81	-0.37*	-0.49	-0.50	0.43	-0.49	-0.15ns	-0.29ns		-0.18ns		-0.01ns	0.06ns
Fear of hypoglycemia while asleep	0.90	-0.33*	-0.41	-0.44	0.25ns	-0.32ns	-0.51	-0.43		-0.15ns		-0.06ns	0.17ns
Fear of hyperglycemia while asleep	0.92	-0.35*	-0.40	-0.39*	0.36*	-0.39*	-0.53	-0.462	-0.473	-0.16ns		-0.13ns	0.05ns
Social functioning	0.82	-0.32*	-0.42	-0.50	0.38*	-0.44	-0.20ns	-0.434	-0.185 ns	-0.176 ns	-0.227 ns	-0.10ns	0.14ns

Given that PROMs typically require respondents to reflect over a given period of time (e.g., ‘past seven days’), daily app scores were averaged over a period of time corresponding to the PROM’s recall period. For example, if the PROM’s recall period was the ‘past seven days’, then the corresponding app scores were averaged across the final seven days of the study period and correlated with the PROM score.

Hypotheses were made for each row (i.e., each app question/scale). In each row, the white cell indicates the correlation that was expected to be highest and the darkest cells indicate the correlations that were expected to be lowest for that question/scale. The lighter grey shading indicates correlations that were expected to fall between the highest and the lowest categories. For example, it was hypothesized that the “energy level” scale would correlate highest with the SF-36 vitality scale, followed by PROMs of related concepts (sleep, depression, anxiety, cognitive function), followed by less conceptually related concepts (diabetes distress, fear of hypoglycemia, financial situation, HbA1c, and diabetes duration).

¹Test-retest reliability was explored by correlating (Spearman’s rho) average scores on each scale from week 3 (test condition) with average scores from week 8 (re-test). Week 3 was selected to allow a two-week run-in phase for participants to familiarize themselves with the app, and week 8 was select to allow for an inter-test interval similar to other studies [34]. Correlations r>0.7 were considered suitable for demonstrating test-retest reliability.

²First number represents correlations with the PAID total score.

³Second number represents correlations with the PAID question “Worrying about the future and the possibility of serious complications?”

⁴First number represents correlations with the PAID total score.

⁵Second number represents correlations with the PAID question “Uncomfortable social situations related to your diabetes care (e.g., people telling you what to eat)?”.

⁶First number represents correlations with the DIDP question “Your financial situation”.

⁷Second number represents correlations with the DIDP question “Your relationship with your family, friends and peers?”.

⁸For these PROMs, there were no specified recall periods, so the average app scores from the last seven days of the study period were used.

Convergent validity was supported if Spearman correlations (between app scales and PROMs) were strong (rs>±0.5) or moderate (rs>±0.3), and divergent validity was supported if they were low (rs<±0.3). ‘Financial situation (DIDP question)’, ‘HbA1c (mmol/mol)’ and ‘Diabetes duration’ were included for exploring divergent validity.

Colours represent hypothesized ranking of correlations

Highest and at least r_s > ±0.3.

Medium.

Lowest.

Numbers in bold represent correlations where hypothesized highest correlations were confirmed.

All correlations are significant at a p < 0.001, unless otherwise noted: *p<0.05, ns = not significant p > 0.05.

Higher scores on all the Hypo-METRICS app scales indicate ‘better’ daily functioning.