Table 2.

Checklist of psychometric features and methodological problems regarding dependent variables and tasks of the ANS.

Methodological problem	Recommendation	Reason
Reliability: not all dependent variables are reliable.	Use accuracy Do not use NRE or NDE	Best reliability. Poor reliability (Sasanguie et al., 2011; Inglis and Gilmore, 2014). Note: Reliability depends on age and is smaller for children.
Convergent validity: many dependent variables are not correlated.	Use accuracy or Weber fraction Do not use NRE	Only accuracy and Weber fraction were strongly related and measure presumably the same construct. The NRE was not related with other dependent variables of the ANS (Inglis and Gilmore, 2014).
Sample distribution: ceiling/floor effects lead to skewed distributions.	Check distribution of the dependent variables using a scatter plot	Many statistic procedures (e.g., Pearson correlation coefficient) assume normality distribution; skewed distributions violate this assumption (Inglis and Gilmore, 2014).
Reliability: not all ANS tasks are reliable.	Use paired dot comparison or the approximate addition task	Reliabilities of both measures are (mostly) acceptable (e.g., Gilmore et al., 2011; Price et al., 2012).
Convergent validity: not all ANS tasks are correlated.	Use paired dot comparison task	The paired dot comparison task is best studied and correlates with almost all other ANS tasks (e.g., Sasanguie et al., 2011; Price et al., 2012).
Discriminant validity: the ANS tasks should measure the ANS and no other cognitive processes.	Use paired dot comparison or the same-different task Do not use sequential/intermixed comparison or approximate arithmetic	These two tasks can be considered as the “purest” ANS tasks with least involved additional cognitive processes. All tasks require additional cognitive processes, like working memory, or visual resolution (e.g., Price et al., 2012).
Measuring ANS in young children: children have to understand the instructions.	Start with training trials and detailed feedback Check task understanding	When children do not understand the task, they might just guess or compare visual properties (Negen and Sarnecka, 2014).
Measuring developmental changes: different tasks, varying regarding difficulty, are used in infants, and children/adult.	Use similar tasks for all age groups (e.g., change detection task, fMRI adaptation paradigm)	Tasks used to measure ANS acuity in infants (e.g., habituation paradigms) are more difficult than tasks employed in children/adults (Gebuis and Van der Smagt, 2011). Thus, the use of different tasks with varying difficulties can bias developmental changes.