Table 3.

Challenges, limitations, and future directions of different computer vision sensing and data processing approaches.

	Current approaches / Problems	Future directions / Improvements/ Challenges
Sensors	Current approaches Mostly 2D single cameras Problems Only 2D information Occlusions	Future directions Multiple 2D cameras 3D (depth) sensors Pressure mat sensors Improvements 3D information Less occlusions More information due to multi-sensory integration
Data	Current approaches Small datasets Problems Not enough data to employ deep learning methods Not publicly available – no benchmarking possible Incorrect or incomplete data in some cases, e.g., inaccurate outcome labelling due to lack of longitudinal studies, the inclusion of incorrect age-specificity cases, use of low-inter-rater agreement or small rater-group or lack of experienced raters in data labelling, disorders or gender misrepresentation	Future directions Collect more data Make it publicly available Make use of home videos Employ DL methods Make use of transfer learning (e.g., Tan et al., 2018) Challenges Need to solve anonymisation issue (automated techniques for face detection and replacement can be applied) Development of methods which can cope with different light conditions, resolution, frame rate
Body areas of interest	Current approaches Mostly movement of arms, legs, head Problems Incomplete information of full-body movement	Future directions Hand, fingers, feet Eye movement data Mimic Challenges Integration and analysis of multimodal information
Motion tracking	Current approaches Mostly in 2D space Problems Only 2D information	Future directions Full-body tracking in 3D using well-established methods in DL (e.g., DeepLabCut and OpenPose frameworks) Challenges DL methods need to be adapted to infants
Motion encoding	Current approaches Conventional features based on: displacement, distance, velocity, acceleration, speed, and time Problems Only 2D features	Future directions Motion encoding using well-established methods from robotics: Dynamic Movement Primitives (e.g., Ijspeert, Nakanishi, Hoffmann, Pastor, & Schaal, 2013), Gaussian Mixture Models (e.g., Calinon, 2016; Khansari-Zadeh & Billard, 2011), Probabilistic Movement Primitives (Paraschos, Daniel, Peters, & Neumann, 2018) Learn features from expert knowledge during observation (e.g., Silva et al., 2018, 2019) Improvements 3D features New motion encoding and features
Classification algorithms	Current approaches Conventional ML methods, e.g., SVM, Decision Trees, Neural Networks, Hidden Markov Models Supervised learning without feedback during learning	Future directions Employ ANN, DL Employ Interactive Machine Learning (learning with feedback) Improvements Better models with more accurate predictions Challenges More data is needed