Table 4.

Result of the literature search.

Reference	Each study's limitation	Reported perspectives
14	•The software is unable to detect points corresponding to the lower area of the face, such as chin movements, or specific points on the tongue; •The face with medical devices is not detected.	•It should be noted that the ideal tool would indicate when pain intensity deserves treatment or treatment adjustments; •The use of an electronic eye, less dependent on humans, could help to integrate pain assessment with pain treatment in the context of neonatal care.
23	•Translational research needs to be done to assess the accuracy of the app for different neonates and clinical situations to assess if the performance of the neural network is less prone to subjective variables that modify pain assessment than the human performance; •The face with medical devices is not detected.	•Apply different explainable AI methods to better understand facial regions that might be relevant to pain assessment and use all the depicted face images to enlarge the face image samples and train our computational model. •Evaluate more recent CNN architectures. •Perform hands-on testing of the mobile application.
30	•Not applicable.	•Not applicable.
30 cited 20	•The current approach was evaluated on a relatively small number of infants; •The face with medical devices is not detected.	•Explore the use of CNNs to develop a highly accurate pain assessment application; •Improve the effectiveness of pain intervention while mitigating the short- and long-term outcomes of pain exposure early in life; •Realize a multimodal approach to pain assessment that allows pain to be assessed during circumstances when all pain responses are not available, clinical condition, activity level, and sedation; •Integrate contextual information, such as medication type/dose, to obtain a context-sensitive pain assessment. •Collect a large multimodal dataset during hospitalization in the NICU; •Investigate the possibility of using neonate sounds as soft biometrics.
30 cited 26	•High False Negative Rate; •The current approach was evaluated on a relatively small number of infants; •The current work does not provide a comparison between the assessment of the proposed automatic system and human; •The face with medical devices is not detected.	•Investigate several directions for minimizing False Negative Rate; •Employ or implement advanced noise reduction methods; •Enlarge the training data using traditional augmentation methods and Generative adversarial networks; •Follow another approach, assessing the level or intensity of the detected pain class; •Evaluate the approach on a larger dataset of infants recorded during both procedural and postoperative pain; •Investigate the association between neonatal pain and the brain's hemodynamic activities using Near-infrared Spectroscopy; •Explore the association between neonatal pain and changes in skin color as well as the association between pain and eye movement/pupil dilation; •Test how well the automatic system performs as compared to human judgments; •Adding points to the total score of infants, based on age, pain history, or other factors, to compensate for their limited ability to behaviorally or physio-logically communicate pain.
31	•In terms of PCC, the performance is relatively low; •The face with medical devices is not detected.	•The goal is pain intensity estimation; •Future work will focus on incorporating other pain modalities, particularly body movements which are part of the NFLAPS and collecting additional data.
32	•Does not report whether the system will be able to evaluate faces with medical devices.	•Providing the relatives (parents) with a tool that allows remote supervising of their newborn's wellness.
33	•The face with medical devices is not detected.	•Evaluate the intensity of the pain, classifying it as mild, moderate, or severe pain; •Future applications may also include patient populations incapable of expressing pain (children with disability, adults with dementia, or mechanically ventilated patients).
34	•It is not possible to accurately assess pain scores or pain grades with AI technology; •The automated NPA system currently requires an additional nurse to record the video; •Does not report whether the system will be able to evaluate faces with medical devices.	•Automate the entire process by recording video with bedside cameras in the future. •The AI technology embedded in the electronic-medical-record system in the future will realize intervention for pain in real-time by medical staff. •The downstream health education system can further perform pain-knowledge education for newborns’ family members to realize the traceability, standardization, and intelligence of the whole process of pain management.
35	•The metrics to validate the presence of artifacts need to be further tested. Mutual information was not able to validate, Pearson's correlation coefficient validated a part of the cases, and only the root means square error showed promise.	•Test more metrics and techniques to detect the presence of artifacts in each facial region.
36	•The system should be more illumination independent and should be able to handle partial component occlusions. •The face with medical devices is not detected.	•Perform an analysis possible to be applied in a real hospital situation.
37	•The face with medical devices is not detected.	•Implement the system in clinical practice, since the classification result shows that the proposed technique could be employed as a valuable tool for classifying the newborn between pain and normal with Fuzzy k-NN Classifier.
38	•The limited size of the dataset requires extension and further experimentation; •Rapid head movements are a problem for landmark tracking; •A thorough exploration of the parameters and alternatives of the KLT algorithm seems necessary to ensure more stability to the system; •The face with medical devices is not detected.	•Conduct additional video acquisition campaigns that will lead to a substantial extension of the original dataset; •Increase the number of expert operators performing manual pain assessment. •To better analyze the consistency and repeatability of the process, multiple scoring sessions on the videos should be performed by the same operators at different periods; •Adopt landmark selection algorithms best suited for child's faces; •Video processing can be effectively complemented by the analysis of audio information.
39	•The model cannot be updated online; •The amount of data in the newborn image database is not very large, and it must amplify the image on the data set during the experiment; •It needs more experiments with faces covered by devices.	•In the follow-up research, the algorithm can be further optimized based on the model algorithm built by this research firstly to improve its real-time performance and recognize the emotions of newborns in real-time. •The algorithm can combine with other auxiliary information such as crying and body movements for multi-modal classification and recognition when the facial expressions of newborns are collected to build a standard newborn image database with a large volume of data.
40	•Not all methods were able to robustly identify patients' faces in complex scenes involving phototherapy lighting, ventilation support, view in dorsal decubitus, and prone position; •One of the proposed settings works better on smaller faces, and the other works better in a brighter environment.	•Perform an enhancement to improve the accuracy of NICU-face; Complement the networks using an ensemble network and combining the strengths of the models; •Use these models for the implementation of other neonatal monitoring applications (e.g., in-home monitoring or intelligent monitoring applications from smartphones)

Note: CNN, convolutional neural networks; Fuzzy k-NN, Fuzzy K-Nearest Neighbor; KLT, Kanade–Lucas–Tomasi; NFLAPS, Neonatal Face and Limb Acute Pain; NICU, Neonatal Intensive Care Unit; NPA, Neonatal Pain Assessment; PCC, Pearson Correlation Coefficient.