Table 6.

Comparative analysis of techniques to mitigate the impact of intrinsic threats

Type/Focus on	Ref.	Concept	Methodology Used	Dataset Used	Performance	Limitation
Occlusion	[158]	Local distribution based occlusion detection for palm print-based uniform patterns.	ULBP (Uniform-LBP), NN, Thresholding	PolyU2D, CASIA, IIITDM palm-print DB	Up to 36% of occlusion is detected	Not tested with other biometric traits.
	[32]	A review on state-of-the-art approaches to detect occlusion in 3D face and subsequently the restoration strategies.	Radial curve, PCA, Template machine	Bosphorus,UMB GavabDB EURECOM KinectFace DB	RR for face occlusion (hairs)- 98.0%, glasses- 94.2%, overall- 96.3 %	Not suitable for Uncontrolled conditions
	[42]	Characterization of the occluded and corrupted region using nuclear norm to get dictionary with best results.	SRC, NNB matrix regression, NNAODL	AR, Extended Yale- B DB, LFW	Accuracy 93.1%	Unable to handle non- monotonic illumination and noise problems.
	[165]	A cascaded deep generative regression model for an occlusion-free face alignment with GAN (locating occlusion), DRM(enhancement), Cascading(facial landmarks).	GAN, DRM, and Cascading	OCFW, COFW, 300W, and AFLW	EER 5.72% for COFW, 6.97% for 300W, 1.80% for AFLW.	Fusion of multiple processes can degrade the performance of occlusion.
	[88]	PCA and infrared thermal-based method to detect the occlusion on face regions is proposed here.	PCA, Infrared thermal imaging, BPNN, SVM	Self-created sample data	Highest face recognition rate 95%	Recognition rate is dependent on kernel parameter.
	[94]	The local and global face features are extracted using high filtering function in the presence of different light source to analyze the halo effect on input images.	Recurrent Neural Network	AR face dataset	Sunglasses occlusion-98.45-88.4, Scarf shield- 95.37-62.3 (in%)	Complex algorithm, which consumes more computation time
	[144]	Wavelet and SURF-based occlusion detection method utilizing gravitational search algorithm for feature extraction and recognition.	Wavelet, SURF, holo-entropy, DNN-RBM FE- appearance-based	Derf’s accumulation (video dataset) and HMDB51	98.72% Accuracy	Complex architecture, computation overhead.
	[191]	A deep SSD-based occlusion detection method to find the target location of the face. It can deal with sunglasses, face mask, hats, and other accessories on face.	SSD (Single Shot Multi-Box Detector)	Seven types of self-built dataset for face occlusion	The average precision has reached to 95.46% for all categories (mAP).	No benchmark dataset is available.
	[8]	This paper proposed BDW-SIFT for partial occlusion. Wavelet SIFT covers 15% to 60% occlusion of the face.	BDW-SIFT (scale invariant feature transform with discrete wavelets), DWT	MUCT dataset	Testing accuracy for occlusion-Vertical- 87%, Horizontal 86%, diagonal 46.5%, Random 90.5%	Due to mass key points the time consuming process
Expression	[63]	User dependent unsupervised approach referred as PADMA, to find affective states from spontaneous facial expressions is proposed here.	PADMA using AMIL, Clustering	CMU Multi-PIE ,UNBC-MSARV	User dependent 58.5%, 59.2% User independent 71.3%, 25.7%	Occurrence of unknown affect in face gestures fails and not detected.
	[40]	A uniform- LBP method is proposed for features extraction with Legendre moments, while k-NN is used for classification.	uLBP- (feature extraction), Legendre- feature vector, k-NN with L1 Norm Classification	IRIS and NVIE Database	RR and accuracy for IRIS 90.63 and 98.83 (visible), 99.83 and 99.98 (infrared). For NVIE 99.46, 99.97 (visible) and 99.82, 99.97(infrared).	Performance reduces for visible images having low illumination.
	[117]	An unsupervised framework for spontaneous expression recognition is proposed with preserved discriminative information of the input (i.e., video).	HOOF, MBH, UAM, SEV, Cosine, LDA, PLDA, SVM	BP4D, AFEW	Accuracy (Expression vector + SVM) for BP4D- 81.3%, AFEW-74.1%	Experiments involve trimmed clipping videos, thus real-time testing result might be poor.
	[187]	An automated machine-based FER system including a dataset development algorithm is proposed with a detailed state-of-the-art review on occlusion effect.	Sparse and reconstruction approach, SVM, DBN, CNN	JAFFE, CK, CAM3D, BU-3DFD	CK+ database shows highest accuracy among all	Hard to detect face if occluded in group of faces
	[177]	A CNN-based method to identify the start to end points (onset, apex, and offset) based on face shape for each expression is proposed.	Scattering CNN	BU-3DFE, BP-4D	Precision 81.35%, Recall 87.19%	Not efficient for 3D and multi model domain
	[184]	An optimized method to recognize the relevant facial expressions for an input image is investigated.	CNN, SVM	BU-3DFE, SFEW	BU-3DFE- 1.72%,, SFEW-1.11%	Not suitable for extent variance in pose.
	[132]	A non-posed image acquisition method, for evoking natural expressions through strong influence of the subjects in various scenarios: playing video games, during interviews, and watching emotional video is proposed.	Appearance based and feature based analysis, Static and dynamic classifiers (BN, HMM)	UT-Dallas FEEDTUM, NVIE, AM-FED	Expression of players of Video game is performed better for emotion diagnosis	Spontaneous expression intensity estimation for uncontrolled profile image is not done
	[103]	A supervised transfer learning method to recognize various facial expressions for multiple subjects (simultaneously) in a single frame is presented.	CNN (MobileNet) with Softmax and center loss function.	JAFFE and CK+	JAFFE (95.24%)and CK+ (96.92%).	Only frontal faces are taken into account
	[12]	A novel software for emotion recognition using webcam data is introduced based on FURIA algorithm and unordered Fuzzy rule induction.	FURIA algorithm with unordered fuzzy rule induction	Cohn-Kanade AU-coded expression extended database (CKplus)	Overall average accuracy of 83.2% (α) level	This method unable to detect the micro expressions
	[10]	An optimized kernel-based SVM method is introduced to classify the various facial expressions.	RBF kernel. SVM, Dimensionality reduction	JAFFE CK	Cross validation result 94.3%	Not able to detect non-basic (despite universal)expressions
Aging	[115]	A 3D shape and wrinkle texture-based aging invariance method is proposed to investigate 3D view-invariant face models.	3D Aging, PCA, AAM, RBF m	2D face aging DB, FG-NET, MORPH, BROWNS using FaceVACS	IA (before& after) aging are 26.4, 37.4 (FGNET), 57.8, 66.4, (MORPH), 15.6, 28.1 (BROWNS)	Automation of Age invariant FR system is hard to implement due to noise, textural info
	[101]	A novel Wrinkle Oriented AAM (WO-AAM) is proposed with new channel to analyze wrinkles, empiric joint probability density by Kernel Density estimation, than synthesize new plausible wrinkles.	PCA,GAN, Recurrent face aging (RFA), VGG-16, CNN	Self-built 400 A Caucasian women, age (43-85 years) with average age- 69	10-years aging & rejuvenating period, the estimation of age can be added by 4.9 and be subtracted by 4 years, respectively.	Result influenced by exposure of sunlight, alcohol consumption, dark spot.
Race	[57]	Other Race Effect (ORE) in FR analyze the different races considering two measures: assessed the quality (and quantity) of social interaction and the time measure (i.e., spent in Western country) with least information of contact.	Race contact questionnaire in study phase and recognition phase.	23 Chinese and 25 Caucasian participant with variability in contact, effect, inversion decrement for other-race, and cross-race	The overall model was significant (2,47) = 30.99; p < 0.0001 and explained 38.2 % of the sample variance.	This method is not generalized and doesn’t perform well for other continental location.
	[160]	A detailed study of the Race condition, includes participants training to recognize African, American (or Hispanic) through their faces at individual level and classify them at on the basis of race.	Electroencephalogram (EEG), Behavioral training	24 students of 18-29 years from university of Victoria, created DB	The fine-grained visual discrimination supported by N250 that reflects the formation of perceptual representations.	Adult face recognition for other race is not inflexible, intuitive experience is meaningless.
Gender	[13]	A novel geomteric curve feature-based method is proposed for gender classification. The saptial features are extracted through circular and radial sets using Euclidean distance. Then, Adaboost algorithm for minimal feature selection.	FE-LBP, Multi-scale local normal patterns (MS-LNPs), Euclidean distance for Geometric (circular/radial) curve feature extraction, FS- Adaboost, CLS- SVM	FRGCv2 dataset	Face recognition rate for a rank-1 is 98% and a gender classification rate is 86%.	It needs a large dataset with annotation, variety in race could affect the results
	[96]	An analysis and evaluation of hormone replacement therapy for classification of gender is presented. The full face is considered for detecting the significant regions. The fusion of texture-based periocular features with patch-based LBP reveals more prominent results.	HRT, Texture-based face matchers, LBP, HOG, and patch-based local binary patterns (p-LBP)	A DB > 1.2 M face image from You Tube with HRT for gender transformation from several months to three years.	The evaluated periocular-fused patch-based face matcher outperforms PittPatt SDK v5.2.2 by 76.83% and Cognetic FaceVACS v8.5 by 56.23% for rank-1 accuracy.	Performance is highly dependent on final alteration after effective medical treatment.

NNAODL-Nuclear Norm based adapted Occlusion Dictionary Learning, POcc-Partial Occlusion, PADMA- Personalized Affect Detection with Minimal Annotation, SRC-Sparse Representation Coding, NNMR-Nuclear norm based matrix regression, DRM-Deep Regression Model, BDW- Biorthogonal Discrete Wavelet, AMIL-Association-based Multiple Instance Learning, HOOF-Histograms of oriented optical flow, AAM- Active appearance model, GAN-Generative Adversarial Networks