Skip to main content
NCBI home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2022 Nov 2;208:145–151. doi: 10.1016/j.procs.2022.10.022

Dual-Proxy Modeling for Masked Face Recognition

Wang Shuhui a, Mao Xiaochen a
PMCID: PMC9626733  PMID: 36337255

Abstract

With the recent worldwide COVID-19 pandemic, almost everyone wears a mask daily, leading to severe degradation in the accuracy of conventional face recognition systems. Several works improve the performance of masked faces by adopting synthetic masked face images for training. However, such methods often cause performance degradation on unmasked faces, raising the contradiction between the face recognition system's accuracy on unmasked and masked faces. In this paper, we propose a dual-proxy face recognition training method to improve masked faces’ performance while maintaining unmasked faces’ performance. Specifically, we design two fully-connected layers as the unmasked and masked feature space proxies to alleviate the significant difference between the two data distributions. The cross-space constraints are adopted to ensure the intra-class compactness and inter-class discrepancy. Extensive experiments on popular unmasked face benchmarks and masked face benchmarks, including real-world mask faces and the generated mask faces, demonstrate our method's superiority over the state-of-the-art methods on masked faces without incurring a notable accuracy degradation on unmasked faces.

Keywords: Masked face recognition, Dual-Proxy, neural network, deep learning

References

  • 1.A. Anwar, A. Raychowdhury, Masked face recognition for secure authentication, arXiv preprint arXiv:2008.11104.
  • 2.G. B. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: A database for studying face recognition in unconstrained environments, Tech. Rep. 07-49, University of Massachusetts, Amherst (October2007).
  • 3.Maze B., Adams J., Duncan J.A., Kalka N., Miller T., Otto C., Jain A.K., Niggel W.T., Anderson J., Cheney J., et al. 2018 International Conference on Biometrics (ICB) IEEE; 2018. Iarpa janus benchmark-c: Face dataset and protocol; pp. 158–165. [Google Scholar]
  • 4.Zhao W., Chellappa R., Phillips P.J., Rosenfeld A. Face recognition: A literature survey. ACM Comput. Surv. 2003:399–458. [Google Scholar]
  • 5.Deng W., Hu J., Guo J. Extended src: Undersampled face recognition via intraclass variant dictionary. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2012;34(9):1864–1870. doi: 10.1109/TPAMI.2012.30. [DOI] [PubMed] [Google Scholar]
  • 6.Deng W., Hu J., Guo J. Face recognition via collaborative representation: Its discriminant nature and superposed representation. IEEE transactions on pattern analysis and machine intelligence. 2017;40(10):2513–2521. doi: 10.1109/TPAMI.2017.2757923. [DOI] [PubMed] [Google Scholar]
  • 7.Chen D., Cao X., Wen F., Sun J. Proceedings of the IEEE conference on computer vision and pattern recognition. 2013. Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification; pp. 3025–3032. [Google Scholar]
  • 8.Cao Z., Yin Q., Tang X., Sun J. 2010 IEEE Computer society conference on computer vision and pattern recognition. IEEE; 2010. Face recognition with learning-based descriptor; pp. 2707–2714. [Google Scholar]
  • 9.Guo Y., Zhang L., Hu Y., He X., Gao J. European conference on computer vision. Springer; 2016. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition; pp. 87–102. [Google Scholar]
  • 10.Cao Q., Shen L., Xie W., Parkhi O.M., Zisserman A. 2018 13th IEEE internationalconference on automatic face & gesture recognition (FG 2018) IEEE; 2018. Vggface2: A datase for recognising faces across pose and age; pp. 67–74. [Google Scholar]
  • 11.Zhu Z., Luo P., Wang X., Tang X. Proceedings of the IEEE International Conference on Computer Vision. 2013. Deep learning identity-preserving face space; pp. 113–120. [Google Scholar]
  • 12.Deng J., Guo J., Xue N., Zafeiriou S. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. Arcface: Additive angular margin loss for deep face recognition; pp. 4690–4699. [DOI] [PubMed] [Google Scholar]
  • 13.Wang H., Wang Y., Zhou Z., Ji X., Gong D., Zhou J., Li Z., Liu W. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. Cosface: Large margin cosine loss for deep face recognition; pp. 5265–5274. [Google Scholar]
  • 14.Z. Wang, G. Wang, B. Huang, Z. Xiong, Q. Hong, H. Wu, P. Yi, K. Jiang, N. Wang, Y. Pei, et al., Masked face recognition dataset and application, arXiv preprint arXiv:2003.09093.
  • 15.Han C., Shan S., Kan M., S.Wu X.Chen. Proceedings of the European Conference on Computer Vision (ECCV) 2018. Face recognition with contrastive convolution; pp. 118–134. [Google Scholar]
  • 16.Kan M., Shan S., Chen X. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. Multi-view deep network for cross-view classification; pp. 4847–4855. [Google Scholar]
  • 17.Qian Q., Shang L., Sun B., Hu J., Li H., Jin R. Proceedings of the IEEE International Conference on Computer Vision. 2019. Softtriple loss: Deep metric learning without triplet sampling; pp. 6450–6458. [Google Scholar]
  • 18.Deng J., Guo J., Liu T., Gong M., Zafeiriou S., arcface Sub-center. Springer; 2020. Boosting face recognition by large-scale noisy web faces; pp. 741–757. European Conference on Computer Vision. [Google Scholar]
  • 19.Liu W., Wen Y., Yu Z., Li M., Raj B., Song L. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. Sphereface: Deep hypersphere embedding for face recognition; pp. 212–220. [Google Scholar]
  • 20.Huang Y., Wang Y., Tai Y., Liu X., Shen P., Li S., Li J., Huang F. proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. Curricularface: adaptive curriculum learning loss for deep face recognition; pp. 5901–5910. [Google Scholar]
  • 21.Sengupta S., Chen J.-C., Castillo C., Patel V.M., Chellappa R., Jacobs D.W. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) IEEE; 2016. Frontal to profile face verification in the wild; pp. 1–9. [Google Scholar]
  • 22.Zheng T., Deng W., Hu J. Beijing University of Posts and Telecommunications; 2018. Cross-pose lfw: A database for studying crosspose face recognition in unconstrained environments. Tech. Rep. 18-01February. [Google Scholar]
  • 23.Moschoglou S., Papaioannou A., Sagonas C., Deng J., Kotsia I., Zafeiriou S. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017. Agedb: the first manually collected, in-the-wild age database; pp. 51–59. [Google Scholar]
  • 24.T. Zheng, W. Deng, J. Hu, Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments, arXiv:1708.08197.
  • 25.He K., Zhang X., Ren S., Sun J. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Deep residual learning for image recognition; pp. 770–778. [Google Scholar]
  • 26.Deng J., Guo J., Xue N., Zafeiriou S. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. Arcface: Additive angular margin loss for deep face recognition; pp. 4690–4699. [DOI] [PubMed] [Google Scholar]
  • 27.A. Paszke, G. et al., Pytorch: An imperative style, high-performance deep learning library, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 8024–8035.

Articles from Procedia Computer Science are provided here courtesy of Elsevier

RESOURCES