A comprehensive experimental comparison of the aggregation techniques for face recognition

Document Type : Research Paper


1 Institute of Institute of Computer Science, Lublin University of Technology, ul. Nadbystrzycka 36B, 20-618 Lublin, Poland

2 Department of Electrical & Computer Engineering, University of Alberta, Edmonton T6R 2V4 AB, Canada


In face recognition, one of the most important problems to tackle is a large amount of data and the redundancy of information contained in facial images. There are numerous approaches attempting to reduce this redundancy. One of them is information aggregation based on the results of classifiers built on selected facial areas being the most salient regions from the point of view of classification both by humans and computers. In this study, we report on a series of experiments and offer a comprehensive comparison between various methods of aggregation of outputs of these classifiers based on essential facial features such as eyebrows, eyes, nose, and mouth areas. For each of them, we carry the recognition process utilizing the well-known Fisherfaces transformation. During the comparisons of the vectors representing the features of images (faces) after the transformations, we consider 16 similarity$/$dissimilarity measures for which we select the best aggregation operator. The set of operators to compare was selected on a basis of the comprehensive literature review regarding aggregation functions.


T. Ahonen, A. Hadid, M. Pietikainen, Face recognition with local binary patterns, In: Proceedings of the 8th
European Conference on Computer Vision, Lecture Notes in Computer Science, 3021 (2004), 469{481.
[2] R. Al-Hmouz, W. Pedrycz, K. Daqrouq, A. Morfeq, Development of multimodal biometric systems with three-way
and fuzzy set-based decision mechanisms, International Journal of Fuzzy Systems, 20(1) (2017), 128{140.
[3] F. R. Al-Osaimi, M. Bennamoun, A. Mian, Spatially optimized data-level fusion of texture and shape for face
recognition, IEEE Transactions on Image Processing, 21 (2012), 859{872.
[4] C. Alsina, M. J. Frank, B. Schweizer, Associative functions. Triangular norms and copulas, World Scienti fic, New
Jersey, 2006.
[5] AT&T Laboratories Cambridge. The Database of Faces. http://www.cl.cam.ac.uk/ research/
dtg/attarchive/facedatabase.html. Accessed 10 May 2016.
[6] A. I. Awad, A. E. Hassanien, Impact of some biometric modalities on forensic science, In: A. K. Muda, Y.-H. Choo,
A. Abraham, and S. N. Srihari (Eds.), Computational intelligence in digital forensics: Forensic investigation and
applications, Studies in Computational Intelligence, 555 (2014), 47-62.
[7] P. N. Belhumeur, J. P. Hespanha, D. J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using class speci fic linear
projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711-720.
[8] G. Beliakov, A. Pradera, T. Calvo, Aggregation functions: A guide for practitioners, Springer-Verlag, Berlin Heidelberg,
[9] M. Bereta, P. Karczmarek, W. Pedrycz, M. Reformat, Local descriptors in application to the aging problem in face
recognition, Pattern Recognition, 46 (2013), 2634{2646.
[10] S. D. Bharkad, M. Kokare, Performance evaluation of distance metrics: Application to fi ngerprint recognition,
International Journal of Pattern Recognition and Arti ficial Intelligence, 25 (2011), 777-806.
[11] R. Brunelli, T. Poggio, Face recognition: Features versus templates, IEEE Transactions on Pattern Analysis and
Machine Intelligence, 15 (1993), 1042-1052.
[12] H. Bustince, J. Fernandez, R. Mesiar, T. Calvo, Aggregation functions in theory and in practice, Springer, Berlin,
[13] T. Calvo, G. Mayor, R. Mesiar, Aggregation operators. New trends and applications, Physica-Verlag, Heidelberg,
[14] C. Campomanes-Alvarez, O. Ibanez, O. Cordon, Experimental study of different aggregation functions for model-
ing craniofacial correspondence in craniofacial superimposition, In: 2016 IEEE International Conference on Fuzzy
Systems (FUZZ-IEEE), (2016), 437{444.
[15] C. H. Chan, M. A. Tahir, J. Kittler, M. Pietikainen, Multiscale local phase quantization for robust component-based
face recognition using kernel fusion of multiple descriptors, IEEE Transactions on Pattern Analysis and Machine
Intelligence, 35 (2013), 1164{1177.
[16] C. Chen, A. Dantcheva, A. Ross, An ensemble of patch-based subspaces for makeup-robust face recognition, Information
Fusion, 32 (2016), 80-92.
[17] J. H. Chiang, Aggregating membership values by a Choquet-fuzzy-integral based operator, Fuzzy Sets and Systems,
114 (2000), 367-375.
[18] J. A. Dargham, A. Chekima, M. Hamdan, Hybrid component-based face recognition system, In: S. Omatu et al.
(Eds.), Distributed computing and artifi cial intelligence, Advances in Intelligent and Soft Computing, 151 (2016),
[19] S. Das, D. Guha, Power harmonic aggregation operator with trapezoidal intuitionistic fuzzy numbers for solving
MAGDM problems, Iranian Journal of Fuzzy Systems, 12(6) (2015), 41-74.
[20] M. Dolecki, P. Karczmarek, A. Kiersztyn, W. Pedrycz, Utility functions as aggregation functions in face recognition,
In: Proc. 2016 IEEE Symposium Series on Computational Intelligence (SSCI 2016), 2016.
[21] H. K. Ekenel, R. Stiefelhagen, Local appearance based face recognition using discrete cosine transform, In: Proceedings
of the 13th European Signal Processing Conference (EUSIPCO), Antalya, (2005), 2484-2488.
[22] H. K. Ekenel, R. Stiefelhagen, Generic versus salient region-based partitioning for local appearance face recognition,
In: M. Tistarelli and M. S. Nixon (Eds.), Advances in Biometrics, Lecture Notes in Computer Science, 5558 (2009),
[23] R. Gottumukkal, V. K. Asari, An improved face recognition technique based on modular PCA approach, Pattern
Recognition Letters, 25 (2004), 429-436.
[24] M. Grabisch, Fuzzy integral in multicriteria decision making, Fuzzy Sets and Systems, 69 (1995), 279-298.
[25] M. Grabisch, J.-L. Marichal, R. Mesiar, E. Pap, Aggregation functions, Cambridge University Press, Cambridge,
[26] J. Haddadnia, M. Ahmadi, N-feature neural network human face recognition, Image and Vision Computing, 22
(2004), 1071-1082.
[27] B. Heisele, V. Blanz, Morphable models for training a component-based face recognition system, In: W. Zhao and
R. Chellappa (Eds.), Face processing: Advanced modeling and methods, 1055 (2006), 439{462.
[28] B. Heisele, P. Ho, J. Wu, T. Poggio, Face recognition: Component-based versus global approaches, Computer Vision
and Image Understanding, 91 (2003), 6-21.
[29] B. Heisele, T. Serre, T. Poggio, A component-based framework for face detection and identi fication, International
Journal of Computer Vision, 74 (2007), 167{181.
[30] G. B. Huang, M. Mattar, T. Berg, E. Learned-Miller, Labeled faces in the wild: A database for studying face
recognition in unconstrained environments, In: Workshop on Faces in 'Real-Life' Images: Detection, Alignment, and
Recognition, ID: inria-00321923 (2007), 1{14.
[31] G. B. Huang, H. Lee, E. Learned-Miller, Learning hierarchical representations for face verifi cation with convo-
lutional deep belief networks, In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on,
(2012) 2518{2525.
[32] Z.-H. Huang, W.-J. Li, J. Wang, T. Zhang, Face recognition based on pixel-level and feature-level fusion of the
top-levels wavelet sub-bands, Information Fusion, 22 (2015), 95-104.
[33] G. Jarillo, W. Pedrycz, M. Reformat, Aggregation of classi fiers based on image transformations in biometric face
recognition, Machine Vision and Applications, 19 (2008), 125-140.
[34] P. Karczmarek, A. Kiersztyn, W. Pedrycz, An evaluation of fuzzy measure for face recognition. In: L. Rutkowski,
M. Korytkowski, R. Scherer, R. Tadeusiewicz, L. Zadeh, and J. Zurada J. (Eds.), ICAISC 2017, LNCS, 10245
(2017), 668{676.
[35] P. Karczmarek, A. Kiersztyn, W. Pedrycz, On developing Sugeno fuzzy measure densities in problems of face
recognition, International Journal of Machine Intelligence and Sensory Signal Processing, 2 (2017), 80-96.
[36] P. Karczmarek, A. Kiersztyn, W. Pedrycz, Generalized Choquet integral for face recognition, International Journal
of Fuzzy Systems, 20(3) (2017), 1047{1055.
[37] P. Karczmarek, A. Kiersztyn, W. Pedrycz, M. Dolecki, An application of chain code-based local descriptor and its
extension to face recognition, Pattern Recognition, 65 (2017), 26-34.
[38] P. Karczmarek, W. Pedrycz, M. Reformat, E. Akhoundi, A study in facial regions saliency: A fuzzy measure
approach, Soft Computing, 18 (2014), 379-391.
[39] T.-K. Kim, H. Kim, H., W. Hwang, J. Kittler, Component-based LDA face description for image retrieval and
MPEG-7 standardisation, Image and Vision Computing, 23 (2005), 631-642.
[40] J. Kittler, M. Hatef, R. P. W. Duin, J. Matas, On combining classi fiers, IEEE Transactions on Pattern Analysis
and Machine Intelligence, 20 (1998), 226{239.
[41] E. P. Klement, R. Mesiar, Logical, algebraic, analytic, and probabilistic aspects of triangular norms, Elsevier,
Amsterdam, 2005.
[42] E. P. Klement, R. Mesiar, E. Pap, Triangular norms. Kluwer Academic Publishers, Dordrecht, 2000.
[43] D. Kurach, D. Rutkowska, E. Rakus-Andersson, Face classi fication based on linguistic description of facial features.
In: L. Rutkowski, M. Korytkowski, R. Scherer, R. Tadeusiewicz, L. A. Zadeh, and J. M. Zurada (Eds.), ICAISC
2014, Part II, LNAI, 8468 (2014), 155-166.
[44] K.-C. Kwak, W. Pedrycz, Face recognition using fuzzy integral and wavelet decomposition method, IEEE Transactions
on Systems, Man, and Cybernetics, Part B: Cybernetics, 34 (2004), 1666-1675.
[45] K.-C. Kwak, W. Pedrycz, Face recognition: A study in information fusion using fuzzy integral, Pattern Recognition
Letters, 26 (2005), 719-733.
[46] Y. Lei, M. Bennamoun, A. A. El-Sallam, An efficient 3D face recognition approach based on the fusion of novel
local low-level features, Pattern Recognition, 46 (2013), 24-37.
[47] W. L. Liu, X. Q. Song, Q. Z. Zhang, S. B. Zhang, (T) fuzzy integral of multi-dimensional function with respect to
multi-valued measure, Iranian Journal of Fuzzy Systems, 9(3) (2012), 111{126.
[48] Z. Liu, C. Liu, Fusion of color, local spatial and global frequency information for face recognition, Pattern Recognition,
43 (2010), 2882-2890.
[49] G. Lucca, J. A., Sanz, G. P. Dimuro, B. Bedregal, R. Mesiar, A. Kolesarova, H. Bustince, Pre-aggregation functions:
Construction and an application, IEEE Transactions on Fuzzy Systems, 24 (2016), 260-272.
[50] G. E. Martnez, P. Melin, O. D. Mendoza, O. Castillo, Face recognition with Choquet integral in modular neural
networks. In: O. Castillo, P. Melin, W. Pedrycz, and J. Kacprzyk (Eds.), Recent advances on hybrid approaches for
designing intelligent systems. Studies in Computational Intelligence, 547 (2014), 437{449.
[51] G. E. Martnez, P. Melin, O. D. Mendoza, O. Castillo, Face recognition with a Sobel edge detector and the Choquet
integral as integration method in a modular neural networks. In: P. Melin, O. Castillo, and J. Kacprzyk (Eds.),
Design of intelligent systems based on fuzzy logic, neural networks and nature-inspired optimization. Studies in
Computational Intelligence, 601 (2015), 59{70.
[52] P. Melin, C. Felix, O. Castillo, Face recognition using modular neural networks and the fuzzy Sugeno integral for
response integration, International Journal of Intelligent Systems, 20 (2005), 275-291.
[53] A. R. Mirhosseini, H. Yan, K.-M. Lam, T. Pham, Human face image recognition: An evidence aggregation approach,
Computer Vision and Image Understanding, 71 (1998), 213-230.
[54] T. Murofushi, M. Sugeno, An interpretation of fuzzy measures and the Choquet integral as an integral with respect
to a fuzzy measure, Fuzzy Sets and Systems, 29 (1989), 201-227.
[55] C. Naveena, V. N. Manjunath Aradhya, S. K. Niranjan, The study of different similarity measure techniques in
recognition of handwritten characters. In: Proceedings of the International Conference on Advances in Computing,
Communications and Informatics (ICACCI-2012), ACM, (2012), 781-787.
[56] S.-K. Oh, S.-H. Yoo, W. Pedrycz, Design of face recognition algorithm using PCA -LDA combined for hybrid
data pre-processing and polynomial-based RBF neural networks : Design and its application, Expert Systems with
Applications, 40 (2013), 1451-1466.
[57] W. Pedrycz, F. Gomide, An introduction to fuzzy sets: Analysis and design. The MIT Press, Cambridge, 1998.
[58] A. Pentland, B. Moghaddam, T. Starner, View-based and modular eigenspaces for face recognition. In: Computer
Vision and Pattern Recognition. Proceedings CVPR '94., 1994 IEEE Computer Society Conference on, (1994),
[59] V. Perlibakas, Distance measures for PCA-based face recognition, Pattern Recognition Letters, 25 (1994), 711-724.
[60] P. J. Phillips, J. Wechsler, J. Huang, P. Rauss, The FERET database and evaluation procedure for face recognition
algorithms, Image and Vision Computing, 16 (1998), 295-306.
[61] P. V. W. Radtke, E. Granger, R. Sabourin, D. O. Gorodnichy, Skew-sensitive boolean combination for adaptive
ensembles An application to face recognition in video surveillance, Information Fusion, 20 (2014), 31-48.
[62] M. Sugeno, Theory of fuzzy integral and its applications, Dissertation. Tokyo Institute of Technology, Tokyo, 1974.
[63] Y. Sun, X. Wang, X. Tang, Hybrid deep learning for face veri fication, IEEE Transactions on Pattern Analysis and
Machine Intelligence, 38 (2016), 1997{2009.
[64] Y. Sun, X. Wang, X. Tang, Deep learning face representation from predicting 10,000 classes. In: The IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), (2014), 1891{1898.
[65] P. Tome, R. Vera-Rodriguez, J. Fierrez, Analysing facial regions for face recognition using forensic protocols. In:
J. M. Corchado et al. (Eds.), Highlights on practical applications of agents and multi-agent systems, 365 (2013),
[66] V. Torra, Y. Narukawa, Modeling decisions. Information fusion and aggregation operators. Springer-Verlag, Berlin
Heidelberg, 2007.
[67] M. Turk, A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3 (1991), 71-86.
[68] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, Y. Ma, Robust face recognition via sparse representation, IEEE
Transactions on Pattern Analysis and Machine Intelligence, 31 (2009), 210{227.
[69] Y. Xue, C. S. Tong, W. Zhang, Survey of distance measures for NMF-based face recognition. In: Y. Wang, Y.
Cheung, and H. Liu (Eds.), Computational Intelligence and Security, CIS 2006, Lecture Notes in Computer Science,
4456 (2007), 1039{1049.
[70] R. R. Yager, J. Kacprzyk, The ordered weighted averaging operators: Theory and Applications. Springer Science+
Business Media, New York, 2012.
[71] D. Yan, Y. Shen, Y. Yan, H. Wang, Non-linear feature fusion based on polynomial correlation filter for face
recognition. In: Lecture Notes in Computer Science, 8261 (2013), 312{319.
[72] Y. Yan, L. A. Osadciw, Intradifference-based segmentation and face identi cation. In: A. K. Jain and N. K. Ratha
(Eds.), Biometric technology for human identi cation, Proceedings of SPIE, 5404 (2004), 502-510.
[73] W. Zhao, R. Chellappa, P. J. Phillips, A. Rosenfeld, Face recognition: A literature survey, ACM Computing
Surveys, 35 (2003), 399458.
[74] Q. Zhu, Y. Xu, Multi-directional two-dimensional PCA with matching score level fusion for face recognition, Neural
Computing and Applications, 23 (2013), 169-174.