Reducing dimensionality using copula-based extropy

Arad, F.; Sheikhi, A.; Keynia, F.

doi:10.22111/ijfs.2026.55828.9912

Articles in Press

Document Type : Original Manuscript

Authors

¹ Dept of Stat. Shahid Bahonar University (SUBK), Kerman, Iran

² Department of Energy Management and Optimization, Institute of Science and High Technology and Environmental Sciences, Graduate University of Advanced Technology, Kerman, Iran

10.22111/ijfs.2026.55828.9912

Abstract

When dealing with high-dimensional datasets, dimensionality reduction techniques become particularly important. Feature selection is one such method, aiming to select a subset of features such that the selected features have the highest relevance to the target variable and the lowest redundancy on each other. In this paper, we intend to utilize extropy, as a complement to entropy, for feature selection. Since common feature selection methods often ignore the dependencies between variables, we will consider these dependencies in the selection process by using copula functions. To this end, a feature selection algorithm based on copula functions and extropy is introduced, and the results of its implementation on real and simulated datasets are presented. More specific, we assess our propose approach with 3 scenarios in simulation study and we used it in feature selection of 3 real word dataset. These results demonstrate that while our proposed feature selection method significantly outperforms other alternatives, it benefits from capturing the nonlinear dependencies.

Keywords

Main Subjects

Decision Making

References

[1] A. H. Abdel-aziem, T. H. M. Soliman, A multi-layer perceptron (MLP) neural networks for stellar classification: A review of methods and results, International Journal of Advances in Applied Computational Intelligence, 3(2) (2023), 29-37. https://doi.org/10.54216/IJAACI.030203

[2] N. Balakrishnan, F. Buono, M. Longobardi, On weighted extropy, Communications in Statistics- Theory and Methods, 51(18) (2022), 6250-6267. https://doi.org/10.1080/03610926.2020.1860222

[3] M. Billah, S. Waheed, Minimum redundancy maximum relevance (mRMR) based feature selection from endoscopic images for automatic gastrointestinal polyp detection, Multimedia Tools and Applications, 79(33-34) (2020), 23633 23643. https://doi.org/10.1007/s11042-020-09151-7

[4] F. Buono, O. Kamari, M. Longobardi, Interval extropy and weighted interval extropy, Ricerche di Matematica, 72(1) (2023), 283-298. https://doi.org/10.48550/arXiv.2112.01152

[5] M. Cord, P. Cunningham, Machine learning techniques for multimedia: Case studies on organization and retrieval, Springer Berlin, Heidelberg, 2008. https://doi.org/10.1007/978-3-540-75171-7

[6] A. Dhindsa, S. Bhatia, S. Agrawal, B. S. Sohi, An improvised machine learning model based on mutual information feature selection approach for microbes classification, Entropy, 23(2) (2021), 257. https://doi.org/10.3390/ e23020257

[7] R. Díaz-Uriarte, S. Alvarez de Andrès, Gene selection and classification of microarray data using random forest, BMC Bioinformatics, 7 (2006), 3. https://doi.org/10.1186/1471-2105-7-3

[8] K. Femmam, B. Brahimi, S. Femmam, An optimized feature selection technique based on bivariate copulas “GBCFS”, Journal of Combinatorial Optimization, 45(2) (2023), 74. https://doi.org/10.1007/ s10878-023-01006-9

[9] K. Femmam, S. Femmam, Fast and efficient feature selection method using bivariate copulas, Journal of Advances in Information Technology, 13(3) (2022), 301-305. https://doi.org/10.12720/jait.13.3.301-305

[10] I. K. Fodor, A survey of dimension reduction techniques, Lawrence Livermore National Laboratory (LLNL), Livermore, CA, United States, 2002.

[11] V. Fonti, E. Belitser, Feature selection using LASSO, VU Amsterdam Research Paper in Business Analytics, 30 (2017), 1-25.

[12] T. Hastie, R. Tibshirani, J. Friedman, The elements of statistical learning: Data mining, inference, and prediction, Springer Series in Statistics, 2nd ed., Springer, New York, 2009. https://doi.org/10.1007/978-0-387-84858-7

[13] M. Jian, S. Zengqi, Mutual information is copula entropy, Tsinghua Science and Technology, 16(1) (2011), 51-54. https://doi.org/10.1016/S1007-0214(11)70008-6

[14] I. T. Jolliffe, J. Cadima, Principal component analysis: A review and recent developments, Philosophical Trans actions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065) (2016), 20150202. https://doi.org/10.1098/rsta.2015.0202

[15] F. Lad, G. Sanfilippo, G. Agró, Extropy: Complementary dual of entropy, Statistical Science, 30(1) (2015), 40-58. https://doi.org/10.1214/14-STS430

[16] S. Lall, D. Sinha, A. Ghosh, D. Sengupta, et al., Stable feature selection using copula based mutual information, Pattern Recognition, 112 (2021), 107697. https://doi.org/10.1016/j.patcog.2020.107697

[17] Y. S. Muhammad, M. Iqbal, I. Hussain, S. Kamal, et al., Modeling non-linear behavior of independent variables, International Journal of Business and Social Science, 5(13) (2014), 192-200.

[18] R. D. Nair, E. I. A. Sathar, Some useful results related to various measures of extropy and their interrelationship, Statistics and Probability Letters, 193 (2023), 109729. https://doi.org/10.1016/j.spl.2022.109729

[19] S. Nanga, A. T. Bawah, B. A. Acquaye, R. Bill, et al., Review of dimension reduction methods, Journal of Data Analysis and Information Processing, 9(3) (2021), 189-231. https://doi.org/10.4236/jdaip.2021.93013

[20] R. B. Nelsen, An introduction to copulas, Springer Series in Statistics, 2nd ed., Springer, New York, 2006. https: //doi.org/10.1007/0-387-28678-0

[21] M. A. Nielsen, Neural networks and deep learning, Determination Press, San Francisco, CA, USA, 2015.

[22] H. J. Park, T. W. Lee, Modeling nonlinear dependencies in natural images using mixture of Laplacian distribution, Advances in Neural Information Processing Systems, 17 (2004), 1041-1048.

[23] H. Peng, G. Fang, P. Li, Copula for instance-wise feature selection and ranking, Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, 216 (2023), 1651-1661. https://doi.org/10.5555/3625834. 3625989

[24] N. Pudjihartono, T. Fadason, A. W. Kempa-Liehr, J. M. O’Sullivan, A review of feature selection methods for machine learning-based disease risk prediction, Frontiers in Bioinformatics, 2 (2022), 927312. https://doi.org/ 10.3389/fbinf.2022.927312

[25] M. A. Rahman, M. Hamada, S. Sharmin, T. A. Rimi, et al., Enhancing early breast cancer detection through advanced data analysis, IEEE Access, 12 (2024), 161941-161953. https://doi.org/10.1109/ACCESS.2024.3483095

[26] P. Ray, S. S. Reddy, T. Banerjee, Various dimension reduction techniques for high dimensional data analysis: A review, Artificial Intelligence Review, 54(5) (2021), 3473-3515. https://doi.org/10.1007/s10462-020-09928-0

[27] D. M. Reif, A. A. Motsinger, B. A. McKinney, J. E. Crowe, et al., Feature selection using a random forests classifier for the integrated analysis of multiple data types, Proceedings of the 2006 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, (2007), 1-8. https://doi.org/10.1109/CIBCB.2006. 330987

[28] S. Saha, S. Kayal, Copula-based extropy measures, properties, and dependence in bivariate distributions, Communications in Statistics- Theory and Methods, 55(1) (2026), 189-216. https://doi.org/10.1080/03610926.2025. 2491720

[29] A. Sheikhi, R. Mesiar, M. Holeňa, A dimension reduction in neural network using copula matrix, International Journal of General Systems, 52(2) (2023), 131-146. https://doi.org/10.1080/03081079.2022.2108029

[30] G. Shi, Y. Sheng, H. Ahmadzade, S. Tahmasebi, Extropy: Dual of entropy for uncertain random variables and its applications, American Institute of Mathematical Sciences, 21(5) (2025), 4025-4040. https://doi.org/10.3934/ jimo.2025041

[31] A. Sklar, Fonctions de répartition á n dimensions et leurs marges, Publications de l’Institut Statistique de l’Universitè de Paris, 8 (1959), 229-231.

[32] C. Strobl, A. L. Boulesteix, A. Zeileis, T. Hothorn, Bias in random forest variable importance measures: Illustrations, sources and a solution, BMC Bioinformatics, 8(1) (2007), 25. https://doi.org/10.1186/1471-2105-8-25

[33] A. Toomaj, M. Hashempour, N. Balakrishnan, Extropy: Characterizations and dynamic versions, Journal of Ap plied Probability, 60(4) (2023), 1333-1351. https://doi.org/10.1017/jpr.2023.7

[34] A. Ullah, T. Wang, W. Yao, Nonlinear modal regression for dependent data with application for predicting COVID 19, Journal of the Royal Statistical Society Series A: Statistics in Society, 185(3) (2022), 1424-1453. https: //doi.org/10.1111/rssa.12849

[35] B. Venkatesh, J. Anuradha, A review of feature selection and its methods, Cybernetics and Information Technologies, 19(1) (2019), 3-26. https://doi.org/10.2478/cait-2019-0001

[36] G. Wang, F. Lauri, A. H. El Hassani, Feature selection by mRMR method for heart disease diagnosis, IEEE Access, 10 (2022), 100786-100796. https://doi.org/10.1109/ACCESS.2022.3207492

Iranian Journal of Fuzzy Systems

Reducing dimensionality using copula-based extropy

References

References

Articles in Press, Accepted Manuscript
Available Online from 11 November 2026

Reducing dimensionality using copula-based extropy

References

References

Articles in Press, Accepted Manuscript Available Online from 11 November 2026

Articles in Press, Accepted Manuscript
Available Online from 11 November 2026