Fuzzy decision tree algorithm based on feature value's class contribution level

Document Type : Research Paper

Authors

School of Mathematics and Statistics, Xidian University, Xi'an, PR China

Abstract

Both fuzzy set theory and probability theory could handle uncertainty, and researchers have always compared the two to try to figure out which is better. Meanwhile, fuzzy decision tree algorithms represent classification knowledge more naturally to the way of human thinking. However, most generalizations of fuzzy decision tree algorithms focus only on numerical fine-tuning and interpretability of the algorithms. In order to make full use of the information of data sets and improve the performance of fuzzy decision trees. In this paper, we apply prior probability knowledge to the construction of fuzzy decision tree. Therefore, first of all, based on prior probability knowledge, for a certain feature variable, we present a concept of feature value's class contribution level which shows the different roles the same feature values play  in different classes. Then, on the basis of feature value's class contribution level, we put forward a new algorithm called Feature Contribution Fuzzy Decision Tree (FCFDT). FCFDT algorithm has good classification results especially on data set with outliers. And it also maintains the interpretability of the fuzzy decision tree algorithms. The proposed algorithm is implemented and validated with seven state-of-the-art decision tree algorithms on 15 real-life data sets coming from the UCI Repository of machine learning. The experimental results obtained clearly indicate the superiority of the proposed scheme over the baseline methods.

Keywords

References

[1] I. D. Borlea, R. E. Precup, A. B. Borlea, D. Iercan, A unified form of fuzzy C-means and K-means algorithms and its partitional implementation, Knowledge-Based Systems, 214 (2021), 106731.
[2] R. A. Borzooei, B. Sheikh Hoseini, M. Mohseni Takallo, Results on t-fuzzy graphs, New Mathematics and Natural Computation, 16(01) (2020), 143-161.
[3] X. Boyen, L. Wehenkel, Fuzzy decision tree induction for power system security assessment, IFAC Proceedings Volumes, 28(26) (1995), 299-304.
[4] X. Boyen, L. Wehenkel, Automatic induction of fuzzy decision trees and its application to power system security assessment, Fuzzy Sets and Systems, 102 (1999), 3-19.
[5] W. Cao, X. Wang, Z. Ming, J. Gao, A review on neural networks with random weights, Neurocomputing, 275 (2018), 278-287.
[6] J. Demsar, Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research, 7 (2006), 1-30.
[7] D. Devi, S. K. Biswas, B. Purkayastha, Redundancy-driven modified Tomek-link based undersampling: A solution to class imbalance, Pattern Recognition Letters, 93 (2017), 3-12.
[8] J. A. Drakopoulos, Probabilities, possibilities, and fuzzy sets, Fuzzy Sets and Systems, 75 (2017), 1-15.
[9] D. Dua, K. Taniskidou, UCI machine learning repository, Irvine, CA: University of California, School of Information and Computer Science, (2017), http://archive.ics.uci.edu/ml.
[10] M. Durairaj, J. H. M. Asha, Fuzzy probability based person recognition in smart environments, Journal of Fuzzy Systems, 40(5) (2021), 9437-9452.
[11] N. EI, I. Karabadji, I. Khelf, H. Seridi, et al., A data sampling and attribute selection strategy for improving decision tree construction, Expert Systems with Applications, 129 (2019), 84-96.
[12] X. Gu, F. L. Chung, S. Wang, Bayesian Takagi-Sugeno-Kang fuzzy classifier, IEEE Transactions on Fuzzy Systems, 25(9) (2017), 1655-1671.
[13] Q. Hu, X. Che, L. Zhang, et al., Rank entropy-based decision trees for monotonic classification, IEEE Transactions on Knowledge and Data Engineering, 24(11) (2012), 2052-2064.
[14] C. Jin, F. Li, Y. Li, A generalized fuzzy ID3 algorithm using generalized information entropy, Knowledge-Based Systems, 64 (2014), 13-21.
[15] A. Kumar, M. Hanmandlu, H. M. Gupta, Fuzzy binary decision tree for biometric based personal authentication, Neurocomputing, 99(1) (2013), 87-97.
[16] M. Lavioleue, J. W. Seaman, J. D. Barrett, W. H. Woodall, A probabilistic and statistical view of fuzzy methods, Technometrics, 37(3) (1995), 249-261.
[17] X. Liu, X. Feng, W. Pedrycz, Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (AFS) approach, Data and Knowledge Engineering, 84 (2013), 1-25.
[18] H. T. Nguyen, Fuzzy sets and probability, Fuzzy Sets and Systems, 90 (1997), 129-132.
[19] A. Paez, F. Lopez, M. Ruiz, Inducing non-orthogonal and non-linear decision boundaries in decision trees via interactive basis functions, Expert Systems with Applications, 122 (2019), 183-206.
[20] C. Pozna, R. E. Precup, Applications of signatures to expert systems modelling, Acta Polytechnica Hungarica, 11(2) (2014), 21-39.
[21] J. R. Quinlan, Induction of decision trees, Machine Learning, 1 (1986), 81-106.
[22] Y. Ren, X. Liu, J. Cao, A parsimony fuzzy rule-based classifier using axiomatic fuzzy set theory and support vector machines, Information Science, 181(23) (2011), 5180-5193.
[23] L. Rokach, O. Z. Maimon, Data mining with decision trees: Theory and applications, World Scientific, (2nd edition), 2015.
[24] S. K. Savas, E. Nasibov, A fuzzy ID3 induction for linguistic data sets, International Journal of Intelligent Systems, 33 (2018), 858-878.
[25] X. Wang, X. Liu, W. Pedrycz, L. Zhang, Fuzzy rule based decision trees, Pattern Recognition, 48 (2015), 50-59.
[26] J. Wang, P. Lu, H. Zhang, X. Chen, Method of multi-criteria group decision-making based on cloud aggregation operators with linguistic information, Information Sciences, 274 (2014), 177-191.
[27] J. Wang, Y. Qian, F. Li, Fusing fuzzy monotonic decision trees, IEEE Transactions on Fuzzy Systems, 28(5) (2020), 887-900.
[28] Y. Yang, X. Bai, A research on classification performance of fuzzy classifiers based on fuzzy set theory, Iranian Journal of Fuzzy Systems, 16 (2019), 15-27.
[29] W. Yi, M. Lu, Z. Liu, Multi-valued attribute and multi-labels data decision tree algorithm, International Journal of Machine Learning and Cybernetics, 2(2) (2011), 67-74.
[30] Y. Yuan, M. J. Shaw, Induction of fuzzy decision trees, Fuzzy Sets and Systems, 69 (1995), 125-139.
[31] L. A. Zadeh, Fuzzy sets, Information and Control, 8 (1965), 338-353.
[32] J. Zhai, X. Z. Wang, S. Zhang, S. Hou, Tolerance rough fuzzy decision tree, Information Sciences, 465 (2018), 425-438.
[33] S. Zhao, H. Chen, C. Li, et al., A novel approach to building a robust fuzzy rough classifier, IEEE Transactions on Fuzzy Systems, 23(4) (2015), 769-786.
[34] H. Zhu, X. Wang, A cost-sensitive semi-supervised learning model based on uncertainty, Neurocomputing, 251 (2017), 106-114.
Iranian Journal of Fuzzy Systems
Volume 19, Issue 4
July and August 2022
Pages 73-88

Files

Share

How to cite

Statistics