A research on classification performance of fuzzy classifiers based on fuzzy set theory

Document Type : Research Paper

Authors

1 School of Mathematics and Statistics, Xidian University, Xi’an 710126, PR China.

2 School of Mathematics and Statistics, Xidian University, Xi’an 710126, PR China,, and School of Sciences, Northwest A&F University, Yangling 712100, PR China.

Abstract

Due to the complexities of objects and the vagueness of the human mind, it has attracted considerable attention from researchers studying fuzzy classification algorithms. In this paper, we propose a concept of fuzzy relative entropy to measure the divergence between two fuzzy sets. Applying fuzzy relative entropy, we prove the conclusion that patterns with high fuzziness are close to the classification boundary. Thus, it plays a great role in classification problems that patterns  with high fuzziness are classified correctly. Meanwhile, we draw a conclusion that the fuzziness of a pattern and the uncertainty of its class label are equivalent. As is well known, entropy not only measures the uncertainty of random variable, but also represents the amount of information carried by the variable. Hence, a fuzzy classifier with high fuzziness would  carry much information about training set. Therefore, in addition to some assessment criteria such as classification accuracy, we could study the classification performance from the perspective of the fuzziness of classifier. In order to try to ensure the objectivity in dealing with unseen patterns, we should make full use of information of the known pattern set and do not make too much subjective assumptions in the process of learning. Consequently,  for problems with rather complex decision boundaries especially, under the condition that a certain training accuracy threshold is maintained, we demonstrate that a fuzzy classifier with high fuzziness would have a well generalization performance.

Highlights

[1] A. Agarwal, J. C. Duchi, The generalization ability of online algorithms for dependent data, IEEE Transactions on Information
Theory, 59(1) (2013), 573–587.
[2] E. Alpaydin, Introduction to machine learning, 3nd edition, MIT Press (2014), 1–5.
[3] G. Atalik, S. Senturk, A new approach for parameter estimation in fuzzy logistic regression, Iranian Journal of Fuzzy Systems,
15(1) (2018), 91–102.
[4] R. Batuwita, V. Palade, FSVM-CIL: Fuzzy support vector machine for class imbalance learning, IEEE Transactions on Fuzzy
Systems, 18(3) (2010), 558–571.
[5] J. C. Bezdek, Pattern recognition with fuzzy objective function algorithms, New York: Plenum Press (1981), 203–239.
[6] J. J. Buckley, Y. Hayashi, Fuzzy neural networks: A survey, Fuzzy Sets and Systems, 66 (1994) 1–13.
[7] O. Chapelle, V. Vapnik, O. Bousquet, S. Mukherjee, Choosing multiple parameters for support vector machines, Machine
learning, 46(1) (2002), 131–159.
[8] S. Y. Chong, P. Tino, X. Yao, Relationship between generalization and diversity in coevolutionary learning, IEEE Transactions
on Computational Intelligence and AI in Games, 1(3) (2009), 214–232.
[9] T. M. Cover, J. A. Thomas, Elements of information theory. Jhon wiley & Sons, INC., Publication, Sec. edition (2006), 13-29.
[10] A. De Luca, S. Termini, A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory, Information and Control,
20 (1972), 301-312.
[11] S. Decherchi, S. Ridella, R. Zunino, P. Gastaldo, D. Anguita, Using unsupervised analysis to constrain generalization bounds
for support vector classifiers, IEEE Transactions on Neural Networks, 21(3) (2010), 424–438.
[12] D. Dua, Karra Taniskidou, UCI machine learning repository, [http://archive.ics.uci.edu/ml], Irvine, CA: University of Cali-
fornia, School of Information and Computer Science, 2017.
[13] R. O. Duda, P. E. Hart, D. G. Stork. Pattern Classification. Wiley Press, (2001), 394–453.
[14] S. Ezghari, A. Zahi, K. Zenkouar, A new nearest neighbor classification method based on fuzzy set theory and aggregation
operators, Expert Systems with Applications, 80 (2017), 58–74.
[15] C. C. Gavin, L. C. Nicola, On over-fitting in model selection and subsequent selection bias in performance evaluation,
Journal of Machine Learning Research, 11 (2010), 2079–2107.

[16] M. Gupta, R. Ragade, P. Yager, Advances in fuzzy set theory and applications, North-Holland Publishing Company, 22(4)
(1979), 623–633.
[17] T. Hastie, R. Tibshirani, J. H. Friedman. The elements of statistical learning: Data mining, inference and prediction, Springer
(2009), 295–481.
[18] L. Hu, K. C. C. Chan, Fuzzy clustering in a complex network based on content relevance and link structures, IEEE Transac-
tions on Fuzzy Systems, 24(2) (2016), 456–470.
[19] A. Kandel, Fuzzy techniques in pattern recognition. New York, John Wiley, 1982.
[20] J. M. Keller, M. R. Gray, J. A. Givens, A fuzzy K-nearest neighbor algorithm, IEEE Transactions on Systems Man and
Cybernetics, SMC-15(4) (1985), 580–585.
[21] C. Lee, D. A. Landgrebe, Decision boundary feature extraction for neural networks, IEEE Transactions on Neural Networks
and Learning, 8(1) (1997), 75–83.
[22] C. F. Lin, S. D. Wang, Fuzzy support vector machines, IEEE Transactions on Neural Networks and Learning, 13(2) (2002),
464–471.
[23] O. Ludwig, U. Nunes, B. Ribeiro, C. Premebida. Improving the generalization capacity of cascade classifers, IEEE Transac-
tions on Systems, Man and Cybernetics, 43(6) (2013), 2135–2146.
[24] R. K. Nowicki, J. T. Starcaewski, A new method for classification of imprecise data using fuzzy rough fuzzification, Informa-
tion Sciences, 414(5) (2017), 33–52.
[25] A. H. M. Pimenta, H. de A. Camargo, Genetic interval type-2 fuzzy classifier generalization: A comparative approach,
Eleventh Brazilian Symposium on Neural Networks, (2010), 194–199.
[26] X. Qiao, L. Zhang, Flexible high-dimensional classification machines and their asymptotic properties, Journal of Machine
Learning Research, 16(8) (2015), 1547–1572.
[27] S. S. Shwartz, S. B. David, Understanding machine learning-from theory to algorithms, New York, NY, USA: Cambridge
Press (2014), 43–54.
[28] S. K. Shukla, M. K. Tiwari, GA guided cluster based fuzzy decision tree for reactive ion etching modeling: A data mining
approach, IEEE Transactions on Semiconductor Manufacturing, 25(1) (2012), 45–56.
[29] M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Statistical Society, Ser. B,
(Statist. Methodol.), 36(2) (1974), 111–147.
[30] G. Varando, C. Bielza, P. Larranaga, Decision boundary for discrete bayesian network classifiers, Journal of Machine Learn-
ing Research, 16(12) (2015), 2725–2749.
[31] P. P. Wang, S. K. Chang, Theory and applications to policy analysis and information systems. New York: Plenum Press,
1980.
[32] X. Z. Wang, H. J. Xing, Y. Li, Q. Hua, C. R. Dong, W. Pedrycz, A study on relationship between generalization abilities and
fuzziness of base classifiers in ensemble learning, IEEE Transactions on Fuzzy Systems, 23(5) (2015), 1638–1653.
[33] I. H. Witten, E. Frank, Data mining: practical machine learning tools and techniques, 2nd edition, Morgan Kaufmann
Publication (2005), 1–36.
[34] Z. Yan, C. Xu, Studies on classification models using decision boundaries, in Proc. 8th IEEE International Conference of
Cognitive Information, (2009), 287-294.
[35] D. S. Yeung, E. Tsang, Measures of fuzziness under di erent uses of fuzzy sets, Advanced Computing Intelligent Communi-
cation Computing Information Sciences, 298 (2012), 25–34.
[36] Y. Yuan, M. J. Shaw, Induction of fuzzy decision trees. Fuzzy Sets and Systems, 69 (1995), 125–139.
[37] L. A. Zadeh, Fuzzy sets, Information Control, 8 (1965) 338–353.
[38] Y. Zhang, R. Maciejewski, Quantifying the visual impact of classification boundaries in choropleth maps, IEEE Transactions on Visualization and Computer Graphics, 23(1) (2017), 371–380.

 

Keywords

Iranian Journal of Fuzzy Systems
Volume 16, Issue 5
September and October 2019
Pages 15-27

Files

Share

How to cite

Statistics