A QUADRATIC MARGIN-BASED MODEL FOR WEIGHTING FUZZY CLASSIFICATION RULES INSPIRED BY SUPPORT VECTOR MACHINES

Document Type : Research Paper

Authors

1 Department of Computer Science & Engineering & IT, Shiraz University, Shiraz, Fars, Iran

2 Department of Electrical Engineering, Science & Research Branch, Islamic Azad University, Marvdasht, Fars, Iran

3 Department of Computer Science & Engineering & IT, Shiraz Uni- versity, Shiraz, Fars, Iran

4 Department of Computer Science & Engineering & IT, Shiraz Univer- sity, Shiraz, Fars, Iran

Abstract

Recently, tuning the weights of the rules in Fuzzy Rule-Base Classification Systems is researched in order to improve the accuracy of classification. In this paper, a margin-based optimization model, inspired by Support Vector Machine classifiers, is proposed to compute these fuzzy rule weights. This approach not only  considers both accuracy and generalization criteria in a single objective function, but also is independent of any order in presenting data patterns or fuzzy rules. It has a global optimum solution and needs only one regularization parameter C to be adjusted. In addition, a rule reduction method is proposed to eliminating low weighted rules and having a compact rule-base. This method is compared with some greedy, reinforcement and local search rule weighting methods on 13 standard datasets. The experimental results show that, the proposed method significantly outperforms the other ones especially from the viewpoint of generalization.

Keywords


\bibitem{r31}
S. M. Chen,
{\it Generating weighted fuzzy rules from relational database systems for estimating values using genetic algorithms},
IEEE Trans. on Fuzzy Systems,
{\bf 11}\textbf{(4)} (2003), 495--506.

\bibitem{r32}
S. M. Chen,
{\it A new weighted fuzzy rule interpolation method based on GA-based weights-learning techniques},
proced. of ICMLC,
{\bf 5} (2010), 2705-2711.

\bibitem{r33}
S. M. Chen,
{\it Weighted fuzzy rule interpolation based on GA-based weight-learning techniques},
IEEE Trans. on Fuzzy Systems,
{\bf 19}\textbf{(4)} (2011),729--744.


\bibitem{r6}
Z. Chi, H. Yan and T. Pham,
{\it Fuzzy algorithms: with applications to image processing and pattern recognition},
World Scientific, Singapore, 1996.

\bibitem{r23}
O. Cordon, M. J. Del Jesus and F. Herrera,
{\it A proposal on reasoning methods in fuzzy rule-based classification systems},
Internat. J. Approx. Reason,
{\bf 20} (1999), 21--45.

\bibitem{r15}
O. Cordon, F. Gomide, F. Herrera, F. Hoffmann and L. Magdalena,
{\it Ten years of genetic fuzzy systems: current framework and new trends},
Fuzzy  Sets and Systems,
{\bf 141} (2004), 5--31.

\bibitem{r25}
C. Cortes and V. Vapnik,
{\it Support vector networks},
Machine Learning,
{\bf 20} (2004).

\bibitem{r29}
J. Demsar,
{\it Statistical comparisons of classifiers over multiple data sets},
Journal of Machine Learning Research,
{\bf 7} (2006), 1--30.


\bibitem{r1}
G. Forman and I. Cohen,
{\it Learning from Little: Comparison of Classifiers Given Little Training},
Springer-Verlag, PKDD 2004, (2004), 161--172.

\bibitem{r11}
L. Fu,
{\it Rule generation from neural networks},
IEEE Transaction on systems, Man, and Cybernetics,
{\bf 24}\textbf{(8)} (1994).


\bibitem{r30}
S. Garcia and F. Herrera,
{\it An extension on statistical comparison of classifiers over multiple data sets for all pair wise comparisons},
Journal of Machine Learning Research,
{\bf 9} (2008), 2677--2694.


\bibitem{r14}
H. B. Gurocak and A. de Sam Lazaro,
{\it A fine tuning method for fuzzy logic rule bases},
Elsevier, Fuzzy Sets and Systems,
{\bf 67} (1994), 147--161.

\bibitem{r26}
C. W. Hsu and C. J. Lin,
{\it A comparison of methods for multiclass support vector machines},
IEEE Transaction on neural networks,
{\bf 13}\textbf{(2)} (2002).

\bibitem{r17}
Q. Hu, P. Zhu, Y. Yang and D. Yu,
{\it Large-margin nearest neighbor classifiers via sample weight learning},
Neurocomputing,{\bf 74}\textbf{(4)} (2011), 656--660.

\bibitem{r20}
H. Ishibuchi, T. Murata and I. B. Turksen,
{\it Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems},
Fuzzy Sets and Systems,
{\bf 89}\textbf{(2)} (1997), 135--150.

\bibitem{r9}
H. Ishibuchi and T. Nakashima,
{\it Effect of Rule Weights in Fuzzy Rule-based Classification Systems},
IEEE Transactions on Fuzzy Systems,
{\bf 9}\textbf{(4)} (2001), 506--515.

\bibitem{r28}
H. Ishibuchi, T. Nakashima and T. Morisawa,
{\it Simple fuzzy rule-based classification systems perform well on commonly used real-world data sets Fuzzy Information Processing Society},
NAFIPS '97., 1997 Annual Meeting of the North American,
(1997), 251--256.

\bibitem{r22}
H. Ishibuchi, T. Nakashima and T. Morisawa,
{\it Voting in Fuzzy Rule-based Systems for Pattern Classification Problems},
Fuzzy Sets and Systems,
{\bf 103}\textbf{(2)} (1999), 223--238.

\bibitem{r5}
H. Ishibuchi, T. Nakashima and M. Nii,
{\it Classification and modeling with linguistic information granules: advanced approaches to linguistic data mining},
Springer Verlag, 2004.

\bibitem{r12}
H. Ishibuchi and M. Nii,
{\it Techniques and applications of neural networks for fuzzy rule approximation},
Fuzzy Theory Systems, (1999), 1491--1519.

\bibitem{r4}
H. Ishibuchi and Y. Nojima,
{\it Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning},
International Journal of Approximate Reasoning,
{\bf 44}\textbf {(1)} (2007), 4--31.

\bibitem{r34}
H. Ishibuchi, K. Nozaki and H. Tanaka,
{\it Distributed representation of fuzzy rules and its application to pattern classification},
Fuzzy Sets and Systems,
{\bf 52}\textbf{(1)} (1992), 21--32.

\bibitem{r19}
H. Ishibuchi, K. Nozaki, N. Yamamoto and H. Tanaka,
{\it Selecting fuzzy if-then rules for classification problems using genetic algorithms},
IEEE Transactions on Fuzzy Systems,
{\bf 3}\textbf{(3)} (1995), 260--270.

\bibitem{r16}
H. Ishibuchi and T. Yamamoto,
{\it Rule weight specification in fuzzy rule-based classification systems},
IEEE Trans. on Fuzzy Systems,
{\bf 13}\textbf{(4)} (2005), 428--435.

\bibitem{r21}
H. Ishibuchi and T. Yamamoto,
{\it Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining},
Fuzzy Sets and Systems,
{\bf 141}\textbf{(1)} (2004), 59--88.


\bibitem{r2}
J. Langford,
{\it Tutorial on practical prediction theory for classification},
Journal of Machine Learning Research,
{\bf 6} (2005), 273--306.

\bibitem{Mansoori2}
E. G. Mansoori, M. J. Zolghadri and S. D. Katebi,
{\it Using distribution of data to enhance performance of fuzzy classification systems}
Iranian Journal of Fuzzy Systems,
{\bf 4}\textbf{(1)} (2007), 21--36.

\bibitem{Mansoori1}
E. G. Mansoori, M. J. Zolghadri, S. D. Katebi and H. Mohabatkar,
{\it Generating fuzzy rules for protein classification},
Iranian Journal of Fuzzy Systems,
{\bf 5}\textbf{(2)} (2008), 21--33.


\bibitem{r3}
R. Mikut, J. Jakel and L. Groll,
{\it Interpretability issues in data-based learning of fuzzy systems},
Elsevier, Fuzzy Sets and Systems,
{\bf 150} (2005), 179--197.

\bibitem{r10}
T. Nakashima, G. Schaefer, Y. Yokota and H. Ishibuchi,
{\it A weighted fuzzy classifier and its application to image processing tasks},
Fuzzy Sets and Systems,
{\bf 158}\textbf{(3)} (2007), 284--294.

\bibitem{r18}
K. Nozaki, H. Ishibuchi and H. Tanaka,
{\it Adaptive fuzzy rule-based classification systems},
IEEE Transactions on Fuzzy Systems,
{\bf 4}\textbf{(3)} (1996), 238--250.

\bibitem{r24}
V. Vapnik,
{\it The nature of statistical learning theory},
Springer Verlag, New York, 1995.


\bibitem{r27}
J. Weston and C. Watkins,
{\it Support vector machines for multi-class pattern recognition},
ESANN'1999 proceedings - European Symposium on Artificial Neural Networks,
Bruges (Belgium),
isbn:2-600049-9-X, (1999), 219--224.

\bibitem{r13}
L. Yu and J. Xiao,
{\it Trade-off between accuracy and interpretability: experience-oriented fuzzy modeling via reduced-set vectors}, Elsevier, Computers and Mathematics with Applications,
{\bf 57} (2009), 885--895.

\bibitem{r8}
M. J. Zolghadri and E. G. Mansoori,
{\it Weighting fuzzy classification rules using receiver operating characteristics (ROC) analysis},
Information Sciences,
{\bf 177}\textbf{(11)} (2007), 2307--2296.

\bibitem{r7}
M. J. Zolghadri and M. Taheri,
{\it A proposed method for learning rule weights in fuzzy rule-based classification systems},
Fuzzy Sets and Systems, {\bf 159} (2008), 449--459.