^{1}Department of Statistics, Ferdowsi University of Mashhad, Mashhad, Iran

^{2}Department of Applied Mathematics, Ferdowsi University of Mashhad, Mashhad, Iran

Abstract

Support Vector Regression (SVR) solves regression problems based on the concept of Support Vector Machine (SVM). In this paper, a new model of SVR with probabilistic constraints is proposed that any of output data and bias are considered the random variables with uniform probability functions. Using the new proposed method, the optimal hyperplane regression can be obtained by solving a quadratic optimization problem. The proposed method is illustrated by several simulated data and real data sets for both models (linear and nonlinear ) with probabilistic constraints.

[1] A. R. Arabpour and M. Tata, Estimating the parameters of a fuzzy linear regression model, Iranian Journal of Fuzzy Systems, 5(2) (2008), 1–19. [2] K. Bache and M. Lichman, UCI machine learning repository, Available on-line at: http://archive.ics.uci.edu/ml/machine-learning-databases, 2013. [3] A. Ben-Tal, S. Bhadra, C. Bhattacharyya and J. S. Nath, Chance constrained uncertain classification via robust optimization, Math. Program., 127(1) (2011), 145–173. [4] P. Bosch, J. Lopez, H. Ramirez and H. Robotham, Support vector machine under uncertainty: an application for hydroacoustic classification of fish-schools in Chile, Expert Systems with Applications, 40 (2013), 4029–4034. [5] K. D. Brabanter, J. D. Brabanter, J. A. K. Suykens and B. D. Moor, Approximate confidence and prediction intervals for least squares support vector regression, IEEE Transactions on Neural Networks, 22 (2011), 110–120. [6] E. Carrizosa, J. E. Gordillo and F. Plastria, Kernel support vector regression with imprecise output, Dept. MOSI. Vrije Univ. Brussel. Belgium. Tech. Rep., Available on-line at: http://www.optimization-online.org/DB_FILE/2008/01/1896.pdf, 2008. [7] E. Carrizosa, J. E. Gordillo and F. Plastria, Support vector regression for imprecise data, Dept. MOSI. Vrije Univ. Brussel. Belgium. Tech. Rep., Available on-line at: http://www.optimization-online.org/DB_HTML/2007/11/1826.html, 2007. [8] J. H. Chiang and P. Y. Hao, Support vector learning mechanism for fuzzy rule-based modeling: a new approach, IEEE Trans. Fuzzy Syst., 12(1) (2004), 1–12. [9] H. Drucker, Ch. J. C. Burges, L. Kaufman, A. Smola and V. Vapnik, Support vector regression machines, Adv. Neural Inform. Process. Syst., 9 (1997), 155–161. [10] B. Efron, Bootstrap methods: Another look at the jackknife, Annals of Statistics, 7 (1979) 1–26. [11] A. Farag and R. M. Mohamed, Classification of multispectral data using support vector machines approach for density estimation, IEEE Seventh International Conference on Intell. Eng. Syst., (2003), 4–6. [12] J. B. Gao, S. R. Gunn, C. J. Harris and M. Brown, A probabilistic framework for SVM regression and error bar estimation, Machine Learning, 46 (2002), 71–89. [13] P. Y. Hao and J. H. Chiang, A fuzzy model of support vector regression machine, International Journal of Fuzzy Systems, 9(1) (2007), 45–49. [14] H. P. Huang and Y. H. Liu, Fuzzy support vector machines for pattern recognition and data mining, International Journal of Fuzzy Systems, 4 (2002), 826–835. [15] G. Huang, S. Song, C. Wu and K. You, Robust support vector regression for uncertain input and output data, IEEE Transactions on Neural Networks and Learning Systems, 23(11) (2012), 1690–1700. [16] R. K. Jayadeva, R. Khemchandani and S. Chandra, Twin support vector machines for pattern classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(5) (2007), 905–910. [17] Y. Jinglin, H. X. Li and H. Yong, A probabilistic SVM based decision system for pain diagnosis, Expert Systems with Applications, 38 (2011), 9346–9351. [18] A. F. Karr, probability, Springer, New york, (1993), 52–74. [19] M. A. Kumar and M. Gopal, Least squares twin support vector machines for pattern classification, Expert Systems with Applications, 36(4) (2009), 7535–7543. [20] J. T. Y. Kwok, The evidence framework applied to support vector machines, IEEE Transactions on Neural Networks, 11 (2000), 1162–1173. [21] G. R. G. Lanckriet, L. E. Ghaoui, Ch. Bhattacharyya and M. I. Jordan, A robust minimax approach to classification, J. Mach. Learn. Res., 3 (2002), 555–582.

[22] Y. J. Lee and S. Y. Huang, Reduced support vector machines: a statistical theory, IEEE Transactions on Neural Networks, 18 (2007), 1–13. [23] H. Li, J. Yang, G. Zhang and B. Fan, Probabilistic support vector machines for classification of noise affected data, Information Sciences, 221 (2013), 60–71. [24] C. F. Lin and S. D. Wang, Fuzzy support vector machine, IEEE Transactions on Neural Networks, 13 (2002), 464–471. [25] W. Y. Liu, K. Yue and M. H. Gao, Constructing probabilistic graphical model from predicate formulas for fusing logical and probabilistic knowledge, Information Sciences, 181(18) (2011), 3828–3845. [26] M. Lobo, L. Vandenberghe, S. Boyd and H. Lebret, Applications of second-order cone programming, Linear Algebra Its Appl., 284 (1998), 193–228. [27] O. L. Mangasarian, Nonlinear Programming, McGraw-Hill, New York, (1969), 69–75. [28] S. Mehrotra, On the implementation of a primal-dual interior point method, SIAM J. Optim., 2 (1992), 575–601. [29] X. Peng, TSVR: an efficient twin support vector machine for regression, Neural Networks., 23(3) (2010), 365–372. [30] J. C. Platt, Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods, Advances in Large Margin Classifiers, 10(3) (1999), 61–74. [31] Z. Qi, Y. Tian and Y. Shi, Robust twin support vector machine for pattern classification, Pattern Recognition, 46 (2013), 305–316. [32] H. Sadoghi Yazdi, S. Effati and Z. Saberi, The probabilistic constraints in the support vector machine, App. Math. Comput., 194 (2007), 467–479. [33] P. K. Shivaswamy, Ch.Bhattacharyya and A.J.Smola, Second order cone programming approaches for handling missing and uncertain data,J. Mach. Learn. Res.,7 (2006), 1283-1314. [34] P. Sollich, Bayesian methods for support vector machines: evidence and predictive class probabilities, Machine Learning, 46 (2002), 21–52. [35] J. A. K. Suykens and J. Vandewalle, Least squares support vector machine classifiers, Neural Processing Letters, 9(3) (1999), 293–300. [36] T. B. Trafalis and S. A. Alwazzi, Support vector regression with noisy data: a second order cone programming approach, Int. J. General Syst., 36 (2007), 237–250. [37] T. B. Trafalis and R. C. Gilbert, Robust classification and regression using support vector machines, Eur. J. Oper. Res., 173(3) (2006), 893–909. [38] URL http://www.csie.ntu.edu.tw/ cjlin/libsvmtools/datasets/regression.html. [39] URL http://www.dcc.fc.up.pt/ ltorgo/Regression/DataSets.html. [40] URL http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml. [41] V. Vapnik, The nature of statistical learning theory, Springer-Verlag, New York, (1995), 123-146, 181-186. [42] V. Vapnik, S. Golowich and A. Smola, Support vector method for multivariate density estimation, Adv. Neural Inform. Process. Syst., 12 (1999), 659–665. [43] Y. Xu and L. Wang, A weighted twin support vector regression, Knowledge-Based Syst., 33 (2012), 92–101. [44] Y. Xu, W. Xi, X. Lv and R. Guo, An improved least squares twin support vector machine, Journal of information and computational science, 9(4) (2012), 1063–1071. [45] X. Yang, L. Tan and L. He, A robust least squares support vector machine for regression and classification with noise, Neurocomputing, 140 (2014), 41–52.