MAN-MACHINE INTERACTION SYSTEM FOR SUBJECT INDEPENDENT SIGN LANGUAGE RECOGNITION USING FUZZY HIDDEN MARKOV MODEL

Document Type: Research Paper

Author

Department of Information Technology, Institute of Graduate Studies and Research. Alexandria University, 163 Horreya Avenue. El Shatby 21526. P.O. Box 832. Alexandria. Egypt

Abstract

Sign language recognition has spawned more and more interest in human–computer interaction society. The major challenge that SLR recognition faces now is developing methods that will scale well with increasing vocabulary size with a limited set of training data for the signer independent application. The automatic SLR based on hidden Markov models (HMMs) is very sensitive to gesture's shape information that makes the accurate parameters of the HMM not capable of characterizing the ambiguous distributions of the observations in gesture's features. This paper presents an extension of the HMMs using interval type-2 fuzzy sets (IT2FSs) to produce interval type-2 fuzzy HMMs to model uncertainties of hypothesis spaces (unknown varieties of parameters of the decision function). The benefit of this enlargement is that it can control both the randomness and fuzziness of traditional HMM mapping. Membership function (MF) of type-2 FS is three-dimensional that provides additional degrees of freedom to evaluate HMM's uncertainties. This system aspires to be a solution to the scalability problem, i.e. has real potential for application on a large vocabulary. Furthermore, it does not rely on the use of data gloves or other means as input devices, and operates in isolated signer-independent modes. Experimental results show that the interval type-2 fuzzy HMM has a comparable performance as that of the fuzzy HMM but is more robust to the gesture variation, while it retains almost the same computational complexity as that of the FHMM.

[1] M. AL-Rousan, K. Assaleh and A. Talaa, Video-based signer-independent arabic sign lan-
guage recognition using hidden markov models, Applied Soft Computing, 9 (2009), 990-999.
[2] V. Athitsos and S. Sclaroff, Estimating 3d hand pose from a cluttered image, Proc. IEEE Int.
Conf. Computer Vision and Pattern Recognition, USA, (2003), 1-8.
[3] S. Bilal, R. Akmeliawati, A. Shafi e and M. Salami, Hidden markov model for human to
computer interaction: a study on human hand gesture recognition, Arti ficial Intelligence
Review, 40(4) (2013), 495-516.
[4] N. Binh, E. Shuichi and T. Ejima, Real-time hand tracking and gesture recognition system,
Proc. Int. Conf. Graphics, Vision and Image Processing, Egypt, (2005), 362-368.
[5] X.-Q. Cao, J. Zeng and H. Yan, Modeling uncertain speech sequences using type-2 fuzzy
hidden markov models, Lecture Notes on Computer Science, 4810 (2007), 315-324.
[6] E. Celik, M. Gul, N. Aydin, A. Gumus and A. F Guneri, A comprehensive review of multi
criteria decision making approaches based on interval type-2 fuzzy sets, Knowledge-Based
Systems, 85 (2015), 329-341.
[7] V. Christian and D. Metaxas, Parallel hidden markov models for american sign language
recognition, Proc. IEEE Int. Conf. Computer Vision, Greece, 1 (1999), 116-122.
[8] P. Doliotis, V. Athitsos, D. Kosmopoulos and S. Perantonis, Hand shape and 3d pose esti-
mation using depth data from a single cluttered frame, Lecture Notes in Computer Science,
7431 (2012), 148-158.
[9] M. Elmezain, A. Al-Hamadi, J. Appenrodt and B. Michaelis, A Hidden markov model-based
isolated and meaningful hand gesture recognition, Int. J. Electrical, Computer, and Systems
Engineering, 3(3) (2009), 156-163.
[10] W. Gao, G. Fang, D. Zhao and Y. Chen, A chinese sign language recognition system based
on sofm/srn/hmm, Pattern Recognition, 37(12) (2004), 2389-2402.
[11] W. Gao, J. Ma and J. Wu, Sign language recognition based on hmm/ann/dr, Int. J. Pattern
Recognition and Arti cial Intelligencet, 4(5) (2000), 587-602.
[12] A. Ghotkar and G. Kharate, Hand segmentation techniques to hand gesture recognition for
natural human computer interaction, Int. J. Human Computer Interaction, 3(1) (2012), 15-
25.
[13] N. Ibraheem, M. Hasan, R. Khan and P. Mishra, Understanding color models: a review,
Journal of Science and Technology, 2(3) (2012), 265-275.
[14] Z. Jia and Z.-Q. Liu, Type-2 fuzzy sets for pattern recognition: the state-of-the-art, Journal
of Uncertain Systems, 1(3) (2007), 163-177.

[15] A. Jmaa, W. Mahdi, Y. Jemaa and A. Hmadou, A new approach for digit recognition based
on hand gesture analysis, Int. J. Computer Science and Information Security, 2(1) (2009),
1-8.
[16] M. Kelarestaghi, M. Slimane and N. Vincent, Introduction of fuzzy logic in the hidden markov
models, Proc. Int. Conf. Fuzzy Logic and Technology, UK, (2001), 14-16.
[17] W. Kong and S. Ranganath, Towards subject independent continuous sign language recogni-
tion: a segment and merge approach, Pattern Recognition, 47(3) (2014), 1294-1308.
[18] K. Kumar, M. Ramakrishna, and B. Prasad, "Feature extraction using sparse svd for bio-
metric fusion in multimodal authentication, Int. J. Network Security and Its Applications,
5(4) (2013), 83-94.
[19] S. Lung, Multi-resolution form of svd for text-independent speaker recognition, Pattern recog-
nition, 35(7) (2002), 1637-1639.
[20] S. Mitra and T. Acharya, Gesture recognition: a survey, IEEE Transactions on Systems,
Man, and Cybernetics, Part C: Applications and Reviews, 37(3) (2007), 311-324.
[21] G. Murthy and R. Jadon, A review of vision based hand gestures recognition, Int. J. Infor-
mation Technology and Knowledge Management, 2(2) (2009), 405-410.
[22] S. Ong and S. Ranganath, Automatic sign language analysis: a survey and the future beyond
lexical meaning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6)
(2005), 873-891.
[23] K. Patwardhan and S. Roy, Dynamic hand gesture recognition using predictive eigen tracker,
Proc. Int. Conf. Computer Vision, Graphics and Image Processing,India, (2004), 1-6.
[24] J. Rekha, J. Bhattacharya and S. Majumder, Hand gesture recognition for sign language:
a new hybrid approach, Proc. Int. Conf. Image Processing, Computer Vision and Pattern
Recognition, USA, (2011), 1-7.
[25] R. Shrivastava, A hidden markov model based dynamic hand gesture recognition system using
opencv, Proc. IEEE Int. Conf. Advance Computing, USA, (2013), 947{950.
[26] H. Sola, J. Fernandez, H. Hagras, F. Herrera, M. Pagola and E. Barrenechea, Interval type-
2 fuzzy sets are generalization of interval-valued fuzzy sets: toward a wider view on their
relationship, IEEE Transactions on Fuzzy Systems, 23(5) (2015), 1876-1882.
[27] M. Su, Fuzzy rule-based approach to spatio-temporal hand gesture recognition, IEEE Trans-
actions on Systems, Man and Cybernetics, 30(2) (2000), 276-281.
[28] N. Tanibata, N. Shimada and Y. Shirai, Extraction of hand Features for Recognition of sign
language words, Proc. Int. Conf. Vision Interface, Canada, (2002), 391-398.
[29] V. Tataru, R. Vieriu and L. Goras, On hand gestures recognition using hidden markov models,
Electronics and Telecommunications, 51(3) (2010), 29-32.
[30] V. Vezhnevets, S. Vassili and A. Andreeva, A survey on pixel-based skin color detection
techniques, Proc. Int. Conf. Graphicon, Russia, 3 (2003), 85-92.
[31] C. Vogler and D. Metaxas, A framework for recognizing the simultaneous aspects of american
sign language, Computer Vision and Image Understanding, 81 (2001), 358-384.
[32] H. Wang, M. Leu and C. Oz, American sign language recognition using multi-dimensional
hidden markov models, Journal of Information Science and Engineering, 22(5) (2006), 1109-
1123.
[33] A. Youssif, A. Aboutabl and H. Ali, Arabic sign language (arsl) recognition system using
hmm, Int. J. Advanced Computer System and Applications, 2(11) (2011), 45-51.
[34] J. Zeng and Z. Liu, Type-2 fuzzy hidden markov models, Studies in Computational Intelli-
gence, 591 (2015), 57-83.
[35] J. Zeng and Z. Liu, Type-2 fuzzy hidden markov models and their application to speech
recognition, IEEE Transaction on Fuzzy Systems, 14(3) (2006), 454-567.
[36] J. Zeng, L. Xie and Z.-Q. Liu, Type-2 fuzzy gaussian mixture models, Pattern Recognition,
41 (2008), 3636-3643.