Home   >   CSC-OpenAccess Library   >    Manuscript Information
Video Audio Interface for recognizing gestures of Indian sign Language
E.Kiran Kumar, S.R.C.Kishore , P.V.V.Kishore, P.Rajesh Kumar
Pages - 479 - 503     |    Revised - 01-09-2011     |    Published - 05-10-2011
Volume - 5   Issue - 4    |    Publication Date - September / October 2011  Table of Contents
MORE INFORMATION
KEYWORDS
Sign language, Wavelet transform, Image Fusion, Elliptical Fourier Descriptors, Principle Component Analysis, Fuzzy Inference System
ABSTRACT
We proposed a system to robotically recognize gestures of sign language from a video stream of the signer. The developed system converts words and sentences of Indian sign language into voice and text in English. We have used the power of image processing techniques and artificial intelligence techniques to achieve the objective. To accomplish the task we used powerful image processing techniques such as frame differencing based tracking, edge detection, wavelet transform, image fusion techniques to segment shapes in our videos. It also uses Elliptical Fourier descriptors for shape feature extraction and principal component analysis for feature set optimization and reduction. Database of extracted features are compared with input video of the signer using a trained fuzzy inference system. The proposed system converts gestures into a text and voice message with 91 percent accuracy. The training and testing of the system is done using gestures from Indian Sign Language (INSL). Around 80 gestures from 10 different signers are used. The entire system was developed in a user friendly environment by creating a graphical user interface in MATLAB. The system is robust and can be trained for new gestures using GUI.
CITED BY (18)  
1 Joudaki, S., Mohamad, D. B., Saba, T., Rehman, A., Al-Rodhaan, M., & Al-Dhelaan, A. (2014). Vision-Based Sign Language Classification: A Directional Review. IETE Technical Review, 31(5), 383-391.
2 Melnyk, M., Shadrova, V., & Karwatsky, B. (2014). Towards Computer Assisted International Sign Language Recognition System: A Systematic Survey. International Journal of Computer Applications, 89(17), 44-51.
3 Babu, S. S., Subrahmanyam, K. V., & Tech, M. Lung Cancer Diagnosis using PET/CT Images based on preprocessing methods.
4 Venkatram, N., reddy, l., kishore, p., & shavya, c. (2014). rsa-dwt based medical image watermarking for telemedicine applications. Journal of Theoretical & Applied Information Technology, 65(3).
5 Gimbi, T. (2014). Recognition of Isolated Signs in Ethiopian Sign Language (Doctoral dissertation, Addis Ababa University).
6 Venkatram, N., Reddy, L. S. S., & Kishore, P. V. V. (2014). Multiresolution medical image watermarking for telemedicine applications. Digital Image Processing, 6(1), 6-15.
7 Futane, P. R., Dharaskar, R. V., & Thakare, V. M. (2014). Survey and analysis of Indian Sign Language Recognition research. International Journal of Advanced Research in Computer Science, 5(4).
8 Divya, S., Kiruthika, S., & Padmavathi, S. Segmentation, Tracking And Feature Extraction For Indian Sign Language Recognition.
9 Venkatram, N., Reddy, L. S. S., Kishore, P. V. V., Fields, G., Vaddeswaram, G. D., & Pradesh, A. (2014). Blind Medical Image Watermarking with LWT–SVD for Telemedicine Applications. image, 20, 23.
10 Venkatram, N., Reddy, L. S. S., & Kishore, P. V. V. Dwt-Bat Based Medical Image Watermarking For Telemedicine Applications.
11 Kishore, P. V. V., Venkatram, N., Sarvya, C., & Reddy, L. S. S. (2014, August). Medical image watermarking using RSA encryption in wavelet domain. In Networks & Soft Computing (ICNSC), 2014 First International Conference on (pp. 258-262). IEEE.
12 Reshna, S., & Jayaraju, M. (2015). A survey on Segmentation and Feature extraction of Images in Indian Sign Language Recognition System.
13 Nair, A. V., & Bindu, V. (2013). A Review on Indian Sign Language Recognition. International Journal of Computer Applications, 73(22), 33-38.
14 Kishore, P. V. V., Rahul, R., Sravya, K., & Sastry, A. S. C. S. (2015, August). Crowd Density Analysis and tracking. In Advances in Computing, Communications and Informatics (ICACCI), 2015 International Conference on (pp. 1209-1213). IEEE.
15 Ghotkar, A. S., & Kharate, G. K. (2015). Dynamic Hand Gesture Recognition and Novel Sentence Interpretation Algorithm for Indian Sign Language Using Microsoft Kinect Sensor. Journal of Pattern Recognition Research, 1, 24-38.
16 Kausar, N., Samir, B. B., & Kuleev, R. (2006). Lung cancer detection using supervised classification with cluster variability on radiographs data. women, 25, 19-04.
17 Savitha, S. K., & Naveen, N. C. Study for Assessing the Advancement of Imaging Techniques in Chest Radiographic Images.
18 Kishore, P. V. V., Mallika, K. L., Prasad, M. V. D., & Narayana, K. L. (2015). Denoising Ultrasound Medical Images with Selective Fusion in Wavelet Domain. Procedia Computer Science, 58, 129-139.
1 Google Scholar 
2 CiteSeerX 
3 refSeek 
4 Scribd 
5 SlideShare 
6 PdfSR 
Abdel-Fattah, M.A., 2005. Arabic sign language: a perspective. Journal of Deaf Studies and Deaf Education 10 (2), 212–221.
ASL corpus: .
Atherton, M., 1999. Welsh today BSL tomorrow. In: Deaf Worlds 15 (1),pp. 11–15.
Baris Sumengen, B. S. Manjunath “ multi-scale edge detection and image segmentation”, ece dept. UC, santa Barbara, USA,(National Conference)2005.
C. C. Lee, “Fuzzy logic in control systems: Fuzzy logic controller-Part-I and Part-II,”IEEE Trans. Syst., Man, Cybem., vol. 20, no. 2, pp. 404-435, 1990.
C. Vogler and D. Metaxas. “A framework for recognizing the simultaneous aspects of american sign language”. Computer Vision & Image Understanding, 81(3):358–384, March 2001.
Casacuberta, F., Vidal, E., 2004. Machine translation with inferred stochastic finite-state transducers. omputational Linguistics 30 (2),205–225.
Christopoulos, C., Bonvillian, J., 1985. “Sign language”. Journal of Communication Disorders 18, 1–20.
ECHO corpus: .
Engberg-Pedersen, E., 2003. From pointing to reference and predication: pointing signs, eyegaze, and head and body orientation in Danish Sign Language. In: Kita, Sotaro (Ed.), Pointing: Where Language, Culture, and Cognition Meet. Erlbaum, Mahwah, NJ,pp. 269– 292.
ESIGN project: .
G. Pajares, “A Wavelet-based Image Fusion Tutorial”, Pattern Recognition, Vol. 37, no.10, pp. 1855-1872, 2004.
Gaolin Fang and Wen Gao, “Large Vocabulary Contineous Sign languagre Recognition Based on Trnsition-Movement Models”, IEEE Transaction on Systems,MAN, and Cybernetics-Vol.37,No.1,January 2007,pp 1-9.
H. Ishibuchi, K. Nozaki, and H. Tanaka, “Distributed representation of fuzzy rules and its application to pattern classification,” Fuuy Sets and syst., vol.52, pp 21-32, 1992.
Indian Sign language, empowering the deaf, Indian Sign Language Database .
J.N. Ellinas, M.S. Sangriotis, “Stereo Image Compression Using Wavelet Coefficients Morphology”, Image and Vision Computing, Vol.22, no.2, pp. 281-290, 2004.
Koehn, P., 2004. Pharaoh: a beam search decoder for phrase-based statistical machine translation models. AMTA.
Kuhl, F. P. and Giardina, C. R., Elliptic Fourier Descriptors of a Closed Contour, CVGIP, 18,pp. 236–258, 1982
L. X. Wang and I. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Trans. Syst., Man, Cyben., vol. 22, no. 6, pp. 1414-1427, 1992.
Lily Rui, Liang, Carl G., Looney, “Competitive Fuzzy Edge Detection”, Applied Soft Computing, Vol.3, no.4, pp. 123-137, 2003.
Lin C. C. and Chellappa, R., Classification of Partial 2D Shapes using Fourier Descriptors,IEEE Trans. PAMI, 9(5), pp. 686–690, 1987.
Liu Cai, “a kind of advanced Sobel image edge detection algorithm”, Guizhou Industrial College Transaction (Natural Science Edition),2004, 33(5):77-79.
M. Sugeno and T. Yasukawa, “A fuzzy-logic-based approach to qualitative modeling,” IEEE Trans. Fuzzy Systems, vol. 1, no. 1, pp. 7-31, 1993.
M. Sugeno, “An introductory survey of fuzzy control,” Inform. Sci., vol. 36, pp. 59-83, 1985.
M.K.Bhuyan and P.K.Bora, “A Frame Work of Hand Gesture Recognition with Applications to Sigb Language”, Annual India Conference, IEEE, pp1-6.
Masataka, N. et al., 2006. Neural correlates for numerical processing in the manual mode. Journal of Deaf Studies and Deaf Education 11 (2), 144–152.
Ming-Hsuan Yang and Narendra Ahuja, “ Extraction of 2D Motion Trajectories and its aApplication to Hand Gesture Recognition”, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.24, No.8, August 2002,pp1061-1074.
Montiel, E., Aguado, A. S. and Zaluska, E., Fourier Series Expansion of Irregular Curves, Fractals, 5(1), pp. 105–199, 1997.
Nariman Habili, Cheng Chew Lim and Alireza Moini , ‘Segmentation Of The Face And Hands In Sign Language Video Sequences Using Color And Motion Cues’, IEEE Transactions on Circuits and Systems For Video Technology 2004 , Vol. 14, No. 8, , pp.1086 – 1097
Nyst, V., 2004. Verbs of motion in Adamorobe Sign Language. Poster. In:TISLR 8 Barcelona, September 30–October 2. Programme and Abstracts. (Internat. Conf. on Theoretical Issues in Sign Language Research; 8), pp. 127–129.
Och J., Ney. H., 2002. Discriminative training and maximum entropy models for statistical machine translation. In: Annual Meeting of the Ass. For Computational Linguistics (ACL), Philadelphia, PA, pp. 295–302.
Och, J., Ney, H., 2003. A systematic comparison of various alignment models. Computational Linguistics 29 (1), 19–51.
Persoon, E. and Fu, K.-S., Shape Description Using Fourier Descriptors, IEEE Trans. SMC, 3, pp. 170–179, 1977
Rini Akmeliawati, Melanie Po-Leen Ooi and Ye Chow Kuang , ‘Real-Time Malaysian Sign Language Translation Using Colour Segmentation and Neural Network’, IEEE on Instrumentation and Measurement Technology Conference Proceeding,Warsaw, Poland 2006 ,pp. 1-6.
Ruiduo Yang, Sudeep Sarkar, and Barbara Loeding. Enhanced level building algorithm to the movement epenthesis problem in sign language. In “CVPR”, MN, USA, June 2007.
Sumita, E., Akiba, Y., Doi, T., et al., 2003. A Corpus-Centered Approachto Spoken Language Translation. Conf. of the Europ. Chapter of the Ass. For Computational Linguistics (EACL), Budapest, Hungary, pp. 171–174.
T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Trans. Syst., Man, cybern., vol. 15, no. 1, pp. 116-132, 1985.
T.Starner and A.Pentland “Real-Time American Sign Language Recognition from video using Hidden Markov Models”,Technical Report, MIT Media laboratory Perceptual computing section, Technical Report number.375,1995.
U. R. Wrobel. Referenz in eb¨ardensprachen: Raum und Person. Forschungsberichte des Instituts f¨ur Phonetik und Sprachliche Kommunikation der Universit ¨at M¨unchen, 37:25– 50, 2001.
W. Stokoe, D. Casterline, and C. Croneberg. “A Dictionary of American Sign Language on Linguistic Principles.” Gallaudet College Press, Washington D.C., USA, 1965.
Yu Zhou and Xilin Chen, “Adaptive sign language recognition with Exemplar extraction and MAP/IVFS”, IEEE signal processing letters, Vol 17, No-3, March 2010, pp297-300.
Mr. E.Kiran Kumar
Dadi Institute of Engineering and Technology - India
Mr. S.R.C.Kishore
Pydah College of Engineering - India
Mr. P.V.V.Kishore
Andhra University - India
pvvkishore@gmail.com
Dr. P.Rajesh Kumar
Andhra University - India


CREATE AUTHOR ACCOUNT
 
LAUNCH YOUR SPECIAL ISSUE
View all special issues >>
 
PUBLICATION VIDEOS