REAL-TIME MALAYSIAN SIGN LANGUAGE SENTENCE-LEVEL GESTURE RECOGNITION ON ANDROID USING GRU-CTC MODEL
| dc.contributor.author | TIANG YU EN | |
| dc.date.accessioned | 2026-04-27T02:16:01Z | |
| dc.date.issued | 2025 | |
| dc.description | The unique structure of sign language sentences, relying on transitions between gestures rather than isolated words, renders traditional word-based or fingerspelling methods inefficient for natural communication. Despite the growing need for accessibility tools, a significant gap exists in real-time recognition systems for Malaysian Sign Language (BIM), particularly at the sentence level, and in the availability of portable mobile translator applications. This paper presents a vision-based gesture recognition system for real-time, sentence-level BIM translation using a Gated Recurrent Unit (GRU) combined with Connectionist Temporal Classification (CTC) to handle temporal dependencies and variable-length sequences. The model was evaluated using 5-fold cross-validation, achieving an average WER of 0.0275 with a standard deviation of 0.0173, and a final test WER of 0.0132 after full-dataset training, confirming its strong generalization capability. Integrated into an Android mobile application, the system was tested in real-time on unseen users, achieving a 69% accuracy and average WER at 0.30, highlighting its potential for practical deployment while also revealing challenges related to inter-user variation and gesture similarity. This work contributes toward bridging the accessibility gap for the Malaysian Deaf community through a portable, vision-based real-time translation solution. | |
| dc.identifier.uri | https://scholarhub.unimas.my/handle/123456789/484 | |
| dc.language.iso | English | |
| dc.publisher | UNIVERSITI MALAYSIA SARAWAK | |
| dc.relation.ispartofseries | Faculty of Computer Science and Information Technology | |
| dc.title | REAL-TIME MALAYSIAN SIGN LANGUAGE SENTENCE-LEVEL GESTURE RECOGNITION ON ANDROID USING GRU-CTC MODEL | |
| dc.type | Final Year Project |