Real-time Bahasa Isyarat Malaysia Recognition System for Greeting Gestures

Phei Chin, Lim and Sherene Saw, Tyng Xin and Suriati Khartini, Jali and Johari, Abdullah (2025) Real-time Bahasa Isyarat Malaysia Recognition System for Greeting Gestures. In: 2025 14th International Conference on Information Technology in Asia (CITA), 5-6 Aug. 2025, Kota Samarahan, Sarawak, Malaysia..

PDF Real-time bim.pdf Download (252kB)

Abstract

Addressing the communication gap between the Deaf and hard-of-hearing and the hearing remains a significant challenge. Most systems focus on static sign language, primarily through fingerspelling that spells out words, which can be inefficient for conveying more complex information. This work introduces a real-time Bahasa Isyarat Malaysia recognition system specifically designed to interpret ten common greeting gestures. It aims to enhance communication efficiency, enabling users to express themselves more effectively through dynamic hand motions rather than relying on static representations. To achieve real-time sign language recognition, VGG16 serves as the foundation architecture that is complemented by MediaPipe for accurate hand detection. The dataset used was collected from 30 participants from the Penang Deaf Association. Modelling experimentation resulted in 80.57% for its average validation accuracy and 71.79% for its average testing accuracy. As for realtime testing with 30 testers, average accuracy of 69.44% is achieved. The model still faces several limitations such as latency and computation constraints which still need to be improved. Future work will focus on enhancing robustness and accuracy by incorporating data from diverse user groups to better represent the BIM sign language users.

Actions (For repository members only: login required)

View Item