Rehman Ullah, Khan (2025) A Dynamic Malaysian Sign Language Dataset for Sign Language Recognition and Translation. [Video] (In Press)
Full text not available from this repository. (Request a copy)Abstract
Sign languages all around the world are unique and diverse. Each sign language shows the differences in cultural nuances of its origin locale giving it is distinctive nature. Thus, despite the positive outcomes of sign language recognition and translation research that has been widely conducted worldwide, there are still notable limitations to each system which are mainly caused by data limitations. The sign language recognition and translation research in Malaysia especially has been set back by the limited size and nature of datasets available that are concurrent with current technological developments. The current datasets available for Malaysian Sign Language (BIM – Bahasa Isyarat Malaysia) are small and limited to fingerspelling of alphanumeric characters and several dynamic words and short phrases. However, given the continuous nature of the sign language communication, these data are not enough to properly train machine learning models to recognize and translate continuous real-world signs. Therefore, in order to address this issue, we introduce a dynamic BIM dataset which comprises of video, gloss, and translation data consisting of alphanumeric characters, dynamic words and short phrases, and continuous sentences. The dataset is split into two versions. The first version, BIM-SSD-V1 dataset comprises of 4,858 parallel video (RGB frames), gloss, and translation data while the second version, BIM-SSD-V2 dataset comprises of 3,143 parallel video (RGB frames), keypoints and gloss data for recognition purposes, and 4,900 parallel gloss and translation data for translation purposes. The raw videos are also available in the dataset. The dataset was developed and compiled with the help of the Deaf and Hard-of-Hearing community. This process also included the development of a Sign Language Module (translations for the video and gloss data) to assist in the development of the dataset. The image and video data were collected using smartphones and the respective gloss annotations for the data were prepared with the help of a BIM expert. The data collection process and participant selection were guided by criteria that ensured data diversity and used a setup similar to real-world scenarios. These criteria and sample differences were set to help models trained on the dataset adapt to different individuals, signing styles, backgrounds, attire, and other contextual variations. The total number of participants involved in the data collection process was four. There are also four samples for every character, word, phrase or sentence in the Sign Language Module. The dataset can mainly be reused by researchers who would like to conduct sign language recognition and translation research using the Sign-to-Gloss-to-Text framework. However, the dataset is not limited to only one framework and can be used for other sign language recognition and translation research frameworks accordingly.
| Item Type: | Video |
|---|---|
| Subjects: | P Language and Literature > P Philology. Linguistics P Language and Literature > PL Languages and literatures of Eastern Asia, Africa, Oceania T Technology > TT Handicrafts Arts and crafts |
| Divisions: | Academic Faculties, Institutes and Centres > Faculty of Cognitive Sciences and Human Development Faculties, Institutes, Centres > Faculty of Cognitive Sciences and Human Development Academic Faculties, Institutes and Centres > Faculty of Cognitive Sciences and Human Development |
| Depositing User: | Khan |
| Date Deposited: | 21 Oct 2025 23:59 |
| Last Modified: | 21 Oct 2025 23:59 |
| URI: | http://ir.unimas.my/id/eprint/49998 |
Actions (For repository members only: login required)
 |
View Item |
