Evaluating the Efficiency of CBAM-ResNet by using Malaysian Sign Language (BIM)

Wong, Woei Sheng (2020) Evaluating the Efficiency of CBAM-ResNet by using Malaysian Sign Language (BIM). [Final Year Project Report] (Unpublished)

[img] PDF
Wong Woei Sheng - 24 pgs.pdf

Download (662kB)
[img] PDF (Please get the password from TECHNICAL & DIGITIZATION MANAGEMENT UNIT, ext: 082-583913/ 082-583914)
Wong Woei Sheng (full permission).pdf
Restricted to Registered users only

Download (3MB)


Malaysian Sign Language has been widely used by deaf-mutes in our nation for communication since created in 1998. However, most Malaysians do not understand this sign language and have difficulties in communicating with deaf-mutes. This had discouraged deafmutes and made them felt helpless when not being understood. Issues such as unfair treatment in education and work sometimes happened. As machine learning and computer vision domain developed, these technologies provided alternates in bridging the communication gap between deaf-mutes and others. This research introduced a competitive CNNs based neural network, namely CBAM-ResNet to prove its efficiency in Malaysian Sign language recognition. A dataset consists of 2071 videos for 19 dynamic signs was built to provide instances for network training. Two different experiments were conducted for static and dynamic signs using CBAM-2DResNet and CBAM-3DResNet respectively by two CBAM integration methods, which known as ‘Within blocks’ and ‘Before classifier’. Performance metrics such as accuracy, loss, precision, recall, F1-score, confusion matrix, and training time were taken to evaluate models’ efficiency. The results showed that all CBAM-ResNet models implemented had good performances in image and video recognition tasks, with recognition rates of over 90 % with little variation. CBAM-ResNet ‘Before Classifier’ is more efficient than ‘Within blocks’ models of CBAM-ResNet in terms of various performance metrics with shorter training time required. The best trained CBAM-2DResNet was chosen to build a realtime sign recognition application based on the image recognition technique. All experiment results indicated the CBAM-ResNet ‘Before classifier’ effectiveness in recognizing Malaysian Sign Language and it’s worth of future research.

Item Type: Final Year Project Report
Additional Information: Project Report (BSc.) -- Universiti Malaysia Sarawak, 2020.
Uncontrolled Keywords: CBAM-ResNet, Malaysian Sign Language, Within blocks, Before classifier, efficiency evaluation.
Subjects: H Social Sciences > H Social Sciences (General)
Divisions: Academic Faculties, Institutes and Centres > Faculty of Cognitive Sciences and Human Development
Depositing User: Gani
Date Deposited: 02 Dec 2020 06:36
Last Modified: 02 Dec 2020 06:36
URI: http://ir.unimas.my/id/eprint/33172

Actions (For repository members only: login required)

View Item View Item