Che Wan Shamsul Bahri, Che Wan Ahmad and Syed Arbaz, Ahmed and Khirulnizam, Abd Rahman and Syarbaini, Ahmad and Mokmin, Basri and Syahrul Nizam, Junaini and Mohd Shahrul Nizam, Mohd Danuri (2025) Evaluation of an Ai-Powered Chatbot for University Student Support: A Framework-Based Analysis. Applied Mathematics and Computational Intelligence (AMCI), 14 (4). pp. 109-132. ISSN 2289-1323
![[img]](http://ir.unimas.my/style/images/fileicons/text.png) |
PDF
Evaluation of an.pdf Download (222kB) |
Abstract
This study presents a comprehensive evaluation of an AI-powered chatbot designed to assist university students with their inquiries. By employing a structured and rigorous framework, the chatbot's performance is assessed based on diverse feedback from participants with both IT and non-IT backgrounds. The evaluation hones in on four critical dimensions: response relevance, understandability, conversation outcomes, and interaction efficiency. Participants were asked ten questions—nine rated on a scale of 1 to 5 and one open-ended—to offer detailed insights into the chatbot’s functionality. Their feedback provided valuable information on the chatbot’s effectiveness in addressing student queries, ease of use, and its potential to enhance the traditional student support system. The findings reveal key strengths, such as improved response accuracy, alongside areas needing further refinement. This study offers practical insights for developers and decision-makers aiming to optimize AI-powered chatbot solutions for enhanced student support, ultimately contributing to a more streamlined and responsive university environment.
| Item Type: | Article |
|---|---|
| Additional Information: | Information, Communication and Creative Technology |
| Uncontrolled Keywords: | Artificial Intelligence (AI), Evaluation. Chatbots, Universities. |
| Subjects: | Q Science > QA Mathematics Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software |
| Divisions: | Academic Faculties, Institutes and Centres > Faculty of Computer Science and Information Technology Faculties, Institutes, Centres > Faculty of Computer Science and Information Technology Academic Faculties, Institutes and Centres > Faculty of Computer Science and Information Technology |
| Depositing User: | Junaini |
| Date Deposited: | 03 Dec 2025 04:51 |
| Last Modified: | 03 Dec 2025 04:51 |
| URI: | http://ir.unimas.my/id/eprint/50678 |
Actions (For repository members only: login required)
 |
View Item |
