Evaluating the Adversarial Vulnerability of YOLOv11 for Indonesian Sign Language (BISINDO) Recognition

Indonesian Sign Language (BISINDO) is a common form of communication used by the deaf community in Indonesia. Various deep learning models have been successfully utilized to recognize BISINDO gestures, including object detection models such as YOLO. However, these models remain vulnerable to adversarial attacks. This study aims to evaluate the robustness of the YOLOv11 model against three types of adversarial attacks, namely the Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and Basic Iterative Method (BIM). We utilize the BISINDO sign image dataset, which consists of 26 alphabet gesture classes (A to Z). The YOLOv11 model achieved a baseline performance of 0.945 mAP on clean data. The model was tested on adversarial images, and the results show a significant mAP drop of 49% under PGD and BIM attacks. In contrast, FGSM only reduces mAP by about 13%. This study provides an initial foundation for further research on model robustness and strength against adversarial attacks in the context of sign language recognition. Furthermore, the findings highlight the importance of developing robust BISINDO recognition systems to ensure reliable communication accessibility in real-world applications.

Recommended citation: Saputra, M. A., Heryadi, Y., Sonata, I., Wulandhari, L. A., & Girsang, A. S. (2025, October). Evaluating the Adversarial Vulnerability of YOLOv11 for Indonesian Sign Language (BISINDO) Recognition. In 2025 8th International Conference on Informatics and Computational Sciences (ICICoS) (pp. 241-246). IEEE.
Download Paper | Download Slides