Generative AI for Sign Language Recognition and Translation

Sign languages differ fundamentally from spoken and written languages, with their own grammar, syntax, and three-dimensional expression involving hand gestures, facial expressions, body movements, and spatial relationships. These non-manual elements are crucial in conveying grammatical structures, nuances, and emotional tones, making sign languages uniquely complex communication systems.

This book provides a comprehensive foundation for understanding the linguistic structures of sign languages and explores the application of artificial intelligence (AI) techniques – ranging from classical machine learning to deep learning and generative AI – for developing effective sign language translation systems. It offers an end-to-end overview, covering linguistic fundamentals, available datasets, text-to-sign and speech-to-sign translation, vision-based sign recognition, pose estimation, and video-based sign language generation. Dedicated chapters focus on model architectures, dataset curation strategies, evaluation metrics, benchmarking tools, and human-centered design approaches for accessible communication systems. Ethical considerations and responsible AI practices are also discussed to promote the development of inclusive and equitable sign language technologies. Complemented by Python code examples, downloadable resources, and implementation insights, this book serves as a practical guide for researchers, engineers, students, and technology professionals aiming to develop AI-powered sign language systems. The multidisciplinary content also supports linguists, accessibility advocates, and application developers working on inclusive language technologies. With its broad coverage and practical orientation, this book is suited to academic and industry professionals in artificial intelligence, computer vision, natural language processing, human-computer interaction, speech technology, and accessibility research, as well as students and early-career researchers seeking a well-rounded introduction to AI-driven sign language translation. By bridging AI methodologies with real-world sign language applications, this book promotes the development of inclusive AI systems supporting communication accessibility for diverse populations.

Author Biography

Elakkiya Rajasekar is an assistant professor in Computer Science at Department of Computer Science, Birla Institute of Technology and Science Pilani, Dubai Campus. She specializes in generative AI and computer vision, applied to language technologies, healthcare, assistive systems, and other interdisciplinary domains. She has led 16 extra-mural funded research projects supported by agencies including DST-RFBR, DRDO, Royal Society UK, and others. She serves as chair of the Dubai ACM-W Chapter.