Speech-impaired individuals face barriers in communication across education, healthcare & workplaces
Keywords:
Sign language recognition; Assistive technology; Speech impairment; MediaPipe; Hand landmarks; BiLSTM; Attention; Text-to-speech; Speech-to-text; Human–computer interaction.Abstract
Speech-impaired individuals face significant communication barriers in education, healthcare and workplace
settings, where interaction increasingly depends on spoken language and trained sign-language interpreters are
not always available. Existing assistive solutions are often hardware-dependent, expensive or limited to a fixed
vocabulary, which makes them impractical for everyday use in resource-constrained institutions. This paper
presents SignBridge, a lightweight, camera-based assistive communication system that recognises hand gestures
and signs from a standard webcam and converts them into readable text and synthesised speech in real time, while
converting spoken or typed replies into text for two-way communication. The system extracts hand and upper
body landmarks with MediaPipe Holistic, normalises them for position and scale, smooths them with a One-Euro
filter and enriches them with joint-angle and velocity features. A hybrid one-dimensional CNN, bidirectional
LSTM and temporal-attention network classifies sliding windows of landmarks, and a sign-spotting and temporal
voting stage turns the frame stream into stable sign decisions. Recognised signs are assembled into sentences and
spoken through offline text-to-speech, while the hearing participant replies through offline speech-to-text or the
keyboard. A context-specific dataset of 60 signs drawn from education, healthcare and workplace scenarios is
defined, with signer-independent evaluation. The proposed recogniser is expected to reach 94.6% accuracy and
a 94.3% macro F1-score with 1.18 M parameters and about 26 ms of processing per frame on a standard CPU,
enabling real-time use without special hardware










