My master's thesis, "Multi-dimensional Performance Evaluation of Streaming ASR Techniques with Whisper," tackles a gap in how streaming speech recognition gets evaluated: standard accuracy metrics like Word Error Rate were built for batch transcription of complete utterances, and don't capture what actually matters in real-time use — latency and output stability as a transcript is built up incrementally.
The thesis proposes a more holistic evaluation framework, including a novel strategy for aligning partial (in-progress) words so that alignment-dependent metrics can be computed even on incomplete output. I used this framework to evaluate Melvin, a Whisper-based streaming ASR system, across multiple model sizes — measuring how accuracy and latency trade off, and identifying instabilities that require dedicated algorithmic handling as a transcript refines over time.
The motivation is practical: streaming ASR underpins real-time AI agents and assistive technology for hearing-impaired users, where both correctness and responsiveness matter simultaneously — and where today's metrics don't tell you enough about either.
Python