Speech

157 results found

Sort

Masked audio text encoders are effective few-shot rescorers

Jinglun Cai, Monica Sunkara, Xilai Li, Anshu Bhatia, Xiao Pan, Sravan Bodapati

ICLR 2023 Workshop on Successful Domain Generalization

2023

Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition (ASR) systems. In this work, we propose Masked Audio Text Encoder (MATE), a multi-modal masked language model rescorer which incorporates acoustic representations into the input space of MLM. We adopt contrastive learning for effectively aligning the modalities by learning shared representations

Machine learning
Two-pass endpoint detection for speech recognition

Anirudh Raju, Aparna Khare, Di He, Ilya Sklyar, Long Chen, Sam Alptekin, Viet Anh Tranh, Zhe Zhang, Colin Vaz, Venkatesh Ravichandran, Roland Maas, Ariya Rastrow

ASRU 2023

2023

Endpoint (EP) detection is a key component of far-field speech recognition systems that assist the user through voice commands. The endpoint detector has to trade-off between accuracy and latency, since waiting longer reduces the cases of users being cut-off early. We propose a novel two-pass solution for endpointing, where the utterance endpoint detected from a first pass endpointer is verified by a 2nd-pass

Conversational AI
Lookahead when it matters: Adaptive non-causal transformers for streaming neural transducers

Grant Strimel, Yi Xie, Brian King, Martin Radfar, Ariya Rastrow, Thanasis Mouchtaris

ICML 2023

2023

Streaming speech recognition architectures are employed for low-latency, real-time applications. Such architectures are often characterized by their causality. Causal architectures emit tokens at each frame, relying only on current and past signal, while non-causal models are exposed to a window of future frames at each step to increase predictive accuracy. This dichotomy amounts to a trade-off for real-time

Related: How dynamic lookahead improves speech recognition

Conversational AI
Modelling low-resource accents without accent-specific TTS frontend

Georgi Tinchev, Marta Czarnowska, Kamil Deja, Kayoko Yanagisawa, Marius Cotescu

ICASSP 2023

2023

This work focuses on modelling a speaker’s accent that does not have a dedicated text-to-speech (TTS) frontend, includ-ing a grapheme-to-phoneme (G2P) module. Prior work on modelling accents assumes a phonetic transcription is avail-able for the target accent, which might not be the case for low-resource, regional accents. In our work, we propose an approach whereby we first augment the target accent data

Related: Modeling low-resource accents without accent-specific TTS front end

Conversational AI
Phonetic RNN-transducer for mispronunciation diagnosis

Daniel Zhang, Soumya Saha, Sarah Campbell

ICASSP 2023

2023

Non-autoregressive models and, in particular, Connectionist Temporal Classification (CTC) models have been the most popular approaches towards mispronunciation detection and diagnosis (MDD) task. In this paper, we identify two important knowledge gaps in MDD that have not been well studied in existing MDD research. First, CTC-based MDD models often assume conditional independence in the predicted phonemes

Related: Pronunciation detection for Alexa’s new English-learning experience

Machine learning
Cross-utterance ASR rescoring with graph-based label propagation

Srinath Tankasala, Long Chen, Andreas Stolcke, Anirudh Raju, Shally Deng, Chander Chandak, Aparna Khare, Roland Maas, Venkatesh Ravichandran

ICASSP 2023

2023

We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity. In contrast to conventional neural language model (LM) based ASR rescoring/reranking models, our approach focuses on acoustic information and conducts the rescoring collaboratively among utterances, instead of individually. Experiments on the VCTK dataset

Related: More-inclusive speech recognition with cross-utterance rescoring

Conversational AI
Dialog act guided contextual adapter for personalized speech recognition

Feng-Ju (Claire) Chang, Thejaswi Muniyappa, Kanthashree Mysore Sathyendra, Kai Wei, Grant Strimel, Ross McGowan

ICASSP 2023

2023

Personalization in multi-turn dialogs has been a long standing challenge for end-to-end automatic speech recognition (E2E ASR) models. Recent work on contextual adapters has tackled rare word recognition using user catalogs. This adaptation, however, does not incorporate an important cue, the dialog act, which is available in a multi-turn dialog scenario. In this work, we propose a dialog act guided contextual

Conversational AI
PROCTER: Pronunciation-aware contextual adapter for personalized speech recognition in neural transducers

Rahul Pandey, Roger Ren, Qi Luo, Jing Liu, Ariya Rastrow, Ankur Gandhe, Denis Filimonov, Grant Strimel, Andreas Stolcke, Ivan Bulyko

ICASSP 2023

2023

End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names and places. Rare words often have non-trivial pronunciations, and in such cases, human knowledge in the form of a pronunciation lexicon can be useful. We propose a PROnunCiation-aware conTextual adaptER (PROCTER) that dynamically

Conversational AI
SumREN: Summarizing reported speech about events in news

Revanth Gangi Reddy, Heba Elfardy, Hou Pong Chan, Kevin Small, Heng Ji

AAAI 2023

2023

A primary objective of news articles is to establish the factual record for an event, frequently achieved by conveying both the details of the specified event (i.e., the 5 Ws; Who, What, Where, When and Why regarding the event) and how people reacted to it (i.e., reported statements). However, existing work on news summarization almost exclusively focuses on the event details. In this work, we propose the

Conversational AI
Multiscale audio spectrogram transformer for efficient audio classification

Wentao Zhu, Mohamed Omar

ICASSP 2023

2023

Audio event has a hierarchical architecture in both time and frequency and can be grouped together to construct more abstract semantic audio classes. In this work, we develop a multiscale audio spectrogram Transformer (MAST) that employs hierarchical representation learning for efficient audio classification. Specifically, MAST employs one-dimensional (and two-dimensional) pooling operators along the time

Conversational AI

...

Speech

Work with us