-
SIGIR 20232023In this paper we introduce SimTDE, a simple knowledge distillation framework to compress sentence embeddings transformer models with minimal performance loss and significant size and latency reduction. SimTDE effectively distills large and small transformers via a compact token embedding block and a shallow encoding block, connected with a projection layer, relaxing dimension match requirement. SimTDE simplifies
-
ICASSP 20232023Recently, there has been an increasing interest in unifying streaming and non-streaming speech recognition models to reduce development, training and deployment cost. The best-known approaches rely on either window-based or dynamic chunk-based attention strategy and causal convolutions to minimize the degradation due to streaming. However, the performance gap still remains relatively large between non-streaming
-
ICASSP 20232023Attention-based contextual biasing approaches have shown significant improvements in the recognition of generic and/or personal rare-words in End-to-End Automatic Speech Recognition (E2E ASR) systems like neural transducers. These approaches employ cross attention to bias the model towards specific contextual entities injected as bias-phrases to the model. Prior approaches typically relied on subword encoders
-
ICASSP 20232023End-to-end ASR models trained on large amount of data tend to be implicitly biased towards language semantics of the training data. Internal language model estimation (ILME) has been proposed to mitigate this bias for autoregressive models such as attention-based encoder-decoder and RNN-T. Typically, ILME is performed by modularizing the acoustic and language components of the model architecture, and eliminating
-
ICLR 2023 Workshop on Deep Learning for Code (DL4C)2023Code understanding and generation require learning the mapping between human and programming languages. As human and programming languages are different in vocabulary, semantic, and, syntax, it is challenging for an autoregressive model to generate a sequence of tokens that is both semantically (i.e., carry the right meaning) and syntactically correct (i.e., in the right sequence order). Inspired by this
Related content
-
November 09, 2022At Amazon, he develops machine learning models to help keeping Amazon stores safe and trustworthy for customers and selling partners.
-
October 13, 2022Fellowships will provide support to pursue research in the fields of artificial intelligence and robotics.
-
October 11, 2022Two doctorate students and five Virginia Tech professors will receive funding to conduct research.
-
October 06, 2022The phrase launches a feature built to help customers navigate an increasingly complex and diverse world of content.
-
October 05, 2022Inaugural recipients named as part of the JHU + Amazon Initiative for Interactive AI (AI2AI).
-
October 05, 2022Dataset that requires question-answering models to look up multiple facts and perform comparisons bridges a significant gap in the field.