-
In-Context Learning (ICL) has enabled Large Language Models (LLMs) to excel as generalpurpose models in zero and few-shot task settings. However, since LLMs are often not trained on the downstream tasks, they lack crucial contextual knowledge from the data distributions, which limits their task adaptability. This paper explores using data priors to automatically customize prompts in ICL. We extract these
-
2024Pre-trained language models, trained on largescale corpora, demonstrate strong generalizability across various NLP tasks. Finetuning these models for specific tasks typically involves updating all parameters, which is resource-intensive. Parameter-efficient finetuning (PEFT) methods, such as the popular LoRA family, introduce low-rank matrices to learn only a few parameters efficiently. However, during
-
In-context learning (ICL) is a powerful paradigm where large language models (LLMs) benefit from task demonstrations added to the prompt. Yet, selecting optimal demonstrations is not trivial, especially for complex or multi-modal tasks where input and output distributions differ. We hypothesize that forming taskspecific representations of the input is key. In this paper, we propose a method to align representations
-
2024Hierarchical Text Classification (HTC) is a sub-class of multi-label classification. It is challenging because the hierarchy typically has a large number of diverse topics. Existing methods for HTC fall within two categories, local methods (a classifier for each level, node, or parent) or global methods (a single classifier for everything). Local methods are computationally expensive, whereas global methods
-
2024Data is a crucial element in large language model (LLM) alignment. Recent studies have explored using LLMs for efficient data collection. However, LLM-generated data often suffers from quality issues, with underrepresented or absent aspects and low-quality data-points. To address these problems, we propose DATA ADVISOR, an enhanced LLM-based method for generating data that takes into account the characteristics
Related content
-
October 29, 2020Watch the replay of Shehzad Mevawalla's Interspeech 2020 keynote talk.
-
October 29, 2020Watch the replay of the Interspeech 2020 industry forum session.
-
October 28, 2020Watch as four Amazon Alexa scientists talk about current state, new developments, and recent announcements surrounding advancements in Alexa speech technologies.
-
October 28, 2020Knowledge distillation technique for shrinking neural networks yields relative performance increases of up to 122%.
-
October 22, 2020Director of speech recognition Shehzad Mevawalla highlights recent advances in on-device processing, speaker ID, and semi-supervised learning.