Active learning

Adaptive batch sizes for active learning: A probabilistic numerics approach

Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen, Xingchen Wan, Vu Nguyen, Harald Oberhauser, Michael A. Osborne

AISTATS 2024

2024

Active learning parallelization is widely used, but typically relies on fixing the batch size throughout experimentation. This fixed ap-proach is inefficient because of a dynamic trade-off between cost and speed—larger batches are more costly, smaller batches lead to slower wall-clock run-times—and the trade-off may change over the run (larger batches are often preferable earlier). To address this trade-off

Machine learning

Leveraging confidence models for identifying challenging data subgroups in speech models

Alkis Koudounas, Eliana Pastor, Vittorio Mazzia, Manuel Giollo, Thomas Gueudre, Elisa Reale, Giuseppe Attanasio, Luca Cagliero, Sandro Cumani, Luca de Alfaro, Elena Baralis, Daniele Amberti

ICASSP 2024

2024

State-of-the-art speech models may exhibit suboptimal performance in specific population subgroups. Detecting these challenging subgroups is crucial to enhance model robustness and fairness. Traditional methods for subgroup identification typically rely on demographic information such as age, gender, and origin. However, collecting such sensitive data at deployment time can be impractical or unfeasible

Conversational AI

Active learning for iterative offline reinforcement learning

Lan Zhang, Luigi Tedesco, Pankaj Rajak, Youcef Zemmouri, Hakan Brunzell

NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World

2023

Offline Reinforcement Learning (RL) has emerged as a promising approach to address real-world challenges where online interactions with the environment are limited, risky, or costly. Although, recent advancements produce high quality policies from offline data, currently, there is no systematic methodology to continue to improve them without resorting to online fine-tuning. This paper proposes to repurpose

Machine learning

TAAL: Target-aware active learning

Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, Sunny Dasgupta

KDD 2023 Workshop on e-Commerce and NLP (ECNLP 6)

2023

Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy with fewer labels compared to random labeling. However, in an industrial setting where we often have class-level business targets to achieve (e.g., 95% recall at 95% precision for each class), active learning techniques continue to acquire labels for classes that have already met their targets

Machine learning

Small is the new big: Pre-finetuned compact models are better for asynchronous active learning

Dantong Liu, Kaushik Pavani, Sunny Dasgupta

ACL 2023 Workshop on SustaiNLP

2023

We examine the effects of model size and pre-finetuning in an active learning setting where classifiers are trained from scratch on 14 binary and 3 multi-class text classification tasks. We make an important observation that, in realistic active learning settings, where the human annotator and the active learning system operate in asynchronous mode, a compact pre-finetuned 1-layer transformer model with

Machine learning

Data-efficient continual learning in Alexa

Pradeep Natarajan

December 14, 2022

EMNLP papers examine constrained generation of rewrite candidates and automatic selection of information-rich training data.

Conversational AI

Uncertainty-based active learning for reading comprehension

Jing Wang, Jie Shen, Xiaofei Ma, Andrew O. Arnold

Transactions on Machine Learning Research

2022

Recent years have witnessed a surge of successful applications of machine reading comprehension. Of central importance to these tasks is the availability of massive amount of labeled data, which facilitates training of large-scale neural networks. However, in many real-world problems, annotated data are expensive to gather not only because of time cost and budget, but also of certain domain-specific restrictions

Machine learning

ALLIE: Active learning on large-scale imbalanced graphs

Limeng Cui, Xianfeng Tang, Sumeet Katariya, Nikhil Rao, Pallav Agrawal, Karthik Subbian, Dongwon Lee

The Web Conference 2022

2022

Human labeling is time-consuming and costly. This problem is further exacerbated in extremely imbalanced class label scenarios, such as detecting fraudsters in online websites. Active learning selects the most relevant example for human labelers to improve the model performance at a lower cost. However, existing methods for active learning for graph data often assumes that both data and label distributions

Machine learning

Amazon's annual machine learning conference featured presentations from thought leaders within academia

Staff writer

October 26, 2021

Amazon Machine Learning Conference highlights use of machine learning across the company’s businesses and its increasing value to customers.

Machine learning

Towards robust episodic meta-learning

Beyza Ermis, Giovanni Zappella, Cédric Archambeau

UAI 2021

2021

Meta-learning learns across historical tasks with the goal to discover a representation from which it is easy to adapt to unseen tasks. Episodic meta-learning attempts to simulate a realistic setting by generating a set of small artificial tasks from a larger set of training tasks for meta-training and proceeds in a similar fashion for meta-testing. However, this (meta-)learning paradigm has recently been

Machine learning

Active learning

Work with us