Image classification

Low-shot image classification using mixture of experts

ICASSP 2025

2025

We propose a low-shot image classification method called LIMO, which can train an accurate image classification model under conditions of acute data scarcity. LIMO uniquely assembles existing knowledge from a set of diverse models and builds a novel mixture of experts architecture for low-shot image classification. LIMO’s architecture introduces minimal number of new model parameters, such that the added

Computer vision

Data pruning via separability, integrity, and model uncertainty-aware importance sampling

Steven Grosz, Rui Zhao, Rajeev Ranjan, Hongcheng Wang, Manoj Aggarwal, Gérard Medioni, Anil Jain

ICPR 2024

2024

This paper improves upon existing data pruning methods for image classification by introducing a novel pruning metric and pruning procedure based on importance sampling. The proposed pruning metric explicitly accounts for data separability, data integrity, and model uncertainty, while the sampling procedure is adaptive to the pruning ratio and considers both intra-class and inter-class separation to further

Computer vision

Building geospatial foundation models via continual pretraining

Matías Mendieta, Boran Han

February 07, 2024

Parallel grids, each containing four satellite images. The four at left have lower feature resolution and more homogenous coloring; the four at right are more detailed.

New approach enables sustainable machine learning for remote-sensing applications.

Computer vision

Deep layers beware: Unraveling the surprising benefits of JPEG compression for image classification pre-processing

Guru Nayak, Gerald Friedland

IEEE ISM 2023

2023

In this paper, we explore the intriguing effects of JPEG compression as a pre-processing technique for image classification tasks. Building upon the findings of a previous study by Friedland et al., which demonstrated that substantial JPEG compression does not significantly degrade classification accuracy, we investigate the potential benefits and limitations of this approach when applied to various classifiers

Computer vision

Prompt pre-training with twenty-thousand classes for open-vocabulary visual recognition

Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alex Smola, Xu Sun

NeurIPS 2023

2023

This work proposes POMP, a prompt pre-training method for vision-language models. Being memory and computation efficient, POMP enables the learned prompt to condense semantic information for a rich set of visual concepts with over twenty-thousand classes. Once pre-trained, the prompt with a strong transferable ability can be directly plugged into a variety of visual recognition tasks including image classification

Computer vision

Learning expressive prompting with residuals for vision transformers

Rajshekhar Das, Yonatan Dukler, Avinash Ravichandran, Ashwin Swaminathan

CVPR 2023

2023

Prompt learning is an efficient approach to adapt transformers by inserting learnable set of parameters into the input and intermediate representations of a pre-trained model. In this work, we present Expressive Prompts with Residuals (EXPRES) which modifies the prompt learning paradigm specifically for effective adaptation of vision transformers (ViT). Our method constructs downstream representations via

Computer vision

FashionNTM: Multi-turn fashion image retrieval via cascaded memory

Anwesan Pal, Sahil Wadhwa, Ayush Jaiswal, Xu Zhang, Yue (Rex) Wu, Rakesh Chada, Pradeep Natarajan, Henrik I. Christensen

ICCV 2023

2023

Multi-turn textual feedback-based fashion image retrieval focuses on a real-world setting, where users can iteratively provide information to refine retrieval results until they find an item that fits all their requirements. In this work, we present a novel memory-based method, called FashionNTM, for such a multi-turn system. Our framework incorporates a new Cascaded Memory Neural Turing Machine (CM-NTM

Computer vision

LD-ZNet: A latent diffusion approach for text-based image segmentation

Koutilya PNVR, Bharat Singh, Pallabi Ghosh, Behjat Siddiquie, David Jacobs

ICCV 2023

2023

Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text description

Computer vision

Prompt pre-training with twenty-thousand classes for open-vocabulary visual recognition

Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alex Smola, Xu Sun

2023

We introduce a prompt pre-training method POMP, which fisrt enables prompt learning on large-scale datasets like ImageNet-21K with over twenty-thousand classes. POMP is memory and computation efficient. Compared with previous methods like CoOp, it achieves comparable accuracy on ImageNet-1K with only 19% GPU memory and 50% training time. POMP achieves new SOTAs on various open-vocabulary visual recognition

Computer vision

Large scale generative multimodal attribute extraction for e-commerce attributes

Anant Khandelwal, Happy Mittal, Shreyas Sunil Kulkarni, Deepak Gupta

ACL 2023

2023

E-commerce websites (e.g. Amazon) have a plethora of structured and unstructured information (text and images) present on the product pages. Sellers often either don’t label or mislabel values of the attributes (e.g. color, size etc.) for their products. Automatically identifying these attribute values from an eCommerce product page that contains both text and images is a challenging task, especially when

Computer vision

Image classification

Work with us