3D modeling / imaging

A metric-driven approach to conformer layer pruning for efficient ASR inference

Dhanush Bekal, Karthik Gopalakrishnan, Karel Mundnich, Srikanth Ronanki, Sravan Bodapati, Katrin Kirchhoff

Interspeech 2023

2023

Conformer-based end-to-end automatic speech recognition (ASR) models have gained popularity in recent years due to their exceptional performance at scale. However, there are significant computation, memory and latency costs associated with running inference on such models. With the aim of mitigating these issues, we evaluate the efficacy of pruning Conformer layers while fine-tuning only on 20% of the data

Conversational AI

Online heavy-tailed change-point detection

Abishek Sankararaman, Balakrishnan (Murali) Narayanaswamy

UAI 2023

2023

We study algorithms for online change-point detection (OCPD), where samples that are potentially heavy-tailed, are presented one at a time and a change in the underlying mean must be detected as early as possible. We present an algorithm based on clipped Stochastic Gradient Descent (SGD), that works even if we only assume that the second moment of the data generating process is bounded. We derive guarantees

Machine learning

Teaching a robot where to park: A scalable crowdsourcing approach

De'Aira Bryant, Tiago Etiene, Ayanna Howard, William D. Smart, Dylan F. Glas

IEEE RO-MAN 2023

2023

For social robots to successfully integrate into daily life in home environments, they will need reliable models of the way people perceive and use space in the home. This paper explores the problem of obtaining annotated training data at scale for subjective judgments about spatial locations. Focusing on the use case of identifying good and bad parking spots for a social robot operating in a home environment

Robotics

A coordinate-regression-based deep learning model for catheter detection during structural heart interventions

Mahdie Aghasizade, Amir Kiyoumarsioskouei, Sara Hashemi, Matin Torabinia , Alexandre Caprio, Muaz Rashid, Yi Xiang, Huzefa Rangwala, Tianyu Ma, Benjamin Lee, Alan Wang, Mert Sabuncu, S. Chiu Wong, Bobak Mosadegh

MDPI Applied Sciences

2023

With a growing geriatric population estimated to triple by 2050, minimally invasive procedures that are image-guided are becoming both more popular and necessary for treating a variety of diseases. To lower the learning curve for new procedures, it is necessary to develop better guidance systems and methods to analyze procedure performance. Since fluoroscopy remains the primary mode of visualizations, the

Computer vision

Cloud-based hybrid simulation model for optimizing warehouse yard operations

Farhan Mohammed, Pascalin Ngoko, Farouq Halawa, Raashid Mohammed

Winter Simulation Conference 2023

2023

Fulfillment centers in the E-commerce industry are highly complex systems that houses inventory and fulfill customer orders. One of the key processes at these centers involves translating customer demands into trucks and yard operations. Truck yards with operational issues can create delays in customer orders. In this paper, we show how a scalable cloud-based hybrid simulation model is used to improve yard

Cloud and systems

Shape of You: Precise 3D shape estimations for diverse body types

Rohan Sarkar, Achal Dave, Gérard Medioni, Ben Biggs

CVPR 2023 Workshop on Computer Vision for Fashion, Art, and Design

2023

This paper presents Shape of You (SoY), an approach to improve the accuracy of 3D body shape estimation for vision-based clothing recommendation systems. While existing methods have successfully estimated 3D poses, there remains a lack of work in precise shape estimation, particularly for diverse human bodies. To address this gap, we propose two loss functions that can be readily integrated into parametric

Computer vision

Selective structured state-spaces for long-form video understanding

Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, Raffay Hamid

CVPR 2023

2023

Effective modeling of complex spatiotemporal dependencies in long-form videos remains an open problem. The recently proposed Structured State-Space Sequence (S4) model with its linear complexity offers a promising direction in this space. However, we demonstrate that treating all image- tokens equally as done by S4 model can adversely affect its efficiency and accuracy. To address this limitation, we present

Machine learning

FlexNeRF: Photorealistic free-viewpoint rendering of moving humans from sparse views

Vinoj Jayasundara, Amit Agrawal, Nicolas Heron, Abhinav Shrivastava, Larry Davis

CVPR 2023

2023

We present FlexNeRF, a method for photorealistic free- viewpoint rendering of humans in motion from monocular videos. Our approach works well with sparse views, which is a challenging scenario when the subject is exhibiting fast/complex motions. We propose a novel approach which jointly optimizes a canonical time and pose configuration, with a pose-dependent motion field and pose-independent temporal deformations

Computer vision

Multimodal neural radiance field

Haidong Zhu, Yuyin Sun, Chi Liu, Lu Xia, Jiajia Luo, Nan Qiao, Ram Nevatia, Cheng-Hao Kuo

ICRA 2023

2023

This paper addresses the challenge of reconstructing a scene with a neural radiance field (NeRF) for robot vision and scene understanding using multiple modalities. Researchers have introduced the use of NeRF to represent an object for synthesizing and rendering novel views of complex scenes by optimizing a 3-D radiance field for ray casting and rendering for 2-D RGB images. However, using RGB images alone

Computer vision

Transferring depth estimation knowledge between cameras

Wei Wang

October 27, 2022

A model that estimates depth from 2-D images learns to adjust to differences between images produced by different cameras, reducing error by about 20%.

Computer vision

3D modeling / imaging

Work with us