Robotics

20 results found

Sort

Multi-view traffic sign localization with high absolute accuracy in real-time at the edge

ABM Musa

ACM SIGSPATIAL 2022

2022

In this paper, we present a system to localize traffic signs with high accuracy in real-time at the edge using 3D reconstruction of an environment. We use Structure-from-Motion (SfM) based 3D reconstruction with only a small number of tracked feature points in video frames. The 3D model obtained from SfM alone has an arbitrary scale and orientation, which is not suitable for localizing traffic signs absolutely

Computer vision
DialFRED: Dialogue-enabled agents for embodied instruction following

Xiaofeng Gao, Qiaozi (QZ) Gao, Ran Gong, Kaixiang Lin, Govind Thattai, Gaurav Sukhatme

IROS 2022

2022

Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural language command to the agent, and the agent can only follow the command passively. We present DialFRED, a dialogue-enabled embodied instruction following benchmark based on the ALFRED benchmark. DialFRED allows an agent to actively

Related: Teaching virtual robots to follow natural-language instructions

Computer vision
Learning to act with affordance-aware multimodal neural SLAM

Zhiwei Jia, Kaixiang Lin, Yizhou Zhao, Qiaozi (QZ) Gao, Govind Thattai, Gaurav Sukhatme

IROS 2022

2022

Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its environment. There are several challenges in solving embodied multimodal tasks, including long-horizon planning, vision-and-language grounding, and efficient exploration. We focus on a critical bottleneck, namely the performance of

Related: Teaching virtual robots to follow natural-language instructions

Machine learning
Learning feature decomposition for domain adaptive monocular depth estimation

Shao-Yuan Lo, Wei Wang, Jim Thomas, Jingjing Zheng, Vishal M. Patel, Cheng-Hao Kuo

IROS 2022

2022

Monocular depth estimation (MDE) has attracted intense study due to its low cost and critical functions for robotic tasks such as localization, mapping and obstacle detection. Supervised approaches have led to great success with the advance of deep learning, but they rely on large quantities of ground-truth depth annotations that are expensive to acquire. Unsupervised domain adaptation (UDA) transfers knowledge

Related: Transferring depth estimation knowledge between cameras

Computer vision
CHMARL: A multimodal benchmark for cooperative, heterogeneous multi-agent reinforcement learning

Vasu Sharma, Prasoon Goyal, Kaixiang Lin, Govind Thattai, Qiaozi (QZ) Gao, Gaurav Sukhatme

Robotics Science and Systems Workshop

2022

We propose a vision-and-language benchmark for cooperative and heterogeneous multi-agent learning. We introduce a benchmark multimodal dataset with tasks involving collaboration between multiple heterogeneous agents in a rich multiroom home environment. We provide an integrated learning framework, multimodal implementation of the state-of-the-art, and consistent evaluation protocol. Our experiments investigate

Machine learning
On the limits of evaluating embodied agent model generalization using validation sets

Hyounghun Kim, Aishwarya Padmakumar, Di Jin, Mohit Bansal, Dilek Hakkani-Tür

ACL 2022 Workshop on Insights from Negative Results in NLP

2022

Natural language guided embodied task completion is a challenging problem since it requires understanding natural language instructions, aligning them with egocentric visual observations, and choosing appropriate actions to execute in the environment to produce desired changes. We experiment with augmenting a transformer model for this task with modules that effectively utilize a wider field of view and

Computer vision
Robotics call for proposals

Pursuing the future of robotics research
A scalable computer vision framework for mobile device auto-typing

Zongyi Liu, Rohit Kumar Gupta, Kun Chen, Bruce Ferry, Simon Lacasse

ICCE 2019

2020

In this paper, we present a computer vision framework that controls robots to auto type on a mobile device such as an Android phone or an iPad. The framework consists of three parts: (i) an image undistortion and segmentation algorithm that supports images captured by a top mounted camera or a side mounted camera, (ii) a deep neural network (DNN) algorithm that detects the keyboard region and recognizes

Computer vision
A deep neural network to detect keyboard regions and recognize isolated characters

Zongyi Liu, Bruce Ferry, Simon Lacasse

ICDAR 2019

2019

Some robot systems need to interactively auto-type on touch screens using a digital camera as the input source. To do so, it is important to have an algorithm that reliably detects the keyboard region and locates and recognizes its characters from an image. However, today most research efforts in the optical character recognition (OCR) area is focused on scene text detection and recognition. Though these

Computer vision
Estimating uncertainty in instance segmentation using dropout sampling

Doug Morrison, Anton Milan, Nontas Antonakos

CVPR 2019 Robotic Vision Probabilistic Object Detection Challenge

2019

Vision is an integral part of many robotic systems, and especially so when a robot must interact with its environment. In such cases, decisions made based on erroneous visual detections can have disastrous consequences. Hence, being able to accurately measure the uncertainty associated with visual information is highly important for making informed decisions. However, this uncertainty is often not captured

Computer vision

Robotics

Work with us