BEAST: Building an embodied action-prediction system with trajectory data

We introduce our system BEAST (Building an Embodied Action-prediction System with Trajectory data) for interactive instruction-following within the Alexa Arena Platform. Our system leverages the abstraction of navigation provided by the Arena to decouple the language and vision predictions. This allows for greater simplicity within the system, and for rapid augmentation of the trajectory data-set and training for our text-only action prediction model. By creating a framework with a focus towards user experience our system is more robust to errors in predictions, and informative to the user.