Selective feature compression for efficient activity recognition inference

By Chunhui Liu, Xinyu (Arthur) Li, Hao Chen, Davide Modolo, Joe Tighe
2021
Download Copy BibTeX
Copy BibTeX
Most action recognition solutions rely on dense sampling to precisely cover the informative temporal clip. Extensively searching the temporal region is expensive for a real-world application. In this work, we focus on improving the inference efficiency of current action recognition backbones on trimmed videos and illustrate that an action model can accurately classify an action with a single pass over the video, unlike the multi-clip sampling common with SOTA, by learning to drop non-informative features. We present Selective Feature Compression (SFC), an action recognition inference strategy that greatly increases model inference efficiency without compromising accuracy. Different from previous works that compress kernel size and decrease the channel dimension, we propose to compress features along the spatio-temporal dimensions without the need to change backbone parameters. Our experiments on Kinetics-400, UCF101 and ActivityNet show that SFC is able to reduce inference speed by 6-7x and memory usage by 5-6x compared with the commonly used 30 crop dense sampling procedure, while also slightly improving Top1 Accuracy. We perform thorough quantitative and qualitative evaluation and show how our SFC learns to attend to important video regions for the task of action recognition.
Research areas

Latest news

IN, TS, Hyderabad
Welcome to the Worldwide Returns & ReCommerce team (WWR&R) at Amazon.com. WWR&R is an agile, innovative organization dedicated to ‘making zero happen’ to benefit our customers, our company, and the environment. Our goal is to achieve the three zeroes: zero cost of returns, zero waste, and zero defects. We do this by developing products and driving truly innovative operational excellence to help customers keep what they buy, recover returned and damaged product value, keep thousands of tons of waste from landfills, and create the best customer returns experience in the world. We have an eye to the future – we create long-term value at Amazon by focusing not just on the bottom line, but on the planet. We are building the most sustainableRead more