Improving compression efficiency using an encoder-aware motion compensated temporal filter

Motion Compensated Temporal Filtering (MCTF) is a pre-processing approach employed prior to video encoding, for improving the compression efficiency. Prior MCTF designs (e.g. [1]) use pre-defined frame-level quantization parameters (QPs) for different slice types and temporal layers, and operate with a fixed Group of Pictures (GOP) structure. However, commercial encoders can adapt GOP structure based upon content characteristics, and can also adapt QPs on a block-basis based upon the frequency of the block being referenced and the spatial complexity of the block, causing prior MCTF to perform sub-optimally with commercial encoders.

To overcome the drawbacks of prior MCTF design, we propose an encoder-aware MCTF (EA-MCTF) that resides within the encoder as shown in Figure 1. The EA-MCTF adapts on a block-basis based upon the spatio-temporal content properties and block-level encoding parameters including block-level QP, block variance, mean-squared error of motion compensated block difference, slice types of adjoining frames, and frequency of a block being used as a reference. Performance was evaluated using ten 1080p SDR 8-bit test clips, and operating the x265 encoder [2] with a GOP length of 16, slow preset, and constant rate factor = {19, 19.5, 20, 20.5}. The EA-MCTF was integrated into the x265 encoder, and it yields an average and peak VMAF [3] BD-rate gains of -12.4% and -50.6%, respectively, over unfiltered encodings. Similar to the EA-MCTF, the MCTF from [1] (“HM-MCTF”) was integrated into the x265 encoder, and their encodings yielded a loss on many of the test clips, with an average BD-rate of -1.9% over unfiltered encodings. On challenging clips, where the encoder typically operates at the peak bit-rate, the use of EAMCTF improves the average VMAF by 0.26 over unfiltered encodings, while the use of HM-MCTF yields no average VMAF improvement. Finally, the EA-MCTF and HMMCTF incur 17% and 784% average encoding time overhead, respectively, demonstrating that the proposed EA-MCTF is significantly faster than the HM-MCTF.