Rate-Distortion Optimization with Non-Reference Metrics for UGC compression
Samuel Fernández-Menduiña, Xin Xiong, Eduardo Pavez, Antonio Ortega, Neil Birkbeck, Balu Adsumilli · 2025
Read Rate-Distortion Optimization with Non-Reference Metrics for UGC compression by Samuel Fernández-Menduiña, Xin Xiong, Eduardo Pavez, Antonio Ortega, Neil Birkbeck, Balu Adsumilli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15003.txt.
Introduction
Samuel Fernández-Menduiña and five co-authors improved in 2025 how user-shot video gets compressed for sharing platforms. Home videos arrive noisy, and classic encoders waste bits saving the noise faithfully. Quality scores that need no clean reference fit this job better, but using them inside the encoder takes far too much computing.
The team bends the no-reference score into a simple straight-line form around the raw video. That trick turns quality into a cost the encoder can use block by block while it works. Bits then flow to what viewers notice, not to flaws. The gain is cleaner uploads at the same file size.
Catalog record
| CREATOR | Samuel Fernández-Menduiña, Xin Xiong, Eduardo Pavez, Antonio Ortega, Neil Birkbeck, Balu Adsumilli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15003
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rate-Distortion Optimization with Non-Reference Metrics for UGC compression Samuel Fernández-Menduiña, Xin Xiong, Eduardo Pavez, Antonio Ortega, Neil Birkbeck, Balu Adsumilli Abstract Service providers must encode a large volume of noisy videos to meet the demand for user-generated content (UGC) in online video-sharing platforms. However, low-quality UGC challenges conventional codecs based on rate-distortion optimization (RDO) with full-reference metrics (FRMs). While effective for pristine videos, FRMs drive codecs to preserve artifacts when the input is degraded, resulting in suboptimal compression. A more suitable approach used to assess UGC quality is based on non-reference metrics (NRMs). However, RDO with NRMs as a measure of distortion requires an iterative workflow of encoding, decoding, and metric evaluation, which is computationally impractical. This paper overcomes this limitation by linearizing the NRM around the uncompressed video. The resulting cost function enables block-wise bit allocation in the transform domain by estimating the alignment of the quantization error with the gradient of the NRM. To avoid large deviations from the input, we add sum of squared errors (SSE) regularization. We derive expressions for both the SSE regularization parameter and the Lagrangian, akin to the relationship used for SSE-RDO. Experiments with images and videos show bitrate savings of more than 30% over SSE-RDO using the target NRM, with no decoder complexity overhead and minimal encoder complexity increase. † † address: ∗ University of Southern California, Los Angeles, CA, USA † Google Inc, Mountain View, CA, USA 1 Introduction Non-professional video, also known as user-generated content (UGC), plays a central role in platforms like YouTube and TikTok [ 1 ] . UGC is often noisy due to amateur production, subpar equipment, and prior compression. Once uploaded, service providers usually re-encode these videos at different qualities and resolutions for streaming [ 2 ] . However, low-quality UGC challenges conventional codecs based on rate-distortion optimization (RDO) [ 3 ] with full-reference metrics (FRMs) like the sum of squared errors (SSE). FRMs measure similarity to the input, converging to perfect quality as the compressed content becomes an identical copy of the original video. While this behavior is suitable for the compression of pristine content, it drives the codec to preserve artifacts in noisy UGC, leading to suboptimal compression. Non-reference metrics (NRMs), which assess visual quality without a reference, can be used instead of FRMs. NRMs are the preferred approach for evaluating compressed UGC videos [ 4 , 1 ] because they reveal that increasing bitrate might not improve visual quality. For instance, Fig. 1 shows that FRMs like the SSE and multi-scale structural similarity index measure (MS-SSIM) [ 5 ] approach zero distortion–i.e., perfect quality–at high bitrates, whereas NRMs, such as video score feature aggregation (VSFA), plateau at the input quality. This behavior shows the mismatch between UGC input quality and perfect quality [ 1 ] , which FRMs fail to capture. While popular for UGC quality assessment, NRMs are seldom used for bit allocation. Since NRMs are computed on decoded videos, parameter selection with NRMs would require encoding, decoding, and metric evaluation for each coding option , which is computationally impractical. Existing methods address this problem indirectly, constraining the coding parameter set by classifying the input based on its statistical properties or perceptual criteria and then applying a compression strategy to each class [ 6 , 7 , 8 ] . However, these methods require training on large datasets and parameters are preset for a class rather than optimized for each input. Alternatively, quality saturation detection methods [ 4 , 9 ] identify the coding parameters at which investing more bits no longer improves quality. None of these techniques can help to adaptively select codec parameters block-wise based on the input to optimize a specific NRM. We address this gap by developing a new approach that allows us to optimize coding for a given NRM via RDO. RDO based on SSE is efficient [ 10 ] because 1) it is additive, so global parameter selection simplifies to block-level optimization, and 2) by Parseval’s identity, we can compute SSE in the transform domain when orthogonal transforms are used. To obtain a cost function based on NRMs with these two properties, we approximate the computation of the NRM of the compressed video using Taylor’s expansion (Fig. 2 ) via autodiff [ 11 ] . This linearized NRM (LNRM) depends on the gradient of the NRM with respect to the individual pixels in the input video, measuring the alignment of the reconstruction error with the direction of maximum variation of the NRM. Since the error in using Taylor’s expansion is likely to increase as the rate decreases, we cannot guarantee a good approximation at all rates. To mitigate this effect, we add SSE regularization. We derive expressions for the SSE regularization parameter and for the Lagrangian, akin to existing SSE-RDO methods [ 12 ] . Since we only modify the RDO cost in the encoder, our compression method remains standard-compliant. Figure 1: (a) Frame from a UGC video, and (b-d) rate-distortion curves obtained by compressing the input video with AVC. Unlike FRMs like SSE (b) and MS-SSIM (c), which converge to zero distortion as the rate increases, NRMs such as VSFA (d) do not converge to perfect quality for high bitrates, revealing the mismatch between zero distortion and the UGC input distortion (UID). Figure 2: Gap y_{\mbox{$\bf x$}}(\hat{\mbox{$\bf x$}}) between the NRMs before and after compression, b(\hat{\mbox{$\bf x$}})-b(\mbox{$\bf x$}) , and the corresponding LNRM. We apply LNRM-RDO to AVC [ 13 ] with both images and videos using corresponding non-reference image [ 14 , 15 ] and video quality assessment methods [ 16 ] . We test videos from the YouTube UGC dataset [ 1 ] , setting the RDO to optimize different NRMs. Moreover, since our method can also be applied to pristine images, we consider pristine content from the KODAK dataset [ 17 ] . We also explore the trade-off introduced by the SSE regularizer. Results in Sec. 4 show that LNRM-RDO achieves bitrate savings of over 30\% when optimizing NRMs compared to conventional SSE-RDO, with no decoder complexity overhead and minimal encoder complexity increase. Given that we only modify the RDO, our approach can also be used for codecs with more flexible bit allocation schemes and more RDO options such as VVC [ 18 ] and AV1 [ 19 ] . Notation. Uppercase and lowercase bold letters, such as \bf A and \bf a , denote matrices and vectors, respectively. The n th entry of the vector \bf a is a_{n} . Regular letters denote scalar values. 2 Preliminaries 2.1 Rate-distortion optimization Let \bf x be the input image with n_{p} pixels and \hat{\mbox{$\bf x$}}(\boldsymbol{\theta}) its compressed version using parameters \boldsymbol{\theta}=[\theta_{1}\ \theta_{2}\ \ldots\theta_{n_{b}}]\in\Theta , where \Theta\in\mathbb{N}^{n_{b}} is the set of all possible operating points and n_{b} the number of blocks in the image. Assume every entry of \boldsymbol{\theta} takes values in the set \{1,\ldots,n_{r}\} , where n_{r} denotes the number of RDO options. Given \mbox{$\bf x$}_{i} for i=1,\ldots,n_{b} , we aim to find [ 20 ] : \boldsymbol{\theta}^{\star}=\argmin_{\boldsymbol{\theta}\in\Theta}\,d(\mbox{$\bf x$},\hat{\mbox{$\bf x$}}(\boldsymbol{\theta}))+\lambda\,\sum_{i=1}^{n_{b}}\,r_{i}(\hat{\mbox{$\bf x$}}_{i}(\boldsymbol{\theta})), (1) where d(\cdot,\cdot) is the distortion metric, r_{i}(\cdot) is the rate for the i th block, and \lambda\geq 0 is the Lagrange multiplier that controls the rate-distortion trade-off. We are especially interested