A Rate-Quality Model for Learned Video Coding
\authorblockN Sang NguyenQuang \authorrefmark 1, Cheng-Wei Chen \authorrefmark 1, Xiem HoangVan \authorrefmark 2 and Wen-Hsiao Peng \authorrefmark 1 \authorblockA \authorrefmark 1 National Yang Ming Chiao Tung University, Taiwan \authorblockA \authorrefmark 2 VNU University of Engineering and Techno · 2025
Read A Rate-Quality Model for Learned Video Coding by \authorblockN Sang NguyenQuang \authorrefmark 1, Cheng-Wei Chen \authorrefmark 1, Xiem HoangVan \authorrefmark 2 and Wen-Hsiao Peng \authorrefmark 1 \authorblockA \authorrefmark 1 National Yang Ming Chiao Tung University, Taiwan \authorblockA \authorrefmark 2 VNU University of Engineering and Techno (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02720.txt.
Introduction
NguyenQuang, Chen, HoangVan, and Peng model the rate-quality tradeoff for learned video coding in a 2025 paper. Neural codecs now beat classic ones, but steering their bitrate precisely is tricky. A good model predicts quality at a given rate.
Their network, RQNet, learns the rate-quality curve from video content and coding context. Predictions from earlier frames are merged by least squares to fit model parameters on the fly. Tests showed much smaller bitrate errors than the baseline approach. Extra complexity stayed minimal. Online adaptation gives the codec flexibility across varied content. The authors target practical learned streaming systems. Accurate rate control is what makes neural codecs deployable.
Catalog record
| CREATOR | \authorblockN Sang NguyenQuang \authorrefmark 1, Cheng-Wei Chen \authorrefmark 1, Xiem HoangVan \authorrefmark 2 and Wen-Hsiao Peng \authorrefmark 1 \authorblockA \authorrefmark 1 National Yang Ming Chiao Tung University, Taiwan \authorblockA \authorrefmark 2 VNU University of Engineering and Techno |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02720
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Rate-Quality Model for Learned Video Coding \authorblockN Sang NguyenQuang \authorrefmark 1, Cheng-Wei Chen \authorrefmark 1, Xiem HoangVan \authorrefmark 2 and Wen-Hsiao Peng \authorrefmark 1 \authorblockA \authorrefmark 1 National Yang Ming Chiao Tung University, Taiwan \authorblockA \authorrefmark 2 VNU University of Engineering and Technology, Vietnam Abstract Learned video coding (LVC) has recently achieved superior coding performance. However, there is a lack of research on the rate-quality (R-Q) model which is important in real-time LVC applications. In this paper, we propose a parametric model to characterize the R–Q relationship in LVC systems. In the proposed model, a neural network, termed RQNet is introduced to characterize the relationship between bitrate and quality level according to video content and coding context. The predicted (R,Q) results are further integrated with those from previously coded frames using the least-squares method to determine the parameters of our R-Q model on-the-fly. Compared to the conventional approaches, our method accurately estimates the R-Q relationship, enabling the online adaptation of model parameters to enhance both flexibility and precision. Experimental results show that our R-Q model achieves significantly smaller bitrate deviations than the baseline methods on commonly used datasets with minimal additional complexity. {IEEEkeywords} Learned video coding, Rate-Quality characteristic, Rate Control 1 Introduction Learned video coding has been considered as a promising coding solution for a wide range of video transmission and storage. Similar to traditional video coding standards, rate control plays a crucial role in deploying LVC for practical video applications. An effective rate control system must deliver high rate-distortion performance while minimizing deviation between actual and target bitrates. The encoder’s efficiency depends on bitrate allocation across frames, while bitrate deviation is determined by encoding parameters. To achieve this, a well-designed model that accurately captures the relationship between bitrate and coding parameters (i.e., quantization parameter or quality level) is indispensable. In learned video coding [ 1 , 2 , 3 , 4 ] , accurately estimating the rate-quality (R-Q) relationship poses a significant challenge. Unlike traditional codecs [ 5 , 6 , 7 ] with well-calibrated analytical models, learned video codecs exhibit distinct R-Q characteristics heavily influenced by model architectures, training protocols, and optimization strategies. Figure 1 : Comparison of the conventional method and our RQ-LVC in predicting rate-quality relationship. In recent years, capturing R-Q relationships in image and video coding has emerged as an important research topic. Jia et al. [ 8 ] utilize exponential and logarithmic functions to characterize the relationship among rate (R), distortion (D), and the Lagrange multiplier ( \lambda ), which indicates the slope of the R-D curve. Xue et al. [ 9 ] propose an exponential function with an additional bias term to model the R- \lambda relationship and enhance the rate-fitting accuracy. Similar approaches have been applied to video coding [ 10 ] , where the R- \lambda and D- \lambda curves are fitted using exponential models, and \lambda values are updated dynamically based on encoding results of previous frames. Inspired by [ 10 ] , Li et al. [ 11 ] propose a learned R-D- \lambda model and parameter updating mechanism for learned video coding. Jia et al. [ 12 ] explore the relationship between bitrate and quantization parameter, while Chen et al. [ 13 ] introduce the scaling factor as a hyperparameter in the R-D model for learned video codecs. Additionally, Gu et al. [ 14 ] design two neural network models to predict (R, \lambda ) and (D, \lambda ) points, leveraging them to model the relationship between R and D. In contrast, Zhang et al. [ 15 ] propose a fully neural network-based rate control system including a rate allocation model and rate implementation network to perform the rate-parameter mapping. Although these methods have proven effective, they remain fundamentally constrained by predefined \lambda values, limiting their adaptability for rate control in learned video coding. Moreover, most methods either update model parameters iteratively by adjusting parameters based on rate-distortion statistics collected from previously encoded frames or use neural networks to predict coding parameters. These approaches operate separately and do not take full advantage of available information. In this paper, we introduce a rate-quality (R-Q) model for learned video coding to estimate the R-Q relationship. In our approach, termed RQ-LVC, the quality level acts as a coding parameter to regulate the variable-rate video coding process. Compared with conventional approaches that independently model the R- \lambda or D- \lambda relationships, our method directly models the link between bitrate and quality, enabling precise rate control for learning-based video codecs. Furthermore, our proposed framework dynamically updates the R-Q model on-the-fly in a batch manner, allowing online adaptation by using information from previously coded frames and predictions from a deep neural network. Our contributions in this work are best illustrated in Fig. 1 . As depicted, multi-pass approaches collect (R,Q) points by encoding a video frame multiple times with different quality levels to fit the R-Q relationship, which is highly time-consuming. In contrast, our proposed RQ-LVC efficiently models the R-Q relationship with low complexity. By leveraging RQNet, our approach significantly reduces encoding complexity while ensuring accurate rate control, offering a more computationally efficient and scalable solution. In Table 1 , we compare the proposed method with several prior works in terms of the rate control model, model parameters updating mechanism, and application scenario. Table 1 : Our method vs. prior works Method Rate Control Model Updating Mechanism Application TIP’14 [ 10 ] \lambda=\alpha R^{\beta} Iterative Mode Traditional Codec ICASSP’24 [ 12 ] R=C\times Q^{-K} Iterative Mode Learned Codec ICLR’24 [ 15 ] Neural Network Neural Network Learned Codec TCSVT’24 [ 13 ] R=\alpha\times r^{\beta} Iterative Mode Learned Codec DCC’25 [ 14 ] R=\alpha\times\lambda^{\beta} Neural Network Learned Codec Our RQ-LVC Q=\alpha\times\operatorname{ln}(R)+\beta Batch Mode Learned Codec *R: bitrate; Q: quality level; \lambda : Lagrange multiplier; \alpha , \beta , C , K : model parameter; r : rescale ratio. 2 Proposed Method Figure 2 : System overview of our RQ-LVC framework with the RQNet, parameter estimation, and R-Q model. RQNet extracts spatiotemporal information to predict (R,Q) points, which are combined with those from previously coded frames to estimate the parametric R-Q model. Then, the predicted Q_{t} is estimated based on the target bitrate R^{target}_{t} to regularize the encoding of an input video frame. Fig. 2 illustrates the overall architecture of the proposed rate-quality model for learned video coding (RQ-LVC). RQ-LVC encodes each input frame X_{t} at variable quality level Q_{t} under a rate constraint R^{target}_{t} by modeling the rate-quality (R-Q) relationship. Specifically, our proposed plug-in module, RQNet, predicts (R,Q) points based on the coding context–such as rate and distortion of reference frames–and content of X_{t} . These predictions serve as prior knowledge, which is further refined using the observed (R,Q) points collected from previously coded frames to update the R-Q model. The updated R-Q model determines the appropriate quality level Q_{t} to satisfy R^{target}_{t} . 2.1 Learning a Variable-Rate Neural Video Codec Our framework features a variable-rate neural video codec that adapts bitrate and quality based on quality level Q . Instead of training separate models for different rates, we develop a variable-rate version using MaskCRT [ 2 ] as the base codec. To train this variable-rate version of MaskCRT [ 2 ] , we adopt