Position dependent prediction combination for intra-frame video coding
Amir Said, Xin Zhao, Marta Karczewicz, Jianle Chen, Feng Zou · 2025
Read Position dependent prediction combination for intra-frame video coding by Amir Said, Xin Zhao, Marta Karczewicz, Jianle Chen, Feng Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23672.txt.
Introduction
Amir Said and four co-authors improved in 2025 the prediction step inside HEVC video compression. A known trick smooths predicted pixels with recursive filters, which sharpens guesses but blocks parallel computation. The team studied why those filters help, then built non-recursive predictors that do better. Their extension mixes values predicted from raw and smoothed reference samples. The mix depends on the prediction mode and block size.
Tested under the standard HEVC conditions, the scheme cut bit rate by an average of 2.0 percent for all-intra coding at equal quality. Small gains matter at the scale of global video traffic. The design also keeps parallel computation open, which suits hardware encoders. The authors present it as a drop-in upgrade for intra-frame prediction. Similar ideas may extend to later video standards.
Catalog record
| CREATOR | Amir Said, Xin Zhao, Marta Karczewicz, Jianle Chen, Feng Zou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23672
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Position dependent prediction combination for intra-frame video coding Amir Said, Xin Zhao, Marta Karczewicz, Jianle Chen, Feng Zou Abstract Intra-frame prediction in the High Efficiency Video Coding (HEVC) standard can be empirically improved by applying sets of recursive two-dimensional filters to the predicted values. However, this approach does not allow (or complicates significantly) the parallel computation of pixel predictions. In this work we analyze why the recursive filters are effective, and use the results to derive sets of non-recursive predictors that have superior performance. We present an extension to HEVC intra prediction that combines values predicted using non-filtered and filtered (smoothed) reference samples, depending on the prediction mode, and block size. Simulations using the HEVC common test conditions show that a 2.0% bit rate average reduction can be achieved compared to HEVC, for All Intra (AI) configurations. † † address: Qualcomm Technologies, Inc., San Diego, CA, USA 1 Introduction Intra-frame coding in the HEVC video coding standard employs the conventional combination of predictive and transform coding, where prediction is based on single reference pixel lines, from previously reconstructed blocks [ 1 , 2 ] . To attain improved compression efficiency while maintaining low decoder complexity, only very simple linear predictors are used, but the encoder can choose among two non-directional and 33 directional predictors. This approach has proved to be quite effective, but one of its limitations is that the same form of prediction is used for blocks sizes that can range from 4\times 4 to 64\times 64 , and while exact directional prediction can work well in the small blocks, it is much less effective in larger blocks. In HEVC this problem is partially addressed by applying a smoothing filter to the reference lines, with a table-based dependence on predictor type and block size [ 3 ] . However, this approach does not address the issue that, in reality, less smoothing is needed for pixels near the top-left block borders, and more for pixels further away from the borders. Furthermore, even for blocks of the same size, there is much statistical variability, and additional coding gains can be obtained by allowing the encoder to test more filter choices. One interesting solution is to apply sets of recursive filters to smooth the predicted values [ 4 , 5 , 6 , 7 ] , since this can naturally provide, up to a certain degree, the desired smoothing control, and experimental results demonstrate that it can improve compression efficiency significantly. The computational complexity of the recursive filters can appear very small, if we measure it with conventional techniques. However, in custom hardware and general purpose processors, video applications are now employing parallel computation. Thus, it is advantageous to use techniques that allow a large number of block pixels to be predicted in parallel, simultaneously. Under this consideration, the recursive filtering approach becomes much less attractive. We propose an extension to HEVC intra prediction, which was accepted to the latest HM-KTA software, 1 1 1 https://vceg.hhi.fraunhofer.de/svn/svn_HMKTASoftware/tags/ HM-14.0-KTA-2.0/ called Position-Dependent Prediction Combination (PDPC). It is meant to address all those issues—while maintaining low complexity and efficient parallel implementation—by employing a simple weighted combination of unfiltered and filtered (smoothed) references, where the weights depend on pixel position, prediction mode, and block size. In the next sections we show how statistical analysis, plus visualization, can help identify what is the type and degree of smoothing needed in the different cases. 2 Exploiting geometric and statistical features Figure 1: Configuration used for intra-frame prediction in the HEVC video coding standard. Fig. 1 shows the HEVC configuration for intra-frame prediction, for 4\times 4 blocks, and the notation we use. In an N\times N block, we have the pixels values represented by v[x,y] , and their set of predicted values by p_{m}[x,y] with x,y\in\{0,1,\ldots,N-1\} , and m\in\{0,1,\ldots,34\} is the index identifying the predictor type (planar, DC, directions 0 to 33). The reference pixels, which are used for prediction, are represented by a two-dimensional array r[x,y] , but with the assumption that only values r[x,-1] and r[-1,y] , with x,y\in\{-1,0,\ldots,2N-1\} are used [ 3 ] . The predictor formulas used by HEVC are designed to exploit geometric features in the blocks. Theoretically, we can improve performance using statistics to design better predictors, but by itself this approach tends to produce solutions that are either not effective, or too complex for the demanding video applications. We show next that a hybrid approach, exploiting statistical distributions conditioned to geometrical features, can be quite powerful, and even if it cannot be directly used, it yields valuable information on how to design better low complexity predictors. We can employ the HEVC encoder as a classifier of geometric features, and analyze the statistics of the blocks where a predictor with index m was chosen. To simplify the notation, we represent the block pixel, predicted, and reference values as columns vectors {\bf v} , {\bf p}_{m}, and {\bf r} , with dimensions N^{2} , N^{2} , and 4N+1 , respectively. Using this notation, we can use training to estimate the matrices {\bf P}_{m}=E_{{\bf r}|m}\{{\bf r}\,{\bf r}^{\mbox{\sf\tiny T}}\},\quad{\bf Q}_{m}=E_{{\bf v},{\bf r}|m}\{{\bf v}\,{\bf r}^{\mbox{\sf\tiny T}}\}, (1) where operators E_{{\bf r}|m} and E_{{\bf v},{\bf r}|m} define expectation on {\bf v} and {\bf r} , conditioned to m being chosen, by the HEVC encoder, as best predictor index for those block pixel and reference values. With those statistics we can compute the linear least-squared-error predictor as {\bf p}_{m}={\bf H}_{m}{\bf r},\quad{\bf H}_{m}={\bf Q}_{m}{\bf P}_{m}^{-1}. (2) Using matrix {\bf H}_{m} for practical prediction is not feasible because it has N^{2}(4N+1) elements, and we need one matrix for each predictor index m , and for each block size. However, unlike matrix {\bf Q}_{m} , which provides only the level of similarity between pixels and reference values, matrix {\bf H}_{m} can show which similarities are the best to exploit for prediction, by identifying the elements with largest magnitudes in the matrix. This can be observed by using a simple technique to visualize the properties of the optimal predictors in matrices {\bf H}_{m} , and how they vary with m . Fig. 2 shows an example of such {\bf H}_{m} visualization image, where gray level represents the weight of a reference pixel (block columns), for a given predictor with index m (block rows). Each 8\times 8 block represents a column of {\bf H}_{m} , but reorganized to represent the corresponding pixel position. Figure 2: Example of an image for visualizing the distribution of coefficients in predictor matrices {\bf H}_{m} . Zeros are represented as gray, positive as lighter, and negative as darker pixels. If we create a similar image, with a predictor matrix {\bf H} equal to the HEVC predictors, we would see that for m=2,\ldots,34 , we would have well-defined straight lines, according to the direction of those predictors. In Fig. 2 we can observe the same directional patterns (it was, after all, obtained using HEVC classification for training), but we can also observe that the lines have different degrees of “blurring,” to compensate for the fact that in real video frames, patterns commonly deviate from perfectly straight lines. From this observation, the advantage of applying recursive filters to the predicted values become quite clear—it makes the result of HEVC predictions more similar