The Implementation of a Fast RMD Method for Audio Video Coding Standard II(AVS2)

2018 International Conference on Computer Science and Software Engineering (CSSE 2018) ISBN: 978-1-60595-555-1

The Implementation of a Fast RMD Method for

Audio Video Coding Standard II(AVS2)

Jianbin Zhao, Yunyao Yan, Xiao Liu, Lei Cheng and Wei Yan

ABSTRACT

1

Intra prediction is the most important part of intra coding that could remove spatial redundancy to enhance the efficiency of video compression. This paper proposes a novel algorithm to speed up intra mode selection and reduce time consuming for AVS2. The proposed algorithm adopts sobel operator to obtain the texture orientation of the predicted unit (PU), selecting 11 possible modes that is adjacent to the angle of PU. Then these modes and four special modes are handled by Hadamard transformation that select 9 modes from all modes to form the candidate pattern list. Overall, it could reduce about 30% encoding time with about 0.5% loss of BD-PSNR for 1080p resolution.

INTRODUCTION

AVS2 is made by Chinese Digital audio and video coding technology standard working group whose primary application is ultra high definition video, supporting 4K resolution and high dynamic range video [1]. The test results show that the performance of AVS2 is similar with HEVC in the digital television broadcasting (progressive), real-time communication and digital cinema or static image domain. But in the digital television broadcasting (interlaced) and video surveillance applications, the performance of AVS2 is superior to HEVC.

The AVS2 was composed of intra and inter code [2]. And the core of intra frame code is Intra-Prediction, which means that the samples of macroblock are predicted by using only information of already transmitted macroblocks of the same frame. It removes spatial redundancy by predicting current pixels with coded pixels in same frame.

Jianbin Zhao1,*, Yunyao Yan2, Wei Yan1, Xiao Liu1, and Lei Cheng1

1

School of Software & Microelectronics, Peking University, Beijing, China

2

Harrison first put forward Intra prediction in video code and it has been improved greatly to reduce time waste. Its specific steps are as follows [3][4]:

• Determine whether the adjacent pixels of current TU are available for prediction and do processing for next step;

• Filter the reference pixels of current TU;

• Calculate predicted pixels of cuurrent TU according to filtered reference pixels.

The most complicated and significant procedure is the third step that selects suitable intra modes from 35 modes to form candidate pattern list. In AVS2, the algorithm of RMD is applied to each Prediction Unit(PU), roughly picking up 9 modes as candidates for Rate Distortion Optimized(RDO) calculation. Then the most optimal mode is elected based on the results of RDO. This procedure needs to calculate RDO result of each mode that needs a lot of time. In order to accelerate intra prediction, the candidate modes list should be selected as soon as possible [5].

Apart from DC mode, each mode corresponds to a texture direction. One way to speed up is Fast RMD in HEVC that obtains angle prediction mode of current PU [6]. The average gradient values of each pixel in current predicted PU are calculated by sobel operator and the angle is figured out to choose 5 adjacent modes. Combined with special modes, these modes are regarded as candidate modes.

The rest of this paper is organized as follows. In section 2, a novel algorithm is proposed. To test the performance of proposed method, it is compared with other methods and experimental results are shown in section3. Section 4 conclude this paper.

PROPOSED ALGORITHM

A fast intra prediction mode selection algorithm is proposed that combines texture information with spatial correlation. The algorithm obtains texture information of current PU by sobel operator and select adjacent modes as candidate modes to choose a best optimal mode.

Sobel Method

Sobel operator is mainly applied to edge detection. Technically, it is a discrete difference operator, which is used to calculate the approximate gray value of luminance. The factors of sobel convolution are shown in Figure.1.

-1 0 +1

-2 0 +2

-1 0 +1

+1 +2 +1

0 0 0

-1 -2 -1

Gx Gy

Figure 1. The factors of Sobel convolution.

1 0 1

2 0 2 *

1 0 1

x G A − +     = −_ + _ − +    (1)

1 2 1

0 0 0 *

1 2 1

y G A + + +     =_{ } _{− − −}    (2)

Then the detailed computation process of convolution is as follows:

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

1 * 1, 1 0 * , 1 1* 1, 1 2 * 1,

0 * , 2 * 1, 1 * 1, 1 0 * , 1 1* 1, 1

1, 1 2 * 1, 1, 1 1, 1 2 * 1, 1, 1

Gx f x y f x y

f x y f x y

f x y f x y f x y

f x y f x y

f x y f x y f x y

f x y f x y f x y

= − − − + − + + − + − − + + + + − − + + + + + + = + − + + + + + − − − + − + −      + _ (3) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

1 * 1, 1 2 * , 1

1 * 1, 1 0 * 1, 0 * ,

0 * 1, 1 * 1, 1

2 * , 1 1 * 1, 1

1, 1 2 , 1 1, 1

1, 1 2 * , 1 1, 1

G y f x y f x y

f x y f x y f x y

f x y f x y

f x y f x y

f x y f x y f x y

f x y f x y f x y

= − − + − + + − + − + + + − − + + − + + − + + = − − + − + + − − − + + + + + +         (4)

After getting Gx and Gy, the gradient value and direction of each pixel can be obtained by equation (5) and (6).

(

)

* * 1/ 2

Grd = Gx Gx + Gy Gy

(5)

(

)

Ang = arctan Gy Gx

RMD Method

In AVS2, there is a fast RMD method in intra prediction. It first does RMD selection for each PU to select 9 modes as candidates of RDO. Then the best optimal mode would be elected based on results of RDO.

There are 33 prediction modes to get the prediction samples in Table.I, index 0, 1 and 2 correspond to DC mode, plane mode, and bilinear mode. The size of luminance prediction block is shown in Fig 2. The other modes are called angular prediction modes. The direction of each mode is shown in Fig.3. In RMD, it would set the most probable mode (MPM) and the calculation of cost function. The rules of MPM are as follows [7]:

• If the upper mode is unavailable, DC mode is set;

• If the left mode is unavailable, DC mode is set;

• If the upper and left mode is available, the lower one is set to fists MPM and another is second one;

• If the upper and left modes are available and the same, and the left mode is not the DC pattern, the first MPM is set to the DC mode, the second to the left pattern, or the second MPM to the bilinear interpolation pattern

TABLE I. PREDICTION MODES.

Mode index Prediction modos

0 DC

1 Plane

2 Bilinear

3-11,13-23,25-32 Angular

12 Vertical

24 Horizontal

Figure 2. The size of luminance prediction block.

After the selection of MPM, the cost function would be computed with equation (7).

.

pred SATD pred pred

J =SATD+λ ×B

(7)

The SATD is the Sum of Absolute transform Difference. λpred is Lagrange

coefficient and Bpredis required encoding bits of current predictive mode [8].

Because the size of luminance prediction block can be either 2N×2N _, N×N_, 2N×0.5N _or 0.5N×2N_，whereN_{can be 32, 16, 8, or 4. In intra prediction, PU is square}

partitioning, which size can be 64, 32, 16, 8, or 4. Then RDO starts from a LCU and finds best suitable CU size based on recursive quadtree.

Improved Fast RMD Method

In order to obtain angular information by sobel operator, the angle of mode 3-32 of the novel fast RMD method is set to -67.5,-60,-52.5,-45,-37.5,-30,-22.5,-15,-7.5,0,7.5,15,22.5,30,37.5,45,52.5,60,67.5,75,82.5,90,97.5,105,112.5,120,127.5,135,14 2.5,150.

In this way, the angular prediction mode can be obtained by sobel operator. The gradient value of each pixel in current PU is calculated by using the Sobel operator introduced previously, and the average value of Gx and Gy (the pixels of the PU) is calculated with equation (8) and (9);

' (1) ' (2) ... ' ( )/

x x x x

G=G +G + +G p p

(8)

' (1) ' (2) ... ' ( )/

y y y y

G G= +G + +G q q

(9)

The angle of PU is obtained with equation(8),which is compared with the angle of mode 3-32 shown before to select the nearest 11 modes.

/

ang = arctan Gy（ Gx） (10)

These 11 modes, DC, Plane, Bilinear, Vertical and Horizontal mode constitute the list of candidate. Then all these 15 modes are handled by Hadamard Transform to select 9 modes for the calculation of RDO.

However, not all CU has 9 candidate prediction modes. The number of candidate prediction patterns should be adjusted on the basis of the size of CU. The corresponding relationship between the size of CU and the number of candidate modes is shown in Table II. The resource of proposed algorithm is shown in Table III.

TABLE II. THE RELATIONSHIP BETWEEN SIZE AND NUMBER OF CU.

The size of CU The number of candidates

Uibitsize = 3 CandModeList = 9

Uibitsize = 4 CandModeList = 5

Uibitsize = 5 CandModeList = 9

TABLE III. THE RESOURCE OF PROPOSED ALGORITHM.

Name Bram Dsp48 FF LUT URAM

DSP / 2 / / /

Expression / 4 26052 27535 /

FIFO / / / / /

Instance 8 8 11746 13584 /

Memory 21 / 76 17 /

Multiplexer / / / 5161 /

Register / / 7035 160 /

Total 29 14 44909 46457 0

Available 1824 2520 548160 274080 0

Utilization 1 ~0 8 16 0

Figure 4. System Structure.

Implementation on ARM-Linux

In this paper, the AVS2 has transformed from PC to ARM-linux to test the algorithm with High-Level Synthesis (HLS). The sobel operator is packed into IP core that can communicate wit ARM and memory on Zynq MP 102 in AXI4 interface. The use of resources is as in Table III and the structure of the systme is illustrated in Fig.4.

PROPOSED ALGORITHM

TABLE IV. EXPERIMENTAL RESULTS ON PC. Sequence

Name

AVS2 Standard AVS2 Sobel QP _{∆PSNR ∆Time}

PSNR(d b)

Time(s) PSNR(d

b)

Time(s)

BQTerrac e_1920x1 080_60

43.32 447.507 43.15 308.272

20 0.17(0.39

%)

139.235(31.11 %)

38.91 390.401 38.70 272.915

26 0.21(0.53

%)

117.486(30.09 %)

35.54 347.737 35.37 244.318

32 0.17(0.48

%)

103.419(29.74 %)

33.14 319.543 33.00 225.112

38 0.14(0.42

%)

94.431(29.55%)

FourPeop le_1280x 720_60

44.32 152.539 44.19 108.116

20 0.13(0.29

%)

44.423(29.12%)

42.23 138.695 42.14 98.867

26 0.09(0.21

%)

39.828(28.72%)

40.16 131.281 40.07 93.467

32 0.09(0.22

%)

37.814(28.80%)

37.78 126.358 37.67 89.924

38 0.11(0.29

%)

36.434(28.83%)

Basketbal lDrill_83 2x480_50

43.01 78.644 42.85 66.190

20 0.16(0.37

%)

12.454(15.84%)

40.17 69.693 40.05 59.703

26 0.12(0.30

%)

9.99(14.33%)

37.50 63.384 37.43 54.758

32 0.07(0.19

%)

8.626(13.61%)

35.05 59.519 34.96 51.316

38 0.09(0.26

%)

8.203(13.78%)

BQSquar e_416x24 0_60

41.79 22.846 41.35 15.924

20 0.44(1.05

%)

6.922(30.30%)

38.26 20.342 37.80 14.259

26 0.46(1.20

%)

6.083(29.90%)

34.72 17.896s 34.32 12.737

32 0.40(1.15

%)

5.159(28.83%)

31.49 16.342s 31.24 11.447

38 0.25(0.79

%)

4.895(29.95%)

TABLE V. EXPERIMENTAL RESULTS ON ZYNQ. Sequence

Name

AVS2 Standard AVS2 Sobel QP _{∆PSNR ∆Time}

PSNR(d b)

Time(s) PSNR(d

b)

Time(s)

BQSquar e_416x24 0_60

41.74 95.063 41.26 73.733

20 0.46(1.15

%)

21.330(22.44%)

38.15 86.420 37.68 68.091

26 0.47(1.23

%)

18.329(21.21%)

35.75 81.742 34.16 63.393

32 1.59(4.45

%)

18.349(22.45%)

31.33 73.840 31.01 59.866

38 0.32(1.02

%)

14.152(19.17%)

that the proposed algorithm could reduce encoding time for all sizes and the performance of high resolution is better.

As for the test on FPGA, the results are shown in Table.V. Because of the limit of RAM and Memory on FPGA, this paper only analyse the sequence of 416*280. Experimental results shows that the proposed algorithm can reduce about 20% encoding time with about 1.2% loss of BD-PSNR.

CONCLUSION

In this paper, we proposed a novel Fast RMD method for AVS2. The algorithm uses sobel operator to estimate the texture direction of current PU, which is applied to calculate the angle of PU via the average of Gx and Gy. Then the adjacent prediction modes and special modes that are handled by Hadamand Transform are selected to form the list of candidate RDO modes. And the candidate number of RDO is different for distinct CU size. In this way, the proposed algorithm can reduce the number of candidate in the Fast RMD selection and accelerate the process of intra mode selection.

ACKNOWLEDGEMENT

This work has been supported by National key science and development plan (2016YFC0801001).

REFERENCES

1. MA S. History and recent developments of AVS video coding standards[J]. Journal of Computer

Research and Development, 2015, 52(1): 27-37.

2. Lei H, Wei X, Yang Z, et al. A fast mode decision algorithm for intra prediction in HEVC[J].

Computer Engineering, 2014, 40(5): 270-273.

3. Piao Y, Chen J, Lee S, et al.Intra coding of AVS2 video coding standard[C]//Proceedings of the

2014 IEEE International Conference on Multimedia and Expo Workshops. Piscataway: IEEE,

2014: 1-5.

4. He Z, Yu L, Zheng X, et al .Framework of AVS2-video coding[C]//Proceedings of the 2013 20th

IEEE International Conference on Image Processing. Piscataway: IEEE, 2013: 1515-1519. 5. Cheng Y, Teng G, Shi X, et al. A fast intra prediction algorithm for HEVC[M]//Proceedings of

the 9th International Forum on Digital TV and Wireless Multimedia Communication. Heidelberg:

Springer, 2012:292-298.

6. Zheng C, Wang G, Fan T, et al. Fast intra prediction mode decision algorithm for AVS2[J]. Video

Engineering, 2016.

7. Chen F, Li G, Wang G, et al. A Fast Inter-frame Prediction Algorithm for AVS2 Based on Mode

Complexity[M]// Digital TV and Wireless Multimedia Communication. 2017.

8. Zhao Chao, Zhao Haiwu, Wang Guozhong, Li Guoping, Teng Guowei. Fast selection algorithm