Padding process

The padding process defines the values of luminance and chrominance samples outside the VOP for prediction of arbitrarily shaped objects. Figure 7-15 shows a simplified diagram of this process.

Vertical

Figure 7-15 Simplified padding process

A decoded macroblock d[y][x] is padded by referring to the corresponding decoded shape block s[y][x]. The luminance component is padded per 16 x 16 samples, while the chrominance components are padded per 8 x 8 samples. A macroblock that lies on the VOP boundary (hereafter referred to as a boundary macroblock) is padded by replicating the boundary samples of the VOP towards the exterior. This process is divided into horizontal repetitive padding and vertical repetitive padding. The remaining macroblocks that are completely outside the VOP (hereafter referred to as exterior macroblocks) are filled by extended padding.

Note - The padding process is applied to all macroblocks inside the bounding rectangle of a VOP. The bounding rectangle of the luminance component is defined by VOP_width and VOP_height extended to multiple of 16, while that of the chrominance components is defined by (VOP_width>>1) and (VOP_height>>1) extended to multiple of 8.

7.5.1.1 Horizontal repetitive padding

Each sample at the boundary of a VOP is replicated horizontally to the left and/or right direction in order to fill the transparent region outside the VOP of a boundary macroblock. If there are two boundary sample values for filling a sample outside of a VOP, the two boundary samples are averaged (//2).

hor_pad[y][x] is generated by any process equivalent to the following example. For every line with at least one shape sample s[y][x] == 1(inside the VOP) :

for (x=0; x<N; x++) {

where x’ is the location of the nearest valid sample (s[y][x’] == 1) at the VOP boundary to the left of the current location x, x” is the location of the nearest boundary sample to the right, and N is the number of samples of a line. s’[y][x] is initialized to 0.

7.5.1.2 Vertical repetitive padding

The remaining unfilled transparent horizontal samples (where s’[y][x] == 0) from Section 7.5.1.1 are padded by a similar process as the horizontal repetitive padding but in the vertical direction. The samples already filled in Section 7.5.1.1 are treated as if they were inside the VOP for the purpose of this vertical pass.

hv_pad[y][x] is generated by any process equivalent to the following example. For every column of hor_pad[y][x] :

where y’ is the location of the nearest valid sample (s’[y’][x] == 1) above the current location y at the boundary of hv_pad, y” is the location of the nearest boundary sample below y, and M is the number of samples of a column.

7.5.1.3 Extended padding

Exterior macroblocks immediately next to boundary macroblocks are filled by replicating the samples at the border of the boundary macroblocks. Note that the boundary macroblocks have been completely padded in Section 7.5.1.1 and Section 7.5.1.2. If an exterior macroblock is next to more than one boundary macroblocks, one of the macroblocks is chosen, according to the following convention, for reference.

The boundary macroblocks surrounding an exterior macroblock are numbered in priority according to Figure 7-16. The exterior macroblock is then padded by replicating upwards, downwards, leftwards, or rightwards the row of samples from the horizontal or vertical border of the boundary macroblock having the largest priority number.

The remaining exterior macroblocks (not located next to any boundary macroblocks) are filled with 2bits_pixel-1

. For 8-bit luminance component and associated chrominance this implies filling with 128.

Exterior macroblock B o u n d a r y

macroblock 3

B o u n d a r y macroblock 0

B o u n d a r y macroblock 1 B o u n d a r y

macroblock 2

Figure 7-16 Priority of boundary macroblocks surrounding an exterior macroblock 7.5.1.4 Padding for chrominance components

Chrominance components are padded according to clauses 7.5.1.1 through 7.5.1.3 for each 8 x 8 block. The padding is performed by referring to a shape block generated by decimating the shape block of the corresponding luminance component. The similar rule is applied to interlaced video based on field to enhance subjective quality of display in 4:2:0 format. For each 2 x 2 adjacent luminance shape samples of the same fields, the corresponding chrominance shape sample is set to 1 if any of the four luminance samples are 1. Otherwise the chrominance shape sample is set to 0. For each 2 x 2 adjacent luminance shape samples, the corresponding chrominance shape sample is set to 1 if any of the four luminance shape samples are 1. Otherwise the chrominance shape sample is set to 0.

7.5.1.5 Padding of interlaced macroblocks

Macroblocks of interlaced VOP (interlaced = 1) are padded according to clauses 7.5.1.1 through 7.5.1.3. The vertical padding of the luminance component, however, is performed for each field independently. A sample outside of a VOP is therefore filled with the value of the nearest boundary sample of the same field. Completely transparent blocks are padded with 128.

7.5.1.6 Vector padding technique

The vector padding technique is applied to generate the vectors for the transparent blocks within a non-transparent macroblock, for an INTRA-coded macroblock and for a skipped macroblock. It works in a similar way as the horizontal followed by the vertical repetitive padding, and can be simply regarded as the repetitive padding performed on a 2x2 block except that the padded values are two dimensional vectors. A macroblock has four 8x8 luminance blocks, let {MVx[i], MVy[i], i=0,1,2,3}

and {Transp[i], i=0,1,2,3} be the vectors and the transparencies of the four 8x8 blocks, respectively, the vector padding is any process numerically equivalent to:

if (the macroblock is INTRA-coded, skipped ) { MVx[0] = MVx[1] = MVx[2] = MVx[3] = 0 MVy[0] = MVy[1] = MVy[2] = MVy[3] = 0 } else {

if(Transp[0] == TRANSPARENT) {

MVx[0]=(Transp[1] != TRANSPARENT) ? MVx[1] :((Transp[2]!=TRANSPARENT) ? MVx[2]:MVx[3]));

MVy[0]=(Transp[1] != TRANSPARENT) ? MVy[1]:((Transp[2]!=TRANSPARENT) ? MVy[2]:MVy[3]));

}

if(Transp[1] == TRANSPARENT) {

MVx[1]=(Transp[0] != TRANSPARENT) ? MVx[0] :((Transp[3]!=TRANSPARENT) ? MVx[3]:MVx[2]));

MVy[1]=(Transp[0] != TRANSPARENT) ? MVy[0]:((Transp[3]!=TRANSPARENT) ? MVy[3]:MVy[2]));

}

if(Transp[2] == TRANSPARENT) {

MVx[2]=(Transp[3] != TRANSPARENT) ? MVx[3] :((Transp[0]!=TRANSPARENT) ? MVx[0]:MVx[1]));

MVy[2]=(Transp[3] != TRANSPARENT) ? MVy[3]:((Transp[0]!=TRANSPARENT) ? MVy[0]:MVy[1]));

}

if(Transp[3] == TRANSPARENT) {

MVx[3]=(Transp[2] != TRANSPARENT) ? MVx[2] :((Transp[1]!=TRANSPARENT) ? MVx[1]:MVx[0]));

MVy[3]=(Transp[2] !=TRANSPARENT) ? MVy[2]:((Transp[1]!=TRANSPARENT) ? MVy[1]:MVy[0]));

} }

Vector padding is only used in I- and P-VOPs, it is applied on a macroblock directly after it is decoded. The block vectors after padding are used in the P-VOP vector decoding and binary shape decoding, and in the B-VOP direct mode decoding.