Motion Interpolation 113 

Previous work reported in [33] used a simple pixel averaging technique to interpolate the missing frames. Such a method gives an erroneous image with high errors/ghosting/blurring effect for 3D video sequences with higher motion and large computational complexity; it also produce large PSNR variation between frames when errors occur. Using Motion Interpolation (MI) approach reduces the complexity, buffering delay and the processing time in both the encoding and decoding of the redundant video data. This can be countered by interpolating at the encoder and sending the residual as described in [32]. However, such a method will cause high redundancy as the residual energy needed to represent a given frame will be high due to the poor prediction making such a method not viable for low bit rate applications over mobile channels. These observations motivate the need to use more powerful motion estimation techniques since the accuracy of the decoder frame interpolation module is a key factor to final compression performance. However, the traditional motion estimation and compensation techniques used at the encoder for video coding are not adequate to perform frame interpolation since they attempt to choose the best prediction for the current frame in the rate-distortion sense. For frame interpolation, we need to find an estimate of the current frame, and therefore a good criterion is to estimate the true motion, and based on that to perform motion compensation between temporally adjacent frames. To provide a solution, we propose to add a MI algorithm aside from the Side Information (SI) in the MDC-Even Odd and Side information [33] and call this new scheme Scalable 3D MDC-SIMI. By adding the new algorithm, the frame quality for an interpolated frame can significantly be improved and therefore it reduces the residual energy needed to represent a given frame when encoding, thus further reducing the bit rate. Figure 4-3 shows the proposed MI Algorithm to be added to the encoder.

This new approach employs a second-order predictor for motion compensation, which predicts a current frame from two previously coded frames. The mismatch between the predicted frames at the encoder and decoder is explicitly coded to avoid error propagation in the MDC channels. By using the second-order predictor and coding the mismatch signal, one can also suppress error propagation in packet lossy networks where packets in either description can be lost. The predictor and the mismatch signal quantizer can be varied to achieve a wide range of tradeoffs between coding efficiency and error resilience.

In the new algorithm, 3D MDC-SIMI, temporal correlation of video sequences is exploited through motion estimation at the encoder, so the estimated frames are known even before decoding. One of the most popular motion estimation algorithms is block-matching algorithm. In order to obtain high quality side information, series of measures is taken as shown in figure 4-3. These are low pass filter (LPF), forward motion estimation (FME), bidirectional motion estimation (BME), Spatial Smoothening and bidirectional motion compensation (BMC) [34][35]. The ME is used to simplify the motion search. The proposed scheme adopts two motion estimation algorithms: Forward motion estimation (FME) and bidirectional motion estimation (BME). In the motion compensation stage, the decision module is implemented to take full advantage of two motion estimations to improve side information. The proposed frame interpolation consists of several parts, i.e forward motion estimation (FME), forward motion compensation, bidirectional motion estimation (BME, filters and decision module.

Figure 4-3: Proposed MI in MDC-SIMI Architecture.

One assumption is made for accurate motion estimation in block-matching methods. That is, all pixels in the entire block have the same motion vector (MVs). The smaller the block size the more possibly the assumption holds, but the stronger the local similarity, the less possibly the assumptions holds. In normal coding, inter-frame motion estimation is completed at the encoder, the estimated frame is known before encoding. The assumption has a significant impact on the interpolated results, however in our method, motion estimation is performed at the decoder and the estimated frame is unknown before encoding. Therefore, block size is of great importance in this new method.

4.3.1 FME and BME

In previous 3D coding techniques, key frames are reconstructed by the conventional intra frame decoder at the decoder side, while in our new method; key frames are estimated based on a new frame interpolation scheme. For a given block in the current frame, motion estimation is meant to search for the best matching block in the reference frame.

The new scheme includes two motion estimation algorithms: FME and BME, both estimation approaches is performed, by offsetting each other’s weakness with each other’s merit, greatly improving the quality of the side information and hence the overall quality of the decoded frame. In FME algorithm, the motion vectors are computed between the previous frame and the following frame . For a given current block, a reference block is obtained by full search method. It is found that FME cannot guarantee each pixel in the interpolated frame to be compensated. Motion Vector selection is shown in figure 4-4 and the process is explained in subsection of 4.3.3. Therefore, BME can be used to remove the problem associated with FME. In order to cover all pixels in the interpolated frame, the BME algorithm is introduced. BME is composed of forward and backward estimation. However, the interpolated frame is not known to

overcome the shortcomings of the FME, the BME first needs to determine the search block location in the interpolated frame which will cover all blocks, one by one.

4.3.2 Decision Mechanism

Frame interpolation adopts two motion estimation algorithms: FME and BME. This results in two interpolated frames at the decoder. Uncovered areas in the interpolated frame from the FME will be filled up by BME. In order to make full use of both motion estimation algorithms, a decision mechanism is adopted to process the over lapped areas between two interpolated frames. In the overlapped areas, the bidirectional motion compensated value for current pixel is

with the corresponding sum of absolute difference ) being while the forward motion compensated value with the corresponding being . The ultimate compensated value is determined by

P = + (4-1)

Where: SD1 and SD2 are sum of absolute differences and Pb the corresponding bidirectional

4.3.3 Motion Vector Selection

Figure 4-4(a) shows the process in selecting a motion vector in the forward motion estimation scheme. A block matching algorithm is used to estimate the motion between the next and previous reference frames. Next motion vectors that intercept the interpolated frame closer to the centre are selected from motion vectors obtained from the previous block matching algorithm. Figure 4-4 (b) shows the motion selection vector in the bidirectional motion estimation architecture. It selects a linear trajectory between next and previous reference frames that passes through the centre of the block located in the interpolated frame. It then performs a small refinement search around the initial block position and the motion vectors between three frames. In choosing new candidates, motion vectors must remain on the same linear trajectory thus motion vectors are symmetric between adjacent key frames generating:

, , (4-2)

, , (4-3)

Where , are the coordinates of the block in the previous reference frame, , are the coordinates of the block in the next frame and represents the motion vector obtained using the forward search which is determined by the distance between the forward and previous reference frames. In figure 4-4(b), the motion vector is divided by two, since the interpolated frame is the same distance between both reference frames.

After that, spatial smoothing is used to reduce the number of false motion vectors. It is based on weighted vector median filters as defined in [33]. For bidirectional motion compensation, the standard bidirectional motion compensation found in conventional video coding is used.