Abstract – In most video coding standards, motion estimation is the most time consuming sub-system. Consequently, in the last few years, a great deal of effort has been devoted to the research of novel algorithms that are capable of saving computations with minimal effects on the coding quality. A novel Reduced Diamond S earch (RDS ) algorithm for fast block matching motion estimation has been proposed in this paper, It is based on the center-biased characteristic of motion vector. S imulation results demonstrate that the proposed algorithm reduces average search points by 20% to 60% according to different motion types with almost the same visual quality compared with other well known block matching algorithms. A threshold to early terminate the block matching process is used to speed up motion estimation time and computation reduction. S imulation results also show that it reduces about 10% to 30% less in search points according to different motion types, with a very low degradation in video quality.
Index Term- Motion estimation, Block matching algorithm, early termination.
I. INTRODUCTION
Motion estimation (ME) has been adopted by all of the existing international standards related to video coding such as the MPEG series[1-2] and H.26x series[3] to remove temporal redundancy between frames. However, ME is also the most computational intensive part, it consumes up to 50% of video encoding time [4]. The Full Search (FS) [5] block matching algorithm is the best in terms of quality and simplicity, but it is the most computation intensive algorithm. There are various fast block matching algorithms such as three-step search (3SS) [6], new-three-step search (NTSS) [7], four-step search (4SS) [8], and diamond search (DS) [9], have been proposed to reduce the computational complexity of ME module.
Recently, a new trend in research work is to reduce the motion estimation computations not only by reducing number of search points in block matching algorithms but also by the use of early termination process, the aim of early termination process is to stop the search algorithm before evaluating each point in the algorithm.
Based on the analysis of above mentioned fast block matching motion estimation algorithms, and a study on motion vector (MV) distribution of real-world test video sequences [10]. A new fast block matching algorithm has been proposed in this paper called reduced diamond search (RDS) it starts the search with only 5 search points using cross pattern, while the other fast search algorithms start with minimum 9 search points. Thus, the RDS algorithm is more simple and efficient when
compared with other search algorithms; as RDS finds the motion vector with reduced number of search points.
The paper is organized as follows: section 2 a brief introduction to previous fast block matching algorithms, Section 3 presents the proposed algorithm (RDS). Section 4 explains the early termination process. Simulation results are shown in Section 5. Finally, the conclusion is in section 6.
II. PREVIOUSFAST BLOCK MATCHINGALGORITHM S The idea behind block matching is to divide the current frame into a matrix 16*16 ‗macro blocks‘ that are then compared with corresponding block and its adjacent neighbors in the previous frame to create a vector that stipulates the movement of a macro block (MB) from one location to another in the previous frame. This movement calculated for all the MB comprising a frame, constitutes the ME in the current frame. The search area for a good MB match is constrained up to ‗p‗ pixels on all fours sides of the corresponding MB in the previous frame. This ‗p‘ is called as the search parameter. Larger motions require a larger ‗p‘ and the larger the search parameter the more computationally expensive the process of ME becomes. Usually the macro block is taken as a square of side 16 pixels, and the search parameter p is ±7 pixels. The idea is represented in figure 1.
The matching of one MB with another is based on the output of a cost function. The MB that results in the least cost is the one that matches the closest to current block. There are various cost functions, of which the most popular and less computationally expensive is Mean Absolute Difference (MAD) [11] given by equation (1). Another cost function is Mean Squared Error (MSE) given by equation (2).
∑ ∑
∑ ∑ (2)
Prof. Atalla I.Hashad, Dr. Rowayda A. Sadek, Member in IEEE, Sara K. Mandour.
A Novel Reduced Diamond Search (RDS)
Algorithm with Early Termination for Fast
Fig 1. MB with size 16*16 and a search parameter size =± 7.
A. Three Step Search (TSS)
This is one of the earliest attempts at fast block matching algorithms. The general idea is represented in figure 2. It starts with the search location at the center and sets the ‗step size‘ S = 4, then it searches at eight locations ± S pixels around location (0,0). From these nine locations searched so far it picks the one giving least cost and makes it the new search origin. Then it sets the new step size S = S/2, and repeats similar search for two more iterations until S = 1.
At that point it finds the location with the least cost function and the macro block at that location is the best match. The calculated motion vector is then saved for transmission.
Fig. 2. 3SS steps ( Step1 step2 step3)
B. New Three Step Search (NTSS)
NTSS was one of the first widely accepted fast algorithms, it improves on TSS and frequently used for implementing earlier standards like MPEG1 [12] and H.261 [13].The NTSS process is illustrated graphically in figure 3. In the first step 16 points are distributed in the following way: in addition to the search origin. 8 are a distance of S = 4 away (similar to TSS) and the other 8 are at S = 1 away from the search origin. If the lowest cost is at the origin then the search is stopped right here and the motion vector is set as (0, 0).
Fig. 3. NT SS steps. ( Inner 1st step outer 1st step)
If the lowest weight is found at any point of the 8 locations at S = 1, then we change the origin of the search to that point and check for weights adjacent to it. On the other hand if the lowest weight after the above mentioned was one of the 8 locations at S = 4, then we follow the normal TSS pro cedure.
C. Four Step Search (4SS)
The 4SS algorithm performs four steps: step1 sets a fixed pattern size of S = 2, it looks at 9 locations in a 5x5 window. If the least weight is found at the center of search window the search jumps to fourth step. Otherwise, step 2 take the point having the least weight to be in the origin and the search window is still maintained as 5x5 pixels wide as shown in figure 4. Step 3 is exactly the same as the second step. In the fourth step the window size is dropped to 3x3, (i.e. S = 1) the location with the least weight is the best matching .
Fig. 4. 4SS steps ( step1 step2 step3 step4)
D. Diamond Search (DS)
The DS algorithm employs two search patterns , the first pattern, called large diamond search pattern (LDSP), it comprises nine checking points from which eight points surround the center one to compose a diamond shape as shown in figure 5a. The second pattern consisting of five checking points forms a smaller diamond shape, called small diamond search pattern (SDSP) as shown in figure5b.
cost function provides the motion vector of the best matching block.
Fig. 5.a (LDSP). Fig. 5.b (SDSP).
III. PROPOSED ALGORIT HM,
REDUCED DIAMOND SEARCH (RDS).
Motion in real-world test video sequence is usually gentle, smooth, and varies slowly. Some basic characteristics of the distribution of global minima must be taken into consideration.
As a consequence, the global minimum distribution is center-biased instead of distributed uniformly, as is demonstrated by the typical examples shown in figure 6. For the Salesman sequence, there are nearly 80% blocks that can be regarded as stationary blocks and most of the motion vectors are enclosed in the central 5 x 5 area. However, it is still highly center-biased [7].
Fig.6. T he motion vector distribution derived from the full search for 100
frames of Salesman sequence.
With such a particular form of distribution, the proposed algorithm RDS is based on this characteristics to achieve much superior performance with fewer number of search points on average. The Reduced diamond search is summarized as follows:
Fig. 7. Fig. 8.
Step1: Start with the cross pattern as shown in figure 7. If the winning point located at the center, go to step 4. Otherwise, go to step 2.
Step2: Check the two points near the winning point to form a half LDSP. As shown in figure 8.
Step3: The winning point found in the previous search step is re-positioned as the center point to form a new LDSP. As shown in figures (9a, 9b).If the point did not change, go to step 4; otherwise, recursively repeat this step.
Step4: Do SDSP. The winning point found in this step is the motion vector which points to the best matching block.
Fig. 9a. Fig. 9b.
Corner winning point. Edge winning point.
IV. EARLYTERM NATION
The objective of early termination process is to reduce computation with minimum effect on video quality. By deciding at each step in the search algorithm if the search point is less than or equal a predefined threshold, otherwise the algorithm proceeds to the next step in the algorithm. So, the best search point for the current block can be found early. A fixed threshold has been introduced in [14]; it states that the average PSNR was in the range of 22 dB to 45 dB.
For early termination of search, selecting PSNR of 22dB will degrade the quality and choosing PSNR much above 45 db increases the number of search points. Considering this fact, we can safely assume that for all practical purposes, a PSNR of 45 can be considered to be a ―good PSNR‖.
the least cost function and check it with the threshold, stop if it is less than or equal to the threshold, if not, reposition the winning point to form a LDSP, check the winning point from the LDSP with the threshold , stop if it is less than or equal the threshold , otherwise, repeat this step until the winning point is not less than the threshold and the winning point does not changed, form a SDSP on this point , then terminate the algorithm. The RDS with early termination is illustrated in figure 10.
V. SIM ULATION RESULTS
Simulation is done on 30 frames and the tested image sequences used in the simulation have different motion characteristics (slow, medium and high motion) using QCIF format (176 * 144).
Two important measures considered for analysis,
Average number of search points per block, in which it has a significant effect on motion estimation time.
Video quality will be evaluated by PSNR.
Simulation results of the proposed algorithm (RDS) when compared with TSS, NTSS, 4SS, and DS are shown in table I. Analysis below compare the RDS results with TSS and DS, as, TSS uses the largest number of average search points and DS uses the least average number of search points among the four algorithms.
All algorithms achieve the same performance in term of video quality for image sequences with slow motion content (e.g. Claire), while the RDS reduces by 61% in average search points compared with TSS and reduces by 28 % when compared with DS. After testing the image sequences with medium motion content (e.g. Foreman) results demonstrate that the RDS average search points compared with TSS were decreased by 52 % and decreased by 25 % compared with DS,
with less than 1 % degradation in video quality. Fig. 10.
For image sequences with high motion content (e.g. Stefan) results reveal that the RDS average search points compared with TSS were decreased by up to 20 % and by 17 % compared with DS , with less than 1 % degradation in video quality .
Figure 11(a, b, c) illustrates a frame-by-frame comparison of the average search points between RDS and the other four algorithms using Akiyo, Carphone and Football sequences, respectively. While figure 12 (a, b, c) shows the comparison of their PSNR.
So that, RDS is highly recommended to use in slow motion content video sequences, as it achieves the same visual quality with less number of search points.
T ABLE I
Average search points and average PSNR for the first 30 frames.
T ABLE II
Average search points and average PSNR for the first 30 frames, after applying threshold on RDS and T SS algorithms.
After applying the threshold on TSS and RDS, Results in table II show that the average search points of RDS in slow motion sequences is reduced by up to 90% compared with TSS, keeping the same video quality.
In medium motion sequences, average search points compared with TSS is reduced by 58% with no change in video quality.
Finally, after applying it on high video motion sequences, the RDS average search points compared with TSS reduced by more than 33 % with no change in video quality.
Figure 13(a, b, c) illustrates a frame-by-frame comparison of the average search points between RDS and TSS algorithm using Akiyo, Carphone and Stefan sequences, respectively. While figure 14 (a, b, c) shows the comparison of their PSNR.
After comparing RDS with and without early termination, RDS with early termination reduces the average search points by 74 % in slow motion sequences, 11% in medium motion sequences and 2 % in high motion sequences by keeping the same video quality.
(a)
(b)
(c)
Fig. 11. Average Search Point comparisons of T SS, NT SS, 4SS, DS and RDS for (a) Akiyo (b) Carphone (c) Football.
(a)
(b)
(c)
Fig. 12. PSNR comparisons of T SS, NT SS, 4SS, DS and RDS for (a) Akiyo (b) Carphone (c) Football.
5 10 15 20 25 30
8 10 12 14 16 18 20 22 Frame number C o mp u ta ti o n s TSScomputations NTSScomputations A4SScomputations DScomputations RDScomputations
5 10 15 20 25 30
8 10 12 14 16 18 20 22 Frame number C o mp u ta ti o n s TSScomputations NTSScomputations A4SScomputations DScomputations RDScomputations
5 10 15 20 25 30
8 10 12 14 16 18 20 22 24 26 Frame number C o mp u ta ti o n s TSScomputations NTSScomputations A4SScomputations DScomputations RDScomputations
5 10 15 20 25 30
38 40 42 44 46 48 50 52 54 Frame number PSN R TSSpsnr NTSSpsnr A4SSpsnr DSpsnr RDSpsnr
5 10 15 20 25 30
30 31 32 33 34 35 36 Frame number PSN R TSSpsnr NTSSpsnr A4SSpsnr DSpsnr RDSpsnr
5 10 15 20 25 30
(a)
(b)
(c)
Fig. 13. Average Search Point comparisons of T SS and RDS after early termination for (a) Akiyo (b) Carphone (c) Stefan.
(a)
(b)
(c)
Fig. 14. PSNR comparisons of T SS and RDS after early termination for (a) Akiyo (b) Carphone (c) Stefan.
5 10 15 20 25 30
0 5 10 15 20 25
Frame number
C
o
mp
u
ta
ti
o
n
s
ThRDScomputations TSSThreshcomputations
5 10 15 20 25 30
4 6 8 10 12 14 16 18 20 22
Frame number
C
o
mp
u
ta
ti
o
n
s
ThRDScomputations TSSThreshcomputations
5 10 15 20 25 30
4 6 8 10 12 14 16 18 20 22
Frame number
C
o
mp
u
ta
ti
o
n
s
ThRDScomputations TSSThreshcomputations
5 10 15 20 25 30
38 40 42 44 46 48 50 52 54
Frame number
PSN
R
ThRDSpsnr TSSThreshpsnr
5 10 15 20 25 30
30 31 32 33 34 35 36
Frame number
PSN
R
ThRDSpsnr TSSThreshpsnr
5 10 15 20 25 30
20 22 24 26 28 30 32
Frame number
PSN
R
(a)
(b)
(c)
Fig. 15. Average Search Point comparisons of RDS with and without early termination for (a) Claire (b) Carphone (c) Stefan.
(a)
(b)
(c)
Fig. 16. PSNR comparisons of RDS with and without early termination for (a) Claire (b) Carphone (c) Stefan.
VI. CONCLUSION
Based on center biased characteristics of motion, the RDS algorithm has been proposed for fast block matching motion estimation. Experimental results show that the RDS outperforms the well known TSS, NTSS, 4SS, and DS search
5 10 15 20 25 30
1 2 3 4 5 6 7 8 9
Frame number
C
o
mp
u
ta
ti
o
n
s
RDScomputations ThRDScomputations
5 10 15 20 25 30
5 6 7 8 9 10 11 12
Frame number
C
o
mp
u
ta
ti
o
n
s
RDScomputations ThRDScomputations
5 10 15 20 25 30
4 6 8 10 12 14 16 18
Frame number
C
o
mp
u
ta
ti
o
n
s
RDScomputations ThRDScomputations
5 10 15 20 25 30
38 40 42 44 46 48 50 52
Frame number
PSN
R
RDSpsnr ThRDSpsnr
5 10 15 20 25 30
30 31 32 33 34 35 36
Frame number
PSN
R
RDSpsnr ThRDSpsnr
5 10 15 20 25 30
20 22 24 26 28 30 32
Frame number
PSN
R
algorithms, by always having a better average search points for (slow, medium and fast motion).
RDS performance in terms of PSNR is the same compared with other algorithms for slow motion video sequences , and achieved a very low degradation in medium and fast motion video sequences.
For stationary blocks, the RDS has to check 5 points only, but for the TSS, NTSS, 4SS, DS have to check 9, 17, 9, 9 points, respectively. Hence, the RDS has the fewest number of search points compared with other algorithms.
After applying the early termination method to the RDS algorithm and TSS algorithm, a dramatic decrease in average search points in RDS, while a very low decrease occurs in average search points in TSS, keeping the same video quality in both algorithms. The reason for the average search points dramatic decrease is that the RDS is a center-biased algorithm while TSS is a uniformly allocated search pattern.
After comparing RDS with and without the use of early termination, the results show that the RDS with early termination is very beneficial not only with slow motion but also with medium motion, keeping the same video quality. Moreover, it makes intensive reduction on the RDS search points leading to a serious reduction in motion estimation time.
REFERENCES
[1] ISO/IEC 11172-2 (MPEG-1 Video), ― Information Technology – Coding
of Moving Pictures and Associated Audio for Digital Storage Media at up to about 1.5 Mbit/s:Video,‖ 1993.
[2] ISO/IEC 13818-2 | IT U-T H.262 (MPEG-2 Video), ― Information
T echnology – Generic Coding of Moving Pictures and Associated Audio Information: Video,‖ 1995.
[3] T . Wiegand, G. Sullivan, G. Bjntegaard, and A. Luthra, ― Overview of
the h.264/avc video coding standard,‖ IEEE Trans. Circuits Syst. Video T echnol., vol. 13, no. 7, pp. 560 –576, Jul. 2003.
[4] Information Technology—Coding of Audio-Visual Objects—Part 10:
Advanced Video Coding. Final Draft International Standard, ISO/IEC FDIS 14 496-10.
[5] F. Dufaux, and F. Moscheni, ―Motion estimation techniques for digital
T V: A review and a new contribution,‖ Proceedings of the IEEE, pp. 858-876,June 1995.
[6] T . Koga, K. Iinuma, A. Hirano, Y. Iijima, and T . Ishiguro, ― Motion
compensated interframe coding for video conferencing,‖ in Proc. Nat.T elecommun. Conf., New Orleans, LA, Nov. 29 –Dec. 3 1981, pp.G5.3.1–5.3.5.
[7] Renxiang Li, Bing Zeng, and Ming L. Liou, ― A New T hree-Step Search
Algorithm for Block Motion Estimation‖, IEEE T rans. Circuits And Systems For Video T echnology, vol 4., no. 4, pp. 438-442, August 1994.
[8] L. M. Po and W. C. Ma, ―A novel four-step search algorithm for fast
block motion estimation,‖ IEEE T rans. Circuits Syst. Video T echnol.,vol. 6, pp. 313–317, June 1996.
[9] Shan Zhu, and Kai-Kuang Ma, ― A New Diamond Search Algorithm for
Fast Block-Matching Motion Estimation‖, IEEE T rans. Image Processing, vol 9, no. 2, pp. 287 -290, February 2000.
[10] C. W. Lam, L. M. Po and C. H. Cheung, "A New Cross-Diamond
Search Algorithm for Fast Block Matching Motion Estimation‖, Proceeding of 2003 IEEE International Conference on Neural Networks and Signal Processing, pp. 1262-1265, Dec. 2003, Nanjing, China.
[11] M.Ezhilarasan and P.T hambidurai,‖Simplified Block Matching
Algorithm For Fast Motion Estimation in Video Compression‖,Journal of Computer Science 4(4):282-289,2008.
[12] ― Video Codec for Audiovisual Services at p_64 kbit=s,‖ IT U-T
SG15,IT U-T Rec. H.261, 2 ed., 1993.
[13] ―Information Technology—Coding of Moving Pictures and Associated
Audio for Digital Storage Media at up to about 1.5 Mbit/s—Part 2:Video,‖ JT C1/SC29/WG11, ISO/IEC 11 172-2 (MPEG-1 Video), 1993
[14] Deepak J. Jayaswal 1, Mukesh A. Zaveri 2‖ Fast Predictive Search
