Object Detection in Video Using Sequence Alignment and Joint Color & Texture Histogram

(1)

paper by using the joint color texture histogram to represent a target and then applying it to the mean shift framework. Apart from the conventional color histogram features, the texture features of the object are also extracted by using the local binary pattern (LBP) technique to represent the object. The major uniform LBP patterns are exploited to form a mask for joint color-texture feature selection. Object tracking is one of the key technologies in intelligent video surveillance and how to describe the moving target is a key issue. The experimental results validate that the proposed method improves greatly the tracking accuracy and high efficiency with mean shift iterations than standard mean shift tracking. It can track the target under complex scenes, such as similar target and background appearance, on which the traditional color based schemes may fail to track.

Keywords: scene detection, sequence alignment, Object tracking, mean shift, local binary pattern, color histogram.

I. INTRODUCTION

The video is first segmented into shots and a set of key-frames is extracted for each shot. Typical scene detection algorithms incorporate time distance in a shot similarity metric, so to overcome the difficulty of having prior knowledge of the scene duration, the shots are clustered into groups based only on their visual similarity and a label is assigned to each shot according to the group that it belongs to. Then, a sequence alignment algorithm is applied to detect when the pattern of shot labels changes, Apart from the conventional color histogram features, the texture features of the object are also extracted by using the local binary pattern (LBP) technique to represent the object. The major uniform LBP patterns are exploited to form a mask for joint color-texture feature selection. Compared with the traditional color histogram based algorithms that uses the whole target region for tracking. Many tracking algorithms have been proposed to overcome the difficulties arising from noise, occlusion, clutter and changes in the foreground object or in the background environment. Among the various tracking algorithms, mean shift tracking algorithms have recently become popular due to their simplicity and efficiency.

The mean shift algorithm was originally proposed by Fukunaga and Hostetler [5] for data clustering. It was later introduced into the image processing community by Cheng. Bradski modified it and developed the Continuously Adaptive Mean Shift (CAMSHIFT) algorithm to track a moving face.

Comaniciu and Meer successfully applied mean shift algorithm to image segmentation [6] and object tracking [7]. In this paper implement object tracking in video in two ways, first while reading the .avi file and another mmreader file, now first discuss the avi file the AVIFILE Create a new AVI file and format is:

AVIOBJ = AVIFILE (FILENAME) creates an AVIFILE object aviobj with the default parameter values. If filename does not include an extension, then '.avi' will be used. Use avifile/close to close the file opened by avifile.

AVIOB=AVIFILE(FILENAME,'PropertyName',VALUE,'P ropertyName',VALUE,...)

Returns an AVIFILE object with the specified property values. Also one of the problems is that matlab does not provided compressor so that large MB video can’t support. while reading mmreader file, MMREADER Create a

multimedia reader object. OBJ =

MMREADER(FILENAME) constructs a multimedia reader object, OBJ, that can read in video data from a multimedia file. FILENAME is a string specifying the name of a multimedia file. There are no restrictions on file extensions. By default, matlab looks for the file filename on the matlab path. If the object cannot be constructed for any reason (for example, if the file cannot be opened or does not exist, or if the file format is not recognized or supported), then MATLAB throws an error.

II. TRACKING

Object tracking is a challenging task. It has a wide range of applications in different machine vision applications such as automated surveillance, video indexing, human computer interaction and traffic monitoring. Different tracking algorithms exist which are categorized into point tracking, kernel tracking and silhouette tracking [18]. Mean‐shift tracking is a kernel‐based algorithm, where a kernel is the object shape and appearance that is supposed to be tracked. The object can be represented using a rectangular or elliptical patch. We applied a colour‐LSN histogram for object representation in the mean‐shift algorithm. This method is compared with an algorithm [9] which utilizes a color‐LBP histogram.

A.Main components for object tracking

Object Detection in Video Using Sequence

Alignment and Joint Color & Texture

Histogram

(2)

1150 In order to address the challenges discussed in the previous

section, we identify ﬁve main logical components of a video tracker:

1.

The definition of a method to extract relevant information from an image area occupied by a target. This method can be based on motion classification, change detection; object classification or simply on extracting low-level features such as color or gradient, or mid-level features such as edges or interest points.

2.

The definition of a representation for encoding the appearance and the shape of a target (the state). This representation defines the characteristics of the target to be used by the tracker. In general, the representation is a trade off between accuracy of the description (descriptiveness) and invariance: it should be descriptive enough to cope with clutter and to discriminate false targets, while allowing a certain degree of flexibility to cope with changes of target scale, pose, illumination and partial occlusions.

3.

The deﬁnition of a method to propagate the state of the target over time. This step recursively uses information from the feature extraction step or from the already available state estimates to form the trajectory. This task links diﬀerent instances of the same object over time and has to compensate for occlusions, clutter, and local and global illumination changes.

Figure1. the video-tracking pipeline. The ﬂowchart shows the main logical components of a tracking algorithm.

4.

The deﬁnition of a strategy to manage targets appearing and disappearing from the imaged scene. This step, also referred to as track management, initializes the track for an incoming object of interest and terminates the trajectory associated with a disappeared target. When a new target appears in the scene (target birth), the tracker must

initialize a new trajectory. A target birth usually happens:

at the image boundaries (at the edge of the ﬁeld of view of the camera),

at speciﬁc entry areas (e.g. doors),

in the far-ﬁeld of the camera (when the size of the projection onto the image plane increases and the target becomes visible), or

When a target spawns from another target (e.g. a driver parking a car and then stepping out).

Similarly, a trajectory must be terminated (target death) when the target:

 leaves the ﬁeld of view of the camera, or  Disappears at a distance or inside another object

(e.g. a building).

In addition to the above, it is desirable to terminate a trajectory when the tracking performance is expected to degrade under a predeﬁned level, thus generating a track loss condition.

B.MEAN SHIFT TRACKING ALGORITHM

Target Representation

In object tracking, a target is usually defined as a rectangle or an ellipsoidal region in the image. Currently, a widely used target representation is the color histogram because of its independence of scaling and rotation and its robustness to partial occlusions [4]. Denote by the normalized pixels in the target region, which is supposed to be centered at the origin point and have n pixels.The probability of the feature u (u=1, 2… m) in the target model is computed as.

(1 )

Where is the target model, is the probability of the uth

element of δ is the Kronecker delta function, associates the pixel to the histogram bin, and k(x) is an isotropic kernel

profile. Constant C is a normalization function defined by

(2)

Similarly, the probability of the feature u in the target candidate model from the candidate region centered at position y is given by

(3)

Where is the target candidate model, is the probability of the uth _{element of our pixels in the target candidate region}

centered at y, h is the bandwidth and Ch is the normalization

function which is independent of y.

In order to calculate the likelihood of the target model and the candidate model, a metric based on the Bhattacharyya coefficient [1] is defined by using the two normalized histograms and as follows

(5)

The distance between and is then defined as

(6)

Minimizing the distance in Eq. (6) is equivalent to maximizing the Bhattacharyya coefficient in Eq. (5). The optimization process is an iterative process and is initialized with the target position, denoted by yo in the

previous frame. By using the Taylor expansion around coefficient, the linear approximation of the Bhattacharyya in Eq. (5) can be obtained as:

(7) Where,

(8)

Since the first term in Eq. (7) is independent of y, to minimize the distance in Eq. (6) is to maximize the second term in Eq. (7). In the mean shift iteration, the estimated target moves from y to a new position y1, which is defined as

(9)

When we choose the kernel k(x) with the Epanechnikov profile, there is g(x) =-k(x) =1, and Eq. (9) can be reduced to.

(10)

By using Eq. (10), the mean shift tracking algorithm finds in the new frame the most similar region to the object. From Eq. (10) it can be observed that the key parameters in the mean shift tracking algorithm are the weights wi. In this project we

will focus on the analysis of wi with which the scale and

orientation of the tracked target can be well estimated, and then a scale and orientation adaptive mean shift tracking algorithm can be developed.

III. THE TRACKING ALGORITHM WITH THE JOINT

COLOR-TEXTURE HISTOGRAM

We use the RGB channels and the LBP patterns extracted by to jointly represent the target and embed it into the mean shift tracking framework. To obtain the color and texture distribution of the target region, we use (1) to calculate the color and texture distribution of the target model ˆq, in which u = 8×8×8×5. The first three dimensions (i.e. 8 ×8 ×8) represent the quantized bins of color channels and the fourth dimension is the bin of the modified LBP texture patterns. Similarly, the target candidate model ˆp(y) is calculated. The whole tracking algorithm is summarized as follows.

Input: the target model ˆq and its location y0 in the previous frame.

(1) Initialize the iteration number k ← 0.

(2) In the current frame, calculate the distribution of the target candidate model ˆp(y0).

(3) Calculate the weights {wi}i=1···nh using (8).

(4) Calculate the new location y1 of the target candidate using (10).

(5) Let k ← k + 1, d ← _y1 − y0_, y0 ← y1. Set the threshold ε and the maximum iteration number N.

If d < ε or k ≥ N Stop and go to Step 6. Otherwise

Go to step 2.

(6) Load the next frame as the current frame with initial location y0 and go to Step 1

IV. APPLICATIONS

The current and upcoming applications that use object tracking. Although the boundaries between these applications are somehow blurred, they can be grouped in six main areas:

Media production and augmented reality

Object tracking is an important element in post-production and motion capture for the movie and broadcast industries. Match moving is the augmentation of original shots with additional computer graphics elements and special eﬀects, which are rendered in the movie. In order to consistently add these new elements to subsequent frames, the rendering procedure requires the knowledge of 3D information on the scene.

 Medical applications and biological research

(4)

1152  Surveillance and business intelligence

In surveillance systems, tracking can be used either as a forensic tool or as a processing stage prior to algorithms that classify behaviours. Moreover, video-tracking software combined with other video analytical tools can be used to redirect the attention of human operators towards events of interest. Smart surveillance systems can be deployed in a variety of diﬀerent indoor and outdoor environments such as roads, airports, ports, railway stations, public and private buildings.

 Robotics and unmanned vehicles

Robotic technology includes the development of humanoid robots, automated PTZ cameras and unmanned aerial vehicles (UAVs). Intelligent visions via one or more cameras mounted on the robots provide information that is used to interact with or navigate in the environment.

 Tele-collaboration and interactive gaming

Video based gaze tracking is used to simulate eye contact among attendees of a meeting to improve the electiveness of interaction in video-conferencing. Object tracking technology for lecture rooms is available that uses a set of PTZ cameras to follow the position of the lecturer. The PTZ cameras exploit the trajectory information in real-time to guide the pan, tilt and zoom parameters of the camera.

EXPERIMENTAL RESULT

In our experiments, we select some standard test video sequences in order to verify the validity of our algorithm. Experiment one proved and compared to the color-texture histogram features and color histogram features. Select a boll video sequence to track the boll. In this section, experiments are performed to illustrate and testify the proposed joint color texture model based Mean shift tracking algorithm. there are two different method used by tracking now first result based on while reading .avi file format video sequence of hand ball playing with 52 frames of spatial resolution 240x352. The tracking target is the moving head. The target is initialized as a rectangular region of size 120×160. Since there are distinctive color differences between the target (the head of the player) and the background

[image:4.595.341.500.48.424.2]

Fig.2. Tracking results of sequence ―hand ball playing‖ using the proposed method. Frames 20, 25, 28, and 40 are displayed.

Method

Mean error 118.0942 Standard deviation 45.8506

[image:4.595.82.241.636.751.2]

Tracking speed (frames/second) 15

Table 2. The target localization accuracies (mean error and standard deviation) and tracking speed on the ball sequence.

(5)

0 10 20 30 40 50 60 70 80 90 100

A

B

C

D

Fig 8.2 Graph indicate ball video tracking percentage (A-frame20, B-25, C-28, D-40)

[image:5.595.345.508.51.587.2]

On the another result based on mmreader in that only frames are used in the different interval, the second experiment is on a video first extract the frames and that frames used in mmreader then track the object. Now used video sequence of table tennis playing with 58 frames of spatial resolution 240×352, the tracking target is the moving head. The target is initialized as a rectangular region of size 120×160.

Fig.2. Tracking results of sequence ―Table tennis‖ using the proposed method (mmreader). Frames 14, 34, 41, 48, 50 and 58 are displayed.

[image:5.595.86.251.399.655.2]

(6)

1154

Fig.2. Tracking results of sequence ―round table‖ using the proposed method. Frames 30, 55, 80 and 100 are displayed.

DISSCUSSION and CONCULSION

In this paper we propose an effective algorithm of object tracking which uses color-texture histogram features and applying it to the Meanshift algorithm.LBP operator is an effective tool to measure the spatial structure of local image texture. To reduce the computational cost and improve the robustness of target representation, we proposed a joint color and LBP texture based mean shift tracking algorithm in this paper. In other papers only implemented the concept of avi file, but in this paper implement both concepts while reading avi files as well as mmreader files to more robust features.

The proposed target representation model effectively extracts the edges and corners, which are important and robust features, of the object while suppressing the smooth background features.

ACKNOWLEDGMENT

The authors would like to thank Prof. A. P. Thakare for his helpful discussions and valuable suggestions for this paper.

REFERENCES

1

Vasileios T. Chasanis, Aristidis C. Likas, and Nikolaos P. Galatsanos ―Scene Detection in Videos Using Shot Clustering and Sequence Alignment‖, IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 11, NO. 1, JANUARY 2009.

2

Jiafu Jiang and Hui Xiong, ―Object Tracking Based on Multi-feature Mean-shift Algorithm‖ National Conference on Information Technology and Computer Science (CITCS 2012)

3

Matti Pietik.inen, Topi M.enp. and Jaakko Viertola ―Color Texture Classifcation with Color Histograms and Local Binary Patterns‖ Machine Vision Group, Infotech Oulu, University of Oulu.

4

M. Heikki¨a andM. Pietik¨ainen, A texture-based

method for modeling the background and detecting moving objects, IEEE Trans. Patt. Anal. Mach. Intell. 28(4) (2006) 657–662.

5

Y. Cheng, Mean shift, mode seeking and clustering, IEEE Trans. Patt. Anal. Mach. Intell. 17(8) (1995) 790–799.

6

K. Fukunaga and L. D. Hostetler, The estimation of the gradient of a density function, with applications in pattern recognition, IEEE Trans. Inform. Th.

21(1) (1975) 32–40.

7

D. Comaniciu, V. Ramesh and P. Meer, Kernel-based object tracking, IEEE Trans. Patt. Anal. Mach. Intell. 25(5) (2003) 564–575.

Ms. Pallavi M. Sune. Her Computer Science & Engg Dept. BE completed in IT and ME Pursuing in Computer.