An Algorithm for Retrieval of Significant Events Near Goal Post From Soccer Videos Using Fuzzy Systems

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459,ISO 9001:2008 Certified Journal, Volume 3, Issue 3, March 2013)

808

An Algorithm for Retrieval of Significant Events Near Goal

Post From Soccer Videos Using Fuzzy Systems

Vilas Naik

1

, Ganesh Rathod

2

1,2_{Department of Computer science and Engineering, Basaveshwar Engineering College,Bagalkot}

Abstract— Event retrieval is one of the important topics of

research in multimedia processing. Event retrieval system is for searching and retrieving significant events relevant to cater user needs. Soccer video analysis plays an important role in research. In football games, there are lots of events that can be considered from different points of view, but the events that maybe change the score and are interest for fans and coaches are “goal-score”, “penalty shootout”, ”free-kick” and “corner kick”. These entire events bound to happen near the goal post, hence proposed method concentrate near goal post. Most retrieval systems make indirect use of human knowledge in their retrieval process. The proposed method aims to efficiently build human knowledge directly for soccer video events retrieval by fuzzy systems. The first phase of proposed scheme is of extracting key-frames, using which the frames are categorized into three views (Far-View, Mid-view and Out-view). Further, considering the frames of mid-view line features are extracted using Hough Transform to detect goal post. Then, using a fuzzy rule base containing the experiences of human knowledge, significant events are extracted. The algorithm is efficient enough to extract significant events near to goal post as per ground truth built.

Keywords—Event retrieval, Significant Events, Goal post

detection, Fuzzy system..

I. INTRODUCTION

Multimedia collections are growing rapidly in both the professional and consumer environment, and are characterized by a steadily increasing capacity and content variety, such as movies, documentaries, sports, news, home videos, e-learning, etc. Therefore, it is more and more difficult for us to find relevant video from an increasing video database. So, there is a strong demand for effective video retrieval to solve the problem. Because of the widespread popularity of sports video, the search of multimedia information retrieval (MIR) mainly focuses on the sports video (basketball, baseball, ice hockey and golf), especially for soccer videos. . Soccer video analysis plays an important role in research area. The basic idea of soccer events retrieval is to infer and retrieve the interesting events, and its goal is to make the results accord with human’s visual perception as much as possible.

(2)

International Journal of Emerging Technology and Advanced Engineering

809 Variant solutions are proposed for event retrieval for sports video. A content-oriented video retrieval system is presented in [8], which is capable of handling high volumes of content as well as various functionality requirements. It allows the audience to access the video contents based on their different interests of selected video program. Shot detection is a fundamental step in video processing and analysis that should be achieved with high degree of accuracy. The work in [9], introduces a unified algorithm for shot detection in sports video using Fuzzy Logic as a powerful inference mechanism. An enhanced query model used for soccer video retrieval using temporal relationships is discussed in [10], which is a general framework for automatically analyzing the sports video, detect the sports events, and finally offer an efficient and user-friendly system for sports video retrieval. Sports video analysis in particular has received much attention in the area of digital video processing and hence paper [11] proposes a novel audio–visual feature based framework for event detection in broadcast video of multiple different field sports. Soccer game is most popular one in the world and people are far more interested in the scoring plays of the game. The work in [12] use Bayesian network to statistically model the scoring event detection based on the recording and editing rules of soccer video. Based on three generally defined shot types, the work presented in [13], detects goal events by combining heuristic rules with unsupervised fuzzy c-means algorithm. An integrated framework for content-based indexing and retrieval in video databases is presented in [14], which has the capability of adapting its performance according to user requirements.

All the methods described above work after selecting the shots which contain the major events. So they do the retrieval operation based on recognition and selection of such shots. The basic idea in this paper is to avoid shot detection and work based on probable scenario of events, A provision in which non significant scenarios are filtered out. As It is known most of the time the ball in soccer game is in the middle of the field and the major events such as goals, corner kicks, penalties, etc bound to happen near goal post. Consider a retrieval that includes removal of all probably useless shots, such as shots of fans, close shots of coaches, shots of the midfield, etc and concentrate near the area of goal post. Such retrieval will mostly cover the attacks and major events of the match. The method here uses human expert knowledge directly for soccer video events retrieval by fuzzy systems. The advantage of applying fuzzy is that, it deals with the complicated systems in simple way. The rest of this paper is organized as follows. In Section 2, provides the details of the proposed algorithm and also show its design flow.

In Section3, the experimental results of the proposed algorithm is presented. Finally, conclusion and recommendations for future research are given in Section 4.

II. PROPOSED ALGORITHM

Most information retrieval systems make indirect use of human knowledge in their retrieval process. The proposed method here aims to efficiently build human knowledge directly into fuzzy systems for retrieval of events from soccer videos. The first phase consists of extracting suitable features from video shots. Then, using a fuzzy rule the significant events are extracted. The Figure 1 depicts over all idea of proposed event detection algorithm. The flow chart for the complete scheme of the methodology is presented in Figure 2.

[image:2.612.354.537.310.524.2]

Figure 1. Methodology for Event Retrieval.

The proposed algorithm can be explained in following steps.

Algorithm

Step 1: The input for this algorithm is soccer videos. The frames are extracted for individual consideration.

Step 2: TheKey frames extraction will be performed using entropy value difference method.

Step 3: From the keyframes Grass detection is done based on dominant color and 3 types of views (Far-view, mid-view and out-mid-view) classification is based on grass color percentage.

Input Video

Fuzzy Inference System

Event Frames near Goal Post

Event Videos Key-Frame Extraction

Grass Detection & View classification

Line Detection using Hough

(3)

International Journal of Emerging Technology and Advanced Engineering

810 Step 4: Considering the mid-view frames, line features are extracted for the detection of goal post and penalty box using Hough Transform.

[image:3.612.40.278.196.455.2]

Step 5: Fuzzy Inference System is built to classify significant and insignificant events from a soccer video.

Figure 2 The overall structure of the proposed algorithm.

A. Key Frame Extraction.

Key-frames play a very important role in video abstraction. Key-frames are also called representative frames, that are a set of salient images extracted from video sequences. They provide a simple yet effective way of summarizing the content of videos for browsing and retrieval and are also widely used in video abstraction due to their compactness. Much research has been conducted in the past few years in understanding the problem of key-frame extraction and developing effective algorithms. Current key-frame extraction techniques can be categorized into five classes. The first class is shot boundary based approach, second is visual content based approaches, third class is motion analysis based approaches, the fourth class is shot activity based approaches and the fifth class is unsupervised clustering based approaches. The method approached in this work is shot activity based approach. The activity is detected in the form of information content change that happens from frame to frame. The detail algorithm for Key-Frame extraction is given below.

Algorithm for Key Frame Extraction

Step1: Reading the sports video file extract individual frames

Step 2: Conversion of every frame from RGB to gray is done.

Step 3: Entropy value of the first frame is calculated and then it is written as a key frame. Further the entropy difference between every frame with its next frame is calculated.

Step 4: If the difference is greater than the pre-defined threshold value then frame second frame is taken as a key frame.

Key-Frames extracted shown in Figure 2(A) are used for the further processing in the event detection mechanism the Grass detection for view classification.

Figure 2 (A). The 119 key frames extracted from clip of 2100 frames Start

Read Video

Key-Frame Extraction

Grass Detection and View Classification

Mid-View

Fuzzy Inference System

Results

Rule Base Out-View Far-View

Line Feature Extraction

[image:3.612.325.583.315.685.2]

(4)

International Journal of Emerging Technology and Advanced Engineering

811 B. Grass Detection and View Classification.

A soccer field has one distinct dominant color (a tone of green) and therefore, a simple method used here for grass detection is to assume a specific value for the dominant color of the field. Once the grass is been detected 3 types of views i.e Far-View, Mid-View and Out-View is been classified. The field where grass percentage is less than 10% is considered as out-view and the grass percentage between 10% and 85% are considered as mid-view whereas remaining field are considered as far-view. The definitions and characteristics of each view are given below.

Far-View:- A far-view displays global view of the field as shown in Figure 3 (a) & (b); here a far view serves for accurate localization of the events of the field.

Mid-View:- A mid-view, where a whole human body is usually visible, is a zoomed-in view of a specific part of the field as in Figure 3 (c) & (d).

Out-View:- The audience, coach and other shots are denoted as out-view (Figure 3(e) & (f)).

(a) (b) (c)

(d) (e) (f)

Figure 3(a). View types in Soccer: (a,b) Far-View, (c,d) Mid-View, and (e,f) Out-View.

C. Line Detection using Hough Transform

Since most of the events in soccer game like ―goal-score‖, ―penalty shootout‖, ―free-kick‖ and ―corner kick‖ bound to happen near the goal post,the goal post detection from the frames become important task for detection of events near it. The proposed work employs Hough transform(HT) for detection of goalpost and penalty box. . The HT is a linear transform originally developed for line detection in digital pictures. The algorithm considers the mid-view frames extracted from procedure explained above section, and then finds the edges from the binary image using Canny edge detection method. Later, Hough Line Features draws lines in terms of pixel location and line length. Extracted line features are shown in Figure 3(b) and used for building Fuzzy Inference system for event detection.

Figure 3(b). The goalpost detected using Houghs transform

D. Fuzzy Inference System to classify significant events.

Line features and grass percentage extracted above are the two inputs to the fuzzy system. Fuzzy rule built explains how much important are the contents of the shot for the significant event. The other goal of designing a fuzzy system is to identify insignificant events. The rule base and three Gaussian curve membership functions (low, medium and high) are presented in Figure 4 & 5. The inference method used here is Sugeno product and defuzzification method used here is weighted average (

w

taver). Result of the execution of the fuzzy inference is

[image:4.612.56.283.357.497.2]

shown in Figure 6, after the execution a threshold value is generated using which significant and insignificant events are separated. The f uzzy system combines the percentage of grass present in the particular frame and appearance of goal post in the frame and detects the frames with significant amount of grass and goal post and consider the event around that frame as event near goal post.

Figure 4. Gaussian curve membership functions (low, medium and high) for In/Out variables.

Figure 5. Fuzzy rule base for classifying significant events.

III. EXPERIMENTAL RESULTS

[image:4.612.342.545.425.501.2]

(5)

International Journal of Emerging Technology and Advanced Engineering

[image:5.612.48.297.159.289.2]

812 The experimental dataset used in this work contains the 10 ―.avi‖ videos of soccer ranging from 1.3MB to 2.88MB. All these videos are downloaded from soccer clips.com and

Figure 6. If (Line Cluster=4.07 and Grass Percent=1.2) then Event Frame=3.21.

Youtube web sites. Proposed algorihm is tested with videos of different types. Results for such samples are described here

[image:5.612.335.553.207.280.2]

 Input Soccer Clip 1

Figure 7. Input clip- Video1 with

Size: 2.20 MB and Duration: 00:00:35.

This clip contains the free kick given to France player Zedaine, who gives a stunning pass to his fellow mate and eventually a goal is been scored through free kick. The clip also shows players and coach celebrating the success. Here, for the above input clip with size 2.20MB and duration 00:00:35 the algorithm gives following frames as event frame near goal post.

Figure 8. Event frames generated by fuzzy system.

[image:5.612.340.523.339.481.2]

As we can see from the the above Figure 8 , it is clear that the frames near the goal post is retrieved, which is the main goal of this algorithm. Once the event frames are extracted the significant video is constructed around the event frame, by adding 60 frames before event frame and 60 frames after it. The following are the event videos constructed:-

Figure 9. Event videos near goal post.

The video constructed above, is of 11 seconds each. Event video 1 constructed above consists of the goal scored by the player in real-time, whereas the event video 2 and 3 shows the reply of the goal scored with different cameras.

[image:5.612.61.242.354.523.2]

 Input Soccer Clip 2

Figure 10. Input clip- Video1 with Size: 2.88 MB and Duration: 00:00:45.

For the above input clip, with size 2.88MB and duration 00:00:45 the algorithm gives following frames as event frame near goal post. This sample video contains the fabulous goal scored by the Italian player for his team. The video shows players and coach celebrating the success; it also shows the reply of the goal scored with different camera.

Figure 11. Event frames generated by fuzzy system.

[image:5.612.337.553.576.638.2] [image:5.612.54.284.618.695.2]

(6)

International Journal of Emerging Technology and Advanced Engineering

[image:6.612.60.280.121.195.2]

813

Figure 12. Event videos near goal post

Below table describes the comparison with few others sample videos for which the proposed algorithm is tested.

Table 1.

Experimental Results for Proposed method

IV. CONCLUSION

The algorithm for retrieval of significant events near goal post in soccer videos is designed and experimented sufficient number of soccer clips. The algorithm is implemented in Matlab 2010b and executed on Pentium® Dual-Core processor with 3GB RAM memory. The significant events near goal post are extracted by beginning with the extraction of key-frames.

From the key-frames the frames are categorized into far-view, view and out-view. Further containing with mid-view frame goal post determination is done using Hough Transform. The events are constructed along the frames containing goal post and fuzzy classifier is employed to classify events as significant and insignificant. The results reveal that the algorithm is efficient enough to extract significant events as per ground truth built.

The proposed work can be extended for classification of events in soccer game, so that the specific events like ―goal‖, ―penalty-shootout‖, ―free kick‖ etc., can be retrieved from the soccer clips based on user desire. In order to further improve accuracy, one can experiment multimodal cues for event retrieval in soccer clips. Proposed method only works on video cues, but video data is composed of multimodal information streams. Therefore multimodal information such as visual information; audio information and textual information can be used for better results to detect the events of sports video.

REFERENCES

[1 ] SUN Xing-hua and YANG Jing-yu, 2007―Inference and retrieval of soccer event‖, International Journal of Communication and Computer, Mar. 2007, Volume 4, No.3 (Serial No.28) ISSN1548-7709, USA

[2 ] J. Assfalg, M. Bertii, A. Del Bimbo, W. Nunziati, and P. Pala, 2002―Soccer highlights detection and recognition using HMMs,‖in proceedings of IEEE International Conference on Multimedia and Expo (ICME), 2002.

[3 ] V. Tovinkere and R. J. Qian, 2001―Detecting semantic events in soccer games: towards a complete solution,‖ in Proceedings of . IEEE International Conference on Multimedia and Expo (ICME)2001.

[4 ] Y. Rui, A. Gupta, and A. Acero,2000 ―Automatically extracting highlights for TV baseball programs,‖ in proceedings of Eighth ACM Conference on Multimedia, 2000.

[5 ] R. Leonardi and P. Migliorati,2003 ―Semantic indexing of multimedia documents,‖ in IEEE journal of Multimedia, vol. 9, no.2. [6 ] Yina Han, Guizhong Liu and Gerard Chollet., 2008―Goal event detection in broadcast soccer videos by combining heuristic rules with unsupervised fuzzy c-means algorithm‖ in proceedings of 10th International Conference on Control, Automation, Robotics and Vision Hanoi, Vietnam.

[7 ] Liu Huayong and Zhang Hui,2005 ―A Content-based broadcasted sports video retrieval system using multiple modalities‖ in proceedings of The Fifth International Conference on Computer and Information Technology, 2005. CIT 2005.

[8 ] Mohammed A. Refaey, Khaled M. Elsayed, Sanaa M. Hanaf and Larry S. Davis,2009 ‖Concurrent transition and shot detection in football videos usingfuzzy logic‖ in proceedings of 16th IEEE International Conference on Image Processing (ICIP), 2009 [9 ] Huang-Chia Shih and Chung-Lin Huang ,2005 ―Content-based

scalable sports video retrieval system‖, in proceedings of IEEE International Symposium on Circuits and Systems, ISCAS2005. .

Ind ex Na m e o f th e fil es. S ize in MB Fu zz y b a se d ev en t fr a m e d ete cte d n ea r g o a l p o st No o f S ig n if ica n t Ev en t Vid eo Co n str u cte d Ty p e o f ev en t

1 Video1 2.11MB 2 2 ―Goal‖

2 Video2 2.71 MB 2 2 ―Goal‖

3 Video3 2.19 MB 0 0 ―-―

9 Video9 2.21 MB 3 2

―Penalty Shootou

t‖

10 Video1

0 2.20 MB 3 3

[image:6.612.51.303.247.613.2]

(7)

International Journal of Emerging Technology and Advanced Engineering

814 [10 ]Guangsheng Zhao , 2008―Event-based soccer video retrieval with

interactive genetic algorithm‖, in proceedings of IEEE International Symposium on Information science and engineering.

[11 ]Shu-Ching Chen, Mei-Ling Shyu and Na Zhao, 2005,―An Enhanced query model for soccer video retrieval using temporal relationships‖, in proceedings of 21st International Conference on Data Mining. [12 ]David A. Sadlier and Noel E. O’Connor, 2005,―Event Detection in

the field of sports video using audio-visual features and a support vector machine‖, in IEEE Transaction on Circuits and Systems for Video Technology.

[13 ]Chen Jianyun, Li Yunhao, Wu Lingda and Lao Songyang, 2004―Semantic Event Detection in Soccer video by Integrating Multi-features using Bayesian Network‖. In Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing.