3D motion detection using neural networks

(1)

CHAPTER 1

CHAPTER 1 INTRODUCTION

INTRODUCTION

In

In vidvideo eo sursurveiveillallance, nce, vidvideo eo sigsignalnals s frofrom m mulmultiptiple le remremote ote loclocatiations ons areare displayed on several TV screens which are typically placed together in a control displayed on several TV screens which are typically placed together in a control room. In the so-called third generation surveillance systems (3GSS), all the parts of room. In the so-called third generation surveillance systems (3GSS), all the parts of the

the sursurveiveillanllance ce sysystestems ms wilwill l be be digdigitaital l and and conconseqsequenuentlytly, , digdigitaital l vidvideo eo wilwill l bebe tra

transmnsmitteitted d and and proprocescessedsed. . AddAdditiitionaonallylly, , in in 3G3GSS SS sosome me 'in'inteltelligligencence' e' has has to to bebe introduced to detect relevant events in the video signals in an automatic way. This introduced to detect relevant events in the video signals in an automatic way. This all

allows filterows filtering of ing of the irrelethe irrelevanvant t timtime e segsegmenments ts of of the the vidvideo eo seqsequenuences ces and theand the displaying on the TV screen only those segments that require the attention of the displaying on the TV screen only those segments that require the attention of the sur

surveiveillallance nce opeoperatorator. r. MoMotiotion n detdetectiection on is is a a basbasic ic opeoperatration ion in in the the selselectiection on of of sig

signifnificaicant nt segsegmenments ts of of the the vidvideo eo sigsignalnals. s. OncOnce e motmotion ion has has beebeen n detdetecteected, d, othother er features can be considered to decide whether a video signal has to be presented to the features can be considered to decide whether a video signal has to be presented to the surve

surveillancillance e operatoperator. If or. If the motion detection is the motion detection is perforperformed after med after the transmissthe transmission of ion of the video signals from the cameras to the control room, then all the bit streams have the video signals from the cameras to the control room, then all the bit streams have to be previously decompressed; this can be a very demanding operation, especially if to be previously decompressed; this can be a very demanding operation, especially if there are many cameras in the surveillance system. For this reason, it is interesting to there are many cameras in the surveillance system. For this reason, it is interesting to co

consnsidider er ththe e ususe e of of momotition on dedetetectiction on alalgogoririththms ms opopereratatining g in in ththe e cocompmpreresssseded (transform) domain.

(transform) domain.

In this thesis we present a motion detection algorithm in the compressed In this thesis we present a motion detection algorithm in the compressed domain with a low computational cost. In the following Section, we assume that domain with a low computational cost. In the following Section, we assume that video is compressed by using motion JPEG (MJPEG), i.e. each frame is individually video is compressed by using motion JPEG (MJPEG), i.e. each frame is individually JPEG compressed.

JPEG compressed.

Motion detection from a moving observer has been a very important Motion detection from a moving observer has been a very important te

techchniniquque e fofor r cocompmpututer er vivisision on apapplplicicatiationons. s. EsEspepeciciallally y in in recrecenent t yeyearars, s, fofor r autonomous driving systems and driver supporting systems, vision-based navigation autonomous driving systems and driver supporting systems, vision-based navigation method has received more and more attention worldwide.

(2)

One of its most important tasks is to detect the moving obstacles like cars, One of its most important tasks is to detect the moving obstacles like cars, bicycles or even pedestrians while the vehicle itself is running in a high speed. bicycles or even pedestrians while the vehicle itself is running in a high speed.

Met

Methodhods s of of imaimage ge difdiffereferencincing ng witwith h the the cleclear ar bacbackgkgrouround nd or or betbetweeween n adjadjacenacentt frames are well used for the motion detection. But when the observer is also moving, frames are well used for the motion detection. But when the observer is also moving, wh

whicich h leleadads s to to ththe e resresulult t of of cocontntininuouoususly ly chchanangiging ng babackckgrgrouound nd scscenene e in in ththee perspective projection image, it becomes more difficult to detect the real moving perspective projection image, it becomes more difficult to detect the real moving objects by differencing methods. To deal with this problem, many approaches have objects by differencing methods. To deal with this problem, many approaches have been proposed in recent years. Previous work in this area has been mainly in two been proposed in recent years. Previous work in this area has been mainly in two categories: 1) Using the difference of optical flow vectors between background and categories: 1) Using the difference of optical flow vectors between background and the moving objects, 2) calibrating the background displacement by using camera’s the moving objects, 2) calibrating the background displacement by using camera’s 3D motion analysis result

3D motion analysis result. . CalculCalculate the ate the opticoptical al flow and estimate the flow and estimate the flow vector’sflow vector’s reliability between adjacent frames. The major flow vector, which represents the reliability between adjacent frames. The major flow vector, which represents the motion of background, can be used to classify and extract the flow vectors of the real motion of background, can be used to classify and extract the flow vectors of the real moving objects. However, by reason of its huge calculation cost and its difficulty for moving objects. However, by reason of its huge calculation cost and its difficulty for determining the accurate flow vectors, it is still unavailable for real applications. To determining the accurate flow vectors, it is still unavailable for real applications. To ana

analyslysis is the the camcamera’era’s s 3D 3D motmotion ion and calibrand calibrate ate the the bacbackgkgrouround nd is is anoanothether r maimainn method for moving objects detection. For on-board camera’s motion analysis, many method for moving objects detection. For on-board camera’s motion analysis, many mot

motionion-det-detectiecting ng algalgoriorithmthms s havhave e beebeen n proproposposed ed whwhich ich alwalways ays depdepend end on on thethe prev

previous recognitioious recognition n resultresults s like like road road lane-mlane-marks arks and horizon and horizon disapdisappoinpointing. Theseting. These methods show some good performance in accuracy and efficiency because of their methods show some good performance in accuracy and efficiency because of their det

detailailed ed anaanalyslysis is of of roaroad d strstructucture ure and and meameasursured ed vehvehiclicle e loclocomoomotiotion, n, whiwhich ch is,is, howev

however, er, compucomputationtationally ally expenexpensive sive and and over-over-dependepended ded upon road upon road featurefeatures s likelike lane-marks, and therefore lead to unsatisfied result when lane mark is covered by lane-marks, and therefore lead to unsatisfied result when lane mark is covered by other vehicles or not exist at all. Compare with these previous works, a new method other vehicles or not exist at all. Compare with these previous works, a new method of moving objects detectio

of moving objects detection n from an on-board camera is from an on-board camera is presenpresented in ted in this paper. Tothis paper. To deal with the background-change problem, our method uses camera’s 3D motion deal with the background-change problem, our method uses camera’s 3D motion analysis results to calibrate the background scene. With pure points matching and the analysis results to calibrate the background scene. With pure points matching and the introduction of camera’s Focus of Expansion (FOE), our method is able to determine introduction of camera’s Focus of Expansion (FOE), our method is able to determine camera’s rotation and translation parameters theoretically by using only three pairs of camera’s rotation and translation parameters theoretically by using only three pairs of matching points between adjacent frames, which make it faster and more efficient for matching points between adjacent frames, which make it faster and more efficient for real-time applications.

(3)

A neural network, also known as a parallel distributed processing network, A neural network, also known as a parallel distributed processing network, is a computing paradigm that is loosely modeled after cortical structures of the brain. is a computing paradigm that is loosely modeled after cortical structures of the brain. It consists of interconnected processing elements called nodes or neurons that work It consists of interconnected processing elements called nodes or neurons that work together to produce an output function. The output of a neural network relies on the together to produce an output function. The output of a neural network relies on the cooperation of the individual neurons within the network to operate. Processing of cooperation of the individual neurons within the network to operate. Processing of information by neural networks is characteristically done in parallel rather than in information by neural networks is characteristically done in parallel rather than in series (or sequentially) as in earlier binary computers or Von Neumann machines. series (or sequentially) as in earlier binary computers or Von Neumann machines. Since it relies on its member neurons collectively to perform its function, a unique Since it relies on its member neurons collectively to perform its function, a unique property of a neural network is that it can still perform its overall function even if property of a neural network is that it can still perform its overall function even if

some of the

some of the neuroneurons are ns are not functionot functioningning. . In other words it In other words it is robust to tolerate error is robust to tolerate error or failure. All neural networks take numeric input and produce numeric output. The or failure. All neural networks take numeric input and produce numeric output. The transfer function of a unit is typically chosen so that it can accept input in any range, transfer function of a unit is typically chosen so that it can accept input in any range, and produces output in a strictly limited range (it has a squashing effect).

and produces output in a strictly limited range (it has a squashing effect).

An artificial neural network (ANN), also called a simulated neural network An artificial neural network (ANN), also called a simulated neural network (SNN) or commonly just neural network (NN) is

(SNN) or commonly just neural network (NN) is an interconnected group of artificialan interconnected group of artificial neurons that uses a mathematical or computational model for information processing neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is an based on a connectionist approach to computation. In most cases an ANN is an

adapt

adaptive system ive system that changes its that changes its structstructure based ure based on external or on external or interninternal al informinformationation that flows through the network.

that flows through the network.

There are different topologies of neural networks that may be employed for There are different topologies of neural networks that may be employed for time series modeling. In our investigation we used radial basis function networks time series modeling. In our investigation we used radial basis function networks whi

which ch havhave e shoshown wn conconsidsideraberably ly betbetter ter scascalinling g proproperpertieties, s, whewhen n incincreasreasing ing thethe number of hidden units, than networks with

number of hidden units, than networks with sigmoid activation function.sigmoid activation function.

RBF networks were introduced into the neural network literature by Broom RBF networks were introduced into the neural network literature by Broom he

headad/L/Lowowe e anand d PoPoggggioio/G/Girirososi i in in ththe e lalate te 19198080s. s. ThThe e RBRBF F nenetwtworork k momodedel l isis motivated by the locally tuned response observed in biologic neurons, e.g. in the motivated by the locally tuned response observed in biologic neurons, e.g. in the vi

visusual al or or in in ththe e auaudiditotory ry sysyststemem. . RBRBFs Fs hahave ve bebeen en ststududieied d in in mumultltivivarariaiatete approximation theory, particularly in the field of function interpolation. The RBF approximation theory, particularly in the field of function interpolation. The RBF neural network model is an alternative to multilayer perceptron which is perhaps the neural network model is an alternative to multilayer perceptron which is perhaps the

(4)

most often used neural network architecture. A radial basis function network (RBF), most often used neural network architecture. A radial basis function network (RBF), therefore, has a hidden layer of radial units, each actually modeling a Gaussian therefore, has a hidden layer of radial units, each actually modeling a Gaussian response surface. Since these functions are nonlinear, it is not actually necessary to response surface. Since these functions are nonlinear, it is not actually necessary to have more than one hidden layer to model any shape of function: sufficient radial have more than one hidden layer to model any shape of function: sufficient radial units will always be

units will always be enough to model any function.enough to model any function.

In surveilla

In surveillance nce systsystem estimation of em estimation of motiomotion n is of is of great importagreat importance, whichnce, which enables the various types of operations to be performed on the detected object. When enables the various types of operations to be performed on the detected object. When using motion estimation, an assumption is made that the objects in the scene have using motion estimation, an assumption is made that the objects in the scene have only translational motion. This assumption holds as long as there is no camera pan, only translational motion. This assumption holds as long as there is no camera pan, zoom, changes in luminance, or rotational motion (quite an assumption!).

zoom, changes in luminance, or rotational motion (quite an assumption!).

After the process of estimation, the detected motion has to be extracted. With After the process of estimation, the detected motion has to be extracted. With the obtained boundary, two objects (with background) can then be extracted from the obtained boundary, two objects (with background) can then be extracted from two image frames (both current image frame and previous image frame). Extracting two image frames (both current image frame and previous image frame). Extracting the movin

the moving g objobject ect frofrom m its backgits backgrouround nd can be can be dondone e by by the the edgedge e enhenhancancemeementnt network and the background remover.

network and the background remover.

In algorithm level, complexity, regularity and precision are main factors that In algorithm level, complexity, regularity and precision are main factors that directly affect the power consumed in extracting an algorithm for motion estimation. directly affect the power consumed in extracting an algorithm for motion estimation. Concurrency and modularity are the requirements on algorithms that are intended to Concurrency and modularity are the requirements on algorithms that are intended to ex

exececutute e on on lolow w popowewer r ararchchititecectuturere. . ThThis is prprojojecect t aiaims ms to to rereduduce ce ththe e popowewer r consumption of motion estimation at algorithm level and architectural level by using consumption of motion estimation at algorithm level and architectural level by using neural network concept.

(5)

1 1..1

1 P

PR

RO

OB

BL

LE

EM S

M ST

TA

AT

TE

EM

ME

EN

NT

T

The goals for this thesis have been the following. The goals for this thesis have been the following.

On

One e gogoal al hahas s bebeen en to to cocompmpilile e an an inintrtrododucuctition on to to ththe e momotition on dedetectectiotionn algorithms. There exist a number of studies but complete reference on real time algorithms. There exist a number of studies but complete reference on real time motion detection is not as common .we have collected materials from journals, papers motion detection is not as common .we have collected materials from journals, papers and confer

and conferences and proences and proposeposed d approapproach that can be best to implemach that can be best to implement a real timeent a real time motion detection.

motion detection.

Another goal has been to search for algorithms that can be used to implement Another goal has been to search for algorithms that can be used to implement the RBF neural network.

the RBF neural network.

A third goal is to evaluate their performance with regard to motion detected. A third goal is to evaluate their performance with regard to motion detected. Th

Thesese e prpropoperertities es werwere e chchososen en bebecaucause se ththey ey hahave ve ththe e grgreaeatetest st imimpapact ct on on ththee implementation effort.

implementation effort.

A final goal has been to design and implement an algorithm including object A final goal has been to design and implement an algorithm including object extraction. This should be done in high level language or matlab. The source code extraction. This should be done in high level language or matlab. The source code should be easy to understand so that it can serve as a reference on the standard for should be easy to understand so that it can serve as a reference on the standard for designers that need to implement real time motion detection.

(6)

CHAPTER 2

CHAPTER 2 OVERVIEW OF NEURAL NETWORKS

OVERVIEW OF NEURAL NETWORKS

Neural network theory is sometimes used to refer to a branch of computational Neural network theory is sometimes used to refer to a branch of computational

sc

scieiencnce e ththat at ususes es neneurural al nenetwtwororks ks as as momodedels ls to to sisimumulalate te or or ananalalyzyze e cocompmplexlex phenomena and/or study the principles of operation of neural networks analytically. phenomena and/or study the principles of operation of neural networks analytically.

It

It addaddresresses ses proprobleblems ms simsimilar ilar to to artiartificficial ial intintellelligenigence ce (AI(AI) ) excexcept ept thathat t AI AI useusess traditional computational

traditional computational algorithmsalgorithms to solve problems whereas neural networks useto solve problems whereas neural networks use 'n

'netetwoworkrks s of of agagenentsts' ' (s(sofoftwtware are or or hahardrdwarware e enentititieties s lilinknked ed totogegeththerer) ) as as ththee computational architecture to solve problems. Neural networks are trainable systems computational architecture to solve problems. Neural networks are trainable systems that can "learn" to solve complex problems from a set of exemplars and generalize that can "learn" to solve complex problems from a set of exemplars and generalize the

the "acq"acquiruired ed knoknowledwledge" ge" to to sosolve lve unfunforeoreseeseen n proprobleblems ms as as in in stostock ck marmarket ket andand environmental prediction. i.e., they are self-adaptive systems.

environmental prediction. i.e., they are self-adaptive systems. Tr

Tradadititioionanalllly, y, ththe e teterm rm neneurural al nenetwtworork k hahas s bebeen en usused ed to to refrefer er to to a a nenetwtworork k of of biological neurons. In modern usage, the term is often used to refer to artificial biological neurons. In modern usage, the term is often used to refer to artificial neural networks, which are composed of artificial neurons or nodes. Thus the term neural networks, which are composed of artificial neurons or nodes. Thus the term 'Neural Network' has two distinct

'Neural Network' has two distinct connotations:connotations: 1.

1. BioBiologlogical neuical neural netral networworks are madks are made up of e up of reareal biolol biologicgical neuroal neurons that arens that are con

connecnected ted or or funfunctioctionalnally-ly-relarelated ted in in the the perperiphipheral eral nernervovous us syssystem tem or or thethe central nervous system. In the field of neuroscience, they are often identified central nervous system. In the field of neuroscience, they are often identified as

as grgrououps ps of of neneururonons s ththat at peperforform rm a a spspeciecifific c phphysysioiolologigical cal fufuncnctition on inin laboratory analysis.

laboratory analysis.

2.

2. ArtArtifiificial neurcial neural netwoal networks are made up of rks are made up of intintercoerconnennectincting g artiartificficial neurial neuronsons (usually simplified neurons) designed to model (or mimic) some properties of (usually simplified neurons) designed to model (or mimic) some properties of biological neural networks. Artificial neural networks can be used to model biological neural networks. Artificial neural networks can be used to model

the

the modmodes es of of opeoperatiration on of of biobiologlogicaical l neuneural ral netnetworworks, ks, whewhereas reas cogcognitnitiveive models are theoretical models that mimic cognitive brain functions without models are theoretical models that mimic cognitive brain functions without

(7)

necessarily using neural networks while artificial intelligence are well-crafted necessarily using neural networks while artificial intelligence are well-crafted alg

algoriorithmthms s thathat t solsolve ve spspecifecific ic intintellelligeigent nt proprobleblems ms witwithouhout t ususing ing neuneuralral network as the computational architecture.

network as the computational architecture.

2.1 2.1 The brain, neural networks and computers

The brain, neural networks and computers

While it is accepted by most scientists that the brain is a type of computer, it While it is accepted by most scientists that the brain is a type of computer, it is a computer with a vastly different architecture to the computers that most of us are is a computer with a vastly different architecture to the computers that most of us are fa

famimilialiar r wiwithth. . ThThe e brbrain ain is is mamassssivivelely y paparalrallellel, , eveven en momore re so so ththan an adadvavancnceded multiprocessor computers. This means that simulating the behavior of a brain on multiprocessor computers. This means that simulating the behavior of a brain on traditional computer hardware is necessarily slow and inefficient.

traditional computer hardware is necessarily slow and inefficient.

Neural networks, as used in artificial intelligence, have traditionally been Neural networks, as used in artificial intelligence, have traditionally been viewed as simplified models of neural processing in the brain, even though the viewed as simplified models of neural processing in the brain, even though the relatio

relation n betweebetween n this model and this model and brain biologbrain biological architecture is ical architecture is very much very much debatdebated.ed. To answer this

To answer this questquestion, David Marr ion, David Marr has proposhas proposed various levels ed various levels of analysis whichof analysis which p

prorovivide de us us wiwith th a a plplauausisiblble e ananswswer er fofor r ththe e rorole le of of neneurural al nenetwtwororks ks in in ththee understanding of human cognitive

understanding of human cognitive functioning.functioning.

The question of what is the degree of complexity and the properties that The question of what is the degree of complexity and the properties that individual neural elements should have in order to reproduce something resembling individual neural elements should have in order to reproduce something resembling animal intelligence is a subject of current research in theoretical neuroscience.

animal intelligence is a subject of current research in theoretical neuroscience.

Historically computers evolved from von Neumann architecture, based on Historically computers evolved from von Neumann architecture, based on seq

sequenuential tial proprocescessinsing g and and exeexecutcution ion of of expexplicilicit t insinstrutructictionsons. . On On the the othother er hanhandd origins of neural networks are based on efforts to model information processing in origins of neural networks are based on efforts to model information processing in biological systems, which may rely largely on parallel processing as well as implicit biological systems, which may rely largely on parallel processing as well as implicit instructions based on recognition of patterns of 'sensory' input from external sources. instructions based on recognition of patterns of 'sensory' input from external sources. In other

In other wordwords, rather s, rather than sequentithan sequential al procesprocessing and execution, at sing and execution, at their very their very heart,heart, neural networks are complex statistic processors.

(8)

2.2 2.2 Artificial

Artificial Neural

Neural

networks

An artificial neural network (ANN), also called a simulated neural network An artificial neural network (ANN), also called a simulated neural network (SNN) or commonly just neural network (NN) is

(SNN) or commonly just neural network (NN) is an interconnected group of artificialan interconnected group of artificial neurons that uses a mathematical or computational model for information processing neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is an based on a connectionist approach to computation. In most cases an ANN is an

adapt

adaptive system ive system that changes its that changes its structstructure based ure based on external or on external or interninternal al informinformationation that flows through the network.

that flows through the network. In

In momore re prpractacticical al terterms ms neneurural al nenetwtwororks ks are are nonon-n-lilinenear ar ststatatisistiticacal l dadatata modeling tools. They can be used to

modeling tools. They can be used to model complex relationships between inputs andmodel complex relationships between inputs and outputs or to find patterns in data.

outputs or to find patterns in data.

2.3 Background

An

An arartitificficiaial l neneurural al nenetwtworork k ininvovolvlves es a a nenetwtworork k of of sisimpmple le prprococesessisingng elements (neurons) which can exhibit complex global behavior, determined by the elements (neurons) which can exhibit complex global behavior, determined by the connections between the processing elements and element parameters. One classical connections between the processing elements and element parameters. One classical type of artificial neural network is the Hopfield net.

type of artificial neural network is the Hopfield net. In

In a a neneurural al nenetwtworork k momodedel, l, sisimpmple le nonodedes s (c(calalled led vavaririououslsly y "n"neueuroronsns",", "neurodes", "PEs" ("processing elements") or "units") are connected together to form "neurodes", "PEs" ("processing elements") or "units") are connected together to form a network of nodes — hence the term "neural network". While a neural network does a network of nodes — hence the term "neural network". While a neural network does not have to be adaptive per se, its practical use comes with algorithms designed to not have to be adaptive per se, its practical use comes with algorithms designed to alter the strength (weights) of the connections in the network to produce a desired alter the strength (weights) of the connections in the network to produce a desired signal flow.

signal flow. In

In momodedern rn sosoftftwaware re imimplplememenentatatitionons s of of arartitifificiacial l neneurural al nenetwtwororks ks ththee appro

approach inspired by biology has more ach inspired by biology has more or less or less been abandonbeen abandoned for ed for a a more practicalmore practical appro

approach based ach based on statistics and on statistics and signasignal l proceprocessingssing. . In some In some of these of these systsystems neuralems neural ne

netwtwororks ks or or papartrts s of of neneurural al nenetwtwororks ks (s(sucuch h as as artartifificicial ial neneururonons) s) are are usused ed asas components in larger systems that combine both adaptive and non-adaptive elements. components in larger systems that combine both adaptive and non-adaptive elements.

(9)

2.4 Models

Neural network models in artificial intelligence are usually referred to as Neural network models in artificial intelligence are usually referred to as artificial neural networks (ANN); these are essentially simple mathematical models artificial neural networks (ANN); these are essentially simple mathematical models d

deeffiinniinng g a a ffuunnccttiioonn . . EEaacch h ttyyppe e oof f AANNN N mmooddeel l ccoorrrreessppoonndds s tto o aa classclass of of such functions.

such functions.

Fig 1 Artificial Neural Network Fig 1 Artificial Neural Network

(10)

Fig 2 A complex neural network Fig 2 A complex neural network

2.5 Employing artificial neural

2.5 Employing artificial neural networks

networks

Perhap

Perhaps the s the greategreatest advantagst advantage of e of ANN is their ability to ANN is their ability to be used as an be used as an arbitrarbitraryary funct

function ion approapproximatiximation on mechanmechanism ism which 'learns' from which 'learns' from obserobserved ved data. data. HoweHowever,ver, using them is not so straightforward and a relatively good understanding of the using them is not so straightforward and a relatively good understanding of the underlying theory is essential.

underlying theory is essential.

•

• ChChoioice ce of of momodedel: l: ThThis is wiwill ll dedepepend nd on on ththe e dadata ta reprepreresesentntatatioion n anand d ththee

application. Overly complex models tend to lead to problems with learning. application. Overly complex models tend to lead to problems with learning.

•

• LeLearnarnining g alalgogoririththm: m: ThTherere e arare e nunumemerorous us trtradadeoeoffffs s bebetwtween een leleararniningng

alg

algoriorithmthms. s. AlmAlmost ost any any algalgoriorithm thm wilwill l worwork k welwell l witwith h the the corcorrecrect t hyphyper er param

parameters for eters for trainitraining on ng on a a particuparticular lar fixed dataset. Howevefixed dataset. However r selectiselecting andng and tuning an algorithm for training on unseen data requires a significant amount tuning an algorithm for training on unseen data requires a significant amount of experimentation.

of experimentation.

•

• Robustness: If the model, cost function and learning algorithm are selectedRobustness: If the model, cost function and learning algorithm are selected

appropriately the resulting ANN can be extremely robust. appropriately the resulting ANN can be extremely robust.

With the correct implementation ANN can be used naturally in online learning and With the correct implementation ANN can be used naturally in online learning and large dataset applicatio

(11)

local dependencies exhibited in the structure allows for fast, parallel implementations local dependencies exhibited in the structure allows for fast, parallel implementations in hardware.

in hardware.

2.6 Types of

2.6 Types of neural networks

neural networks

2.6.1 Feed forward neural network

The feed forward neural networks are the first and arguably simplest type of The feed forward neural networks are the first and arguably simplest type of artificial neural networks devised. In this network, the information moves in only one artificial neural networks devised. In this network, the information moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network.

output nodes. There are no cycles or loops in the network.

2.6.2 Single-layer perceptron

Th

The e earearliesliest t kinkind d of of neuneural ral netnetworwork k is is a a sinsinglegle-lay-layer er perpercepceptrotron n netnetworwork,k, which consists of a single layer of output nodes; the inputs are fed directly to the which consists of a single layer of output nodes; the inputs are fed directly to the outputs via a series of weights. In this way it can be considered the simplest kind of outputs via a series of weights. In this way it can be considered the simplest kind of feed-forward network. The sum of the products of the weights and the inputs is feed-forward network. The sum of the products of the weights and the inputs is calculated in each node, and if the value is above some threshold (typically 0) the calculated in each node, and if the value is above some threshold (typically 0) the neu

neuron fires ron fires and takes and takes the activathe activated ted valvalue ue (ty(typicpically 1); ally 1); othotherwerwise ise it it taktakes es thethe deactivated value (typically -1). Neurons with this kind of activation function are deactivated value (typically -1). Neurons with this kind of activation function are also called McCulloch-Pitts neurons or

also called McCulloch-Pitts neurons or threshold neurons.threshold neurons. A

A pepercrcepeptrtron on cacan n be be crcreaeateted d ususining g anany y vavalulues es fofor r ththe e acactitivavateted d anandd de

deactactivivateated d ststatates es as as lolong ng as as ththe e ththresreshohold ld vavalulue e lilies es bebetwtweeeen n ththe e twtwo. o. MoMostst perceptrons have outputs of 1 or -1 with a threshold of 0 and there is some evidence perceptrons have outputs of 1 or -1 with a threshold of 0 and there is some evidence that such networks can be trained more quickly than networks created from nodes that such networks can be trained more quickly than networks created from nodes with different activation and deactivation values.

with different activation and deactivation values.

Perceptrons can be trained by a simple learning algorithm that is usually Perceptrons can be trained by a simple learning algorithm that is usually called the delta

called the delta rule. It rule. It calculcalculates the ates the errors between calculerrors between calculated output and ated output and sampsamplele output data, and uses this to create an adjustment to the weights, thus implementing a output data, and uses this to create an adjustment to the weights, thus implementing a form of gradient descent.

(12)

Sin

Singlegle-un-unit it perpercepceptrotrons ns are are onlonly y capcapablable e of of learlearninning g linlinearlearly y sepseparabarablele patte

patterns; in rns; in 1969 in 1969 in a a famoufamous s monomonograph entitlegraph entitled d PercepPerceptrons Marvin Minsky andtrons Marvin Minsky and Seymour Papert showed that it was impossible for a single-layer perceptron network Seymour Papert showed that it was impossible for a single-layer perceptron network to learn an XOR function.

to learn an XOR function.

2.6.3 Multilayer layer perceptron

This class of networks consists of multiple layers of computational units, This class of networks consists of multiple layers of computational units, usually interconnected in a feed-forward way. Each neuron in one layer has directed usually interconnected in a feed-forward way. Each neuron in one layer has directed connections to the neurons of the subsequent layer. In many applications the units of connections to the neurons of the subsequent layer. In many applications the units of these networks apply a sigmoid function as an activation function.

these networks apply a sigmoid function as an activation function. The universal approxi

The universal approximatiomation n theoretheorem m for neural for neural netwonetworks states rks states that everythat every continuous function that maps intervals of real numbers to some output interval of continuous function that maps intervals of real numbers to some output interval of real numbers can be approximated arbitrarily closely by a multi-layer perceptron real numbers can be approximated arbitrarily closely by a multi-layer perceptron with just one hidden layer. This result holds only for restricted classes of activation with just one hidden layer. This result holds only for restricted classes of activation functions, e.g. for the sigmoid functions.

functions, e.g. for the sigmoid functions.

Multi-layer networks use a variety of learning techniques, the most popular Multi-layer networks use a variety of learning techniques, the most popular be

being ing bacback-pk-propropagaagatiotion. n. HerHere e the the ououtputput t valvalues ues are are comcomparpared ed witwith h the the corcorrectrect an

answswer er to to cocompmputute e ththe e vavalulue e of of sosome me prprededefefinined ed errerroror-fu-funcnctitionon. . By By vavarioriousus techniques the error is then

techniques the error is then fed back through the network. Using this information, thefed back through the network. Using this information, the algorithm adjusts the weights of each connection in order to reduce the value of the algorithm adjusts the weights of each connection in order to reduce the value of the error function by some small amount. After repeating this process for a sufficiently error function by some small amount. After repeating this process for a sufficiently large number of training cycles the network will usually converge to some state large number of training cycles the network will usually converge to some state where the error of the calculations is small. In this case one says that the network has where the error of the calculations is small. In this case one says that the network has learned a certain target function. To adjust weights properly one applies a general learned a certain target function. To adjust weights properly one applies a general met

methohod d for for nonnon-li-lineanear r optoptimiimizatization on thathat t is is calcalled led gragradiedient nt desdescencent. t. For For thithis, s, thethe derivative of the error function with respect to the network weights is calculated and derivative of the error function with respect to the network weights is calculated and the weights are then changed such that the error decreases (thus going downhill on the weights are then changed such that the error decreases (thus going downhill on the surface of the error function). For this reason back-propagation can only be the surface of the error function). For this reason back-propagation can only be applied on networks with differentiable.

(13)

Fig 3 XOR perceptron Fig 3 XOR perceptron

A three layer Perceptron net capable of calculating XOR. The numbers within A three layer Perceptron net capable of calculating XOR. The numbers within the

the percepperceptrons represent each trons represent each percepperceptronstrons' ' explicexplicit it thresthreshold. The hold. The numbnumbers ers thatthat annotate arrows represent the weight of the inputs. This net assumes that if the annotate arrows represent the weight of the inputs. This net assumes that if the threshold is not reached, zero (not -1) is output. Note that the bottom layer of inputs threshold is not reached, zero (not -1) is output. Note that the bottom layer of inputs is not always considered a real perceptron layer.

is not always considered a real perceptron layer.

2.6.4 Radial basis function (RBF) network

Ra

Radidial al BaBasisis s FuFuncnctitionons s are are popowewerfrful ul tetechchniniquques es fofor r ininteterprpololatiation on inin multidimensional space. A RBF is a function which has built into a distance criterion multidimensional space. A RBF is a function which has built into a distance criterion with respect to a centre. Radial basis functions have been applied in the area of with respect to a centre. Radial basis functions have been applied in the area of neura

neural l netwonetworks where they rks where they may be may be used as a used as a replacereplacement for ment for the sigmoidathe sigmoidal l hiddhiddenen layer transfer characteristic in multi-layer perceptrons.

layer transfer characteristic in multi-layer perceptrons.

2.6.5 Echo State Network

The Echo State Network (ESN) is a recurrent neural network with a sparsely The Echo State Network (ESN) is a recurrent neural network with a sparsely connected random hidden layer. The weights of output neurons are the only part of connected random hidden layer. The weights of output neurons are the only part of the network that can change and be learned. ESN are good to (re)produce temporal the network that can change and be learned. ESN are good to (re)produce temporal patterns.

(14)

2.6.6 Stochastic neural networks

A stochastic neural network differs from a regular neural network in the fact A stochastic neural network differs from a regular neural network in the fact that it introduces random variations into the network. In a probabilistic view of that it introduces random variations into the network. In a probabilistic view of neural networks, such random variations can be viewed as a form of statistical neural networks, such random variations can be viewed as a form of statistical sampling, such as Monte Carlo sampling.

sampling, such as Monte Carlo sampling.

2.6.7 Neuro-fuzzy networks

A neuro-fuzzy network is a fuzzy inference system in the body of an artificial A neuro-fuzzy network is a fuzzy inference system in the body of an artificial neural network. Depending on the FIS type, there are several layers that simulate the neural network. Depending on the FIS type, there are several layers that simulate the processes involved in a fuzzy inference like fuzzification, inference, aggregation and processes involved in a fuzzy inference like fuzzification, inference, aggregation and defuzzification. Embedding an FIS in a general structure of an ANN has the benefit defuzzification. Embedding an FIS in a general structure of an ANN has the benefit of using available ANN training methods to find the parameters of a fuzzy system. of using available ANN training methods to find the parameters of a fuzzy system.

(15)

CHAPTER 3

CHAPTER 3 RBF NETWORK

RBF NETWORK

3.1 Radial Functions

Radial functions are a special class of function. Their characteristic feature is that Radial functions are a special class of function. Their characteristic feature is that their response decreases (or increases) monotonically with distance from a central their response decreases (or increases) monotonically with distance from a central point. The centre, the distance scale, and the precise shape of the radial function are point. The centre, the distance scale, and the precise shape of the radial function are parameters of the model, all fixed if it is linear.

parameters of the model, all fixed if it is linear.

A typical radial function is the Gaussian which, in the case of a scalar input, is A typical radial function is the Gaussian which, in the case of a scalar input, is

Its parameters are its centre

Its parameters are its centre cc and its radius r. The figure illustrates a Gaussian RBFand its radius r. The figure illustrates a Gaussian RBF with centre

with centre cc = 0 and radius r = 1.= 0 and radius r = 1.

A Gaussian RBF monotonically decreases with distance from the centre. In contrast, A Gaussian RBF monotonically decreases with distance from the centre. In contrast, a multiquadric RBF which, in the case of scalar input, is

a multiquadric RBF which, in the case of scalar input, is

monotonically increases with distance from the centre. Gaussian-like RBFs are local monotonically increases with distance from the centre. Gaussian-like RBFs are local (give a significant response only in a neighbourhood near the centre) and are more (give a significant response only in a neighbourhood near the centre) and are more commonly used than multiquadric-type RBFs which have

commonly used than multiquadric-type RBFs which have a global response.a global response.

3.2 Radial Networks

A RBF is a function which has built into a distance criterion with respect to a A RBF is a function which has built into a distance criterion with respect to a centre. Radial basis functions have been applied in the area of neural networks where centre. Radial basis functions have been applied in the area of neural networks where th

they ey mamay y be be usused ed as as a a rereplplacacememenent t fofor r ththe e sisigmgmoioidadal l hihidddden en lalayeyer r trtranansfsfer er characteristic in multi-layer perceptrons. RBF networks have 2 layers of processing: characteristic in multi-layer perceptrons. RBF networks have 2 layers of processing: In the first, input is mapped onto each RBF in the 'hidden' layer. The RBF chosen is In the first, input is mapped onto each RBF in the 'hidden' layer. The RBF chosen is us

usuaualllly y a a GaGausussisianan. . In In reregrgresessision on prproboblemlems s ththe e ououtptput ut lalayeyer r is is ththen en a a lilinenear ar co

(16)

interpretation of this output layer value

interpretation of this output layer value is the same as a ris the same as a regression model in statistics.egression model in statistics. In classification problems the output layer is typically a sigmoid function of a linear In classification problems the output layer is typically a sigmoid function of a linear co

combmbininatatioion n of of hihidddden en lalayeyer r vavalulueses, , rereprpresesenentiting ng a a popoststererioior r prprobobababililitity.y. Performance in both cases is often improved by shrinkage techniques, known as Performance in both cases is often improved by shrinkage techniques, known as ridge regression in classical statistics and known to correspond to a prior belief in ridge regression in classical statistics and known to correspond to a prior belief in sma

small ll parparameameter ter valvalues ues (an(and d thethereforefore re smosmooth oth outoutput put funfunctioctions) ns) in in a a BayBayesiesianan framework.

framework.

RBF network

RBF networks have s have the advantagthe advantage e of not of not suffersuffering from local ing from local minima in theminima in the same way as multi-layer perceptrons. This is because the only parameters that are same way as multi-layer perceptrons. This is because the only parameters that are adjusted in the learning process are the linear mapping from hidden layer to output adjusted in the learning process are the linear mapping from hidden layer to output layer. Lineari

layer. Linearity ensures that the ty ensures that the error surface is quadratic and therefore has error surface is quadratic and therefore has a singlea single easily found minimum. In regression problems this can be found in one matrix easily found minimum. In regression problems this can be found in one matrix ope

operatrationion. . In In claclassissificaficatiotion n proprobleblems ms the the fixfixed ed nonnon-lin-lineariearity ty intintrodroduceuced d by by thethe sigmoid output function is most efficiently dealt with using iterated reweighed least sigmoid output function is most efficiently dealt with using iterated reweighed least squares.

squares.

RBF networks have the disadvantage of requiring good coverage of the input RBF networks have the disadvantage of requiring good coverage of the input space by radial basis functions. RBF centers are determined with reference to the space by radial basis functions. RBF centers are determined with reference to the distribution of the input data, but without reference to the prediction task. As a result, distribution of the input data, but without reference to the prediction task. As a result, rep

represresententatiationaonal l resresourources ces may be may be waswasted ted on on areaareas s of of the the inpinput ut spaspace ce thathat t areare irrelevant to the learning task. A common solution is to associate each data point irrelevant to the learning task. A common solution is to associate each data point with its own centre, although this can make the linear system to be solved in the final with its own centre, although this can make the linear system to be solved in the final layer rather large, and requires shrinkage techniques to avoid over fitting.

layer rather large, and requires shrinkage techniques to avoid over fitting.

Associating each input datum with an RBF leads naturally to kernel methods Associating each input datum with an RBF leads naturally to kernel methods such as Support Vector Machines and Gaussian Processes (the RBF is the kernel such as Support Vector Machines and Gaussian Processes (the RBF is the kernel function). All three approaches use a non-linear kernel function to project the input function). All three approaches use a non-linear kernel function to project the input data into a space where the learning problem can be solved using a linear model. data into a space where the learning problem can be solved using a linear model. Like Gaussian Processes, and unlike SVMs, RBF networks are typically trained in a Like Gaussian Processes, and unlike SVMs, RBF networks are typically trained in a Maximum Likelihood framework by maximizing the probability (minimizing the Maximum Likelihood framework by maximizing the probability (minimizing the error) of the data under the model. SVMs take a different approach to avoiding over error) of the data under the model. SVMs take a different approach to avoiding over fitting by maximizing instead a margin. RBF networks are outperformed in most fitting by maximizing instead a margin. RBF networks are outperformed in most

(17)

cl

clasassisifificacatiotion n apapplplicicatatioions ns by by SVSVMsMs. . In In regregresressision on apapplplicicatiationons s ththey ey cacan n bebe competitive when the dimensionality of the input space is relatively small.

competitive when the dimensionality of the input space is relatively small.

3.3 RBF Architecture

Artificial networks typically have three layers: an input layer, a hidden layer Artificial networks typically have three layers: an input layer, a hidden layer with a non-linear RBF activation function and a linear output layer. The output, with a non-linear RBF activation function and a linear output layer. The output,

, of the network is thus , of the network is thus

Where

Where N N is the number of neurons in the hidden layer,is the number of neurons in the hidden layer, ccii is the center vector for is the center vector for neuron

neuron ii, and, and aaiiare the weights of the linear output neuron. In the basic form all inputare the weights of the linear output neuron. In the basic form all input are connected to each hidden neuron. The norm is typically taken to be the Euclidean are connected to each hidden neuron. The norm is typically taken to be the Euclidean distance and the basis function is taken to be Gaussian.

distance and the basis function is taken to be Gaussian.

..

T

Thhe Ge Gaauussssiiaan bn baassiis fs fuunnccttiioonns as arre le looccaal il in tn thhe se seennsse te thhaatt .. Changing parameters of one neuron has only a small effect for input values that are Changing parameters of one neuron has only a small effect for input values that are far away from the center of that neuron.

far away from the center of that neuron. RBF

RBF netwonetworks rks are are univuniversal ersal approapproximateximates s on on a a compacompact ct subssubset et of of . . This This meansmeans that a RBF network with enough hidden neurons can approximate any continuous that a RBF network with enough hidden neurons can approximate any continuous function with arbitrary precision.

function with arbitrary precision. The weights

The weights aaii,, , an, and β are dd β are detetermerminined ied in a man a mannnner ther that opat optitimimizezes ths the fit be fit betetweweenen and the data.

(18)

Fig 4 Architecture of a radial basis function network. Fig 4 Architecture of a radial basis function network.

3.4 Training

In a RBF network there are three types of parameters that need to be chosen In a RBF network there are three types of parameters that need to be chosen to adapt the network for a particular task: the center vectors

to adapt the network for a particular task: the center vectors ccii, the output weights, the output weights wwii,, an

and d ththe e RBRBF F wiwidtdth h paparamrameteters ers ββii. In the sequential training of the weights are. In the sequential training of the weights are updated at each time step as data streams in.

updated at each time step as data streams in.

For some tasks it makes sense to define an objective function and select the For some tasks it makes sense to define an objective function and select the parameter values that minimize its value. The most common objective function is the parameter values that minimize its value. The most common objective function is the

least squares function least squares function

Where, Where,

(19)

We have explicitly included the dependence on the weights. Minimization of the We have explicitly included the dependence on the weights. Minimization of the least squares objective function by optimal choice of weights optimizes accuracy of least squares objective function by optimal choice of weights optimizes accuracy of fit.

fit.

3.5 Interpolation

RBF

RBF netwonetworks rks can can be be used used to to interpinterpolate olate a a functfunction ion when when thethe v

vaalluuees s oof f tthhaat t ffuunnccttiioon n aarre e kknnoowwn n oon n ffiinniitte e nnuummbbeer r oof f ppooiinnttss:: . Taking the known points

. Taking the known points x xii to be the centers of theto be the centers of the radial basis functions and evaluating the values of the basis functions at the same radial basis functions and evaluating the values of the basis functions at the same points g

It can be shown that the interpolation matrix in the above equation is non-singular, if It can be shown that the interpolation matrix in the above equation is non-singular, if the point’s

the point’s x x _i are distinct, and thus the weights _i are distinct, and thus the weights ww can be solved by simple linear can be solved by simple linear algebra:

algebra:

3.6 Function approximation

If the purpose is not to perform strict interpolation but instead more general If the purpose is not to perform strict interpolation but instead more general function approximation or classification the optimization is somewhat more complex function approximation or classification the optimization is somewhat more complex because there is no obvious choice for the centers. The training is typically done in because there is no obvious choice for the centers. The training is typically done in two phases first fixing the width and centers and then the weights. This can be two phases first fixing the width and centers and then the weights. This can be justified by considering the different nature of the non-linear hidden neurons versus justified by considering the different nature of the non-linear hidden neurons versus

the linear output neuron. the linear output neuron.

(20)

3.7 Training the basis function centers

Basis function centers can be either randomly sampled among the input instances or Basis function centers can be either randomly sampled among the input instances or found by clustering the samples and choosing the cluster means as the centers. The found by clustering the samples and choosing the cluster means as the centers. The RB

RBF F wiwidtdths hs arare e ususuaualllly y all all fifixexed d to to sasame me vavalulue e whwhicich h is is prpropoporortitiononal al to to ththee maximum distance between the chosen centers.

maximum distance between the chosen centers.

3.8 Pseudoinverse solution for the linear weights

After the centers c

After the centers cii have been fixed, the weights that minimize the error at the outputhave been fixed, the weights that minimize the error at the output are computed with a linear pseudo inverse solution:

are computed with a linear pseudo inverse solution: ,,

Where the entries of

Where the entries of GG are the values of the radial basis functions evaluated at theare the values of the radial basis functions evaluated at the points x

points xii: g: g ji ji= ρ (| | x= ρ (| | x j j− c− cii| |).| |).

The existence of this linear solution means that unlike Multi-layer perceptron (MLP) The existence of this linear solution means that unlike Multi-layer perceptron (MLP) networks the RBF networks have a unique local minimum (when the centers are networks the RBF networks have a unique local minimum (when the centers are fixed).

fixed).

3.9 Advantages/Disadvantages

•

• RBF trains faster than a MLP.RBF trains faster than a MLP. •

• Another advantage that is claimed is that the hidden layer is easier to interpretAnother advantage that is claimed is that the hidden layer is easier to interpret

than the hidden layer in an MLP. than the hidden layer in an MLP.

•

• Although the RBF is quick to train, when training is finished and it is beingAlthough the RBF is quick to train, when training is finished and it is being

used it is slower than a MLP, so where speed is a factor a MLP may be more used it is slower than a MLP, so where speed is a factor a MLP may be more appropriate.

(21)

CHAPTER 4

CHAPTER 4 OVERVIEW OF MOTION DETECTION

OVERVIEW OF MOTION DETECTION ALGORITHMS

ALGORITHMS

Given a number of sequentia

Given a number of sequential video frames from the l video frames from the same source the goal issame source the goal is to detect the motion in the area observed by the source. When there is no motion all to detect the motion in the area observed by the source. When there is no motion all the sequential frames have to be similar up to noise influence. In the case when the sequential frames have to be similar up to noise influence. In the case when motion is present there is some difference between the frames. For sure, each motion is present there is some difference between the frames. For sure, each low-cost system has some aspect of noise influence. And in case of no motion every two cost system has some aspect of noise influence. And in case of no motion every two sequential frames will not be the identical. This is why the system must be smart sequential frames will not be the identical. This is why the system must be smart en

enouough gh to to didiststininguguisish h bebetwtween een nonoisise e anand d rereal al momotitionon. . WhWhen en ththe e sysyststemems s araree calibrated and stable enough the character of noise is that every pixel value may be calibrated and stable enough the character of noise is that every pixel value may be slightly different from that in other frame. And in first approximation it is possible to slightly different from that in other frame. And in first approximation it is possible to define some noise per pixel threshold parameter (adaptable for any given state) the define some noise per pixel threshold parameter (adaptable for any given state) the me

meananining g of of whwhicich h is is hohow w ththe e pipixexel l vavalulue e (o(of f ththe e sasame me ororieientnted ed pipixexel l in in twtwoo sequential frames) might differ but actually the indicating value is the same one. sequential frames) might differ but actually the indicating value is the same one. More precisely, if the pixel with coordinates (Xa,Ya) in frame A differs from the More precisely, if the pixel with coordinates (Xa,Ya) in frame A differs from the pixe

pixel with coordinal with coordinates (Xb,Ybtes (Xb,Yb) in frame B ) in frame B less than on TPP less than on TPP (thres(threshold per pixelhold per pixel)) value so we will see them as pixels with equal values. And we can write it by value so we will see them as pixels with equal values. And we can write it by formulae:

formulae:

Pixel (Xa, Ya) equal to Pixel (Xb, Yb) I Pixel (Xa, Ya) equal to Pixel (Xb, Yb) I

if if

{abs (Pixel (Xa,Ya)-Pixel(Xb,Yb)) < TPP } {abs (Pixel (Xa,Ya)-Pixel(Xb,Yb)) < TPP }

By adapting the TPP value to current system state we can make the system to be By adapting the TPP value to current system state we can make the system to be noise-stable. By applying this threshold operation to every pixel pair we may assume noise-stable. By applying this threshold operation to every pixel pair we may assume that all the preprocessed pixel values are noise-free. The element of noise that is not that all the preprocessed pixel values are noise-free. The element of noise that is not cancelled will be significantly small relative to other part.

cancelled will be significantly small relative to other part.

Ok, if so we have to post-process these values to detect the motion if any. As Ok, if so we have to post-process these values to detect the motion if any. As it

it wawas s mememomorirized zed ababovove e we we hahave ve mamaninipupulatlate e wiwith th didiffffereerent nt pipixexels ls ininsiside de twtwoo sequential frames to make conclusion about the motion.

(22)

Firstly, to make the system sensitive enough we have not to fix the TPP value Firstly, to make the system sensitive enough we have not to fix the TPP value too big. It mean that keeping the sensitivity of the system high in any two frames too big. It mean that keeping the sensitivity of the system high in any two frames there will be some little number (TPP related) of different pixels. And in this case we there will be some little number (TPP related) of different pixels. And in this case we have not to see them as noise. It is the first of the reasons to define a TPF (threshold have not to see them as noise. It is the first of the reasons to define a TPF (threshold per frame) value (adaptab

per frame) value (adaptable for le for any given state) the meaning of any given state) the meaning of whicwhich is h is how manyhow many pixels at least, inside two sequential frames must differ in order to see them as pixels at least, inside two sequential frames must differ in order to see them as motion. The second reason to deal with TPF is to filter (to drop) small motion. For motion. The second reason to deal with TPF is to filter (to drop) small motion. For instance, by playing with TPF values we can neutralize motion of the small object instance, by playing with TPF values we can neutralize motion of the small object (bugs etc.) by still detect the motion of people. And we can write the exact meaning (bugs etc.) by still detect the motion of people. And we can write the exact meaning of TPF by formulae:

of TPF by formulae:

Let’s define NDPPP to be the Number of Different Pre-Processed by TPP Pixels. Let’s define NDPPP to be the Number of Different Pre-Processed by TPP Pixels. So,

So,

There is a motion i.e. NDPPP > TPF. There is a motion i.e. NDPPP > TPF.

Both of TPP and TPF values are variable through the UI to get the optimal system Both of TPP and TPF values are variable through the UI to get the optimal system sensitivity. Also the TPF value has its visual equivalent and it is used as following. sensitivity. Also the TPF value has its visual equivalent and it is used as following. After the pixels pre-processing (by TPP) lets color all static (which do not include After the pixels pre-processing (by TPP) lets color all static (which do not include motion) pixels by lets say black color and all the dynamic (which indicate the motion) pixels by lets say black color and all the dynamic (which indicate the motion) pixels will be left with their original color. This will bring the effect of motion) pixels will be left with their original color. This will bring the effect of motion extraction. In the other words, all the static parts of the frames will be black, motion extraction. In the other words, all the static parts of the frames will be black, and only the moving parts will be seen normally. The enabling/disabling of this and only the moving parts will be seen normally. The enabling/disabling of this effect is possible to control through the GUI.

effect is possible to control through the GUI. Th

The e CamCamera era ManManageager r proprovidvides es rouroutintines es for for acqacquiruiring ing vidvideo eo framframes es fromfrom CCD cameras. Any process can request a video frame from any video source. The CCD cameras. Any process can request a video frame from any video source. The system manages a request queue for each source and executes them cyclically.

(23)

CHAPTER 5

CHAPTER 5 PROPOSED DETECTION SYSTEM

PROPOSED DETECTION SYSTEM

This chapter presents the main software design and implementation issues. It This chapter presents the main software design and implementation issues. It starts by describing the general flow chart of the main program that was implemented starts by describing the general flow chart of the main program that was implemented in MATLAB. It then explains each component of the flow chart with some details. in MATLAB. It then explains each component of the flow chart with some details. Finally it shows how the graphical user interface GUI was designed.

Finally it shows how the graphical user interface GUI was designed.

5.1 Basic Architecture

Fig 5 A basic architecture of surveillance system Fig 5 A basic architecture of surveillance system

The above block diagram shows the surveillance system which consists of a The above block diagram shows the surveillance system which consists of a camera system which monitors the particular area, a video daughter card which camera system which monitors the particular area, a video daughter card which tr

tranansmsmitits s ththe e vividedeo o sisigngnal al to to elelectectricrical al sisigngnalal, , a a nenetwtworork k carcard d whwhich ich hehelplps s inin con

connecnectinting g to to a a netnetworwork k and and motmotion ion detdetectiection on algalgoriorithm thm (SA(SAD D and and CorCorrelarelatiotion)n) along with RBF network.

(24)

5.2 Main Program Flow Chart

The main task of

The main task of the softwarthe software was e was to read the to read the still images recordstill images recorded from theed from the camera and then process these images to detect motions and take necessary actions camera and then process these images to detect motions and take necessary actions accordingly. Figure 6 below shows the general flow chart of the main program.

accordingly. Figure 6 below shows the general flow chart of the main program.

Start Start Setup & Setup & Initializations Initializations What is What is Flag Flag value? value? Flag=0 Flag=0 Flag=1 Flag=1 Image Image Acquisition Acquisition Motion Detection Motion Detection Algorithm

Algorithm _{Break & clear}_{Break & clear}

Is image > Is image > threshold threshold Yes Yes No No Actions on Actions on Motion Motion Detection Detection Data Data Record Record Stop Stop

Figure 6 Main Program Flow Diagram Figure 6 Main Program Flow Diagram

(25)

It

It starts with general initializatistarts with general initialization of on of softwsoftware parameters and are parameters and objectobjects s setupsetup.. Then, once the program started, the flag value which indicates whether the stop Then, once the program started, the flag value which indicates whether the stop button was pressed or not is checked. If the stop button was not pressed it start button was pressed or not is checked. If the stop button was not pressed it start

reading the images then process them

reading the images then process them using one of the two algorithms as using one of the two algorithms as the operator the operator was selected. If a motion is detected it starts a series of actions and then it go back to was selected. If a motion is detected it starts a series of actions and then it go back to read the next images, otherwise it goes directly to read the next images. Whenever read the next images, otherwise it goes directly to read the next images. Whenever the stop button is pressed the flag value will be set to zero and the program is the stop button is pressed the flag value will be set to zero and the program is stopped, memory is cleared and necessary results are recorded. This terminates the stopped, memory is cleared and necessary results are recorded. This terminates the program and returns the control for the operator to collect the results.

program and returns the control for the operator to collect the results.

The next sections explain each process of the flow chart in figure 6 with some The next sections explain each process of the flow chart in figure 6 with some details.

details.

5.2.1 Setup and Initializations

Start Start Launch GUI Launch GUI Start Start button button pressed pressed Yes Yes No No Read Threshold Read Threshold Value Value Stop Stop

Read Algorithm Type Read Algorithm Type

Setup Serial Port Setup Serial Port

Setup Video Object Setup Video Object

Figure 7 Setup and Initializations Process Figure 7 Setup and Initializations Process