OpenCV and Visual c++ Programming in image processing

(1)

OpenCV, MATLAB or AForge ??

Image processing is the process of manipulating image data in order to make it suitable for Image processing is the process of manipulating image data in order to make it suitable for computer vision applications or to make it suitable to present it to humans. For example, computer vision applications or to make it suitable to present it to humans. For example, changing brightness or contrast

changing brightness or contrast is a image processing tais a image processing task which make the imsk which make the image visuallyage visually ple

pleasiasing ng for for humhumans ans or or suisuitabtable le for for furfurthether r proprocescessinsing g for for a a cercertaitain n comcomputputer er visvisionion application.

application.

Computer vision which go beyond image processing, helps to obtain relevant information Computer vision which go beyond image processing, helps to obtain relevant information from images and make decisions based

from images and make decisions based on that information. In other words, on that information. In other words, computer visioncomputer vision is making the computer see as humans do. Basic steps for a typical computer vision is making the computer see as humans do. Basic steps for a typical computer vision application as follows. application as follows. 1 1.. IImmaagge e aaccuuiissiittiioonn ! !.. IImmaagge e mmaanniippuullaattiioonn " ".. ##bbttaaiinniinng rg reelleevvaannt it innffoorrmmaattiioonn $ $.. %%eecciissiioon n mmaakkiinngg

If you are new to computer vision, you may be wondering where to start. First If you are new to computer vision, you may be wondering where to start. First you have to understand the basic principles of image processing and computer you have to understand the basic principles of image processing and computer visio

vision. Then n. Then you have you have to choose a to choose a suitasuitable language to ble language to develdevelop your op your compucomputerter visio

vision n appliapplicatiocation. n. Some of Some of the most the most populpopular methods are ar methods are usingusingOpenCV withOpenCV with C/C++

C/C++,, MATLABMATLAB and and AForgeAForge. If you don't really know why you would choose. If you don't really know why you would choose one over the other,

one over the other, here is my explanation.here is my explanation.

!T"!# is the most easiest and the ine$cient way to process images and !T"!# is the most easiest and the ine$cient way to process images and %pen& is the most e$cient and hardest way to process images. !Forge has %pen& is the most e$cient and hardest way to process images. !Forge has (ualities in between %pen&

(ualities in between %pen& and !Tand !T"!#."!#.

%pen& has become hardest only because there is no proper documentation %pen& has become hardest only because there is no proper documentation and error ha

and error handling codes. ndling codes. #ut %pen& has lots of basic in#ut %pen& has lots of basic inbuilt image prbuilt image processingocessing functions so that those who want to learn computer vision can develop their functions so that those who want to learn computer vision can develop their applications through prop

applications through proper understanding about what they er understanding about what they do.do.

So, I think that it is worthy to learn computer vision with %pen&. Therefore in So, I think that it is worthy to learn computer vision with %pen&. Therefore in this blog, it is presented basic image processing functions and computer vision this blog, it is presented basic image processing functions and computer vision applications with line by line

(2)

(3)

Read & Display Image

Read Image from File and Displa

)ere I am going to explain how to read an image from a *le and display the )ere I am going to explain how to read an image from a *le and display the content using %pen& library functions. First of all, open your &++ I- and content using %pen& library functions. First of all, open your &++ I- and crea

create a te a new pronew proect. /ect. /ou have to con*gurou have to con*gure your new proect e your new proect in order to usein order to use %pen& library functions. If you have not con*gured the proect for %pen& yet, %pen& library functions. If you have not con*gured the proect for %pen& yet, please refer to

please refer to Installing 0 &on*guring with isual Studio.Installing 0 &on*guring with isual Studio.

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111

11111111111111111111111 !in"l#de

!in"l#de $open"%&/highg#i/highg#i'hpp$$open"%&/highg#i/highg#i'hpp$ !in"l#de

!in"l#de (iostream)(iostream)

using namespace

using namespacec!c! using namespace

using namespaces"d!s"d! in"

in" main#main# in" in" argcargc, cons" c$ar%%, cons" c$ar%% arg arg  '

'

Ma"

Ma" img img ( ( imread#imread#)My*ic+*-))My*ic+*-), CV.LOAD.IMA-/.01C2A1-/D!, CV.LOAD.IMA-/.01C2A1-/D! 33read "$e 33read "$e image da"a in "$e 4ile )My*ic+*-) and s"ore

image da"a in "$e 4ile )My*ic+*-) and s"ore i" in 5img5i" in 5img5 i4

i4 #img+emp"y# #img+emp"y# 33c$ec6 7$e"$er "$e image is 33c$ec6 7$e"$er "$e image is loaded or no"loaded or no" '

'

cou"

cou" 8888 )/rror 9 Image canno" :e loaded++;;))/rror 9 Image canno" :e loaded++;;) 88 endl! 88 endl! //

//sys"em#sys"em#)pause))pause)!! 337ai" 4or a 6ey press 337ai" 4or a 6ey press re"urn

re"urn <=!<=! >

>

namedindo7#

namedindo7#)Myindo7))Myindo7), CV.I1DO.A0TO@I/!, CV.I1DO.A0TO@I/! 33crea"e a 7indo7 7i"$ "$ 33crea"e a 7indo7 7i"$ "$ee name )Myindo7)

name )Myindo7) ims$o7#

ims$o7#)Myindo7))Myindo7), img!, img! 33display "$e image 7$ 33display "$e image 7$ic$ is s"ored in "$e 5imic$ is s"ored in "$e 5img5 in "$eg5 in "$e )Myindo7) 7indo7

)Myindo7) 7indo7 7ai"ey#!

7ai"ey#! 337ai" in4ini"e "ime 4or a 6eypress 337ai" in4ini"e "ime 4or a 6eypress des"royindo7#

des"royindo7#)Myindo7))Myindo7)!! 33des"roy "$e 7indo7 7i"$ 33des"roy "$e 7indo7 7i"$ "$e name, )Myindo7)"$e name, )Myindo7) re"urn re"urn !! > > 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111111111111111111 1111111111111111111111

(4)

#efore you run this program, put any image *le

#efore you run this program, put any image *le 2y3ic.4356 into the folder where2y3ic.4356 into the folder where your c++ *le is.%therwise you have

your c++ *le is.%therwise you have to change the to change the *rst argument of imread26*rst argument of imread26 function and give the absolute path to your image *le.

function and give the absolute path to your image *le. /

/ou can download this %pen&ou can download this %pen& visual c++ pro visual c++ proect fromect from herehere. 2The downloaded. 2The downloaded *le is a compressed .rar folder

*le is a compressed .rar folder. So, you . So, you have to extract it usinghave to extract it using 7inrar7inrar or other or other suitable software6

suitable software6

Load & Diplay Image from File Load & Diplay Image from File *planation

*planation

"et's review the above %pen& code line by "et's review the above %pen& code line by lineline..

•

• !in"l#de!in"l#de $stdaf'h$$stdaf'h$

If you are developing your application in isual Studio, don't forget to put this If you are developing your application in isual Studio, don't forget to put this line above the code.

line above the code.

•

• !in"l#de!in"l#de $open"%&/highg#i/highg#i'hpp$$open"%&/highg#i/highg#i'hpp$

imread#

imread#,, namedindo7#namedindo7#,, ims$o7#ims$o7# andand 7ai"ey#7ai"ey# functions are declared in the abovefunctions are declared in the above header file. &o you must include it.

header file. &o you must include it. '

'e e aarre e uussiinnggMa"Ma" ddaatta a ssttrruuccttuurre e iin n tthhe e aabboovve e pprrooggrraamm. . IIt t iis s ddeeccllaarreedd in

(5)

because $open"%&/highg#i/highg#i'hpp$ header *le include that header *le inside it. So, we don't need to include it again in our program.

• using namespace c!

(ll the data structures and functions

in )openc3core3core+$pp) and )openc3$ig$gui3$ig$gui+$pp) are declared inside c namespace. &o, we have to add the above line in the top of our program. #therwise we have to append )c995 specifier before each #penC* functions and data structures. +e.g  cv--at, cv--imread+/ , etc/. If you are not familiar with namespaces, please refer this article.

,!in"l#de (iostream), and )using namespace s"d5 are added because we are using )cou"5 to display some strings in the console. 0his is basic C and you should be familiar with this.

• Ma" img ( imread#const string0 -lename,

int .ags9C*23#(%2I(452C#3#6

Mat is a data structure to store images in a matrix. It is declared in )openc3core3core+$pp) header file.

imread0 is a function declared in $open"%&/highg#i/highg#i'hpp$ header *le. It loads an image from a *le and stores it in at data structure.

(rguments of imread+/ function

• -lename  location of the file. If you 7ust give the filename only, that image should

be in the same folder as your C file. #therwise you have to give the full path to your image.

• 4lags  0here are four possible inputs

o &:"%!:I!5-:;<&)!<5- = image=depth9> bits per pixel in

each channel, no. of channels9unchanged

o &:"%!:I!5-:5?!/S&!"- = image depth9> bits, no. of

channels9@

o &:"%!:I!5-:&%"%? = image=depth98, no. of channels9A

o &:"%!:I!5-:!</-3T) = image=depth9unchanged , no. of

channels98

o &:"%!:I!5-:!</&%"%? = image=depth98, no. of

channels9unchanged

/ou can combine these aboveparameters to get desired image output.

e.g 

C*23#(%2I(452(89%5:0; < C*23#(%2I(452(89C#3#6  image depth=unchanged, no. of channels=unchanged

C*23#(%2I(452C#3#6 < C*23#(%2I(452(89%5:0;  image depth=unchanged, no. of channels="

(6)

If you are not sure what to do, use C*23#(%2I(452C#3#6 as the !nd parameter of

imread+/ function.

0o understand imagedepth and concept of channels, you should be familiar with theory of image processing. &o, let)s discuss little bit of theory of image processing.

(ny digital image consists of pixels. (ny pixel should have some value. 0he minimum value for a pixel is > and it represents black.'hen the value of the pixel is increased, the intensity of that pixel is also increased. In a computer memory, /. In decimal, it is !??. fixed number of bits are allocated for every pixel. &ay the number of allocated bits per pixel is @. 0hen the maximum number that a pixel can have is !?? +11111111 in binary/

8ow what is imagedepthA 0he image<dep"$ means the number of bits allocated for each pixel. If it is @, each pixel can have a value between > and !??. If it is $, each pixel can have a value between > to 1? +1111 in binary/.

;ere is a simple model of a image with imagedepth of @ bits. 5ach small box represents a pixel. &o, each box may contain a value bet ween > to !??.

;ere is some properties of the following image.

o Imagedepth @ bit

o _{1 channel + &o, this is a grayscale image /} o 0he height is $ pixel

o _{0he width is ? pixels}

o _{0he resolution of this image is $x?.}

0his is a grayscale image +black and white image/ because this image has no color content. If the value of this pixel is higher, it will be shown more brighter. If the value is low, it will be shown more darker.

Grayscale Image with image depth of 8

(7)

planes 6ed, 4reen and Blue. (ny color can be created using a particular combination of these " colors. (ny pixel is a combination of three " values. +!??, >, >/ represent pure red. +>, !??, >/ represent pure green. +!??, >, !??/ represents pure violate. In the same way, you can create many color. Imagedepth is !$ because each pixel is represented with @ x " bits +@ bits from each channel/.

;ere is some properties of the following image.

o _{Imagedepth !$ bit}

o _{" channels + &o, this is a color image /} o _{0he height is $ pixel}

o _{0he width is ? pixels}

o 0he resolution of this image is $x?.

R, G, B planes of a color image

In the above model, top left pixel is +!", !"1, $/. It will be shown as a greenish color because the green value+!"1/ of that pixel is larger than the red+!"/ and blue+$/ value.

• i4 #img+emp"y#

If imread+/ function fails to load the image, )img5 will not be loaded any data. 0herefore 5img+emp"y#) should return true. It)s a good practice to check whether the image is loaded successfully and if not exit the program. #therwise your program will crash when executing ims$o7#function.

• :ool Ma"99emp"y#

This function returns true, if atBBdata99<;"" or atBBtotal26 99 C

• //sys"em#)pause)!

If you are using *isual &tudio, it)s better to uncomment this line because it will pause the program until user press any key. If we don)t uncomment it, the program will exit immediately so that user will not see the er ror message.

• oid namedindo7#cons" s"ring& 7inname, in" 4lags ( I1DO.A0TO@I/!

This function creates a window. 3arameters =

(8)

o 7inname  0itle of the window. 0hat name will display in the title bar of the

newly created window

o 4lags  determine the siDe of the window. 0here are two options

 I1DO.A0TO@I/ < Eser cannot resiDe the image. Image will be displayed in its original siDe

 CV.I1DO.1ORMAL < Image will resiDed if you resiDe the the window

• oid ims$o7#cons" s"ring& 7inname, Inpu"Array ma"!

This function shows the image which is stored in the 'mat, in a window speci*ed by winname. If the window is created with1I2DO13A4TO5I6* Dag, image will be displayed in its original siEe. %therwise image may be scaled to the siEe of the window.

3arameters =

o _{7inname  0itle of the window. 0his name is used to identify the window}

created by named'indow+/ function.

o _{ma"  hold the image data}

• in" 7ai"ey#in" delay ( 

waitey26 function wait for keypress for certain time, speci*ed by dela 2in milliseconds6. If dela is Eero or negative, it will wait for in*nite time. If any key is pressed, this function returns the !S&II value of the key and your program will continue. If there is no key press for the specified time, it will return 1 and program will continue.

• oid des"royindo7#cons" s"ring& 7inname

This function closes the opened window, with the title of winname and deallocate any associated memory usage. This function is not essential for this application because when the program exits, operating system usually close all the opened windows and deallocate any associated memory usage.

5#mmar

7hen running this program, the image of 'M7i"'879' is loaded into the variable, 'img, of type at. Then a window named 'M1indow' is opened. !fter that 'img' is loaded to that window. The window with the image will be displayed until any key is pressed.

(9)

This program is also very much similar to the previous application. The only diGerence is that this program creates a blank image instead of loading an existing image from a *le.

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111 !in"l#de $open"%&/highg#i/highg#i'hpp$ !in"l#de (iostream) using namespace c! using namespace s"d!

in" main# in" argc, cons" c$ar%% arg  '

Ma" img#E, =, CV.0CG, @calar#,, =! 33crea"e an image # G c$annels,  :i" image dep"$, E $ig$, = 7ide, #, , =  assigned 4or Blue, -reen and Red plane respec"iely+ 

i4 #img+emp"y# 33c$ec6 7$e"$er "$e image is loaded or no" '

cou" 88 )/rror 9 Image canno" :e loaded++;;) 88 endl!

//sys"em#)pause)! 337ai" 4or a 6ey press re"urn <=!

>

namedindo7#)Myindo7), CV.I1DO.A0TO@I/! 33crea"e a 7indo7 7i"$ "$e name )Myindo7)

ims$o7#)Myindo7), img! 33display "$e image 7$ic$ is s"ored in "$e 5img5 in "$e )Myindo7) 7indo7

7ai"ey#! 337ai" in4ini"e "ime 4or a 6eypress

des"royindo7#)Myindo7)! 33des"roy "$e 7indo7 7i"$ "$e name, )Myindo7) re"urn !

>

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111111111111111111

#efore you run this program, put any image *le 2y3ic.4356 into the folder where your c++ *le is.%therwise you have to change the *rst argument of imread26 function and give the absolute path to your image *le.

/ou can download this %pen& visual c++ proect from here. 2The downloaded *le is a compressed .rar folder. So, you have to extract it using 7inrar or other suitable software6

(10)

Create & Display Image

2ew OpenCV f#n"tions

• Mat<<Matint rows= int "ols= int tpe= "onst 5"alar; s0>

This is the one of the many constructor available in Mat class. It initialiEe the at obect with the value given by the Scalar obect

3arameters B

o rows = <umber of rows in the H array 2 height of the image in

pixels6

o "ols = <umber of columns in the H array 2 width of the image in

pixels6

o tpe = specify the bit depth, data type and number of channels of

the image. I gave CV3?4C@ and it specify > bit unsigned integers with A channels. )ere are some of possible inputs for this parameter

 &:>;&@ = > bit unsigned integers with single channel  &:>;&A = > bit unsigned integers with A channels  &:JF&@ = J bit Doating point value with @ channels

If you want more details about this, please refer to Data Tpes for Arras in the #asics of %pen& !3I

o s  InitialiEe each array element with the value given by s. In the

above application, I gave Scalar2C,C,@CC6. So, it initialiEe my *rst channel 2#lue plane6 with C, Hnd channel 25reen plane6 with C and Ard channel 2?ed 3lane6 with @CC. So, my *nal image is red. /ou can try diGerent combinations of these three and see the output image.

(11)

5#mmar

In this program, I created a A channel image with KCC height and @CCC width. > bit unsigned integer is allocated for each pixel in each channel. 2>xA 9 HJ bits per each pixel6 !nd each pixel is assigned with 2C,C,@CC6 scalar value. That means @st channel is all Eero, Hnd channel is also all Eero and the Ard channel is all @CC. Therefore we can see a red image as the output of the program.

(12)

Cap"ure Video 4rom File or Camera

Capt#re Video From File

In this lesson, I am going to show you how to read a video *le. It's fairly simple. /ou ust need to initialiEe a ideo&apture obect which will open a video *le and

read frame by frame from that opened video. )ere is the sample %pen& code. If you are using isual Studio, you will need to include Lstdafx.h header file.

1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Hinclude )openc3$ig$gui3$ig$gui+$pp) Hinclude 8ios"ream

using namespace c! using namespace s"d!

in" main#in" argc, c$ar % argJK '

VideoCap"ure cap#)C930sers3@2/RMAL3Des6"op3@ampleVideo+ai)! 33 open "$e ideo 4ile 4or reading

i4 # ;cap+isOpened#  33 i4 no" success, ei" program '

cou" 88 )Canno" open "$e ideo 4ile) 88 endl! re"urn <=!

>

33cap+se"#CV.CA*.*RO*.*[email protected]@/C, G! 33s"ar" "$e ideo a" Gms

dou:le 4ps ( cap+ge"#CV.CA*.*RO*.F*@! 33ge" "$e 4rames per seconds o4 "$e ideo

cou" 88 )Frame per seconds 9 ) 88 4ps 88 endl!

namedindo7#)MyVideo),CV.I1DO.A0TO@I/! 33crea"e a 7indo7 called )MyVideo)

7$ile#= '

Ma" 4rame!

:ool :@uccess ( cap+read#4rame! 33 read a ne7 4rame 4rom ideo i4 #;:@uccess 33i4 no" success, :rea6 loop

'

(13)

:rea6! >

ims$o7#)MyVideo), 4rame! 33s$o7 "$e 4rame in )MyVideo) 7indo7

i4 #7ai"ey#G ((  337ai" 4or 5esc5 6ey press 4or G ms+ I4 5esc5 6ey is pressed, :rea6 loop

'

cou" 88 )esc 6ey is pressed :y user) 88 endl! :rea6! > > re"urn ! > 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

This is how I pla a %ideo #sing m OpenCV program

2ew OpenCV f#n"tions whi"h are not fo#nd earlier are eplained here

• VideoCap"ure99VideoCap"ure#cons" s"ring& 4ilename

This is one of few constructors available in ideo&apture class. This constructor open the video *le and initialiEes the ideo&apture obect for reading the video stream from the speci*ed *le.

(14)

The destructor of this class will deallocated any associated memory with this obect. Therefore you don't need to deallocate memory explicitly in your program.

• :ool VideoCap"ure99IsOpened#

If the previous call to *ideoCapture constructor is successful, this method will return true. #therwise it will return false.

It is essential to check that whether the *ideoCapture initialiDe successfully. If it is

unsuccessful, program should be exited. #therwise when you try to read a frame from the *ideoCapture ob7ect, your program will crash.

• :ool VideoCap"ure99se"#in" propId, dou:le alue

9ou can change some properties of *ideoCapture ob7ect. If it is successful, this method will return true. #therwise it will return false. 9ou should try to change some properties of the video stream in your code. In my code, I have shown you how to change the position of the video, by changing the CV3CA737RO737O53M5*C property.

:arameters

-o in" propID < 0his argument specify the property you are going to change. 0here are many

options for this argument. &ome of them are listed here.

 CV3CA737RO737O53M5*C  current position of the video in milliseconds  CV3CA737RO737O53FRAM*5  current position of the video in frames  CV3CA737RO73FRAM*31IDT  width of the frame of the video stream  CV3CA737RO73FRAM*3*I9T  height of the frame of the video stream  CV3CA737RO73F75  frame rate 2frames per second6

 CV3CA737RO73FO4RCC = four character code of codec

o _{dou:le alue < 0his is the new value you are going to assign to the property, specified by}

the propI%

• dou:leVideoCap"ure99ge"#in" propId

0his function returns the value of the property which is specified by propId+ 9ou should try to obtain some properties of the video stream in your code. In my code, I have shown you how to obtain the frame rate +frames per second/ of the video, by using

the CV.CA*.*RO*.F*@ argument.

:arameters 

o _{in" propID < 0his argument specify the property you are going to obtain. 0here are many}

options for this argument. &ome of them are listed here.

 CV3CA737RO737O53M5*C  current position of the video in milliseconds  CV3CA737RO737O53FRAM*5  current position of the video in frames  CV3CA737RO73FRAM*31IDT  width of the frame of the video stream  CV3CA737RO73FRAM*3*I9T  height of the frame of the video stream  CV3CA737RO73F75  frame rate +frames per second/

(15)

 CV3CA737RO73FO4RCC = four character code of codec

• :ool VideoCap"ure99read#Ma"& image!

The function grabs the next frame from the video, decodes it and stores it in the 'image, variable. Inside this function, ideo&aptureBBgrap26 and ideo&aptureBBretrieve26 will be called. If you want, you can use these two functions instead of ideo&aptureBBread26 function.

If the operation successful, it will return true. %therwise it will return false.

• 7ai"ey#@

The function waits for AC milliseconds. If a key was pressed before the speci*ed time, it returns the !S&II value of the pressed key. If that value is HM 2!S&II value of 'esc' key is HM6, the program will execute inside the if block. If no key is pressed during that ACms, the function returns =@ program will continue the while loop.

• VideoCap"ure99NVideoCap"ure#

estructor of ideo&apture obect will destroy any associated memory of that particular obect. This destructor will be called implicitly on exit of the main method of the above program.

5#mmar

!t *rst, this program captures a video from a *le. Then the program enters into a in*nite loop. In that loop, it grabs frames from the captured video se(uentially, decodes it, shows it in a window and waits for @ milliseconds. If the video *le has no more frames or if the user presses the 'esc' key, the program will break the in*nite loop.

2ote<

;sing waiteint0 function is very important because imshowstring;= MAT0 function need time to paint the image in the window and 7ai"ey#in"will give that necessary time.

Capt#re Video From Camera

The main diGerence of the following program from the above program is the argument to the ideo&apture constructor. )ere you ust need to give the index of your camera to the constructor of the ideo&apture obect instead of the *lename in the above program. )ere is the sample %pen& code.

(16)

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 !in"l#de $open"%&/highg#i/highg#i'hpp$

!in"l#de (iostream) #sing namespa"e "%> #sing namespa"e std>

int mainint arg"= "harE arg%G0 H

VideoCapt#re "ap0> // open the %ideo "amera no'  if "ap'isOpened00 // if not s#""ess= eit program H

"o#t (( $Cannot open the %ideo "am$ (( endl> ret#rn J>

K

do#le d1idth  "ap'getCV3CA737RO73FRAM*31IDT0> //get the width of frames of the %ideo

do#le deight  "ap'getCV3CA737RO73FRAM*3*I9T0> //get the height of frames of the %ideo

"o#t (( $Frame siNe < $ (( d1idth (( $  $ (( deight (( endl> named1indow$MVideo$=CV31I2DO13A4TO5I6*0> //"reate a window "alled $MVideo$

while J0 H

Mat frame>

ool 5#""ess  "ap'readframe0> // read a new frame from %ideo if 5#""ess0 //if not s#""ess= rea: loop

H

"o#t (( $Cannot read a frame from %ideo stream$ (( endl> rea: >

K

imshow$MVideo$= frame0> //show the frame in $MVideo$ window if waite@0  &0 //wait for ,es", :e press for @ms' If ,es", :e is pressed= rea: loop

H

"o#t (( $es" :e is pressed  #ser$ (( endl> rea: >

K K

(17)

ret#rn > K

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

/ou can download this %pen& visual c++ proect from here. 2The downloaded *le is a compressed .rar folder. So, you have to extract it using 7inrar or other

suitable software6

T$is is $o7 I cap"ure mysel4 7i"$ my 7e:cam using "$e OpenCV program

• VideoCapt#re<<VideoCapt#reint de%i"e0

0his is one of constructors available in *ideoCapture class. 0his constructor open the camera indexed by the argument of this constructor and initialiDes the *ideoCapture ob7ect for reading the video stream from the specified camera.

)ere the 'C' means the index of the camera to be used. /ou can use @,H,A.. instead of C, if your computer is attached to more than @ camera. The destructor of this class will deallocated any associated memory with this

obect. Therefore you don't need to deallocate memory explicitly in your program.

(18)

If the *ideoCapture ob7ect initialiDation unsuccessful, the expression inside the )if )parentheses will evaluate to true and the statements inside the )if) block will be executed. It is a good practice to check this scenario and handle it accordingly because otherwise it may cause the program to crash.

• dou:le did"$ ( cap+ge"#CV.CA*.*RO*.FRAM/.IDT2

0his function obtain the width +in pixels/ of frames of the camera output.

• dou:le d2eig$" ( cap+ge"#CV.CA*.*RO*.FRAM/.2/I-2T

0his function obtain the height +in pixels/ of frames of the camera output.

(19)

ri"e Image & Video "o File

In this lesson, I am going to show you how to write images and videos to a *le using %pen&.

If you have not install and con*gure %pen& yet, please refer to Installing 0 &on*guring with isual Studio.

1rite Image to File

In the following example, a yellow image is created and written into a file. 3et)s see how to

do it with #penC*. 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 111111111111111 !in"l#de $open"%&/highg#i/highg#i'hpp$ !in"l#de (iostream) using namespace c! #sing namespa"e std>

in" main# in" argc, cons" c$ar%% arg 

H

Mat imgPQ= P= CV3JP4C@= 5"alar=Q= Q00> //"reate an image  @ "hannels= JP it image depth= PQ high= P wide= = Q= Q0 assigned for Bl#e= 9reen and Red plane respe"ti%el' 0

i4 #img+emp"y# 33c$ec6 7$e"$er "$e image is loaded or no"

H "o#t (( $*RROR < Image "annot e loaded''$ (( endl>

//sstem$pa#se$0> //wait for a :e press

ret#rn J> K

%e"tor(int) "ompression3params> //%e"tor that stores the "ompression parameters of the image

"ompression3params'p#sh3a":CV3IM1RIT*387*934ALITS0> //spe"if the "ompression te"hni#e

"ompression3params'p#sh3a":U?0> //spe"if the compression #alit

ool 5#""ess  imwrite$D</TestImage'pg$= img= "ompression3params0> //write the image to -le

(20)

if  5#""ess 0 H

"o#t (( $*RROR < Failed to sa%e the image$ (( endl> //sstem$pa#se$0> //wait for a :e press

K

namedindo7#)Myindo7), CV.I1DO.A0TO@I/! 33crea"e a 7indo7 7i"$ "$e

name )Myindo7)

ims$o7#)Myindo7), img! 33display "$e image 7$ic$ is s"ored in "$e 5img5 in "$e

)Myindo7) 7indo7

7ai"ey#! 337ai" 4or a 6eypress

des"royindo7#)Myindo7)! 33des"roy "$e 7indo7 7i"$ "$e name, )Myindo7) re"urn !

K

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 111111111111111

The above program is very much similar to the program under '&reate a #lank Image 0 isplay' section in the lesson of ?ead 0 isplay Image. If you need further clari*cations of any %pen& functions which are not explained here, please refer to 6ead G %isplay Image lesson.

2ew OpenCV f#n"tions whi"h are not fo#nd earlier

• :ool im7ri"e# cons" s"ring& 4ilename, Inpu"Array img, cons" ec"or8in"&

params(ec"or8in"#

The function saves the image in the variable 'img, to a *le, speci*ed by ,-lename, . If this function fails to save the image, it will return false. %n success of writing the *le to the harddisk, it will return true. 3arameters =

o 4ilename < specify the location and name of the file to be saved o img < hold the image which is going to save

(21)

o params < 0his is a int vector to which you have to insert some int parameters

specifying the format of the image

 H:54 format  9ou have to

puch2backCV3IM1RIT*387*934ALITS *rst and then a number between C and @CC 2higher is the better6. If you want the best (uality output, use @CC. I have used N> in the above sample program. #ut higher the value, it will take longer time to write the image

 :84 format  9ou have to

puch2backCV3IM1RIT*37293COM7R*55IO2 first and then a number between > and  +higher is the better compression, but slower/.

The image format is chosen depending on the *le name extension. %nly images with > bit or @ bit unsigned single channel or A channel+ C*2@EC1, C*2@EC", C*2@&C1, C*2@&C",&:@;&@, &:@;&A6 with 'B9R' channel order, can be saved. If the depth or channel order of the image is diGerent, use ,Mat<<"on%ertTo0, or ,"%tColor, functions to convert the image to

supporting format before using imwritefunction.

The above program is very much similar to the program under '&reate a #lank Image 0 isplay' section in the lesson of ?ead 0 isplay Image. If you need further clari*cations of any %pen& functions which are not explained here,

please refer to6ead G %isplay Image lesson.

@ummary

0his program creates an yellowish image 2 A channels, @ bit image depth, KC high, CC wide, 2C, KC,CCC, KC,CCC6 assigned for #5? channels6. #ecause the image has @ bit depth, you can use values between C and KKAK for each element in each channel. I have used KC,CCC for each element in the green and red planes which gives the yellowish color. /ou can try diGerent values. Then it speci*es the compressing techni(ue. I have used 43-5 as the

compression techni(ue in the above example. Then it saves the image in the %-J0estImage.7pg location. 0hen it displays the newly created image in a window and wait indefinitely until an user presses a key.

1rite Video to File

In the following example, a video is captured from the webcam and written into a *le. "et's see how to do it with %pen&.

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111

!in"l#de $open"%&/highg#i/highg#i'hpp$ !in"l#de (iostream)

(22)

#sing namespa"e std>

int mainint arg"= "harE arg%G0 H

VideoCapt#re "ap0> // open the %ideo "amera no'  if "ap'isOpened00 // if not s#""ess= eit program H

"o#t (( $*RROR< Cannot open the %ideo -le$ (( endl> ret#rn J>

K

named1indow$MVideo$=CV31I2DO13A4TO5I6*0> //"reate a window "alled $MVideo$

do#le d1idth  "ap'getCV3CA737RO73FRAM*31IDT0> //get the width of frames of the %ideo

do#le deight  "ap'getCV3CA737RO73FRAM*3*I9T0> //get the height of frames of the %ideo

"o#t (( $Frame 5iNe  $ (( d1idth (( $$ (( deight (( endl>

5iNe frame5iNestati"3"ast(int)d1idth0= stati"3"ast(int)deight00> Video1riter oVideo1riter $D</MVideo'a%i$ = CV3FO4RCC,7,=,I,=,M,=,J,0= &= frame5iNe= tr#e0> //initialiNe the Video1riter oe"t

if  oVideo1riter'isOpened0 0 //if not ini"ialie the Video1riter s#""essf#ll= eit the program

H

"o#t (( $*RROR< Failed to write the %ideo$ (( endl> ret#rn J>

K

while J0 H

Mat frame>

ool 5#""ess  "ap'readframe0> // read a new frame from %ideo if 5#""ess0 //if not s#""ess= rea: loop

H

"o#t (( $*RROR< Cannot read a frame from %ideo -le$ (( endl> rea: >

K

(23)

imshow$MVideo$= frame0> //show the frame in $MVideo$ window if waiteJ0  &0 //wait for ,es", :e press for @ms' If ,es", :e is pressed= rea: loop

H

"o#t (( $es" :e is pressed  #ser$ (( endl> rea: > K K ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111

ere is how I re"ord m %ideo from m we"am and sa%e to a -le in D dri%e

• 5iNe

(24)

&reate a 5iNe obect with a given width and height. )ere I have cast the width and height to integers because they are originally double values and the 5iNe constructor does not accept double values as its parameters.

• Video1riter<<Video1riter "onst string;

-lename= int fo#r""= do#le fps= 5iNe frame5iNe= ool isColortr#e0

This is the constructor of the ideo7riter class. It initialiEes the obect with following parameters

o "onst string; -lename = Specify the name and the location of the

output *le. The video stream is written into this *le

o int fo#r"" = specify the J character code for the codec which is

used to compress the video. /our computer may not be supported some codecs. So, if you fail to save the video, please try other codecs. )ere are some popular codecs.

 &:F%;?&&2'', 'I', '', 'A'6 for ivO 3-5=J codec  &:F%;?&&2'', '3', 'J', 'H'6 for 3-5=J codec  &:F%;?&&2'', 'I', '', 'O'6 for ivO codec  &:F%;?&&2'3','I','','@'6 for 3-5=@ codec  &:F%;?&&2'I', 'H', '', 'A'6 for IT; ).HA codec  &:F%;?&&2'', '3', '-', '5'6 for 3-5=@ codec

&omplete list of codec can be found here

)ere I have used CV3FO4RCC,7,= ,I,= ,M,= ,J,0 as the four character code of codec to compress the video. If the output *le cannot be opened, you can try diGerent four character code of codecs.

For 1indows #sers, it is possible to use =@ instead of the above codecs in order to choose compression method and additional compression parameters from a dialog box. It is a best method for icrosoft 7indows users.

o do#le fps  frames per seconds of the video stream. I have used

HC. /ou can try diGerent values. #ut the codec should support the fps value. So, use an appropriate value.

o 5iNe frame5iNe = SiEe obect which specify the width and the

height of each frame of the video stream.

o ool isColor = If you want to save a color video, pass the value as

true. %therwise false. ?emember codec should support whatever value, you pass. In the above example, you have to give true as the Kth argument. %therwise it will not work.

• if  oVideo1riter'isOpened0 0

&heck whether the ideo7riter obect initialiEe successfully. If not exit the program immediately.

(25)

7rite a frame to the video stream. The siEe of the frame should be same as the siEe you speci*ed when initialiEing the ideo7riter obect.

!ll other %pen& functions have been discussed in earlier lessons. So, if you are not familiar with those %pen& functions yet, please go through Capt#re Video from File or Camera

(26)

Fil"ering Images

Image *ltering is an important part of computer vision. For most of computer vision applications, *ltering should be done before anything else. %pen& supports lots of in=build *ltering methods for images. )ere is the list of *ltering methods that I am going to discuss with you in the following posts 2with %pen& H.J.K and &++ 6

•

Change Brightness of Image or Video

•

Change Contrast of Image or Video

•

istogram *#aliNation of 9ras"ale or

Color Image

•

5mooth / Bl#r Images

)ere is the list of image *ltering methods which are explained using examples with %pen& H.@ in & style 2not &++6

• -roding

• ilating

• Inverting

(27)

Original Image

If you have not install and con*gure %pen& yet, please refer to Installing 0 &on*guring with isual Studio.

*roding

-roding is a simple way of *ltering images. )ere is how it can be done with %pen&. 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 !in"l#de $stdaf'h$ !in"l#de ("%'h) !in"l#de (highg#i'h) int main0 H

//displa the original image

IplImageE img  "%LoadImage$C</M7i"'pg$0> "%2amed1indow$M1indow$0>

"%5howImage$M1indow$= img0>

//erode and displa the eroded image

"%*rodeimg= img= = &0>

"%2amed1indow$*roded$0> "%5howImage$*roded$= img0> "%1aite0> //"leaning #p "%Destro1indow$M1indow$0> "%Destro1indow$*roded$0> "%ReleaseImage;img0> ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 /ou can download this %pen& visual c++ proect from here.

(28)

roded Image

• "%*rodeimg= img= = &0

The @st parameter is the source image.

The Hnd parameter is the destination image which is to be the eroded image. )ere the Ard parameter is the structuring element used for erosion. If it is 0,

a APA rectangular structuring element is used.

The Jth parameter is the number of times, erosion is applied.

This function can process images in place. That means same variable can be used for the @st and Hnd parameters.

If you want more explanation about various methods in the above computer application , please refer to &apturing Images 0 ideos.

Dilating

ilating is something like opposite of the eroding an image. )ere is the %pen& code.

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

(29)

!in"l#de ("%'h)

!in"l#de (highg#i'h) int main0

H

"%5howImage$M1indow$= img0> //dilate and displa the dilated image "%Dilateimg= img= = &0>

"%2amed1indow$Dilated$0> "%5howImage$Dilated$= img0> "%1aite0> //"leaning #p "%Destro1indow$M1indow$0> "%Destro1indow$Dilated$0> "%ReleaseImage;img0> ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

(30)

Dilated Image

• "%Dilateimg= img= = &0

The @st parameter is the source image.

The Hnd parameter is the destination image which is to be the dilated image. )ere the Ard parameter is the structuring element used for dilation. If it is 0,

a APA rectangular structuring element is used.

The Jth parameter is the number of times, dilation is applied.

This function can process images in place. That means same variable can be used for the @st and Hnd parameters.

In%erting

Inverting an image is like taking the negative of an image. 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 !in"l#de $stdaf'h$

!in"l#de ("%'h)

!in"l#de (highg#i'h) int main0

(31)

H

"%5howImage$M1indow$= img0>

//in%ert and displa the in%erted image "%2otimg= img0> "%2amed1indow$In%erted$0> "%5howImage$In%erted$= img0> "%1aite0> //"leaning #p "%Destro1indow$M1indow$0> "%Destro1indow$In%erted$0> "%ReleaseImage;img0> ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 /ou can download this %pen& visual c++ proect from here.