A generic framework for colour texture segmentation

(1)

A G E N E R I C F R A M E W O R K F O R C O L O U R T E X T U R E

S E G M E N T A T I O N

BY

P A D M A P R IY A N A M M A L W A R , B .S c ., M .S c., M .P h il.,

Th is t h e s i s i s s u b m i t t e d t o Du b l i n Ci t y Un i v e r s i t y a s t h e f u l f i l m e n t OF THE REQUIREMENT FOR THE AWARD OF THE DEGREE OF

D o c t o r o f P h i l o s o p h y In E l e c t r o n i c E n g i n e e r i n g SU P E R V IS E D B Y P R O F .P A U L .F .W H E L A N , B .E n g ., M .E n g ., P h .D ., S c h o o l o f E l e c t r o n i c E n g i n e e r i n g D u b l i n C i t y U n i v e r s i t y A P R I L 2 0 0 4

(2)

D ecla ra tio n

I hereby certify th a t this m aterial, which I now subm it for assessment on the programme of study leading to th e award of Doctor of Philosophy, is entirely my own work and has not been taken from the work of others and to th e extent th a t such work has been cited and acknowledged within th e tex t of my work.

t w

„ .

_

l

*

j?a

Signed : --- 1 — •--- D ate : Padm apriya Nammalwar

(3)

To m y dear

(4)

A ck n ow led gem en ts

Many thanks to all who helped and encouraged me to complete this research work. In particular,

I would like to thank my supervisor Prof.Paul.F.W helan for all the ideas, guid ance and expertise provided by him throughout this research. My sincere grati tude to him for all his help.

I wish to thank the School of Electronic Engineering and Dublin City University for funding this research.

I wish to thank Dr.Ovidiu G hita for his valuable suggestions during the imple m entation of the programming and for looking at this thesis for corrections and suggestions for improvement. I th an k Pradeep for his valuable suggestions dur ing the initial stages of this research.

I wish to thank all my collègues in Vision Systems Group, R obert Sadleir, Tarik Chowdhry, Micheál Lynch, John Mallon, Kevin Robinson and Nicholas Sezille. I thank collègues from the VRAL Lab Dr.Derek Molloy and John Higgins.

I wish to thank Robert Clare and Conor Maguire for th e assistance given to me towards the computer maintenance.

Many thanks to my brothers P rasan th and Bhaskar for their constant support and encouragement during this research work.

Lastly, I would like to say my special thanks to Paddu for all his love, support and help.

(5)

A b s tr a c t

This thesis proposes a novel m ethod to combine the colour and the texture for colour texture segmentation. The objective of this research work is to derive a framework for colour texture segmentation and to determ ine the contribu tion of colour in colour texture analysis. The colour tex tu re processing is based on the feature extraction from colour-textured images. The texture features were obtained from the luminance plane along w ith the colour features from the chrominance planes. Based on the above mentioned approach, a m ethod was developed for colour texture segmentation. The proposed m ethod unifies colour and texture features to solve the colour texture segm entation problem. Two of the grey scale texture analysis techniques, Local Binary P a tte rn (LBP) and Discrete Cosine Transform (DCT) based filter approach were extended to colour images. An unsupervised fc-means clustering was used to cluster pixels in the chrominance planes. Non-parametric test was used to test th e similarity between colour texture regions. An unsupervised texture segm entation m ethod was fol lowed to obtain the segmented image. The evaluation of th e segmentation was based on the ROC curves. A quantitative estim ation of colour and texture per formance in segmentation was presented. The use of different colour spaces was also investigated in this study. The proposed m ethod was tested using different mosaic and natural images obtained from VisTex and other predom inant im age database used in com puter vision. The applications for the proposed colour texture segmentation m ethod are, Irish Script On Screen (ISOS) images for the segmentation of the colour textured regions in the document, skin cancer images to identify the diseased area, and Sediment Profile Imagery (SPI) to segment un derwater images. The inclusion of colour and texture as distributions of regions provided a good discrimination of the colour and the texture. The results indi cated th a t the incorporation of colour information enhanced the texture analysis techniques and the methodology proved effective and efficient.

(6)

A b b rev ia tio n s and S ym b ols

ID One Dimension

2D Two Dimension

3D Three Dimension

CIE International Commission on Illum ination CIE-XYZ Non-uniform colour space, defined by CIE CIE-LAB Uniform colour space, defined by CIE CIE-LUV Uniform colour space, defined by CIE DCT Discrete Cosine Transform

GIF Graphics Interchange Format

HSI Hue, Saturation, Intensity, a colour model HSV Hue, Saturation, Value, a colour model ISOS Irish Script on Screen

JPEG Joint Photographic Experts Group

KS Kolmogorov Smirnov

LBP Local Binary P attern

LB P/C Local Binary P attern and C ontrast LVQ Learning Vector Q uantisation M-KS Modified Kolmogorov Smirnov

MRF Markov Random Field

MRMRF M ulti-resolution Markov Random Field MeasTex M easurement of Texture

NLC Nearest linear Combination

NN Nearest Neighbour

NTSC N ational Television System Com m ittee RGB Red, Green, Blue, a colour model

ROC Receiver O perating Characteristic Curve SAC Symmetric auto-correlations

(7)

s c o v Symmetric auto-correlations w ith a covariance measure

SLR Single Lens Reflex

SPI Sediment Profile Imagery

SRAC Symmetric auto-correlations w ith rank order version SVR Symmetric auto-correlations w ith variance ratio

TU Texture Unit

YIQ Colour model used in American television transm ission sys tems

YUV Colour model used in European television transm ission sys tems

VisTex Vision Texture

Vi Pixels in the 3 x 3 neighbourhood

Ei Threshold value in the 3 x 3 neighbourhood

uk Basis Vector

Eff Local discontinuity along horizontal direction E yVxy Local discontinuity along vertical direction

E Cxy Local discontinuity along counter-diagonal direction Ediv Local discontinuity along diagonal direction

E

¿-‘xy Local discontinuity measure based on th e four detectors Nxy{R) Contextual neighbourhood associated w ith pixel (x, y) ^xy(R) Mean of pixels on N xy(R)

« U R ) Spatial variance of pixels on N xy(R) Sty(R) Normalised spatial variance

vliax(R ) M aximal spatial variance across the image

°L n (R ) Minimal spatial variance across the image

H ^ y ( R ) A ) Transform ation based on the normalised variance

9a Threshold to limit th e degree of contextual discontinuities j(t+1)

(8)

Vij (*) Tij a

E n co d es th e effect of c o n te x tu a l d isc o n tin u itie s E n co d es th e effect of local d isc o n tin u itie s

To d e te rm in e th e e x te n t of fe a tu re p re s e rv a tio n in te rm s of c o n te x tu a l d isc o n tin u itie s

To d e te rm in e th e p re se rv a tio n e x te n t of lo cal d isc o n tin u ities

X T h e m e a n of th e colour p lan e P (i,j) pix el value a t th e p o sitio n (i, j)

G L og-likelihood p se u d o m e tric G -s ta tis tic , s im ila rity m ea su re

ft T h e freq u en cy a t b in i

s, m T w o sa m p le h isto g ram s

D (s, m) D iscrep an cy s ta tis tic or M odified K olm ogo rov S m irnov s ta tis tic

Fs(i), Fm(i) S am p le c u m u la tiv e d is trib u tio n s

Gmax M ax im u m G value n K~TTnin M in im u m G value R R a tio of th e m ax im u m a n d m in im u m G value X T h re sh o ld v alue in s p littin g q . _{M in im um block size}

M I M erger Im p o rta n c e value M IR M erger Im p o rta n c e ra tio Y T h re sh o ld v alu e in m erg ing

r a disc w ith ra d iu s r on th e pixelw ise classificatio n

d a sq u a re w ith a d im e n sio n d o n th e b o u n d a ry refinem ent m , w2 W eights a c co rd in g to th e d is trib u tio n of colo ur c lu ste re d

labels

M K S i , M K S 2 M odified K olm ogorov S m irnov s ta tis tic for th e in te n sity h isto g ra m a n d th e colour h isto g ra m resp e c tiv e ly

(9)

M in M I M in im um o f m erg er im p o rta n c e v alu e kj U n ifo rm ity fa c to r of th e sa m p le regions

CLj[i] H isto g ra m of th e colour c lu ste re d lab els in th e sa m p le re gions

Np N u m b e r of pixels in th e c o rre sp o n d in g regions

e R a tio betw een th e n u m b e r o f pixels in c o rre c tly seg m en ted by th e to ta l n u m b e r of pixels in th e region

(10)

List o f F igures

1.1 (a), (b) - M icro te x tu r e s (c), (cl) - M acro t e x t u r e s ... 5

1.2 (a), (c) - In p u t Im a g e (b ), (d) - S y n th e sise d Im a g e [ 1 5 ] ... 7

1.3 (a) B a rk (b) B rick (c) S a n d (d) F ab ric. D ifferent ty p e s of V isT ex [27] te x tu re s a n d th e ir i d e n t i t i e s ... 8

1.4 D ifferent h om ogeneous te x tu r e s were lab elled in te x tu r e classificatio n 8 1.5 T e x tu re s e g m e n ta tio n - A n e x a m p l e ... 9 2.1 Section su m m a ry - T e x tu re s e g m e n t a t i o n ... 22 2.2 S ection s u m m a ry - T e x tu re C la s s if ic a tio n ... 28 2.3 S ection s u m m a ry - T e x tu re C la s s if ic a tio n ... 29 2.4 Section s u m m a ry - R e la te d te x tu r e b a se d a p p ro a c h e s . . . 34 2.5 S ection s u m m a ry - R e la te d te x tu r e b ased a p p r o a c h e s ... 35

2.6 S ection su m m a ry - C o lour se g m e n ta tio n ... 40

2.7 S ectio n su m m a ry - C o lo u r te x tu r e s e g m e n t a t i o n ... 44

2.8 S ectio n s u m m a ry - C o lou r te x tu r e s e g m e n t a t i o n ... 45

2.9 S ectio n su m m a ry - C o lou r te x tu r e c l a s s i f i c a t i o n ... 49

2.10 S ectio n su m m a ry - C o lo u r te x tu r e c l a s s i f i c a t i o n ... 50

(11)

L I S T OF F IG U R E S

3.2 k -m ean s c lu ste rin g m e th o d , (a) re p re se n ts th e o rig in al im age w ith th e differen t p a tte r n s a n d th e c e n tro id of th e im a g e d a ta , (b) illus tr a te s th e c lu ste r m em b e rsh ip a fte r th e first ite ra tio n , (c) show s th e c lu ste r m em b e rsh ip a fte r th e second ite ra tio n , (d) re p re se n ts

th e final c lu ste re d im a g e ... 63

3.3 A ty p ic a l ex am p le of th e s p littre e a n d th e c o rre sp o n d in g im age r e p r e s e n t a t i o n ... 72

3.4 P se u d o co de for th e s p littre e a p p r o a c h ... 73

3.5 A ty p ic a l ex am p le of m e r g e g r a p h ... 75

3.6 P se u d o co de for th e m e rg e g ra p h a p p r o a c h ... 76

4.1 F lo w ch art rep re sen tin g fe a tu re e x tra c tio n a p p ro a c h e s a n d th e colour te x tu r e s e g m e n ta tio n m e t h o d s ... 80

4.2 T h e colour te x tu r e s e g m e n ta tio n for M e t h o d - I ... 81

4.3 T h e colour te x tu r e s e g m e n ta tio n for M e th o d - I I ... 84

4.4 T h e colour te x tu r e se g m e n ta tio n for M e t h o d - I l l ... 89

5.1 Sam p le seg m en ted re s u lts fro m greyscale im ag es in M e th o d -I using L B P / C ... 96

5.2 R O C curves re p re se n tin g t h e p e rfo rm a n c e of grey scale L B P /C a n d D C T ... 97

5.3 (a) - (d) S eg m en ted re su lts of a m osaic in g rey scale a n d R , G a n d B p lan es resp e c tiv e ly u sing L B P / C ... 98

5.4 (a) - (d) S egm en ted re s u lts of b ird in g reyscale a n d R , G a n d B p lan es resp e c tiv e ly u sin g L B P /C ... 99

5.5 Sam p le seg m en ted re s u lts of m o saic a n d n a tu r a l im ag es from M e th o d -I using D C T ... 101

5.6 (a), (c), (e) a n d (g) show s th e seg m en ted re su lts, (b ), (d ), (f) an d (h) re p re se n ts th e c o rre sp o n d in g pixelw ise classified im ages . . . . 102

(12)

L I S T O F F IG U R E S

5.7 S am ple seg m en ted re su lts from M e th o d -II u sin g L B P /C a n d a

c h ro m a tic ity fe a tu re in R G B sp ace ...104

5.8 S am ple se g m e n ted re s u lts from M e th o d -II u sin g D C T a n d a ch ro m a tic ity fe a tu re in R G B s p a c e 105 5.9 R O C curves for L B P /C ap p ro a c h in greyscale a n d L B P /C w ith s ta tis tic a l c o lo u r fe a tu re s ta n d a r d d e v ia tio n in R G B sp a ce . . . . 106

5.10 R O C curves for D C T a p p ro a c h in greyscale a n d D C T w ith s ta tis tic a l colour fe a tu re s ta n d a r d d e v ia tio n in R G B s p a c e ...107

5.11 B a r c h a rt re p re se n tin g th e im p o rta n c e of c h ro m a tic ity fe a tu re s u sin g th e L B P /C a p p r o a c h ...108

5.12 B a r c h a rt re p re se n tin g th e im p o rta n c e of c h ro m a tic ity fe a tu re s u sin g th e D C T a p p r o a c h ... 109

5.13 C o m p a riso n of th e colour spaces in L B P / C ... 109

5.14 C o m p a riso n of th e colour spaces in D C T ... 110

5.15 C o m p ariso n of th e different a p p ro ach es in M e t h o d - I I ... 110

5.16 Influence of th e colour a n d te x tu re ...112

5.17 S am ple se g m e n ted re s u lts of n a tu r a l im ages ... 113

5.18 S am p le re su lts illu s tra te s th e p o o r p e rfo rm a n c e of th e pixelw ise c la s s if ic a tio n ... 115

5.19 R O C curves for L B P /C in greyscale a n d L B P /C w ith co lou r clus te rin g in R G B sp a ce ... 117

5.20 R O C curves for D C T in greyscale a n d D C T w ith co lo ur c lu ste rin g in R G B s p a c e ...118

5.21 B a r c h a rt re p re se n tin g t h e p e rfo rm a n c e of th e tw o ap p ro a c h e s in R G B , Y IQ a n d H SI colour s p a c e ...119

(13)

5.23 (a), (d ), (g) S e g m e n ta tio n b ased on colour, (b ), (e), (h) S egm en ta tio n b a sed on te x tu r e , (c), (f), (i) S e g m e n ta tio n b a se d on colour

a n d t e x t u r e ... 122

5.24 S am p le se g m e n ted re su lts from M e th o d -Ill u sin g L B P w ith colour c l u s t e r i n g ...124

5.25 S am p le se g m e n ted re su lts from M e th o d - Ill u sin g L B P w ith colour c l u s t e r i n g ...125

5.26 S am p le se g m e n ted re su lts from M e th o d - Ill u sin g D C T w ith colour c l u s t e r i n g ...126

5.27 S eg m en ted re s u lts of n a tu r a l im ages from M e th o d - Ill usin g L B P /C w ith colour c l u s t e r i n g ... . 1 2 7 5.28 S eg m en ted re s u lts of n a tu r a l im ages from M e th o d - Ill u sin g D C T w ith colour c l u s t e r i n g ... 128

5.29 T h e influence of a d a p tiv e sm o o th in g ... 130

5.30 T h e influence of noise, a-1% , b-2% , c-4% , in th e co lou r te x tu re se g m e n ta tio n ... 131

5.31 S e g m e n ta tio n effect of th e r o ta te d i m a g e s ... 132

5.32 C o m p a riso n o f M e th o d -II using L B P /C w ith co lo u r fe a tu re s a n d M e th o d - Ill u sin g L B P /C w ith colour c l u s t e r i n g ... 135

5.33 T h e e x e c u ta b le file for M e t h o d - I ... 136

5.34 T h e e x e c u ta b le file for M e th o d -II usin g L B P /C ... 137

5.35 T h e e x e c u ta b le file for M e th o d -II usin g D C T ...139

5.36 T h e e x e c u ta b le file for M e t h o d - I l l ...139

6.1 ( la ) , (2a), (3a) a n d (4a) rep re sen ts se g m e n ted re s u lts of ISO S im ages usin g M e th o d -Ill, ( lb ) , (2b), (3 b), (4b) re p re s e n ts re su lts of ISO S im ag es u sin g M e th o d - Ill a fte r th e b o u n d a ry refinem ent s t a g e ... 143

(14)

6.2 ( la ) , (2a), (3 a), (4a), (5a) rep re sen ts se g m e n te d re s u lts of skin can cer im ag es from M e th o d -Ill, ( lb ) , (2 b ), (3 b ), (4 b), (5b) rep re

se n ts th e re s u lts a fte r th e b o u n d a ry refin em e n t s ta g e ... 146

6.3 ( la ) , (2a) re p re se n ts seg m en ted re su lts o f S P I im ag es u sin g M eth o d -I ll, ( lb ) , (2b) re p re se n ts se g m e n ta tio n re s u lts a fte r th e b o u n d a ry refinem en t s t a g e ...148 B .l R G B C o lou r C u b e ...180 B .2 H ex ago n al P y r a m i d ...181 B .3 C IE c h ro m a tic ity d ia g ra m [ 9 9 ] ... 184 B .4 C o m p le m e n ta ry colour p a i r s ... 186

C .l D ifferent classes u sed in M e t h o d - I I ...188

C .2 D ifferent classes used in M e t h o d - I l l ...189

D .l M 1-M 6 are th e m osaic im ages c o n s tru c te d fro m V isT ex d a ta b a s e . 190 D .2 M 7-M 18 a re th e m osaic im ages c o n s tru c te d fro m V isT ex d a ta b a s e 191 D .3 M 19-M 30 a re th e m osaic im ages c o n s tru c te d fro m V isT ex d a ta b a s e 192 D .4 M 31-M 37 a re th e m osaic im ages from V isT ex d a ta b a s e , M 38 a n d M 39 w ere m osaic im ages u sed by M irm eh d i a n d P e tro u [86] . . . . 193

D .5 N 1-N9 a re th e n a tu r a l im ages from V isT ex d a ta b a s e , N10 an d N i l a re th e n a tu r a l im ages w hich w ere u sed by P a n jw a n i a n d H ealey [ 2 4 ] ...194

E . l S am ple se g m e n ted re s u lts of m osaic im ag es fro m M e th o d -I usin g L B P /C in different colour p lan es ... 195

E .2 S am p le se g m e n ted re s u lts of m osaic a n d n a tu r a l im ag es from M e th o d -I usin g L B P /C in different colour p l a n e s ...196

F . 1 Sam ple seg m en ted re s u lts from M e th o d -II u sin g L B P / C a n d colour fea tu re s ... 197

(15)

L I S T O F F IG U R E S

F.2 S am p le se g m e n ted re su lts from M e th o d -II u sin g L B P /C a n d colour f e a t u r e s ... 198 F.3 S am p le se g m e n ted re s u lts from M e th o d -II u sin g D C T a n d colour

f e a t u r e s ... 199

G .l S am p le se g m e n ted re s u lts from M e th o d -Ill u sing L B P /C w ith colour c lu s t e r i n g ...200 G .2 S am p le se g m e n ted re s u lts from M e th o d - Ill u sing L B P /C w ith

colour c lu s t e r i n g ...201 G .3 S am p le seg m en ted re s u lts o f n a tu r a l im ag es fro m M e th o d -Ill using

L B P /C w ith c o lo u r c l u s t e r i n g ... 202 G .4 S am p le se g m e n ted re s u lts from M e th o d -Ill u sin g D C T w ith colour

c l u s t e r i n g ...202 G .5 S am p le se g m e n ted re s u lts from M e th o d -Ill u sin g D C T w ith colour

c l u s t e r i n g ...203 G .6 S am p le se g m e n ted re s u lts of m osaics a n d n a tu r a l im ages from

(16)

List o f T ables

5.1 P e rce n tag e o f se g m e n ted re su lts - S ignificance of co lo ur fea tu re s in co lo ur te x tu r e a n a l y s i s ... 108 5.2 Q u a n tita tiv e e v a lu a tio n for different b lock s i z e ... 116 5.3 T h e p e rfo rm a n c e of th e L B P /C w ith c o lo u r c lu ste rin g a n d D C T

w ith c o lo u r c lu s te rin g in R G B , Y IQ a n d IISI sp a ce s as p e rc e n ta g e of se g m e n ted re s u lts ... 119 5.4 A verage s e g m e n ta tio n erro r (%) d e p e n d in g on th e colour a n d th e

L B P te x tu r e w e i g h t s ... 121 5.5 A verage s e g m e n ta tio n erro r (%) d e p e n d in g on th e colour a n d th e

(17)

C on ten ts

1 In tro d u ctio n 1 1.1 R esearch O b je c tiv e a n d M o t i v a t i o n ... 1 1.2 D efinitions o f T e x tu re , C o lour a n d C o lo u r T e x t u r e ... 3 1.3 S tu d y o f T e x t u r e s ... 6 1.4 C o lo u r A n a l y s i s ... 12 1.5 Issues in C o lo ur T e x tu re S e g m e n t a t i o n ... 13 1.6 O rg a n isa tio n of th e T h e s i s ... 16 1.7 S u m m a ry ... 17 2 R eview o f R e la te d W ork 18 2.1 I n t r o d u c t i o n ... 18 2.2 R eview of T e x tu re S t u d i e s ... 18 2.2.1 T e x tu re S e g m en ta tio n ... 18 2.2.2 T e x tu re C la s s ific a tio n ... 23 2.2.3 R e la te d T e x tu re B ased A p p r o a c h e s ... 30

2.3 R eview o f C o lo u r Im age S e g m e n ta tio n ... 36

2.4 R eview of C o lo u r T e x tu re R e s e a r c h ... 41

2.4.1 C o lo u r T e x tu re S e g m e n t a t i o n ... 41

2.4.2 C o lo u r T e x tu re C lassificatio n ... 46

2.4.3 C o lo u r T e x tu re Im age R e t r i e v a l ... 51

(18)

C O N T E N T S

2.4.5 R e la te d C olour T e x tu re B a se d A p p r o a c h e s ... 53

2.5 S u m m a ry ... 55

3 Im age P r o c essin g A n a ly sis T ech n iq u es 57 3.1 I n t r o d u c t i o n ... 57 3.2 F e a tu re E x tra c tio n T e c h n iq u e s ... 57 3.2.1 T e x tu re F e a tu re E x tra c tio n T e c h n i q u e s ... 58 3.2.2 C o lo ur F e a tu re E x tra c tio n T ec h n iq u e s ... 61 3.3 S electio n of C o lo ur Space ... 64 3.4 A d a p tiv e S m o o t h i n g ... 65 3.5 N o n -p a ra m e tric T e s t ... 68 3.5.1 G - s t a t i s t i c ... 69

3.5.2 M odified-K olm ogorov S m irn o v ( M - K S ) ... 69

3.6 U n su p e rv ise d T e x tu re S e g m e n ta tio n M e t h o d ... 70 3.7 B o u n d a ry R e f in e m e n t... 77 3.8 S u m m a ry ... 78 4 M eth o d o lo g y 7 9 4.1 I n t r o d u c t i o n ... 79 4.2 C o lo ur T e x tu re S e g m en ta tio n M e th o d s a n d D e s c r i p t i o n ... 80 4.2.1 C o lo u r T e x tu re S e g m e n ta tio n - M e t h o d - I ... 80

4.2.2 C o lo u r T e x tu re D e sc rip tio n for M e t h o d - I ... 82

4.2.3 C o lo u r T e x tu re S e g m e n ta tio n - M e t h o d - I I ... 83

4.2.4 C o lo u r T e x tu re D e sc rip tio n for M e th o d -II ... 85

4.2.5 C o lo u r T e x tu re S e g m e n ta tio n - M e t h o d - I l l ... 88

4.2.6 C o lo u r T e x tu re D e sc rip tio n for M e t h o d - I l l ... 90

4.3 S u m m a ry ... 93

5 R esu lts and D iscu ssio n 94 5.1 I n t r o d u c t i o n ... 94

(19)

C O N T E N T S 5.2 S e g m e n ta tio n R e su lts U sing M e th o d -I ... 95 5.3 S e g m e n ta tio n R e su lts U sing M e t h o d - I I ...103 5.3.1 P e rfo rm a n c e E v a l u a t i o n ...105 5.3.2 Influence of T e x tu re a n d C o l o u r ...I l l 5.4 S e g m e n ta tio n R e su lts U sing M e t h o d - I l l ... 116 5.4.1 P e rfo rm a n c e E v a l u a t i o n ...116 5.4.2 Influ ence of T e x tu re a n d C o l o u r ... 123 5.5 D iscu ssion ...134 5.6 A lg o rith m I m p l e m e n ta t io n ... 136 5.7 S u m m a r y ... 140

6 A p p lica tio n o f th e D ev elo p ed C olour T ex tu re S eg m e n ta tio n M e th o d olo g y 141 6.1 I n t r o d u c t i o n ...141

6.2 C o lo u r T e x tu re S e g m en ta tio n A p p l i c a t i o n s ... 142

6.2.1 Iris h S c rip t on Screen Im ag es ... 142

6.2.2 S k in C a n c e r Im ages ... 144

6.2.3 S e d im e n t P rofile I m a g e r y ... 147

6.3 S u m m a ry ...150

7 C o n clu sio n s 151 8 T h esis C o n trib u tio n and F u ture W ork 156 8.1 T h esis C o n t r ib u t io n ...156

8.2 F u tu re W o r k ... 159

B i b l i o g r a p h y 161

List o f P u b lic a tio n s 172

(20)

C O N T E N T S

A T extu re F ea tu re E x tra ctio n T ech n iqu es 173

A .l C o - o c c u r r e n c e ...173 A .2 G a b o r F i l t e r s ...175 A .3 M ark ov R a n d o m F i e l d ... 176 A .4 F r a c t a l s ... 177 A p p en d ix B 179 B C olour Sp aces 179 B .l R G B S p a c e ... 179 B .2 H S I S p a c e ...180 B .3 H S V S pace ... 181 B .4 Y IQ S p a c e ...182 B .5 Y U V S p a c e ... 183 B.6 C IE -X Y Z S p a c e ... 183 B .7 C IE -L U V S p a c e ... 185 B .8 C IE -L A B S p a c e ... 186 A p p en d ix C 188 C Im p le m e n ta tio n 188 A p p en d ix D 190 D D a ta b a se Im ages 190 D .l D a ta b a s e o f M osaic Im ages . ...190 D .2 D a ta b a s e o f N a tu ra l I m a g e s ... 194 A p p en d ix E 195 E R e su lts - M e th o d -I 195 A p p e n d ix F 197

(21)

C O N T E N T S F R e su lts - M eth o d -II 197 F . l R e s u lts u sin g L B P /C a n d colour f e a t u r e s ... 197 F .2 R e s u lts u sin g D C T a n d co lo ur f e a t u r e s ... 199 A p p en d ix G 200 G R e su lts - M e t h o d - I ll 2 0 0

G .l R e su lts u sin g L B P /C w ith colour c l u s t e r i n g ...200 G .2 R e s u lts u sin g D C T w ith colour c l u s t e r i n g ... 202

(22)

C h ap ter 1

In trod u ction

1.1 R e s e a r c h O b je c tiv e a n d M o tiv a tio n

M o st n a tu r a l su rfaces e x h ib it te x tu re s . T e x tu re is observ ed as s tr u c tu r a l p a tte r n s in th e g rain , grass, clo th , w ood, b rick a n d sa n d . It is a n im p o r ta n t c h a ra c te ris tic u sed in in te rp re tin g th e im ag e in fo rm a tio n . T e x tu re refers to th e p a tt e r n of in te n s ity v a ria tio n s in a n im ag e a n d h a s v a ry in g degrees of ran d o m n e ss a n d reg u larity. T h e co arseness or fineness of te x tu r e d im ages a re re la tiv e a n d d e p e n d s on th e scale o f th e im age. F or e x am p le in e x a m in in g an im ag e from a larg e d ista n c e th e im ag e m a y a p p e a r sm o o th , w ith o u t a n y d istin g u ish e d te x tu re s . O n d e c re a s in g th e d ista n c e , th e te x tu re a p p e a rs fine a n d if th e d ista n c e is f u rth e r d e c re a sed th e te x tu r e m a y a p p e a r coarse. H ence te x tu r e d e sc rip tio n is scale d e p e n d e n t.

C olour is a p r o p e rty of en o rm o u s im p o rta n c e to h u m a n v isu a l p e rc e p tio n a n d is a rich so u rce of in fo rm a tio n for im a g e analy sis. I t is a n in trin sic a tt r ib u t e of an im ag e a n d p rovides m o re in fo rm a tio n th a n a single in te n s ity v alue. In c o m p u te r vision, colour is used in im a g e se g m e n ta tio n , im ag e classificatio n a n d im ag e d a ta b a s e retriev al. H u m a n p e rc e p tio n of colour d e p e n d s on th e s p a tia l freq u en cy of th e colour co m p o n en t. Specifically, th e p e rc e p tu a l resp o n se of th e

(23)

C H A P T E R 1. IN T R O D U C T IO N

human visual system to a certain p art of the electromagnetic spectrum depends on the frequency with which this stimulus is spatially distributed. Hence, colours in multi-colour p attern are perceived differently than from uniform areas [1].

Texture and colour are widely accepted as two key factors in image analysis. Texture and colour properties have been regarded separately rath er th an collec tively. Texture and colour are inseparable, since textures have a colour aspect and coloured surfaces are textured. Most work in the past on texture analysis and segmentation derived structural description for texture, example coarseness, regularity, blobiness, orientation etc., with the colour information used as an extra cue [1]. Recently, several studies were directed to the problem of joint rep resentation of texture and colour. Numerous approaches for texture analysis and colour analysis have been developed. Though colour and texture are inherently related to each other there does not yet exist a reliable m ethod to combine both colour and texture for colour texture segmentation. Hence, the main objective of this research work is to find a methodology for colour texture segmentation and to examine the contribution of colour in the analysis of textures i.e., to determine the use of colour information in the colour texture segm entation process.

Colour texture analysis plays a vital role in many applications of com puter vision. Im portant applications include defect detection in industrial surface inspection, detection of defects in textile and paint inspection, assessment of carpet wear in quality control and detection of damage regions in script images. In addition, colour texture analysis is used in rem ote sensing for ground classification, seg m entation of satellite or aerial images, segmentation of underwater images and in several medical applications. Image textures are used in combination with colour features to diagnose leukemic malignancy in samples of stained blood cells [2]. Colour textures are used in the classification of blood cells and in the segmenta tion of skin cancer images for disease detection in biomedical image processing.

(24)

Some other useful applications are, th e segmentation of textured regions in doc ument analysis, to find the quality of th e food in food industries, and in image retrieval for classification [3]. The existence of a variety of applications is the m otivating factor for this research work. The applications involved in this re search work are th e segmentation of script images, skin cancer images and th e underwater images.

1.2 D e fin itio n s o f T e x tu r e , C o lo u r a n d C o lo u r

T e x tu r e

D efin itio n o f T ex tu re

Extensive work has been carried out in the area of tex tu re analysis for several decades and many papers have been published, b u t there does not exist a widely accepted definition of texture. Formulation of a m athem atical description to define a texture has not been carried out so far due to the variation in the textures.

Some definitions of texture from image processing handbooks and well known research publications are as follows:

• S k lansky-Im age S eg m en ta tio n and F eatu re E x tr a ctio n [4]

A region in an image has a constant texture if a set of local statistics or other local properties of the picture function are constant, slowly varying or approxim ately periodic.

• W ilso n and Sp an n -Im age S eg m e n ta tio n a n d U n c e rta in ty [5]

Textured regions are spatially extended patterns based on the more or less accurate repetition of some unit cell called texton or subpatterns.

• A .K .J a in -F u n d a m en ta ls o f D ig ita l Im age P r o c e ssin g [6]

(25)

called texels. T h e tex e l c o n ta in s several pixels, w hose p la c e m e n t could b e p erio d ic, q u asi-p erio d ic or ra n d o m . N a tu ra l te x tu re s are g e n e ra lly ra n d o m , w h ereas artificial te x tu re s a re o ften d e te rm in is tic or p e rio d ic. T e x tu re m ay b e coarse, fine, sm o o th , g ra n u la te d , rip p le d , reg u la r, irre g u la r or linear.

• IE E E Stan dard G lossary o f Im age P r o c essin g and P a tte r n R e c o g  n itio n T erm in ology [7]

T e x tu re is an a ttr ib u te re p re s e n tin g th e s p a tia l a rra n g e m e n t of th e grey levels of th e pixels in a region.

• R .C .G o n za lez and R .E .W o o d s-D ig ita l Im age P r o c e ssin g [8]

W e in tu itiv e ly view te x tu r e as a d e sc rip to r t h a t p ro v id es a m ea su re of p ro p e rtie s such as sm o o th n e ss, coarsen ess a n d reg u larity.

• R .M .H aralick and L .G .S h a p iro -C o m p u ter an d R o b o t V isio n . V ol um e 1 [9]

T e x tu re of an im age is n o n -fig u ra tiv e a n d cellular. It is d e sc rib e d b y th e n u m b e r a n d ty p e s of its (to n a l) p rim itiv e s a n d th e s p a tia l o rg a n is a tio n or lay o u t of its (to n a l) prim itiv es. T h e s p a tia l o rg a n isa tio n m a y b e ra n d o m , m ay have a p airw ise d e p e n d en c e of one p rim itiv e on a n e ig h b o u rin g p rim i tive, or m ay have a d e p e n d en c e of n p rim itiv e s a t a tim e. T h e d e p e n d en c e m ay b e s tru c tu ra l, p ro b a b ilistic or fu n ctio n al.

• B ern d J a h n e-D ig ita l Im age P r o c essin g [10]

P a tte r n s w hich c h a ra c te rise o b je c ts a re called te x tu re s in im ag e processing.

T h is collection of d efin itio ns im p lies t h a t th e te x tu r e is fo rm u la te d b y different p eo p le d e p e n d in g u p o n th e p a rtic u la r a p p lic a tio n a n d su g g ests t h a t a fo rm al m o d el to d escrib e te x tu r e does n o t ex ist. T h is is d u e to th e fa c t t h a t th e te x tu r e h a s b een d e scrib ed u sin g in tu itiv e m o d els r a th e r t h a n a n a n a ly tic a l fo rm u la tio n . B u t a lth o u g h a u n iv ersa lly a c c e p te d d efin itio n h as n o t b een agreed, tw o m ain

(26)

c h a ra c te ris tic s t h a t a re c o m m o n for all abo ve m e n tio n e d d efin itio n s c a n b e id e n tified. T h e first reflects th e f a c t t h a t th e te x tu r e c a n b e a s so c ia te d w ith im ag e regio ns w h ere t h e n e ig h b o u rin g pixels p re s e n t a sig n ific a n t v a ria tio n in in te n s ity levels w h ereas th e seco n d re g a rd s th e te x tu r e as a h o m o g e n o u s p r o p e r ty t h a t is r e p e a te d a t a w ell d efin ed s p a tia l scale. E a rly w o rk d iv id e d te x tu r e s in to tw o m a in ty p es, m icro a n d m a c ro te x tu r e s . M icro te x tu r e s a re te x tu r e s w ith sm a ll g rey level p rim itiv e s a n d th e s p a tia l in te r a c tio n b e tw e e n p rim itiv e s is c o n s tra in e d t o b e v ery local. M acro te x tu r e s a re te x tu r e s of g rey level p rim itiv e s la rg e r in size th a n th e in d iv id u a l p ixel. T h e s e p rim itiv e s h av e th e ir ow n id e n tifia b le s h a p e p ro p e rtie s . A n e x a m p le fo r m ic ro a n d m a c ro te x tu r e s is sh o w n in F ig u re 1.1. D if fe re n t te c h n iq u e s a n d m e th o d o lo g ie s w ere d e v e lo p e d to d e sc rib e th e s e te x tu r e s . A d e ta ile d e x p la n a tio n o n th e s e te c h n iq u e s is fo u n d in [11].

(a) _(b)

(c) (d)

(27)

D e fin itio n o f C olour

C o lo u r is a p e rc e p tu a l re s u lt of light in th e visible region of th e s p e c tru m , h a v in g w aveleng th s from 400nm to 700nm , in cid e n t u p o n th e re tin a . T h e h u m a n r e tin a h a s th re e ty p e s of colour p h o to re c e p to rs, h ence th re e c o m p o n e n ts (R - R e d , G - G reen a n d B - B lue) are n e c essa ry a n d sufficient to d e scrib e a colour. C olour science h a s m a in ly c o n c e n tra te d on th e rese a rc h p ro b le m s, w h ich a re b a se d on th e tri-c h ro m a tic ity m o d el of th e h u m a n v isu al sy stem . In th is m o d el, th e id ea is t h a t th e colour visio n is b a s e d on th re e colour sen sitiv e sensors, w hich h ave different w av eleng th s e n sitiv ity w ith each o th e r [12]. C o lo u r vision is in h e re n tly tri-c h ro m a tic [13].

C olour is a n im p o r ta n t p r o p e r ty a n d a d v a n ta g e o u s over in te n sitie s. E a c h im ag e p ixel is re p re se n te d b y th re e b y te s c o rre sp o n d in g to th e v alu es of th e red , green a n d b lue co m p o n en ts. H ence th re e values in th e p lace of single in te n s ity value re su lts in m o re in fo rm a tio n for im age analysis.

D e fin itio n o f C olou r T ex tu re

T w o im ages w ith th e sam e colour a n d differen t te x tu r e p a tte r n s o r th e sam e te x tu r e p a tte r n w ith different colours a re con sid ered as tw o different colour tex - tu r e d im ages. C o lo u r te x tu r e can b e re g a rd e d as a p a tt e r n d e scrib ed b y th e rela tio n sh ip b etw een its c h ro m a tic a n d s tr u c tu r a l d is tr ib u tio n [14].

1 .3

S tu d y o f T e x tu r e s

T ex tu re S y n th e sis an d A n a ly sis

T h e th re e im p o r ta n t issues in te x tu re s a re te x tu r e sy n th e sis, te x tu r e classifica tio n a n d te x tu r e se g m e n tatio n .

(28)

(c) (d)

F ig u re 1.2: (a), (c) - I n p u t Im a g e (b ), (d) - S y n th e sis e d Im a g e [15]

to re n d e r te x tu r e im ag es w h ic h a re p e rc e p tu a lly sim ila r to t h e o b se rv e d te x tu r e ex am p les. T h is is o fte n u se d in im a g e co m p re ssio n a p p lic a tio n s a n d in c o m p u te r g ra p h ic s to re n d e r o b je c t su rfa c es. T e x tu re fe a tu re e x tr a c tio n te c h n iq u e s su c h as M a rk o v R a n d o m F ie ld (M R F ) a n d f ra c ta l m o d els (see A p p e n d ix A ) a re u se d to g e n e ra te s y n th e tic a lly te x tu r e d im ag es as sh o w n in F ig u re 1.2.

T h e se co n d issu e is o n th e texture classification. F ig u re 1.3 sho w s d iffe re n t ty p e s o f te x tu r e s t h a t a re id e n tifie d as b a rk , b rick , s a n d a n d fab ric. Id e n tific a tio n a n d

(29)

nr

Sic.

i :

i t

(a) (b)

(c) (d)

F ig u re 1.3: (a) B a rk (b) B rick (c) S a n d (d) F a b ric . D iffe ren t ty p e s of V isT ex [27] te x tu r e s a n d th e ir id e n titie s

Sand _Stone

Water Tile

F ig u re 1.4: D ifferent h o m o g en e o u s te x tu r e s w ere la b e lle d in te x tu r e c lassificatio n

classificatio n of th e s e ty p e s of h o m o g en eo u s reg io n s is c a lle d t e x tu r e classification . T h is a p p ro a c h is c o n c ern e d w ith th e p a tt e r n re c o g n itio n t a s k o f te x tu r a l fe a tu re e x tr a c tio n i.e., to e x tr a c t t h e te x tu r e fe a tu re s u sin g v ario u s te c h n iq u e s su c h as

(30)

co -o ccu rren ce, D isc re te C osine T ra n sfo rm (D C T ), m o rp h o lo g y e tc ., follow ed by te x tu r e classificatio n . T h e th ir d issue is o n th e texture segmentation. T e x tu re s e g m e n ta tio n involves p a rtitio n in g a n im ag e in to a se t of n o n -o v e rla p p in g regio ns w hose u n io n fo rm s th e e n tire im ag e. V ario u s m e th o d s su c h as sp lit a n d m erg e, reg io n grow ing a re a d o p te d to o b ta in th e s e g m e n ta tio n re su lts.

C lassificatio n a n d s e g m e n ta tio n p rocesses h av e closely r e la te d o b jec tiv e s. C la s sifica tio n c a n lead to s e g m e n ta tio n , a n d vice v ersa. C la ssific a tio n of p ix els in a n im ag e is a n o th e r fo rm o f c o m p o n e n t lab e llin g , t h a t c a n re s u lt in s e g m e n ta tio n of th e o b je c ts in th e im a g e [6]. S im ilarly, im a g e s e g m e n ta tio n b y te m p la te m a tc h in g , as in c h a ra c te r rec o g n itio n , lead s to c la ssific a tio n o r id e n tific a tio n of e ach o b je c t. T h e b a sic a p p ro a c h for s e g m e n ta tio n is to c o m p u te th e te x tu r e fe a tu re s over lo cal n e ig h b o u rh o o d s . T h e n e ig h b o u rh o o d is p a r titio n e d a n d clu s te r e d u sin g reg io n g row ing a n d s h rin k in g m e th o d s . T e x tu re s e g m e n ta tio n u se s a h o m o g en e ity m e a su re w h ich is a fu n c tio n of th e t e x tu r e fe a tu re s.

T h e re a re tw o ty p e s of t e x tu r e s e g m e n ta tio n , su p e rv is e d a n d u n su p e rv is e d . S u p e rv ise d s e g m e n ta tio n re q u ire s a p rio ri in fo rm a tio n re g a rd in g th e d iffe re n t te x tu re s p re s e n t in a n im ag e. If no a s s u m p tio n s c a n b e m a d e a b o u t th e ty p e of te x tu r e s , th e n u n s u p e rv is e d m e th o d is used . I n s u p e rv is e d t e x tu r e s e g m e n ta tio n

(31)

each fe a tu re v e c to r is assig n ed to a class as it is g e n e ra te d , w h ereas in th e u n s u p erv ised se g m e n ta tio n , th e s ta tis tic a l analysis m u st b e p e rfo rm e d on th e e n tire d istrib u tio n of v ecto rs. T h e goal is to recognise c lu ste rs in th e d is trib u tio n a n d assign th e sam e la b e l to th e m all. In general, th is is a m u ch m o re co m p lex ta s k to accom plish.

S e g m en ta tio n of a n im ag e e n ta ils th e division o r s e p a r a tio n of th e im age in to regions of sim ila r a ttr ib u te s . T h e m o st basic a tt r ib u t e for s e g m e n ta tio n is th e im age a m p litu d e -lu m in a n c e for a m o no chrom e im ag e a n d th e colo ur c o m p o n e n ts for a colour im ag e [16]. A s ta n d a r d th e o ry of im age s e g m e n ta tio n does n o t ex ist. As a co n sequence, no single s ta n d a r d m e th o d of im ag e s e g m e n ta tio n h a s em erged. T h e re is a co llection of m e th o d s t h a t h av e received som e d egree of p o p u larity . H aralick a n d S h ap iro [9] have e sta b lish e d g u id elin es for a g o o d im age seg m en tatio n :

• R egions of a n im ag e s e g m e n ta tio n sh o u ld b e u n ifo rm a n d hom ogeneous w ith re sp e c t to som e c h a ra c te ris tic such as g rey to n e or te x tu re .

• R egion in te rio rs sh o u ld b e sim ple a n d w ith o u t m a n y sm all holes.

• A d ja c en t reg io n s of a s e g m e n ta tio n sh o u ld have sig n ifican tly differen t values w ith re sp e c t to th e c h a ra c te ris tic on w h ich th e y a re unifo rm .

• B o u n d a rie s o f each seg m en t sh o u ld b e sim ple, n o t rag g e d a n d m u st b e s p a tia lly a c c u ra te .

U n til recently, a lim ite d n u m b e r of q u a n tita tiv e im ag e s e g m e n ta tio n p erfo rm an ce m etric s has b een develo p ed. T h e m e th o d s a d o p te d for co lo ur te x tu r e se g m e n ta tio n in th is rese a rc h w o rk a re e x p lain ed in c h a p te r 4.

(32)

T ex tu re D e sc r ip tio n M eth o d s

T u cery an et al. [2] d iv id e d th e te x tu re fe a tu re e x tra c tio n tec h n iq u es u sed for te x tu re d e sc rip tio n in to fo ur categories as: s ta tis tic a l, g eo m etrical, m o d el b a sed a n d signal pro cessin g.

• S ta tis tic a l m e th o d s an a ly se th e s p a tia l d is tr ib u tio n of grey values, b y com p u tin g local fe a tu re s a t each p o in t in th e im ag e, a n d d eriv in g a set of s ta tis tics from t h e d is trib u tio n s of th e local fe a tu re s. B a se d on th e p e rfo rm a n c e of th e m e th o d a n d th e te x tu r e in fo rm a tio n c a p tu r e d , th e m o st w idely used s ta tis tic a l m e th o d s a re co-occurrence fe a tu re s, g re y level differences, sign ed differences a n d L o cal B in a ry P a tte r n (L B P ). O th e r s ta tis tic a l ap p ro a c h e s in clu de th e a u to c o rre la tio n fu n ctio n a n d th e g rey level ru n le n g th m e th o d .

• G eo m etrical m e th o d s consider te x tu re s c o m p o se d o f te x tu r e p rim itiv e s a n d th e ru les b a se d on th e ir sp a tia l o rg an isa tio n . T h e p rim itiv e s w ere e x tra c te d b y edge d e te c tio n w ith a L a p lac ian -o f-G au ssian [2], b y a d a p tiv e regio n e x tra c tio n or b y m a th e m a tic a l m orphology.

• M odel b a se d m e th o d s a re b a sed on th e te x tu r e p rocess b y c o n s tru c tin g a p a ra m e tric g e n e ra tiv e m odel w hich c re a te s t h e observ ed in te n s ity d is trib u tio n . T h e m o d el can e ith e r b e p ix el b a s e d or region based . P ix e l b a sed m o d els v iew a n im age as a collection o f pixels, w hereas region b a sed m odels re g a rd a n im ag e as a se t of s u b p a tte r n s p lac e d acco rd in g to th e given rules. V ario u s ty p e s of m odels c a n b e o b ta in e d w ith d ifferent n eig h b o u rh o o d s. E x a m p le s for th is ty p e of m o d el in clu d e s one d im en sio n al tim e series m o d el, a u to re g re ssiv e a n d m oving av erag e m odel. R a n d o m field m o d els an aly se s p a tia l v a ria tio n s in tw o dim en sio n s. E x a m p le s of ra n d o m field m odels in clu d e M R F m odel, G a u ssia n M R F m o d el a n d G ib bs ra n d o m field m odel [17] (see A p p e n d ix A).

(33)

tia l d o m a in filters su ch as L aw s’ m ask s, lo cal lin e a r tra n sfo rm s a n d vario u s m ask s desig n ed for edge d e te c tio n are th e m o st d ire c t a p p ro a c h for c a p tu rin g fre q u e n c y in fo rm a tio n . T h e fre q u e n c y a n a ly sis of th e te s te d im ag e w as also e v a lu a te d b y th e m e th o d s such as F o u rie r analysis. O th e r sig nal processin g m e th o d s in clu de G a b o r tra n s fo rm a n d w avelet b a se d m e th o d s.

1.4 C o lo u r A n a ly s is

C olour im age a n a ly sis h a s received in cre ase d a tte n tio n in c o m p u te r vision d u r ing th is decade. T h e a v ailab ility of pow erful c o m p u ta tio n a l m ach in es co m b ined w ith novel te c h n iq u e s d ev elop ed a n d th e tec h n o lo g ic a l p e rfo rm a n c es achieved by colour sensors h av e im p o sed th e use of co lo u r as a p refe ra b le a lte rn a tiv e to s ta n d a rd grey level ap p ro ach es. In a d d itio n , colo u r is a n im p o r ta n t p a r t of th e h u m a n v isu al sy stem . I t is a n in trin sic a tt r ib u t e of a n im age a n d p ro vid es m ore in fo rm a tio n t h a n th e g rey level in te n sity values. A colour sp ace is a m e th o d by w hich colour c a n b e specified, c re a te d a n d visu alised . T h e co n cep t of colour space refers to t h e c a rte s ia n space in w hich th e v isu a l se n sa tio n of colour c a n b e un iq u ely defin ed b y a se t of n u m b ers re p re s e n tin g c h ro m a tic featu res. C olour re p re se n ta tio n is b a se d on th e classical th e o ry of T h o m a s Y oung [2], H e s ta te d t h a t an y colour c a n b e re p ro d u c e d b y m ix in g a n a p p ro p ria te set of th re e p rim a ry colours.

C olour is a p e rc e p tu a l p h e n o m e n o n re la te d to h u m a n resp o n se to d ifferent w ave len g th s in th e visib le e le c tro m ag n e tic sp e c tru m . C o lo u r sp ace is a m o d el w hich rep re sen t colours in te rm s of in te n s ity values. A colour m o d el is th e geo m etric re p re se n ta tio n of colours in a th re e d im e n sio n a l space. E a c h colour m o d el h a s its own c h a ra c te ristic , th e c h a ra c te ristic s g e n e ra lly u se d to d istin g u ish one colour from a n o th e r a re b rig h tn e ss, hue a n d s a tu ra tio n . H u e a n d s a tu r a tio n ta k e n to g e th e r are called c h ro m a tic ity a n d th ere fo re a co lou r m ay b e c h a ra c te rise d b y its

(34)

b rig h tn e ss a n d eh ro m aticity . S angw ine [18] p re s e n te d a n exten siv e s tu d y on th e n a tu re of colour, its electro n ic re p re s e n ta tio n a n d th e co n c ep ts of lu m in an c e a n d chrom inance. T h e fu n d a m e n ta l colour sp ace in im ag e pro cessin g is R G B , w hich is an a d d itiv e co lo ur space. In c o m p u te r p ro ce ssin g of colour im ages, v ariou s colour m o dels a re u sed for different p u rp o se s. I n p ra c tic e a m o re a p p ro p ria te a p p ro ach is to u tilise a colour space, w h e re th e in fo rm a tio n is re p re se n te d in a b e tte r w ay for th e ap p lic atio n . T h e re a re a n u m b e r of colour sp aces in use d e p en d in g on th e a p p lic a tio n involved. Som e o f th e w idely u sed colour spaces are HSI, H SV , Y IQ , C IE -X Y Z , C IE -L A B , C IE -L U V , etc., [19].

In colour an aly sis, colour plays tw o im p o r ta n t roles, q u a n tita tiv e a n d q u a lita tive. In a q u a n tita tiv e role th re e im age fe a tu re s fro m th re e different p la n e s a re o b ta in e d as c o m p a re d to one im age fe a tu re from g rey scale im age. T h e a lg o rith m is derived from th o se u sed in m o n o ch ro m atic im a g e an aly sis w ith th e colour b e in g th e a d d itio n a l in fo rm a tio n . C olour im proves th e s e g m e n ta tio n w ith o u t in c re a s ing th e c o m p le x ity o f th e alg o rith m s [20]. In a q u a lita tiv e role, colour is u se d to acquire in trin sic p h y sical p ro p e rtie s from n a tu r a l im ag es [20].

1.5 I s s u e s in C o lo u r T e x tu r e S e g m e n ta tio n

T h e re are som e issues in colour te x tu r e s e g m e n ta tio n such as:

• S e g m e n ta tio n is a c o m p u ta tio n a lly in te n siv e ta s k . T h e re is a d e p e n d en c y b etw een th e im ag e reso lu tio n a n d th e sp e e d o f processing. T h e sp e e d of s e g m e n ta tio n also d e p e n d s on fe a tu re e x tra c tio n from te x tu r e a n d colour p lan es a n d th e se g m e n ta tio n process. H ence, th e speed of p ro cessin g is a n im p o rta n t c o n sid e ra tio n th ro u g h o u t th e d e v elo p m en t of th e colour te x tu r e s e g m e n ta tio n a lg o rith m in th is stu d y . T h e re q u ire m e n t for s e g m e n ta tio n is a c o m p u ta tio n a lly sim ple, effective a n d efficient tech n iq u e. T h is re su lte d in th e selectio n of sim ple tec h n iq u es su ch as th e L B P a n d th e D C T b a sed

(35)

filter a p p ro a c h , t h a t derives te x tu r e fe a tu re s fro m th e lu m in a n c e p lanes. T h e e x tra c tio n of colour using s ta tis tic a l fe a tu re s or u n su p e rv ise d fc-means colour c lu ste rin g pro ved an efficient w ay to e x tr a c t colour fea tu re s.

• T h e te x tu r e a n d th e colour in n a tu r a l im ag es a re o ften n o n -u n ifo rm d u e to th e c h an g e in scale, o rie n ta tio n o r o th e r v isu a l d isto rtio n s [21]. I t is also affected b y un even illu m in a tio n c o n d itio n s. T h is causes p ro b le m s in colour te x tu r e im age se g m e n tatio n . T h e m e th o d s for te x tu r e s e g m e n ta tio n develo p ed h av e only occasionally evolved in real-w o rld a p p lic a tio n s [22], In general, th e an alysis o f n a tu r a l im ages h a s pro v ed to be e x tre m e ly h a rd . H ence th e ap p ro a c h e s sh o u ld b e c o m p u ta tio n a lly cheap a n d ro b u s t a g a in s t th e v a ria tio n s aforesaid.

• N a tu ra l im ag es do n o t h ave well defin ed b o u n d a rie s. A s a re s u lt, seg m e n ta tio n e rro rs will arise w hile definin g th e b o u n d a rie s b etw e en tw o no n- hom o g en eo u s regions. T h e se e rro rs a re overcom e b y a p p ly in g a se g m e n ta tio n refin em en t a lg o rith m , pixelw ise classificatio n or a b o u n d a ry refinem ent a lg o rith m , in th is research work.

• T h e in c o rp o ra tio n of colour in colour te x tu r e a n a ly sis is a n o th e r issue in colour te x tu r e se g m e n tatio n . V arious m e th o d s are a d o p te d to in c o rp o ra te th e colour in fo rm a tio n in to te x tu r e an aly sis. T h e selectio n of a p p ro p ria te m e th o d s to in c o rp o ra te th e c h ro m a tic in fo rm a tio n in to te x tu r e an aly sis p lays a v ita l role for colour te x tu r e se g m e n ta tio n .

T h e w idely u sed m e th o d s a re as follows:

T h e t h r e e s p e c t r a l b a n d m e t h o d T h is is a ty p ic a l m e th o d t h a t h a s th e a d v a n ta g e of sim p licity w h ich e x te n d s t h e s ta n d a r d te x tu r e an aly sis te c h n iq u es su ch as co -o ccurrence, L B P, D C T , G a b o r filters, etc., to colour im ages a n d th e re su lt is o b ta in e d in each colour p la n e in d iv id u a lly [23].

(36)

I n t e r a c t i o n b e t w e e n s p e c t r a l b a n d s T h is a p p ro a c h uses th e sp a tia l in te ra c tio n b etw e en different sp e c tra l b a n d s. P a n jw a n i a n d H ealey [24] p re s e n te d a n u n su p e rv ise d te x tu re se g m e n ta tio n a lg o rith m b a se d on M R F m o d els for colour te x tu re s . T h e ir m odels c h a ra c te ris e d a te x tu r e in te rm s of s p a tia l in te ra c tio n b etw een s p e c tra l b a n d s.

I n f o r m a t i o n f r o m u n i c h r o m e f e a t u r e s a n d o p p o n e n t f e a t u r e s T h is m e th o d em ploys th e in fo rm a tio n from b o th in d iv id u a l colour ch an n el a n d cro ss c o rre la tio n fea tu re s. J a in a n d H ealey [25] in tro d u c e d a m e th o d b a se d on u n ich ro m e fe a tu re s c o m p u te d from th e th re e s p e c tra l b a n d s in d e p e n d e n tly a n d o p p o n e n t fe a tu re s t h a t u tilised th e s p a tia l c o rre la tio n b etw e en s p e c tra l b a n d s u sin g G a b o r filters. T h e y c o n c lu d e d t h a t th e o p p o n e n t co lou r fea tu re s sig n ifican tly im pro ved th e classificatio n a c c u ra c y over sim p ly u sin g u n ich ro m e featu res.

C o m b i n a t i o n o f c o l o u r a n d t e x t u r e i n f o r m a t i o n T h is a p p ro a c h div id es th e colour sig n al in to lu m in an c e a n d c h ro m in an c e c o m p o n e n ts, th e in fo rm a tio n from b o th in te n s ity a n d colour p lan es w ere e x tr a c te d se p a ra te ly a n d m erg ed to g e th e r for colour te x tu r e processing. G re y scale alg o rith m s a re ap p lie d to th e in te n s ity p lan e a n d th e colour in fo rm a tio n is u sed as a d d itio n a l in fo rm a tio n [26]. D rim b a re a n a n d W h e la n [14] e x a m in e d th e c o n trib u tio n of colour in fo rm a tio n to th e overall classificatio n p e rfo rm a n c e b y using th is ap p ro ach .

T h is research w ork focusses on th e la s t a p p ro a c h w h ere colour an d te x tu r e in fo r m a tio n a re p ro cessed se p e ra te ly a n d th e d is trib u tio n o f b o th th e lu m in an c e a n d th e c h ro m in an ce fe a tu re s a re used for colour te x tu r e se g m e n tatio n .

(37)

1.6 O r g a n isa tio n o f t h e T h e s is

T h e th e sis s tr u c tu r e is organised as follows:

In c h a p te r 2, th e re la te d w ork from th e p a s t lite r a tu r e a re review ed. A n e x te n sive in tro d u c tio n to th e re la te d rese a rc h w ork is p re se n te d . T h e rese a rc h w ork on te x tu re an aly sis, colour an aly sis, colour te x tu r e s e g m e n ta tio n a n d classificatio n a n d colour te x tu r e im ag e retriev al m e th o d s a re also review ed.

In c h a p te r 3, a d e ta ile d e x p la n a tio n is p ro v id e d o n v ario u s tec h n iq u es u se d in th is research w ork. T h e fe a tu re e x tra c tio n tec h n iq u es, th e fe a tu re p rese rv in g a d a p tiv e sm o o th in g , th e u n su p e rv ised /c-means c lu ste rin g , th e n o n -p a ra m e tric te s ts used for t h e colour te x tu r e d e scrip tio n a n d th e u n su p e rv ise d te x tu r e s e g m e n ta tio n m e th o d a re ex p lain ed . A b o u n d a ry refin em e n t a lg o rith m w hich e n h a n ce s th e se g m e n ted re s u lt is also d escrib ed. T h e tec h n iq u es a re com b ined in to th e pro cess a p p lie d to im ages to o b ta in th e final se g m e n ted resu lt.

In c h a p te r 4, a d e ta ile d d e sc rip tio n o n th e m eth o d o lo g y used in th is s tu d y is p resen ted . T h is c h a p te r ex p lain s th e th re e m e th o d s d ev elo p ed a n d a d o p te d for colour te x tu r e se g m e n ta tio n a n d d e ta ils t h e differen t colo u r te x tu r e d e sc rip tio n for th e th re e m e th o d s. F lo w c h a rts illu s tr a te th e ste p s followed in th e colour te x tu r e m od el for seg m en tatio n .

In c h a p te r 5, differen t ex p e rim e n ts t h a t a re p e rfo rm e d to observe th e effective ness a n d th e feasib ility of th e a p p ro a c h is ex p lain ed . T h e p e rfo rm a n c e of th e sy stem is e v a lu a te d u sin g R eceiver O p e ra tin g C h a ra c te ris tic (R O C ) cu rve a n a l ysis. In a d d itio n , th e e x p la n a tio n a b o u t th e V isio n T e x tu re (V isT ex) [27] im age d a ta b a s e , c o n s tru c tio n of different colour te x tu r e im ages a re also discussed. T h e re su lts of th e d e ta ile d e x p e rim e n ts from m osaic a n d n a tu r a l im ages a re p rese n ted . Finally, t h e sy ste m im p le m e n ta tio n of th e a lg o rith m is explained.

(38)

In chapter 6, th e application of the developed colour tex tu re segmentation m ethod ology in ISOS script images, skin cancer images and SPI images are presented. In chapter 7, concluding remarks on this research work is presented.

Finally, in chapter 8, the contribution of the thesis and th e future extensions of this work are suggested.

1 .7 S u m m a r y

Texture and colour are the two innate characteristics and key factors in image analysis. Various m ethods such as statistical m ethods, signal processing m ethods etc., and techniques such as co-occurrence, grey level differences, signed differ ences, M RF etc., were developed in texture analysis over the past few decades. Colour images provide more information th an grey scale images. Colour analysis finds an increasing attention in the recent years. The recent techniques developed for image analysis imposed the use of colour. In com puter vision, there are only a few m ethods th a t include colour information in texture analysis. Hence, this research focusses on the development of a model for colour texture segmentation. The objective of this research is to evaluate the colour texture segm entation and to determine the role of colour in colour texture analysis.

(39)

C hapter 2

R ev iew o f R ela ted W ork

2 .1 I n t r o d u c t i o n

This chapter presents a literature review on texture, colour and colour texture analysis. The study is related to the range of research work published on texture classification, texture segmentation, colour segmentation, colour texture classi fication, colour texture segmentation, colour texture image retrieval and some general approaches on texture and colour textures. The survey was based on a representative sample of literature th a t is available in th e above mentioned areas and the advantages of the approaches were also explored.

2 .2 R e v i e w o f T e x t u r e S t u d i e s

2.2.1 T ex tu re S e g m en ta tio n

Unser and Eden [28] described an approach for unsupervised segmentation of texture images. Local linear transforms were employed to extract the local tex ture properties. Texture energy measures were estim ated at the output of a filter bank by means of non-linear transform ation followed by an iterative Gaussian smoothing algorithm. This procedure generated a multi-resolution sequence of

(40)

C H A P T E R 2. R E V IE W OF R E L A T E D W O R K

feature planes w ith a half octave scale progression. A new feature reduction technique was applied. The feature reduction technique is an improvement of Karhunen-Loeve transform. The authors concluded th a t the m ethod provided efficient texture segmentation. The advantage of the m ethod is th a t it required no prior knowledge about th e textures present in the image.

Jain and Farrokhnia [29] presented a multi-channel filtering based texture seg m entation technique th a t used a bank of G abor filters to characterise the chan nels. Texture features were obtained by subjecting each filtered image to a non linear transform ation and computing the energy in a window around each pixel. A square error clustering algorithm was used to integrate the feature images and to obtain segmentation.

Dunn and Higgins [30] presented an algorithm for designing optimal Gabor filters. The procedure used a decision on theoretical framework, based on modelling a Gabor filter output as a Rician distribution, for designing optim al filters. A mul tiple filter segmentation scheme was also proposed in order to gain more robust results. The authors determined the efficiency of th e m ethod experimentally. They claimed th a t th e m ethod performed b etter and provided useful G abor fil ters for a wide range of texture pairs.

Puzicha et al. [31] proposed and examined non-param etric statistical tests to measure texture similarity. The statistical tests were applied to the coefficients of images filtered by a multi-scale G abor filter bank. The authors found th a t the similarity measures were useful for bo th texture based image retrieval and unsupervised texture segmentation. Experiments were conducted on B rodatz micro textures and a collection of real world images.

Hofmann et al. [32] presented novel approaches to segment textured images. This was followed in four steps: First, a scale space approach for d a ta representation

(41)

based on Gabor filters has been suggested. Secondly, a non-param etric sta tisti cal test was followed for texture comparison. Thirdly, an unsupervised tex tu re segmentation was formulated as a pair wise d ata clustering problem based on dissimilarities between texture blocks with a sparse neighbourhood structure. Finally, they developed a general m athem atical framework to apply the opti misation principle of determ inistic annealing to arbitrary partitioning problems. Segmentation algorithm was tested and validated on B rodatz textures and real world images. They concluded th a t their approach constitutes a tru ly unsuper vised m ethod for texture segmentation.

Randen and Husoy [33] designed filters for texture feature extraction. They de veloped and evaluated several approaches for the design of linear finite impulse response filters w ith optimised energy separation. A model for th e feature m ean and the variance was developed and the model was used for filter optim isation. This approach was compared with alternative filter optim isation approaches. The approaches were assessed by supervised segmentation experiments.

Ojala and Pietikainen [21] presented an unsupervised tex tu re segmentation m ethod based on the comparison of feature distributions for measuring the homogeneity of texture image regions and to localise boundaries between regions. Texture in formation was extracted using LBP and contrast. A region-based algorithm was developed for coarse image segmentation and a pixelwise classification scheme for improving th e localisation of region boundaries. The advantage of this m ethod is th a t it does not require any prior knowledge about th e number of textures or regions in the image and this m ethod can be easily generalised to utilise other texture features, multi-scale information, colour features and combinations of multiple features.

(42)

texture segmentation. A novel MRM RF param eter estim ation m ethod based on Markov Chain Monte Carlo approach was presented. Experim ents were per formed on different mosaics of natural textures and th e authors found th a t the m ethod was suitable to segment textured images. The disadvantage of this m ethod was th e com putational complexity.

S u m m ary

This section presents a survey on texture segmentation. The review shows th a t techniques such as Local linear transform , G abor filter, M RMRF, LBP and Linear finite impulse response filter are used to extract texture features. Non- param etric tests are used to discriminate th e various texture regions. Different supervised and unsupervised m ethods are followed for texture segmentation. The performance evaluation of the m ethods is based on th e application of th e devel oped algorithm to B rodatz images and a large num ber of natural and artificial texture images. Most of th e unsupervised texture segmentation m ethods require no prior knowledge about the texture present in the image. The D CT filter based approach was selected from this survey and the LBP based unsupervised texture segmentation forms the basis for the research work.