Adaptive Structuring Element in Fuzzy
Morphology for Automatic Extraction of
Urban Road Network from High Resolution
Aerial Images
Dr. S. LETITIA† Associate Professor,
Dept. of Electronics & Communication Engineering Thanthai Periyar Govt. Institute of Technology, Vellore
NISHAT KANVEL
Associate Professor,
Dept. of Electronics & Communication Engineering Thanthai Periyar Govt. Institute of Technology, Vellore
Dr. ELWIN CHANDRA MONIE
Principal,
R.M.K. Engineering College, Chennai
This paper presents a novel algorithm for automatic extraction of road network using fuzzy morphological operations with adaptive structuring elements. The shape, size and gray-scale values of the structuring elements are dynamically chosen, based on the geometric and radiometric properties of the objects to be retained. As the image and structuring elements are considered as fuzzy sets, fast implementation of mathematical morphology is possible. Though this proposed work is having some limitations, it gives better results in extracting the roads from high resolution satellite images with higher Quality factor.
Key words: Fuzzy Sets, Automatic Extraction, Adaptive Structuring Elements, Fuzzification, Defuzzification
1. Introduction
The automatic extraction of road network from high resolution images has drawn considerable interest in the past few years. In one such method, extraction is done by first classifying roads in an image and extracting the roads after edge detection (Tupin et al. 1997). However the extraction becomes very difficult in an urban environment where the roads are surrounded by buildings and other objects. Depending on the prior knowledge of the environment, appropriate Directional Filters are also used (Gamba et al. 2006). In this method, Fuzzy Hough Transform identifies the direction in a finite road segment and the extraction of the road is completed by Directional filtering. This is not appropriate for a complex road network which changes its direction frequently. The roads are plotted in the form of straight lines in the methods proposed by Gamba et al. 2006 and Shackelford et al.2003. But, errors and discontinuities are introduced when the roads have frequent bends.
computed to determine the best size and direction of the morphological filter to be used. As these methods are applicable to binary images only, it needs to be extended to grey scale images. In Fuzzy Mathematical Morphology (FMM), the image and structuring element are generally of gray scale type. It is interesting to observe that gray scale images can be represented as fuzzy sets. Hence, fuzzy concepts can be adapted to Traditional Mathematical Morphology which leads to Fuzzy Mathematical Morphology (FMM) operations. FMM is considered to be a good tool to segment tree structured medical images (Bouchet et al. 2007) whose lines are fuzzy and their contours are not sharp. It has been indicated that their work on eye fundus images can be extended to road network since it is also a tree structured image having similar properties of eye fundus image (Zana and Klein 2002). In our work, Fuzzy Mathematical Morphology is taken as the basic concept to extract roads.
Another aspect to be considered in applying Mathematical Morphological operations is the selection of the shape and the size of the structuring element. If the structuring element is dynamically chosen depending on the local features in the input image, it is called an Adaptive Structuring Element (Debayle and Pinoli 2005). Roerdink and Heijmanns 1988, consider segmentation of branches of the trees from the photographs of trees in a forest using adaptive morphology. In the field of Euclidean morphology, the size of the SE increases with the distance to the centre of the image. Verly and Delanoy (1993), has applied mathematical morphology to range imagery, a modality where the value of each pixel is equal to the distance to the imaging device. Thus in order to take perspective into account, the SE size should be adapted locally to the image content. The extraction of features from an ambiguous image is very effective when this Adaptive Structuring Element is used. In an image of an urban area, the denser the area is inhabited and the more intensively it is used, the denser the road network is (Baumgartner 1999). Hence, the extraction of road network from high resolution images of urban area is very challenging. In this paper, we propose a novel method for automatic extraction of road network from AIRSAR (C-band) and Ikonos panchromatic images using FMM with Adaptive Structuring Element. In the method proposed, the image and the SE are converted into fuzzy sets and fuzzy operations such as fuzzy dilation, fuzzy erosion and fuzzy opening are performed to extract the road network. The last step in this work is Adaptive Alternating Sequential filtering (Debayle and Pinoli 2005), which involves sequential and alternative opening and closing operations with Adaptive Structuring Element, utilized to extract the roads, from ‘No Road’ elements such as tall buildings, trees, shadows, etc., which are situated very close to the road network. The remarkable feature of this algorithm is that it removes the ‘No road’ elements without deviation in the original direction of the roads, in spite of frequent bends and curvilinear nature of the roads in the test image. The method proposed in this paper is innovative because the algorithm used for extraction can be applied to different kinds of imagery, for both active and passive remote sensing which is quite uncommon.
2. Proposed work
2.1 Adaptive Structuring Elements
It is observed from the statistical analysis that the Adaptive SE outperforms the morphological filters in the space-invariant Structuring Element. Chen et al. 1996 considered the SEs of variable size to filter a one-dimensional signal. But Adaptive SE based on Adaptive Neighborhood set is a very effective as morphological filter (Cheng and Venesanopoulous 1992). If
V
mh(
x
)
xD is the Adaptive Neighborhood set, then the Adaptive SE
x Dh m
x
R
(
)
can be expressed as:
m
h
f
C
D
)
,
,
(
D z
h m h
m h
m
x
V
z
x
V
z
R
(
)
(
)
)
(
(1)whereC is the general set basedon a local measurement such as luminance, contrast and thickness, related to the image f.2
D
is the domain of definition of the images
z is a point in the image after the transformation where
z
D
andf
I
I is the natural set of image mappings from D into
Now the adaptive SE
R
mh(
x
)
xD is applied to the local details of the image. The shape of usual disc SE for three homogenous tolerance m=1, 2, 3 is shown in Figure. 1(a).The Adaptive SE is anisotropic and satisfies symmetrical property, viz.
x
R
(
y
)
y
R
mh(
x
)
h
m
.It is alsoreflective such that
R
mh(
x
)
R
~
mh(
x
)
whereR
~
mh(
x
)
z
;
x
R
mh(
z
)
is the reflected Adaptive SE. This definition enables morphological duality necessary to build morphological filters (Soille and Pesaresi 2002).Figure.1(a) Non Adaptive and Adaptive Structuring Elements for three values m=1, 2, 3 pixels
In contrast to the Adaptive Disc Structuring Element discussed above in Figure. 1(a), the Adaptive Line Structuring Element is used in the proposed work shown in Figure. 1(b). If the length of the line structuring element is less than the road width in the image, then those roads will appear in the output image. When the structuring element and the road have parallel directions, the road will stay nearly unchanged. Considering these facts, if adaptive opening is applied to an image using line structuring element whose length varies from 11 to 15 pixels, every isolated round and bright zone whose width is less than 11 pixels will be removed. The length of the SE is varied between a set of values, and the output image is continuously monitored to see whether all the required road elements are extracted in it. The above concept is diagrammatically explained in Figure. 1(b).
Figure 1(b) Adaptive Line SEs for homogeneity tolerance parameter m=11 to 15 pixels
2.2 Adaptive Morphological Operators and Filters
Adaptive dilation and Adaptive Erosion are the basic dual operators which are defined as:
I
C
f
h
m
(
,
,
)
AdaptiveDilation
)
(
)
(
sup
:
)
(
x
R
w
w
f
x
D
f
D
h m h
Adaptive Erosion
)
(
)
(
inf
:
)
(
x
R
w
w
f
x
D
f
E
h m hm
(3)The Adaptive Dilation (2) and Erosion (3) are connected operators. The attribute of connectivity helps defining several connected operators, which are of great morphological importance. The complex adaptive morphological filters are derived based on the lattice theory and the connected operators. Accordingly, the Adaptive Closing and Opening are respectively defined as:
(
m
,
h
,
f
)
C
I
Adaptive Closing is performing the adaptive Erosion on the adaptive Dilated Image
Adaptive Closing
)
)(
(
)
(
:
)
(
x
f
D
f
E
x
D
f
C
h m h m hm
(4)Adaptive Opening is performing the adaptive Dilation on the adaptive Eroded Image
Adaptive Opening
)
)(
(
)
(
:
)
(
x
f
E
f
D
x
D
f
O
h m h m hm
(5)2.3 Fuzzy mathematical morphology
A gray value image is a fuzzy set in the sense that, it is a fuzzy version of a binary image, or that a gray value represents the degree to which the pixel belongs to the image foreground (Smith 2000, Pasha 2006, Deng and Heijmans 2002). The three basic operations used in Fuzzy Mathematical Morphology are Conjunction, Disjunction and Negation. If p and q are two predicates given, the three basic operations are defined as follows: Conjunction: p
q; Disjunction: p q;Negation: ~p. The truth values of the above operations are based on the values of p and q in {0, 1}. The implication pq is obtained from disjunction and negation combination, where p q = ~pq. In fuzzy logic, it is necessary to introduce a new scale of values [0, 1] in which values are assigned to the predicate depending on their truthfulness. The above operations can once again be defined as:
Conjunction: u (p q ) = min (u(p) ,u (q) ) Disjunction: u (p q ) = max (u(p) ,u (q) ) Negation : u (~p) = 1-u (p)
where p and q are fuzzy predicates and u(.) is their truth value. Indeed, these operations are an extension of those above. A function C: [0,1] X [0,1] [0,1] given by C (s ,t ) = st is called a fuzzy conjunction if it increases in both arguments and satisfies the Boolean Conjunction on {0,1}.
C(0,0)=C(0,1)=C(1,0)=0 and C (1,1) = 1 (6)
A function I: [0,1] X [0,1] [0,1] given by I (s,t ) = st is called a fuzzy implication if it decreases in the first argument, increases in the second and satisfies the following in Boolean implication:
s
a
s
a
I
t
t
a
t
t
a
C
)
1
(
)
,
(
a
-1
>
,
1
,
0
,
for all a, s, t
[
0
,
1
]
(8) The above formulae meet thedefinition of conjunction and implication. The fuzzy erosion of an image f by a Structuring Element B, in a point x is defined as:
f
y
y
f
y
B
I
x
B
f
F
inf
,
)
)(
,
(
(9)The fuzzy dilation of an image f by a structuring element B, in a point x is defined as:
f
y
y
f
y
B
C
x
B
f
F
sup
,
)
)(
,
(
(10)Following the steps of the morphological theory, fuzzy opening is described as:
F(
f
,
B
)
F(
F(
f
,
B
),
B
)
(11) Fuzzy Closing is defined as
F(
f
,
B
)
F(
F(
f
,
B
),
B
)
(12) On the basis of these equations, fuzzy Top- Hat by opening is defined asTop
Hat
yF(
f
)
f
F(
f
,
B
)
(13)2.4 Proposed algorithm
The key steps involved in the proposed algorithmare: (1) Fuzzification, (2) Fuzzy Dilation and Fuzzy Erosion (3) Fuzzy Top hat by opening and (4) Adaptive Sequential Filtering (5) Defuzzification. The sequence of operations is shown in Figure. 2a.
Step: 1 The pixel values of the gray scale image are converted into fuzzy values in a set [0, 1] using Sigmoid function,
t
1
arctan
t
2
1
Figure.2 (a) Fuzzy Morphology with Adaptive SE
1. Fuzzification 3.Fuzzy
Top Hat filter
Processed Image 5. Defuzzification
4. Adaptive Sequential filters
2. Fuzzy Dilation & Fuzzy Erosion Input
Step 2: Fuzzy Erosion and Fuzzy Dilation are performed using (9) and (10) respectively. The Structuring Element B in (9) and (10) are in fact the adaptive SE, defined in (1). The structuring element in FMM is considered as a fuzzy set and implemented using Kleene – Dienes formula shown in (8).
Step 3: The Fuzzy Top-hat by opening is obtained by subtracting the original image from the fuzzy dilation on the fuzzy eroded image defined in (13). The purpose of Fuzzy top-hat is to extract the locally brilliant elements from the image. The objective is to obtain the graph peak representing the object to be segmented (brilliant object). The strategy is to create a new image with the irrelevant information, that is to say, to eliminate the peak applying an opening to the original image. Then, by subtracting it from the original image, a new one is obtained built from the information of interest. This has been clearly explained in Figure. 2b.
Figure. 2(b) The original image appears in black. The image in red shows the irrelevant information; and the one in green, the image Top-Hat.
Step 4: Adaptive alternating sequential filters of order n is performed using the adaptive SEs with the criterion mapping f and homogeneity tolerance m is given by an Alternating Sequential Filter with Openings and Closings (ASFOC).
)
(
,
f
ASFOC
mhn :
)
)(
(
...
, 1,
n
OC
f
x
OC
f
I
I
h p m h
p
m
The top-hat image is filtered sequentially using the adaptive alternating sequential filters for more than 5 times to eliminate the background noise.
Step 5: Then the image is defuzzified using the inverse function of the sigmoid function to view the output image.
2.5 Implementation
3 Results and evaluation
The road images extracted by applying FMM algorithm using Adaptive Structural Element on two test images given in Figure.3 are shown in Figure. 4(c) and (d). The factors used in this work to measure the results of the Fuzzy Adaptive Method are Completeness, Correctness and Quality of the output image with respect to the ground truth which is manually extracted. The definition of Completeness (Wiedemann, C., 2003 and Hinz, S. et al.2000) of the extracted data is presented as the percentage of the reference data, i.e., the percentage of the reference network which could be extracted. Similarly, Correctness represents the percentage of correctly extracted road data, i.e. the percentage of the extraction, which is in accordance with the reference. Completeness and Correctness are the basic factors which assess the results of the proposed work in this paper. Quality can be directly calculated from Completeness (compl) and Correctness (corr)using the formula:
Quality =
The same factors are also measured for the same set of images through Adaptive Directional Filtering using Hough Transform proposed by Gamba et al. 2006 shown in in Figure. 4(a) and 4(b). This method is having clearly extracted output lines in the direction of the roads along with some erroneous lines in the places of frequently changing directions in the case of Ikonos Panchromatic image of Pavia, Italy. But the algorithm proposed in this paper is extracting the roads even in the places of frequently changing bends. But it has a drawback of more noise in the output. The reason is due to the shadows of the tall building, trees and foliages that fall on the road paths. Also this is the cause for decreasing the Completeness factor. Completeness is observed to be higher for Figure. 4(c) and marginally less for Figure.4(d). In the case of Correctness, Figure.4 (d) has a higher value and Figure. 4(c) is fair marginally poor. However in Quality, both Figure. 4(c) and Figure. 4(d) are superior. Quantitatively, as seen from Table I, the Correctness calculated for Figure. 4(d) is higher than for Figure. 4(b) by 24.76% and Quality by 15.96%. The Completeness is better by 8.57% for Figure. 4(c) than 4(a) and there is an improvement in the Quality by 7.24%. The inference is that the proposed method extracts most of the roads in a regularly laid out roads as in the test image in Figure. 3 (a). This method also extracts roads more accurately for images such as Figure. 3(b), where the direction of the road changes frequently. This shows that the proposed method is extracting the roads correctly, even when the road directions are changing frequently in the test image Figure. 3(b). Another important outcome of the proposed work is that, even if the roads are curved and circular as in Figure. 5(a),
Figure.3 Data sets. (a) AIRSAR image of Santa Monica, CA, (b) Ikonos panchromatic image of Pavia, Italy. (c) Road network in (a), (d) Road network in (b).
Figure.3(a) Figure.3 (b) corr
+ compl.corr
-compl
Figure. 3(c) Figure.3 (d)
Table 1 Quantitative evaluation and comparison of the results
Figure.4(a) Figure.4(b) Figure.4(c) Figure.4 (d)
Completeness 63.1% 81.8% 71.67% 79.31%
Correctness 95.2% 64.6% 93.62% 89.36%
Quality 61.1% 56.5% 68.34% 72.46%
which is the Ikonos image of part of Denver city, it has correctly extracted without losing any of the road segments, as seen in Figure. 5(b). The Adaptive Structuring Element has eliminated the problem of over segmentation and is capable of extracting the roads of varying width, provided that the variation of the size of the adaptive structuring element is suitably selected to operate over the image. This is not possible with the methods using directional filters as the one described in Gamba et al 2006. ‘No Road’ areas are the ones that are consequences of different relations between roads and buildings. In down town areas, buildings typically are closer and more parallel to roads. The presence of noise in the extracted road network closer to it due to ‘No road’ elements is the major drawback of the proposed
Figure 4. Results. Figure. 4(a) and (b) Road network extracted from Figure. 3(a) and (b) using Adaptive Directional Filtering and road regularization. Figure. 4(c) and (d) Road network extracted from Figure. 3(a) and (b) using Fuzzy Adaptive Structuring Element
Figure.4 (c) Figure.4 (d)
method. ‘Normalized Mean Square Error’ (NMSE) is used in our work to measure the ‘No Road’ elements in the output images. NMSE is an estimator of the overall deviations between predicted and measured values. In our work, the NMSE obtained for Santa Monica, CA image is 0.267 and that for Pavia, Italy image is 0.068 while the ideal value is 0. Since the NMSE value is very near to zero, the extraction is almost perfect. Another advantage of the proposed method is the simplicity of the arithmetic operations of the fuzzy morphology compared to traditional morphological one for gray scale images. Since the fuzzy arithmetic is simple, the implementation using fuzzy morphological operations is faster compared to traditional morphological operations, for applications over the changing width of the roads.
Figure.5 Ikonos Image of Part of Denver city. (a) Sample image (b) Extracted image through Fuzzy Adaptive Structuring Element
Figure.5 (a) Figure.5 (b)
4 Conclusion
References
[1] F. Tupin, H. Maitre and J. M. Nicolas, “Detection of Linear Features in SAR images: Application to Road Network Extraction”, IEEE Trans. Geosci. Remote Sens., vol. 36, no.2, pp. 434-453, Mar. 1998.
[2] P.Gamba, F.Dell’Acqua and G. Lisini, “Improving Urban Road Extraction in High-Resolution Images Exploiting Directional Filtering, Perceptual Grouping and Simple Topological Concepts”, IEEE Geosci. Remote Sens. Letters, vol.3, no.3, pp. 387-391, July 2006.
[3] J.Serra, “Image Analysis and Mathematical Morphology”, vol.2: Theoretical Advances, Academic Press, New York, 1988.
[4] P. Soille and M. Pesaresi, “Advances in Mathematical Morphology Applied to Geoscience and Remote Sensing”, IEEE Trans. Geos. and Remote Sens., vol. 40, no. 9, pp. 2042 – 2055, Sept. 2002.
[5] R.Bellens, S.Gautama , L. Martinez-Fonte, W.Philips, J.C.-W. Chan and F. Canters, “Improved Classification of VHR Images of Urban Areas Using Directional Morphological Profiles”, IEEE Trans. Geosci. Remote Sens., vol. 46, no. 10, 2803-2813, Oct. 2008.
[6] A. Bouchet, J. Pastore and V.Ballarin, “Segmentation of Medical Images using Fuzzy Mathematical Morphology”, International Journal JCS & T, vol.7, no.3, pp. 256 – 262, Oct. 2007.
[7] F. Cheng and A.N. Venesanopoulous, “Adaptive Morphological filter for image processing”, IEEE Transactions on Image Processing, vol.1, no.4, pp. 533 -539, Oct. 1992.
[8] C.S. Chen, J. L. Wu and Y.P. Hung, “Statistical analysis of space–varying morphological openings with flat structuring elements”, IEEE Trans. Signal Processing, vol.44, no.4, pp.1010-1014, Apr. 1996.
[9] J. Roderdink and H. Heijmans, “Mathematical Morphology for structures without translation symmetry”, Signal Processing 15 (3), pp. 271-277, 1988.
[10] J.G. Verly and R.L. Delanoy, “Adaptive Mathematical Morphology for range imagery”, IEEE Trans. Image Processing, vol.2, no. 2, pp. 272 – 275, Apr. 1993.
[11] F. Smith, “Selection of Fuzzification, inference and Defuzzification Techniques for Fuzzy Logic Control”, Technical report on NUIG –IT – 230800 – August, 2000.
[12] A. Pasha, “Morphological Image Processing with Fuzzy Logic”, Journal of Aeronautics and Space Technologies, vol .2, no. 3 pp. 27 -34, Jan 2006.
[13] T.Q. Deng and H. Heijmans, “Grey–scale Morphology Based on Fuzzy Logic”, Journal of Mathematics Imaging and Vision, vol.16, pp. 155 -171, 2002.
[14] C. Wiedemann, “External Evaluation of Road Networks”, ISPRS Archives, Vol. XXXIV, Part 3/W8, Munich, 17 -19, Sept. 2003.
[15] S.Hinz, C. Wiedemann and A. Baumgartner, “A Scheme for Road Extraction in Rural Areas and its Evaluation”,5th IEEE Workshop on Applications of Computer Vision, pp. 134-139, 2000.
[16] J. Debayle and J.C. Pinoli , “ Multi scale Image Filtering and Segmentation by means of Adaptive Neighborhood mathematical Morphology,” IEEE International Conference on Image Processing ICIP , vol. 3, pp. 537-540, Sept. 2005
[17] F. Zana and J.C. Klein, “Segmentation of Vessel-Like Patterns Using Mathematical Morphology and Curvature Evaluation”, IEEE Trans. Image Processing, vol. 10, no.7, pp.1010-1019, July 2001.
[18] A.K. Shackelford and C.H. Davis, “Fully automated road network extraction from high-resolution satellite multispectral imagery”, IEEE International Geoscience and Remote Sensing Symposium, IGARSS '03. Proceedings. 2003, vol.1, pp. 461- 463, 2003.
