Template Matching method

The general approach is to classify the objects of interest in several steps i.e. first the algorithm does a rough classification of points on the objects of interest. Then it improves the classification results in next steps by excluding the wrongly classified points and including the desirable points which were not classified in previous steps. Figure 4-1 indicates the flowchart of TM method.

4.4.1. First classification of rails 4.4.1.1. Pre classification

First a rough classification is done based on the height of points. This leads to the classification of points as 4 classes:

1. Class 1: contains all points on the track bed and rails.

It is assumed that the most common elevation in the dataset represents the elevation of track bed.

This assumption was made due to higher point density of track bed in comparison to other objects. Besides, due to slope of the terrain, the elevation of track bed may change. So to include all points on rails and track bed in class 1, all points with elevation 0.5 m more or less than the most common elevation in the dataset are classified as class1.

Maximum Track bed height= the most common elevation + 0.5 meter Minimum Track bed height= the most common elevation - 0.5 meter

2. Class 2: contains all points with the maximum elevation 5 meters above track bed. Class 2 does not include any points on the objects of interest. So all points with a maximum elevation of 5 meters above maximum track bed height are classified as class 2.

3. Class 3: contains all points with the elevation 5-5.5 meters above track bed including points on contact wires. So all points with an elevation between 5 to 5.5 meters above maximum track bed height are classified as class 3.

4. Class 4: contains all points with the elevation more than 5.5 meters above track bed including the points on catenary wires. So all points with an elevation more than 5.5 meters above maximum track bed height are classified as class 4.

As a result, rail points should be detected out of class 1, contact wire points out of class 3 and catenary wire points out of class 4 and all points in class 2 are undesirable for us.

Relative heights of contact and catenary wires to rails are constant (5-5.5 m and more than 5.5 m above rails respectively) due to railway environment specification of The Netherlands.

4.4.1.2. Grid generation

As mentioned in section 4.4.1.1, class 1 contains all rail points so to classify rail points only points of class 1 are processed. The main idea is to inspect the data locally and this grid facilitates the inspection of data in details and locally.

So to implement, a 2D grid with cell size of 50 by 50 cm² was generated. In this step, cell size is very influential on results and is highly dependent on data resolution, point density. A smaller cell size would result in a cell with small number of points which decrease the reliability of results and a bigger cell size can lead to cells containing two rails or rails with other objects which makes the analysis more

complicated. In fact, it is preferred that each cell contains maximum of one rail as it makes the process more efficient. Direction is a characteristic of rails that can be used if there is just one piece of rail in a cell.

As the grid is 2D, the location of each cell is indicated by two elements i.e. row and column. So to identify a cell in which a point falls, two elements of row and column should be determined by following steps:

1. Finding minimum of Xs and Ys of all points in class 1: MinX and MinY.

2. Determining that each point lies in which cell in a way described below:

a. For a point with coordinates: P(X,Y,Z).

b. For a grid cell size of 0.5 m.

c. Row = ((X-MinX)/0.5)+1 d. Column = ((Y-MinY)/0.5)+1

e. Row and Column indicate the location of the cell in which the point P falls in.

3. Continue the above steps for all points of class 1.

In this way, based on planimetric position of points, their X and Y coordinates, it was determined each point falls in which cell. Then, further analysis was done on all points which lie in one cell.

The idea is to have enough number of points in each cell so that they can represent the characteristics of objects in that cell which leads to reliable statistical analysis results. By an inspection of cells containing different number of points, it was found that the ones having less than 30 points do not represent true information about the characteristics of objects inside. Therefore, just cells with more than 30 points inside were considered for further analysis.

4.4.1.3. Height jump detection

The height jump detection (HJD) algorithm uses the fact that the height difference between the points on top of the rail and the points on the track bed is about 10 cm which is the rail height. The main idea is that the rails cause a height jump on the track bed due to their heights. In Figure 4-2, grey arrows show rails which cause height jumps and the white arrow shows another part of track bed without any height jumps.

Figure 4-2. Grey arrows point to height jumps caused by rails while the white arrow points to other parts of track bed without any height jumps

The HJD method was applied to detect rail points and as only class 1 contains points on rails, only points of class 1 were processed for this stage and the strategy was:

1. To determine if a cell contains rail points or not by inspecting:

a. Whether there is a height jump in a cell or not.

b. If the size of the height jump is the same as the one caused by rails.

2. To differentiate the points on rail tracks from the points on track bed in a cell containing rails.

To determine if there is a height jump in a cell, the height difference between the highest and lowest points in a cell is inspected. If there is a rail in a cell, the height difference between the highest and lowest points should be at least 10 cm which is the rail height. But as the data contains noise, usually the lowest and highest points in each cell do not represent the rail and track bed respectively. This is evident in Figure 4-3 in which rail points are indicated by grey colour and track bed points by white colour and 10, 80 and 90 percentile elevations represent the elevations of track bed, bottom of rail and top of rail respectively.

Moreover, the size of the height jump needs to be checked. Besides rails, there are some other objects that cause height jumps like signs, electrical devices, bumps on track bed and the short walls where the railway stations and track bed intersect (Figure 4-4). So to differentiate between these objects from rails, the algorithm inspects the size of height jump. In other words, the size of height jump determines if it is a piece of rail or not. For instance, the short walls of intersection of track bed and railway stations and some signs make a bigger height jump while electrical devices and bumps on track bed make a smaller height jumps than that of caused by rails.

Figure 4-3. Using 10, 80 and 90 percentiles to remove noises for implementation of height jump detection method

Therefore, with respect to the rail height, the algorithm considers only the height jumps in the range of 10- 25 cm as rails and ignores the rest. In Figure 4-4, two arrows on left indicate the height jumps caused by rails and the one on the right point to height jump caused by railway station.

As a result, the constraint is defined as:

If a cell contains a piece of rail, the height difference between the 10 and 90 percentile elevations of its points is more than 10 cm and less than 25 cm:

10 (centimetres) < (90%Elevation) – (10% Elevation) < 25 (centimetres)

Then for the cells which are considered to have rail points, the points on rails needs to be differentiated from the ones on track bed. To do this, the points higher than 80 percentile elevation of all points in that cell were considered as points on rails (Figure 4-3).

One should pay attention that this is the first classification of rail points and some wrong classifications and some classification gaps are expected which will be taken care of in the next steps.

4.4.2. First classification of wires

To classify the points on wires, three characteristics of wires were considered:

1. Contact wires are in elevation of 5-5.5 m above rails (Figure 4-5).

2. Catenary wires are in elevation of more than 5.5 m above rails (Figure 4-5).

Figure 4-4. Big height jump caused by railway station platform V.S. small height jumps caused by rails

3. In top view, as if all rail and wire points are projected on a planimetric plane, both contact and catenary wire points lie between rail points (Figure 4-6).

To this stage, some rail points are classified in previous step and all wire points are in class 3 and 4 of pre classification based on height. So if all classified rail points and all points in class 3 and 4 are projected on a planimetric plane, out of all points in class 3 and 4, only points on wires lie in a close neighbourhood of classified rail points.

To implement, all projected classified rail points were put in one KD tree data structure and all projected points of class 3 and 4 were put in another KD tree data structure. Then by doing the nearest neighbour analysis, only points of class 3 and 4 which were in a close neighbourhood of rail points were considered as contact and catenary wire points respectively.

4.4.3. Excluding wrongly classified rail points

It should be noticed that the first classification of wire points is dependent on the results of first classification of rail points. That is so because the first classification of wire points is done based on nearest neighbour analysis between points of class 3 and 4 and the classified rail points in previous step.

As in the results of first classification of rail points, some wrong classifications and some classification gaps are expected, the results of first classification of wires are expected to have some wrong

classifications and some classification gaps as well.

Figure 4-5. Relative height of wires to rails

Figure 4-6. In top view, wires shown by yellow and orange colour lie between rails shown by green colour

The idea of TM was to seek the rail pattern in the classified points of previous steps to detect only truly rail points and exclude the wrongly classified ones. This was done by considering the spatial relation of wires to rails.

In fact, there are three points that should be noticed:

1. The purpose of this step is to exclude wrong classification of just rail points and not wire points.

2. TM was aimed to exclude wrong classifications and not aimed to include any more rail or wire points which were not classified in previous steps. Classification of the rest of rail and wire points which were not classified in previous steps is done in next steps described in sections 4.4.4 and 4.4.5.

3. The input of this step is the classified rail and wire points which is obtained in previous steps and the output is only true rail points.

So the point cloud is converted to a greyscale image and a kernel based on the defined rail pattern is generated. The rail pattern is defined based on the characteristics of rails and contact wires, mentioned in section 4.2, and contains:

1. Two straight rails:

a. 1 meter long

b. Distance between the axis of rail tracks:1.4 m c. Width of rail tracks: 20 cm

2. One contact wire:

a. In a certain elevation above rail tracks

b. In top view, its planimetric position lies between rail tracks c. Width of contact wire: 30 cm

3. A black area in surroundings of rails

Then to find rail patterns, the kernel is moved through the greyscale image with the following strategy:

1. Move the kernel through the first 5 meters of rails to get the main direction of rails.

2. Move the kernel through whole greyscale image every 5 rows and in each row every 3 cells to find some of rail patterns as a starting point for the next step.

3. Continue searching for rail pattern from the detected rail patterns in previous step.

a. Continue the search for rail pattern just in the neighbourhood of its two ends and follow them to find the rest of rail patterns.

b. Update the rails direction every 1 meter after each detection of rail pattern.

The strategy explained above is an optimized one as first it commences the rail pattern seeking with 5 rows and 3 columns steps. Then, it continues the search in the neighbourhood of detected patterns and follows them to detect the rest of rails. This algorithm is more efficient and faster than searching for rail pattern in each cell of greyscale image one by one.

The result up to this section would be a greyscale image in which most of the wrongly classified rail points are excluded. However, some wrongly recognized rail patterns are still expected which are mostly scattered

component analysis is applied and small disconnected components are eliminated. As a result, only big components representing true rails are remained.

To implement, the kernel was generated as a greyscale image containing a pair of white colour lines of 1 meter long as rail tracks and one grey colour line of 1 meter long as contact wire between rail tracks.

Figure 4-7 shows the rail pattern in a railway environment and Figure 4-8 indicates the rail pattern as a greyscale image.

In implementation, to convert the point cloud to a greyscale image, the steps below were done:

1. A greyscale image was generated and the grey value of all its cells was set to zero.

2. It was determined each rail and wire point falls in which cell.

3. The grey value of each cell with at least one rail point inside was set to 255 (shown by white colour)

4. The grey value of each cell with at least one contact wire point inside was set to 128 (shown by grey colour)

The resolution, cell size, of greyscale image and kernel was chosen according to point cloud resolution. In addition, step 2 above was done based on planimetric position of points i.e. all rail and wire points which were classified in previous steps were projected on a planimetric plane. Next, as the location of each cell in

Figure 4-7. Rail and wire pattern in a railway environment

Figure 4-8. Rail and wire pattern as a greyscale image

greyscale image is identified by two elements of row and column, to determine the cell in which a point P(X,Y,Z) falls, following steps were pursued:

1. The minimum of Xs and Ys of all points were found: MinX and MinY 2. For a grid cell size of 0.05 m

3. Row = ((X-MinX)/0.05)+1 4. Column = ((Y-MinY)/0.05)+1

5. Continue the above steps for all points

For each cell the kernel was rotated by different angles from -90 to +90 degrees and checked if the pattern of greyscale image in that cell matches with the rail pattern represented by the kernel. For each cell as the kernel rotated, the correlation between the image and kernel was calculated for each angle. Out of all correlations calculated for each cell, only the maximum correlation was considered and checked whether it exceeded a threshold or not. If the maximum correlation between the kernel and a part of the greyscale image was more than the threshold, it was concluded that part of the image contained rail pattern. In other words, the high correlation between the greyscale image and the kernel means there are some rail points in that part of the greyscale image. Then the rail points in that part are recognized according to the kernel pattern with respect to the angle by which the correlation exceeded the threshold.

To calculate the correlation threshold, some sampling on correlation between the kernel and different parts of greyscale image was done. Based on achieved results, the correlation between the kernel and parts containing rail patterns exceeded 0.3 in a 0-1 scale in which 0 represents no match and 1 represents complete match between kernel and a cell in greyscale image.

Finally, connected component analysis was applied to remove wrongly recognized rail patterns. In fact, it eliminated components with maximum size of a rail pattern of 1 meter long.

4.4.4. Classifying the rest of rail points

The result of previous step is a greyscale image containing some white and some black cells. White cells represent the rails and the rest of cells are black. To convert back the greyscale image to point clouds, like its first conversion in section 4.4.3, it is determined that each point falls in which cell. If a point fell in a white cell then it is labelled as a rail point and if it fell in a black cell, it is not labelled.

Therefore, the result is point clouds including some areas where no rail points are classified. It is assumed that not-classified pieces of rails are small enough to be considered as straight so to fill this classification gap, Hough Transform algorithm is applied. By applying Hough Transform algorithm, the parameters of corresponding lines of rails are calculated and by following these lines the gap areas are filled. The points in these gaps are labelled differently from true rail points and as potential rail points. To verify the potential rail points, it should be checked if in an elevation more than 5 m above them, there are some wire points or not. If so, they would be classified as rails.

To implement, it should be noticed that the main classification gap is an intersection between railway and a road as it is a cross levelling and rails in that part do not cause a height jump. The length of this gap which is the width of road intersecting railway was about 20 meters. For this length, rails were assumed to be straight. To use the Hough Transform algorithm some parameters needed to be set and they were set based on railway environment specifications mentioned in section 4.2.

The Hough Transform algorithm parameters were:

1. Maximum number of lines to be detected

a. i.e. maximum number of rails exist next to each other, set to 20 rails

b. Can be increased or decreased for other railway environments with more or less number of rails

2. Neighbourhood suppression

a. Which make the algorithm to detect maximum one rail in defined neighbourhood and suppress the rest of rails in that neighbourhood

b. Set based on gauge, meaning that in a neighbourhood of smaller than gauge maximum one rail exists

3. Maximum length of gap areas to be filled by algorithm a. Maximum width of roads intersecting with railway b. Was set to 50 meters

After converting the greyscale image back to 3D point clouds, it was checked whether in an elevation more than 5 m above potential rail points, there were some wires or not. To do so, again the nearest neighbour analysis was done between projected points of class 3 and 4 and all potential rail points. Only

After converting the greyscale image back to 3D point clouds, it was checked whether in an elevation more than 5 m above potential rail points, there were some wires or not. To do so, again the nearest neighbour analysis was done between projected points of class 3 and 4 and all potential rail points. Only