Target Detection and Classification Based on LiDAR

28

© Global Society of Scientific Research and Researchers http://asrjetsjournal.org/

Target Detection and Classification Based on LiDAR

TANG Chun-ming

a

, ZHANG Xiao-yu

b*

,

YU Xiang

c

, ZHU Wen-yan

d

a,b,c,d_{School of Electronics and Information Engineering, Tianjin Polytechnic University, Tianjin 300387, China} a_{Email: [email protected]}

b_{Email: [email protected]} c_{Email: [email protected]} d_{Email: [email protected]}

Abstract

To solve the problem of difficult classification of air baggage, we use LMS511 LiDAR to collect the distance data from the baggage surface to the light-center of LiDAR, propose a new detection and classification algorithm, and the baggage detection and classification system is designed, thus, the self-service of air baggage check is realized. Firstly, an object-based classification method is proposed by considering the characteristics of target. The geometry, texture, corner features and shape descriptors of the baggage are extracted to construct the feature vectors, and the feature vectors are imported into the RDforest new model to classify the baggage samples. Secondly, based on the three-dimensional characteristics of LiDAR data, a classification method based on Seed area is proposed. By comparing the classification credibility values of two classification methods, the further classification results reached 91.33%. In addition, the filling rate and the average Gaussian curvature entropy were used to classify the hard shell packing box and the Luggage case in detail, and the classification results reached 100%. The experimental results show that target detection and classification system is more robust and has better recognition and classification effects.

Keywords: LiDAR; Target detection and classification; RDforest new model; Seed area; Classification

credibility.

1. Introduction

With the development of artificial intelligence (AI), the Light Detection And Ranging (LiDAR), i.e., light detection and measurement, has become the "smart eye" of AI robots. The applications of the LiDAR technology in the field of AI robots has become the focus of the industry.

--- * Corresponding author.

29

The range-finding data of the two-dimensional LiDAR is widely used in the target recognition and classification, but it is seldom used in the air baggage detection and classification. Moreover, the existing baggage detection and classification technology is usually based on the depth image. For example, Gao and his colleagues [1] used the Kinect sensor to collect the depth image of the baggage, the author used the mean square deviation of the depth value of the target surface for rough classification, and then the number and area of the high-rise units after clustering are used for detailed classification, and finally the classification accuracy is 85%. In addition, the range-finding data of the LiDAR is generally used to detect the quantity and the size of the baggage. In [2], the author used a dual laser range-finder to scan the outer contour of the baggage, and proposed an algorithm based on the hierarchical clustering and the cube fitting. It effectively checks the number and the size of the air baggage and raises the speed at which passengers can use the self-service baggage devices. In this article, we apply the LiDAR sensor to the self-service baggage check-in system, which can detect the three-dimensional shape of the baggage and identify the type of baggage automatically, so as to select the right way of the baggage check, meeting the growing needs of airlines for the automatic check-in and the baggage check.

In the target classification, the feature extraction is the most important. The geometric and statistical features obtained from the range-finding data of the LiDAR are very intuitive. In the research of the autonomous driving and the obstacle recognition, Lee M and his colleagues [3] used the LiDAR point cloud data to extract three features of width, strength and range variance, and accurately classified obstacles into four categories through Euclid's theorem: four-wheel cars, rubber cones, two-wheel bicycles and pedestrians. In addition, for the surface of the object, the angular point [4] and curvature [5] of the object can also be extracted.

In order to solve the problem of the difficulty of the baggage classification, the baggage detection and classification system is designed. The LiDAR is used to scan the baggage and collect the data, and then the data is preprocessed. We proposed two classification algorithms based on the object and the seed region respectively, and the classification results of the two classification algorithms are compared with each other. Finally, the experimental results of the baggage detection classification system are obtained by combining the two classification algorithms. In addition, the hard shell packing boxes and pull bar boxes were detailed classified by combining with the filling rate and the mean gaussian curvature entropy.

2. Air baggage detection equipment

The LiDAR used in this equipment is the SICK LMS 511, which belongs to a two-dimensional LiDAR sensor. Figure 1 (a) is an air baggage detection simulation device. The LiDAR is installed above the direction of the baggage movement, driven by a stepper motor, and based on the speed of air conveyor. The baggage is scanned in tangential direction along the slide bar above the baggage. The Scanning range is 90 degrees and the resolution is 0.25 degrees. The data is stored through the PC, and the measurement data sequence format is hexadecimal. As shown in Figure 1 (b), the vertical distance between the ground and the center of the LiDAR scanning is D=240cm . The direction of v is parallel to the direction of motion of the LiDAR, x is perpendicular to v and parallel to the ground, and y is perpendicular to v and perpendicular to the ground.

30

3. Classification algorithm based on the object

3.1. Feature extraction

In the classification algorithm based on the object, the geometric features, the texture features, the angular features and the shape descriptors of the baggage are extracted, taking into account the characteristics of the baggage.

(1) Geometric features: we define two kinds of geometric features, one based on the height and width and the other on the "height difference". The former includes the average change in width, the average change in height, the high variance, the average value of the section area, the average value of the section girth, and the latter includes the average value of height difference, the variance of height difference, the total number Aof the height differences greater than the threshold s , the ratio 1

Aa

=

A M

/

of the height difference to the

threshold s , 1 M is determined by the scanning range and the resolution. Where, the "height difference" is

defined as the difference between any two adjacent columns of the same row in the preprocessed data matrix of the height.

(2) Texture features: the variance, the angular second moment, the information entropy and the moment of inertia.

(3) Angular features: for the angular points in the diagonal direction of the baggage surface, we extracted the diagonal direction data from the height matrix, and set the threshold s . If the absolute value of the 2

difference between two adjacent data in the extracted data is greater than s , it is said that the point is an 2

angular point in the diagonal direction. In addition, we use the Harris angular point detection method to detect the baggage surface. Firstly, the optical imaging feature matrix 'Y is constructed from the matrix Y , i.e., the data of the matrix Y is extracted according to the column, keep the position of the column data unchanged, and make up 0 at other positions of Y . Thus, M optical imaging feature matrices '_{Y of the same size as Y are} constructed to represent M vertical sections of the sample. Then, the angular point of the vertical cross-section of the sample surface can be obtained by detecting the Harris angular point on Y . '

(4) Shape descriptors: the effective height feature matrixs is transformed into two-dimensional gray image, and the filling rate and the convex hull character can be calculated.

3.2. RDforest new model

Deep forest [9] is a forest set with a cascade structure, including the multi-particle scan and the cascade forest. It x y baggage ground r v� Sick LMS511 LiDAR D

(a) Physical map (b) Installation schematic Figure 1: LiDAR physical map and installation schematic

31

is proved that this model is effective in the classification of small-scale samples. Considering the advantages of deep forest, in order to enhance the generalization ability of the model and reduce the error, we introduced the ridge regression before the multi-particle scan, and reconstructed the original features. In addition, in order to reduce the effect of trees with poor classification effect, increase the effect of trees with good classification effect, and reduce the distance between similar trees, expand the distance between trees with large differences, the category probability estimation in deep forest was optimized, and the RDforest model was constructed as shown in Figure 2.

Figure 2: Architecture of Rdforest

Firstly, the original eigenvector is defined as l , n y is the sample category. The ridge regression model as

formula (1), find ξ that minimizes J( )ξ ,

2 2

( ) min || _n ||_F || ||_F

J l y

δ

ξ = ξ− +l ξ (1)

Where, l is the parameter that affects the weight, each l corresponds to a set of feature weights. The optimum parameters l are found quantitatively by Leave One Out Cross Validation (LOOCV). Thus, the corresponding feature weight of l is found, and the original feature vector is reconstructed as L . n

Secondly, the category probability estimation in deep forest is optimized and improved. The idea of category probability distribution is generated based on the calculation of the percentage of the instances of different categories falling into the leaf node of the decision tree in the deep forest. The category probability is defined as the weighted sum of the probability vector of the decision tree, which is input into the next cascade layer as a new feature vector. Thus, the classification accuracy is further improved.

The RDforest model contains Q cascade layer, each layer contains Mq random forests, and each forest is

composed of Tk q, decision trees. We use pn c( , , )t k q, to represent the probability that the sample n generated by the

the cascade layer q of the t -th tree of the k-th forest belongs to category c . The probability that the k-th forest forecast sample instance n is category c is ( , )

,

k q n c

v , which satisfies the formula(2),

, ( , ) ( , , ) , ₁ , , 1 Tk q k q t k q n c _t n c k q v p T = =

∑

(2)

32

The category vector of the k-th forest of the q-th cascade layer is expressed as ( , ) ( , ) ( , )

,1 ,

( k q , , k q)

k q

n n n C

v = v  v , and

the category vector of the q-th cascade layer is expressed as ( )q ₌₍ (1, )q _{, ,} (Mq, )q ₎

n n n

v v  v . It is assumed that the reconstructed feature vector L is n x after multi-particle scan, and n x is connected with category vector n vn( )q

and input to the next cascade layer. For example, the concatenated vector (1)

n

z of the concatenated forest after the first level training is expressed as (1) _{( ,} (1)₎

n n n

z = x v , then, (1)

n

z is inputed into the second cascade. By that analogy, we can get the concatenation vector of the category q is ( )q _{( ,} ( )q ₎

n n n

z = x v .

In order to improve the classification accuracy, we improved the method of solving category probability in formula (2), such as formula (3).

Figure 3:Instance of the category vector generation of the first cascade

, ( , ) ( , , ) ( , , ) , ₁ , k q T k q t k q t k q n c _t n c v =

∑

₌ p ⋅

ω

(3)

Where,

ω

( , , )t k q _{represents the weight of the t -th tree of the k-th forest at the level q.}

Figure 3 shows the improved first instance of the cascading layer category vector generation, and the generation process of category probabilities is shown in the Figure 3. It's worth noting that, we weight the decision tree in the forest, which does not depend on the category c .

As shown in Figure 3, the category probability of the three categories training of the q-th cascade layer can be calculated respectively, as shown in formula (4), (5) and (6),

( , ) (1, , ) (2, , ) (3, , ) ,1 0.2 0.3 0.2 k q k q k q k q n v =

ω

+

ω

+

ω

(4) ( , ) (1, , ) (2, , ) (3, , ) ,2 0.4 0.5 0.8 k q k q k q k q n v =

ω

+

ω

+

ω

(5) ( , ) (1, , ) (2, , ) (3, , ) ,3 0.4 0.2 0.0 k q k q k q k q n v =

ω

+

ω

+

ω

(6)

Where, the weight

ω

( , , )t k q _satisfies , _{( , , )}

1 1

k q

T _{t k q}

t= ω =

33

Next, the weight is optimized with Pearson correlation coefficient ρ [10], which is used to measure the distance between two decision trees in the Distance Metric Learning. Define a category probability vector as

( , , ) ( , , ) ( , , ) ,1 , =( t k q,..., t k q ) t k q n n n C p p p , the distance ₍ ( , , )t k q_, (t 1, , )k q ₎ n n

d p p + between two category probability vectors can be calculated by formula (7),

( , , ) ( 1, , ) ( , , ) ( 1, , )

( t k q, t k q ) ( t k q , t k q )

n n n n

d p p + =ρ p p + (7)

When discussing decision tree weights, ρ only reduces the distance between similar decision trees and increase the distance between different decision trees, however, the classification ability of each decision tree itself is not considered.

Therefore, a new approach is proposed to evaluate the classification capability of the decision tree by using the out-of-bag data, and the weight of each decision tree is further optimized.

The training data is randomly divided into in-of-bag data and out-of-bag data, where the in-of-bag data is used to construct the tree, and the out-of-bag data provides an unbiased estimate of the training error.

Since the out-of-bag data does not involve constructing the trees, learning weights from the out-of-bag data can avoid overfitting.

We ran each tree on the corresponding out-of-bag data, and obtianed the classification accuracy oob of the tree, which represents the classification ability of the decision tree.

The oob is used as part of the decision tree weight optimization, combined with formula (7), the weight of the t -th tree of the k-th forest at the q-th level can be expressed as formula (8),

, ( , , ) ( 1, , ) ( , , ) ( , , ) ( 1, , ) 1 | ( , ) | ( ( , )) k q t k q t k q t n n t k q T _{t k q t k q} t n n t oob d p p oob d p p ω + + = + = +

∑

(8)

Substituting formula (8) into formula (3), the category probability ( , ) ,

k q n c

v of the k-th forest at the q-th level can be obtained as shown in formula (9),

, , , ( , , ) ( 1, , ) ( , ) ( , , ) ( , , ) ( , , ) , ₁ , ₁ , ( , , ) ( 1, , ) 1 | ( , ) | ( ( , )) k q k q k q t k q t k q T T t n n k q t k q t k q t k q n c _t n c _t n c _T t k q t k q t n n t oob d p p v p p oob d p p ω + = = ₊ = + = ⋅ = ⋅ +

∑

∑

_∑

(9)

The method that solve the weighted class probability, not only reduce the influence of trees with poor classification effect, increase the influence of trees with good classification effect, but also reduce the distance between similar trees, enlarge the distance between the larger differences trees, thus the classification accuracy is improved

34

4. Classification algorithm based on the seed region

The three-dimensional point cloud constructed by the LiDAR data can better reflect the morphological characteristics of the object. According to the characteristics of the surface of the baggage, i.e., the shape of the local area in the same type of baggage is similar, and the surface in the different types of baggage is quite different. Therefore, the target is classified based on the three-dimensional point cloud and the European geometry [11].

Firstly, the original coordinate system was transformed into the Euclidean three-dimensional coordinate system, and the three-dimensional model can be established with taking the projection point of the bottom left corner of the scanned baggage on the ground as the origin, as shown in Figure 4. Secondly, in order to accommodate different categories of the baggage, and avoid the influence of small parts such as the wheel on classification results, we divided the baggage plane xOy into θ (θ=6) zones on average, these zones was mapped to the surface of the baggage, and the center of each zone is the seed point O xθ( θ,y zθ, θ). The circular seed zone is obtained by taking the shortest distance from the seed point Oθ to the boundary of this region as the radius R .

The point in the θ seed region of an object is denoted as

{

pθ1,pθ2,pθ3,...,pθk

}

.

The average value of the L2 norm of all points and seed points in the seed region of the n -th sample can be calculated by the formula (10),

2 1 1 = K || || n _k pk O K θ θ ρ

∑

₌ − (10) Where, _{|| || = |}2 _||2 _{|| ||}2 _{|| ||}2 k k k k

p −Oθ x −xθ + y −yθ + z −zθ represents the number of points in the seed area.

Figure 4: Three-dimensional model

After comparing the average L2 norm matrix of each type of the baggage, it can be seen that the average L2 norm of the class I baggage is more stable, and the average L2 norm change of the class III baggage varies greatly. Therefore, the variance of the average L2 norm matrix of each baggage is calculated separately, as shown in formula (11).

35

6 6 ₂ 1 1 1 ₍ 1 ₎ 6 6 n n n SD =

∑

_θ= ρθ−

∑

_θ= ρθ (11)

The smaller the variance, the more stable the data. After a lot of training, the variance ranges of Class I, Class II and Class III baggages are: 0≤SDn ≤0.3, 0.3<SDn ≤ , 1 SDn ≥ . 1

Due to the different complexity of different surfaces, and the more complex distribution, the more information it contains.

In order to improve the discrimination accuracy, the average Gaussian curvature entropy ηn of the six regions

of each baggage is calculated by formula (12),

6 1 1 = 6 n _θ θ η

∑

₌τ (12) Where, 0 ln l θ α θα θα τ = −

∑

= β β (13) ' ' 0 ( ) / l ( ) k k θα θ α θ β = α

∑

= α (14)

l represents the maximum gaussian curvature in the seed region, _k'_{( )}

θ α and βθα represent the number and probability that gaussian curvature is α in the θ-th seed region, τθ represents the gaussian curvature entropy

of the θ-th seed region.

The more complex the surface, the more information it contains. According to the classification of baggage, the average Gaussian curvature entropy of Class III baggage is the largest.

In summary, the L2 norm of the six regions in the average L2 norm matrix of Class I baggage varies smoothly, and the average Gaussian curvature entropy is small.

The L2 norm of Class III baggage varies irregularly, and the average Gaussian curvature entropy is large.

5. Experiment

In order to verify the effectiveness of the system, a total of 360 valid baggage samples were selected, and 120 for each type of sample.

The ratio of training set to test set is 7:5.

According to the characteristics of the air baggage and the actual baggage handling in the process of classification operation, the description of the baggage category is shown in Table 1.

36

Table 1: Baggage samples

Class of

baggage Surface characteristics of baggage

Shipping

method Part sample picture

Class I: Regular Smooth or small regular change Direct shipment

Class II: Subangular

Poor flatness, obvious concave convex ,

wrinkles, etc. Put in the tray

Class III: Irregular

Amplitude changes are not regular, or the baggage is out of scale

Not allowed to check

5.1. LiDAR data preprocessing

In order to facilitate the calculation and remove the noise from data, we preprocessed the raw LiDAR data, as shown in Figure 5.

Where, R represents the distance from the light center of LiDAR to the surface of the object, i i is scan frames.

The dimension of W and H matrix is N×M,

N

is the scan frames of the LiDAR.

Considering that the running rate of the LiDAR is far less than the scanning rate of LiDAR, and the data amount of each frame is very small, then N� M .

Due to the photons of the LiDAR are easily absorbed and attenuated by the object, the sample data has noise points. Therefore, the median filtering [12] is used to process these noise points.

The threshold segmentation is used to segment the sample feature data from the surrounding environment.

5.2. Analysis of the experiment and results

The baggage was classified according to two classification algorithms based on the object and the seed region respectively, and the classification result is obtained.

Then, according to the classification result, the credibility values of the two methods can be obtained, as shown in Table 2.

37

Figure 5: LiDAR data preprocessing flow chart Table 2: Credibility comparison

Method Class I ClassII ClassIII

Object-based 0.55 0.6 0.42

Seed region-based 0.45 0.4 0.58

Combining the credibility values of the two methods, the specific determination method of the baggage category can be obtained: For example, the classification algorithm based on the object determines that an object is regular, and the classification algorithm based on the seed region determines that the object is subangular, since the credibility of the two is 0.55 and 0.4 respectively, thus, the object is finally determined to be regular, and so on. Therefore, when determining the category of a baggage, the final classification result can be further optimized by the credibility of two methods. As shown in Table 3, the classification accuracy based on credibility values classification method, object classification method and seed region classification algorithm. The classification algorithm based on seed region takes the average of the classification accuracy of three types of the baggage.

Table 3: Comparison of classification algorithm in this paper

Classification algorithm Classification accuracy（%） Object-based 90.67%

Seed region-based 88.67% Credibility value-based 91.33%

38

In addition, the feature vectors was input into the traditional classifier to obtain the classification results, and the results were compared with the classification results based on the credibility value proposed in this paper, the results are shown in Table 4.

Table 4: Comparison of classification algorithm in this paper with traditional classification algorithm

Classification method Classification accuracy（%）

SVM 84%

Random forest 88.67%

Deep forest 88% Our method 91.33%

In addition, in the experimental samples, the regular object can also be divided into the packaged hard shell packing box and the Luggage case. Considering that the Luggage case has the wheels, the tie rods and the locks, and the contour of the hard shell packing box is nearly rectangular, thererfore, 55 groups of the hard-shell packing boxes and the Luggage cases were taken respectively for the detailed classification experiment. The filling rate R is taken as the first evaluation measure, and the boundary threshold is set as 0.925. When

0.925

R< , the object is a Luggage case, otherwise it is a hard-shell packing box. Secondly, the average Gaussian curvature entropy ηn is taken as the second evaluation measure, and the category of the baggage

classified by the first evaluation quantity is further determined. When η ≥n 4.75, the object is judged as a Luggage case, otherwise it is judged as a hard-shell package box. Finally, the classification accuracy of the hard shell packing box and the Luggage case is 100%.

The experimental results show that, the scanning data of the two-dimensional LiDAR can effectively classify the air baggage into three categories: regular, subangular and irregular. Where, the classification algorithm based on the seed region makes full use of the characteristics of the three-dimensional model and plays an important role in the classification of the irregular baggage. The accurate discrimination of the hard shell packing box and the Luggage case shows that the baggage filling rate and the average Gaussian curvature entropy are important evaluation indicators. Moreover, the RDforest model classifier has higher classification accuracy and better robustness than SVM and random forest.

6. Conclusion

By using the widely used LiDAR, the baggage detection and classification system is constructed, and the baggage is scanned, identified and classified. Firstly, according to the characteristics of the baggage, the classification algorithm based on the object is proposed, i.e., the feature vector of the baggage is extracted and input into the RDforest model for experiment. Secondly, according to the three-dimensional characteristics of the LiDAR data, the classification algorithm based on the seed region is proposed. By calculating the credibility values of the two classification algorithms for classifying objects, a more accurate classification accuracy of 91.33% is obtained by comparing and optimization.

39

application scenarios, and the classification accuracy is relatively high, but the efficiency in the process of the feature extraction is relatively low. In future research, other more effective features can be studied to improve the efficiency of the feature extraction. Furthermore, samples can be classified by combining various effective features to increase the diversity of the sample classification. In addition, the combination of LiDAR data and CCD image data can effectively improve the classification efficiency and accuracy.

References

[1] GAO Q J, WEI Y Y. Airline baggage classification based on the mean square error of the surface depth [J]. Computer Engineering & Science, 2017, 39(1):125-130.

[2] GAO Q, LI T, LUO Q. An Algorithm for Inspecting the Number of Self Check-In Airline Luggage Based on Hierarchical Clustering[C]// Sixth International Conference on Measuring Technology and Mechatronics Automation. IEEE, 2014:71-74.

[3] LEE M, HUR S, PARK Y. An Obstacle Classification Method Using Multi-feature Comparison Based on 2D LIDAR Database[C]// International Conference on Information Technology - New Generations. IEEE, 2015:674-679.

[4] LI P, WEI Z H, HE X, et al. Object recognition based on shape feature fusion under multi-views[J]. Optics & Precision Engineering, 2014, 22(12):3368-3376.

[5] ZHANG Y H, GENG G H, WEI X R, SHI C C, ZHANG S L. Feature extraction of point clouds using the DBSCAN clustering [J]. JOURNAL OF XIDIAN UNIVERSITY, 2017, 44(2): 114-120.

[6] SUN J, LAI Z. Airborne LiDAR Feature Selection for Urban Classification Using Random Forests[J]. Geomatics & Information Science of Wuhan University, 2014, 39(11): 1310-1313.

[7] SHANG M S, WANG K C. Advanced Image Registration Method Based on Harris and SIFT Algorith [J]. MICROELECTRONICS & COMPUTER, 2018, 35(6) :132-134.

[8] SREEVALSAN-NAIR J, JINDAL A. Using gradients and tensor voting in 3D local geometric descriptors for feature detection in airborne lidar point clouds in urban regions[C]// IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2017:5881-5884.

[9] ZHOU Z H, FENG J. Deep forest: Towards an alternative to deepneural networks[J]. arXiv preprint arXiv:1702.08835, 2017.

[10] LY A, MARSMAN M, WAGENMAKERS E J. Analytic posteriors for Pearson's correlation coefficient[J]. Statistica Neerlandica, 2018, 72(1):4.

[11] ZHOU B N, MIN H S, KANG Y W. Research on 3D Point Cloud Segmentation Algorithm in PCL Environment [J]. MICROELECTRONICS & COMPUTER, 2018, 35(6):101-105.

[12] KIM D, JANG H U, MUN S M, et al. Median Filtered Image Restoration and Anti-Forensics Using Adversarial Networks[J]. IEEE Signal Processing Letters, 2018, PP(99):1-1.