Automatic individual tree detection and canopy segmentation from three-dimensional point cloud images obtained from ground-based lidar
Kenta ITAKURA and Fumiki HOSOI†
Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Yayoi, Bunkyo-ku, Tokyo 113-8657, Japan
Abstract
Lidar (light detection and ranging) has been widely utilized for estimating the structural parameters of plants, such as tree height, leaf inclination angle, and biomass. However, individual trees have been primarily manually extracted from three-dimensional (3D) point cloud images. Automatically detecting each tree and analyzing its structural parameters is desirable. In this study, we propose a method to (1) detect each tree from 3D point cloud images obtained from ground-based lidar, (2) estimate the number of trees and diameter at breast height (DBH) from the detected 3D point cloud images of trees, and (3) segment each tree canopy. First, we focused on point clouds whose height ranged from 0.5 to 1.5 m and detected each cluster of tree trunks. Then, the clusters were expanded by classifying other points to the clusters that are located near the points and then repeating this process. The process assigns the points in the 3D point cloud image to each tree in the upward direction and separates not only tree trunks but also tree canopies. As a result, the trees in 3D point cloud images were detected with high accuracy, and the number of trees and DBH was estimated.
Moreover, each tree canopy was segmented.
Key words: Automatic tree detection, Canopy segmentation, DBH estimation, Lidar, Three-dimensional imaging
1. Introduction
The accurate determination of tree structure is necessary for a variety of applications within the fields of urban forestry, ecology, environmental protection, and forestry (Omasa et al., 2002; Oshio et al., 2013; Morgenroth and Gomez, 2014). For example, the spatially complicated structure of forests influences the light conditions and microclimate within the forest (Hosoi and Omasa, 2014). Recently, lidar (light detection and ranging) has gained popularity as a useful tool for obtaining the three-dimensional (3D) structural properties of plants in lieu of manual measurements, which is time consuming and laborious (Hopkinson, 2004; Maas et al., 2008; Kato et al., 2014).
Ground-based lidar enables measurements from underneath the canopy and provides information for scales ranging from needle/leaf to forest canopy (Lefsky et al., 2002; Van Leeuwen and Nieuwenhuis, 2010; Dassot et al., 2011; Kato et al., 2014).
Lidar has been widely utilized to obtain a range of various measurements, for example, for measuring diameter at breast height (DBH) and tree height (Hopkinson et al., 2004; Omasa et al., 2006; Strahler et al., 2008), leaf inclination angle (Hosoi et al., 2009; Hosoi and Omasa., 2015), leaf area density (Hosoi and Omasa, 2006; Hosoi et al., 2010; Hosoi and Omasa, 2012), amount of biomass (Lefsky et al., 1999; Urano and Omasa, 2005), and green area index (Liu et al., 2017).
It is desirable to automatically detect each tree and analyze its structural parameters (Huang et al., 2011). There are some
studies on automatic tree detection from 3D point cloud images.
Lovell et al. (2011) proposed a method to detect the trees in the 3D point cloud model using the reflection intensity of a lidar laser beam. However, some types of lidars, e.g., a lightweight lidar for mounting on a mobile platform (Pan et al., 2017), cannot provide accurate information about beam intensity. Tree trunks can be detected by conducting Hough transform or circle (cylinder) fitting to a cross section at a certain height of 3D point cloud tree images, and the detection helps in DBH estimation (Simonse et al., 2003; Henning and Radtke, 2006; Bienert et al., 2007; Tansey et al., 2009; Huang et al., 2011). Nevertheless, if a tree trunk cross section forms an incomplete circle and/or includes tree branches and other noise, the cross section cannot be identified as a tree trunk; therefore, it is difficult to identify trees that are at a distance from a lidar scanner, thereby leading to errors while estimating their DBH (Thies and Spiecker, 2004).
Moreover, to obtain structural information on individual tree canopies, each canopy should be segmented, which is a challenging task with 3D point cloud images from ground-based lidars.
In this study, we propose a method to (1) detect each tree from 3D point cloud images obtained from ground-based lidar even if the tree trunks form incomplete shapes, (2) estimate the number of trees and DBH from the detected 3D point cloud images of trees, and (3) segment each tree canopy.
2. Materials and Methods
2.1 Study site and trees selected for automatic tree detection The campus of the University of Tokyo in Tokyo, Japan was selected as a study site. The campus has several types of trees, such as the Himalayan cedar (Cedrus deodara), Japanese zelkova (Zelkova serrata), maidenhair tree (Ginkgo biloba), camellia Received; March 2, 2018
Accepted; April 20, 2018
†Corresponding Author: [email protected] DOI: 10.2480/agrmet.D-18-00012
(Camellia japonica), ginkgo (Ginkgo biloba L.) and sasanqua (Camellia sasanqua). The 58.3 ha Shinjuku Gyoen National Garden in Tokyo, Japan was also selected, which has more than 10,000 trees, such as the cherry tree (Cerasus Mill.), tulip tree (Liriodendron tulipifera), plane tree (Platanus), Himalayan cedar (Cedrus deodara), Formosan sweetgum (Liquidambar formosana), and bald cypress (Taxodium distichum).
Four plots were selected as study sites. Rows of ginkgo trees, planted at an interval of about 3 m, and rows of Japanese zelkova, planted at an interval of about 1 to 2 m, at the University of Tokyo were chosen. The diameters of the ginkgo trees and Japanese zelkova were about 45 to 80 cm and 30 to 60 cm, respectively. Himalayan cedar and cherry trees, with diameters of about 50 to 70 and 30 to 80 cm, respectively, were selected from the Shinjuku Gyoen National Garden. The area of the plot with Himalayan cedar and cherry trees were about 1600 m2 and 5000 m2, respectively.
2.2 Lidar measurement and acquisition of 3D point cloud images The lidar used in this study was a VLP-16 (Velodyne Lidar Inc., USA) with 16 laser sources rotating at a frequency of 10 Hz. The weight, measurement accuracy, and distance range are 830 g, ±3 cm, and 100 m, respectively. The lidar captures 3D point cloud images while it moves on a platform, enabling wide area measurements, unlike other widely used ground-based lidar systems, which are installed at one point. The lidar was attached to a handheld tripod, which raised the lidar’s height to about 2 m, and we walked with it along the target trees. The inclination angle of the lidar directly influences the measurable area (Itakura et al., 2018a), so the angle was adjusted relative to the study sites. One scan (i.e., one rotation of 16 laser beams) captures sparse 3D models. Making use of the Simultaneous Localization and Mapping (SLAM) method proposed by Zhang and Singh (2014), detailed 3D point cloud models were constructed as shown in Fig. 1 (a) (Pan et al., 2017).
2.3 Tree detection and canopy segmentation method with 3D point cloud models
Point cloud data whose distance from the ground was about 0.5 to 1.5 m were extracted, where mainly tree trunks were included but, additionally, tree branches and other noise existed. All points constituting the 3D model were converted into voxel coordinates in which each X, Y, Z value of the point cloud data was rounded off to the nearest integer value, allowing calculation of structural parameters efficiently (Hosoi and Omasa, 2006; Hosoi and Omasa, 2007; Hosoi et al., 2010; Grau et al., 2017). Voxel size was selected according to tree size, and in this study voxel size was set to 0.1 m or 0.2 m. Voxels corresponding to coordinates converted from points within the data were assigned an attribute value of 1, and the attribute value of vacant voxels was set as zero (Hosoi and Omasa, 2006). In the voxel coordinate, each non-zero voxel was given a tree number, that is, a label for identifying a particular tree. At first, no non-zero voxel contained a tree number. The numbers were assigned by the tree detection method explained below.
For tree trunk detection, firstly, one voxel containing the attribute value 1 within the extracted data whose distance
from the ground is about 0.5 to 0.6 m was focused on (point of interest) and the label “Tree number 1” was given in addition to the attribute value. Next, a cube was formed, 7 voxels on each side, with the center cube as the point of interest. If the cube is too big, tree trunk detection is prone to error; if the cube is too small, the calculation time takes much longer. Tree number 1 was assigned to the non-zero voxels in the cube that had not been given any tree numbers. Subsequently, the voxels that had been given a tree number in the previous step went through the same expanding procedure (i.e., one voxel became the next point of interest and the non-zero voxels surrounding it were assigned the same tree number), and so the procedure went on until all the points of interest had been dealt with, expanding the region of the cluster. The process ended when there were no non-zero voxels that had not been assigned a tree number around the cluster. Then, a new tree number was assigned to a voxel which belonged to another cluster and had no tree numbers with a height of 0.5 to 0.6 m, and the tree number was assigned to the voxels in the cluster likewise. Repeating this process, each cluster corresponding to each tree trunk could be detected.
Although a few groups of points corresponding to tree branches and noise were clustered like tree trunks, the number of points of the cluster was much smaller than that of the tree trunks. Thus, the number of voxels of each cluster was counted, and those not considered to be tree trunks were discarded based on a threshold.
The threshold was not the same across all species, study sites, and voxel sizes; hence, the threshold was set after determining how many voxel points noise clusters had.
After the number of trees was identified, as shown in Fig. 1 (b), the region of the tree trunks was expanded. However, tree numbers were not provided to the voxels whose height was more than 1.5 m. Tree numbers were assigned to the voxels whose attribute value was 1 in ascending order of height, in accordance with the tree detection method discussed in the previous paragraph. In this process, a new tree number was not assigned, and the method was used only for the expansion of each tree trunk region in a vertical direction, as shown in Fig. 1 (c). When expanding the region of a point of interest, some cubes involved in the expansion included two tree numbers when the point of interest was located between trees. In such cases, the mode of the tree number in the cube was assigned to the point of interest.
Repeating this process, each tree, including its canopy, was also segmented, as represented in Fig. 1 (d). In order to ensure the accuracy of tree canopy segmentation, 20 cherry blossoms in Shinjuku Gyoen National Garden whose tree canopies overlapped each other were selected randomly, and the border of their canopies was discriminated manually (i.e., by sight). Next, it was compared with the border which had been discriminated automatically using the present method to calculate the error of automatic canopy segmentation.
2.4 DBH estimation
To validate the DBH estimation, measured and estimated DBH were compared. Twelve ginkgo trees at the University of Tokyo were selected, and their DBH were measured with tree calipers. After canopy segmentation, the region corresponding to each tree trunk was extracted. In this process, the raw 3D points
Fig. 1. Tree detection and segmentation process. (a) Three-dimensional point cloud image of four trees. The enclosed tree trunk represents the extracted parts applied for each tree trunk detection. (b) Detected tree trunks, at a height of about 0.5 to 1.5 m, are assigned different tree numbers. The difference of brightness represents each detected tree. (c) The expanding process of each tree. Each cluster expands in the direction of the arrows. (d) Four trees automatically segmented in the 3D point cloud image. The rectangular cuboids enclose each tree.
of each tree trunk were recalled instead of the corresponding voxels and utilized for DBH estimation. For the estimation, circle fitting was conducted on the recalled points of each tree trunk (Omasa et al., 2002; Simonse et al., 2003).
3. Results and Discussion
Table 1 shows the results of the number of trees counted through automatic tree detection. In the rows of ginkgo and Japanese zelkova, all of the trees (Tree numbers 18 and 16) were correctly separated. In the study site with the Himalayan cedar and cherry trees, all of the trees on the site (Tree numbers 23 and 60) were detected; however, a total of three and two errors of commission occurred, respectively. The segmented cherry trees are represented in Fig. 2 (a) side view and (b) bird’s-eye view.
The absolute estimation error of DBH was 3.8 cm, and its mean absolute percentage error (MAPE) was 6.7%. The mean of the difference of the border of the tree canopies which was manually and automatically discriminated was 1.31 m, while the mean of the major axis of the canopies was 11.58 m.
In this study, to detect an individual tree, a part of its tree trunk was focused on. Some points that are close to each other are connected, and a cluster whose point number was greater than the threshold was recognized as a tree trunk; this ensured that tree trunk detection worked even if the cross section did not form a complete circle. In this method, the height of the initial points of each cluster was limited to about 0.5 to 0.6 m. Tree branches themselves may form a cluster and be misidentified as tree trunks. However, branches are located at a higher level
than tree trunks. Therefore, by limiting the height of the initial points of each cluster such that it was low enough to avoid the branches (herein 0.5 to 0.6 m), the branches could be prevented from constituting a cluster. Consequently, the point cloud of the trunk could be accurately extracted. In previous studies, using circle fitting and Hough transform, tree trunks were detected with high accuracy (i.e., about 86%-98%) (Hopkinson et al., 2004). However, if the trunk was incompletely depicted, the estimation accuracy dropped below 52% (Thies and Spiecker, 2004). It is inevitable that all of the points of the trees cannot be fully scanned from the lidar, as shown in Fig. 2 (c). Therefore, it is advantageous that our proposed tree detection method detects incompletely scanned trees.
Commissions occurred with Himalayan cedar and cherry species because, in a few cases, the number of points constituting tree branches close to the ground were larger than the tree trunks themselves. In those cases, the branch was misidentified as a trunk. The method should be implemented under the condition that the target trees do not have branches close to the ground.
By regarding a new point that has been assigned a tree number as an initial point and expanding the cluster to the whole tree, the canopies of each tree could be segmented. Neighboring canopies had small gaps in the 3D point cloud data; thus, each canopy could be segmented without growing up to neighboring canopies.
The accuracy of automatic DBH estimation in this study was similar to the manually estimated value from the lidar used in this study, which was about 1-3 cm (Pan et al., 2017;
Itakura et al., 2017; Itakura et al., 2018b). In the present study’s tree trunk detection method, the noise around the trunk is not included because points far from the cluster are not assigned a tree number. In contrast, when tree trunks are detected through the conventional circle (cylinder) fitting method (Omasa et al., 2002; Simonse et al., 2003), the noise around the trunk influences the result because noise cannot be eliminated in that method. Moreover, in the present study, estimations were conducted with raw point cloud data instead of the points in the voxel coordinates, as discussed in section 2-4, which has higher resolution. By using the raw point cloud data, this study’s DBH estimation was conducted with high accuracy.
It is expected that the tree detection method used in this study can be applied to large areas of trees. The lidar used in this study
is not only light and portable, but also applicable to moving platforms. This property enables measurements over a wide range compared to conventional lidar, which should be fixed on the ground. If the tree detection method in this study is used in conjunction with lidar, the structural parameters of trees in large areas can be automatically and effectively estimated.
References
Bienert A, Scheller S, Keane E, Mohan F, Nugent C, 2007:
Tree detection and diameter estimations by analysis of forest terrestrial laser scanner point clouds. ISPRS Workshop on Laser, Scanning 2007 and SilviLaser 2007, 12-14 September, Espoo, Finland, 50-55.
Dassot M, Constant T, Fournier M, 2011: The use of terrestrial LiDAR technology in forest science: application fields, benefits and challenges. Annals of Forest Science 68, 959-974.
Grau E, Durrieu S, Fournier R, Gastellu-Etchegorry J-P, Yin T, 2017: Estimation of 3D vegetation density with Terrestrial Laser Scanning data using voxels. A sensitivity analysis of influencing parameters. Remote Sensing of Environment 191, 373-388.
Henning JG, Radtke PJ, 2006: Detailed stem measurements of standing trees from ground-based scanning lidar. Forest Science 52, 67-80.
Hopkinson C, Chasmer L, Young-Pow C, Treitz P, 2004:
Assessing forest metrics with a ground-based scanning lidar.
Canadian Journal of Forest Research 34, 573-583.
Hosoi F, Omasa K, 2006: Voxel-based 3-D modeling of individual trees for estimating leaf area density using high-resolution portable scanning lidar. IEEE Transactions on Fig. 2. Sixty trees segmented in the Shinjuku National Garden. Each segmented tree is denoted by a different
color. (a) Side view. (b) Bird’s-eye view. The rectangular cuboids separate each tree. (c) An example of point cloud representing a part of a tree trunk.
Table 1. Results of the number of trees through automatic tree detection.
Species All
trees Estimated
trees Omission
error Commission
error Accuracy (%)
Ginkgo 18 18 0 0 100
Japanese
zelkova 18 18 0 0 100
Himalayan
cedar 23 26 0 3 88.4
Cherry
blossom 60 62 0 2 96.8
Geoscience and Remote Sensing 44, 3610-3618.
Hosoi F, Omasa K, 2007: Factors contributing to accuracy in the estimation of the woody canopy leaf area density profile using 3D portable lidar imaging. Journal of Experimental Botany 58, 3463-3473.
Hosoi F, Nakai Y, Omasa K, 2009: Estimating the leaf inclination angle distribution of the wheat canopy using a portable scanning lidar. Journal of Agricultural Meteorology 65, 297-302.
Hosoi F, Nakai Y, Omasa K, 2010: Estimation and error analysis of woody canopy leaf area density profiles using 3-D airborne and ground-based scanning lidar remote-sensing techniques.
IEEE Transactions on Geoscience and Remote Sensing 48, 2215-2223.
Hosoi F, Omasa K, 2012: Estimation of vertical plant area density profiles in a rice canopy at different growth stages by high-resolution portable scanning lidar with a lightweight mirror. ISPRS Journal of Photogrammetry and Remote Sensing 74, 11-19.
Hosoi F, Omasa K, 2014: 3-D remote sensing for measurement and analysis of forest structure. Japanese Journal of Ecology, 64, 223-231.
Hosoi F, Omasa K, 2015: Estimating leaf inclination angle distribution of broad-leaved trees in each part of the canopies by a high-resolution portable scanning lidar. Journal of Agricultural Meteorology 71, 136-141.
Huang H, Li Z, Gong P, Cheng X, Clinton N, Cao C, Ni W, Wang L, 2011: Automated methods for measuring DBH and tree heights with a commercial scanning lidar. Photogrammetric Engineering and Remote Sensing 77, 219-227.
Itakura K, Kamakura I, Hosoi F, 2017: Estimation of tree trunk diameter by LIDAR while moving on foot or by car.
Eco-Engineering 29, 107-113.
Itakura K, Kamakura I, Hosoi F, 2018a: Calculation of moving distance when measuring tree height using portable scanning lidar and tree height measurement by using registration of images obtained on the ground and high places, Eco-Engineering 30, 7-14.
Itakura K, Kamakura I, Hosoi F, 2018b: A Comparison study on three-dimensional measurement of vegetation using Lidar and SfM on the ground, Eco-Engineering 30, 15-20.
Kato A, Ishii H, Enoki T, Osawa A, Kobayashi T, Umeki K, Sasaki T, Matsue K, 2014: Application of laser remote sensing to forest ecological research. Journal of The Japanese Forestry Society 96, 168-181.
Lefsky MA, Cohen WB, Acker SA, Parker GG, Spies TA, Harding D, 1999: Lidar remote sensing of the canopy structure and biophysical properties of Douglas-fir western hemlock forests. Remote Sensing of Environment 70, 339-361.
Lefsky MA, Cohen WB, Parke, GG, Harding DJ, 2002. Lidar remote sensing for ecosystem studies. Bioscience 52, 19-30.
Liu S, Baret F, Abichou M, Boudon F, Thomas S, Zhao K, Fournier C, Andrieu B, Irfan K, Hemmerlé M, 2017:
Estimating wheat green area index from ground-based LiDAR measurement using a 3D canopy structure model. Agricultural
and Forest Meteorology 247, 12-20.
Lovell JL, Jupp DLB, Newnham GJ, Culvenor DS, 2011:
Measuring tree stem diameters using intensity profiles from ground-based scanning lidar from a fixed viewpoint. ISPRS Journal of Photogrammetry and Remote Sensing 66, 46-55.
Maas HG, Bienert A, Scheller S, Keane E, 2008: Automatic forest inventory parameter determination from terrestrial laser scanner data. International Journal of Remote Sensing 29, 1579-1593.
Morgenroth J, Gomez C, 2014: Assessment of tree structure using a 3D image analysis technique - A proof of concept, Urban Forestry & Urban Greening 13, 198-203.
Omasa K, Urano Y, Oguma H, Fujinuma Y, 2002: Mapping of tree position of larix leptolepis woods and estimation of diameter at breast height (DBH) and biomass of the trees using range data measured by a portable scanning lidar. Journal of The Remote Sensing Society of Japan 22, 550-557.
Omasa K, Hosoi F, Konishi A, 2006: 3D lidar imaging for detecting and understanding plant responses and canopy structure. Journal of Experimental Botany 58, 881-898.
Oshio H, Asawa T, Hoyano A, Miyasaka S, 2013: Accuracy of the information on the external crown form of individual trees extracted by airborne LIDAR in urban spaces. Journal of The Remote Sensing Society of Japan 33, 350-359.
Pan Y, Kuo K, Hosoi F, 2017: A study on estimation of tree trunk diameters and heights from three-dimensional point cloud images obtained by SLAM. Eco-Engineering 29, 17-22.
Simonse M, Aschoff T, Spiecker H, Thies M, 2003: Automatic determination of forest inventory parameters using terrestrial laserscanning. Proceedings of the ScandLaser Scientific Workshop on Airborne Laser Scanning of Forests, Umeå, 251-257.
Strahler AH, Jupp DLB, Woodcock CE, Schaaf CB, Yao T, Zhao F, Yang X, Lovell J, Culvenor D, Newnham G, 2008:
Retrieval of forest structural parameters using a ground-based lidar instrument (Echidna®). Canadian Journal of Remote Sensing 34, 426-440.
Tansey K, Selmes N, Anstee A, Tate NJ, Denniss A, 2009:
Estimating tree and stand variables in a Corsican Pine woodland from terrestrial laser scanner data. International Journal of Remote Sensing 30, 5195-5209.
Thies M, Spiecker H, 2004: Evaluation and future prospects of terrestrial laser scanning for standardized forest inventories.
International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 36, 192-197.
Urano Y, Omasa K, 2005: Estimation of tree stem diameters and biomass in Japanese cedar forest using portable imaging lidar data. Journal of Agricultural Meteorology 60, 1175-1177.
Van Leeuwen M, Nieuwenhuis M, 2010: Retrieval of forest structural parameters using LiDAR remote sensing. European Journal of Forest Research 129, 749-770.
Zhang J, Singh S, 2014: LOAM: Lidar odometry and mapping in real-time. Robotics: Science and Systems Conference (RSS).
Berkeley, USA.
