Dynamically-Assembled Regression Forest

Motivation

Regression forest is a powerful regression tool, especially when it is difficult to find an explicit mathematical expression for modeling the complex relationship between the input values and output values. This intrinsic characteristic makes the regression forest quite suitable for predicting the rapid and nonlinear early development of cortical attributes. However, a single conventional regression forest (CRF) may not be sufficient for estimating vertex-wise cortical morphological attributes of the entire cortical surface. Because the cortical morphological attributes and their developments in infants are highly regionally heterogeneous, using a single CRF cannot make a precise estimation at the vertex level. An intuitive way to solve this problem is to partition the whole cortical surface into a set of small regions of interest (ROIs), and then train a local regression forest for each ROI. However, this strategy will lead to spatially unsmooth estimations around the boundaries of neighboring ROIs. This is because the cortical attributes of the vertices by the two sides of a ROI boundary are estimated using two different regression forests, which are trained independently with different training samples. Intuitively, increasing the overlapping area among ROIs could produce smoother estimation results, but it unexpectedly

smooth as the real data, more than 90% of the area of an ROI needs to be overlapped with its neighboring ROIs. Unfortunately, such a large portion of overlap requires quite a large number of ROIs in order to cover the whole cortex, and thus leads to a large computational workload, as a respective set of individual trees need to be trained for each ROI. Taking account of all these issues, I propose a Dynamically-Assembled Regression Forest (DARF). By first training a single decision tree for each vertex in the training stage and then locally grouping decision trees of neighboring vertices as forests in the testing stage, DARF is able to produce spatially smooth regression results and meanwhile also save a lot of computational cost.

Training Stage

In the training stage, one individual binary decision tree is trained at each vertex on the cortical surface. As shown in Figure 3.1, for the given vertex on the spherical cortical surface (mapped from the original cortical surface), one individual tree is trained using the nearby vertices within a training-neighborhood (i.e., the red region) as training samples. Each training sample can be denoted as a pair of a feature vector and a scalar regression target ∈ , ∈ . The feature vector consists of a set of features extracted from the local cortical attribute maps around the vertex at the input time point(s) (see Section 3.1.2), and the scalar target is the cortical attribute value of vertex at the target time point.

Testing Stage

In the testing stage, to predict the cortical attribute value of a given vertex at the target time point, as shown in Figure 3.1, all nearby individual trees within a testing-neighborhood (i.e., the blue region) are grouped together to form a vertex-specific forest. The feature vector of the given vertex is computed and then fed into each individual tree of the formed forest. The prediction

result is finally computed as the average of regression outputs from all trees of the formed forest. Of note, the optimal size of testing-neighborhood can be learned via a cross validation.

Figure 3.1. Training and testing stages for DARF. (a) The red region is the training neighborhood, where all the vertices on the spherical space are used as training samples. (b) The blue region is the testing neighborhood, where all the individual trees are combined together to form a forest in the testing stage. Note that the red and blue regions in (a) and (b) could have different sizes.

Different from the original way of using a regression forest, which trains a set of trees and makes them a fixed forest at the training stage, our method does not assign trees to any forest in the training stage. Instead, the forest is formed by group neighboring trees during the testing stage, and thus it is named “Dynamically-Assembled” Regression Forest. This novel way of forming forest has two advantages.

The first advantage is that, by using DARFs, the predicted cortical attribute map is smooth without any border-artifact near the boundary between ROIs as that in the ROI-based method. The reason is two-fold. First, as the neighboring DARFs share a large number of the same decision trees, they are very similar. Second, features of neighboring vertices, which are extracted from the

local cortical attribute maps, are also similar. By feeding the similar input features to the similar DARFs, the outputs at neighboring vertices generally have small differences, and thus the predicted cortical attribute map is smooth.

The second advantage is that, since one single decision tree can be shared by many nearby forests, the computational cost for training forests is significantly reduced. For instance, if in each forest there are 100 trees, using DARF would save 99% computational cost compared to the case of using the original regression forests (i.e., training a forest at each vertex). Even compared to the ROI-based strategy, DARF still significantly saves the computational cost in the training stage. Based on my experiments, to achieve the smooth predicted cortical attribute maps, highly- overlapped ROIs are needed, with the number of ROIs being nearly 20% of the number of vertices. If each forest owns 100 trees, using DARF would still save 19% computational cost.