Selection of Aggregation Interval

One of the most important goals of interval aggregation is to find reliable estimations of aggregate features within the interval, thus using them to calibrate models with the highest classification accuracy about the interval’s RSC type. Although there is more than one way to achieve this goal, for our study, we solidify this idea with the following steps:

1. The continuous segments, over which CFMs were collected are divided by different interval lengths from 100 meters to 2000 by 100-meter increments, i.e., the intervals of 100, 200, 300, . . . , 2000 meters are respectively tested.

2. The aggregate features are calculated for each divided interval of CFMs.

3. The aggregate features are used to calibrate classification models for each aggregation interval length.

4. Choose the aggregation interval, which results in a classification model with the highest classification accuracy. The classification accuracy is measured using “hit rate”, which is the proportion of correct estimates of all the estimates made by the model.

The aggregate features include standard deviation, skewness, mean high-frequency (0.25∼0.5 periods/point) spectra, and mean low-frequency (0.00∼0.25 periods/point) spec- tra. An additional term, mean high-frequency spectra divided by the total spectra, is added as the interaction term of high-frequency spectra and low-frequency spectra. Physically speaking, it is just the proportion of high-frequency spectra in the total variance of the spatial series. Additionally, the mean friction does not enter the model at this stage for the following reasons:

ˆ As previous studies suggested, mean friction is a strong discriminator of RSC types. If it enters the model, sometimes its effect may overwhelmingly mask the discriminating power of other features.

ˆ A main purpose of this analysis on the aggregation interval length selection is to choose an appropriate interval so that the aggregate features can substantially show their discriminating powers.

The RSC types are first divided into two classes–Type (0,1,2) and Type (3,4,5), and a binary logit model and a support vector machine (SVM) with the Gaussian radial basis

function as the kernel function are respectively calibrated with all available aggregated samples for each aggregation length. In other words, 20 logit models and 20 SVMs are calibrated for 20 aggregation interval lengths. Logit models and SVMs will be discussed in detail in the methodology part of this chapter. These models are used to estimate RSC types in terms of the two classes. The hit rates of all these models are shown in Figure 3.15. Another classification scheme is also tested, i.e., RSC types are divided into two classes: Type (0,1,2,3) and Type (4,5), and the hit rates of the new 40 models are shown in Figure 3.16.

Figure 3.15: Model Hit Rates for Different Aggregation Interval Lengths Type (0,1,2) vs. Type (3,4,5)

For both classification schemes, the hit rates of both logit models and SVMs increase with the aggregation interval length. When the aggregation interval length is shorter than 500 meters, the hit rates increase faster, i.e., increasing the aggregation interval length can significantly improve model hit rates. When the aggregation interval length is longer than 500 meters, this improving effect still exists but is not that significant, as suggested by the curves becoming more and more flat in both figures. The most significant changes of the slopes of all these four curves coincidentally occurs within the area where the aggregation interval length is between 500 and 1000 meters. Another observation from the figures is that the hit rates of SVMs are higher than logit models at all aggregate interval lengths with the biggest difference being less than 10% and most less than 5%. The following are some interpretations of these observed patterns.

Figure 3.16: Model Hit Rates for Different Aggregation Interval Lengths Type (0,1,2,3) vs. Type (4,5)

ˆ With the aggregation interval becoming longer, more point-wise CFMs are used to calculate aggregate features, which makes those feature estimations more reliable, thus reflecting the RSC characteristics of the interval in a more accurate and stable way. Consequently, the performance of the resulting models using these features keeps improving with the increase of aggregation interval. Therefore, there is an upward trend for all four curves.

ˆ For both classification schemes and both classifiers, the improvement of the model performance becomes more and more marginal after the aggregation interval reaches 500 meters. This model performance transition zone exists in the aggregation interval length between 500 to 1000 meters.

ˆ SVMs are better classifiers than logit models for all tested aggregation interval lengths, suggesting the RSC classification features have some high dimensional inter- action effects, which are difficult to capture using traditional classifiers; however, the overall performance of SVMs is not significantly better than logit models as their hit rates are not substantially different, suggesting that those high dimensional interac- tion effects are not playing an important role.

From the perspective of model performance, a longer aggregation interval length is preferable; however, as mentioned previously, a longer aggregation interval means a fewer aggregated sample size; and our available dataset does not allow the interval length to be larger than 1000 meters. Furthermore, from the operational perspective of maintenance personnel and decision makers, a shorter aggregation interval is a better choice. Therefore, in our study, 700 meters is chosen as the primary aggregation interval length for extracting aggregate features and calibrating models. Again, this interval length is determined based on a carefully selected trade-off point of sample availability, model performance and the easiness of model application.

Another implication by Figure 3.15 and 3.16 is that although traditional linear classifi- cation models, like logit models, are not as efficient as modern classifiers, such as SVMs, in capturing high dimensional feature patterns, their performance is comparably acceptable, at least for the purpose of this study. Moreover, one major objective of this study is to give explicit interpretations for the resulting models so that they can be applied in the real world with more confidence.