Crash and injury severity model comparisons

Several methods of assessing the validity of a model have been well discussed by the literature. This summary of previous works involving injury severity analysis is intended as a guide of the accepted methods to compare different statistical models. In general, logistic regression analysis can be assessed using several measures of performance. For comparing the reliability models that use different data sets, calibration and discrimination measures have been used. More on these measures is discussed in Chapter 3. Several studies have dealt with these comparisons, some of which are presented next.

In the study be O’Donnell and Connor (1996), the two models (ordered probit and logit) exhibited similar goodness of fit (Veal-Zimmerman Pesudo-R²). The coefficients of the ordered probit were consistently lower than their logit counterparts. The asymptotic t-ratios suggested that the standard errors were lower for the ordered logit model, but this could not be verified.

The coefficient signs of the two models agreed, except for the effects of time of crash, which were not found to be significant in the ordered logit model. In summary, none of the models showed a significant advantage over the other.

In a study using a sample of 43,913 crashes reported in Ontario, Canada during 1986 the investigators assessed the reliability of different crash severity models (Saccomanno, et al., 1996). The criteria utilized included goodness of fit, robustness of risk factor coefficients, and

whether the resulting coefficients where acceptable and consistent with previous research and scientific principles. In this study, three model structures were tested. Models 1a and 1b were disaggregating sequential binary logit models (five injury severity levels, four injury severity expressions). Two sequencing options (a, b) were developed: from No Injury to Fatal Injury and vice versa. Model 2 was a disaggregate two-stage binary logit model, where two injury severities were considered at each stage. In stage 1, injury severities were classified as severe and non-severe. In stage 2, the severe cases were further classified into Fatal and Major Injury; while the non-severe cases were split into minor and minimal injuries and no injury. Model 3 was an aggregate binary logit model with only two severity levels: severe (fatal and major injury) and non-severe (minor, minimal, and no injury). The model comparison of statistical goodness of fit at the injury expression level employed a similarity index (to measure predictive reliability of each injury severity in each model) and the expected percent correct (case by case using Monte Carlo statistical estimating techniques). At the overall model level, two success index indicators that measured correct case-specific classifications in each severity model for each injury level as well as the whole model (all the injury expressions treated together). Also, the Predicted Less Observed Injury Severity Share was used for the overall model only. Each model was finally compared in terms of the statistical significance of the injury expression coefficients (t-tests), and whether the results were scientifically acceptable. The results of the analysis suggested that model reliability is not sensitive to the number of injury classes specified in the model or to the level of model aggregation. The most important factors explaining most of the variation in injury severity were the dynamics of the crash, seating position of the occupant, use of seat belts, and age of occupant involved. The accuracy of the information provided in the crash reports was the primary determinant of model reliability according to the authors.

Krull et al. (2000) used logit models to analyze injury severity for drivers involved in a single-vehicle crash. Three-year crash and road inventory data from Michigan (1994-1996, N=35,447) and Illinois (1993-1995, N=24,296) were collected from the Highway Safety Information System (HSIS) maintained by FHWA. The KABC0 injury scale is used in both jurisdictions and categories K (fatal) and A (incapacitating injury) were grouped together to represent severe injuries. Three single vehicle crash models were developed from the Illinois, Michigan and pooled data. A total of 16 driver, vehicle and environmental variables were included in the first regression analysis. However ADT was excluded due to a high correlation (0.533) with rural functional class, as shown by the correlation matrix. Likewise, the right shoulder width and left shoulder width were highly correlated and excluded as well. The significant factors found to increase injury severity for the three models were: alcohol involvement, daylight, driver age, rural functional class, speed limit and rollover crash; while restrain use, slick roadway and heavier vehicle types had a decreasing effect on severe injury.

Only vehicle type showed non-significant coefficients for the Michigan and pooled models.

When comparing the goodness of fit of these models, the pooled model performed better on the Likelihood Ratio Test statistic (not a formal goodness of fit measure), also the pooled model performed a little better on the R-square measure, which can be used to compare models using different data sets. Missing data-dummy variables for driver age and restraint use were found to be significant at the 0.10 level, suggesting a systematic reason for the missing variables. A similar effort was undertaken as part of the final analysis presented in later chapters of this thesis.

In a study focused on driver characteristics (Dissanayake and Lu, 2002), two sets of sequential binary logistic regression models were developed to describe the injury severity

relationship of older drivers involved in fixed object-passenger car crashes in Florida between 1994 and 1996. The dependent variable in one set of models was driver injury severity, while it was the crash injury severity for the other set. For each of the sets of models, crash or injury severity was varied from the least severe (no injury) to the most severe (fatality) and vice versa.

The injury severity models were found to have better fit and predictive accuracy. The fit was compared using the rank correlation measures and the predictive accuracy was computed as the ratio of the true positives and true negatives to the total cases with a 0.5 cut point. The percent accuracy is equivalent to the percent concordant in a binary logit model.

A study by Abdel-Aty (2003) using data from three counties in Central Florida developed three driver injury severity models for different road entities. For roadway sections, crash data from 1996-1997 (17,647 drivers involved in 7,891 crashes) was used. For signalized intersections the same crash data years were used with 2,336 drivers involved in 1,168 crashes.

Meanwhile only the 1999-2000 police reports were available for toll plaza crashes for a total of 447 crashes and 803 involved drivers (725 with complete information). Different modeling methods were tested: multinomial logit, nested logit, and ordered probit. The goodness of fit measures likelihood ratio index and classification accuracy for each model were compared. The nested logit was the best model, while the ordered probit performed very well with considerable less data and modeling efforts. After testing several combinations, four categories of driver injury severity levels were found to produce the best models: no injury, possible injury, evident injury, and severe/fatal injury. The factors related to driver’s age, gender, seat belt use, point of impact, speed ratio and vehicle type were found significant in all models. Driver at fault, land use and light-weather interaction were significant in the signalized intersection. Alcohol-seat belt interaction, lighting conditions, and the existence of a horizontal curve were found significant in

the roadway section model. The model for toll plazas included weather condition, number of impacts, E-pass lane, alcohol-seat belt, passenger car-speed ratio and two additional E-pass interactions.

In a study of crash severity levels at signalized intersections, Abdel-Aty and Keller (2005) explored the differences between ordered probit models using complete datasets and restricted datasets. The complete data included short forms (minor crashes) and long forms (crashes available in the CAR and FDHSMV crash databases). The restricted dataset included only the long form crashes. Crash data from four counties in Central Florida during the years 2000-2001 was used to develop five models: 7,833 crashes reported on long forms (restricted dataset) and 21,204 crashes in the complete dataset (including short forms). The first two models analyzed the restricted and the complete dataset crash severity with only crash type and county indicator as independent variables. The next two models used the same datasets, but with intersection characteristics as their independent variables. In both cases, the models with complete datasets fared much better in classification accuracy. Also, right turn crashes were significant in the complete dataset model and not in the restricted model. Meanwhile, most gains in variable information were achieved in the intersection characteristics complete dataset model (major road no. of lanes, left and right turn lanes, division on minor road, and ADT on major road). It was decided to use the complete dataset for the final model with a combination of independent variables of the previous models. This final model achieved a high level of classification accuracy (79.1%), but lost all but two (median and speed limit on minor road) intersection characteristic variables. The combined model also lost the right turn and sideswipe crash types (which are less severe). In this study, goodness of fit and variable information was used for comparison and demonstrated the usefulness of the complete dataset for the less severe

crash types. It also showed a tradeoff between the amount of significant factors (especially road related) and the overall risk assessment provided by the combined variable model.

A summary of the methods used to compare the statistical models discussed above is shown in Table 2-2 below. The coefficient signs are always checked for agreement with previous studies and scientific principles. Coefficient robustness and classification accuracy are also important in comparing models. This is not an exhaustive list, but the fundamental issues in comparing models have been adequately covered in this section.

Table 2-2 Summary of goodness of fit comparison methods from past studies Regression Unit of analysis Occupant

involvement Crash

Finally, there are other studies that include additional goodness of fit statistics for the logistic regression model. Valenti et al. (2002) did not discuss model comparison, but provided the necessary goodness of fit data for the different logistic models. It was found that although all models have acceptable fit, the overall driver models had less favorable calibration (Hosmer-Lemeshow statistic) when compared to the models that only considered one group of drivers or

pedestrians. The authors indicated that the study had several limitations, among them not having separate analyses for each crash type. In another logistic regression analysis (Sze and Wong, 2007) the model goodness of fit was verified using the Hosmer-Lemeshow test and logistic regression graphical diagnosis, including leverage and residuals.