Non-Parametric Method for Confidence Interval Estimation

Due to data availability, traffic parameter statistics (e.g., mean and median) used for traffic system evaluation are generally estimated from samples rather than the whole population. The sample-based estimates, however, might not be exactly equal to the true population parameters, resulting in uncertainties for performance evaluation. Standard error is one indicator of such uncertainty. Naik (2010) applied ordinary bootstrap, block bootstrap, and gap bootstrap to estimate the uncertainty of the travel time prediction model. In this dissertation, the confidence interval of traffic parameter estimates will be used to evaluate the uncertainty of traffic system performance. In this section, various bootstrap methods for interval estimation are reviewed.

2.3.1.1 Standard Error Based Confidence Interval

Assuming that the estimator (𝜃̂) of the true parameter (𝜃) follows a normal

distribution, the (1 − 2𝛼) confidence interval can be approximated as 𝜃̂ ± 𝑧1−𝛼_{∙ 𝑠𝑒̂,}

where 𝜃̂ is the point estimate of 𝜃 and 𝑠𝑒̂ is the estimated standard error. When the

sample size (n) is not large enough to make the assumption of normal distribution hold,

𝜃̂ ± 𝑡_𝑛−11−𝛼_{∙ 𝑠𝑒̂ can generate efficient average estimates. These two methods are named as}

the standard confidence interval and the Student’s t interval. They yield equal-tail intervals that are unable to represent the distribution skewness or other errors when 𝜃̂ represents other statistics (e.g., median) instead of the mean.

To relieve the constraints of normal theory assumption and account for unequal tail, the bootstrap-t interval was proposed to estimate the distribution 𝑡̂ directly from the data instead of making the assumption of normal or t distribution. The resulting interval is in the form of [𝜃̂ − 𝑡̂1−𝛼_{∙ 𝑠𝑒̂, 𝜃̂ − 𝑡̂}𝛼_{∙ 𝑠𝑒̂]. It is important to note that 𝑡̂}1−𝛼 _{is not equal}

to 𝑡̂𝛼 _{in regards to skewness.}

To apply this method, an efficient way to estimate the standard error estimator is necessary for the dataset with dependent structure. It is well-established that the standard

error of the sample mean could be estimated using√𝑠2_{/𝑛, where 𝑠}2 _{= ∑ (𝑥}

𝑖− 𝑥̅)2 𝑛

𝑖=1 /

(𝑛 − 1). However, there is no such equation for most statistical estimators (e.g., median). In these instances, the bootstrap estimate of standard error first proposed by Efron in 1979, can be used. It is illustrated using the statistical estimator median as an example. The basic bootstrap algorithm starts with generating a large number of independent

bootstrap samples: 𝑥∗1_{, 𝑥}∗2_{,… , 𝑥}∗𝐵_{, each of size n. The number of samples (B), generally}

ranges from 50 to 200 for standard deviation estimation. Bootstrap median replicates

𝑠(𝑥∗1_{), s(𝑥}∗2_{),… , 𝑠(𝑥}∗𝐵_{) can be calculated for each sample. The standard deviation of}

these replicates is the standard error estimator of the median 𝑠(𝑥), as shown in equation 2.8. 𝑠𝑒 ̂𝑏𝑜𝑜𝑡 = {∑[𝑠( 𝐵 𝑏=1 𝑥∗𝑏_{) − 𝑠(∙)]}2_{/(𝐵 − 1)}}1/2 _(2.8a) 𝑠(∙) = ∑ 𝑠( 𝐵 𝑏=1 𝑥∗𝑏_{)/𝐵 (2.8b)} where:

𝑠𝑒

̂_{𝑏𝑜𝑜𝑡} = the estimated standard error of median using bootstrap-t method,

B = the size of bootstrap sample, and

𝑠(𝑥∗𝑏_{) = bth bootstrap median replicate.}

Different from the standard intervals which are symmetric around zero, the asymmetric intervals resulting from bootstrap-t percentiles represent an improvement in coverage. It is particularly applicable to location statistics like the sample mean, median, and other percentiles, but is not trustworthy for more general problems such as setting a confidence interval for a correlation coefficient. An overall assessment of the three standard-error based confidence intervals are quoted from Efron and Tibshirani (1993):

“The increase in accuracy of estimation for Bootstrap-t approximation is at the price of generality. The standard confidence interval applies to all samples, and all sample sizes; the student-t table applies to all samples of a fixed size n; the bootstrap-t table applies only to the given sample.”

2.3.1.2 Percentile Based Confidence Interval

Although the bootstrap-t method can theoretically account for skewness and yield good theoretical coverage probabilities, it can yield somewhat erratic results in practice. Improved methods use percentiles instead of the standard error of bootstrapped estimates to identify the confidence limits.

If bootstrap distribution of 𝜃∗ _{= 𝑠(𝑥}∗_{) is roughly normal, then the standard}

normal and percentile intervals will nearly agree. The bootstrap distribution can be regarded as a normal distribution if sample size n approaches infinite, according to the

central limit theorem. However, this might not hold for small samples in which case the percentile interval is superior to the standard normal interval. Also, a percentile interval has transformation-respecting and range-preserving property. By range-preserving property, a percentile interval always falls within the allowable range of its estimator. Although percentile intervals are less erratic in practice compared to bootstrap-t intervals, they have less satisfactory coverage properties.

Given independent bootstrapped samples 𝑥∗1_{, 𝑥}∗2_{,… , 𝑥}∗𝐵_{, each of size n,}

bootstrap replicates 𝜃̂∗_{(𝑏)= 𝑠(𝑥}∗𝑏_{), 𝑏 = 1, 2, … , 𝐵. Denote 𝜃̂}

𝐵∗(𝛼) as the 100αth empirical

percentile (i.e., the value in the ordered list of the B replications of 𝜃̂_𝐵∗_{). The (1 − 2𝛼)}

percentile interval would be [𝜃̂_𝐵∗(𝛼), 𝜃̂_𝐵∗(1−𝛼)]. It needs more bootstrap samples (B) for

percentile estimation than for standard error estimation. Variable B should be greater than 500 or 1000 to make the variability of percentile estimators acceptably low.

2.3.1.3 Bias-Corrected and Accelerated (BCa) Interval

The BCa interval is an improved version of the percentile method in both theory and practice. Given enough sample size, the resulting interval would closely match exact confidence intervals in special situations, where the statistically exact interval is

accessible through statistical theory, and give dependably accurate coverage probabilities in all situations. In addition, the BCa method is also transformation-respecting.

Integrating the performance on accuracy and flexibility, the BCa method is recommended for general use by Efron and Tibshirani (1993).

The end points of the BCa interval is modified by acceleration (𝑎̂) and bias-

The notation Φ(∙) is the standard, normal cumulative distribution function and 𝑧(𝛼) is the 100αth percentile point of a standard normal distribution. For example, Φ(1.645) = 0.95

and 𝑧(0.95) = 1.645. It can be assessed from equation 2.9 that if 𝑎̂ and 𝑧̂0 are zero, the

interval is equal to the percentile interval. Non-zero 𝑎̂ and 𝑧̂₀ correct deficiencies of the

previous standard and percentile methods. BCa: (𝜃̂_𝐵∗(𝛼1)_{, 𝜃̂} 𝐵∗(𝛼2)) (2.9a) 𝛼₁ = Φ(𝑧̂₀+ 𝑧̂0+ 𝑧(𝛼) 1 − 𝑎̂(𝑧̂₀+ 𝑧(𝛼)₎) (2.9b) 𝛼₂ = 𝛷(𝑧̂₀+ 𝑧̂0+ 𝑧(1−𝛼) 1 − 𝑎̂(𝑧̂₀+ 𝑧(1−𝛼)₎) (2.9c) where: 𝑎̂ = acceleration, 𝑧̂0 = bias-correction,

𝑧(𝛼) _{= the 100α th percentile point of a standard normal distribution.}

2.3.1.4 Modified Bootstrap

When the dataset is not composed of independent observations, the standard bootstrap method is not enough to get independent bootstrap samples, and modified bootstrap (e.g., block bootstrap) is needed. Specifically for the traffic dataset with dependent observations within one day, Lahiri et al. (2012) applied gap bootstrap to generate consistent and asymptotically unbiased estimates of standard error for a massive dataset with certain dependent structure.

2.3.1.5 Summary

This dissertation compares the coverage of the standard error based confidence interval and the BCa confidence interval. The BCa method is selected to calculate the confidence interval of individual traffic parameters.