Parameter Estimation in Survival Analysis

This section investigates fitting the two distributions presented in Section 3.4, the exponen-tial and Weibull distributions, to a data set of failure times. Other distributions, such as

Reliability 3-23 the gamma distribution or the exponential power distribution, have analogous methods of parameter estimation. Two sample data sets are introduced and are used throughout this section.

The analysis in this section assumes that a random sample ofn items from a popula-tion has been placed on a test and subjected to typical field operating condipopula-tions. The data values are assumed to be independent and identically distributed random lifetimes from a particular population distribution. As with all statistical inference, care must be taken to ensure that a random sample of lifetimes is collected. Consequently, random num-bers should be used to determine which n items to place on test. Laboratory conditions should adequately mimic field conditions. Only representative items should be placed on test because items manufactured using a previous design may have a different failure pattern than those with the current design.

A data set for which all failure times are known is called a complete data set. Figure 3.10 illustrates a complete data set of n = 5 items placed on test, where the X’s denote fail-ure times. (The term “items placed on test” is used instead of “sample size” or “number of observations” because of potential confusion when right censoring is introduced.) The likelihood function for a complete data set ofn items on test is given by

L(θ) =ⁿ

i=1

f(t_i)

wheret1, t2, . . ., t_nare the failure times andθ is a vector of unknown parameters. (Although lowercase letters are used to denote the failure times here to be consistent with the notation for censoring times, the failure times are nonnegative random variables.)

Censoring occurs frequently in lifetime data because it is often impossible or impractical to observe the lifetimes of all the items on test. A censored observation occurs when only a bound is known on the time of failure. If a data set contains one or more censored observations, it is called a censored data set. The most frequent type of censoring is known as right censoring. In a right-censored data set, one or more items have only a lower bound known on the lifetime. The number of items placed on test is still denoted by n and the number of observed failures is denoted byr.

One special case of right censoring is considered here. Type II or order statistic censoring corresponds to terminating a study upon one of the ordered failures.Figure 3.11shows the case ofn = 5 items are placed on a test that is terminated when r = 3 failures are observed.

The third and fourth items on test had their failure times right censored.

Writing the likelihood function for a censored data set requires some additional notation.

As before, let t1, t2, . . ., t_n be lifetimes sampled randomly from a population. The corre-sponding right-censoring times are denoted byc1, c2, . . ., c_n. The setU contains the indexes

0 t

1

X 2

X

3 X

4

X

5

X

FIGURE 3.10 A complete data set withn=5.

t

of the items that are observed to fail during the test (the uncensored observations):

U = {i|ti≤ ci}

The setC contains the indexes of the items whose failure time exceeds the corresponding censoring time (they are right censored):

C = {i|t_i> c_i}

The reason that the survivor function is the appropriate term in the likelihood function for a right-censored observation is thatS(ci) is the probability that itemi survives to censoring timeci. The log likelihood function is

logL(θ) =

i∈U

logf(x_i) +

i∈C

logS(x_i)

As the density function is the product of the hazard function and the survivor function, the log likelihood function can be simplified to

logL(θ) =

Finally, asH(t) = − log S(t), the log likelihood can be written in terms of the hazard and cumulative hazard functions only as

The choice of which of these three expressions for the log likelihood may be used for a particular distribution depends on the particular forms ofS(t), f(t), h(t), and H(t).

Reliability 3-25 3.5.1 Data Sets

Two lifetime data sets are presented here that are used to illustrate inferential techniques for survivor (reliability) data. The two types of lifetime data sets presented here are a complete data set, where all failure times are observed, and a Type II right-censored data set (order statistic right censoring).

Example 3.11

A complete data set ofn = 23 ball bearing failure times to test the endurance of deep-groove ball bearings has been extensively studied (e.g., Meeker and Escobar, 1998, page 4). The ordered set of failure times measured in 10⁶ revolutions is

17.88 28.92 33.00 41.52 42.12 45.60 48.48 51.84 51.96 54.12 55.56 67.80 68.64 68.64 68.88 84.12 93.12 98.64

105.12 105.84 127.92 128.04 173.40 

Example 3.12

A Type II right-censored data set ofn = 15 automotive a/c switches is given by Kapur and Lamberson (1977, pages 253–254). The test was terminated when the fifth failure occurred.

Ther = 5 ordered observed failure times measured in number of cycles are 1410 1872 3138 4218 6971

Although the choice of “time” as cycles in this case is discrete, the data will be analyzed as

continuous data. 

3.5.2 Exponential Distribution

The exponential distribution is popular due to its tractability for parameter estimation and inference. Using the failure rateλ to parameterize the distribution, recall that the survivor, density, hazard, and cumulative hazard functions are

S(t) = e^−λt f(t) = λe^−λt h(t) = λ H(t) = λt for all t ≥ 0

We begin with the analysis of a complete data set consisting of failure timest1, t2, . . ., t_n. Since all of the observations belong to the index setU, the log likelihood function derived earlier becomes

The maximum likelihood estimator forλ is found by maximizing the log likelihood function.

To determine the maximum likelihood estimator forλ, the single element “score vector”

U(λ) = ∂ log L(λ)

∂λ =n

λ− ⁿ

i=1

t_i

often called the score statistic, is equated to zero, yielding ˆλ = _nn

i=1t_i Example 3.13

Consider the complete data set ofn = 23 ball bearing failure times. For this particular data set, the total time on test is_n

i=1ti= 1661.16, yielding a maximum likelihood estimator ˆλ = _nn

i=1ti

= 23

1661.16 = 0.0138 failure per 10⁶ revolutions.

As the data set is complete, an exact 95% confidence interval for the failure rate of the distribution can be determined. Sinceχ²46,0.975= 29.16 andχ²46,0.025= 66.62, the confidence interval derived in Section 3.4:

Note that, due to the use of the chi-square distribution for this confidence interval, the interval is not symmetric about the maximum likelihood estimator. For this and subsequent examples, care has been taken to perform intermediate calculations involving numeric quan-tities such as critical values or total time on test values to as much precision as possible;

then final values are reported using only significant digits.

Figure 3.12 shows the empirical survivor function, which takes a downward step of

1/n = 1/23 at each data point, along with the survivor function for the fitted exponen-tial distribution. It is apparent from this figure that the exponenexponen-tial distribution is a very poor fit. This particular data set was chosen for this example to illustrate one of the short-comings of using the exponential distribution to model any data set without assessing the adequacy of the fit. Extreme caution must be exercised when using the exponential distri-bution since, as indicated in Figure 3.12, the exponential distridistri-bution might be a poor fit.

The appropriate distribution is probably in the IFR class, as the ball bearings are wearing out. As shown subsequently, the Weibull distribution is a much better approximation to this particular data set. As the exponential distribution can be fitted to any data set that has at least one observed failure, the adequacy of the model must always be assessed. The point and interval estimators associated with the exponential distribution are meaningful only if the data set is a random sample from an exponential population.  The importance of model adequacy assessments, such as those indicated in the previous example, applies to all fitted distributions, not just the exponential distribution. Further-more, if a modeler knows the failure physics (e.g., fatigue crack growth) underlying a process, then an appropriate model consistent with the failure physics should be chosen.

We now turn to the analysis of right-censored data sets drawn from exponential popu-lations. The previous discussion concerning complete data sets is a special case of Type II

Reliability 3-27

0 0.0 0.2 0.4 0.6 0.8 1.0

t

S(t)

150 100

50

FIGURE 3.12 Empirical and exponential fitted survivor functions for the ball bearing data.

censoring whenr = n. As before, assume that the failure times are t1, t2, . . ., tn, the test is terminated upon therth ordered failure, the censoring times are c1=c2=· · · = cn=t(r)for all items, andxi= min{ti, ci} for i = 1, 2, . . ., n.

The log likelihood function is

logL(λ) =

i∈U

logh(x_i)− ⁿ

i=1

H(x_i)

=

i∈U

logλ − n i=1

λxi

=r log λ − λ n i=1

xi

Since there arer observed failures, the expression n i=1

xi

is often called the total time on test as it represents the total accumulated time that the n items accrue while on test. To determine the maximum likelihood estimator, the log likelihood function is differentiated with respect toλ,

U(λ) = ∂ log L(λ)

∂λ = r

λ− ⁿ

i=1

x_i

and is equated to zero, yielding the maximum likelihood estimator ˆλ = r

n i=1x_i

Exact confidence intervals and hypothesis tests concerning λ can also be derived in the Type II censoring case by using the result

2λ ⁿ

i=1

x_i= 2rλ

λˆ ∼ χ²(2r)

where χ²(2r) is the chi-square distribution with 2r degrees of freedom. This result can be proved in an analogous fashion to the case of a complete data set. Using this fact, it can be stated with probability 1− α that

χ²_2r,1−α/2< 2rλ

λˆ < χ²_2r,α/2

Rearranging terms yields an exact 100(1− α)% confidence interval for the failure rate λ:

ˆλχ²_2r,1−α/2

2r < λ < ˆλχ²_2r,α/2 2r Example 3.14

Consider the Type II right-censored data set of automotive switches, wheren = 15 and there arer = 5 observed failures, which are

t(1)= 1410, t(2)= 1872, t(3)= 3138, t(4)= 4218, t(5)= 6971 For this particular data set, the total time on test is _n

i=1x_i= 87,319 cycles, yielding a maximum likelihood estimator

ˆλ =_nr

i=1xi = 5

87,319 = 0.00005726

failure per cycle. Equivalently, the maximum likelihood estimator for the mean of the dis-tribution is

μ =ˆ n

i=1xi

r = 87,319

5 = 17,464

cycles to failure. As the data set is Type II right censored, an exact 95% confidence interval for the failure rate of the distribution can be determined. Using the chi-square critical values, χ²_10,0.975= 3.247 andχ²_10,0.025= 20.49, the formula for the confidence interval

ˆλχ²_2r,1−α/2

2r < λ < ˆλχ²_2r,α/2 2r becomes

(0.00005726)(3.247)

10 < λ < (0.00005726)(20.49) 10

or

0.00001859 < λ < 0.0001173

Taking reciprocals, this is equivalent to a 95% confidence interval for the mean number of cycles to failure of

8525< μ < 53,785

Reliability 3-29

0 0.0 0.2 0.4 0.6 0.8 1.0

t

S(t)

2000 4000 6000

FIGURE 3.13 Empirical and exponential fitted survivor functions for the automotive a/c switch data.

Not surprisingly, with only five observed failures, this is a rather wide confidence interval forμ, and hence there is not as much precision as in the ball bearing example, where there were 23 observed failures. Assessing the adequacy of the fit is more difficult in the case of censoring, as it is impossible to determine what the lifetime distribution looks like after the last observed failure time, which is 6971 cycles in this case. Figure 3.13 shows the empirical survivor function and the associated fitted exponential survivor function. In this case, the exponential distribution appears to adequately model the lifetimes through 6971 cycles.  Situations often arise when it is useful to compare the failure rates of two exponential populations based on data collected from each of the two populations. Examples include comparing the survival of one brand of integrated circuit versus another and comparing the survival of a single item at two levels of an environmental variable. Let the failure rate in the first population beλ1 and the failure rate in the second population beλ2.

As in the previous subsections,x denotes the minimum of the lifetime t and the censoring timec. The two data sets are denoted by

x11, x12, . . ., x1n1

then1 values from the first population, and

x21, x22, . . ., x2n2

then2values from the second population. Thusx_jiis observationi (failure or right-censoring time) from populationj. Assume further that r1> 0 failures are observed in the first popu-lation and thatr2> 0 failures are observed in the second population. For tractability, it is assumed that the tests performed on both populations use Type II censoring. This assump-tion allows exact confidence intervals forλ1/λ2 to be derived. Approximate methods exist when other types of censoring are used. In addition, these methods generalize to the case of comparing more than two populations.

Since 2λ1

_n1

i=1x1i has the chi-square distribution with 2r1 degrees of freedom and 2λ2

_n2

i=1x2i has the chi-square distribution with 2r2degrees of freedom, the statistic 2λ1

has theF distribution with 2r1and 2r2degrees of freedom. This is true because the ratio of two independent chi-square random variables divided by their respective degrees of freedom results in anF random variable. So with probability 1 − α

F2r1,2r2,1−α/2<λ1ˆλ2

Two points are important to keep in mind with respect to this confidence interval. First, it is typically of interest to see whether this confidence interval contains 1, which indicates that there is no statistical evidence to conclude that the failure rates of the two populations are different. Second, if the null hypothesis

H0:λ1=λ2

is to be tested directly, then there is cancellation in λ1λˆ2/λ2ˆλ1 under H0, so that the test statistic ˆλ2/ˆλ1 has theF distribution with 2r1and 2r2degrees of freedom.

3.5.3 Weibull Distribution

As mentioned earlier, the Weibull distribution is typically more appropriate for modeling the lifetimes of items with increasing and decreasing failure rates, such as mechanical items.

We present the most general case of random censoring, rather than looking at each censoring mechanism individually.

As before, let t1, t2, . . ., tn be the failure times, c1, c2, . . ., cn be the censoring times, and xi= min{ti, ci} for i = 1, 2, . . ., n. Recall that the Weibull distribution has hazard and cumu-lative hazard functions

h(t) = κλ(λt)^κ−1 and H(t) = (λt)^κ

fort ≥ 0. When there are r observed failures, the log likelihood function is logL(λ, κ) =

and the 2× 1 score vector has elements U1(λ, κ) = ∂ log L(λ, κ)

∂λ = κr

λ − κλ^{κ−1 n}

i=1

x^κ_i

Reliability 3-31 When these equations are equated to zero, the simultaneous equations

κr

have no closed-form solution for ˆλ and ˆκ. One piece of good fortune, however, to avoid solving a 2× 2 set of nonlinear equations, is that this first equation can be solved for λ in terms ofκ as follows:

Using this expression for λ in terms of κ in the second element of the score vector yields a single, albeit more complicated, expression withκ as the only unknown. Applying some algebra, this equation reduces to

which must be solved iteratively using the Newton–Raphson technique or a fixed point method.

Example 3.15

It was seen in a previous example that the exponential distribution poorly approximated the ball bearing data set. The Weibull distribution is fit to the ball bearing failure times yielding maximum likelihood estimators ˆλ = 0.0122 and ˆκ = 2.10. Figure 3.14 shows the empirical survival function along with the exponential and Weibull fits to the data. It is clear that the Weibull distribution is far superior to the exponential for modeling the ball bearing failure times. This is due to the fact that the Weibull distribution is capable of modeling wear out forκ > 1.

In the case of the exponential distribution, we found a confidence interval for the param-eterλ. In the case of the Weibull distribution, we desire a confidence region for the param-etersλ and κ. Using the fact that the likelihood ratio statistic, 2[log L(ˆλ, ˆκ) − log L(λ, κ)], is asymptoticallyχ²(2), a 95% confidence region for the parameters is allλ and κ satisfying

2[−113.691 − log L(λ, k)] < 5.99

where logL(ˆλ, ˆκ) = −113.691 and χ²_2,0.05= 5.99. The two degrees of freedom for the χ² distribution come from the fact that the Weibull distribution has two unknown parameters λ and κ. The 95% confidence region is shown inFigure 3.15, and, not surprisingly, the line κ = 1 is not interior to the region. This is further proof that the exponential distribution is not an appropriate model for this particular data set. The entire confidence region is in the κ > 1 region of the graph; this is statistically significant evidence provided by the data that

the ball bearings are indeed wearing out. 

0 0.0 0.2 0.4 0.6 0.8 1.0

t

S(t) Weibull

Exponential

150 100

50

FIGURE 3.14 Empirical, exponential fitted, and Weibull fitted survivor functions for the ball bearing data.

0 0.0 0.005 0.010 0.015 0.020

l

k

1 2 3 4

FIGURE 3.15 Confidence region forλandκfor the ball bearing data.

In document Operations Research Applications. a. Ravi Ravindran (Page 100-110)