Recall thechurn data set [4], where we were interested in predicting whether a customer would leave the company’s service (churn) based on a set of predictor variables. For this simple logistic regression example, assume that the only predictor available isVoiceMail Plan, a flag variable indicating membership in the plan. The cross-tabulation ofchurnby VoiceMail Plan membership is shown in Table 4.3.
The likelihood function is then given by
l(β|x)=[π(0)]403×[1−π(0)]2008×[π(1)]80×[1−π(1)]842
Note that we may use the entries from Table 4.3 to construct the odds and the odds ratio directly.
r Odds of those with VoiceMail Plan churning=π(1)[1−π(1)]=80/842= 0.0950
r Odds of those without VoiceMail Plan churning =π(0)[1−π(0)]= 403/2008=0.2007 and OR= π(1) [1−π(1)] π(0)[1−π(0)] = 80842 4032008 =0.47
That is, those who have the VoiceMail Plan are only 47% as likely to churn as are those without the VoiceMail Plan. Note that the odds ratio can also be calculated as
TABLE 4.3 Cross-Tabulation ofChurnby Membership in the VoiceMail Plan
VoiceMail=No, VoiceMail=Yes,
x=0 x=1 Total Churn=false, 2008 842 2850 y=0 Churn=true, 403 80 483 y=1 Total 2411 922 3333
SPH SPH
JWDD006-04 JWDD006-Larose November 23, 2005 14:51 Char Count= 0
164 CHAPTER 4 LOGISTIC REGRESSION
TABLE 4.4 Results of Logistic Regression of Churn on the VoiceMail Plan
Logistic Regression Table
Odds 95% CI
Predictor Coef SE Coef Z P Ratio Lower Upper
Constant -1.60596 0.0545839 -29.42 0.000
VMail -0.747795 0.129101 -5.79 0.000 0.47 0.37 0.61
Log-Likelihood = -1360.165
Test that all slopes are zero: G = 37.964, DF = 1, P-Value = 0.000
the following cross-product:
OR= π(1) [1−π(0)] π(0) [1−π(1)] =
80(2008) 403(842) =0.47
The logistic regression can then be performed in Minitab, with the results shown in Table 4.4. First, note that the odds ratio reported by Minitab equals 0.47, the same value that we found using the cell counts directly. Next, equation (4.3) tells us that odds ratio=eβ1. We verify this by noting thatb
1 = −0.747795, so thateb1 =0.47. Here we have b0= −1.60596 and b1= −0.747795. So the probability of churning for a customer belonging (x=1) or not belonging (x=0) to the VoiceMail Plan is estimated as ˆ π(x)= e ˆ g(x) 1+eg(x)ˆ = e−1.60596+ −0.747795x 1+e−1.60596+ −0.747795x with the estimated logit
ˆ
g(x)= −1.60596−0.747795x
For a customer belonging to the plan, we estimate his or her probability of churning:
ˆ
g(x)= −1.60596−(0.747795)(1)= −2.3538
INTERPRETING A LOGISTIC REGRESSION MODEL 165 and ˆ π(x)= e ˆ g(x) 1+eg(x)ˆ = e−2.3538 1+e−2.3538 =0.0868
So the estimated probability that a customer who belongs to the VoiceMail Plan will churn is only 8.68%, which is less than the overall proportion of churners in the data set, 14.5%, indicating that belonging to the VoiceMail Plan protects against churn. Also, this probability could have been found directly from Table 4.3:
P(churn|VoiceMail Plan)= 80
922 =0.0868
For a customer not belonging to the VoiceMail Plan, we estimate the probability of churning: ˆ g(x)= −1.60596−(0.747795)(0)= −1.60596 and ˆ π(x)= e ˆ g(x) 1+eg(x)ˆ = e−1.60596 1+e−1.60596 =0.16715
This probability is slightly higher than the overall proportion of churners in the data set, 14.5%, indicating that not belonging to the VoiceMail Plan may be slightly indicative of churning. This probability could also have been found directly from Table 4.3:
P(churn|VoiceMail Plan)= 403
2411 =0.16715
Next, we apply the Wald test for the significance of the parameter for the VoiceMail Plan. We haveb1= −0.747795 and SE(b1)=0.129101, giving us
ZWald=
−0.747795
0.129101 = −5.79
as reported underzfor the coefficientVoiceMail Planin Table 4.4. The p-value is P(|z|>5.79)∼=0.000, which is strongly significant. There is strong evidence that theVoiceMail Planvariable is useful for predicting churn.
A 100(1−α)% confidence interval for the odds ratio may be found thus: exp b1±z·
∧
SE(b1)
where exp[a] representsea. Thus, here we have a 95% confidence interval for the odds ratio given by
exp b1±z· ∧ SE(b1) =exp [−0.747795±(1.96)(0.129101)] =(e−1.0008, e−0.4948) =(0.37, 0.61)
as reported in Table 4.4. Thus, we are 95% confident that the odds ratio for churn- ing among VoiceMail Plan members and nonmembers lies between 0.37 and 0.61. Since the interval does not includee0=1, the relationship is significant with 95% confidence.
We can use the cell entries to estimate the standard error of the coefficients directly, as follows (result from Bishop et al. [5]). The standard error for the logistic
SPH SPH
JWDD006-04 JWDD006-Larose November 23, 2005 14:51 Char Count= 0
166 CHAPTER 4 LOGISTIC REGRESSION
TABLE 4.5 Reference Cell Encoding for theCustomer
Service CallsIndicator Variables
CSC-Med CSC-Hi
Low (0 or 1 calls) 0 0 Medium (2 or 3 calls) 1 0 High (≥4 calls) 0 1 regression coefficientb1forVoiceMail Planis estimated as follows:
∧ SE(b1)= 1 403+ 1 2008+ 1 80+ 1 842 =0.129101
In thischurnexample, the voice mail members were coded as 1 and the nonmembers coded as 0. This is an example ofreference cell coding, where the reference cell refers to the category coded as zero. Odds ratios are then calculated as the comparison of the membersrelative tothe nonmembers or with reference to the nonmembers.
In general, for variables coded asaandbrather than 0 and 1, we have ln [OR(a,b)]=gˆ(x=a)−gˆ(x=b)
=(b0+b1a)−(b0+b1b)
=b1(a−b) (4.4)
So an estimate of the odds ratio in this case is given by exp [b1(a−b)] which becomes eb1whena=1 andb=0.