• No results found

Comparison of four methods of computing confidence intervals for a proportion

N/A
N/A
Protected

Academic year: 2021

Share "Comparison of four methods of computing confidence intervals for a proportion"

Copied!
19
0
0

Loading.... (view fulltext now)

Full text

(1)

Comparison of four methods of computing confidence

intervals for a proportion

Gilles Guillot

June 28, 2017, 12:25:11

Contents

Summary 1

Results 2

Comparing confidence interval bounds returned by the various methods 6

Appendix: R code 17

Summary

We compare four methods of computing confidence intervals for a proportion. These four methods are • the Normal approximation method,

• the Wilson score method,

• the Wislon score method with continuity correction • the Clopper-Pearson method.

They are described on the Wikipedia pageBinomial proportion confidence interval.

We estimate the coverage for the various methods listed above for an intended nominal coverage ofα= 0.05 for various sample sizesnand various probabilitiesp. We do so with Monte Carlo simulations with 10ˆ{6} replicates per combination of parameters (n, p). The Normal approximation fails to achieve the nominal coverage for small sample size, more so when the truepis close to 0 (or 1). The three other methods are more accurate. We note that the Wilson score method with continuity correction and the Clopper-Pearson method tend to be a bit conservative for smallnorp.

(2)

Results

0

200

400

600

800

1000

0.0

0.2

0.4

0.6

0.8

1.0

Normal approximation

Sample size

Estimated co

v

er

age

p= 0.001

p= 0.01

p= 0.02

p= 0.05

p= 0.1

p= 0.2

(3)

0

200

400

600

800

1000

0.0

0.2

0.4

0.6

0.8

1.0

Wilson score method (no correction)

Sample size

Estimated co

v

er

age

p= 0.001

p= 0.01

p= 0.02

p= 0.05

p= 0.1

p= 0.2

(4)

0

200

400

600

800

1000

0.0

0.2

0.4

0.6

0.8

1.0

Wilson score with continuity correction

Sample size

Estimated co

v

er

age

p= 0.001

p= 0.01

p= 0.02

p= 0.05

p= 0.1

p= 0.2

(5)

0

200

400

600

800

1000

0.0

0.2

0.4

0.6

0.8

1.0

Clopper−Pearson method

Sample size

Estimated co

v

er

age

p= 0.001

p= 0.01

p= 0.02

p= 0.05

p= 0.1

p= 0.2

(6)

Comparing confidence interval bounds returned by the various

methods

Normal

Sample size= 10 , p= 0.001

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0.5 0 30000

Wilson

Width CI w Frequency 0.30 0.35 0.40 0.45 0 30000

Wilson cc

Width CI w Frequency 0.30 0.35 0.40 0.45 0 30000

CP

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 30000

(7)

Normal

Sample size= 10 , p= 0.01

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0 30000

Wilson

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 30000

Wilson cc

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 30000

CP

Width CI w Frequency 0.30 0.40 0.50 0.60 0 30000

Normal

Sample size= 10 , p= 0.05

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0 20000

Wilson

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 20000

Wilson cc

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 20000

CP

Width CI w Frequency 0.30 0.40 0.50 0.60 0 20000

(8)

Normal

Sample size= 10 , p= 0.1

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0 15000

Wilson

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 15000

Wilson cc

Width CI w Frequency 0.30 0.35 0.40 0.45 0.50 0 15000

CP

Width CI w Frequency 0.30 0.40 0.50 0.60 0 15000

Normal

Sample size= 20 , p= 0.001

Width CI w Frequency 0.00 0.10 0.20 0 30000

Wilson

Width CI w Frequency 0.16 0.20 0.24 0 30000

Wilson cc

Width CI w Frequency 0.16 0.20 0.24 0 30000

CP

Width CI w Frequency 0.16 0.20 0.24 0.28 0 30000

(9)

Normal

Sample size= 20 , p= 0.01

Width CI w Frequency 0.00 0.10 0.20 0.30 0 30000

Wilson

Width CI w Frequency 0.20 0.25 0.30 0 30000

Wilson cc

Width CI w Frequency 0.20 0.25 0.30 0 30000

CP

Width CI w Frequency 0.20 0.25 0.30 0.35 0 30000

Normal

Sample size= 20 , p= 0.05

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0 10000

Wilson

Width CI w Frequency 0.20 0.25 0.30 0.35 0 10000

Wilson cc

Width CI w Frequency 0.20 0.25 0.30 0.35 0 10000

CP

Width CI w Frequency 0.15 0.25 0.35 0 10000

(10)

Normal

Sample size= 20 , p= 0.1

Width CI w Frequency 0.0 0.1 0.2 0.3 0.4 0 10000

Wilson

Width CI w Frequency 0.20 0.25 0.30 0.35 0.40 0 10000

Wilson cc

Width CI w Frequency 0.20 0.25 0.30 0.35 0.40 0 10000

CP

Width CI w Frequency 0.15 0.25 0.35 0.45 0 10000

Normal

Sample size= 50 , p= 0.001

Width CI w Frequency 0.00 0.04 0.08 0 30000

Wilson

Width CI w Frequency 0.07 0.09 0.11 0 30000

Wilson cc

Width CI w Frequency 0.07 0.09 0.11 0 30000

CP

Width CI w Frequency 0.07 0.09 0.11 0.13 0 30000

(11)

Normal

Sample size= 50 , p= 0.01

Width CI w Frequency 0.00 0.05 0.10 0.15 0.20 0 20000

Wilson

Width CI w Frequency 0.08 0.12 0.16 0.20 0 20000

Wilson cc

Width CI w Frequency 0.08 0.12 0.16 0.20 0 20000

CP

Width CI w Frequency 0.08 0.12 0.16 0.20 0 20000

Normal

Sample size= 50 , p= 0.05

Width CI w Frequency 0.00 0.05 0.10 0.15 0.20 0 8000

Wilson

Width CI w Frequency 0.10 0.15 0.20 0 8000

Wilson cc

Width CI w Frequency 0.10 0.15 0.20 0 8000

CP

Width CI w Frequency 0.10 0.15 0.20 0.25 0 8000

(12)

Normal

Sample size= 50 , p= 0.1

Width CI w Frequency 0.00 0.05 0.10 0.15 0.20 0.25 0 6000

Wilson

Width CI w Frequency 0.10 0.15 0.20 0.25 0 6000

Wilson cc

Width CI w Frequency 0.10 0.15 0.20 0.25 0 6000

CP

Width CI w Frequency 0.10 0.15 0.20 0.25 0 6000

Normal

Sample size= 200 , p= 0.001

Width CI w Frequency 0.00 0.01 0.02 0.03 0.04 0 30000

Wilson

Width CI w Frequency 0.020 0.030 0.040 0 30000

Wilson cc

Width CI w Frequency 0.020 0.030 0.040 0 30000

CP

Width CI w Frequency 0.020 0.030 0.040 0 30000

(13)

Normal

Sample size= 200 , p= 0.01

Width CI w Frequency 0.00 0.02 0.04 0.06 0 8000

Wilson

Width CI w Frequency 0.02 0.03 0.04 0.05 0.06 0 8000

Wilson cc

Width CI w Frequency 0.02 0.03 0.04 0.05 0.06 0 8000

CP

Width CI w Frequency 0.02 0.03 0.04 0.05 0.06 0.07 0 8000

Normal

Sample size= 200 , p= 0.05

Width CI w Frequency 0.00 0.02 0.04 0.06 0.08 0 6000

Wilson

Width CI w Frequency 0.02 0.04 0.06 0.08 0 3000

Wilson cc

Width CI w Frequency 0.02 0.04 0.06 0.08 0 3000

CP

Width CI w Frequency 0.02 0.04 0.06 0.08 0.10 0 3000

(14)

Normal

Sample size= 200 , p= 0.1

Width CI w Frequency 0.04 0.06 0.08 0.10 0 3000

Wilson

Width CI w Frequency 0.05 0.07 0.09 0.11 0 6000

Wilson cc

Width CI w Frequency 0.05 0.07 0.09 0.11 0 6000

CP

Width CI w Frequency 0.05 0.07 0.09 0.11 0 6000

Normal

Sample size= 1000 , p= 0.001

Width CI w Frequency 0.000 0.004 0.008 0 10000

Wilson

Width CI w Frequency 0.004 0.006 0.008 0.010 0 10000

Wilson cc

Width CI w Frequency 0.004 0.006 0.008 0.010 0 10000

CP

Width CI w Frequency 0.004 0.006 0.008 0.010 0 10000

(15)

Normal

Sample size= 1000 , p= 0.01

Width CI w Frequency 0.005 0.010 0.015 0.020 0 3000

Wilson

Width CI w Frequency 0.005 0.010 0.015 0.020 0 3000

Wilson cc

Width CI w Frequency 0.005 0.010 0.015 0.020 0 3000

CP

Width CI w Frequency 0.005 0.010 0.015 0.020 0 3000

Normal

Sample size= 1000 , p= 0.05

Width CI w Frequency 0.020 0.025 0.030 0.035 0 3000

Wilson

Width CI w Frequency 0.020 0.025 0.030 0.035 0 3000

Wilson cc

Width CI w Frequency 0.020 0.025 0.030 0.035 0 3000

CP

Width CI w Frequency 0.020 0.025 0.030 0.035 0 3000

(16)

Normal

Sample size= 1000 , p= 0.1

Width CI w Frequency 0.030 0.034 0.038 0.042 0 3000

Wilson

Width CI w Frequency 0.030 0.034 0.038 0.042 0 3000

Wilson cc

Width CI w Frequency 0.030 0.034 0.038 0.042 0 3000

CP

Width CI w Frequency 0.032 0.036 0.040 0.044 0 4000

(17)

Appendix: R code

## en.wikipedia.org/wiki/Binomial_proportion_confidence_interval

## functions to compute CI for a proportion CI_prop = function(x ,

alpha = 0.05, # 1-(nominal coverage)

method=c("normal","Wilson","Wilson_cc","CP") )

{

CI_normal = CI_Wilson = CI_Wilson_cc = CI_CP = NULL

p.hat = x/size

if("normal" %in% method) {

z = qnorm(p=1-alpha/2)

delta = z* sqrt(p.hat*(1-p.hat)/size) CI_normal = cbind(p.hat-delta,p.hat+delta) }

if("Wilson" %in% method) {

z = qnorm(p=1-alpha/2)

delta = z* sqrt(p.hat*(1-p.hat)/size + z^2/(4*size^2))

CI_Wilson = cbind((p.hat + z^2/(2*size) - delta)/(1+z^2/size), (p.hat + z^2/(2*size) + delta)/(1+z^2/size)) }

if("Wilson_cc" %in% method) {

z = qnorm(p=1-alpha/2)

delta = z* sqrt(p.hat*(1-p.hat)/size + z^2/(4*size^2))

CI_Wilson_cc = cbind((p.hat + z^2/(2*size) - delta)/(1+z^2/size), ((p.hat + z^2/(2*size) + delta)/(1+z^2/size))) }

if("CP" %in% method) {

lb = qbeta(p=alpha/2,shape1=x , shape2 = size-x+1)

ub = qbeta(p = 1-alpha/2, shape1 = x+1 , shape2 = size-x) CI_CP = cbind(lb,ub)

}

return(list(CI_normal=CI_normal,

CI_Wilson=CI_Wilson,

CI_Wilson_cc=CI_Wilson_cc,

(18)

## function to estimate coverage of CI by Monte Carlo simulation ## Normal approximation method

coverage_normal = function(size, #sample size

prob, # actual proportion

rep, # number of Monte Carlo replicates alpha = 0.05 # 1-(nominal coverage)

) {

X = rbinom(size=size,prob=prob,n=rep) p.hat = X/size

z = qnorm(p=1-alpha/2)

delta = z* sqrt(p.hat*(1-p.hat)/size)

mean((prob > p.hat-delta) &

(prob < p.hat+delta )) }

## function to estimate coverage of CI by Monte Carlo simulation ## Wilson score method

coverage_Wilson = function(size, #sample size

prob, # actual proportion

rep, # number of Monte Carlo replicates alpha = 0.05 # 1-(nominal coverage)

) {

X = rbinom(size=size,prob=prob,n=rep) p.hat = X/size

z = qnorm(p=1-alpha/2)

delta = z* sqrt(p.hat*(1-p.hat)/size + z^2/(4*size^2))

mean((prob > (p.hat + z^2/(2*size) - delta)/(1+z^2/size)) & (prob < (p.hat + z^2/(2*size) + delta)/(1+z^2/size))) }

## function to estimate coverage of CI by Monte Carlo simulation ## Wilson score with continuity correction method

coverage_Wilson_cc = function(size, #sample size

prob, # actual proportion

rep, # number of Monte Carlo replicates alpha = 0.05 # 1-(nominal coverage)

(19)

{

X = rbinom(size=size,prob=prob,n=rep) p.hat = X/size

z = qnorm(p=1-alpha/2)

delta = z*sqrt(z^2 - 1/size + 4*size*p.hat*(1-p.hat) + (4*p.hat-2)) + 1

lb = (2*size*p.hat+z^2 - delta) / (2*(size+z^2)) subs = lb < 0

lb[lb < subs] = 0

delta = z*sqrt(z^2 - 1/size + 4*size*p.hat*(1-p.hat) - (4*p.hat-2)) + 1

ub = (2*size*p.hat+z^2 + delta) / (2*(size+z^2)) subs = ub > 1

ub[subs] = 1

mean((prob > lb) & (prob < ub)) }

## function to estimate coverage of CI by Monte Carlo simulation ## Clopper-Pearson method

coverage_CP = function(size, #sample size

prob, # actual proportion

rep, # number of Monte Carlo replicates alpha = 0.05 # 1-(nominal coverage)

) {

X = rbinom(size=size,prob=prob,n=rep)

# F = qf(p=alpha/2 , df1=2*X , df2=2*(size-X+1)) # lb = 1/(1+ (size-X+1) / (X*F))

# F = qf(p=1-alpha/2 , df1=2*(X+1), df2 = 2*(size-X) ) # ub = 1/(1+ (size-X) / ((X+1)*F) )

lb = qbeta(p=alpha/2,shape1=X , shape2 = size-X+1)

## lb2 = 1 - qbeta(p=1-alpha/2,shape2=X , shape1 = size-X+1) # just checking SAS formula p.7 OK ub = qbeta(p = 1-alpha/2, shape1 = X+1 , shape2 = size-X)

## ub2 = 1- qbeta(p = alpha/2, shape2 = X+1 , shape1 = size-X) # just checking SAS formula p.7 OK

mean((prob > lb) & (prob < ub))

References

Related documents

In this paper, the sinc collocation method is proposed for solving linear and nonlinear multi-order fractional differential equations based on the new definition of

In comparing the calculation-to-calculation reaction rates using the BUGLE-93 cross-section library at the thermal shield, pressure vessel, and cavity capsules, for eleven

IJEDR1604115 International Journal of Engineering Development and Research ( www.ijedr.org ) 755 The offloading decision mainly depends on the local computing resources and

This proposition is that if a change in the quantity of money were exogenously engineered by the monetary authority, then the long-run effect would be a change in the price level

Radio modem, cellular, and also satellite communication with an automatic channels choice of communication in the conditions of the powerful electromagnetic fields having

One note that, for the assessment point associated with a maximum service pressure of 70 bar or a circumferential stress of 125 MPa, the failure probability is less than 10 -6

Among other changes, all plans offered in the individual and small group markets must cover a minimum set of essential health benefits (EHBs) and limit each member’s overall