N-dimensional Likelihood Profiling (NLP) An Efficient Alternative to Bootstrap. Bill Denney PAGE 2012 Venice

N-dimensional Likelihood Profiling (NLP)

An Efficient Alternative to Bootstrap

Bill Denney

Outline

• Motivation: Bootstrapping is hard!

• Solution: N-dimensional Likelihood Profiling

• Faster solution than bootstrapping with easy

convergence

• Examples of application

A day in the life of a quantitative clinical pharmacologist…

A semi-mechanistic, highly nonlinear PK/PD model to assess drug effects.

Need to discuss simulation with uncertainty with the study team... And we need it soon…

…. But, bootstrapping takes forever

How can I simulate with

uncertainty?

?

How can I simulate with uncertainty?

Bootstrap Covariance

Matrix

LLP

Gold standard Quick (if possible) Reliable one

parameter CI Completes slowly

Doesn’t always complete

May not have enough subjects

Sometimes not available

Often not a good estimate of the CI Assumption heavy

Doesn’t work with >1 parameter.

Cannot simulate

…? …? …?

NLP: Filling the Gap

The Idea:

Extend LLP to higher dimensionality allowing to sample the empirical distribution for simulation.

Faster solution than bootstrapping with easy

convergence.

The Problem: Not so simple!

Requires integration of the likelihood

surface in N dimensions.

THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.6 -0.5 -0.4 -0.3 -0.2 THETA2 T H ETA3 -2.7 -2.6 -2.5 -0.45 -0.35 -0.25 THETA2 T H ETA3 -2.7 -2.6 -2.5 -0.45 -0.35 -0.25 THETA2 T H ETA3 -2.70 -2.65 -2.60 -2.55 -0.40 -0.35 -0.30 -0.25 THETA2 T H ETA3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2

How does NLP work?

• From the initial estimates (θ₀)

• Relative likelihood values around θ₀ are searched maximizing information until convergence. Log(EC50) Log(E max )

NLP in one dimension

Compare LLP, Bootstrap, and NLP with one

parameter of a two

compartment PK model.

All provide equal results.

Model evaluations : LLP - 5 Bootstrap - 1000 NLP - 8 -0.80 -0.75 -0.70 -0.65 -0.60 -0.55 -0.50 024 68 1 0 dO F V -0.759 -0.562 -0.661 De n sit y -0.8 -0.7 -0.6 -0.5 02 468 -0.763 -0.564 appr ox im at e. O B J -2427 -2426 -2425 -2424 -2423 -2422 -0.75 -0.70 -0.65 -0.60 -0.55 -0.759 -0.562 -0.661 LLP Bootstrap NLP

Emax EC 5 0 1 2 3 4 6 8 10 12 0 0.451.322.713.84 NLP in multiple dimensions

In a “no regrets dose” study…

What can we infer about Emax/EC50? LLP Bootstrap NLP Emax EC 5 0 0 2 4 6 8 8 10 12 14 Concentration E ffe ct 0 2 4 6 8 10 12 0 5 10 15 20 25 30 Concentration E ffe ct 0 5 10 15 0 5 10 15 Concentration E ffe ct 0 2 4 6 8 10 12 0 5 10 15 20 25 30

A day in the life of a quantitative clinical pharmacologist… made simpler.

NLP

Silber, H. E., Jauslin, P. M., Frey, N. and Karlsson, M. O. Basic & Clinical Pharmacology & Toxicology, 106: 189–194.

Bootstrap did not complete after 10 days on a computational cluster. NLP converged in 1 day on my laptop. THETA13 T H ET A5 0.0024 0.0026 0.0028 0.0030 0.0032

7.0e-05 7.5e-05 8.0e-05 8.5e-05 9.0e-05 9.5e-05

0 0.45 1.32 2.713.84 Insulin Secretion Glucose Production

Conclusions

NLP allows estimation and sampling of the likelihood surface in multiple dimensions (≤5).

Faster solution than bootstrap with easy convergence for

Long parameter estimation times,

Small populations,

Fixed parameters,

Acknowledgements

Gianluca Nucci

Wei Gao

Ted Rieger

THANK YOU!

Theory

Covariance matrix

Assumes that local curvature at the minimum defines full

distribution. LLP

Find the point where OBJ increases by X (3.84 = 95% CI)

through iteration. NLP

Find the (piecewise, continuous) equation that defines the

OBJ surface as a function of model parameters.

Can be used like LLP for one dimension, but allows

sampling.

Provides multivariate empirical distribution.

Bootstrap

Summary

LLP Bootstrap NLP

Can Sample for Simulation

No Yes Yes

CPU Time ~1x N ~1000-2000x N ~5-100x N

Dimensions 1 Full Intermediate

Uncertainty in FIXED

Parameters

Yes No Yes

Start with the initial parameter estimates (θ₀).

Fix the parameters of interest at

values away from θ₀ forming an initial set of simplexes.

For each simplex (i):

While (∫ new_i dθ - ∫ old_i dθ) > tol

Store ∫ old_i dθ = ∫ new_i dθ

Refine points and run model at new points.

–Adding where error was previously

observed; allows parallelization.

Compute ∫ new_i dθ

User-defined parameters are for point selection and ∆ integration tolerance.

T H ET A3 -2.7 -2.6 -2.5 -0.45 -0.35 -0.25 T H ET A3 -2.70 -2.65 -2.60 -2.55 -0.40 -0.35 -0.30 -0.25 THETA2 T H ET A3 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2

Future extensions

Analytical integration of n-dimensional likelihood surface will very significantly decrease estimation time in higher dimensions.

Improved interpolation methods will similarly reduce

integration time and number of iterations required. If estimating full dimensionality of the problem

(all non-fixed θs, ηs, and σs), faster methods

(using maxeval=0) can be used to simply sample the likelihood surface at that point.

Typically will require above improvements in integration