Efficient Computation with Taste Shocks
Grey Gordon FRB of Richmond
EEA/ESEM Manchester, UK August 28, 2019
Discrete choices result in discontinuities.
E.g., think about a savings policy functionb0(b;r) with
• r an interest rate,
• b ∈ B the current bond holding of an agent,
• b0 ∈ B the bond choice, and
• B finite.
Equilibrium may requireB(r;µ) :=Rb0(b;r)dµ(b) = 0.
Asr moves,b0 either stays the same or jumps.
Theoretically, fine: Can exploit indifference, upper hemicontinuity.
Motivation
An alternative is to addtaste shocks~.
Instead of maxb0∈BU(b,b0;r), use maxb0∈BU(b,b0;r) +σb0
Replacesb0(b;r) with P(b0|b;r).
Forb0 ∼Type I extreme value, the choice probabilities are
P(b0|b;r) = exp(U(b,b
0;r)/σ)
P#B
1 exp(U(b,·;r)/σ)
Big advantage:
B(r;µ) =
Z Z
b0(b, ~;r)f(~)d~dµ(b) =
Z X
b0
P(b0|b;r)b0dµ(b)
That’s great, but it creates a problem:
P(b0|b) = exp(U(b,b
0)/σ)
P#B
1 exp(U(b,·)/σ)
To construct the choice probabilities, the denominator requires evaluatingU(b,·) everywhere.
As #Bgrows, this means cost grows quadratically, O(#B2).
But, imagineU(b,b0∗)/σ= 0 and elsewhereU(b,·)/σ=−50?
Then we could obtain anumerically equivalentsolution by only evaluating atb0=b0∗ (provided we knew where b0∗ was) because
#B
X
1
exp(U(b,·)/σ) = 1+(#B−1)e−50On a P.C.= 1 = exp(U(b,b0∗)/σ)
The solution
I propose algorithms that avoid wastingU evaluations by determining a priori whether a choice is numerically irrelevant.
For small taste shocks, the algorithms make value function maximization20-200times faster for small taste shocks in a
• standard incomplete markets model and a
• sovereign debt model that needs taste shocks for convergence.
For more moderate sized taste shocks, the algorithms are still often 10 times faster.
Taste shocks date toLuce (1959); McFadden (1974); Rust (1987).
Taste shocks have been used in
• maximum likelihood estimation (Rust, 1987),
• facilitating indirect inference estimation (Bruins et al., 2015), • smoothing out nonconcavities (Iskhakov et al., 2017),
• consumer default w/ imperfect info (Chatterjee et al., 2015),
• sovereign default (Dvorkin et al., 2018;Gordon and Querron-Quintana, 2018),
• marriage markets (Santos and Weiss, 2016), • dynamic migration models (Childers, 2018), and
• quantal response equilibria (McKelvey and Palfrey, 1995).
I build on algorithms inGordon and Qiu (2018).
Choice probabilities
ConsiderU(i,i0) = log(bi −bi0/2) + log(bi0).
This hasincreasing differences, essentiallyUi,i0 ≥0.
• Gordon and Qiu (2018) collect many sufficient conditions. • Loosely, this implies monotone policies (w/o taste shocks) • And monotonicity partly implies increasing differences.
Also,U(i,·) is concave—upper contour sets discretely convex.
Contour plots of log10(P(i0|i)) for two σ. -16 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100
−16 is machine epsilonfor PCs: x <10−16 implies1+x=1.
Choice probabilities
Contour plots of log10(P(i0|i)) for two σ.
-16 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100
Increasing differences⇒ contours are monotone increasing
Contour plots of log10(P(i0|i)) for two σ. -16 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100
Suppose one knew the lower and upper contour ati = 50.
Choice probabilities
Contour plots of log10(P(i0|i)) for two σ.
-16 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100
Lower prob. than the square Lower prob. than
the square Lower prob.
than the square
Lower prob. than the square
With increasing differences, can rule out the rectangular regions as being numerically relevant.
Contour plots of log10(P(i0|i)) for two σ. -16 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100 -16 -16 -16 -8 -8 -8 -4 -4 -4
20 40 60 80 100
20 40 60 80 100
Concavity instead can be used to quickly find the squares.
First locate σ= 0 max using Heer et al.’s 2005 algorithm.
Algorithms
Algorithms in the paper show how to exploit
• monotonicity with 1 state / 1 choice, • concavity with 1 state / 1 choice,
• monotonicity and concavity jointly with 1 state / 1 choice, and
Empirical performance
I use theAiyagari (1994) model to test all the algorithms jointly. Below are the results formonotonicity only.
100 200 300 400 500
0 50 100
100 200 300 400 500
0 20 40
100 200 300 400 500
0 10 20 30 40
100 200 300 400 500
0 5 10 15 20 25
100 200 300 400 500
0 10 20 30
100 200 300 400 500
0 5 10 15
Evaluation count “speedup” is n2(the naive ##UevaluationsUevaluations)
Theoretical and empirical efficiency
Forsmall taste shocks, the theoretical worst case evaluations ofU
andempirical speedup atn= 500 are
Theoretical Empirical
Algorithm Evaluations Asymptotics Speedup
Monotonicity only nlog2n+ 5n O(nlog2n) 40 Concavity only 2nlog2n+ 2n O(nlog2n) 30 Mono. + conc. 18n+ 2 log2n O(n) 130 Two-state mono. Messy O(n2)∗ 200
∗only applies for a subset of problems.
#1: Reducing computational errors
Compare “the” consumption policy,E~c(a, ~).
0 10 20 30 40 50
0 1 2 3 4 5
0 10 20 30 40 50
-6 -5 -4 -3 -2 -1
0 10 20 30 40 50
0 1 2 3 4 5
0 10 20 30 40 50
-6 -5 -4 -3 -2 -1
Four reasons to use taste shocks
#2: Market clearing
5 10 15 20 25
-0.03 -0.02 -0.01 0 0.01 0.02 0.03
7 8 9 10
0.012 0.013 0.014 0.015 0.016 0.017 0.018
5 10 15 20 25
-0.03 -0.02 -0.01 0 0.01 0.02 0.03
7 8 9 10
0.012 0.013 0.014 0.015 0.016 0.017 0.018
#3: Smoothness in calibration and estimation
5 10 15 20 25
-0.03 -0.02 -0.01 0 0.01 0.02 0.03
7 8 9 10
0.012 0.013 0.014 0.015 0.016 0.017 0.018
5 10 15 20 25
-0.03 -0.02 -0.01 0 0.01 0.02 0.03
7 8 9 10
0.012 0.013 0.014 0.015 0.016 0.017 0.018
Four reasons to use taste shocks
#4: Facilitating fixed point calculations
In long-term sovereign debt models, bond pricesq must satisfy a fixed point like
(T◦q)(b0,y) =R−1Ey0|y(1−d(b0,y0)) λ+ (1−λ)q(a(b0,y0),y0)
whereais the debt issuance policy and d is the default decision.
The difficulty is the sovereign may be nearly indifferent between two, far-awayachoices.
A small change inq may cause a large jump ina, making T ◦q
500 1000 1500 2000 2500 -6
-4 -2
500 1000 1500 2000 2500
-6 -4 -2
500 1000 1500 2000 2500
Performance in the long-term debt model
Note: Here, increasing differences does not hold globally, but does locally about the optimal policy. But, one can use guess-and-verify.
50 100 150 200 250 300 350 400 450 500
0 10 20 30 40
50 100 150 200 250 300 350 400 450 500
6 6.5 7 7.5
Taste shocks are useful in many applications.
However, the naive approach comes with significant cost.
In many models, a numerically equivalent approximation can be obtained at less cost.
Exploiting monotonicity and/or concavity in one or two states generates large speedups for small taste shocks.
The speedup can be extremely large, but is decreasing inσ.
I will show now that asn increases, one “should” sendσ to zero.
Optimal taste shock sizes
Mean Euler errors for different taste shock and grid sizes:
0 100 200 300 400 500 600 700 800 900 1000 -4
-3.5 -3 -2.5 -2 -1.5 -1
Note that as n increases, the bestσ decreases.
This graph fromChatterjee and Eyigungor (2012) shows the indifference.
Consequently, small changes inq can make thea policy jump from
Exploiting monotonicity
1 3 5 7 9 11 13 15 18 20 22 24 26 28 30 32 35 5
10 15 20 25 30 35
1 16 32 47 63 78 94 109 125 140 156 171 187 202 218 234 250 50
100 150 200 250
Ati = 1, know nothing: Have to evaluate at alli0.
1 3 5 7 9 11 13 15 18 20 22 24 26 28 30 32 35 5
10 15 20 25 30 35
1 16 32 47 63 78 94 109 125 140 156 171 187 202 218 234 250 50
100 150 200 250
At i = 35, know something—the lower bound is i0 = 1.
However, this is not helpful.
Exploiting monotonicity
1 3 5 7 9 11 13 15 18 20 22 24 26 28 30 32 35 5
10 15 20 25 30 35
1 16 32 47 63 78 94 109 125 140 156 171 187 202 218 234 250 50
100 150 200 250
However,
• ati = 9, upper contour ati = 18 provides a useful bound.
• ati = 26,lower contour at i = 18 provides a useful bound.
1 3 5 7 9 11 13 15 18 20 22 24 26 28 30 32 35 5
10 15 20 25 30 35
1 16 32 47 63 78 94 109 125 140 156 171 187 202 218 234 250 50
100 150 200 250
As divide and conquer continues, bounds become more useful.
Exploiting monotonicity
1 3 5 7 9 11 13 15 18 20 22 24 26 28 30 32 35 5
10 15 20 25 30 35
1 16 32 47 63 78 94 109 125 140 156 171 187 202 218 234 250 50
100 150 200 250
Only a very small percent of wasted evaluations.
Largest gain is where only a few numerically-relevant choices.
S. R. Aiyagari. Uninsured idiosyncratic risk and aggregate savings.
Quarterly Journal of Economics, 109(3):659–684, 1994. M. Bruins, J. A. Duffy, M. P. Keane, and A. A. Smith, Jr. Generalized indirect inference for discrete choice models. Mimeo, July 2015.
S. Chatterjee and B. Eyigungor. Maturity, indebtedness, and default risk. American Economic Review, 102(6):2674–2699, 2012.
S. Chatterjee and B. Eyigungor. Continuous Markov equilibria with quasi-geometric discounting. Journal of Economic Theory, 163: 467–494, 2016.
Bibliography II
D. Childers. Solution of rational expectations models with function valued states. Mimeo, 2018.
M. Dvorkin, J. M. S´anchez, H. Sapriza, and E. Yurdagul. Sovereign debt restructurings: A dynamic discrete choice approach.
Working paper, Federal Reserve Bank of St. Louis, 2018. G. Gordon and S. Qiu. A divide and conquer algorithm for
exploiting policy function monotonicity. Quantitative Economics, 9(2):521–540, 2018.
G. Gordon and P. A. Querron-Quintana. A quantitative theory of hard and soft sovereign defaults. Mimeo, 2018.
F. Iskhakov, T. H. Jørgensen, J. Rust, and B. Schjerning. The endogenous grid method for discrete-continuous dynamic choice models with (or without) taste shocks. Quantitative Economics, 8(2):317–365, 2017.
D. McFadden. Conditional logit analysis of qualitative choice behavior. In P. Zarembka, editor, Frontiers in Econometrics, chapter 4. Academic Press, 1974.
R. D. McKelvey and T. R. Palfrey. Quantal response equilibria for normal form games. Games and Economic Behavior, 10:6–38, 1995.
J. Rust. Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica, 55(5):999–1033, 1987. C. Santos and D. Weiss. “Why not settle down already?” A