3.3 Solving by dynamic programming
3.4.1 Intertemporal utility maximization with a CES utility function
The individual’s budget constraint is given in the dynamic formulation
at+1 = (1 + rt) (at+ wt ct) : (3.4.1) Note that this dynamic formulation corresponds to the intertemporal version in the sense that (3.1.3) implies (3.4.1) and (3.4.1) with some limit condition implies (3.1.3). This will be shown formally in ch. 3.5.1.
The budget constraint (3.4.1) can be found in many papers and also in some textbooks.
The timing as implicit in (3.4.1) is illustrated in the following …gure. All events take place at the beginning of the period. Our individual owns a certain amount of wealth at at the beginning of t and receives here wage income wt and spends ct on consumption also at the beginning. Hence, savings st can be used during t for production and interest is paid on st which in turn gives at+1 at the beginning of period t + 1.
t t+1 ct+1
ct
( )
at+1= +1 r st t st =at +wt −ct
Figure 3.4.1 The timing in an in…nite horizon discrete time model
The consistency of (3.4.1) with technologies in general equilibrium is not self-evident.
We will encounter more conventional budget constraints of the type (2.5.13) further below.
As (3.4.1) is widely used, however, we now look at dynamic programming methods and take this budget constraint as given.
The objective of the individual is to maximize her utility function (3.1.1) subject to the budget constraint by choosing a path of consumption levels ct;denoted by fc g ; 2 [t; 1] : We will …rst solve this with a general instantaneous utility function and then insert the CES version of it, i.e.
u (c ) = c1 1
1 : (3.4.2)
The value of the optimal program fc g is, given its initial endowment with at, de…ned as the maximum which can be obtained subject to the constraint, i.e.
V (at) max
fc g Ut (3.4.3)
subject to (3.4.1). It is called the value function. Its only argument is the state variable at: See ch. 3.4.2 for a discussion on state variables and arguments of value functions.
DP1: Bellman equation and …rst-order conditions
We know that the utility function can be written as Ut= u (ct) + Ut+1: Now assume that the individual behaves optimally as from t+1: Then we can insert the value function.
The utility function reads Ut = u (ct) + V (at+1) : Inserting this into the value function, we obtain the recursive formulation
V (at) = max
ct fu (ct) + V (at+1)g ; (3.4.4) known as the Bellman equation.
Again, this breaks down a many-period problem into a two-period problem: The objective of the individual was maxfc g(3.1.1) subject to (3.4.1), as shown by the value function in equation (3.4.3). The Bellman equation (3.4.4), however, is a two period decision problem, the trade-o¤ between consumption today and more wealth tomorrow (under the assumption that the function V is known). This is what is known as Bellman’s
principle of optimality: Whatever the decision today, subsequent decisions should be made optimally, given the situation tomorrow. History does not count, apart from its impact on the state variable(s).
We now derive a …rst-order condition for (3.4.4). It reads d
dctu (ct) + d
dctV (at+1) = u0(ct) + V0(at+1)dat+1 dct = 0:
Since dat+1=dct= (1 + rt) by the budget constraint (3.4.1), this gives
u0(ct) (1 + rt) V0(at+1) = 0: (3.4.5) Again, this equation makes consumption a function of the state variable, ct = ct(at) : Following the …rst-order condition (3.3.5) in the general example, we wrote ct = c (xt) ; i.e. consumption ct changes only when the state variable xt changes. Here, we write ct= ct(at) ;indicating that there can be other variables which can in‡uence consumption other than wealth at: An example for such an additional variable in our setup would be the wage rate wt or interest rate rt; which after all is visible in the …rst-order condition (3.4.5). See ch. 3.4.2 for a more detailed discussion of state variables.
Economically, (3.4.5) tells us as before in (3.3.5) that, under optimal behaviour, gains from more consumption today are just balanced by losses from less wealth tomorrow.
Wealth tomorrow falls by 1 + rt, this is evaluated by the shadow price V0(at+1) and everything is discounted by :
DP2: Evolution of the costate variable
Using the envelope theorem, the derivative of the maximized Bellman equation reads ,
V0(at) = V0(at+1)@at+1
@at : (3.4.6)
We compute the partial derivative of at+1with respect to atas the functional relationship of ct= ct(at) should not (because of the envelope theorem) be taken into account.
From the budget constraint we know that @a@at+1
t = 1 + rt: Hence, the evolution of the shadow price/ the costate variable under optimal behaviour is described by
V0(at) = [1 + rt] V0(at+1) : This is the analogon to (3.3.6).
DP3: Inserting …rst-order conditions
Let us now be explicit about how to insert …rst-order conditions into this equation. We can insert the …rst-order condition (3.4.5) on the right-hand side. We can also rewrite the
…rst-order condition (3.4.5), by lagging it by one period, as (1 + rt 1) V0(at) = u0(ct 1) and can insert this on the left-hand side. This gives
u0(ct 1) 1(1 + rt 1) 1 = u0(ct), u0(ct) = [1 + rt] u0(ct+1) : (3.4.7) This is the same result as the one we obtained when we used the Lagrange method in equation (3.1.6).
It is also the same result as for the two-period saving problem which we found in OLG models - see e.g. (2.2.6) or (2.6.1) in the exercises. This might be surprising as the planning horizons di¤er considerably between a 2- and an in…nite-period decision problem. Apparently, whether we plan for two periods or for many more, the change between two periods is always the same when we behave optimally. It should be kept in mind, however, that consumption levels (and not changes) do depend on the length of the planning horizon.
The CES and logarithmic version of the Euler equation
Let us now insert the CES utility function from (3.4.2) into (3.4.7). Computing mar-ginal utility gives u0(c ) = c and we obtain a linear di¤erence equation in consumption, ct+1 = ( [1 + rt])1= ct: (3.4.8) Note that the logarithmic utility function u (c ) = ln c ; known for the two-period setup from (2.2.1), is a special case of the CES utility function (3.4.2). Letting approach unity, we obtain
lim!1u (c ) = lim
!1
c1 1
1 = ln c
where the last step used L’Hôspital’s rule: The derivative of the numerator with respect to is
When the logarithmic utility function is inserted into (3.4.7), one obtains an Euler equa-tion as in (3.4.8) with set equal to one.