A User''s Guide to Solving Dynamic Stochastic Games Using the Homotopy Method

(1)

Vol. 58, No. 4, Part 2 of 2, July–August 2010, pp. 1116–1132

issn0030-364Xeissn1526-54631058041116 doi10.1287/opre.1100.0843

A User’s Guide to Solving Dynamic Stochastic

Games Using the Homotopy Method

Ron N. Borkovsky

Rotman School of Management, University of Toronto, Toronto, Ontario M5S 3E6, Canada, [email protected]

Ulrich Doraszelski

Department of Economics, Harvard University, Cambridge, Massachusetts 02138, [email protected]

Yaroslav Kryukov

Tepper School of Business, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, [email protected]

This paper provides a step-by-step guide to solving dynamic stochastic games using the homotopy method. The homotopy method facilitates exploring the equilibrium correspondence in a systematic fashion; it is especially useful in games that have multiple equilibria. We discuss the theory of the homotopy method and its implementation and present two detailed examples of dynamic stochastic games that are solved using this method.

Subject classiﬁcations: homotopy method; dynamic stochastic games; Markov-perfect equilibrium; equilibrium correspondence; game theory; industrial organization; numerical methods.

Area of review: Special Issue on Computational Economics.

History: Received February 2008; revision received November 2009; accepted December 2009. Published online in

Articles in AdvanceJuly 14, 2010.

1. Introduction

There has been much interest in game-theoretic models of industry evolution and, in particular, in the framework introduced by Ericson and Pakes (1995) that is at the heart of a largeand growing literaturein industrial organiza-tion and other ﬁelds (see Doraszelski and Pakes 2007 and the references therein). Ericson and Pakes (1995) provide a model of dynamic competition in an oligopolistic indus-try with investment, enindus-try, and exit. Their framework is designed to facilitate numerical analysis of a wide variety of phenomena that are too complex to be explored in ana-lytically tractable models. Methods for computing Markov-perfect equilibria are therefore a key part of this stream of research. This paper contributes by providing a step-by-step guide to solving dynamic stochastic games using the homotopy method.

Computing a Markov-perfect equilibrium of a dynamic stochastic gameamounts to solving a largesystem of equa-tions. To date, the Pakes and McGuire (1994) algorithm has been used most often to compute equilibria of dynamic oligopoly models. This backward solution method falls into the broader class of Gaussian methods. The idea behind Gaussian methods is that it is harder to solve a large sys-tem of equations once than to solve smaller syssys-tems many times, and that it may therefore be advantageous to break up a large system into small pieces.

The drawback of Gaussian methods is that they offer no systematic approach to computing multiple equilibria. The potential for multiplicity in the Ericson and Pakes (1995) framework is widely recognized; see p. 570 of Pakes and

McGuire(1994) and, morerecently, theexamples of mul-tipleequilibria in theonlineappendix to Doraszelski and Satterthwaite (2010) and in Besanko et al. (2010a). To identify more than one equilibrium (for a given parame-terization of the model), the Pakes and McGuire (1994) algorithm must be restarted from different initial guesses. However, different initial guesses may or may not lead to different equilibria. A similar remark applies to the stochas-tic approximation algorithm of Pakes and McGuire (2001), the other widely used method for computing equilibria.

This, however, still understates the severity of the prob-lem. When there are multiple equilibria, the trial-and-error approach of restarting the Pakes and McGuire (1994) algo-rithm from different initial guesses is sure to miss a sub-stantial fraction of them, regardless of how many initial guesses are tried: As shown by Besanko et al. (2010a), if a dynamic stochastic gamehas multipleequilibria, then someof them cannot possibly becomputed by thePakes and McGuire (1994) algorithm. It is important, therefore, to consider alternative algorithms that can identify multiple equilibria and thus provide us with a more complete picture of the set of solutions to a dynamic stochastic game.

Thehomotopy method allows us to exploretheequilib-rium correspondence in a systematic fashion. The homo-topy method is a type of path-following method. Starting from a single equilibrium that has already been computed for a given parameterization of the model, the homotopy algorithm traces out an entire path of equilibria by varying one or more parameters of the model. Whenever we can ﬁnd such a path and multipleequilibria aretheresult of

1116 INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(2)

the path bending back on itself, then the homotopy method is guaranteed to identify them. We note at the outset that it is not assured that any given path includes all possible equilibria at a given value of the parameter vector.

In this paper wediscuss thetheory of thehomotopy method as well as HOMPACK90, a suite of Fortran90 rou-tines developed by Watson et al. (1997) that implements this method. To explore the equilibrium correspondence of a dynamic stochastic game, we set up the homotopy algo-rithm to vary oneor moreparameters of themodel. This type of application is referred to as a natural-parameter homotopy. We discuss potential problems that one may encounter in using HOMPACK90 in this way and offer someguidanceas to how to resolvethem.

We present two examples of dynamic stochastic games and show, step by step, how to solve them using the homotopy method. In order to use the homotopy method, one must formulate a problem as a system of equations. Our ﬁrst example, the learning-by-doing model of Besanko et al. (2010a), is particularly well suited for the homo-topy method because it is straightforward to express the equilibrium conditions as a system of equations. We dis-cuss in detail how this is done. Moreover, we illustrate the computational demands of the homotopy method using the learning-by-doing model as an example.

Our second example, the quality ladder model of Pakes and McGuire (1994), presents a complication. As invest-ment cannot be negative, the problem that a ﬁrm has to solve is formulated using a complementary slackness con-dition, a combination of equalities and inequalities. We show how to reformulate this complementary slackness condition as a system of equations that is amenable to the homotopy method.

In sum, this paper provides a step-by-step guide to solving dynamic stochastic games using the homotopy method. Our goal here is to enable the reader to start using HOMPACK90 as quickly as possible. To this end, we also makethecodefor thelearning-by-doing and quality ladder models available on our homepages. The code is accom-panied by additional detailed instructions on how to set up and useit.

2. The Theory of the Homotopy Method

A Markov-perfect equilibrium of a dynamic stochastic game consists of values, i.e., expected net present values of per-period payoffs, and policies, i.e., strategies, for each player in each state. Values are typically characterized by Bellman equations and policies by optimality conditions (e.g., ﬁrst-order conditions). These equilibrium conditions depend on the parameterization of the model. Collect-ing Bellman equations and optimality conditions for each player in each state, the equilibrium conditions amount to a system of equations of the form

Hx =0 (1)

whereH N+1_→N_, _x_∈N _{is thevector of unknown}

values and policies, and 0∈N _{is a vector of zeros.} _∈

01is the so-called homotopy parameter.1 _{Depending on} theapplication at hand, thehomotopy parameter maps into oneor moreof theparameters of themodel. Theobject of interest is the equilibrium correspondence

H−1₌ _x _H_x ₌₀

Thehomotopy method aims to traceout entirepaths of equilibria inH−1_{by varying both}_x _and_.

To this end, we deﬁne a parametric path as a set of func-tionsxs ssuch that xs s∈H−1_{. Thepoints} on thepath areindexed by theauxiliary variables; as we movealong thepath,seither increases or decreases mono-tonically; however, neitherx nornecessarily vary mono-tonically. DifferentiatingHxs s=0 with respect to

syields

Hxs s x xs+

Hxs s

s=0 (2)

where Hxs s/x is the N ×N Jacobian of H

with respect tox,x_s_and_H_x_{s s/}_are_N_×₁

vectors, and _s _{is a scalar. This system of equations}

captures the conditions that must be satisﬁed in order to remain “on path.” As this is a system ofNdifferential equa-tions in N +1 unknowns, x

is, i=1 N, and s,

it has many solutions. Although these solutions differ by monotonetransformations ofs, they all describe the same path. One solution obeys the so-called basic differential equations y is=−1i+1det _H_y_s y −i i=1 N+1 (3) where ys=xs s and thenotation ·₋_i is used to indicatethat theith column is removed from the

N×N+1JacobianHys/yof Hwith respect to y. A proof that the basic differential equations (3) satisfy theconditions (2) for remaining on path can befound in Garcia and Zangwill (1979) and on pp. 27–28 of Zangwill and Garcia (1981).

An implementation of the homotopy method—a homo-topy algorithm—can beused to traceout entirepaths of equilibria by numerically solving the basic differential equations (3). An already computed equilibrium provides theinitial condition. From therea homotopy algorithm uses the basic differential equations to determine the next step along the path. In this manner, it continues to follow the path step by step.

Regularity and Smoothness Requirements. A closer inspection of the basic differential equations (3) reveals a potential difﬁculty. If the Jacobian Hys/y is not of full rank at somepointyson thesolution path, then the

INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(3)

Figure 1. Examples of solution paths ifHis regular (left panel) and not regular (right panel). 0 0.2 0.4 0.6 0.8 1 0 0.5 1 1.5 2 2.5 3 3.5 4 λ x A C B D 0 0.2 0.4 0.6 0.8 1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 λ x H I G F E

determinant of each of its square submatrices is zero. Thus, according to the basic differential equations (3),y

is=0,

i=1 N +1, and thehomotopy method is stuck at point ys. A central condition in the mathematical liter-atureon thehomotopy method is thus that theJacobian must havefull rank at all points on thesolution path. If so, thehomotopy is called regular. Moreformally, H is regular if rankHy/y=N for all y∈H−1_. Theregu-larity requirement—and a certain smoothness requirement to be discussed below—ensures that the set of solutions H−1 _{consists only of continuous paths. The left panel of} Figure1 shows examples of possiblesolution paths if H is regular: (A) paths that start at =0 and end at=1, (B) paths that start and end at=0 or=1, (C) loops, and (D) paths that start at=0 or=1 but never end because

x (or a component of x in thecaseof a vector) tends to

+ or −.2 _{Theright panel of Figure1 shows} exam-ples of solution paths that are ruled out by the regularity requirement: (E) isolated equilibria, (F) continua of equilib-ria, (G) branching points,3_{(H) paths of inﬁnitelength that} start at =0 or =1 and convergeto singlepoints (spi-rals), and (I) paths that start at=0 or=1 but suddenly terminate.

In practice, it is often hard to establish regularity because the Jacobian of a system of equations that characterizes theequilibria of a dynamic stochastic gameformulated in the Ericson and Pakes (1995) framework tends to be intractable. This stems partly from the fact that the Jacobian for such a system is typically quitelargebecausethesystem includes at least two equations (Bellman equation and opti-mality condition) for each stateof theindustry, and even “small” models with few ﬁrms and few states per ﬁrm tend to have hundreds of industry states.

The other major requirement of the homotopy method is smoothness in the sense of differentiability. This yields solution paths that aresmooth and freeof sudden turns or kinks. Formally, if H is continuously differentiable in

addition to regular, then the set of solutionsH−1 _consists only of continuously differentiable paths. This result is known as the path theorem and essentially follows from the implicit function theorem (see, e.g., p. 20 of Zangwill and Garcia 1981). Moreover, for a path to be described by thebasic differential equations (3), it must bethecasethat H is twice continuously differentiable in addition to reg-ular. This result is known as the BDE theorem (see, e.g., pp. 27–28 of Zangwill and Garcia 1981).

The smoothness requirement is nontrivial and easily vio-lated, for example, by nonnegativity constraints on compo-nents ofx, say because investment cannot become negative, or by distributions with nondifferentiable cumulative dis-tribution functions such as theuniform disdis-tribution that is often used to model random scrap values and setup costs. Section 5 explains how to deal with such complications.

There is a subtle difference between the homotopy method, a mathematical theory, and the homotopy algo-rithm, a numerical method. In theory, the homotopy method is used to describe solution paths. In practice, a homotopy algorithm takes discrete steps along such a path. This can be beneﬁcial because the homotopy algorithm may suc-ceed in tracing out a solution path even if the regularity and/or smoothness requirements are violated; as the homo-topy algorithm proceeds along the solution path in discrete steps, it may skip over points at which one or both of these requirements are violated. However, this also can lead to a complication. As the homotopy algorithm proceeds in dis-crete steps, it may jump from one solution path to another, thus failing to trace out either path in its entirety. These issues are discussed further in §5.5.

3. The HOMPACK90 Software Package

HOMPACK90 is as a suite of Fortran90 routines that traces out a path in H−1₌ _y_H_y₌₀ INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(4)

Table 1. Path-following algorithms and dense vs. sparse Jacobian in HOMPACK90.

DenseJacobian SparseJacobian

ODE-based FIXPDF FIXPDS

Normal ﬂow FIXPNF FIXPNS

Augmented Jacobian FIXPQF FIXPQS

Thenotation y=x ∈N+1 _{underlines that the} homo-topy method does not make a distinction between the unknown variablesx∈N _{and the homotopy parameter}_∈

01. Here we just give a brief overview that is meant to enable the reader to start using HOMPACK90 as quickly as possible. A detailed description of HOMPACK90 is given in Watson et al. (1987, 1997).

To useHOMPACK90, theuser must provideFortran90 codefor thesystem of equations and its Jacobian. In addi-tion, theuser must supply HOMPACK90 with an initial condition in theform of a solution to thesystem of equa-tions for a particular parameterization. HOMPACK90 then traces out a solution path. HOMPACK90 offers several dif-ferent path-following algorithms as well as storage formats for theJacobian of thesystem of equations. Table1 gives an overview. Below we proceed to discuss the differences between the various path-following algorithms and storage formats as well as ways to generate initial conditions. In §4.3 wethen comparetheimplications of thevarious path-following algorithms and storageformats for theperfor-manceof HOMPACK90.

The output of HOMPACK90 includes a sequence of solutions to the system of equations, saved to binary ﬁles,4 and an exit ﬂag that indicates a normal ending or several kinds of failure. Wediscuss someof thepotential problems in §5.5.

3.1. Path-Following Algorithms

HOMPACK90 traces out a parametric path ys∈H−1 _as a sequence of points, indexed by k. The kth point in the sequence is sk_yk_{, where} _yk _{is understood to represent} ysk_{. Thestep along thepath from onepoint to thenext}

starts by choosing s=sk+1₋_sk_{. HOMPACK90 adjusts}

this step length based on the curvature of the path. Then HOMPACK90 computes the next point yk+1 _{using a} two-phase method. The predictor two-phase generates a guess for yk+1_{; the corrector phase then improves this guess using} a version of Newton’s method. The difference between thevarious availablepath-following algorithms lies in the implementation of the predictor and corrector phases.

ODE-Based. The predictor phase directly applies the system of differential equations (2). It ﬁrst solves the sys-tem of linear equations

Hyk

y y=0 (4)

to obtainyand then computes the guess for the next point asyk+1₌_yk₊_y_s_{. As this predictor step tends to be very}

precise, typically several such steps are executed before a corrector step becomes necessary.

Normal Flow.The predictor phase uses a Hermite cubic extrapolation from the previous two points as a guess for thenext point. Although theHermitecubic extrapolation is much easier to compute than solving the system of linear equations (4), it is also much cruder. The corrector phase is thus necessary at every step.

Augmented Jacobian.Thepredictor phaseis thesame as in thenormal ﬂow algorithm, but theaugmented Jacobian algorithm takes a different approach to the cor-rector phase (quasi-Newton instead of Newton steps). 3.2. Jacobian

HOMPACK90 requires the user to provide a routine that returns the JacobianHy/y at a given point y. Dueto thekey roleof theJacobian in thehomotopy algorithm, we next discuss some details on ways to compute and repre-sent it.

Numerical vs. Analytical Jacobian. The easiest way to computetheJacobian is to do so numerically using a one- or two-sided finite-difference scheme (see, e.g., Chap-ter 7 of Judd 1998). However, we found that, due to the lim-ited precision of numerical differentiation, the ODE-based algorithm takes small steps, and this significantly increases the time needed to traverse an entire path; normal flow and augmented Jacobian algorithms are more robust to impre-ciseJacobians.

Theobvious solution is to useanalytical instead of numerical differentiation, but this carries a high ﬁxed cost of deriving, coding, and debugging the Jacobian. Instead, we turn to automatic differentiation, a technique for gen-erating computer programs with statements for the compu-tation of derivatives based on the chain rule of differential calculus. Automatic differentiation relies on the fact that every function, no matter how complicated, is executed on a computer as a sequence of elementary operations (addi-tion, multiplica(addi-tion, etc.) and functions (exp, sin, etc.). By applying thechain ruleover and over again to thecomposi-tion of those elementary operathecomposi-tions, one can compute, in a completely mechanical fashion, a derivative of the function. WeuseADIFOR to automatically differentiateFortran code. ADIFOR is described in Bischof et al. (1996); here wejust givea brief overview.5 _{Theinput to ADIFOR is} theFortran90 codethat returns Hy at a given point y. ADIFOR analyzes the code and from it generates new code. This code receives a pair ofN+1×1vectorsy y

and returns theN×1vector

H=H_yyy

Thus, weobtain thejth column of theJacobian via a sin-gle call to the ADIFOR-generated code withy set to the

jth basis vector. Repeating this for j =1 N +1 we assembletheentireJacobian. INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(5)

This column-by-column approach is necessary because the ADIFOR-generated code never computes the elements

Hiy/yjof theJacobianHy/yexplicitly, but instead

transforms y y into H in a process that parallels thecomputation of Hy. For example, if Hy₁ y₂=

uy₁vy₂ is a function of thescalars y₁ and y₂, where

u and v are themselves functions, then the ADIFOR-generated code computes

u =u_y

1y1

v =v_y

2y2

H =uy1v+vy2u

Dense vs. Sparse Jacobian. In many applications, including dynamic stochastic games, the number of equa-tions and unknowns is large, but any given equation involves only a small number of unknowns (because the transitions from onestateto thenext aretypically restricted to a small set of “nearby” states), leading to a Jacobian comprising mostly zeros. Such a Jacobian is called sparse and can be more efﬁciently represented using a sparse matrix storageformat that consists of a list of thenonzero elements with corresponding row and column indices rather than as a densematrix that consists of theentireset of

N N+1elements.

Taking advantageof thesparsenatureof theJacobians in dynamic stochastic games offers a decrease in compu-tation time, and in fact we show in §4.3 that this decrease is substantial. The additional efﬁciency comes from lower memory requirements and faster linear algebra operations. In addition, the Jacobian of a very large system of equa-tions may exceed the available memory unless it is stored as a sparsematrix.

Theuseof sparseJacobians is complicated by two issues. First, there is additional coding because the user must spec-ify the “sparsity structure,” i.e., the row and column indices of potentially nonzero elements. In practice, this means going through the system of equations and identifying the elements ofythat appear in a given equation.

Second, thesparseand denseversions of thevari-ous path-following algorithms take different approaches to solving systems of linear equations. In all cases, the lin-ear algebra routines in HOMPACK90 were selected for speed, not reliability, which means that they can and do fail for certain problems. Our experience has been that the sparse linear solver is more likely to fail than the dense linear solver; §5.6 provides further detail and offers some solutions.

3.3. Initial Condition

Theﬁnal input to HOMPACK90 is an initial condition in theform of a solution to thesystem of equations for the particular parameterization associated with =0. In some cases, if the parameterization associated with =0 is trivial, thesolution can bederived analytically. A good

exampleis thecaseof a zero discount ratethat turns a dynamic stochastic game into a set of disjoint static games played out in every state. Another example is a particu-lar parameterization that makes movements through state spaceunidirectional and thus allows thegameto besolved by backwards induction (see, e.g., Judd et al. 2007).

More generally, a solution for a particular parame-terization can be computed numerically using a number of approaches such as Gaussian methods including (but not limited to) the Pakes and McGuire (1994) algorithm, other nonlinear solvers (see Ferris et al. 2007), and arti-ﬁcial homotopies (see Zangwill and Garcia 1981, Chap-ter 1), which can also be implemented using HOMPACK90. Finally, onecan usea solution obtained by tracing out a path along a different parameter as an initial condition (see§5.1 for an exampleof path following along several parameters).

4. Example 1: The

Learning-by-Doing Model

We begin with the learning-by-doing model of Besanko et al. (2010a). The description of the model is abridged. Please refer to Besanko et al. (2010a) for economic moti-vation and greater detail on some derimoti-vations.

4.1. Model

Firms and States. We consider a discrete-time, inﬁnite-horizon stochastic game. Firmn∈ 12is described by its stock of know-how (or experience)e_n∈ 1 M. At any point in time, the industry is completely characterized by a vector of ﬁrms’ statese=e1 e2∈ 1 M2. We refer toeas thestateof theindustry. Weusee2 _{to denote the}

vectore₂ e₁found by interchanging the stocks of know-how of ﬁrms 1 and 2.

Each period, firms observethestateof theindustry and set prices for their respective goods. By making a sale, a firm can add to its stock of know-how. At thesametime, thefirm faces thepossibility of organizational forgetting, leading to the law of motione

n=en+qn−fn, whereenand

en areﬁrm n’s stock of know-how in the subsequent and

current period, respectively, the random variableqn∈ 01

indicates whether ﬁrm n makes a sale, and the random variablef_n ∈ 01 represents organizational forgetting. If

qn=1, theﬁrm gains a unit of know-how through learning

by doing, whereas it loses a unit of know-how through organizational forgetting iff_n=1.

Learning by Doing.Firm n’s marginal cost of produc-tioncendepends on its stock of know-howen through a

learning curve with a progress ratio of∈01:

ce_n=    e_n if 1en< m m ifmenM

where =log₂. Marginal cost decreases by 1001−

percent as the stock of know-how doubles, so that a

(6)

lower progress ratio implies a steeper learning curve. The marginal cost of production at thetop of thelearning curve,

c1, is >0, andmrepresents the stock of know-how at which a ﬁrm reaches the bottom of its learning curve.

Organizational Forgetting.Theprobability that ﬁrm n

loses a unit of know-how through organizational forget-ting is

e_n=Prf_n=1=1−1−$en

where$∈01is the forgetting rate.

Demand. Each period, one (nonstrategic) buyer enters the market and purchases a unit of the good from one of thetwo ﬁrms. Thenet utility of good n to a buye r is

v−pn+&n, wherepn is theprice,vis the ﬁxed component

of utility, and&nis a stochastic component that captures the

buyer’s idiosyncratic preference for good n. Thebuyer’s idiosyncratic preferences&1 &2areunobservableto ﬁrms and are independently and identically type 1 extreme value distributed. The buyer purchases the good that gives it the highest net utility, so the probability that ﬁrm n makes a saleis given by

Dnp=Prqn=1=2expv−pn

k=1expv−pk =₁₊_exp_p1

n−p−n

wherep=p1 p2is the vector of prices and we adopt the convention of usingp−nto denotethepricecharged by the

other ﬁrm.

State-to-State Transitions.From oneperiod to thenext, thetransition probabilities are

Pre nen qn=    1−e_n ife n=en+qn e_n ife n=en+qn−1

where, at the upper and lower boundaries of the state space, wemodify thetransition probabilities to be PrMM1=1 and Pr110=1, respectively.

Bellman Equation and First-Order Condition.Deﬁne

Vne to be the expected net present value of ﬁrm n’s

cash ﬂows if theindustry is currently in statee. Thevalue functionV_n 1 M2_→_{is implicitly deﬁned by the} Bellman equation V_ne=max pn Dnpn p−nepn−cen +) 2 k=1 Dkpn p−neVnke (5)

where p−ne is thepricecharged by theother ﬁrm in

state e,)∈01 is thediscount factor, andVnkeis the

expectation of ﬁrm n’s valuefunction conditional on the

buyer purchasing the good from ﬁrmk∈ 12 in state e as given by Vn1e= e1+1 e 1=e1 e2 e 2=e2−1 VnePre1e11Pre2e20 (6) Vn2e= e1 e 1=e1−1 e2+1 e 2=e2 VnePre1e10Pre2e21 (7)

Thepolicy function p_n 1 M2_→ _{speciﬁes the} price p_ne that ﬁrm n sets in state e. To determine it, let hn· bethemaximand in theBellman equation (5).

Differentiatinghn·with respect topnweobtain

theﬁrst-order condition

0=Dnpn p−ne1−pn−cen−)Vnne+hn·

It is straightforward to show that thepricing decisionp_ne

is uniquely determined by the solution to the ﬁrst-order condition.

Equilibrium. We restrict attention to symmetric Markov-perfect equilibria. In a symmetric equilibrium the pricing decision taken by ﬁrm 2 in stateeis identical to the pricing decision taken by ﬁrm 1 in statee2_{, i.e.,} _p

2e=

p1e2, and similarly for thevaluefunction. It therefore suffices to determine the value and policy functions of firm 1, and wedefineV e=V₁eand pe=p₁e for each state e. To simplify thenotation, wefurther define

Vke= V1ke to betheconditional expectation of ﬁrm

1’s valuefunction andDke=Dkpe pe2to bethe

probability that the buyer purchases from ﬁrmk∈ 12in thestatee.

Parameterization.Wefocus on theways in which learn-ing by dolearn-ing and organizational forgettlearn-ing affect priclearn-ing behavior, and the industry dynamics implied by that behav-ior. Besanko et al. (2010a) prove that the model has a uniqueequilibrium if$=0 or$=1. It is therefore natural to usethehomotopy method to traceout theequilibrium correspondence by varying $ from 0 to 1. Wethus make theforgetting rate$a function of the homotopy parameter

and set

$=$start₊_$end₋_$start

In particular, if$start₌_{0 and}_$end₌_{1, then the homotopy} method traces out the equilibrium correspondence from

$0=0 to $1=1. To exploretheroleof learning by doing, we repeat this procedure for 100 evenly spaced val-ues of ∈0011. We hold the remaining parameters ﬁxed at the values shown in Table 2.

System of Equations.Wearenow ready to describethe equilibrium as a system of equations in the form given by (1). Deﬁne the vector of unknown values and policies in equilibrium as x=V 11V 21V M1V 12V MM p11pMM INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(7)

Table 2. Parameter values.

Parameter M m )

Value30 15 ₁1₀₅=09524

Note. Learning-by-doing model.

The Bellman equation and ﬁrst-order condition in state e are H1 ex= −V e+D1epe−ce1 +) 2 k=1 DkeVke=0 (8) H2 ex=1−1−D1epe−ce1−)V1e +) 2 k=1 D_keV_ke=0 (9)

wherewesubstitutefor V₁e and V₂e using deﬁni-tions (6) and (7) (imposing symmetry). The collection of Equations (8) and (9) for all statese∈ 1 M2 _{can be} written more compactly as

Hx=         H1 11x H1 21x H2 M Mx         =0 (10) where 0∈2M2

is a vector of zeros. Any solution to this system of 2M2 _{equations in 2}_M2 _unknowns _x_∈2M2

is a symmetric equilibrium in pure strategies (for a given value of ∈01).6

4.2. Equilibrium Correspondence

Figure2 shows thenumber of equilibria as a function of theprogress ratioand theforgetting rate$. Darker shades indicate more equilibria. The subset of parameterizations that yield three equilibria is fairly large, and we have found up to nineequilibria for somevalues ofand$. Themul-tiple equilibria describe a rich array of pricing behaviors that areeconomically meaningful and that arequitedif-ferent in terms of implied industry structure and dynamics (see Besanko et al. 2010a for a discussion).

Recall that wemadetheforgetting rate$ a function of the homotopy parameter . To visualizetheset of equi-libria as a correspondence of $ for a specific value of , we need a way to summarize each equilibrium as a sin-glenumber. Thevaluefunction in theinitial state11is thevalueof a firm at theonset of theindustry;V 11is thus an economically meaningful summary of an equilib-rium. Because we are also interested in long-run industry concentration, we further compute the expected Herfindahl

Figure 2. Number of equilibria.

0 0.01 0.03 0.05 0.1 0.2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ ρ 1 3 5 7 9

Note. Learning-by-doing model.

index. Weproceed in two steps. First, weusethepolicy function to construct theprobability distribution over the next period’s state e _{given this period’s state}_e_{, and from}

it wecomputethelimiting (or ergodic) distribution over states,+_{. Second, weusethis distribution to computethe}

expected Herﬁndahl index

HHI₌

e∈1 M2

D1e2+D2e2+e

Asymmetric industry structures arise and persist to the extent thatHHI_>₀_5.

Figures 3 and 4 visualize the equilibrium correspondence in terms ofV 11andHHI_{, respectively, for a variety of}

different progress ratios. If the system of equations that characterizes the equilibria is regular, then there must be a path connecting theuniqueequilibria at $=0 and $=1. Multiple equilibria arise whenever this main path bends back on itself. Similar to (A) in the left panel of Figure 1, in thebottom-right panel of Figure4, thepath from$=0 to$=1 is S-shaped around$=01; when the sign of_s

changes from positiveto negativeat $=01181, thepath is bending backward, and when the sign of_s _changes

from negativeto positiveat$=00745, thepath is bending forward.7 _{Consequently, Figure 2 indicates that there are} three equilibria in the vicinity of$=01 and=005.

Wehavealso been ableto identify oneor moreloops, similar to (C) in theleft panel of Figure1, that aredis-joint from the main path. These loops further add to the multiplicity of equilibria. In the upper-left panel of Fig-ure 4, for example, there is a loop around$=01; accord-ingly, Figure2 indicates that therearethreeequilibria in thevicinity of $=01 and =095. However, still other loops may exist because, in order to trace out a loop, we must ﬁrst somehow compute at least one equilibrium on the loop. Unfortunately, there is no way to determine whether one’s search for loops—or more generally equilibria—has

(8)

Figure 3. Initial ﬁrm valueV 11. 0 0.03 0.1 0.3 1 0 5 10 15 20 δ V(1,1) ρ = 0.95 0 0.03 0.1 0.3 1 0 5 10 15 20 δ V(1,1) ρ = 0.35 0 0.03 0.1 0.3 1 0 5 10 15 20 δ V(1,1) ρ = 0.75 0 0.03 0.1 0.3 1 0 5 10 15 20 δ V(1,1) ρ = 0.05

Figure 4. Limiting expected Herﬁndahl indexHHI_.

0 0.03 0.1 0.3 1 0.5 0.51 0.55 0.7 1 δ HHI ∞ ρ = 0.95 0 0.03 0.1 0.3 1 0.5 0.51 0.55 0.7 1 δ HHI ∞ ρ = 0.35 0 0.03 0.1 0.3 1 0.5 0.51 0.55 0.7 1 δ HHI ∞ ρ = 0.75 0 0.03 0.1 0.3 1 0.5 0.51 0.55 0.7 1 δ HHI ∞ ρ = 0.05

(9)

been exhaustive. Figures 2–4 are therefore not necessarily a completemapping of theequilibria.

In some places, the various paths appear to intersect one another. A case in point is the loop in the upper-left panel of Figure 4 that appears to twice intersect the path from

$=0 to $=1. Although such an intersection seems to resemblethebranching point (G) in theright panel of Fig-ure1, thefact that two equilibria giveriseto thesame expected Herﬁndahl index does not mean that the equilibria themselves are the same. We have indeed veriﬁed that the various solution paths do not intersect. Thus, the intersec-tions in Figures 3 and 4 do not stem from violaintersec-tions of the regularity requirement.

4.3. Performance

HOMPACK90 offers several different path-following algo-rithms and storageformats for theJacobian of thesys-tem of equations. Moreover, the user can compute the Jacobian either numerically or analytically. To highlight theimplications of thesechoices for theperformanceof HOMPACK90, we have conducted a series of experiments. Throughout, wefocus on themain path of theequilibrium correspondence from$=0 to$=1 for a progress ratio of

=075 (as shown in the upper-right panel of Figure 4). Weset theprecision in HOMPACK90 to 10−10 _{(see §5.5} for a discussion). WeuseADIFOR to analytically compute the Jacobian. All experiments are conducted on a Linux machinewith a 64-bit 1 GHz AMD Athlon CPU and 4 GB of RAM.

Path-Following Algorithms. A major issueis the trade-off between robustness and computation time. Com-putation timeis theproduct of thenumber of steps it takes to traceout theentirepath and theaveragetimeper step. This involves yet another trade-off because these two deter-minants of computation timeareaffected in oppositeways by thesizeof thestep that thehomotopy algorithm takes from onepoint to thenext. Optimally adjusting thestep sizeis a highly nontrivial problem, and thealgorithm that does this is a major part of HOMPACK90.

Turning to thechoiceof a speciﬁc path-following rithm, Watson et al. (1997) describe the normal ﬂow algo-rithm as the baseline offering a reasonable compromise between robustness and computation time. The ODE-based algorithm is described as the most robust, but the slowest, and the augmented Jacobian algorithm as the least robust, but fastest.

Table 4. Performance: “Complicated” vs. “simple” segment of path.

“Complicated” ($∈0003) “Simple” ($∈0031)

Algorithm/Jacobian Time (h:m) No. of steps Time/step (s) Time (h:m) No. of steps Time/step (s)

ODE-based/sparse, ana. 0:31 507 37 0:57 1072 32

Normal ﬂow/sparse, ana. 0:18 292 39 1:26 1905 28

Aug. Jacobian/sparse, ana. 0:26 290 54 2:17 1905 43

Note.Learning-by-doing model.

Table 3. Performance: Path-following algorithms and densevs. sparseJacobian.

Time No. of Time/step Algorithm/Jacobian (h:m) steps (s) ODE-based/dense, ana. 22:50 1596 515 Normal ﬂow/dense, ana. 28:59 2197 475 Aug. Jacobian/dense, ana. 25:25 2250 407 ODE-based/sparse, ana. 1:28 1579 034 Normal ﬂow/sparse, ana. 1:44 2197 029 Aug. Jacobian/sparse, ana. 2:43 2195 045

Note.Learning-by-doing model.

Table3 shows that theODE-based algorithm is not always slower than the other path-following algorithms. On the contrary, in our experiments the ODE-based algorithm turns out to befastest: Whileit takes moretimeto complete each step, it takes fewer steps to complete the path.

To further investigate this somewhat unexpected ﬁnding, in Table4 wecontrast theperformanceof thedifferent path-following algorithms on separateportions of thesolu-tion path. The ODE-based algorithm is faster on the “sim-ple” segment of the path ($∈0031) without multiplicity and much curvature(so that theunknowns changegradu-ally with the homotopy parameter), but slower on the “com-plicated” segment of the path ($∈0003). Thereason may lie in the different step-size adjustment procedures of the different path-following algorithms. Indeed, as a closer analysis of theoutput of HOMPACK90 reveals, theODE-based algorithm takes much larger (and thus fewer) steps than theother path-following algorithms on the“simple” segment of thepath. In contrast, on the“complicated” seg-ment of the path, the ODE-based algorithm takes much smaller (and thus more) steps than the other path-following algorithms.8

Returning to Table 3, the comparison between the normal ﬂow and augmented Jacobian algorithms is not clear-cut either. The dense augmented Jacobian algorithm takes less time for each step but requires more steps, and this leads to an overall decrease in computation time. In contrast, the sparse augmented Jacobian algorithm takes more time for each step but requires fewer steps, and this leads to an over-all increasein computation time. Although thesparseaug-mented Jacobian algorithm takes only two fewer steps than thesparsenormal ﬂow algorithm in Table3, Borkovsky et al. (2007) havefound that in someapplications both the denseand thesparseversions of theaugmented Jacobian

(10)

algorithm sometimes take up to 20% fewer steps than their normal ﬂow counterparts.

Jacobian. As is obvious from Table3, thedenseJaco-bian algorithms require considerably more computation time. A closer analysis reveals that the additional computa-tion time required by the dense Jacobian algorithms is spent performing linear algebra operations on the Jacobians. Overall, this makes an overwhelmingly strong case for using sparseJacobians.

With regard to thedenseJacobian algorithms, wehave found that the choice between a numerical, hand-coded analytical, or ADIFOR-generated analytical Jacobian has a negligible effect on the time per step. This is because the timerequired to computetheJacobian is dwarfed by the time required to solve the system of linear equations that thealgorithm must solveto computethenext step along thepath.

Turning to thesparseJacobian algorithms, precision is a key advantageof analytically computed Jacobians. Table5 shows that although the ODE-based algorithm succeeds in completing the solution path with an analytical Jacobian, it fails to do so with a numerical Jacobian; in particular, it spends much time tracing out a short segment of the path and stops at$=0096 where it reaches the maximum number of steps.

On the other hand, the normal ﬂow algorithm requires thesamenumber of steps to completethesolution path regardless of whether an analytical or numerical Jacobian is used. Interestingly, the path is computed more quickly when a numerical Jacobian is used (presumably because computing the numerical Jacobian requires less time than computing theanalytical Jacobian dueto thecolumn-by-column approach that ADIFOR requires to assemble to Jacobian). Thus, it appears that the lower precision of the numerical Jacobian is problematic for the ODE-based algo-rithm but not for theother path-following algoalgo-rithms. The likely reason is that, in contrast to the other two path-following algorithms, the ODE-based algorithm uses the Jacobian not just in thecorrector, but also in thepredictor phase.

Overall, our results make an overwhelmingly strong case for using sparseJacobians. Therearealso good reasons to prefer analytical over numerical Jacobians, especially Table 5. Performance: Path-following algorithms and

numerical vs. analytical Jacobian.

Time No. of Time/step Algorithm/Jacobian (h:m) steps (s) ODE-based/sparse, ana. 1:28 1579 34 ODE-based/sparse, num. >6:27 >10000 23 Normal ﬂow/sparse, ana. 1:44 2197 29 Normal ﬂow/sparse, num. 1:22 2197 23

Note. Learning-by-doing model.

because ADIFOR makes the process of computing ana-lytical Jacobians very easy. Finally, we conclude that per-formance is at least partly problem speciﬁc. We therefore recommend conducting experiments on the particular appli-cation at hand. The gains from experimentation can be sub-stantial, and experimentation is virtually costless once the system of equations and the Jacobian have been coded.

5. Example 2: The Quality Ladder Model

We next consider the quality ladder model of Pakes and McGuire(1994). To simplify theexposition, werestrict attention to a duopoly without entry and exit. The descrip-tion of the model is abridged; please see Pakes and McGuire(1994) for details.

5.1. Model

Firms and States. Thestateof ﬁrm n∈ 12 is -_n ∈

1 M and reﬂects its product quality. The vector of ﬁrms’ states is =-1 -2∈ 1 M2, and weuse 2 _{to denotethevector}

-2 -1. Each period, firms first compete in the product market and then make investment decisions. The state in the next period is determined by the stochastic outcomes of these investment decisions and an industrywide depreciation shock that stems from an increasein thequality of an outsidealternative. In partic-ular, firmn’s stateevolves according to thelaw of motion

-

n=-n+.n− , where .n∈ 01 is a random variable

governed by ﬁrmn’s investmentxn0, and ∈ 01is an

industrywide depreciation shock. If.n=1the investment

is successful and the quality of ﬁrm n increases by one level. The probability of success is /xn/1+/xn, where

/ >0 is a measure of the effectiveness of investment. If

=1, the industry is hit by a depreciation shock and the qualities of all ﬁrms decrease by one level; this happens with probability$∈01.

Below weﬁrst describethestatic model of product mar-ket competition and then turn to investment dynamics.

Product Market Competition. Theproduct market is characterized by price competition with vertically differen-tiated products. There is a continuum of consumers. Each consumer purchases at most one unit of one product. The utility a consumer derives from purchasing product n is

g-n−pn+1n, where g-_n=    -n if 1-n-∗ -∗₊_ln₂₋_exp_-∗₋ -n if-∗< -nM

maps thequality of theproduct into theconsumer’s valua-tion for it,pnis theprice, and1n represents the consumer’s

idiosyncratic preference for productn. There is an outside alternative, product 0, that has utility1₀. Assuming that the idiosyncratic preferences10 11 12are independently and identically type1 extremevaluedistributed, thedemand for ﬁrmn’s product is Dnp2=m₁₊exp2 g-n−pn j=1expg-j−pj INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(11)

wherep=p1 p2is the vector of prices andm >0 is the sizeof themarket (themeasureof consumers).

Firm n chooses the price p_n of product n to maximize profits. Hence, firmn’s profits in stateare

3_n=max

pn Dnpn p−n2pn−c

where p−n is thepricecharged by theother ﬁrm and

c0 is themarginal cost of production. Given a state, thereexists a uniqueNash equilibrium of theproduct mar-ket game (Caplin and Nalebuff 1991). It is found easily by numerically solving the system of first-order conditions corresponding to firms’ profit-maximization problem. Note that the quality ladder model differs from the learning-by-doing model in that product market competition does not directly affect state-to-state transitions and, hence, 3n

can be computed before the Markov-perfect equilibria of thedynamic stochastic gamearecomputed via thehomo-topy method. This allows us to treat3nas a primitive

in what follows.

Bellman Equation and Complementary Slackness Condition. Define V_n to be the expected net present valueof firm n’s cash flows if theindustry is currently in state. ThevaluefunctionVn 1 M2→is

implic-itly deﬁned by the Bellman equation

Vn=max_xn₀3n−xn +) _/x n 1+/x_nWn1+ 1 1+/x_nWn0 (11) where)∈01 is thediscount factor andW.n

n is the

expectation of ﬁrm n’s valuefunction conditional on an investment success (.n =1) and failure(.n=0),

respec-tive ly, as give n by

W.n n = ∈01 .−n∈01 $ 1−$1− /x−n 1+/x₋_n .−n · ₁ 1+/x₋_n 1−.−n ×Vnmax min -n+.n− M1 max min -−n+.−n− M1 (12)

wherex−nis the investment of the other ﬁrm in state.

Note that the min and max operators merely enforce the bounds of thestatespace.

Thepolicy function x_n 1 M2_→₀_speciﬁes the investment of ﬁrmnin state. Solving themaximiza-tion problem on theright-hand sideof theBellman equathemaximiza-tion (11), we obtain the complementary slackness condition

−1+) / 1+/xn2W 1 n−Wn00 xn −1+) / 1+/xn2W 1 n−Wn0 =0 (13) x_n0

The investment decisionxnis uniquely determined by

the solution to complementary slackness condition (13) as

xn=−1+

max 1 )/W1

n−Wn0

/ (14)

Equilibrium. We restrict attention to symmetric Markov-perfect equilibria. In a symmetric equilibrium, the investment decision taken by ﬁrm 2 in stateis identical to the investment decision taken by ﬁrm 1 in state2_{, i.e.,}

x2=x12, and similarly for thevaluefunctions. It therefore suffices to determine the value and policy func-tions of firm 1, and wedefineV =V1andx=

x₁ for each state . Similarly, wedeﬁneW.1₌

W.1

1 for each state .

Parameterization.Weallow thehomotopy algorithm to vary/and$: / $ = /start $start + /end₋_/start $end₋_$start

For example, if$start₌_{0 and}_$end₌_{1 while}_/start₌_/end_,

then the homotopy algorithm traces out the equilibrium cor-respondence from $0=0 to$1=1. In general, given any starting and ending values for the parameter vector, the homotopy algorithm can traceout an entirepath of equi-libria by moving along thelinein parameter spacethat connects the starting and ending values. The effectiveness of investment/is a natural parameter to vary because the equilibrium trivially involves no investment if/=0. More-over, this equilibrium is unique. In addition to/, weallow therateof depreciation$ to vary because experience sug-gests that the rate of depreciation is often a key determinant of industry structure and dynamics (see, e.g., Besanko and Doraszelski 2004, Besanko et al. 2010b). Note that in the quality ladder model the equilibrium is not unique at$=0 (seeFigure5) and may not beuniqueat$=1. Wehold the remaining parameters ﬁxed at the values shown in Table 6. Figure 5. Number of equilibria.

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0 5 10 15 20 25 δ α 1 3

Note. Quality ladder model.

(12)

Table 6. Parameter values.

Parameter M m c -∗ ₎

Value18 5 5 12 0925

Note. Quality ladder model.

5.2. The Zangwill and Garcia (1981) Reformulation of the Complementary Slackness Condition Dueto thenonnegativity constraint on investment in the quality ladder model, we obtained a complementary slack-ness condition instead of a ﬁrst-order condition as in the learning-by-doing model in §4. To apply the homotopy method, we must reformulate the system of equations and inequalities in (13) as a system of equations.

Consider a general complementary slackness condition on a scalar variablex:

Ax0

Bx0 (15)

AxBx=0

where Ax and Bx arefunctions of x. Zangwill and Garcia (1981, pp. 65–68) offer a reformulation of the com-plementary slackness condition that consists entirely of equations that are continuously differentiable to an arbitrary degree. The idea is to introduce another scalar variable ;

and consider the system of equations

Ax+max 0 ;k₌₀ ₍₁₆₎

Bx+max 0−;k₌₀ ₍₁₇₎

wherek∈. From Equations (16) and (17), it follows that

;=            −Ax1/k _if_{Ax <}₀ −−Bx1/k _if_{Bx <}₀ 0 ifAx=Bx=0 (18)

Using thefact that max 0−;×max 0 ;=0 and the solution for ; in Equation (18), it is easy to see that the system of Equations (16) and (17) is equivalent to the com-plementary slackness condition in (15). Moreover, this sys-tem isk−1times continuously differentiable with respect to ;. Although wetakek=2 in what follows, wecould easily satisfy the smoothness requirement of the homotopy method by choosing a larger value fork.

The Quality Ladder Model. The complementary slackness condition (13) can be restated as

−1+/x2₊_)/W1₋_W0₀

x−1+/x2₊_)/W1₋_W0₌₀ ₍₁₉₎

x0

whereweusethefact that wefocus on symmetric equilib-ria in order to eliminate ﬁrm indices and multiply through by1+/x2 _{to simplify theexpression. Applying the} Zangwill and Garcia (1981) reformulation to the comple-mentary slackness condition (19) yields the equations

−1+/x2₊_)/W1₋_W0

+max 0;k₌₀ ₍₂₀₎

−x+max 0−;k₌₀ ₍₂₁₎

Theterms max 0 ;k _and _max ₀₋_;k

serve as slack variables that ensure that the inequali-ties in (13) are satisﬁed and the fact that max 0 ;×

max 0−;=0 ensures that the equality in (13) holds. We can now proceed to deﬁne the system of homotopy equations using Equations (20) and (21) along with the Bellman equation −V +31−x +) _/x 1+/xW1+ 1 1+/xW0 =0 (22)

wherewesubstitutefor W1 _and _W0 _using deﬁni-tion (12) (imposing symmetry). This yields a system of 3M2 equations in the 3M2 _unknowns _V₁₁_{V M M}_,

x11 xM M, and;11 ;M M.

Two problems arise. First, because we have added the slack variables, this system of equations is relatively large with 3M2_{equations and unknowns. This leads to increased} memory requirements and computation time. Second, this system of equations yields an extremely sparse Jacobian. Notethat therows of theJacobian corresponding to Equa-tion (21) each have only one or two nonzero elements. Also notethat each column of theJacobian corresponding to a slack variable has only one nonzero element. We have found that such a Jacobian tends to cause HOMPACK90’s sparse linear equation solver to fail; this is discussed further in §5.6.

We address these problems by solving Equation (21) for

x=max 0−;k ₍₂₃₎

and then substituting forxin Equations (20) and (22).9 This reduces the system of 3M2 _{equations in 3}_M2 unknowns to a system of 2M2_{equations in 2}_M2_unknowns. Moreover, it eliminates the rows and columns of the Jacobian that included only one or two nonzero elements; thus, wehaveeliminated theexcessivesparsity that tends to cause HOMPACK90’s sparse linear equation solver to fail. To this end, deﬁne the vector of unknowns in equilib-rium as x=V 11V 21V M1V 12V MM ;11;MM INFORMS holds cop yright to this ar ticle and distr ib uted this cop y as a cour tesy to the author(s). Additional inf or mation, including rights and per mission policies ,is av ailab le at http://jour nals .inf or ms .org/.

(13)

Theequations in stateare H1 x=−V +31−x +) /x 1+/xW1+ 1 1+/xW0 =0 (24) H2 x=−1+/x2+)/W1−W0 +max 0;k₌₀ ₍₂₅₎

wherewesubstitutefor W1 _and _W0 _using defini-tion (12) (imposing symmetry) and for xusing defini-tion (23). Notethat (24) and (25) areequadefini-tions that areused to construct the system of homotopy equations, whereas (12) and (23) are simply definitional shorthands for terms that appear in Equations (24) and (25). The collection of Equations (24) and (25) for all states ∈ 1 M2 _can bewritten morecompactly as Hx =         H1 11x H1 21x H2 M Mx         =0 (26) where 0∈2M2

is a vector of zeros. Any solution to this system of 2M2_{equations in 2}_M2_unknowns,_x_∈2M2

is a symmetric equilibrium in pure strategies (for a given value of ∈01).10 _{The equilibrium investment decision}_x in stateis recovered by substituting the equilibrium slack variable;into deﬁnition (23).

Our approach of replacing a model variable with a slack variablecan betaken only if oneof theequations in the Zangwill and Garcia (1981) formulation admits a closed-form solution for a model variable. (In the case of the quality ladder model, we solved Equation (21) for the investment decision x.) This is always thecaseif a model variable is constrained to be above/below a constant, as with thenonnegativity constraint in thequality ladder model. However, it is possiblethat noneof theequations in theZangwill and Garcia (1981) formulation admits a closed-form solution for a model variable. Suppose, for example, weimposean upper bound on thesum of ﬁrms’ investments in each state, i.e., xn-+x−n-L-, in

the quality ladder model (say because ﬁrms are compet-ing for a scare resource). Then, in solvcompet-ing an equation corresponding to (16) or (17) for xn, oneﬁnds that

xn=f ;n x−n=f ;n xn2. That is,

theclosed-form solution for ﬁrm n’s policy in state -,

x_n, is a function of its rival’s policy in state,x₋_n, and thus its own policy in state2_,_x

n2. In this case,

it is impossibleto ﬁnd a closed-form solution for x_n

as a function of only ;_n, and thus it is impossibleto eliminate the model variablex_n. On theother hand, in

this case, the Jacobian of the system formulated using the “pure” version of the Zangwill and Garcia (1981) formula-tion is no longer as sparse, thereby reducing our motivaformula-tion for replacing a model variable with a slack variable in the ﬁrst place.

5.3. Equilibrium Correspondence

Figure5 shows thenumber of equilibria as a function of the effectiveness of investment/and the rate of depreciation$. Wehavefound up to threeequilibria for somevalues of/

and$. The extent of multiplicity is not nearly as dramatic as in the learning-by-doing model.

To visualize the equilibrium correspondence, we graph the expected Herﬁndahl index

HHIT₌ 1

m2

∈1 M2

D₁p22

+D₂p22₊T

where+T _{is the transient distribution over states in period}

T∈ 101001000, starting from state11in period 0. Weusetransient distributions rather than thelimiting dis-tribution as in the learning-by-doing model because there may be several closed communicating classes.11

As can be seen in Figure 6, industry concentration, both in theshort and in thelong run, is affected by / and $

in nontrivial ways. Whereas the homotopy algorithm com-putes continuous solution paths, the expected Herﬁndahl indices in Figure 6 appear to change almost discontinuously in someplaces. This happens becausetheshapeof the transient distribution, and with it thevalueof theexpected Herﬁndahl index, changes abruptly as investment in certain states goes to zero. In particular, if investment in state11

is zero, then both ﬁrms are stuck at the lowest-possible quality level. As soon as x11 >0, however, the indus-try takes off, thereby giving rise to a nontrivial transient distribution that assigns positiveprobability to asymmetric industry structures. For example, with $=07 investment rises from zero to positive around /=217 to causethe abrupt change in the expected Herﬁndahl index in the left panel of Figure 6; with/=3 investment drops from posi-tiveto zero around$=074 in theright panel.

Figure7 illustrates theability of thehomotopy algorithm to crisscross the parameter space.12 _{It combines several} slices through the equilibrium correspondence to show how the expected Herﬁndahl indexHHI1000_{depends jointly on} the effectiveness of investment/and therateof deprecia-tion$.

To illustratewhy themultiplicity that is displayed in Fig-ure5 is not discerniblein Figure7, wediscuss thethree equilibria that wehavefound at /=18 and $=005. Across these equilibria, investment differs most in state

1515, where it is 0, 00092, and 00365. Table 7 presents the corresponding transient distributions+1000 _{in a subset} of thestatespacearound state1515. The differences

(14)

Figure 6. Transient expected Herﬁndahl index HHIT _at _T _∈ ₁₀₁₀₀₁₀₀₀ _along _/ _with _$₌₀_{7 (left panel) and}

along$ with/=3 (right panel).

0 3 6 9 12 15 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 α 0 0.2 0.4 0.6 0.8 1 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 δ HHI T T=10 T=100 T=1,000 HHI T

in investment lead to qualitative differences in industry structure; whereas the equilibria in the left and middle panels giveriseto transient distributions +1000 _with sym-metric modal state 1515, theequilibrium in theright panel gives rise to a transient distribution+1000_with asym-metric modal states 1516 and 1615. Despite this, the expected Herfindahl indices are virtually identical (and equal to 05000) because significant differences between thetransient distributions +1000 _{are limited to states in} which each firm has a quality level greater than -∗₌_12.

Figure 7. Transient expected Herﬁndahl index

HHI1000_along_/_and_$_.

0 5 10 15 ₀ 0.2 0.4 0.6 0.8 1 0.5 0.6 0.7 0.8 0.9 1 δ α HHI 1 , 000

For this subset of thestatespace, a changein thestate has a negligible impact on product market competition and, accordingly, on theHerﬁndahl index.

5.4. Scalability

To assess thescalability of HOMPACK90, wechangethe number of quality levelsM, and thus the number of equa-tions 2M2_{, and adjust thequality cutoff}_-∗_{accordingly. We}

trace out the equilibrium correspondence along/∈015

with$=07 held ﬁxed. From Table 8 it appears that the total computation time increases more than linearly in the number of equations. It is to be expected that the time per step increases in the number of equations because solving the systems of linear equations described in §3.1 becomes more burdensome. More surprisingly, the number of steps increases in the number of equations.

Table 7. Transient distributions +1000 _{for thethre}_e equilibria at/=18 and$=005.

15 16 15 16 15 16

15 0.6888 10−5 _{15 0}_{4274 0}₁₄₁₈ _{15 0}_{1921 0}₂₆₈₃

16 10−5 ₀ _{16 0}_{1418 0}₀₀₂₁ _{16 0}_{2683 0}₀₁₃₉ Note.Quality ladder model.

Table 8. Scalability: Normal ﬂow algorithm with sparse analytic Jacobian.

No. of Time No. of Time/step

M -∗ _equations _(h:m:s) _steps _(s)

9 6 162 0:00:15 931 002

18 12 648 0:24:27 7608 019 27 18 1458 5:08:27 21457 086

Note.Quality ladder model.