SELECTION-MIGRATION REGIMES CHARACTERIZED BY A GLOBALLY STABLE EQUILIBRIUM

SELECTION-MIGRATION REGIMES CHARACTERIZED BY A

GLOBALLY STABLE EQUILIBRIUM*

SAMUEL KARLIN+ A N D R. B. CAMPBELLS

+

Department of Mathematics, Stanford University, Stanford, CA 94305

$ Department of Mathematics, Purdue University, Lafayette, Indiana 47907

Manuscript received July 9,1979 Revised copy received November 2, 1979

ABSTRACT

The principle that a subdivided population subject to overdominance viability selection in each habitat will manifest a unique, globally attractng polymorphic equilibrium is posited. This follows as a corollary to the stronger principle that, if haploid selection or submultiplicative diploid selection (definition: the geo- metric mean of the homozygote viabilities is less than or equal to the heterozy- gote viability) is operating in each habitat,there is a unique, globally attracting stable equilibrium that may be monomorphic or polymorphic. These principles are proven for a broad spectrum of migration patterns. I n all such migration selection systems, multiple fixation states cannot be simultaneously stable under submultiplicative viability regimes. Contrasting examples where sub- multiplicative viabilities are not in force are given.

is the role of natural history in the genetic composition of extant W:oglations? T o what extent are allele-frequency configurations determined by the selection, migration, recombination and other biological parameters, as against sensitivity to initial conditions, founder effects, sampling and general random-drift fluctuations? These considerations are related to the question: For

a given set of genetic, environmental and population structure forces, when is there a unique, globally attracting equilibrium and what are its characteristics in terms of the model parameters?

This question has plagued geneticists who pofider the diversity of life. A classical phrasing might ask when the adaptive landscape of WRIGHT (1932;

see also DOBZHANSKY 1970, pp. 24 ff.) has a single versus several adaptive peaks or, more generally, what is the shape of the adaptive landscape. However, some of the concepts used to define the adaptive landscape (e.g., Hardy-Weinberg ratios, mean fitness of the population) are not appropriate in the multideme context; therefore. we phrase the question more generally.

The use of theoretical equilibrium studies for nonequilibrium situations is irequently challenged (e.g., THOMSON 1977). Although there certainly are cxamples of populations ( e . g . , some bacteria, fungi, deep sea fish) at equilibrium (perhaps in a stochastic sense) or close to equilibrium for significant durations, other populations (especially humans) are subjected to constantly changing

* Support in part by NIH Grant GMt0452-16 and NSF Grant MCS 80624-AOl.

1066 S. KARLIN A N D R. B. CAMPBELL

parameters that may not permit the allele frequencies to approach an equilib- rium. However, knowledge of the stable equilibrium configurations f o r appro- priate models provides some information on the dynamics of the system. One can infer from the equilibrium contingencies, to some extent, possibilities for the underlying selection or population-migration structure. A description of enough equilibrium states also provides a control f o r simulation or computation schemes used in studying related transient phenomena.

The global equilibrium structure is rather inaccessible under migration-selec- tion balance, and usually only certain aspects of it, especially the fixation states, have been studied (see reviews by

FELSENSTEIN

1976 and HEDRICK, GINEVAN and

EWING

1976). Clinal selection-migration patterns allow the study of poly- morphic equilibria and under some circumtances the global equilibrium struc- ture (e.g., SLATKIN 1973; NAGYLAKI 1976a,b). Incorporation of more forces governing genotype frequencies (e.g., assortative mating in CAMPBELL 1980) or stochastic elements (e.g., fluctuating selection intensities in GILLESPIE 1974 and

TEMPI ETON and

ROTHMAN

1978 or sampling in finite populations in SLATKIN and CHARLESWORTH 1978) significantly reduces the scope of the conclusions one can draw h o m the models.

I n the case of a single panmictic deme experiencing only viability selection at a single locus with two alleles, there is a unique, globally attracting fixation state in the case of directional selection and a unique. globally attracting poly- morphic equilibrium under overdominance selection. A general principle of wide applicability in terms of the model parameters states that superimposing recom- bination, bisexuality. o r multideme interactions on viability regimes prescribing globally attracting polymorphic equilibria (the local regimes may specify differ- ent equilibria in isolation) will retain the existence of a globally attracting poly- morphic equilibrium ( KARLIN 1979). Hence, viability selection, when present, is, in a sense, a major force determining the equilibrium structure of a population system.

I n this paper, we discuss the prospects that a mu1 tideme population structure subject to a selection regime entailing overdominance in each habitat linked by migration flow will evolve to a unique, globally attracting polymorphic equilib- rium. We also investigate conditions for the existence of a globally attracting equilibrium (not necessarily a polymorphism) without assuming overdominant selection in each separate habitat.

GLOBALLY STABLE EQUILIBRIUM 1067 on initial conditions (KARLIN 1977a). These realizations involve rather extreme selection coefficients. On the other hand, there is no evidence countering the principle that overdominance in each habitat (which produces a stable globally attracting polymorphic equilibrium in isolation) will maintain a stable, globally attracting polymorphic equilibrium for any level or form of migration super- imposed. We present several results buttressing a slightly stronger principle, as follows: If the local deme selection regimes express submultiplicative viabilities in the sense that the geometric mean of the homozygote viabilities is less than or equal to the heterozygote viability in each habitat, excluding the case of strict neutrality throughout, then there is a unique, globally attracting stable equi- librium with respect to the extended population range, which is either a poly- morphism or a fixation state. These circumstances include additive and multi- plicative allelic viability effects and the condition of local overdominance. We formally highlight these principles.

Principle I: In a multideme population system, if there is an overdominant

selection regime in each habitat, then there is a globally stable polymorphic equilibrium

for any migration structure (i.e., a unique, globally attracting polymorphism).

Principle 11: In a multideme population system, i f submultiplicative viabilities are manifested in each habitat, then :here is a globally attracting equilibrium (not necessarily a polymorphism).

The existence of a unique stable equilibrium entails less sensitivity to initial conditions and retains continuity with respect to the parameters delimiting the iorces. Catastrophies (sharp changes in solutions) cannot occur when a unique,

stable equilibrium exists for all specifications of the model parameters.

Because overdominance in each habitat assures that both fixation states are unstable, Principle I is subsumed in Principle 11. We verify Principle I1 under LEVENE migration (cf.,

KARLIN

1977a), when temporal selection variation is manifested in a single habitat, for a model of seeds manifesting facultative dor- mancy o r germination and in the general selection-migration model of two habitats.

The relevance of the principles can be better appreciated by contrasting the equilibrium configurations associated with natural classes of global selection regimes. We focus on four classes in order to address the scope of applicability of Principles I and 11. The hypotheses for Principle I1 include many cases in the first class as well as the second class.

1068 S. KARLIN A N D R. B. CAMPBELL

one homozygous type quite inconspicuous, while the other homozygous type, which is adapted to another background, is most visible. The difference in direc- tion of selection among habitats may be due to, e.g., ambient temperature or different background patterns as in the case of cryptic coloration.

Ouerdominance in each habitat: Overdominance (i.e., the heterozygote is more fit than both homozygotes) is the only means of maintaining a polymorphism in a single panmictic population, There are a paucity of currently documented cases of this nature (the standard cited example is that of the haemoglobin S

variant)

,

but the difficulties inherent to demonstrating selective mechanisms preclude removing overdominance from consideration. Models postulating that heterozygote enzyme activity levels are exactly intermediate between the homo- zygote activity levels, but fitness is a concave function of the activity level (GILLISPIE 1976), can produce overdominance where each habitat is a fine grained environment, due to an average heterozygote advantage over the life of an individual. Moreover, there appear to be cases of overdominance emerging in the multiallele context involving clusters of tightly linked genes; especially in the HLA complex and immunoglobulin systems

(BODMER

1978; see also KARLIN 1979). It is expected that the assertion of Principle I applies to these cases. Underdominance (disruptiue selection) in each habitat: The circumstance in which the heterozygote is less fit than either homozygote is the third manner in which selection can be manifested in a single habitat. When underdominance is acting in each habitat, both fixation states are stable and the equilibrium struc- ture for the corresponding multideme system may encompass many stable equilibria; which one is established can depend on initial conditions, historicity, or random effects. Underdominance may be appropriate for cryptically colored species in a heterogeneous erivironmen t where the homozygous phenotypes mimic backgrounds that their bearers can seek for protection, while the hetero- zygote bears a pattern that is conspicuous everywhere. Another motivation con- siders homozygotes as specialists that exercise habitat selection in fine-grained environments, while heterozygotes are generalists not adapted to any particular environment.

A mixture of ouerdominance and underdominance selection forms ouer the population range: This case exemplifies opposing selective forces prescribing complementary equilibrium configurations mediated by migration. It merits study as a complement to the study of opposing directed selection regimes. It is quite tractable in some special cases, which we delineate below. Of course, we also consider other mixtures of the three basic single-habitat selection regimes.

THE MODEL

We briefly review the standard one-locus, two-allele model for local viability selection medi- ated by migration among demes, the soft selection case (cf., CHRISTIANSEN 1974; KARLIN 1976). Viability selection acts within demes in accordance with the transformation equation

[ S i 2 2

+

xi ( 1 - - X i ) ]

[ S i X i Z

+

2xi (I-zi)

+

ti (I-&)

'1

GLOBALLY STABLE EQUILIBRIUM 1069

where the subscripts specify the demes to which the parameters refer; x i is the frequency of the A allele preceding reproduction; si, 1 and ti specify the relative viabilities of the AA, Aa, and UCI genotypes, respectively; and caret designates that time of the life cycle following selection, but preceding migration. Migration then completes the generation cycle [the prime (’) denotes the subsequent generation] by

where M is a constant stochaPtic matrix called the backward migration matrix. The specifications si

<

1

<

ti or t i

<

1

<

si, si,ti

<

1 and s,,ti

>

1 represent local directional selection, over- dominance and underdominance, respectively.

Remark: The particular specialization of (1) that has siti ’= 1 is called multiplicative allelic viability expression. Under this circumstance we may rewrite ( 1 ) as

which is clearly equivalent to haploid selection with viabilities si and 1 for alleles A and a, respectively, This simplification is fundamental to the later results.

RESULTS

We proceed with the formal statement of some results that reflect on the equi- librium structure of multiple deme genetic systems. They are of independent interest and, taken in concert, provide some circumstances under which the principles stated in the introduction are valid. The proofs are deferred to the appendices. We consider only two alleles at one locus.

Definitions: If, in ( l ) , sst%

<

1 or equivalently if the geometric mean of the homozygote viabilities is less than or equal to the heterozygote viability, we say that the viability array i s submultiplicative. Recall that haploid viabilities are equivalent to multiplicative viabilities with s p t i = 1.

We employ “stable” to mean strongly stable in the sense that there is local convergence to the equilibrium at a geometric rate (or formally that the spectral radius of the gradient matrix evaluated at the equilibrium is less than one). Analogously, we connote by “unstable” strong instability in the sense that there is divergence from the equilibrium at a geometric rate (or the spectral radius of the gradient matrix evaluated at the equilibrium exceeds one). The inclusion of the possibility that the spectral radius is equal to one into the following results entails some technical changes, but does not affect the qualitative picture.

Stability of the fixation states: If a transformation of the form (2) incorporat- ing viability selection and migration has a globally attracting equilibrium, then, in particular, both fixation states cannot be simultaneously stable. We address the stability nature of the fixation states as a first step toward delineating the global equilibrium structure and providing circumstances under which principles

I and I1 are valid.

1070 S . HARLIN .4ND R. B. CAMPBELL

fixation state can be stable for slight migration. However. for the LEVENE (1953)

migration pattern (total panmixia), both fixation states can be simultaneously stable with directional selection manifested in each habitat (KARLIN 1977a). In order to assure that at least one fixation state is unstable for general migration patterns, it is further necessary to restrict the selection regimes. One class of selection regimes precluding simultaneous stability of both fixation states is that of submultiplicative viabilities.

stales cannot be simultaneously stable.

Result I. W h e n subrnulliplicatiue uiabilities are acting in each habitat, both fixation

The specific sufficient criteria precluding simultaneous stability of the fixation states delineated below indicate that the conclusion of Result I holds under more general hypotheses than stated.

Case

i:

If the backward migration matrix. M , is doubly stochastic (i.e., the columns sum to unity, as well as the rows), a sufficient condition assuring instability of at least one of the fixation states is

This condition is certainly satisfied if s p t l

<

1 with strict inequality for some i

as characterizes submultiplicative viabilities.

Case

ii:

We can generalize Case i by introducing

<,

the principal left eigen- vector (which necessarily exhibits positive components) of M , normalized such that its components sum to 1. A sufficient condition precluding simultaneous stability of both fixation states associated with a n arbitrary stochastic matrix.

M ,

is

1 n

The specialization that M is doubly stochastic entails that d1

=

-

for all i. and hence Case i, is subsumed by Case ii.

Case

iii:

If M is positive (semi)definite and there exist positive definite diag- onal matrices E,,

E ,

such that EIME, is symmetric positive definite (this con- straint is always satisfied by 2 x 2 matrices with a positive determinant and is satisfied by symmetric stepping-stone migration if migration exchange from each deme is restricted to at most 5 0 % ) , then a sufficient condition precluding simultaneous stability of both fixation states is:

The arithmetic-geometric mean inequality

GLOBALLY STABLE EQUILIBRIUM 1071

be applied for general migration patterns, while (6) is only relevant for a

migration matrix entailing at most a moderate amount of dispersal.

Polymorphic equilibria whose existence can be inferred: The next result is somewhat technical in nature, and its utility for population genetic systems is primarily covered in the corollary that follows. The readily ascertainable equi- libria of ( 2 ) , in general, are limited to the two fixation states and, in the presence of special symmetry relations, additional polymorphic equilibria may be explic- itly determined. I t is necessary to exploit this information in order to character- ize as fully as possible the global equilibrium structure indicating, i n particular, where other equilibria must exist.

An important property of the allele-frequency array transformation (2) is its intrinsic monotonicity in the following sense. If two given frequency arrays,

x = (xl,

. .

.

,

x,)

and y = (yl, . . . , y,)

(xi

and yi are the A allele frequencies in deme

i),

are ordered x

<

y (meaning that

xi

<

y i for all

i

with strict ine- quality for some component; i.e., the y configuration entails a larger A allele frequency i n each derne compared to x), then the resulting frequency vectors

x’ and y’ of the next generation [see (2)] retain the same ordering (in fact, strict): x’ Q y’ (i.e., x’i

<

y’i for all i). This endowment for the migration-selection structure implies the following fact:

Result II: If there exist two stable equilibria, U and v, for ( 2 ) satisfying U

<

v (< desig- nates strictly less f o r all corresponding components), then there is an unstable equilibrium, y, satisfying U Q y

<

v. I f there exist two unstable equilibria, U and v satisfying U Q v,

then there exists a stable equilibrium, y, which is strictly less than a n y other equilibrium interior to the right parallelpiped {x[u

<

x Q v} defined b y U and v; this equilibrium satisfies U Q y

<

v.

There are many cases where polymorphic equilibria (stable o r unstable) do not satisfy a strict ordering relation (<), but the fixation states are necessarily well ordered with respect to any other equilibrium. The nature of the transfor- mation equations (2) usually prescribe only a finite number of equilibria and periodic or oscillating trajectories seem not to occur. I n particular, if both the

0 and 1 fixation states are unstable and there is a unique internal equilibrium, then there is global convergence to the polymorphic equilibrium.

The following corollary to Result I1 has particular interest for circumstances where we have tight bounds on the number of polymorphic equilibria.

Corollary: If both fixation states are unstable, then either (a) there is a unique, globally attracling polymorphic equilibrium, (b) there are one unstable and two stable polymorphic equilibria, or ( c ) there are more t h n three polymorphic equilibria, in-

clud‘ng at least :WO stable and at least two unstable polymorphic equilibria.

This assertion, in particular, implies that, if all polymorphic equilibria are stable, then there is at most one polymorphic equilibrium which is globally attracting. (We use this fact below.)

1072 S . K A R L I N A N D R . B. C A M P B E L L

manifested. It is also relevant to diploid selection because the transformation equation (2) coincides for haploid viabilities and diploid multiplicative allelic viability contributions, as noted in ( 3 ) . But the main utility of Result 111 is its immediate generalization to a spectrum of diploid viability regimes provided by Result 1V below.

Result I l l : Under the action of haploid viabilities (or equiualenlly multiplicative diploid Liability effects) in systems of two demes, there is a unique, stable equilibrium lhnt is glob- ally aitracling. This globally atiracting equilibrium m a y be a fixation slate, in which case there are no polymorphic equilibria; or a polymorphic equilibrium, in which case the only oiher equilibria are ihe two unstable fixation states.

The proof of this result is given in APPENDIX c . Although we have not proved

the analog for general migration entailing more than two demes, the proofs for several migration patterns that have been studied are included in APPENDIX C .

The migration patterns for which we have extended Result I11 include: Levene migration pattern: The case where the backward migration matrix is rank one entailing total panmixia in each generation was demonstrated by

KARLIN (1977a). He further demonstrated uniqueness of the polymorphic equilibriuni when the same allele is dominant in each habitat in addition to the submultiplicative selection patterns.

Temporal variation: We actually discuss in APPENDIX c the slightly more gen-

eral case where the backward migration matrix is any permutation matrix; systematic cyclical temporal variation is the most relevant subcase from the biological perspective. I n this case, there cannot be any polymorphic equilibria, except in the degenerate case where there is no net selection over a cycle of generations and the system is neutral.

Distinguished deme: These models have been discussed by KARLIN (1976, 1977b) and are of particular relevance to the study of seed pools (TEMPLETON and LEVIN 1979). They are characterized by a backward migration matrix that has only one row with multiple nonzero entries, They are similar to Leslie mat- rices occurring in life history and population dynamics studies. The proof in

APPENDIX c actually extends to the (slightly) more general case where some rows

are identical, as is characteristic of the LEVENE migration structure, while the other rows have only oiie nonzero entry, as characterizes permutation matrices. The proofs for the above migration patterns are evidence that Result I11 gen- eralizes to systems of several demes. In fact, we have not encountered any migra- tion patterns to which Result I11 does not extend. Our search for counterexamplei has included extensive numerical runs. Therefore, we posit its generalization as a principle that we formally state:

Principle I l l : 1Jndw [he action of haploid liabilities ( o r equivalently multiplicaiiue diploid uiability effects) with an arbitrary number of demes, there is a unique, siable equi- libriurn that is globrrlly atlraciing. This equilibrium m a y be a fixation state. in which case there are no polymorphic equilibria, or a polymorphic equilibrium. in which case the only

o ~ h r equilibria are the uns:able fixation sta:es.

GLOBALLY STABLE EQUILIBRIUM 1073

selection. Demonstrating uniqueness of a polymorphic equilibrium directly for specific diploid viability regimes (e.g., additive allelic contributions) is more diffi- cult than for the haploid case. However, in view of Result 11, we are able to extend Result 111 to submultiplicative viabilities; hence, demonstrating unique- ness for haploid viability selection suffices to establish uniqueness for diploid submultiplicative viabilities. We state this formally as:

Result IV: A n y migration structure (i.e., backward migration matrix) that assures the existence of a unique, globally attracting stable equilibrium under the action of haploid viability selection also assures the existence of a unique, globally artracting stable equi- librium when submultiplicative viabilities (the geometric mean of the homozygote viabili- ties is less than or equal to the heterozygote viability) are operating in each habitat.

In particular, this provides that if a polymorphic equilibrium under submulti- plicative viabilities exists, it is stable and globally attracting as occurs in the haploid case. The proof, which is deferred to APPENDIX D, demonstrates that any polymorphic equilibrium must be stable when Principle I11 is valid (e.g., in the circumstance of Result I11 and the other examples discussed afterward) and, hence, by Result 11 there is only one stable equilibrium.

SimuZation confirmation: Because the above analytic results do not encompass all migration patterns, we conducted some computer simulations for systems of three, five and eight habitats to buttress o w belief that Principle I1 is in general valid, i.e., that Principle I11 holds without qualification. No counterexamples were generated. We shall first discuss the results for three demes and then indicate how the five and eight deme examples complement the three-deme results.

With three habitats, 90 randomly generated migration matrices representing symmetric nearest neighbor stepping-stone, DEAKIN (homing superimposed on total panmixia) and general migration patterns were assigned randomly gen- erated haploid viability parameters. (Ten examples of the LEVENE migration mode were also generated.) The action of selection and migration was then iterated starting from frequencies near both fixation states [i.e., (0.01,0.01,0.01)

and (0.99,0.99,0.99) were taken as the initial frequencies]. I n all cases, the allele frequencies converged to the same equilibrium from both initial frequencies, precluding the existence of multiple stable equilibria.

If there is, indeed, only one stable equilibrium, it is of interest to consider whether there is a relationship between the nature of the selection regime or migration structure and whether the stable equilibrium is polymorphic (inter- nal) or monomorphic (a fixation state). With a linear stepping-stone migration model, where the

A

allele is favored in two habitats and disfavored in the third, one might expect polymorphism to be more likely if the favored habitats are adjacent, rather than at the termini; but the limited simulations we ran (which did not contrast two orderings for the same selection coefficients) do not suggest that any ordering provides better prospects for polymorphism independent of the migration structure.

1074 S. KARLIN AND R. B. CAMPBELL

rate, m, uniform on (0,0.25) ; the

DEAKIN

migration matrices were generated by choosing two random numbers ( x 1 , x 2 ) uniform ( 0 , l ) to create a rank-one stochastic matrix, R, with the row elements imin(z,,z,), Ixl-zzj, 1 - max(zl,xz)] and then choosing i(y uniform (0,l) and letting M =

aJ

+

(1-a)R; for the general

migration pattern each row was generated independently i n the manner of the row for R above.

For

the stepping-stone and

DEAKIN

migration patterns, nine of

30 and 10 of 30 of the unique stable equilibria were fixation states, respec- tively, whereas, for the general migration patterns, 21 of 30 of the stable equi- libria were monomorphic. (For LEVENE migration patterns, seven of the 10 stable equilibria were monomorphic.) This suggests that the stepping-stone and

DEAKIN

migration models have a relatively greater identification of demes with their habitats, perhaps due to the relatively larger homing tendency, which fosters polymorphism contrasting

with

the realizations for a general migration pattern of the total panmixia of the LEVENE migration pattern.

We

summarize the three deme results.

LEVENE Stepping-stone DEAKIN General

s+-z

<

1

<

$3 4/5 4/15 4/15 12/15

SpSQ

<

1

<

sz 3J5 5/15 6/15 9/15

The entries are the number of monomorphic stable equilibria divided by the number of parameter sets.

We did not investigate the role of environmental heterogeneity on the pros- pects for polymorphism for systems of five or eight demes. However, the nature of the migration structure appears to play a stronger role in the prospects for polymorphism than with three demes. For the stepping-stone migration model, only three of the 30 equilibria attained were monomorphic, while general migration patterns had 17 of 48 monomorphic equilibria, and the

LEVENE

migra- tion model produced 17 of 36 monomorphic equilibria. The migration matrices were generated analogously to the three-deme case. We note, in particular, that the greater tendency for monomorphism to the

LEVENE

migration structure is further reflected by the fact that, for two of the selection regimes, monomor- phisms for both fixation states occurred (five migration patterns were assigned to each selection regime). The results are summarized below.

LEVENE Stepping-stone General

Five habitats 9/20 1 / 1 5 6/25

Eight habitats 8/16 2 / 1 5 11/22

The entries are the number of monomorphic stable equilibria divided by the number of parameter sets. Migration patterns and selection regimes were generated randomly.

OVER/UNDERDOMINANT SELECTIOX REGIMES WITH A COMMON EQUILIBRIUM

If the selection regimes in all the habitats can be expressed in the form (cf.,

(8)

KARLIN 1977a)

GLOBALLY STABLE EQUILIBRIUM 1075 where a

>

0 is constant among habitats and si varies between habitats, then the frequency array maniEesting the

A

allele frequency a/( I+a) in each habitat is

an equilibrium that we shall refer to as the common polymorphic equilibrium. The choice of QI = 1 provides standard symmetric overdominance and under-

dominance. The ability to specify this particular equilibrium allows us t o explicate more fully the global equilibrium structure. I n particular, we prove Principle

I

for a selection regime of this nature. The contrasts of over- and under- dominant selection can be illuminated by considering overdominant, underdomi- nant and a mosaic of over/underdominant selection regimes separately. This was done for LEVENE migration by KARLIN (1977a, pp. 370,371). The analysis below allows a general migration pattern.

Overdominance in each habitat: When a global selection regime of the form

(8) manifests overdominance in each habitat (i.e., si

<

I for all

i),

then both fixation states are unstable and the common equilibrium is stable and globally attracting. The proof is given in APPENDIX E.

The above finding is structurally stable, meaning that the existence of a globally attracting polymorphism applies under small perturbations of the selec- tion parameters.

The “symmetry assumption” that a has the same value in each habitat enables us to specify the allele frequency at the common polymorphic equilibrium. This equilibrium value is used in the demonstration that the polymorphic equilibrium is unique. For the general problem of overdominance in each habitat, OL varies among the habitats and, although Result

I1

provides that there is at least one polymorphic equilibrium, we cannot evaluate it. Hence, the circumstance of (8)

enables a concise proof of Principle I; we believe that the principle holds for general overdominant selection regimes with any superimposed population structure.

Underdominant selection

in

each habitat: If a selection regime of the form (8)

manifests underdominance in each habitat, then both fixation states are locally stable and the common polymorphic equilibrium is unstable. The global equilib-

rium structure depends on the particular migration pattern. I n fact, under suffi- ciently small migration flow there are 2”-2 distinct stable polymorphic equilib- ria, in addition to the stable fixation states (KARLIN and MCGREGOR 1972).

If

the initial frequency is less than a/ ( 1 +a) in each deme, the population will converge

to the 0 fixation state; if the frequency is greater than ,a/( 1 +a) in each deme, the

population will converge t o the

1

fixation state. This provides lower bounds on the domains of attraction of the fixation states and precludes any cyclic or oscillatory behavior passing through these regions.

I n the two-deme model with underdominance in each deme, as many as seven polymorphic equilibria are possible with, at most, two polymorphisms stable and both fixations also stabIe (KARLIN and MCGREGOR 1972).

Mosaic of over- and underdominance on the population range: Under LEVENE migration, the cases of a unique, glcbally attracting common polymorphism

1076 S. KARLIN A N D R. B. CAMPBELL

state is stable, while the other fixation state and the common polymorphic equilibrium are unstable-a stable equilibrium between the two unstable equilibria completes the stability structure; and [b) both fixation states and the common polymorphic equilibrium are unstable-two stable equilibria separating the unstable equilibria complete the stability structure. Consideration of pure underdominance, as discussed above, shows that the situation for general migra- tion cannot be as simple. We can, however, draw some robust conclusions.

[A)

If at least one habitat manifests overdominance and at least one habitat manifests underdaminance, then both of the fixation states and the common polymorphic equilibrium must be unstable for sufficiently small migration [i.e.,

M is sufficiently close to the identity matrix

I).

(B) If the backward migration matrix, M , is positive semidefinite and can be made symmetric by multiplication by positive-definite diagonal matrices

[i.e., M = E,KE,, where E, and E, are positive-definite diagonal matrices and

K

is symmetric and positive definite [actually. positive semi-definite suffices for I()

1,

then stability of the common polymorphic equilibrium precludes stability of either fixation state. This was shown to be true f o r the LEVENE migration structure in KARLIN (1977a).

However. simultaneous stability of the common polymorphic equilibrium and both fixation states is possible with a generdl migration pattern. We present a n example representing temporal variation of two habitats in APPENDIX E.

[ C ) The above examples illustrate that under a mosaic of under- and over- dominance and general migration, the hypercubes bounded by the fixation states

(0 o r 1)

,

and the common polymorphic equilibrium a / [ 1 +a) may have interior stable or unstable equilibria. This contrasts the cases of universal overdominance and underdominance that preclude the existence of equilibria in those hyper- cubes and the LEVENE migration structure. which only permits stable equilibria in those hypercubes. However, monotonicity still provides that those hypercubes are mapped into themselves; hence, cyclic o r oscillatory trajectories cannot pass through them. In fact, because those hypercubes are mapped into themselves, they may be studied in isolation from the rest of the system.

DISCUSSION

The focus of this paper has been to demonstrate circumstances under which Principles I and I1 are valid, i.e., when there is a unique, globally attracting stable equilibrium under submultiplicative viabilities. Although we have not proved the principles for all migration patterns, we are n ot aware of any counter- examples. Examples of possible equilibrium structures where viabilities are not submultiplicative were presented for contrast. It is relevant to indicate various interpretations and implications of our results.

GLOBALLY STABLE EQUILIBRIUM 1077 pany most electrophoretic alleles if we believe that heterozygote activity levels are almost exactly intermediate between the homozygote activity levels and fit- ness is a concave function of activity levels (GILLESPIE 1976).

(B) When there is a unique, globally attracting stable equilibrium, it is natural to inquire as to the relation between the selection regime and migration pattern and whether it is monomorphic or polymorphic. I n an isolated deme, there is never a stable polymorphism with haploid viabilities. I n fact, this is true with an arbitrary number of alleles and/or loci, even with superimposed recombination. Hence, diploid multiplicative viabilites and, in fact, any mani- festation of directional selection will not maintain a polymorphism in an iso- lated deme, but overdominance will. This suggests that, as the geometric means in each habitat of the homozygote viabilities decrease, the prospect that the stable equilibrium is polymorphic should increase. This can be easily demon- strated if both homozygote viabilities decline with the mean, but does not hold in general. The effect of the migration structure on the location of the stable equilibrium is harder to quantify because of the problems inherent to ordering matrices. However, the simulation runs support the intuitive notion that the prospects for polymorphism should decrease with greater panmixia.

(C) A harbinger o€ Principles I and I1 was the fact that they were proven for the LEVENE migration pattern by

KARLIN

(1977a). If the same allele is domi- nant in all habitats, it Wac shown that there is a unique, globally attracting stable equilibrium under the LEVENE migration pattern. It was further shown that both fixation states cannot be simultaneously stable under a general migra- tion pattern if the same allele is dominant in all habitats (KARLIN 1977b). This suggests that the analog of Result I11 might hold for the case of dominance. If

this is valid, there is a notion of subdominant viability analogous to submulti- plicative viability that will generalize Result IV. The caveat should be added that the same allele must be dominant in all habitats; if different alleles are dominant in different habitats, it is known that multiple stable equilibria are possible.

(D) It is not a trivial matter to extend these results to multiple alleles and/ or multiple loci, or even to interpret them in that context. We mentioned above that, in a single habitat, haploid selection precludes polymorphism consistent with the one-locus, two-allele case. Diploid directional selection is easily defined, providing a unique stable equilibrium characterized by fixation of the fittest gamete in an isolated habitat, but the concept of overdominance is more recondite

(KARLIN

1978). Pairwise overdominance (i.e., each heterozygote is more fit than the corresponding homozygotes) will not assure a unique stable equilibrium in

1078 S. KARLIN A N D R. B. CAMPBELL

LITERATURE CITED

BODMER, W. F., 1978

BULMER, M. G., 1972

CAMPBELL, R. B., 1980 Polymorphic equilibria in subdivided populations with assortative

CHRISTIANSEN, F. B., 1974 Sufficient conditions for protected polymorphism in a subdivided

CHRISTIANSEN, F. B. and M. W. FELDMAN, 1975 Subdivided populations: A review of the one-

DOBZHANSKY, TH., 1970 Genetics of the Evoluiionury Process. Columbia University Press, New

FELSENSTEIN, J., 3 976 The theoretical population genetics of variable selection and migration.

FRIEDLAND, S. and S. KARLIN, 1975 Some inequalities for the spectral radius of nonnegative

GILLESPIE, J., 1974

The HLA System. Proc. Fifth Int.’l Cong. Human Genet. (Mexico City,

Multiple niche polymorphism. Amer. Nat. 106: 254-257. 1976) Excerpta Medica, Amsterdam.

mating. Theor. Pop. Biol. 18: (in press).

population. Amer. Nat. 108: 157-164.

and two-locus deterministic theory. Theor. Pop. Biol. 7: 13-28.

York,

Ann. Rev. Genet. 10: 253-280.

matrices and applications. Duke Math. J. 42 : 459-490.

1976

acterization of the fitness function. Amer. Nat. 110: 809-821.

Polymorphism in patchy environments. Amer. Nat. 108: 145-151. -, A general model to account for enzyme variation in natural populations. 11. Char-

HEDRICK, P. W., M. E. GINEVAN and E. P. EWING, 1976 Genetic polymorphism in heterogene- ous environments. Ann. Rev. Ecol. Syst. 7: 1-32.

KARLIN, S, 1976 Population subdivision and selection interaction. In: Population Genetics and Ecology. Edited by S. KARLIN and E. NEVO. Academic Press. New York. pp. 617-657. ---,

1977a Gene frequency patterns in the Levene subdivided population model. Theor. Pop. Biol. 11: 356-385. --, 1977b Protection of recessive and dominant traits in a sub- divided population with general migration structure. Amer. Nat. 111: 1145-1162. --, 1978 Theoretical aspects of multilocus selection balance I. pp. 503-587. In: Sfudies in

Mathematics and Biology. Edited by S. LEVIN. Mathematical Assoc. of America. Washing- ton, D.C. -, 1979 Principles of polymorphism and epistasis for multilocus systems. Proc. Natl. Acad. Sci. U.S. 76: 541-545.

KARLIN, S. and U. LIBERMAN, 1975 Random temporal variation in selection intensities one-locus

KARLIN, S. and J. L. MCGREGOR, 1972 Application of the method of small parameters t o multi-

LEVENE, H., 1953 Genetic equilibrium when more than one ecological niche is available.

NAGYLAKI, T., 1976a Clines with variable migration. Genetics 83: 867-886. --, 1976b The relation between distant individuals in geographically structured populations. Math. Biosci. 28 : 73-80.

two-allele model. J. Math. Biol. 2 : 1-1 7.

niche population genetic models, Theor. Pop. Biol. 3: 186-209.

Amer. N2t. 8 5 : 331-333.

SLATKIN, M., 1973

SLATKIN, M. and D. CHARLESWORTH, 1978

TEMPLETON, A. R. and D. A. LEVIN, 1979

TEMPLETON, A. R. and E. D. ROTHMAN, 1978

Gene flow and selection in a cline. Genetics 7 5 : 733-756.

The spatial distribution of transient alleles in a sub-

Evolutionary consequences of seed pools. Amer. Nat.

Evolution in fine grained environments: I. Environmental runs and the evolution of homeostasis. Theor. Pop. Biol. 13: 340-355. divided population: A simulation study. Genetics 87: 793-810.

GLOBALLY STABLE EQUILIBRIUM 1079

THOMSON, G., 1977 The effect of a selected locus on linked neutral loci. Genetics 85: 753-788.

WRIGHT, S., 1932 The roles of mutation, inbreeding, crossbreeding, and selection in evolution.

Corresponding editor: M. NEI

Proc. VI Int’l. Cong. Genet., I: 356-366.

APPENDIX A

The proof of Result

I

follows directly from the lower bounds on the spectral radii of product matrices given by

FRIEDLAND

and

KARLIN

(1975)

(cf.

KARLIN

1977b).

If

M

i s a n n X n stochastic matrix and

D

is a positive definite

n

X n

matrix with zero entries off the diagonal, then

where p designates the spectral radius and

5

is the principal left eigenvector of

M

(which is necessarily nonnegative), normalized so that

&

= 1. When

sub-

multiplicative viabilities are in force, stability of the fixation states 0 and

1

depends on the magnitudes of (BULMER 1972;

CHRISTIANSEN

1974; KARLIN 1976)

?&

k 1

p[M

diag(t-l)] and

p[M

diag(s-l)], (A.@

respectively, where diag (.) specifies the matrix with the indicated entries on

the

diagonal and 0’s off the diagonal and inverses are taken componentwise. Since

siti

<

1, then (siti)Pi

<

1. It follows with the help of

( A.l)

that the product of

the spectral radii in (A.2) exceeds 1 and, hence, at least one of the spectral radii is greater than one.

The

argument with respect to (6) is similar, based on the inequality

which is applicable whenever M is equivalent to a positive definite matrix (e.g., DEAKIN or stepping-stone model with local exchange rate less than

50%).

APPENDIX B

The proof of Result I1 relies decisively on the monotonicity of (2). In par- ticular, if

2

>

y, then G(x)

>

x(y) and, therefore, x‘(x)

>

x’(y). Accordingly, if there are two fixed points u,v satisfying U

<

v and y is constrained such that

U

<

y

<

v, then. by virtue of monotonicity, the transformed vector y’ = ~ ’ ( y )

obeys U = x’(u>

<

y’

<

x’(v) = v. This property asserts that the rectangular parallelpiped

K

= {x(u

<

x f v} is mapped into itself. It also provides that any point greater than the equilibrium point U and less than some point i n the

domain of attraction of U lies in the doman of attraction of U.

If U Q v are- stable equilibria, consider the boundary (denoted aD) of the

domain of attraction of U intersected with K . It has positive n-I dimensional

1080 S . KARLIN A N D R. B. CAMPBELL

continuity) contains points in the domain of attraction of x and points not attracted to U (hence a point in do). The intersection of

X

I

with {xlx

>

U} is

homeomorphic to the n-1 dimensional ball, since every ray originating in

K

at

U contains exactly one point in dD by monotonicity of the transformation (2).

(Monotonicity also guarantees continuity.) Monotonicity and continuity provide that aD n K is also homeomorphic to the n-I dimensional ball. Since

aD

rlK inherits closure and the property of mapping into itself from its defining sets, we may apply BROUWERS fixed-point theorem, which guarantees the existence of a fixed point in. XI

n

K .

We may repeat this procedure until we generate an unstable equilibrium, which must occur since an accumulation point of stable equilibria must be an equilibrium that cannot be stable.

If U

<

v are unstable equilibria, then, by monotonicity of the transformation

(2), successive iterations of the transformation on points in K near U (near v)

increase (decrease) monotonely to an equilibrium. [By the Frobenius theory of positive matrices, the principal eigenvector of the linear approximation is strictly positive (assuming that

M

is irreducible and the local transformations are strictly monotone) ; therefore, after a few preliminary iterations, successive iterations will increase (decrease) strictly.] This (or these) equilibria y are necessarily distinct from U and v are strictly less (greater) than any other equilibrium in

K by monotonicity.

The Corollary follows by considering the individual cases. If both fixation states are unstable and there is an unstable polymorphic equilibrium, then there must be a stable equilibrium in each of the smaller parallelpipeds defined by the unstable equilibria. If there are multiple stable polymorphic equilibria, at least two of them must be well ordered because of convergence from near zero to an equilibrium strictly less than all others. Hence, there must be an unstable equi- librium between the two well-ordered stable equilibria. If there are at least three stable equilibria, one must be strictly less than the others by convergence from

0, one must be strictly greater than the others by convergence from 1, and, hence, one must lie between those two. The parallelpipeds defined by those three equi- libria entail the existence of at least two unstable equilibria.

We repeat the observation that under the circumstance of two unstable boun- dary equilibria and a unique internal equilibrium, points near the boundary equilibria must converge to the internal equilibrium and hence all internal points must converge to that equilibrium by monotonicity.

APPENDIX C

GLOBALLY STABLE EQUILIBRIUM 1081

This case is most expeditiously treated by recalling that the inverses of linear fractional transformations are linear fractional transformations so that the trans- formation equations may be written as

sx

= (l-m)z+my, tY = J . A s + ( l - p ) y ,

sz+ ( 1 -s) _{tY+ ( 1 -y}

1

which provide, respectively,

(s-l)s(l-s) - (t-1) Y ( 1 7 ) m (ss+l -s) - y-2, x-y= _{P ( tY+ 1 7 )}

Equation (C.2) gives

s[ (s-1) (l-z)+m(sx+l-x)]

_,

(l-x) [ (l-s)x+m(sx+(l-z)] l - y =

m (ss+ 1 -z) m[sx+l-s]

Y =

Substituting these expressions into the equality implied by (C.2) and factoring out z (1 -5) gives

(C.3)

(s-1 ) p{ t s [I (s-1) (1 -s) +m ( s s f 1 -z)

3

+

(1-s) [ (1 -s) s f n (sz+l-x] }

+(t-1) [(s-1) ( 1 3 ) + m ( s s + l - s ) ] [ ( l - s ) s + m(ss+(l-x)]

= o

.

We assume s

>

1

>

t because s,t

>

1 is a trivial case.

We evaluate the resulting quadratic at

x*

=L

<

0, which provides sx

4-

I - s = O and at

x**

=

>

1, which provides (s-1) (1-z)

4-

1 -s

(1 -s),-m (l-m) (1-s)

m[sx+l-x] = 0. At z*, the left side of ( C . 3 ) is (t-l)s(l-p), which is negative. At

x**,

the left hand side is (s-1)p __-- ms

,

which is positive. Hence, there is

(1-m)

1 -s-m

and _______

.

Since

x*

is negative and

x**

is positive

a root between

-

1

l-s (1-ml(1-s)

under the assumption s

>

1

>

t,

'which we made without loss of generality, it follows that all the roots of the original quartic are real; hence, as parameter values are changed, polymorphic equilibria can occur only by passing through (bifurcating off) one of the fixation states. If s = 1 and t

<

1, there are no poly- morphic equilibria, 0 is stable and 1 is unstable. As the magnitude of s is increased, 0 must become unstable before 1 becomes stable and a stable poly- morphic equilibrium bifurcates off 0 when it becomes unstable. No further polymorphic equilibria can appear as s increases until 1 becomes stable, which for s larger remains stable.

Since the proof for the

LEVENE

migration pattern is available

(KARLIN

1977a), we next verify Principle I11 for the case of temporal variation.

1082 S. K A R L I N A N D R . B. CAMPBELL

sition of n linear fractional transformations in the case of haploid selection. which is a linear fractional transformation. Fixed points of the composite linear fractional transformation are exclusively 0 and 1, except when the composite transformation is the identity mapping. [This is equivalent in the notation of

( 3 ) to

, n

si =

1.1

a = i

This latter contingency presents a degenerate situation.

In

all other circum- stances, only the fixations constitute equilibria possibilities. The a allele fixation is globally stable when si

<

1. and the outcome is A fixation in the circum- stance of ,TI si

>

1.

The distinguished deme migration model can be considered as a hybrid between the LEVENE migration model and temporal selection variation (permutation migration). The analysis consists of a modification of their proofs.

n

2 =1

n

Z = 1

A P P E N D I X D

Proof of Result

IV

(assuming Principle

111)

For ready reference, we restate the corollary to Results

I

and 11.

Corollary D.I: Under the supposition that a haploid selection regime exhibits at most one polymorphism, then if such exists. it is globally stable. Otherwise one of the fixation alternatives is globally attracting.

The hypothesis of the corollary has been validated for several migration models as set forth in APPENDIX c.

Recall the hypothesis that s,ti

< 1

for all i. Consider an equilibrium vector

x* = (z: ,I:,

. . .

,z,*) where

x:

is the equilibrium frequency in the ith deme. Let

si ( 2 2 ) 2

+

xx (1-2:)

Set si = s i

+

h, ti = ti

+

k, with h and k nonnegative and determined by the

conditions

g,f. 2 1= 1

(D.2)

h(zy(1-y:) = k(l-2:)Zyf

.

(D.3)

and

Il/ranifestly, (D.2) and (D.3) are possible for 0 <

x:

<

1 and 0

<

y: C 1, Consider the modified selection regime with fitness parameters in deme

i

aa

c

AA Aa

&

1 ti

.

_(D.4)

This expresses multiplicative allelic effects [by (D.2)]. The relation (D.3)

implies that f i

(2)

= y:: i = 1,2,

.

.

. ,

n . Therefore, X * = ( z * , x * ,

. .

.

,z,t)

GLOBALLY STABLE EQUILIBRIUM 1083 We claim that. using the prime (') to denote differentiation in what follows,

fi

(Xi*)

>

fi

( Z f ) *

(D.6)

Indeed, observe first that

(D.7)

c 1 1 z

1

1

f ; ( O ) =-<-=f'f'(0) and

yi(l)

=.;;-<-=fi(l)

.

2

ti si q

A direct comparison shows that f.(x) and &(z) intersect exactly once x (at 2:)

for

x

confined to the open unit interval [fi (0) =

5

(0). = 0, f i (1) = f i (1) =

11.

These facts manifestly validate the claim of (D.6).

With (D.6) i n hand, an elementary de_duction comparing the principal eigen- values of the matrices Mf'(x*) and Mf'(x*) indicates that any stable poly- morphism for $ = ['l(z),~(x),

. . .

, f , ( x ) ] i s also stable for f = [ f l ( x ) , f ~ ( x ) ,

.

.

.

,fn

(x)

] with submultiplicative viabilities. By hypothesis, a polymorphism under haploid selection is uniquely globally attracting. This analysis establishes that with submultiplicative viabilities every polymorphism is strongly stable. I t follows i n cognizance of Result

I1

that either a globally stable polymorphism exists or no polymorphic equilibria exist and one of the fixation states is globally attracting.

APPENDIX E

In order to show that overdominance of the iorm (8) i n each habitat provides a globally attracting polymorphic equilibrium, we shall show that the equi- librium characterized by the common A allele frequency (a/( l+,a) in each deme

is the only polymorphic equilibrium and, hence, by the corollary to Result

11,

is

globally stable. To this end, introduce the frequency variables yi =

xl.

-

CY/ (14-a)

,

specifying the signed deviation of frequencies from the equilibrium a/( l+a)

.

Then,

n

(E.1)

q ( q - I ) (l-,.)

+

l - ' a + s j a six;

+

22j (1

-xi)

+

(1 -CY+aq) (1 -zj)2

Yj

.

The denominator in the quotient is positive on (0,l) and concave in

xi,

while the numerator is linear. Hence, the quotient achieves its maximum at one of the endpoints; in fact, the ratio is equal to one for x i = 0 and

xi

= 1. Thus, excluding the fixation states, an equilibrium y must be an eigenvector of a substochastic matrix with eigenvalue one, an impossibility. Thus,

y

= 0 specifies the only polymorphic equilibrium, which must be stable and globally attracting by Result

11. since both fixation states are unstable.

I n order to address the stability of equilibria under circumstances other than overdominance throughout, it is useful to recall that,

_-

for the local transformations

2s@

+

l-a

,

8

(1) = (l-4Y+si(Y)-1 (without migration), f : (0) =si',

8

(&)

=

1084 S. KARLIN A N D R. B. CAMPBELL

equilibrium is unstable (actually repelling in all directions) when there is under- dominance throughout (si

>

1 f o r all ;).

Convergence to 0 (or

1)

when -si

<

(>)

-

for all

i

follows from considering

.

(E.2)

The numerator is a cubic with zeros at 0, a/(l f a and 1 and, hence, changes sign

at those values and nowhere else. The denominator is positive on (0,l). Evalu- ation at x=F (x=I - e ) for F sufficiently small shows f i

(x)

<

( >)s for Z< (>)

a / ( l + U ) and, since this holds in all habitats and M is stochastic, convergence ensues. More rigor requires the demonstration that f i ( x ) -

x

is bounded away

from zero outside neighborhoods of the fixation states.

With a mosaic of over- and underdominance instability of the fixation states and the common polymorphic equilibrium persists for sufficiently slight migra- tion since, when one of the entries in diag(f') is greater than one and

M

is sufficiently close to the identity, one of the entries on the diagonal of the product matrix M diag(f') is greater than one, which suffices to assure an eigenvalue greater than one by positivity.

The lower bound on the spectral radii of the product of symmetrizabIe positive semi-definite stochastic matrices with a positive diagonal matrix, cf. (A.3)

,

in terms of the left eigenvector of M provides that it suffices to consider the LEVENE migration pattern (where this bound is sharp) in order to preclude simultaneous stability of the common polymorphic equilibrium and the fixation states. This case is shown in KARLIN (1977a).

W e illustrate simultaneous stability of the common polymorphis equilibrium and both fixation states in the case of temporal variation involving two habitats so that

M

=

(

o>

.

Let rr = 1 in (21) and s1 = 0.25, sz = 5, so that symmetric over- and underdominance is manifested in the two habitats, respectively. Then, at either fixation state, stability depends on

a l+ff

sixz+-s(i-x) -ssi-s3-29(i-x) - (I--~+&~)Z-(I-X)Z

s,zz

+

22( 1 -2)

+

(l-a+rrsi) (l--s)

f i ( 2 ) - x = -

0 1