ANALYSIS OF SOME TWO-LOCUS SYSTEMS FOR TRAITS EXHIBITING CONTINUOUS VARIATION

ANALYSIS OF SOME %WO-LOCUS SYSTEMS FOR TRAITS EXHIBITING CONTINUOUS VARIATION

DORIT CARMELLI

Department of Medical Biophysics and Computing, University of Utah, Salt Lake City, Utah 84112

Manuscript received June 26, 1979 Revised copy received September 17, 1979

ABSTRACT

Some generalized two-locus, major-gene models for traits exhibiting con- tinuous expression are investigated. Conditions of gametic equilibria are stated as functions of the parameters characterizing the within phenotypic-genotypic distributions. Selection is stabilizing in favor of a n optimum phenotype. It is established that a n increase in the phenotypic variance facilitates stability of a central polymorphic state with tighter linkage. Also, with increased pheno- typic variance in an additive two-locus selection model, a HARDY-WEINBERG

type of equilibrium will be less central. The rate d convergence, i n this case, is slowed down with an augmented environmental background.

HERE exists a point of view suggesting that, in many cases of observed quantitative traits, most of the variability can be ascribed to a few major

loci, plus a number of independent environmental factors. This does not preclude the existence of numerous other genes that each contribute minor effects to the overall variability. The study of quan titatiT-e inheritance in these cases reduces to the analysis of suitable major-gene models meshed with the multifactorial background.

KARLIN and CARMELLI (1978) investigated the changes in gene frequencies of some one-locus models, as depending on the background distributions asso- ciated with different genotype classes, under various forms of selection functions acting on the individual phenotype. In this paper, some two-locus classical models are integrated into the framework of the generalized major-gene hypotheses.

In this respect, the study of multi-locus genetic systems, embodying various interactions of selection and linkage, permits more versatility in the range and form of the genotypic-phenotypic associations; they also provide the natural bridge between the dichotomy of one locus and polygenic inheritance (a large number of genes). Moreover, classifications and characterizations pertaining to the dynamics and equilibrium behavior in n-locus theory, incorporating com- ponents of selection-recombination events, nonrandom mating patterns and facets of population structure are increasingly amenable to theoretical analyses, numerical simulation and relevant interpretations (e.g., KARLIN 1977, 1979).

1002 D. CARMELLI

Accordingly, we would expect progress from this approach on the problem of the mode of inheritance of complex traits, as well as the investigations of the effects of the background distribution on the nature of the equilibrium at the major loci.

Complementing the sensitivity studies in KARLIN and CARMELLI (1978), I

investigated in this study how the central values (i.e., the means or modes), as compared to the variance parameters, affect the changes in gamete frequencies of

some two-locus major-gene models. Inter alia, we will deal with the following problems: (1) What is the range of recombination rates as a function of the phenotypic-genotypic distribution where stable polymorphisms might exist? (2) How do the parameters of the phenotypic-genotypic distributions influence the expected disequilibrium value at a polymorphic state? ( 3 ) What are the conditions on the parameters of the phenotypic distributions ensuring segrega- tion at one of the loci involved, while the second locus is monomorphic?

T o keep this paper complete in itself,

I

review the notation and conceptualiza- tion of the generalized major-gene model, as set forth in KARLIN and CARMELLI

(1978).

I assume that the phenotypic value is a real variable (multivariate cases are

also accessible) and stipulate that the phenotypic expression of the genotype class

i

is governed by an appropriate density f i (z)

.

I n this formulation, the com-

plex background influences inherent tc the genotype-phenotype associations are summarized by the functions f Z (x)

,

which are assumed to be approximately time invariant. These distributions can accommodate variable expressitivity, forms of penetrance and pleiotropic effects of other genes. I n the present investi- gation, the background distributions are assumed to follow Gaussian densities.

I

indicate a number of possible generalized two-locus modeIs involving the four basic gametes: A B , Ab, aB and ab. Of the many different ways that the ten possible genotypes may be partitioned into phenotypic subgroups,

I

investigated only a few that represent simple examples in which two loci may interact.

A. Standard additiue allelic and independent loci efects: The simplest version has allelic values for A and B equal to 1, and alleles a and b confer value 0 , in- ducing five classes of genotypes.

1

2

3

4

5

ab

A B Ab Ab aB

A B

A B A b

’

aB ab

’

aB Ab aB ab ab

-

ab

Ab aB

~

__

--

-

-

A B A B - _ _

--

and associated phenotype distributions f Z (z)

,

i

= 1,2,3,4,5.

B. Aggregate heterozygosity determinations:

All homozygotes One heterozygous locus Double heterozygotes

AB Ab

A B A b aB ab A B aB A B

Ab

AB’=

’ a B ’

-

ab A b ’ a 2 b ’

’

ab

- -

ab

’

aB

In the corresponding major-gene setting, the three classes of genotypes generate three phenotypic distributions.

TWO-LOCUS SYSTEM 1003

determination, and if recessive at this locus, then dominance at the second locus

{ B,b} becomes decisive. The corresponding grouping has three phenotypic classes,

viz.,

1 2 3

ab AB AB Ab AB AB A b Ab

Z ' A b ' A b ' Z ' a b ' a B ' a b

aB

'

ab ab

The major-gene models,

B

and

C,

express levels and forms of phenotype epistasis. As demonstrated later for model

B,

the extent of linkage between the two major loci becomes a determining factor.

To study the changes in gamete frequencies under selection forces, we stipu- late that selection (natural and/or artificial) acts on the phenotype as described by a function +(z) that effcctively prescribes the relative viability of an indi- vidual carrying phenotype value z. It follows that the ith genotypic class is endowed with average relative viability value

-

aB

aB

- -

- - -

By judicious assessments of +(z), we can incorporate forms of stabilizing or directed selection schemes motivated from concepts of evolutionary trends or

disease liability criteria. In this context, stabilizing selection means that

+

(5)

attains an absolute maximum at one or more intermediate values. Directional selection conveys an enhanced survival probability with increased phenotype value so that, as z traverses --OO to + w , + ( x ) ranges from 0 to 1.

Definition (1.1) allows for a parametrization of the fitness values in terms of means, variances and other properties of the genotype-phenotype distributions

coupled to the means, modes and range of the phenotype selection function.

In

order to appraise the effects of differential viability selection on the evolutionary outcome, we must translate relations with respect to { f 4

(x)}

and +(x) into com- parisons among the fitness values 7;. For sensitivity and robustness properties regarding the corresponding evolutionary aspects of the one-locus two-allele model, the reader should consult KARLIN and CARMELLI (1978).

A GENERALIZED TWO-LOCUS SYMMETRIC SELECTION MODEL

Generalized symmetric selection is an extension of the notions of symmetric under and overdominance to multiple loci (see KARLIN 1977). I n this frame- work, fitness depends on which loci are homozygous or heterozygous and other- wise is not influenced by the allelic components at the loci. We confine the dis- cussion in this paper to the case of two loci carrying two alleles each, without accounting for position effects of the heterozygous loci. The prototype fitness matrix has the form

AB A b aB ab

1004 D. CARMELLI

I n the case above, three phenotypic classes, Po, P, and P,, are involved, char- acterized by the aggregate heterozygosity over loci. Thus, all genotypes of phenotypic class Po are double homozygotes, those of class P I single heterozygotes and those of class P, double heterozygotes (model B).

The plasticity of the trait’s value for the three phenotypic classes, Po, P I and P,, is governed by the densities f o

(x)

,

f l (z) and f 2 (1). The corresponding mean and variance parameters are { d o , u ~ } , {dl,af} and {dz,az}, so that, with a prescribed selection function +(z), the fitness values

{rl.},

i

= 0, 1, 2 are determined as in

(1.1).

The complete equilibria configuration of the selection model (2.1) in terms

of relationships among the fitness coefficients yo, yl and y 2 is familiar (e.g., see

LEWONTIN and KOJIMA 1960;

KARLIN

and

FELDMAN

1970). We review some of the basic facts relevant to our analysis.

The case of light linkage: With complete linkage ( r = 0)

,

the four gametes, A B , Ab, aB and ab, are equivalent to four alleles, and (2.1) is the matrix of a single-locus four-allele system. I n this case. the classical single-locus theory is sufficient for the verification of the equilibrium structure. For the viability regime (2.1), the stability regions of the various equilibrium types with no recombination are delineated by linear inequalities among the fitness parameters

yi (see

KARLIN

and LIBERMAN 1976, Table 1 )

.

Central polymorphism: The population state that assigns equal frequency (in this case

%)

to all haplotypes is an equilibrium that we refer to as the central equilibrium. The central equilibrium exhibits linkage equilibrium or no associa- tion between the two loci. It is well known (LEWONTIN and KOJIMA 1960; BODMER and FELSENSTEIN 1967) that for y2

>

yo, this equilibrium is stable, provided

>

r* = Y 2

+

Y O

-

2 Y l

4 Y 2

In the range ( O ,r* ) , generally, two stable polymorphic equilibria exist having marginal allele frequencies pA = pB = 1/2 and carrying

D

# 0. The exact value of the linkage disequilibria at these points is

b = & % [ l - yz

+

4yzr yo - 2y1

1

‘/e

.

Note that

]dl

in ( 2 . 3 ) as a function of the recombination rate r is monotone decreasing, so that with tighter linkage, stronger association between loci is expected.

We consider next, for convenience, phenotypic densities following Gaussian distributions within each genotypic class

i,

( i = 0,1,2) distinguished by their mean value,

di,

and variance U : . We also assume optimizing selection acting on the

TWO-LOCUS SYSTEM 1005

.

An explicit evaluation of yi on and corresponding environmental variance

the basis of (1.1) yields (cf., KARLIN and CARMELLI 1978)

With relatively strong selection, that is, environmental variance U;-+ 0, then

Yi’fi(2)

i=o,

1 , 2

.

(2.5)

Thus, the phenotypic density evaluated at the optimal phenotype prescribes the appropriate genotypic fitness value. In this circumstance, the equilibrium struc- ture determined by relationships among the yi will depend on comparisons among the values of (2.5).

Stability conditions of boundary equilibria for tight linkage: Under weak selection, such that +:U 0 0 , we demonstrated in

KARLIN

and CARMELLI (1978),

that the fitness comparison yi

>

yi holds if and only if

(2.6)

In this respect, with complete linkage ( r = 0) and weak selection, the conditions of stability of a boundary type equilibrium, e.g., {AB,Ab} are assured when

y Z

<

yo

<

yl or

U;

+

(z-d+)2

<

U;

+

(z-dj)z

.

U;

+

(z-dx) 2

<

U,”

+

(z-do) 2

<

cr;

+

( 2 4 , )

.

(2.7)

Similarly, total fixation of the gamete type { A B } occurs if y o

>

max(yl,yz), which is equivalent to

U,”+ ( z - d o ) Z

<

min{u,2+ ( ~ - d ~ ) ~ , U;+ ( ~ - - d ~ ) ~ }

.

_(2.8)

Conditions (2.7) and (2.8) assert the essentiality of both the mean values and variances as determinant in evaluating fitness relationships.

I n the case of equal class variances such that uo = u1 = u2, the evaluations

jz-di( serve as key determinants, so that the largest fitness is attained with minimal deviation. For approximately equal mean phenotypic expression,

do z d , z d,, then independent o,f the optimal value and the common central phenotype prescription, the genotype with the minimal phenotype variance carries superior fitness. On the other hand, in the presence of strong selection where y i z f z (z)

,

the determining criterion involves a “weighted mixture” of the variance and its reciprocal. When ~i are all large, a canalizing principle again emerges, and the smallest among the confers the maximum fitness among the genotypes. In sharp contrast, where are all small and, provided

( z - d ) 2

>>

0, the largest variance confers the maximum fitness among the genotype classes.

1006 D. CARMELLI

a stable central polymorphism. To help elucidate this matter, we consider U: = U: = U," = 2 . W e assume that the genotype-phenotype density distributions { f i ( z ) } i = O , 1, 2 differ only in the mean values d,

( i = O ,

1, 2) and Iz-dzI

<

jz-d,

I,

which is equivalent to y z

>

yo.

Result: (1)

For

( ~ - d , ) ~

<

( z - & ) ~

<

( ~ - d ~ ) ~ , the central equilibrium is stable for r = 0 and, hence, stable for all r, independent of the phenotypic variance.

(2) If (z-d2)2

<

(z-d,)2

<

(z-do)" and 2(z-d,)'

<

(z-do)z

4-

(z-dZ)',

then the central equilibrium is always stable, provided the phenotypic variance

is large enough.

(3) When (z-d,)

<

(z-d,)

<

(z-do) and 2 (z-dl)

>

(z-d,) f (z-d,)

',

then the central equilibrium is stable f o r I

>

I*, where I* is determined by

(2.2). In this case, a larger phenotypic variance extends the range of recom- bination rates where stability of the central equilibrium prevails.

We sketch the proof. Let 1/2 (z-di) = ai for

i

= 0, 1 , 2 and ( U ;

4-

U')-' = w,

SO that y i of (2.4) attains the form

yi = d z e x p ~ - a i w l for

w

>

0

.

(9.9)

Plainly, y i

>

y j if and only if ai

<

ai. Where a,

<

a,

<

a,, as in (1)

,

we have

y2

>

yo and yo

-

2y,

4-

y2

<

0 and, hence, the equilibrium of the form $.4 = f i B =

1/2,

D

= 0 is stable for all r independent of w.

The verification f o r (2) and ( 3 ) follows by examination of the critical value r* = R

(w),

expressed as a function of the variance parameter w, viz.,

R ( w )

= (1/4)+(1/4+) exp~-w(ao-a2>l

-

(1/2) exp[-w(a~-adl

for w

>

0

.

(2.10)

Also, by direct differentiation

--

''

-

(1/2) (al-a2> exp[-w(al-az)l - (1/4.)(a,--J dw

exp C-w (a,

-

a,>

1

.

(2.11)

Note that large values of w correspond to a smaller total variation U;

4-

u2 and

also R ( 0 ) = 0 , independent of the ordering in the vaIues of ai. The condition 2al

>

a.

+

a2 in (3) implies

-

dR

>

o

at

w =

0. AISO as a.

-

a,

>

al -az,

declines quicker than e-w(ul-aJ as w increases, so that

R(w),

in this case, is a nonnegative increasing function throughout the range w 2 0. There- fore, an increased phenotypic variance will diminish the boundary value R (w),

assuring stability of the central equilibria with smaller recombination rates. I n ( 2 ) , the condition 2a,

<

a.

+

a, implies - ( O )

<

0. Hence, R(w) is negative In the neighborhood of zero. In the range where R ( w ) is negative, the

dw

dR

TWO-LOCUS SYSTEM 1007 central equilibrium is always stable independent of r. The value ZZ; that pre- scribes the appropriate variance bound will be the nonzero solution of

(2.12) For

w

<

W

or equivalently u2

>

E ~ , the central equilibrium is always stable, while for U?

<

i

F

stability holds i n the range (7-*?1/2).

In summary, an increased phenotypic variance reflecting influences of

sampling effects, environmental noise or an augmented polygenic background will protect stability of a central type of equilibrium with tighter linkage. Where stable equilibria with

Li

# 0 exist, then inspection of the disequilibrium function

(2.13)

as a function of w, for a prescribed 7-

<

r*, reveals that

I

S

1

increases with w or

decreases with increased variance. Therefore, the effect of an increased pheno- typic variance is t o involve a set of gametes that manifest a weaker amount of

genic association.

SOME NONEPISTATIC FITNESS MODELS

The consequences of additive and multiplicative viability arrays

will

be sur- veyed in this section with respect to selection coefficients, expressed in terms of

phenotype parameters. We will assume, for illustration, a two-locus phenotype system of a bivariate trait

( X , Y )

measured on some continuous scale, e.g.,

X-

level of an enzyme Y-activity performance. Locus A controls the expression of the trait

X,

and locus

B

that of

Y.

The marginal fitness coefficients at each locus are determined vi2 the definition (l.l), depending on the genotype-

phenotype distribution and the phenotypic selection function.,

If

the two-locus fitness matrix has the form

BB

Fitnesses\ Fitnesses

at locus 1 at locus2

81

B b

82

A A ff1 w11 =tal

4-

p1

Wl2 = a1

i-

p 2

Aa f f 2 w13 a2 $-

81.

w14 = f f 2 -b p 2 = w23

aa f f 3 w33 = a3 -b

81

w34= ff3 i - 8 2 then we say there is no additive epistasis.

If

the fitness array takes the form

Fitnesses\ Fitnesses

BB

B b

at locus 1 at locus 2

p1

82

A A ff1 w11 = alp1 WlZ=

4%

aa ff3 Was = ff$1 w34 = 4 3 P 2

Aa f f 2 w13 = "ZPl w14 = d 2

1008 D. CARMELLI

review and recent results see

KARLIN

1975). It is known that if heterozygote advantage prevails at each locus, i.e., the conditions

a2

>

max(a,,.aS)

p2

>

max(P1,P3) (3.3) hold; then the equilibrium

1, =$1$2,12=$1(1-$2) , 1 3 = (l-$l)$2,i4= (1-$1)(1-$2) _(3.4)

with

012 - ff3 8 2

-

P 3

= 2a, - 011 - f f Q $2 = 2& -

p, -p3

(3.5)

is globally stable for all r

>

0, for the additive fitness array (3.1). If only one of the conditions of (3.3) holds, then one of the genes becomes fixed, while the other gene is maintained segregating.

I n the context of generalized major-gene models, the above comparisons of fitnesses are translated into comparisons of the marginal mean effects and vari- ance parameters characterizing the genotypic-phenotypic distributions. Possible inquiries in this respect are the effects of an increased environmental variance on the nature of stable equilibria. We investigated this question in detail in

CARMELLI and KARLIN (1980) with respect to the outcomes of single major-gene models with underlying Gaussian phenotypic distributions. We demonstrated that whenever a unique, globally stable equilibrium is established, the rate of convergence is slowed down with increasing phenotypic variance. Also7 under the condition ( 3 . 3 )

,

where the Hardy-Weinberg equilibrium (3.4) is globally stable, increased phenotypic variances will cause larger deviations of 1&-1/21 and 1$2-1/2], so that, consequently, a less “central” gametic equilibrium is realized.

With respect to multiplicative nonepistatic situations, the equilibrium con- figuration is more complex. For simplici ty, let us assume that the trait expres- sions, X and Y , are properly standardized and that the marginal fitnesses matrices at each locus are equal. In addition, only the distribution of the heterozygotes differs from that of the homozygote genotypes, so that a1 = =

p1

=

p3,

a2 = P z and the fitness matrix has the form shown in (2.1), where yo = a:, yl = alaZ

and y z a,”. Paraphrasing the previous analysis, we know that when a2

>

(Y,, the

central equilibrium (1/4, 1/4, 1/4, 1/4) is locally stable if and only if

r

>

R(w)

= (1/4)(1-exp~-w(al - a , ) ] ) * (3.6) where

Obviously, given that a,

>

a,, R ( w ) is monotone increasing as w increases, so that an increased phenotypic variance assures stability of the central equilibrium with smaller recombination rates.

w = ( U ~ + U ~ , ) - ~ and a, = (1/2) ( z - & ) ~ for i = 1,2

.

TWO-LOCUS SYSTEM 1009 On the other hand, if the marginal phenotypic distributions differ only with respect to the variance parameters, SO that

then fitness superiority depends on the relative magnitude of the phenotypic variances compared to the deviation

1

z-dl

.

Suppose that the distance Iz-dl is small compared to the variances; then, in

this case, the largest fitness occurs with minimum variance, i.e., (z-d)

<

a2,

<

U; implies a2

>

a1 o r equivalently

w z

>

wl. The boundary value

R

as a function of the deviation 1/2(z-d) = a, i.e.,

-

lnwi-lnwl 2(wz-w,) is monotone decreasing in the range 0

<

a

<

Thus, for deviations Iz-dl in this range: a smaller distance of the common mean,

d , from the optimum, z, implies stability of the central equilibrium with looser linkage.

For a = 0, the central equilibrium will be stable for

I

>

(1/4)

{

1- [ u 2 2 f ~ 2 E ] 1 ’ 2 } 2 2 1

+

U 2 E

.

(3.9)

a2 In this special case, the determining factors are the variances only. The ratio 7i

41

is always less than one since fitness superiority is achieved with less phenotypic variance. Therefore, as this ratio decreases, the looser is the recombination where stability of the central equilibrium occurs.

DISCUSSION A N D SUMMARY

The study of generalized major-gene models for traits exhibiting continuous expression has concentrated, for the most part, on the one-locus, two-allele case. The standard theory has been mainly statistical, and virtually no evolutionary aspects of the changes in gene frequencies at the major loci, as functions of the intrinsic phenotypic distributions or penetrances, were considered.

This paper seeks to shift the focus of oligogenic models from the study of

I010 D. CARMELLI

In KARLIN

and CARMELLI (1978) and CARMELLI and

KARLIN

(1980), an im-

portant objective was to discern contrasts, sensitivities and robustness of the evolutionary realizations in gene frequency equilibria at the major locus, relative to the parameters characterizing the genotypic-phenotypic associations and the phenotypic selection function. A number of the findings in this paper extend the one-locus results io include the purview of linkage and selection for some simple classical two-locus models. The main conclusions are as follows:

(1) An increase in the phenotypic variance will always decrease the amount

of recombination needed to keep a central type of polymorphism stable, i.e.,

environmental noise and/or a polygenic background facilitate the establishment

of a central type polymorphism with tighter linkage.

(2) Polymorphic equilibria, where

d

# 0, will exist near r = 0 and entail weaker linkage disequilibria between loci with increasing phenotypic variance.

(3) For additive nonepistatic two-locus models, a Hardy-Weinberg type equilibrium will be less central. with increasing phenotypic variance. Also, the rate of convergence is slowed with an increase in the variances.

The analyses were illustrated for special cases of equal means or equal vari- ances in the phenotypic-genotypic distributions. I t is worth reiterating that our results are structurally stable, meaning they are qualitatively invariant under small perturbations on these assumptions, i.e., they also hold when the variances or the means of the distributions are approximately equal.

We

limited ourselves to a background Gaussian distribution, but the results are likely to apply for

a much broader class of symmetric unimodal densities with the corresponding scaling paramelers replacing the variances of the normal distribution.

I wish t o thank S. KARLIN for helpful comments and many discussions essential to the completion of this paper.

LITERATURE CITED

BODMER, W. F. and J. FELSENSTEIN, 1967

CARMELLI, D. and S. KARLIN, 1980

KARLIN, S., 1975.

Linkage and selection: theoretical analysis of the deterministic two-locus random mating model. Genetics 57: 237-265.

The effects of increased phenotypic variance on the evolu- tionary outcomes of generalized major-gene models. Ann. Hum. Genet. 44: 81-93.

General two-locus selection models: Some objectives, results, and interpreta- tions. Theor. Popul. Biol. 7: 364-398. -, 1977 Selection with many loci and possible relations to quantitative genetics. pp. 207-226. Proc. Internatl. Conf. Quan. Genet., Iowa State University Press. -- , 1979 Principles of polymorphism and epistasis for multi- locus systems. Proc. Natl. Acad. Sci. U.S. 76: 541-545.

KARLIN, S. and M. W. FELDMAN, 1970 Linkage and selection: Two-locus symmetric viability model. Theor. Pop. Biol. 1: 39-71.

KARLIN, S. and U. LIBERMAN, 1976 A phenotypic symmetric selection model for three loci, two alleles. The case of tight linkage. Theor. Pop. Biol. 10: 334-364.

KARLIN, S. and D. CARMELLI, 1978 Evolutionary aspects and sensitivity studies of some major- gene models. J. Theor. Biol. 7 5 : 197-225.

LEWONTIN, R. C. and K. KOJIMA, 1960 The evolutionary dynamics of complex polymorphisms. Evolution 14: 458-472.