COMBINATORIAL THEORY OF Q,T-SCHR ODER POLYNOMIALS, PARKING FUNCTIONS AND TREES

COMBINATORIAL THEORY OF Q,T-SCHR ¨

ODER

POLYNOMIALS, PARKING FUNCTIONS AND TREES

Chunwei Song

A Dissertation in Mathematics

Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for

the Degree of Doctor of Philosophy

2004

Supervisor of Dissertation

COPYRIGHT

Chunwei Song

To Athena (Lingyun), Jie,

Acknowledgements

I would like to express my deep gratitude to my advisor, James Haglund, for his encourage-ment and caring guidance. I tremendously benefited from his sharp insight of the subjects, in depth knowledge of the field and extraordinary skills of attacking problems.

There have been many people who ever played an important role in my mathematical development. In special, I must thank Herb Wilf, who attracted me into the field of combi-natorics through a very first lesson of “generatingfunctionology” [Wil94]; Andre Scedrov, who instructed me mathematical logic in addition to cryptology; Jennifer Morse, who in a graceful manner displayed to me Tableau theory and symmetric functions; Amy Myers, who shared me with the theory of order; Paul Seymour, who brought me into his realm of graph theory; and Felix Lazebnik, who for a whole year tutored me extremal combinatorics and probabilistic/algebraic methods. I am also grateful to the following people who lent their expertise to me during the progress of this dissertation: Doron Zeilberger, Ira Gessel, Robert Sulanke, Michael Steele and Nick Loehr. They, and the other teachers throughout my maturation, have shaped my view on MATHEMATICS.

array of professionals entrusted with the preservation and perpetuation of certain specific knowledge or ideas and privileged to be the most indoctrinated members of society, one has to possess an ultimate concern toward one’s nation, society, and the entire humanity. This concern is for everything pertinent to the public benefits and must transcend self as well as coterie interests, which in some sense coincides with the religious spirit of responsibility.

I am very thankful to the math departmental staff, in particular Janet Burns and Monica Pallanti, for their assistance in many aspects during the past years when I was a graduate student.

It has been a great pleasure to discuss mathematics at Penn with many of our excellent fellow students, among whom forgive me to mention only my peers Fred Butler, Irina Gheorghiciuc and Aaron Jaggard.

My mathematical career started at my age of five due to the enlightenment of my mother, Ms. Guiying Li, an extremely talented woman who had no chance for the best education. Athena (Lingyun) Song, a little angel born at the beginning of this dissertation, has presented much inspiration and joy. Last but not least, I am indebted to the infinite love and emotional support from my wife, Jie, who has shared every piece of my excitements and frustrations and provided to me an enriched beautiful life.

ABSTRACT

COMBINATORIAL THEORY OF Q,T-SCHR ¨

ODER

POLYNOMIALS, PARKING FUNCTIONS AND TREES

Chunwei Song

James Haglund

Contents

Acknowledgements iv

Contents vii

List of Figures ix

1 Introduction 1

1.1 Overview . . . 1

1.2 A Survey of the Literature . . . 2

1.3 Summary of New Results . . . 9

2 Dyck Paths and Permutation Paths 17 2.1 On the Symmetry of the (q, t)-Catalan Polynomial . . . 17

2.1.1 A Bijection BetweenDnand a Special Set of Permutations . . . 18

2.1.2 The Foata-Sch¨utzenberger Involution . . . 27

2.2 The Theory of Permutation Paths . . . 29

2.2.1 Then!Permutation Paths and the Bijectionf betweenSnandPn . 29 2.2.2 Restrictingf to Some Pattern Forbidding Permutations . . . 33

2.2.3 The Signed Permutation PathsBn . . . 40

3.1 Haglund and Loehr’s (q, t)-Parking Function Polynomial . . . 43

3.2 Counting Special Families of Labelled Trees . . . 45

3.3 Major Sequences, Tree Inversion Enumerator and the Tutte Polynomial . . 64

4 The Limit Case of the (q, t)-Schr¨oder Theorem 70 4.1 Background and Basic Properties . . . 70

4.2 Investigation of Zeros . . . 77

4.3 Whenq =t . . . 83

4.4 Agreement oftkCoefficients . . . 91

5 Higher Dimensional Schr¨oder Theory 106 5.1 m-Schr¨oder Paths andm-Schr¨oder Number . . . 106

5.2 q-m-Schr¨oder Polynomials . . . 109

5.3 (q, t)-m-Schr¨oder Statistics and the Shuffle Conjecture . . . 112

List of Figures

1.1 A Dyck pathΠ∈ D6 with area(Π)=3. . . 4

1.2 A Dyck pathΠand its bounce pathB. . . 5

1.3 A Schr¨oder pathΠ∈S8,4 witharea(Π) = 5andmaj(Π) = 30. . . 8

1.4 A2-Schr¨oder path of order6and with2diagonal steps. . . 15

2.1 A balanced Dyck path of 4 blocks . . . 19

2.2 Add strips toΠ(1) to obtainΠ(2) . . . 23

2.3 Correspondence betweenP3andS3underf . . . 31

2.4 A Permutation pathΠ∈ P8with 4 caves and a 2-triangle . . . 38

2.5 σ = 57836241∈S8(4123)butΠis not 2-triangle-forbidding. . . 39

2.6 A Signed Permutation path. . . 40

3.1 A parking functionP ∈ P8 witharea(D(P)) = 6. . . 44

3.2 The number of “Least-Child-Being-Monk” trees on{0,1,2,3}is22 = 4. . 47

3.3 A primary parking functionP ∈ P₈∗, witharea(D(P)) = 12. . . 49

3.4 The3members ofT3,1. . . 52

4.1 A Dyck path (on the left) and its inverse (on the right). . . 89

5.1 A2-Dyck path inD₆2. . . 107

Chapter 1

Introduction

1.1

Overview

The primary focus of this dissertation is on various properties of lattice path enumeration and their (q, t)-analogues, a rapidly developing topic in the frontier of combinatorics, which as a domain of mathematics is itself going through a profound revolution.

1.2

A Survey of the Literature

There are two noteworthy sequences of numbers that are of central interests to combinato-rial mathematicians. ThenthCatalan number is

Cn=

1

n+ 1

µ

2n n

¶

.

[Sta99, Ex.6.19, pages 219–229] gives 66 combinatorial interpretations of these numbers, and updated additions are provided in [Sta].

ThenthSchr¨oder number is

Sn= n

X

d=0

1

n−d+ 1

µ

2n−d d, n−d, n−d

¶

.

[Sta99, Ex.6.39, pages 239–240] provides 19 combinatorial interpretations of these num-bers.

Among the interpretations, we are primarily interested in those associated to the lattice paths.

Definition 1.2.1. A Dyck path of ordern is a lattice path from(0,0)to(n, n)that never goes below the main diagonal{(i, i),0≤i≤n}, with steps(0,1)(or NORTH, for brevity N) and(1,0)(or EAST, for brevity E). LetDndenote the set of all Dyck paths of ordern.

Definition 1.2.2. A Schr¨oder path of ordernis a lattice path from(0,0)to(n, n)that never goes below the main diagonal{(i, i),0≤i≤n}, with steps(0,1)(or NORTH, for brevity N),(1,0)(or EAST, for brevity E) and (1,1) (or Diagonal, for brevity D). LetSndenote the

set of all Schr¨oder paths of ordern.

The Catalan numberCn = 1,2,5,14,42, ...,counts the number of Dyck paths of order

paths of ordern. In this dissertation, we are often more concerned with the Schr¨oder paths with d diagonal steps.

Definition 1.2.3. A Schr¨oder path of order n and with d diagonal steps is a lattice path from (0,0)to (n, n) that never goes below the main diagonal {(i, i),0 ≤ i ≤ n}, with

(0,1) (or NORTH, for brevity N), (1,0)(or EAST, for brevity E) and exactlyd (1,1) (or Diagonal, for brevity D) steps. LetSn,ddenote the set of all Schr¨oder paths of ordernand

withddiagonal steps.

The number of Schr¨oder paths of ordernand withddiagonal steps is counted by

Sn,d=

µ

2n−d d

¶

Cn

= 1

n−d+ 1

µ

2n−d d, n−d, n−d

¶

.

ClearlySn =

P_n

d=0Sn,dandCn =Sn,0.

By a statistic on a given setS, we mean a combinatorial rule that associates a nonneg-ative integer to each element ofS. Efforts and progresses have been made by considering 1 variable ([CR64], [FH85], [BSS93], etc) and 2 variable ([GH96], [HL], [EHKK03], etc) generalizations of these numbers, through studying various invented statistics associated with these lattice paths. The applications of these works expand from almost every sub-field of discrete mathematics to other areas such as representation theory and algebraic geometry.

Two important statistics onDnareareaandbounce.

Given Π∈ Dn, area(Π) is defined to be the number of complete squares between

Π and the main diagonal line y = x. More specifically, let ai(Π) be the number of

(a1(Π), a2(Π), . . . , anΠ) is the area vector of Π. Finally, area(Π) =

P_n

i=1ai(Π). An

example of a Dyck path of order 6 with area vector(1,0,0,1,1,0)is illustrated in Figure 1.1.

0 1

0 1 1 0

Figure 1.1: A Dyck pathΠ ∈ D6 with area(Π)=3.

Carlitz and Riordan [CR64] defined the following naturalq-analogue ofCn,

Cn(q) =

X

Π∈Dn

qarea(Π)_,

and showed that

Theorem 1.2.1.

Cn(q) =qk−1 n

X

k=1

Ck−1(q)Cn−k(q), n≥1.

The statistic bounce was introduced by Haglund in [Hag03]. Here we adopt the de-scription of [HL] to define it: start by placing a ball at the upper corner (n, n)of a Dyck pathΠ, then push the ball straight left. Once the ball intersects a vertical step of the path, it “ricochets” straight down until it intersects the diagonal, after which the process is iterated; the ball goes left until it hits another vertical step of the path, then follows down to the diagonal, etc. On the way from(n, n)to(0,0)the ball will strike the diagonal at various points(ij, ij). We definebounce(Π)to be the sum of theseij. For convenience, we also let

In addition, we say Π is balanced if and only if Π = b(Π). In Figure 1.2, a Dyck path

Π is represented by the solid line and its bounce path b(Π) = B is the dashing line. As illustrated,bounce(Π) = 2 + 6 = 8.

6

2

Figure 1.2: A Dyck pathΠand its bounce pathB.

Throughout this dissertation we use the standard notation

[n] := (1−qn_{)/(1−q),[n]! := [1][2]· · ·[n],}

·

n k

¸

:= [n]! [k]![n−k]!

for the q-analogue of the integer n, the q-factorial, and the q-binomial coefficient and

(a)n := (1−a)(1 −qa)· · ·(1−qn−1a) for the q-rising factorial. Sometimes it is

nec-essary to write the baseqexplicitly as in[n]q,[n]!q,

£_n

k

¤

qand(a;q)n, but we often omitqif

it is clear from the context. Occasionally, wheni+j+k=n, we also use£_i,j,kn ¤:= _[i]![[n_j]![]!_k]! to represent theq-trinomial coefficient.

We frequently make use of the following “q-binomial theorem” as a tool to prove iden-tities.

Theorem 1.2.2. [And98, page 36] The “q-binomial theorem”. Forn∈N,

n

X

k=0

q(k2)

·

n k

¸

and

∞

X

k=0

·

n+k−1

k

¸

zk ₌ 1

(z;q)n

.

In [GH96], Garsia and Haiman introduced a complicated rational function Cn(q, t)

which they proved has the following properties:

Cn(q,1) =

X

Π∈Dn

qarea(Π) =Cn(q)

q(n2)C

n(q,1/q) =

1 [n+ 1]

·

2n n

¸

.

In order to interpretCn(q, t), Haglund [Hag03] introduced the distribution function

Fn(q, t) =

X

Π∈Dn

qarea(Π)_tbounce(Π)

and conjectured thatFn(q, t) = Cn(q, t). Garsia and Haglund ( [GH02], [GH01]) proved

this by using symmetric function methods, and as a byproduct also the conjecture in [GH96] thatCn(q, t)is a polynomial with positive integer coefficients. Therefore,Cn(q, t)is now

called the (q, t)-Catalan polynomial.

There is a pair of basic statistics on the symmetric groupSn,invandmaj. In general,

for any integer word or multiset permutationw=w1w2· · ·wn,invandmajare defined as

inv(w) = X

i<j wi>wj

1

maj(w) = X

i wi>wi+1

i.

For later use, we also define the descent set of a wordw

and the number of descents ofw

des(w) :=|Des(w)|.

The following result due to MacMahon [Mac60] is now classical.

Theorem 1.2.3. For any fixed integersand any vectorα∈Ns, ifMα denotes the set of all

permutations of the multiset{0α01α1· · ·sαs}_{, then}

X

w∈Mα

qinv(w) ₌

·

n α1,· · · , αs

¸

= X

w∈Mα

qmaj(w)_.

Accordingly we say that inv andmaj are multiset Mahonian statistics. If we lets = n,

α0 = 0, α1 = · · · = αn = 1in the above theorem, thenMα specializes to the symmetric

groupSn,

£ _n

α1,···,αs

¤

= n!, and therefore we say that the two statistics invand maj onSn

are both Mahonian statistics.

Given a Dyck pathΠ, if we encode each N step by a 0, and eachE step by a 1, then from(0,0)to(n, n)we obtain a wordw(Π)ofn 0’s andn 1’s. Thus, the subset ofMn,n

each element of which has at least as many 0’s as 1’s in any initial segment is in bijection withDn. We call this special subset of 01 words the Catalan words of ordernand denote it

byCWn. Hence we may associate with eachΠthe statistics ofinvandmajbyinv(Π) =

inv(w(Π)) andmaj(Π) = maj(w(Π)). It is easy to see that¡n₂¢−inv(Π) = area(Π). The following classical result of MacMahon [Mac60, page 214] has a simple combinatorial proof in [FH85].

Theorem 1.2.4.

X

Π∈Dn

qmaj(Π) ₌ 1

[n+ 1]

·

2n n

¸

.

for a lattice pathΠthat never goes below the diagonal linex=y, define lower triangle to be a triangle with vertices(i, j),(i+ 1, j)and(i+ 1, j+ 1), and let theareaofΠ, denoted byarea(Π), be the number of lower triangles betweenΠand the main diagonal. This new definition ofareaagrees with the old one for Dyck paths, and is well defined for Schr¨oder paths. Similarly, if we mapSn,dto the words ofn−d0’s,d1’s andn−d2’s by replacing

eachN step by a 0, eachDstep by a 1 and eachE step by a 2 in a Schr¨oder path Π, then we have themajstatistic for Schr¨oder paths. Bonin, et. al. showed that [BSS93]

Theorem 1.2.5.

X

Π∈Sn,d

qmaj(Π) ₌ 1

[n−d+ 1]

·

2n−d n−d, n−d, d

¸

.

In Figure 1.3 below, the Schr¨oder pathΠ ∈ S8,4 is encoded by 001221010221, which

implies thatmaj(Π) = 5 + 6 + 8 + 11 = 30, and has area vector (0,1,1,0,0,2,1,0), which saysarea(Π) = 1 + 1 + 2 + 1 = 5. The length of each row, as computed from the number of lower triangles, is shown on the right.

0 1 1 0 0 2 1 0

Figure 1.3: A Schr¨oder pathΠ∈S8,4witharea(Π) = 5andmaj(Π) = 30.

procedure and defined the (q, t)-Schr¨oder polynomial

Sn,d(q, t) =

X

Π∈Sn,d

qarea(Π)_tbounce(Π)_.

They generalized Garsia and Haiman’s result to the following

q(n2)−(

d

2)S_n,d(q,1

q) =

1 [n−d+ 1]

·

2n−d n−d, n−d, d

¸

,

They also conjectured that the (q, t)-Schr¨oder polynomial is symmetric and made a stronger conjectural interpretation ofSn,d(q, t)involving a linear operator∇defined on the modified

Macdonald basis (for details see [EHKK03], [Hag04] or [HL]).

Conjecture 1.2.1. For all integersn,dwithd≤n,

Sn,d(q, t) =<∇en, en−dhd> .

This was recently proved in [Hag04] and thus became the (q, t)-Schr¨oder Theorem.

1.3

Summary of New Results

In this section, we list the main theorems in the chapters that follow.

First, in Chapter 2 we obtain some partial results about the symmetry of the (q, t )-Catalan polynomial and develop the theory of Permutation paths, which is a kind of gener-alized lattice path that contains Dyck paths as a subset.

distribu-tion funcdistribu-tion defined onTn, whereTnis a subset of the symmetric groupSn.

Cn(q, t) =

X

σ∈Tn

qinv(σ)_t(n2)−maj(σ).

Definition 1.3.1. A Permutation path of ordernis a lattice path from(0,0)to(n, n), which never goes below the main diagonal(i, i),0≤i≤n, or above the liney=n, and consists of NORTH(0,1), EAST(1,0)and SOUTH(0,−1)steps but never repeats (i.e. no NORTH step followed or preceded by a SOUTH step). LetPndenote the collection of Permutation

paths of ordern. Theorem 1.3.2.

|Pn|=n!.

Furthermore, there exists a weight-preserving bijection f between Sn and Pn that maps

the inversion statistic to the area statistic. Namely, for anyσ ∈Sn, we have

inv(σ) = area(f(σ)).

Next we consider the restriction of f toSn(312), the312-avoiding permutations, and call

itf∗. We show thatf∗ is a bijective map betweenSn(312)and Dyck pathsDn, a subset of

the image set Permutation paths.

Theorem 1.3.3. f∗ is a weight-preserving bijection betweenSn(312)and Dyck pathsDn

that maps the inversion statistic to the area statistic, and therefore

X

σ∈Sn(312)

qinv(σ) ₌ X Π∈Dn

qarea(Π)_.

Signed Permutation paths, which may be viewed as a generalization of both Permutation paths and Schr¨oder paths. Some parallel results on Signed Permutation paths are also included.

In Chapter 3, we prove some graph theory enumeration results while investigating the parking function polynomial Rn(q, t) as introduced in [HL]. We are able to show that

Rn(q,1)is equivalent to a group of other combinatorial statistics.

Theorem 1.3.4. (“Least-Child-Being-Monk”) DefineTn+1,0 to be the set of labelled trees

on{0,1,2, ..., n+ 1}, such that the least labelled child of 0 has no children (we say such trees have the Least-Child-Being-Monk property). Then the cardinality ofTn+1,0, which we

denote bytn+1,0, is equal tonn.

Corollary 1.3.5. Whenngoes to infinity, the probability for a labelled tree to be

“Least-Child-Being-Monk” ise−2.

Theorem 1.3.6. DefineTn+1,pto be the set of labelled trees on{0,1,2, ..., n+ 1}, such that

the total number of descendants of the least labelled child of 0 isp. Then, the cardinality

ofTn+1,p, denoted bytn+1,p, is equal to

(n−p)n−p_{(p+ 1)}p−1

µ

n+ 1

p

¶

.

Corollary 1.3.7. Whenngoes to infinity, the probability for a labelled tree on{0,1,2,· · · , n} to have the property that the least labelled child of 0 has exactlypdescendants is

(p+ 1)p−1

p! e

−2−p_.

Theorem 1.3.8. (Least-Single Trees Recurrence) A rooted labelled tree is

Hereditary-Least-Single if it has the property that every least child has no children. Let the number

Thenhnsatisfies the following recurrence:

hn=(n−1)hn−1−2

X

1≤i≤n−2

hn−ihi+1

µ

n−2

i−1, n−i−1

¶

+ X

1≤i≤n−2

X

1≤j≤n−i−1

ihihjhn+1−i−j

µ

n−2

i−1, j−1, n−i−j

¶

.

The following list contains{hn}, fornfrom 1 to 10, which is computed by Maple using

our recurrence: 1, 1, 1, 4, 15, 96, 665, 6028, 60907, 725560 ...

Theorem 1.3.9. Consider the exponential generating function H(x) = P_n≥0hn+1

n! xn.

ThenH(x)satisfies the simple functional equation

H2(x)−H(x) + 1 =exH(x).

Let

Rn(q,1) =

X

P∈Pn

qarea(P)_,

wherePnis the set of parking functions [HL], and

Mn(q) =

X

ˆ

s∈Mn

qarea(ˆs)_,

whereMnis the set of major sequences [Kre80].

Theorem 1.3.10.

Rn(q,1) =Mn(q).

Corollary 1.3.11. The 5 combinatorial statistics (see [Bei82], [Bj¨o92] and [Ste02]) are all

equal, i.e.

and they all satisfy the following same recurrence:

Stat1(q) = 1,

Statn(q) = n

X

i=1

µ

n−1

i−1

¶

[i] Stati−1(q)Statn−i(q).

In Chapter 4, we attack a combinatorial proof of a interesting identity derived from the limit case of the (q, t)-Schr¨oder theorem. That is,

Theorem 1.3.12. Forn ∈N,

n

X

k=1

X

a1+···+_ak=n ai>0

qPki=1(ai2)t

Pk−1

i=1(k−i)ai 1

(tk_;q)a

1(q;q)ak ×

k−1

Y

i=1

·

ai+ai+1−1

ai

¸

1 (tk−i_;q)

ai+ai+1

× (q;q)n(t;t)n

= [zn_] Y i,j≥0

(1 +qi_tj_{z)× (q;q)}

n(t;t)n

= X

σ∈Sn

qmaj(σ)_t(n₂)−maj(σ−1₎

,

Above we use[zn]f(z)to denote the coefficient ofzninf(z), a series in powers of z. Sometimes we also use [zn]{f(z)}, especially whenf(z)is a long formula. We analyze several special cases, make parallels of some results by Carlitz [Car56] and also obtain some refined results and conjectures relating the (q, t)-Schr¨oder polynomial statistics to the permutations whose longest increasing subsequence is of a fixed size. One of our byproducts is Theorem 1.3.13.

Definition 1.3.2. The inverse of a Catalan wordw∈CWnis defined to be

where r denotes the reverse operation and − denotes the complement operation that ex-changes 0 and 1. We saywis aninvolutionif and only ifw=w−1.

Example 1.3.1. Whenn=3,

(000111)−1 _{= 000111,}

(001011)−1 _{= 001011,}

(001101)−1 _{= 010011,}

(010011)−1 _{= 001101,}

(010101)−1 _{= 010101.}

So the involution set consists of 000111, 001011 and 010101.

It is easy to see thatw−1 ∈ CWnif and only if w ∈ CWn, so the inverse operation is

closed onCWn. Geometrically, givenw, we may obtainw−1 by finding the Dyck pathΠ

thatwcorresponds to under the natural map, reflecting Πover the NW-SE main diagonal to obtain a new Dyck pathΠ−1_{, and then taking the Catalan word that corresponds toΠ}−1_.

Theorem 1.3.13.

X

w∈CWn:

wis an involution

qmaj(w)−ndes(w) ₌ X

σ∈Sn(123):

σ is an involution

qmaj(σ)−maj(σ−1₎

.

In Chapter 5 we turn to higher dimensional Schr¨oder theory. That is, we study general-ized Schr¨oder paths inside a rectangle of lengthmnand widthn. We derive a formula for the number ofm-Schr¨oder paths and study itsqand(q, t)-analogues.

Figure 1.4: A2-Schr¨oder path of order6and with2diagonal steps.

for brevity N) and (1,0)(or EAST, for brevity E). LetD_nm denote the set of allm-Dyck paths of ordern.

Definition 1.3.4. Anm-Schr¨oder path of ordernand withddiagonal steps is a lattice path from(0,0)to(mn, n), which never goes below the main diagonal{(mi, i) : 0 ≤i ≤ n}, with(0,1)(or NORTH, for brevityN),(1,0)(or EAST, for brevityE) and exactlyd(1,1) (or Diagonal, for brevity D) steps. LetS_n,dm denote the set of allm-Schr¨oder paths of order

nand withddiagonal steps.

Figure 1.3 illustrates a2-Schr¨oder pathΠ∈ S₆2_,4.

Theorem 1.3.14. The number ofm-Schr¨oder paths of ordern and withddiagonal steps,

denoted byS_n,dm , is equal to

1

mn−d+ 1

µ

mn+n−d mn−d, n−d, d

¶

.

Remark 1.3.1. Whenm=1, the theorem above counts the ordinary Schr¨oder paths. When

d= 0, them-Dyck paths are counted. Actually the later result, i.e.|Dm_n|= _mn1₊₁¡mn_n+n¢is quite new [GH96] [HPW99], and not a single niceq-version seems to exist.

Definition 1.3.5. Define them-Narayana polynomialdm_n(q)overm-Schr¨oder paths of or-dernto be

dm n(q) =

X

Π∈Sm n

qdiag(Π)_,

where diag(Π)is the number ofDsteps in them-Schr¨oder pathΠ. Theorem 1.3.15. dm_n(q)hasq=−1as a root.

In [FH85], there is a refinedq-identity, X

k≥1

X

w∈CWn,k

qmajw ₌X

k≥1

1 [n]

·

n k

¸·

n k−1

¸

= 1

[n+ 1]

·

2n n

¸

,

whereCWn,kis the set of Catalan words consisting ofn0’s,n1’s, withkascents (i.e.k−1

descents). For the generalized version, Cigler proved that there are exactly

1 n µ n k ¶µ mn k−1

¶

m-Dyck paths with k peaks (consecutive NE pairs) [Cig87]. In order to generalize the results of [FH85], we prove a generalizedq-identity.

Theorem 1.3.16.

X

k≥d

·

k d

¸

1 [n]

·

n k

¸·

mn k−1

¸

q(k−d)(k−1) ₌ 1

[mn−d+ 1]

·

mn+n−d mn−d, n−d, d

¸

.

In the last section of Chapter 5, we mention a conjecture of Haglund, Haiman, Loehr, Remmel and Ulyanov which defines the (q, t)-m-Schr¨oder polynomial and relates it to the

Chapter 2

Dyck Paths and Permutation Paths

2.1

On the Symmetry of the (

q, t

)-Catalan Polynomial

The (q, t)-Catalan polynomialCn(q, t), introduced in [GH96] as a rational function, is

sym-metric inq andt from its definition. However, the original definition is very complicated and it is only because of the fact that Fn(q, t) = Cn(q, t), which is proved in [GH02]

[GH01], do we know thatCn(q, t)is a polynomial and has positive coefficients. Here

Fn(q, t) =

X

Π∈Dn

qarea(Π)tbounce(Π),

whereareaandbounceare statistics on Dyck pathsDnas introduced in Chapter 1. There is

no direct proof thatFn(q, t)is symmetric, i.e.,Fn(q, t) =Fn(t, q). Therefore it is desirable

to prove this combinatorially.

In this section we construct a bijection g between Dyck paths Dn and a special

sub-group of Sn, which we call Tn, interchanging areaand inv, and bounce and

¡_n

2

¢

−maj

onTn.

2.1.1

A Bijection Between

D

and a Special Set of Permutations

Given a Dyck pathΠ ∈ Dn, we construct an injectiong, which mapsΠ to a permutation

σ ∈Sn, with the properties that

area(Π) = inv(σ), bounce(Π) =

µ

n

2

¶

−maj(σ).

We define this map by a procedure involving two steps. Step 1: whenΠ∈ Dnis a balanced path.

First consider the case that Π is a balanced path. That is, Π = b(Π). Suppose Π is made up ofkblocks, i.e. Πhaskright (from NORTH to EAST) turns and hits the diagonal exactlyk+ 1 times including at(0,0)and at (n, n). To better illustrate, we consider the case k = 4, as it will be easy to extend this to generaln. As illustrated by Figure 2.1, let the sizes of the 4 blocks be a, b, candd, respectively, from bottom to top. Notice that

n=a+b+c+d.

The image permutationσ =g(Π)is defined as follows.

σ =a(a−1)· · ·1(a+b)(a+b−1)· · ·(a+ 1)(a+b+c) (a+b+c−1)· · ·(a+b+ 1)n(n−1)· · ·(a+b+c+ 1).

a b c d

Figure 2.1: A balanced Dyck path of 4 blocks

than any element in the(j + 1)st block, for1≤j ≤3. Apparently,

area(Π) =

µ a 2 ¶ + µ b 2 ¶ + µ c 2 ¶ + µ d 2 ¶ ,

inv(σ) =

µ a 2 ¶ + µ b 2 ¶ + µ c 2 ¶ + µ d 2 ¶ ,

and thereforearea(Π) =inv(σ).

It is also easy to observe that σ−1 =σ. Becauseσhas descents everywhere except the last position of each block, we have

maj(σ−1_{) = maj(σ) =}

µ

n

2

¶

−a−(a+b)−(a+b+c).

Note that

bounce(Π) =a+ (a+b) + (a+b+c).

Therefore,

bounce(Π) =

µ

n

2

¶

For convenience we define the set of “balanced permutations”.

Definition 2.1.1. A permutation σ = σ1· · ·σn ∈ Sn is said to be balanced if its one line

notation can be partitioned into a number of continuously descending blocks, such that any element in a preceding block is smaller than any element in a later block, i.e., σis of the form

σ=a1(a1−1)· · ·1 (a1+a2)(a1+a2−1)· · ·(a1+ 1) · · ·

n(n−1)· · ·(ak−1+. . .+a1+ 1),

for some integerk≥1.

Let the number of balanced permutations in Sn be bn. Above we have established

a bijection g between the balanced Dyck paths and balanced permutations. From both combinatorial structures, it is not hard to observe the following recurrence relation:

bn= n

X

i=0

bi,

b0 = 1.

Thereforebn= 2n−1.

Step 2: for generalΠ ∈ Dn .

Next we turn to the general case. It is useful to introduce the notion of right balanced path. Given a Dyck pathΠ, letB =b(Π)be the bounce path ofΠ. ClearlyB is a balanced path. Suppose B consists of m + 1blocks. That is, B has m left turns (from EAST to NORTH), which are just the hits at the main diagonal, andm+ 1right turns (from NORTH to EAST). In general, Πhas more area squares thanB geometrically. For1 ≤ j ≤ m, if

balanced. In the caseΠitself is a balanced path, i.e. B andΠare identical from the origin

(0,0), we say thatΠis 0-right balanced (so 0-right balanced means balanced). Clearly,Π being j-right balanced implies that Πis (j+1)-right balanced. Furthermore we allow j to be any integer by convention.

In order to extend the map g to the entire set Dn, we start with B = b(Π) and let

Π(0) _{=B. According to Step 1, there is a permutationσ}(0) _=g(Π(0)_{)such that}

area(Π(0)_{) = inv(σ}(0)_),

bounce(Π(0)_{) =}

µ

n

2

¶

−maj(σ(0)_).

Intuitively, we obtaing(Π)by each time adding squares to the area between two consec-utive blocks ofΠ(0), so that it gets closer toΠ, and finding the corresponding permutation that is the image of the modified path. SinceΠ(0) =B hasm+ 1blocks, we will reachΠ together withg(Π)afterm steps. In other words, each time we modify the path obtained earlier to become “more” right balanced, until we getΠ:

B = Π(0) →Π(1) → · · · →Π(m) = Π,

where eachΠ(j)is j-right balanced,0≤j ≤m.

Inductively, for each j,1 ≤ j ≤ m, we modify σ(j−1) = g(Π(j−1))to obtain σ(j) =

g(Π(j)_{), which satisfies}

area(Π(j)) = inv(σ(j)), bounce(Π(j)_{) =}

µ

n

2

¶

−maj(σ(j)_).

work on its inverse(σ(j−1))−1, instead of σ(j−1) itself, to obtain (σ(j))−1, and afterwards take the inverse again to obtainσ(j). We illustrate this process for the casem+ 1 = 4, i.e.,

B consists of 4 blocks, and use the same setup ofΠin Step 1 forB. W.O.L.G., we choose to show the second stage: assume forj = 2, we have already foundσ(1) =g(Π(1))which satisfies the two statistical identities withΠ(1), we go on to constructσ(2) = g(Π(2)). For technical as well as symbolic convenience, let ρ = (σ(1))−1 and τ = (σ(2))−1. Also, we make the inductive assumption thatρis of the particular form

ρ=ρ1· · ·ρa+b(a+b+c)(a+b+c−1)· · ·(a+b+ 1)

n(n−1)· · ·(a+b+c+ 1),

whereρ1· · ·ρa+b could be any permutation inSa+b. Finally letΠ(2) be obtained fromΠ(1)

by adding a strip of b1 squares to the top of the second EAST segment (from the bottom)

ofB, adding a strip ofb2 squares to the top of the just added strip of lengthb1,. . ., and in

the end adding a strip ofbr squares to the top of the previously added strip of lengthbr−1.

To understand this construction, be aware that

1≤br ≤ · · · ≤b1 ≤b,

1≤r ≤c−1,

and see Figure 2.2 (if r = 0then just let τ = ρ, we are done and pass on; ifr = c, then

a b c d

b1 b2 br

r<c

Figure 2.2: Add strips toΠ(1)to obtainΠ(2) Nowτ is ready.

τ =ρ1· · ·ρa+b−b1(a+b+c)ρa+b−b1+1· · ·ρa+b−b2(a+b+c−1)

ρa+b−b2+1· · · ·ρa+b−br(a+b+c−(r−1))ρa+b−br+1· · ·ρa+b

(a+b+c−r)· · ·(a+b+ 1)n(n−1)· · ·(a+b+c+ 1).

Intuitively, in the process to findτ, we just move(a+b+c)b1positions left to

corre-spond to theb1 squares in the newly added first row, and(a+b+c−1)b2 positions left,

etc, while leave the remaining untouched.

Obviously,area(Π(2))−area(Π(2)) =b1+· · ·+br.Note that for any permutationς,

we haveinv(ς) = inv(ς−1). Therefore,

inv(σ(2)_)−inv(σ(1)_{) =inv(τ)−inv(ρ)}

and hence we have the first statistical identity forσ(2)andΠ(2) by the inductive hypothesis. Since B is the balanced path of all the Π(j)s, bounce(Π(2)) = bounce(Π(1)) by the definition of the bounce statistic. Next we prove thatmaj(σ(2)) = maj(σ(1))by showing that σ(1) = τ−1 and σ(2) = ρ−1 have the same descent set, so that we have the desired second statistical identity forσ(2)andΠ(2).

Lemma 2.1.1. For any integeri, with1≤i≤n−1,

i∈Des(ρ−1_{) ⇔ i∈Des(τ}−1_).

Proof. There are three cases fori.

• a+b+c+ 1 ≤i≤n. This case is trivial.

• 1≤i≤a+b. Observe that by the construction ofτ,

i∈Des(ρ−1) ⇔ ∃x < y, s.t.

          

1≤x < y ≤a+b, ρ(x) = i+ 1,

andρ(y) = i.

⇔ ∃x0 < y0, s.t.

          

1≤x0 < y0 ≤a+b+r, τ(x0) = i+ 1,

andτ(y0) =i.

⇔ i∈Des(τ−1_).

• a+b + 1 ≤ i ≤ a+b +c. Then ρ(i) = a+b+ 1 + (a+b +c−r). Because

also the case forτ since it is also true that

τ−1_{(a+b+c)< τ}−1_{(a+b+c−1)<· · ·< τ}−1_(a+b),

andτ−1(a+b+c+ 1) =n.

So we have found a map g from Dyck paths to permutations preserving the statistical identities. The last thing we need to prove is thatgis an injection so that it is reversible.

Theorem 2.1.2. The mapg described in the above algorithm is an injection.

Proof. Given two Dyck pathsΠ1andΠ2, we prove that they result in different images.

Case 1: supposeΠ1andΠ2have the same bounce pathBand they are obtained by the

following procedures, respectively,

B = Π(0)₁ →Π(1)₁ → · · · →Π(₁m)= Π1,

B = Π(0)₂ →Π(1)₂ → · · · →Π(₂m)= Π2.

where Π(j) is j-right balanced, 0 ≤ j ≤ m. For eachj, let σ₁(j) = g(Π(₁j)) and σ₂(j) =

g(Π(₂j)). So,g(Π1) = g(Π(1m)) =σ (m)

1 andg(Π2) =g(Π(2m)) = σ (m) 2 .

Assume the two procedures do not agree with each other for the first time at the ith step, i.e., Π(₁j) = Π₂(j), and thereforeσ₁(j) = σ₂(j),0 ≤ j ≤ i−1, but Π(₁i) 6= Π(₂i). For convenience, letρ = (σ₁(i))−1 and τ = (σ₂(i))−1. W.O.L.G, assume the first disagreement ofΠ1 andΠ2 is thatΠ1 adds a strip of more squares thanΠ2 does to the same row of the

jth _{block. As a result, while constructingρthere will be an integerxmovingrplaces left,}

passingrnumbersy1,· · · , yr; but in the process of constructingτ the same integerxonly

relevant order ofxandy1,· · · , yr will never change again after that in the later process of

constructingσ₁(m) andσ₂(m). So,x is to the right ofyr in(σ1(m))−1, but to the left ofyr in

(σ(₂m))−1_{, and thusσ}(m) 1 6=σ

(m) 2 .

Case 2: supposeΠ1 andΠ2 have different bounce paths, respectivelyB1 andB2, and

Π1 andΠ2 are obtained by the following procedures,

B1 = Π(0)1 →Π(1)1 → · · · →Π(1m1) = Π1,

B2 = Π(0)2 →Π (1)

2 → · · · →Π (m2)

2 = Π2.

Also letσ(₁j) =g(Π(₁j))andσ(₂j) =g(Π₂(j)). Sog(Π1) = σ(1m1)andg(Π2) = σ2(m2).

W.O.L.G., assumeB1andB2do not agree for the first time at theithblock (from bottom

to top), and that theith block ofB1 is larger in size than theithblock ofB2. Then, theith

descending block of(σ₁(0))−1 will have the form(x+r)· · ·(x+ 1)xand accordingly, the

ith_{descending block of(σ}(0)

2 )−1has the form(x+s)· · ·(x+ 1)xwithr≥s+ 1. Be aware

that(x+s+ 1)is always to the left of(x+s)in every(σ₁(j))−1 during our procedure of reachingΠ1 because we maintain the relative order of each block. Hence(x+s+ 1)is to

the left of (x+s)in(σ₁(m1))−1. On the other hand, observe that the (i+ 1)st descending block of(σ₂(0))−1is of the formy(y−1)· · ·(x+s+ 1). That is,(x+s+ 1)is at the the end of the next block of(x+s)in(σ₂(0))−1. By the rule of our algorithm,(x+s+ 1)stays unmoved during the modification from(σ(₂i))−1to(σ(₂i+1))−1 (again, because otherwiseΠ2

should have a different bounce path fromB2). Therefore(x+s+1)is to the right of(x+s)

in(σ₂(m2))−1, and soσ₁(m1)6=σ₂(m2).

Hencegis a bijection betweenDnand a subset ofSn, which we callTn, and accordingly

we have the following corollary.

function defined onTn.

Cn(q, t) = Fn(q, t) =

X

σ∈Tn

qinv(σ)_t(n2)−maj(σ).

Example 2.1.1. Whenn = 3,T3 ={321,231,213,132,123}.

Whenn = 4,T4 ={4321,3214,1324,2134,1234,2143,1243,1432,2314,3421,3241,2431,

3142,1342}.

2.1.2

The Foata-Sch ¨utzenberger Involution

An involution ψ is described in [FS78] that interchanges inv and maj and preserves the descent set. More specifically, we have the following theorem.

Theorem 2.1.4. [FS78] There exists an involution ψ : Sn → Sn with the property that

inv(ψ(σ)) =maj(σ)andinv(σ) = maj(ψ(σ))hold simultaneously. Lemma 2.1.5. The following polynomial is symmetric inqandt:

Sn(q, t) =

X

σ∈Sn

qinv(σ)_t(n₂)−maj(σ)_.

Proof. Letcbe the “complement” map such thatc(σ) = (n+ 1−σ1)(n+ 1−σ2)· · ·(n+

1−σn)forσ ∈ Sn. Note thatκ =cψ, whereψ is the Foata-Sch¨utzenberger involution in

Theorem 2.1.4, is a bijection fromSntoSn. Furthermore,

inv(κ(σ)) =

µ

n

2

¶

−inv(ψ(σ)) =

µ

n

2

¶

and

µ

n

2

¶

−maj(κ(σ)) =

µ

n

2

¶

−(

µ

n

2

¶

−maj(ψ(σ))) =maj(ψ(σ))

=inv(σ).

Therefore,

Sn(q, t) =

X

σ∈Sn

qinv(σ)_t(n₂)−maj(σ)

= X

σ∈Sn

qinv(κ(σ))t(n2)−maj(κ(σ))

= X

σ∈Sn

q(n2)−maj(σ)tinv(σ)

=Sn(t, q).

Notice that if we could replace Sn byTn in the above theorem, then the symmetry of

the (q, t)-Catalan polynomial would be proved due to the bijectiong betweenDn andTn.

That however, would require a proof for κ = cψ to be closed on Tn and would require a

2.2

The Theory of Permutation Paths

2.2.1

The

n!

Permutation Paths and the Bijection

f

between

S

and

P

As an attempt to extend the idea of Dyck paths, we introduce the notion of Permutation paths and develop the related theory.

Definition 2.2.1. A Permutation path of ordernis a lattice path from(0,0)to(n, n)which never goes below the main diagonal(i, i),0≤i≤n, or above the liney=n, and consists of NORTH(0,1), EAST(1,0)and SOUTH(0,−1)steps but never repeats (i.e. no NORTH step followed or preceded by a SOUTH step).

LetPndenote the collection of Permutation paths of ordern. Figure 2.3 is an

illustra-tion of the 6 members inP3.

Theorem 2.2.1.

|Pn|=n!.

Proof. Note that any Permutation path Π ∈ Pn consists of some NORTH steps, some

SOUTH steps and exactlynEAST steps made at different columns. In more detail, for j from 1 ton−1, thesenEAST steps are in the form of(j−1, hj)→(j, hj)wherehjcould

be any integer satisfying j ≤ hj ≤ n becauseΠ never goes below the main diagonal or

above the liney=n. In factΠis uniquely decided by these EAST steps or equivalently the sequence of their heights (y-values)(h1,· · · , hn). Once these EAST steps are fixed, we just

connect them up by continuous NORTH or SOUTH steps, or possibly an empty vertical move ifhj =hj+1. Since repeats are not allowed, the connection is unique. Therefore the

number of Permutation paths of ordernis equal to the number of sequences(h1,· · · , hn).

For each j, sincej ≤ hj ≤ n, there aren+ 1−j ways to choose hj. Furthermore, the

The cardinality of n! naturally motivates us to give a bijection between Pn and the

symmetric groupSn. Actually the previous proof already provides hints of this bijection.

Lemma 2.2.2. There exists a bijectionf between the symmetric groupSnand the

Permu-tation pathsPn.

Proof. For anyΠ ∈ Pn, define the height sequence ofΠto be the sequence of the heights

(y-values) of Π’s EAST steps, from left to right, as in the previous proof. Denote it by

h(Π) = (hΠ

1, . . . , hΠn). Clearlyj ≤hΠj ≤nfor eachjand any integer vector satisfying this

requirement is a height sequence for some uniquely decided Permutation path.

Givenσ =σ1· · ·σn ∈ Sn, find its “lifted word”l(σ) =l1σ· · ·lnσ where for1≤ j ≤n,

lσ

j is what σj would become if we map {σj,· · ·, σn} to the set {j,· · · , n} keeping the

relative order of each element.

For example, if n = 6andσ = 6 2 4 3 5 1, thenl(σ) = 6 3 5 5 6 6: l₁σ = 6because

σ1 = 6 is the biggest among {6,2,4,3,5,1} when the set {6,2,4,3,5,1} is mapped to

{1,2,3,4,5,6}where 6 is also the biggest;l₂σ = 3 becauseσ2 = 2is the second smallest

in {2,4,3,5,1} when the set {2,4,3,5,1} is mapped to {2,3,4,5,6} where the second smallest element is 3, etc. Notice that l₁σ is always equal to σ1, lnσ is always n and that

i≤lσ

i ≤nfor everyi.

So, l(σ) is a height sequence. Find its corresponding Permutation path Π and let

f(σ) = Π.

Conversely, given any Permutation path Π, locate its height sequenceh(Π). Actually we will useh(Π)as the “lifted word” to findσ=f−1(Π). Letσ1 =hΠ1. Forifrom 2 ton,

letσibe the(hΠi + 1−i)thsmallest number in the set{1,· · · , n} − {σ1,· · · , σi−1}. Clearly

That is, we may writeΠdirectly as Π = (hΠ₁, . . . , hΠ_n), whereh(Π)is the height sequence ofΠ.

Example 2.2.1. Whenn= 3, there are3! = 6Permutation paths, and their correspondence with the permutations inS3 throughf is indicated in Figure 2.3.

(1,3,3)=f(132) (2,2,3)=f(213) (1,2,3)=f(123)

(2,3,3)=f(231) (3,2,3)=f(312) (3,3,3)=f(321)

Figure 2.3: Correspondence betweenP3andS3underf

Theareastatistic, previously defined on Dyck pathsDn, may be extended toPn

natu-rally since a Permutation path never goes below the main diagonal. Simply, we letarea(Π) be the number of complete squares between Π and the main diagonal liney = x. This agrees with the old definition of areaonDn, which is a subset ofPn, ifΠis also a Dyck

path.

The bijectionf has the nice property of mappinginvtoarea.

Theorem 2.2.3. f is a weight-preserving bijection betweenSn andPn that maps the

in-version statistic to the area statistic. Namely, for anyσ ∈Sn, we have

Proof. Letf(σ) = Πandh(Π) = (hΠ₁,· · ·, hΠ_n). Notice that

area(Π) =

n

X

i=1

hΠ

i −i

and for1≤i≤n−1(hΠ_n −n = 0) we have

hΠ

i −i=|{j :σi > σj andi < j ≤n}|.

So it is clear.

Corollary 2.2.4.

X

Π∈Pn

qarea(Π) _{= [n]!.}

Proof. Recall that theinvstatistic is Mahonian onSn[Mac60],

X

σ∈Sn

qinv(σ) = [n]!.

So the conclusion follows from Theorem 2.2.3. Alternatively, it could also be obtained easily by induction.

As we did for Dyck paths, we may associate each Permutation path with an appropriate word. Still encode eachN step by a0, each E step by a1, and in addition encode eachS step by a special character0∗.

Definition 2.2.2. A Permutation word of ordernis a permutation of the multiset{0n+s1n0∗s}, where1≤s≤ bn₂cbn−₂1c, with the property that for1≤i≤2n+2s, in the initial subword

w=w1w2· · ·wi,

• the number of0’s is at least as many as the sum of the numbers of1’s and the number

• the number of0’s minus the number of0∗’s is at mostn. We letAWndenote the set of Permutation words of ordern.

Definition 2.2.3. The inversion statistic of a Permutation word w = w1w2· · ·w2n+2s is

defined to be

inv(w) = X

i:wi=1

n1(i)−n2(i),

wheren1(i)is the number of 0’s afterwiandn2(i)is the number of0∗’s afterwi.

The inversion statistic so defined on AWn is apparently an extension of the inversion

statistic on the Catalan wordsCWn, wheres= 0, i.e. no0∗’s exist.

Corollary 2.2.5.

X

w∈AWn

q(n2)−inv(w) = [n]!.

Proof. For anyw∈AWn, it is easy to see that

µ

n

2

¶

−inv(w) = area(Π(w)),

whereΠ(w)is the Permutation path thatwcorresponds to.

2.2.2

Restricting

f

to Some Pattern Forbidding Permutations

Since Dyck pathsDn is a subset of Pn and at least thearea statistic is extended toPn in

a nice way, we study f−1(Dn)and some other related objects in order to understand the

Catalan phenomena as well as derive more general theories.

First we need some preliminary background on the theory of patterns.

Given permutationsτ ∈ Sk andσ ∈Sn, we define an occurrence of the patternτ inσ

to be a choice ofkslots

such that the sequence σi1, . . . , σik is in the same order of relative size as the sequence τ1, . . . , τk. In other words, for1≤j1 < j2 ≤k,

σij₁ < σij₂ iffτj1 < τj2.

Sometimes we also say that{σi1,· · · , σik}is aτ-occurrenceinσ.

Accordingly, if σ does not contain any τ-occurrences of the pattern , we say that σ is

τ-avoiding. Denote the set of allτ-avoiding permutations inSnbySn(τ)[Pri97].

Example 2.2.2. Consider σ = 51324 ∈ S5 and τ = 123 ∈ S3. σ is NOT τ-avoiding

because

{σ2, σ3, σ5}={1,3,4}

is aτ-occurence inσ. Notice that

{σ2, σ4, σ5}={1,2,4}

is also aτ-occurence inσ, but finding one occurence is sufficient for our purpose here. Alternatively, let’s consider σ0 = 32541 ∈ S5 and τ

0

= 312 ∈ S3. Then σ

0

is τ0 -avoiding because we can not find any 312-occurence inσ0 = 32541. So we can say that

32541∈S5(312).

The theory of pattern avoidance has been studied extensively. It is now well known (see, for example, [Knu73]) that for anyτ ∈S3, |Sn(τ)| =Cn. Partly because of this, we

are motivated to considerf−1(Dn), and we find that this pre-image is indeed Sn(312). In

fact, there have been two direct bijections betweenSn(312) andDn which have occurred

bijection exchanging theinversionandareastatistics but it is different from ourf. Let’s consider the restriction off onSn(312), the312-avoiding permutations, and call

itf∗. We prove thatf∗ is a bijective map betweenSn(312)and Dyck pathsDn, a subset of

the image set of Permutation paths.

Theorem 2.2.6. f∗ is a weight-preserving bijection betweenSn(312)and Dyck pathsDn

that maps the inversion statistic to the area statistic, and therefore

X

σ∈Sn(312)

qinv(σ) ₌ X Π∈Dn

qarea(Π)_,

where the right hand side is Carlitz-Riordan’sq-Catalan polynomialCn(q)[CR64] which

satisfies the recurrence

Cn(q) = n

X

k=1

qk−1_C

k−1(q)Cn−k(q).

Proof. Any Permutation path Π ∈ Pn can be uniquely represented by its height vector

h(Π) = (hΠ

1, . . . , hΠn). Notice thatΠ∈ Dnif and only if for1≤i≤n−1, so we have

hΠ

i ≤hΠi+1.

Givenσ ∈Sn(312), we provef(σ) = Π ∈ Dn. Recall thath(Π) = l(σ). This means for

1≤i≤n,hΠ_i =l_iσis whatσiwould become if we map{σi,· · · , σn}to the set{i,· · · , n}

keeping the relative order of each element. Revising this a little bit, let(l_iσ₊₁)−denote what

σi+1 would be at the “previous stage”, i.e, when we map{σi,· · · , σn}to{i,· · ·, n}rather

than replacingibyi+ 1(for which we would getl_iσ₊₁ ). Now note that

• Ifσi+1 < σi, thenliσ+1 = (lσi+1)−+ 1. So

lσ i+1 ≥lσi

⇔(lσ

i+1)− ≥liσ−1

⇔@k,s.t.i < i+ 1< kandσi+1 < σk < σi.

From the above conditions, it is clear thatσ∈Sn(312)impliesΠ ∈ Dn.

Conversely, givenΠ∈ Dn, consider its pre-imagef−1(Π) =σ. Supposeσ /∈Sn(312).

Take a minimal 312-occurence{σi, σj, σk}in the sense

i < j < k,

σj < σk < σi,

and|j−i|+|k−i|is minimal.

Π ∈ Dnimplies thathΠi ≤hΠi+1, and henceliσ ≤lσi+1. This requiresσi−σi+1 ≤1, and

hencej 6=i+ 1. Then what aboutσi+1? Ifσi+1> σk, then{σi+1, σj, σk}would be another

312-occurence, violating the minimality. Soσi+1 < σk. But then{σi, σi+1, σk}would be a

“less” 312-occurence. This shows thatσ /∈Sn(312)is impossible.

Therefore f(Sn(312)) = Dn, or we may say that there exists a bijection f∗ from

Sn(312)toDn.

Sincef is weight-preserving, its restrictionf∗is also weight-preserving.

The above result gives rise to more general questions. Since we now knowf−1(Sn(312)) =

Dn, what is f−1(Sn(k12. . . k−1)) for general k? We answer this question partially by

giving a lower bound.

For convenience, let the restriction offonSn(k12· · ·k−1), the set ofk12· · ·(k−1)

following definition.

Definition 2.2.4. An m-cave of a Permutation path Π, with height sequence h(Π) =

hΠ

1hΠ2 · · ·hΠn, is a stepisatisfying that

max{hΠ

1, . . . , hΠi−1} −hΠi =m,

wherem ≥ 1. For anm-cave c, sometimes we say thatcis of depth m. Anm-triangle is a sequence of caves,c1, c2,· · · , cm, not necessarily continuous but in order from left to

right, wherecj is of depth at leastm+ 1−j for each1≤j ≤m. If a Permutation pathΠ

does not contain anym-triangle, we say that P ism-triangle-f orbidding. Denote the set ofm-triangle-forbidding Permutation paths of ordernbyFn,m.

Geometrically, an m-cave of Π ∈ Pn is a step which is m squares down compared

with the highest level thatΠhas reached earlier (observe that the highest level thatΠwill reach later is alwaysy=n). The juxtaposition of the not-necessarily continuous sequence of caves in anm-triangle contains an isosceles right triangle with leg lengthm. Since we requirem ≥ 1, a Dyck path has no m-cave orm-triangle. In addition,Fn,1 = Dn. The

following figure illustrates a pathΠ ∈ P8 with four caves: c1 at step 2 is of depth 1,c2 at

step 4 is of depth 1, c3 at step 5 is of depth 3 and c4 at step 6 is of depth 2. The cavec1

itself is a 1-triangle, and so is every other cave. The sequence of caves{c3, c4}forms the

only 2-triangle ofΠand there is nom-triangle form ≥3.

The next theorem provides a combinatorial interpretation of the(m1. . . m−1)-avoiding permutations in terms of the(m−2)-triangle-forbidding paths.

Theorem 2.2.7. Form ≥3, ifσ ∈Sncontains anm1· · ·m−1pattern, then the

Permu-tation pathΠ =f(σ)must have an(m−2)-triangle. That is,

Figure 2.4: A Permutation pathΠ∈ P8with 4 caves and a 2-triangle

Whenm = 3,⊆is replaced by equality.

Proof. For anyσ /∈Sn(m12. . . m−1), we proveΠ =f(σ)contains some(m−2)-triangle.

Assume{σi1,· · · , σim}is a(m12. . . m−1)-occurence inσ, namely

i1 < i2 <· · ·< im,

σi2 <· · ·σim < σi1.

In fact, forjfrom 2 tom−1, we show that there is adj-cave at stepij, wheredj ≥m−j.

Note that

hΠ

i2 <· · ·< h

Π

im−1.

Furthermore becauseσim−1 < σim < σi1, we have

hΠ_im−1 < hΠ_i1.

So,

max{hΠ

1, . . . , hΠij−1} −hΠij ≥h

Π

i1 −h

Π