Lecture 2
The main concepts of combinatorial analysis
Plan of the lecture:
1. Fundamental of combinatorics. 1.1Introduction.
1.2Rule of sum.
1.3Multiplicative rule, the Counting Principle. 2. Permutations.
3. Combinations.
3.1 Number of 𝑘-combinations from a set
3.2 Number of combinations with repetition 4. Partitions.
4.1 Multinomial coefficient 4.2 Ordered partition of a set
1 Fundamental of combinatorics
1.1 Introduction
Combinatorics is a branch of pure mathematics concerning the study of discrete (and usually finite) objects. It is related to many other areas of mathematics, such as algebra, probability theory, ergodic theory and geometry, as well as to applied subjects in computer science and statistical physics. Aspects of combinatorics include “counting” the objects satisfying certain criteria (enumerative combinatorics), deciding when the criteria can be met, and constructing and analyzing objects meeting the criteria (as in combinatorial designs and matroid theory), finding “largest”, “smallest”, or “optimal” objects (extremal combinatorics and combinatorial optimization), and finding algebraic structures these objects may have (algebraic combinatorics).
1.2 Rule of sum
In combinatorics, the rule of sum is a basic counting principle. Stated simply, it is the idea that if we have 𝑎 ways of doing something and 𝑏 ways of doing another thing and we can not do both at the same time, then there are 𝑎 + 𝑏 ways to choose one of the actions.
More formally, the rule of sum is a fact about set theory. It states that sum of the sizes of a finite collection of pairwise disjoint sets is the size of the union of these sets. That is, if 𝑆1,𝑆2,...,𝑆𝑛 are pairwise disjoint sets, then we have:
𝑆1 + 𝑆2 + ⋯ + 𝑆𝑛 = 𝑆1∪ 𝑆2∪ ⋯ ∪ 𝑆𝑛 .
Simple example
A woman has decided to shop at one store today, either in the north part of town or the south part of town. If she visits the north part of town, she will either shop at a mall, a furniture store, or a jewelry store (3 ways). If she visits the south part of town then she will either shop at a clothing store or a shoe store (2 ways).
Thus there are 3 + 2 = 5 possible shops the woman could end up shopping at today.
In combinatorics, the rule of product is a basic counting principle (a.k.a. the
fundamental principle of counting). Stated simply, it is the idea that if we have 𝑎 ways of doing something and 𝑏 ways of doing another thing, then there are 𝑎 · 𝑏 ways of performing both
actions.
To choose one of these 𝐴, 𝐵, 𝐶 and one of these 𝑋, 𝑌 is to choose one of these 𝐴𝑋, 𝐴𝑌, 𝐵𝑋, 𝐵𝑌, 𝐶𝑋, 𝐶𝑌 .
In this example, the rule says: multiply 3 by 2, getting 6.
The sets {𝐴, 𝐵, 𝐶} and {𝑋, 𝑌} in this example are disjoint, but that is not necessary. The number of ways to choose a member of {𝐴, 𝐵, 𝐶}, and then to do so again, in effect choosing an
ordered pair each of whose components is in {𝐴, 𝐵, 𝐶}, is 3 × 3 = 9.
In set theory, this multiplication principle is often taken to be the definition of the product of cardinal numbers. We have
𝑆1 ∙ 𝑆2 ⋯ 𝑆𝑛 = 𝑆1× 𝑆2 × ⋯ × 𝑆𝑛 ,
where × is the Cartesian product operator. These sets need not be finite, nor is it necessary to have only finitely many factors in the product.
The Counting Principle
The counting principle is based on a divide-and-conquer approach, whereby the counting is broken down into stages through the use of a tree. For example, consider an experiment that consists of two consecutive stages. The possible results of the first stage are 𝑎1, 𝑎2, …, 𝑎𝑚; the possible results of the second stage are 𝑏1, 𝑏2, …, 𝑏𝑛. Then, the possible results
of the two-stage experiment are all possible ordered pairs (𝑎𝑖, 𝑏𝑗), 𝑖 = 1, … , 𝑚, 𝑗 = 1, … , 𝑛. Note that the number of such ordered pairs is equal to 𝑚 ∙ 𝑛. This observation can be generalized
as follows (see also Fig. 1).
The Counting Principle
Consider a process that consists of 𝑟 stages. Suppose that: (a) There are 𝑛1 possible results for the first stage.
(b) For every possible result of the first stage, there are 𝑛2 possible results at the second
stage.
Then, the total number of possible results of the 𝑟-stage process is
𝑛1 ∙ 𝑛2⋯ 𝑛𝑟.
Figure 1: Illustration of the basic counting principle. The counting is carried out in 𝑟 stages
(𝑟 = 4 in the figure). The first stage has 𝑛1 possible results. For every possible result of the first 𝑖 − 1 stages, there are 𝑛𝑖 possible results at the 𝑖th stage. The number of leaves is 𝑛1∙ 𝑛2⋯ 𝑛𝑟.
This is the desired count.
It should be noted that the Counting Principle remains valid even if each first-stage result leads to a different set of potential second-stage results, etc. The only requirement is that the number of possible second-stage results is constant, regardless of the first-stage result. This observation is used in the sequel.
Many problems in probability theory require that we count the number of ways that a particular event can occur. For this, we study the topics of permutations and combinations. We will focus primarily on two types of counting arguments that involve the selection of 𝑘 objects
out of a collection of 𝑛 objects. If the order of selection matters, the selection is called a
permutation, and otherwise, it is called a combination. We will then discuss a more general type of counting, involving a partition of a collection of 𝑛 objects into multiple subsets.
We start with 𝑛 distinct objects, and let 𝑘 be some positive integer, with 𝑘 ≤ 𝑛. We wish to count the number of different ways that we can pick 𝑘 out of these 𝑛 objects and arrange them
in a sequence, i.e., the number of distinct 𝑘-object sequences. We can choose any of the 𝑛 objects to be the first one. Having chosen the first, there are only 𝑛 − 1 possible choices for the second; given the choice of the first two, there only remain 𝑛 − 2 available objects for the third stage, etc. When we are ready to select the last (the 𝑘th) object, we have already chosen 𝑘 − 1
objects, which leaves us with 𝑛 − (𝑘 − 1) choices for the last one. By the Counting Principle, the number of possible sequences, called 𝑘-permutations, is
𝑛(𝑛 − 1) ⋯ (𝑛 − 𝑘 + 1)
𝑘 𝑓𝑎𝑐𝑡𝑜𝑟𝑠
=𝑛(𝑛−1)⋯(𝑛−𝑘+1)(𝑛−𝑘)⋯2∙1(𝑛−𝑘)⋯2∙1 = (𝑛−𝑘)!𝑛! .
In the special case where 𝑘 = 𝑛, the number of possible sequences, simply called
permutations, is
𝑛 ∙ 𝑛 − 1 ∙ 𝑛 − 2 ⋯ 2 ∙ 1 = 𝑛!.
The product of the positive integers from 1 to 𝑛 inclusive is denoted by 𝑛! (read as “𝑛 factorial”):
𝑛! = 1 ∙ 2 ∙ 3 ∙ ⋯ ∙ (𝑛 − 2) ∙ (𝑛 − 1) ∙ 𝑛.
Let 𝑘 = 𝑛 in the formula for the number of 𝑘-permutations, and recall the convention 0! = 1.
Example 1.26. Let us count the number of words that consist of four distinct letters. This is the problem of counting the number of 4-permutations of the 26 letters in the alphabet. The desired number is
𝑛! (𝑛−𝑘)!=
26!
22!= 26 ∙ 25 ∙ 24 ∙ 23 = 358,800.
The count for permutations can be combined with the Counting Principle to solve more complicated counting problems.
all elements of the given set. For example, given the set of letters {𝐶, 𝐸, 𝐺, 𝐼, 𝑁, 𝑅}, some permutations are 𝐼𝐶𝐸, 𝑅𝐼𝑁𝐺, 𝑅𝐼𝐶𝐸, 𝑁𝐼𝐶𝐸𝑅, 𝑅𝐸𝐼𝐺𝑁 and 𝐶𝑅𝐼𝑁𝐺𝐸, but also 𝑅𝑁𝐶𝐺𝐼 – the sequence need not spell out an existing word. 𝐸𝑁𝐺𝐼𝑁𝐸, on the other hand, is not a permutation, because it uses the elements 𝐸 and 𝑁 twice.
If 𝑛 denotes the size of the set – the number of elements available for selection – and only
permutations are considered that use all 𝑛 elements, then the total number of possible permutations is equal to 𝑛!, where “!” is the factorial operator.
In general the number of permutations is denoted by 𝑃 𝑛, 𝑟 , or sometimes 𝑃𝑟𝑛, where: 𝑛 – the number of elements available for selection,
𝑟 – the number of elements to be selected (0 ≤ 𝑟 ≤ 𝑛).
For the case where 𝑟 = 𝑛 it has just been shown that 𝑃 𝑛, 𝑟 = 𝑛!. The general case is
given by the formula:
𝑃 𝑛, 𝑟 = 𝑛−𝑟 !𝑛! .
As before, this can be shown informally by considering the construction of an arbitrary permutation, but this time stopping when the length 𝑟 has been reached. The construction proceeds initially as above, but stops at length 𝑟. The number of possible permutations that has
then been reached is:
𝑃(𝑛, 𝑟) = 𝑛 ∙ (𝑛 − 1) ∙ (𝑛 − 2) ⋯ (𝑛 − 𝑟 + 1).
So:
𝑛! = 𝑛 ∙ 𝑛 − 1 ∙ 𝑛 − 2 ⋯ 2 ∙ 1 =
= 𝑛 ∙ 𝑛 − 1 ∙ 𝑛 − 2 ⋯ 𝑛 − 𝑟 + 1 ∙ 𝑛 − 𝑟 ⋯ 2 ∙ 1 = = 𝑃 𝑛, 𝑟 ∙ 𝑛 − 𝑟 ⋯ 2 ∙ 1 = 𝑃 𝑛, 𝑟 ∙ 𝑛 − 𝑟 !.
But if 𝑛! = 𝑃(𝑛, 𝑟) ∙ (𝑛 − 𝑟)!, then 𝑃(𝑛, 𝑟) = 𝑛!/(𝑛 − 𝑟)!. In the special case where 𝑛 = 𝑟 the formula above simplifies to:
𝑃 𝑛, 𝑟 =𝑛!0!=𝑛!1! = 𝑛!.
The reason why 0! = 1 is that 0! is an empty product, which always equals 1. An empty
the multiplicative identity, just as the empty sum – the result of adding no numbers – is zero, or the additive identity.
Since it may be impractical to calculate 𝑛! if the value of 𝑛 is very large, a more efficient
algorithm is to calculate:
𝑃(𝑛, 𝑟) = 𝑛 ∙ (𝑛 − 1) ∙ (𝑛 − 2) ⋯ (𝑛 − 𝑟 + 1).
3 Combinations
In combinatorial mathematics, a combination is an un-ordered collection of distinct elements, usually of a prescribed size and taken from a given set. (An ordered collection of distinct elements would sometimes be called a permutation, but that term is ambiguous; it can also mean “reordering of all terms”, among other related notions.) Given such a set 𝑆, a combination of elements of 𝑆 is just a subset of 𝑆, where, as always for (sub)sets the order of the
elements is not taken into account (two lists with the same elements in different orders are considered to be the same combination). Also, as always for (sub)sets, no elements can be repeated more than once in a combination; this is often referred to as a “collection without repetition”. For instance, {1,1,2} is not a combination of three digits; as a set this is the same as
{1,2,1} or {2,1,1} or even {1,2}. On the contrary, a poker hand can be described as a combination
of 5 cards from a 52-card deck: the order of the cards doesn't matter, and there can be no identical cards among the 5.
Thus for example, whereas the 2-permutations of the letters A, B, C, and D are
AB, AC, AD, BA, BC, BD, CA, CB, CD, DA, DB, DC,
the combinations of two out four of these letters are
AB, AC, AD, BC, BD, CD.
A 𝑘-combination (or 𝑘-subset) is a subset with 𝑘 elements (an 𝑛-set is a set containing exactly 𝑛 elements, where 𝑛 is a natural number).
The set of 𝑘-combinations of a set 𝑋 may be denoted by 𝑋𝑘 .
The number of 𝑘-combinations (each of size 𝑘) from a set 𝑆 with 𝑛 elements (size 𝑛) is
the binomial coefficient (also known as the “choose function”):
C 𝑛, 𝑘 = Ckn = nC
k = nk =k! n−k !n! ,
where 𝑛 is the number of objects from which you can choose and 𝑘 is the number to be chosen, and 𝑛! denotes the factorial.
The definition can be understood by considering a list of 𝑛 elements; the list can be ordered 𝑛! ways, and for each possible ordering can be partitioned into the first 𝑘 elements followed by the remaining 𝑛 − 𝑘 elements. The first partition is then a selection of 𝑘 elements
from the original list and all those partitions from every ordering cover all such selections. The complete permutation of the original list produces duplicate selections, however; some permutations result in a permuted but identical set for the first partition, and so we divide by 𝑘!
to remove these, and other permutations result in permuted second partitions, and so we divide by (𝑛 − 𝑘)! to remove these.
Base cases: 𝑛0 = 1, 𝑛1 = 𝑛.
Since as explained above a combination is a special case of a partition of a set; specifically, a partition into two sets of size 𝑘 and 𝑛 − 𝑘, you get the same number of combinations if you substitute 𝑘 with 𝑛 − 𝑘. Therefore, when 𝑘 is more than half of 𝑛, it may be easier to compute the binomial coefficient using 𝑛 − 𝑘 in place of 𝑘.
Example. The number of combinations of two out of the four letters 𝐴, 𝐵, 𝐶, and 𝐷 is found by letting 𝑛 = 4 and 𝑘 = 2. It is
4 2 =
4! 2! 4−2 ! =
4∙3∙2∙1 1∙2∙1∙2 = 6,
consistently with the listing given earlier.
3.2 Number of combinations with repetition
The number of combinations with repetition can be calculated as:
𝑛+𝑘−1 ! 𝑘! 𝑛−1 ! =
𝑛+𝑘−1
For example, if you have ten types of donuts (𝑛) on a menu to choose from and you want three donuts (𝑘) the number of ways to choose can be calculated as:
10+3−1 !
3! 10−1 ! = 220.
There is an easy way to understand the above result. Imagine we have 𝑛 + 𝑘 identical boxes arranged on a line. From these boxes (except the first one), we arbitrarily choose 𝑘 of them and mark the chosen boxes as empty. The rest of the boxes can be filled by the 𝑛 elements in the set 𝑆. For each non-empty box, if it is followed by 𝑀 successive empty boxes, we choose
the corresponding element in the non-empty box 𝑀 times. As a result, each arrangement of choosing empty boxes corresponds to a way of choosing 𝑘 out of the 𝑛 elements with repetition. The total number is therefore the number of combinations with repetition, which equals 𝑛+𝑘−1𝑘 .
4 Partitions
4.1 Multinomial coefficient
Recall that a combination is a choice of 𝑘 elements out of an 𝑛-element set without regard to order. This is the same as partitioning the set in two: one part contains 𝑘 elements and
the other contains the remaining 𝑛 − 𝑘. We now generalize by considering partitions in more than two subsets.
We have 𝑛 distinct objects and we are given nonnegative integers 𝑛1, 𝑛2, …, 𝑛𝑟, whose
sum is equal to 𝑛. The 𝑛 items are to be divided into 𝑟 disjoint groups, with the 𝑖th group containing exactly 𝑛𝑖 items. Let us count in how many ways this can be done.
We form the groups one at a time. We have 𝑛𝑛
1 ways of forming the first group. Having
formed the first group, we are left with 𝑛 − 𝑛1 objects. We need to choose 𝑛2 of them in order to
form the second group, and we have 𝑛−𝑛1
𝑛2 choices, etc. Using the Counting Principle for this
𝑟-stage process, the total number of choices is
𝑛 𝑛1
𝑛−𝑛1
𝑛2
𝑛−𝑛1−𝑛2
𝑛3 ⋯
𝑛−𝑛1−⋯−𝑛𝑟−1
𝑛𝑟 ,
𝑛! 𝑛1! 𝑛−𝑛1 !∙
𝑛−𝑛1 !
𝑛2! 𝑛−𝑛1−𝑛2 !⋯
𝑛−𝑛1−⋯−𝑛𝑟−1 !
𝑛−𝑛1−⋯−𝑛𝑟−1−𝑛𝑟 !𝑛𝑟!.
We note that several terms cancel and we are left with
𝑛! 𝑛1!𝑛2!⋯𝑛𝑟!.
This is called the multinomial coefficient and is usually denoted by
𝑛 𝑛1,𝑛2,…,𝑛𝑟 .
4.2 Ordered partition of a set
An ordered partition 𝑂 of a set 𝑆 is a sequence 𝐴1, 𝐴2, 𝐴3, ..., 𝐴𝑛 of subsets of 𝑆, with union is 𝑆, which are non-empty, and pairwise disjoint. This differs from a partition of a set, in that the order of the 𝐴𝑖 matters.
For example, one ordered partition of {1, 2, 3, 4, 5} is {1, 2} {3, 4} {5}
which is equivalent to {1, 2} {4, 3} {5}
but distinct from
{3, 4} {1, 2} {5}.
The number of ordered partitions 𝑇𝑛 of {1, 2, . . . , 𝑛} can be found recursively by the formula:
𝑇𝑛 = 𝑛−1 𝑛𝑖 𝑇𝑖
𝑖=0 .
An ordered partition of “type 𝑘1+ … + 𝑘𝑚“ is one in which the 𝑖-th part has 𝑘𝑖 members, for 𝑖 = 1, . . . , 𝑚. The number of such partitions is given by the multinomial coefficient
𝑛 𝑘1,…,𝑘𝑚 =
𝑛! 𝑘1!…𝑘𝑚!.
type 1 + 1 + 1: 6; type 2 + 1: 3; type 1 + 2: 3; type 3: 1.
Together this is the ordered Bell number 13 (the 𝑛th Bell number is the number of partitions of a set with 𝑛 members).
Summary of Counting Results
Permutations of 𝑛 objects: 𝑛!.
𝑘-permutations of 𝑛 objects: 𝑛!/(𝑛 − 𝑘)!.
Combinations of 𝑘 out of 𝑛 objects: nk =𝑘! 𝑛−𝑘 !𝑛! .
Partitions of 𝑛 objects into 𝑟 groups with the 𝑖th group having 𝑛𝑖 objects:
𝑛
𝑛1,𝑛2,…,𝑛𝑟 =
