Trials, events and experiments

It is beyond our scope to define probabilities, events, probability spaces, sample spaces and chance experiments rigorously. The exposition below is heuristic and therefore not entirely correct because (technical) details must be omitted. However, you should

get a feel for what these concepts mean. The following sections are based on Papoulis (1965) and Evans et al. (2000).

We start with the following definition:

Outcome Any observable phenomenon is said to be an outcome.

In the context of probability theory, we define a set of outcomes from the description of an experiment. The outcomes may not be unique, so we must agree upon their definition to avoid ambiguity. We associate uncertainty with outcomes. The uncer-tainty is measured with probability. The latter ranges from 0 to 1. The probability of an outcome that is certain to occur is 1 and the probability of an outcome that never occurs is 0. An experiment here does not necessarily mean some activity that we undertake. It refers to anything we wish to observe.

Example 4.12. Table 4.1 illustrates some experiments and their outcomes. Note that the outcomes can be factors, integers, real numbers or anything else you wish to define

as an outcome. ut

Table 4.1 Experiments and outcomes.

Experiment Outcomes

Observing a sick person sick, healthy, dead Treating a sick person sick, healthy, dead Rolling a die number of dots facing up

Earthquake magnitude

Weighing an elephant the elephant’s weight

The definition of outcome leads to another important concept in probability theory, namely the definition of

Sample space The set of all possible outcomes, denoted by S, is called the sample space. A sample space is also known as an event space, possibility space or simply the space.

Example 4.13. The sample space of the state of two organisms (dead (d) or alive (a)) is

S ={aa, ad, da, dd} . The sample space of the magnitude of an earthquake is

S = the set of all real numbers .

u t With the concept of sample space, we have the definition of

Event An event is a subset of the sample space.

In notation, if S is a sample space, then E⊂ S is an event. Because sets are subsets of themselves, S is also an event.

Example 4.14. In the context of an experiment, we may define the sample space of observing a person as

S ={sick, healthy, dead} .

Trials, events and experiments 105

Therefore, the following are all events:

{sick} , {healthy} , {dead} ,

{sick, healthy} , {sick, dead} , {healthy, dead} , {sick, healthy, dead} , {none of the above} .

The sample space of elephant weights is

S = real numbers . Therefore, the following are all events:

the set of all real numbers between − ∞ and ∞ , the set of all real numbers between − 10 and 5.32 , any real number .

u t You may object to some event definitions for elephant weights. However, we can assign 0 probability to events. Therefore, negative weights are acceptable as weights, as long as we assign zero probability to them. A special kind of event is an

Elementary (or simple) event An event that cannot be divided into subsets.

Example 4.15. Consider a study of animal movement. We classify behaviors as a -standing, b - walking and c - running. Then A :={a}, B := {b} and C := {c} are all elementary events. a, b and c are not events at all because they are not sets. We may consider an observation as a complete sequence (in any order) of A, B and C. Then {a, b, c} is an elementary event, but {A, B, C} is not. ut Because events are subsets of S, they can include more than one outcome. If one of these outcomes occurred, we say that the event occurred.

Example 4.16. In a study of animal behavior, we classify the following events: A = standing, B = walking, C = running, D = lying and E = other. We are interested in two events: F - moving, G - not moving. Then if we observe the animal walking, we say that event F occurred. Similarly, if we observe the animal laying, then we say

that event G occurred. ut

Recall that we defined disjoint sets as those sets whose intersection is empty. For events, we define

Disjoint events A and B are said to be disjoint events if A∩ B = ∅.

Example 4.17. Let A = pneumonia, B = gangrene and C = dead be possible out-comes of the observation that a person is alive. Then A and C are disjoint events. A

and B are not. ut

The ideas of outcome, sample space and event lead to the following definition:

Trial A single performance of an experiment whose outcome is in S. The following are examples of trials: flipping a coin, rolling a die, treating a pond with rotenone, treating a patient with a particular drug and recording the magnitude of an earthquake. The simplest trial is defined as

Bernoulli trial A trial with only two possible outcomes, one arbitrarily named a success and the other a failure.

Example 4.18. A single flip of a coin is one example of a Bernoulli trial. It may consists of an idealized coin—a circular disk of zero thickness. When flipped, it will come to rest with either face up (“heads”, H, or “tails”, T ) with equal probability. A regular coin is a good approximation of the idealized coin. ut Chance experiment A chance experiment (or experiment for short) is a trial with more than one possible outcomes where the amount of uncertainty of different outcomes and their combinations is known or deducible.

Example 4.19. Flipping a fair coin, with outcomes defined as H and T is a chance experiment. The sample space is

S ={H, T } .

and the amount of uncertainty of any outcome or their combinations is known.

In a medical study, giving patients a drug and observing the outcome is a chance experiment. The outcome is uncertain and we assign hypothetical probability to out-comes (e.g. healthy, sick, or dead). When the experiment is over, we may use the results to test if our hypotheses about the probabilities—of being healthy, sick, or

dead—were justified. ut

Here is an example where a chance experiment is defined and the sample space is determined.

Example 4.20. You observe deer crossing the highway. The experiment consists of observing the sex of two consecutive deer. Let M be the event that a male crossed the highway and F the event that a female did. To define the sample space, we use a tree diagram (Figure 4.7). Here, the set of all possible outcomes is a female crossed the road and then a female or a male, a male crossed the road and then a female or a male. Therefore,

S ={F F, F M, MF, MM} .

Figure 4.7 A tree diagram.

Trials, events and experiments 107 To create the sample space with R, we use combn().

> no.dimnames(t( combn(c('F', 'M', 'F', 'M'), 2)))

"F" "M"

"F" "F"

"F" "M"

"M" "F"

"M" "M"

"F" "M"

From the innermost parentheses out: We create a vector that labels the possible out-comes in the first and second pair of observations. Next, combn() creates a matrix of all possible combinations of two from the vector. Next, we transpose the matrix (i.e. rows become columns) with t(). Finally, we print this matrix with no.dimnames() (see

page 32). ut

When the number of possible outcomes is small, we can present the possible outcomes with a tree diagram. In Example 4.20, an elementary event consists of two (not one) crossings. Here is another example.

Example 4.21. Mist nets are used to catch birds. They are made of fine nylon mesh so birds do not see them. The nets hang somewhat loosely and when a bird flies into one, it gets tangled. Different meshes are used to catch different sizes of birds. Suppose you have four mist nets, each of a different mesh. Call them nets 1, 2, 3 and 4. You wish to allocate each of these nets to one of four study areas named a, b, c and d.

Here are the possible outcomes:

> p <- t(expand.grid(letters[1 : 4], 1 : 4))

> no.dimnames(noquote(p))

a b c d a b c d a b c d a b c d 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4

The function expand.grid() creates a data frame from all combinations of the sup-plied vectors. We give it a vector of the letters a through d and the sequence of numbers 1 through 4. With t() we switch columns and rows in the resulting data frame p. To see the combinations without quotes, we use noquote() and to see it with-out the dimension names we use no.dimnames(). The latter is discussed on page 32.

The output from the calls above produces the sampling space S. Here we label an elementary event by a pair of one letter and one digit—for example, (a, 1) means that net 1 was assigned to area a. Because they are subsets of S, the following are events. The set of all events such that the chosen area is a is

> (A <- no.dimnames(noquote(p[, p[1, ] == 'a']))) a a a a

1 2 3 4

From the inside out, the expression p[1, ] =='a' returns TRUE for all those columns in the first row of p that have the value a. Next, p[, p[1, ] == 'a'] returns the

subset of the data frame that has a in its first row. To avoid clutter, we print A with no dimension names. The events where mist net 1 is allocated to all areas are obtained with

> (B <- no.dimnames(noquote(p[, p[2, ] == 1]))) a b c d

1 1 1 1

Again, from the inside out, the expression p[2, ] == 1 returns TRUE for all those columns in the second row of p that have the value 1. Next, p[, p[2, ] == 1]

returns the subset of the data frame that has 1 in its second row. Again, to avoid clutter, we print B with no dimension names. Now the intersection of the events A and B gives the single event

> noquote(intersect(A, B)) [1] a 1

You could accomplish all of this without R. However, if your sets are more complex and the combinations and subsets more elaborate, deriving the results by hand may be

tedious. ut

One of the most fundamental and pervasive experiments is the Bernoulli experiment.

Bernoulli experiment We call an experiment with a single event and two outcomes a Bernoulli experiment.

Example 4.22. The simplest example of a Bernoulli experiment is a flip of a coin.

The event is a side of a coin landing face up. The two possible outcomes are heads

(H) or tails (T ). ut

A Bernoulli experiment is important in its own right. In many situations in life we face binary choices with no certain outcomes. We drive a car and may or may not get into an accident. Any aspect of computer logic and computations whose outcome may not be certain involves binary operations. In decision making we often reduce choices to binary operations—to act or not to act. Bernoulli trials serve as the starting point for many other probability models which we shall meet as we proceed.