Conditional probability and subjective probability

There is a tendency to refer to “probability” as objective, and to conditional probability as subjective. First, note that proba-bility is always conditional. When we say that the probaproba-bility of the outcome “4” of throwing a die is ¹/6, we actually mean that the conditional probability of the outcome “4,” given that one of the possible outcomes: 1, 2, 3, 4, 5, 6 has occurred, or will occur, that the die is fair and that we threw it at random, and any other information that is relevant. We usually suppress this given information in our notation and refer to it as the unconditional probability. This is considered as an objective probability.²⁰

20Here, we want to stress the change in the extent of “objectivity” (or subjectivity), when moving from probability of an event to a conditional probability.

Introduction to Probability Theory, Information Theory, and all the Rest 43

Now, let us consider the following two pairs of examples:

O₁: The conditional probability of an outcome “4,” given that Jacob knows that the outcome is “even,” is¹/3.

O2: The conditional probability of an outcome “4,” given that Abraham knows that the outcome is “odd,” is zero.

S₁: The conditional probability that the “defendant is guilty,”

given that he was seen by the police at the scene of the crime, is 9/10.

S2: The conditional probability that the “defendant is guilty,”

given that he was seen by at least five persons in another city at the time of the crime’s commission, is nearly zero.

In all of the aforementioned examples, there is a tendency to refer to conditional probability as a subjective probability. The reason is that in all the abovementioned examples, we involved personal knowledge of the conditions. Therefore, we judge that it is highly subjective. However, that is not so. The two proba-bilities, denoted O₁ and O₂, are objective probabilities.

The fact that we mention the names of the persons, who are knowledgeable of the conditions, does not make the conditional probability subjective. We could make the same statement as in O₁, but with Rachel instead of Jacob. The conditional prob-ability of an outcome “4,” given that Rachel knows that the outcome is even, is ¹/3. The result is the same. The subjectivity of this statement is just an illusion resulting from the involve-ment of the name of the person who “knows” the condition. A better way of rephrasing O1 is:

The conditional probability of an outcome “4,” given that we know that the outcome is even, is¹/3, or even better; the con-ditional probability of an outcome “4,” given that the outcome is “even,” is¹/3.

In the last two statements, it is clear that the fact that Jacob or Rachel, or anyone of us knowing the condition does not

have any effect on the conditional probability. In the last state-ment, we made the condition completely impersonal. Thus, we can conclude that the given condition does not, in itself, con-vert an objective (unconditional) probability into a subjective probability.

Consider the following paragraph from Callen (1983):

“The concept of probability has two distinct interpreta-tions in common usage. ‘Objective probability’ refers to a frequency, or a fractional occurrence; the assertion that

‘the probability of newborn infants being male is slightly less than one half’ is a statement about census data. ‘Sub-jective probability’ is a measure of expectation based on less than optimum information. The (subjective) prob-ability of a particular yet unborn child being male, as assessed by a physician, depends upon that physician’s knowledge of the parents’ family histories, upon accumu-lating data on maternal hormone levels, upon the increas-ing clarity of ultrasound images, and finally upon an edu-cated, but still subjective, guess.”

Although it is not explicitly said, what the author implies is that in the first example: “the probability of a newborn infant being male is slightly less than one half” as stated is an answer to an unconditional probability question, “What is the probability of a newborn infant being male?” The second example is cast in the form of an answer to a conditional probability question:

“What is the probability of a particular yet unborn child being male, given. . . all the information as stated.”

Clearly, the answer to the second question is highly subjec-tive. Different doctors who are asked this question will give dif-ferent answers. However, the same is true for the first question if given different information. What makes the second ques-tion and its answer subjective is not the condiques-tion or the specific

Introduction to Probability Theory, Information Theory, and all the Rest 45

information, or the specific knowledge of this or that doctor, but the lack of sufficient knowledge. Insufficient knowledge confers liberty to give any (subjective) answer to the second question.

The same is true for the first question. If all the persons who are asked have the same knowledge as stated, i.e. no informa-tion, they are free to give any answer they might have guessed.

The first question is not “about the census data” as stated in the quotation; it is a question on the probability of the occur-rence of a male gender newborn infant, given the information on “census data.” If you do not have any information, you can-not answer this “objective question,” but anyone given the same information on the “census data” will necessarily give the same objective answer.

There seems to be a general agreement that there are essen-tially two distinct types of probabilities. One is referred to as the judgmental probability which is highly subjective, and the sec-ond is the physical or scientific probability which is considered as an objective probability. Both of these can either be condi-tional or uncondicondi-tional. In this book, we shall use only scien-tific, hence, objective probabilities. In using probability in all the sciences, we always assume that the probabilities are given either explicitly or implicitly by a given recipe on how to calcu-late these probabilities. While these are sometimes very easy to calculate, at other times, they are very difficult,²¹ but you can

21For instance, the probability of drawing three red marbles, five blue marbles and two green marbles from an urn containing 300 marbles, 100 of each color, given that the marbles are identical and that you drew 10 marbles at random, and so on. . . This is a slightly more difficult problem, and you might not be able to calculate it, but the probability of this event is “there” in the event itself. Similarly, the probability of finding two atoms at a certain distance from each other, in a liquid at a given temper-ature and pressure, given that you know and accept the rules of statistical mechanics, and that you know that these rules have been extremely useful in predicting many average properties of macroscopic properties, etc. This probability is objective! You might not be able to calculate it, but you know it is “there” in the event.

always assume that they are “there” in the event, as much as mass is attached to any piece of matter.