Chapter 3: Probability Topics
3.1 Terminology
Learning Objectives
By the end of this section, you should be able to:
- Define and use common probability terms
- Compute probabilities for common experiments with equally likely outcomes
- Combine events and compute the resulting probabilities
- Compute conditional probabilities
Probability is a measure that is associated with how certain we are of outcomes of a particular experiment or activity. An experiment is a planned operation carried out under controlled conditions. If the result is not predetermined, then the experiment is said to be a probability experiment. Flipping one fair coin twice is an example of an experiment.
A result of an experiment is called an outcome. The sample space of an experiment is the set of all possible outcomes. Three ways to represent a sample space are: to list the possible outcomes, to create a tree diagram, or to create a Venn diagram. The uppercase letter [latex]S[/latex] is used to denote the sample space. For example, if you flip a fair coin once, the possible outcomes are [latex]H[/latex] = heads and [latex]T[/latex] = tails, so the sample space is [latex]S = \{ H, T \}[/latex].
An event is a set containing any combination of outcomes. Upper case letters like [latex]A[/latex] and [latex]B[/latex] represent events. For example, if the experiment is to flip one fair coin twice, event [latex]A[/latex] might be getting at most one head.
Example
Consider the sample space [latex]S[/latex] that contains the whole numbers (starting at one) less than 20.
Let event [latex]A[/latex] = the even numbers and event [latex]B[/latex] = numbers greater than 13.
List the elements of [latex]S, A, \text{ and }B.[/latex]
Definition of Probability
The probability of any outcome is the long-term relative frequency of that outcome. Probabilities are between zero and one, inclusive (that is, zero and one and all numbers between these values). The probability of an event [latex]A[/latex] is written [latex]P(A)[/latex]. If an event [latex]A[/latex] cannot occur, then [latex]P(A) = 0[/latex]. If the event is guaranteed to occur, [latex]P(A) = 1[/latex]. If [latex]P(A) = \frac{1}{2} = 0.5[/latex], the event A is equally likely to occur or not to occur. For example, if you flip one fair coin repeatedly (from 20 to 2,000 to 20,000 times) the relative frequency of heads approaches 0.5 (the probability of heads).
Equally likely means that each outcome of an experiment occurs with equal probability. For example, if you toss a fair, six-sided die, each face (1, 2, 3, 4, 5, or 6) is as likely to occur as any other face. If you toss a fair coin, a Head ([latex]H[/latex]) and a Tail ([latex]T[/latex]) are equally likely to occur. If you randomly guess the answer to a true/false question on an exam, you are equally likely to select a correct answer or an incorrect answer.
To calculate the probability of an event [latex]A[/latex] when all outcomes in the sample space are equally likely, count the number of outcomes for event [latex]A[/latex] and divide by the total number of outcomes in the sample space.
Example
If you toss a fair dime and a fair nickel, the sample space is [latex]\{HH, TH, HT, TT\}[/latex]. The sample space has four outcomes. Let [latex]A =[/latex] getting exactly one Head. There are two outcomes that meet this condition, so [latex]A = \{HT, TH \}[/latex], so [latex]P(A) = \frac{2}{4} = 0.5[/latex].
The Law of Large Numbers
An important characteristic of probability experiments, known as the law of large numbers, states that as the number of repetitions of an experiment is increased, the relative frequency obtained in the experiment tends to become closer and closer to the theoretical probability. Even though the outcomes do not happen according to any set pattern or order, overall, the long-term observed relative frequency will approach the theoretical probability. (The word empirical is often used instead of the word observed.)
You toss a coin and record the result. What is the probability that the result is heads? If you flip a coin two times, does probability tell you that these flips will result in one heads and one tail? You might toss a fair coin ten times and record nine heads. Probability does not describe the short-term results of an experiment, rather it gives information about what can be expected in the long term. To demonstrate this, Karl Pearson once tossed a fair coin 24,000 times! He recorded the results of each toss, obtaining heads 12,012 times. In his experiment, Pearson illustrated the Law of Large Numbers.
Example
Suppose you roll one fair six-sided die, with the numbers {1, 2, 3, 4, 5, 6} on its faces. Let event E = rolling a number that is at least five.
There are two outcomes {5, 6}, so [latex]P(E) = \frac{2}{6} = \frac{1}{3}[/latex].
If you were to roll the die only a few times, you would not be surprised if your observed results did not match the probability. But if you were to roll the die a very large number of times, you would expect that, overall, [latex]\frac{2}{6}[/latex] of the rolls would result in an outcome of “at least five”. You would not expect exactly [latex]\frac{2}{6}[/latex]. The long-term relative frequency of obtaining this result would approach the theoretical probability of [latex]\frac{2}{6}[/latex] as the number of repetitions grows larger and larger.
It is important to realize that in many situations, the outcomes are not equally likely. A coin or die may be unfair, or biased. Two math professors in Europe had their statistics students test the Belgian one Euro coin and discovered that in 250 trials, a head was obtained 56% of the time and a tail was obtained 44% of the time. The data seem to show that the coin is not a fair coin; more repetitions would be helpful to draw a more accurate conclusion about such bias. Some dice may be biased. Look at the dice in a game you have at home; the spots on each face are usually small holes carved out and then painted to make the spots visible. Your dice may or may not be biased; it is possible that the outcomes may be affected by the slight weight differences due to the different numbers of holes in the faces. Gambling casinos make a lot of money depending on outcomes from rolling dice, so casino dice are made differently to eliminate bias. Casino dice have flat faces; the holes are completely filled with paint having the same density as the material that the dice are made out of so that each face is equally likely to occur. Later we will learn techniques to use to work with probabilities for events that are not equally likely.
Combining Events
Often we are not just interested in a single event, but multiple events which may or may not happen at the same time. In order to find probabilities relating to multiple events, we first look at how to combine two events into a new one.
First, consider an "AND" Event: An outcome is in the event [latex]A \text{ AND } B[/latex] if the outcome is in both [latex]A[/latex] and [latex]B[/latex] at the same time. In set notation, this is called the intersection of two events, denoted [latex]A \cap B[/latex], and the set [latex]A \cap B[/latex] contains the outcomes that are in both events [latex]A \text{ AND } B[/latex].
Example
Let [latex]A = \{ 1, 2, 3, 4, 5\}[/latex] and [latex]B = \{4, 5, 6, 7, 8\}.[/latex]
Then [latex]A \text{ AND } B = A \cap B = \{ 4, 5\}[/latex].
Next, consider an "OR" Event: An outcome is in the event [latex]A \text{ OR } B[/latex] if the outcome is in [latex]A[/latex] or is in [latex]B[/latex] or is in both [latex]A[/latex] and [latex]B[/latex]. In set notation, this is called the union of two events, denoted [latex]A \cup B[/latex], and the set [latex]A \cup B[/latex] contains the outcomes that are in either event A or B or both.
Example
Let [latex]A = \{ 1, 2, 3, 4, 5\}[/latex] and [latex]B = \{4, 5, 6, 7, 8\}.[/latex]
Then [latex]A \text{ OR } B = A \cup B = \{1, 2, 3, 4, 5, 6, 7, 8\}[/latex]. Notice that 4 and 5 are NOT listed twice; duplicates are not listed in set notation.
Third, the complement of event [latex]A[/latex] is denoted [latex]A'[/latex] (read “A prime”) and consists of all outcomes that are NOT in A. As a result, [latex]P(A) + P(A') = 1[/latex].
Example
Let [latex]S = \{1, 2, 3, 4, 5, 6\}[/latex], like when rolling a fair six-sided die, and let [latex]A = \{1, 2, 3, 4\}[/latex]. Compute [latex]P(A)[/latex] and [latex]P(A')[/latex].
Finally, the conditional probability of [latex]A[/latex] given [latex]B[/latex], written [latex]P(A | B)[/latex], is the probability that event [latex]A[/latex] will occur given that the event [latex]B[/latex] has already occurred. A conditional reduces the sample space: we calculate the probability of [latex]A[/latex] from the reduced sample space containing the outcomes where [latex]B[/latex] has occurred. Assuming [latex]P(B)[/latex] is greater than zero, the formula to calculate [latex]P(A | B)[/latex]is
[latex]P(A | B) = \dfrac{P\left(A\text{ AND }B\right)}{P\left(B\right)}[/latex], or equivalently,
[latex]P(A | B) = \dfrac{P\left(A \cap B\right)}{P\left(B\right)}[/latex].
Example
Suppose we toss one fair, six-sided die. The sample space is [latex]S = \{1, 2, 3, 4, 5, 6\}[/latex]. Let [latex]A =[/latex] “face is 2 or 3” and [latex]B =[/latex] “face is even.” Calculate [latex]P(A | B)[/latex] two ways, first finding the smaller sample space, and then using the formula.
It is important to read each problem carefully to think about and understand what the events are. Understanding the wording is the first very important step in solving probability problems. Reread the problem several times if necessary. Clearly identify the event of interest. Determine whether there is a condition stated in the wording that would indicate that the probability is conditional; carefully identify the condition, if any.
Example
Consider the sample space [latex]S[/latex] that contains the whole numbers (starting at one) less than 20. Let event [latex]A[/latex] = the even numbers and event [latex]B[/latex] = numbers greater than 13. Compute each set or probability.
- [latex]P(A) = \underline{\hspace{2in}}[/latex]
- [latex]P(B) = \underline{\hspace{2in}}[/latex]
- [latex]A \cap B = \underline{\hspace{2in}}[/latex]
- [latex]A \cup B = \underline{\hspace{2in}}[/latex]
- [latex]P(A \cap B) = \underline{\hspace{2in}}[/latex]
- [latex]P(A \cup B)= \underline{\hspace{2in}}[/latex]
- [latex]A' = \underline{\hspace{2in}}[/latex]
- [latex]P(A') = \underline{\hspace{2in}}[/latex]
- [latex]P(A) + P(A')= \underline{\hspace{2in}}[/latex]
- [latex]P(A | B ) = \underline{\hspace{2in}}[/latex]
- [latex]P(B | A) = \underline{\hspace{2in}}[/latex]
Your turn!
The sample space S is the ordered pairs of two whole numbers, the first from one to three and the second from one to four (Example: (1, 4)). Let event A = the sum is even and event B = the first number is prime.
- Enumerate the set [latex]S[/latex].
- Enumerate the set [latex]A[/latex].
- Enumerate the set [latex]B[/latex].
- Compute [latex]P(A)[/latex].
- Compute [latex]P(B)[/latex].
- Enumerate the set [latex]A \cap B[/latex].
- Enumerate the set [latex]A \cup B[/latex].
- Compute [latex]P(A \cap B)[/latex].
- Compute [latex]P(A \cup B)[/latex].
- Enumerate the set [latex]B'[/latex].
- Compute [latex]P(B')[/latex]
- Check that [latex]P(B) + P(B') = 1[/latex].
- Compute [latex]P(A|B)[/latex] and [latex]P(B|A)[/latex]. Are they equal?
Your Turn!
A fair, six-sided die is rolled. Describe the sample space S, identify each of the following events with a subset of S and compute its probability (an outcome is the number of dots that show up).
- Event T = the outcome is two.
- Event A = the outcome is an even number.
- Event B = the outcome is less than four.
- The complement of A.
- A GIVEN B
- B GIVEN A
- A AND B
- A OR B
- A OR B′
- Event N = the outcome is a prime number.
- Event I = the outcome is seven.
Solution
- T = {2}, P(T) = [latex]\frac{1}{6}[/latex]
- A = {2, 4, 6}, P(A) = [latex]\frac{1}{2}[/latex]
- B = {1, 2, 3}, P(B) = [latex]\frac{1}{2}[/latex]
- A′ = {1, 3, 5}, P(A′) = [latex]\frac{1}{2}[/latex]
- Reduced sample space: A|B = {2}, P(A|B) = [latex]\frac{1}{3}[/latex]
- Reduced sample space: B|A = {2}, P(B|A) = [latex]\frac{1}{3}[/latex]
- A AND B = {2}, P(A AND B) = [latex]\frac{1}{6}[/latex]
- A OR B = {1, 2, 3, 4, 6}, P(A OR B) = [latex]\frac{5}{6}[/latex]
- A OR B′ = {2, 4, 5, 6}, P(A OR B′) = [latex]\frac{2}{3}[/latex]
- N = {2, 3, 5}, P(N) = [latex]\frac{1}{2}[/latex]
- A six-sided die does not have seven dots. P(7) = 0.
Your turn!
The table below shows the enrollment of 100 students in two freshman courses, English and Speech.
| Taking English | Not Taking English | |
| Taking Speech | 43 | 9 |
| Not Taking Speech | 44 | 4 |
Denote the events [latex]S =[/latex] taking Speech, [latex]NS =[/latex] Not taking Speech, [latex]E = [/latex] taking English, and [latex]NE =[/latex] Not taking English. Compute the following probabilities.
- [latex]P(E)[/latex]
- [latex]P(S)[/latex]
- The probability that a student is not taking English.
- The probability that a student is not taking Speech.
- [latex]P(NE \cap NS)[/latex]
- The probability that a student is taking both English and Speech.
- The probability that a student is taking English and not Speech.
- The probability that a student is taking English or Speech.
- [latex]P(E')[/latex]
- The probability that a student is taking English or not taking Speech.
- The probability that a given Speech student is taking English.
- The probability that a given English student is taking Speech.
References
“Countries List by Continent.” Worldatlas, 2013. Available online at http://www.worldatlas.com/cntycont.htm (accessed May 2, 2013).
a number between zero and one, inclusive, that gives the likelihood that a specific event will occur; the foundation of statistics is given by the following 3 axioms (by A.N. Kolmogorov, 1930’s): Let S denote the sample space and A and B are two events in S. Then:
• 0 ≤ P(A) ≤ 1
• If A and B are any two mutually exclusive events, then P(A OR B) = P(A) + P(B).
• P(S) = 1
A random experiment where the result is not predetermined
a planned activity carried out under controlled conditions
the set of all possible outcomes of an experiment
a subset of the set of all outcomes of an experiment; the set of all outcomes of an experiment is called a sample space and is usually denoted by S. An event is an arbitrary subset in S. It can contain one outcome, two outcomes, no outcomes (empty subset), the entire sample space, and the like. Standard notations for events are capital letters such as A, B, C, and so on.
a particular result of an experiment
Each outcome of an experiment has the same probability.
As the number of trials in a probability experiment increases, the difference between the theoretical probability of an event and the relative frequency probability approaches zero.
An outcome is in the event A AND B if the outcome is in both A AND B at the same time.
The shared outcomes of two events
An outcome is in the event A OR B if the outcome is in A or is in B or is in both A and B.
The set of all outcomes in two events; outcomes can be in either or both events
The complement of event A consists of all outcomes that are NOT in A.
the likelihood that an event will occur given that another event has already occurred