\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior\n",
"Bowl 1 0.5\n",
"Bowl 2 0.5"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table['prior'] = 1/2, 1/2\n",
"table"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And a column for the likelihoods:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.841661Z",
"iopub.status.busy": "2021-04-16T19:35:10.840937Z",
"iopub.status.idle": "2021-04-16T19:35:10.843527Z",
"shell.execute_reply": "2021-04-16T19:35:10.843881Z"
},
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"prior | |
---|---|

Bowl 1 | 0.5 |

Bowl 2 | 0.5 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood\n",
"Bowl 1 0.5 0.75\n",
"Bowl 2 0.5 0.50"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table['likelihood'] = 3/4, 1/2\n",
"table"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we see a difference from the previous method: we compute likelihoods for both hypotheses, not just Bowl 1:\n",
"\n",
"* The chance of getting a vanilla cookie from Bowl 1 is 3/4.\n",
"\n",
"* The chance of getting a vanilla cookie from Bowl 2 is 1/2.\n",
"\n",
"You might notice that the likelihoods don't add up to 1. That's OK; each of them is a probability conditioned on a different hypothesis.\n",
"There's no reason they should add up to 1 and no problem if they don't.\n",
"\n",
"The next step is similar to what we did with Bayes's Theorem; we multiply the priors by the likelihoods:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.847394Z",
"iopub.status.busy": "2021-04-16T19:35:10.846784Z",
"iopub.status.idle": "2021-04-16T19:35:10.856472Z",
"shell.execute_reply": "2021-04-16T19:35:10.856847Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | |
---|---|---|

Bowl 1 | 0.5 | 0.75 |

Bowl 2 | 0.5 | 0.50 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm\n",
"Bowl 1 0.5 0.75 0.375\n",
"Bowl 2 0.5 0.50 0.250"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table['unnorm'] = table['prior'] * table['likelihood']\n",
"table"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I call the result `unnorm` because these values are the \"unnormalized posteriors\". Each of them is the product of a prior and a likelihood:\n",
"\n",
"$$P(B_i)~P(D|B_i)$$\n",
"\n",
"which is the numerator of Bayes's Theorem. \n",
"If we add them up, we have\n",
"\n",
"$$P(B_1)~P(D|B_1) + P(B_2)~P(D|B_2)$$\n",
"\n",
"which is the denominator of Bayes's Theorem, $P(D)$.\n",
"\n",
"So we can compute the total probability of the data like this:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.860857Z",
"iopub.status.busy": "2021-04-16T19:35:10.860423Z",
"iopub.status.idle": "2021-04-16T19:35:10.863717Z",
"shell.execute_reply": "2021-04-16T19:35:10.864069Z"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.625"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"prob_data = table['unnorm'].sum()\n",
"prob_data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notice that we get 5/8, which is what we got by computing $P(D)$ directly.\n",
"\n",
"And we can compute the posterior probabilities like this:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.872145Z",
"iopub.status.busy": "2021-04-16T19:35:10.871531Z",
"iopub.status.idle": "2021-04-16T19:35:10.875239Z",
"shell.execute_reply": "2021-04-16T19:35:10.874623Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | |
---|---|---|---|

Bowl 1 | 0.5 | 0.75 | 0.375 |

Bowl 2 | 0.5 | 0.50 | 0.250 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"Bowl 1 0.5 0.75 0.375 0.6\n",
"Bowl 2 0.5 0.50 0.250 0.4"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table['posterior'] = table['unnorm'] / prob_data\n",
"table"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The posterior probability for Bowl 1 is 0.6, which is what we got using Bayes's Theorem explicitly.\n",
"As a bonus, we also get the posterior probability of Bowl 2, which is 0.4.\n",
"\n",
"When we add up the unnormalized posteriors and divide through, we force the posteriors to add up to 1. This process is called \"normalization\", which is why the total probability of the data is also called the \"normalizing constant\"."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Dice Problem\n",
"\n",
"A Bayes table can also solve problems with more than two hypotheses. For example:\n",
"\n",
"> Suppose I have a box with a 6-sided die, an 8-sided die, and a 12-sided die. I choose one of the dice at random, roll it, and report that the outcome is a 1. What is the probability that I chose the 6-sided die?\n",
"\n",
"In this example, there are three hypotheses with equal prior\n",
"probabilities. The data is my report that the outcome is a 1. \n",
"\n",
"If I chose the 6-sided die, the probability of the data is\n",
"1/6. If I chose the 8-sided die, the probability is 1/8, and if I chose the 12-sided die, it's 1/12.\n",
"\n",
"Here's a Bayes table that uses integers to represent the hypotheses:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.879515Z",
"iopub.status.busy": "2021-04-16T19:35:10.878981Z",
"iopub.status.idle": "2021-04-16T19:35:10.881188Z",
"shell.execute_reply": "2021-04-16T19:35:10.880775Z"
}
},
"outputs": [],
"source": [
"table2 = pd.DataFrame(index=[6, 8, 12])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'll use fractions to represent the prior probabilities and the likelihoods. That way they don't get rounded off to floating-point numbers."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.886867Z",
"iopub.status.busy": "2021-04-16T19:35:10.886335Z",
"iopub.status.idle": "2021-04-16T19:35:10.894831Z",
"shell.execute_reply": "2021-04-16T19:35:10.894457Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

Bowl 1 | 0.5 | 0.75 | 0.375 | 0.6 |

Bowl 2 | 0.5 | 0.50 | 0.250 | 0.4 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood\n",
"6 1/3 1/6\n",
"8 1/3 1/8\n",
"12 1/3 1/12"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from fractions import Fraction\n",
"\n",
"table2['prior'] = Fraction(1, 3)\n",
"table2['likelihood'] = Fraction(1, 6), Fraction(1, 8), Fraction(1, 12)\n",
"table2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once you have priors and likelhoods, the remaining steps are always the same, so I'll put them in a function:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.898394Z",
"iopub.status.busy": "2021-04-16T19:35:10.897966Z",
"iopub.status.idle": "2021-04-16T19:35:10.899624Z",
"shell.execute_reply": "2021-04-16T19:35:10.899989Z"
}
},
"outputs": [],
"source": [
"def update(table):\n",
" \"\"\"Compute the posterior probabilities.\"\"\"\n",
" table['unnorm'] = table['prior'] * table['likelihood']\n",
" prob_data = table['unnorm'].sum()\n",
" table['posterior'] = table['unnorm'] / prob_data\n",
" return prob_data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And call it like this."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.904682Z",
"iopub.status.busy": "2021-04-16T19:35:10.904149Z",
"iopub.status.idle": "2021-04-16T19:35:10.905883Z",
"shell.execute_reply": "2021-04-16T19:35:10.906265Z"
}
},
"outputs": [],
"source": [
"prob_data = update(table2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is the final Bayes table:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.913780Z",
"iopub.status.busy": "2021-04-16T19:35:10.913058Z",
"iopub.status.idle": "2021-04-16T19:35:10.916229Z",
"shell.execute_reply": "2021-04-16T19:35:10.915856Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | |
---|---|---|

6 | 1/3 | 1/6 |

8 | 1/3 | 1/8 |

12 | 1/3 | 1/12 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"6 1/3 1/6 1/18 4/9\n",
"8 1/3 1/8 1/24 1/3\n",
"12 1/3 1/12 1/36 2/9"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The posterior probability of the 6-sided die is 4/9, which is a little more than the probabilities for the other dice, 3/9 and 2/9.\n",
"Intuitively, the 6-sided die is the most likely because it had the highest likelihood of producing the outcome we saw."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Monty Hall Problem\n",
"\n",
"Next we'll use a Bayes table to solve one of the most contentious problems in probability.\n",
"\n",
"The Monty Hall problem is based on a game show called *Let's Make a Deal*. If you are a contestant on the show, here's how the game works:\n",
"\n",
"* The host, Monty Hall, shows you three closed doors -- numbered 1, 2, and 3 -- and tells you that there is a prize behind each door.\n",
"\n",
"* One prize is valuable (traditionally a car), the other two are less valuable (traditionally goats).\n",
"\n",
"* The object of the game is to guess which door has the car. If you guess right, you get to keep the car.\n",
"\n",
"Suppose you pick Door 1. Before opening the door you chose, Monty opens Door 3 and reveals a goat. Then Monty offers you the option to stick with your original choice or switch to the remaining unopened door."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To maximize your chance of winning the car, should you stick with Door 1 or switch to Door 2?\n",
"\n",
"To answer this question, we have to make some assumptions about the behavior of the host:\n",
"\n",
"1. Monty always opens a door and offers you the option to switch.\n",
"\n",
"2. He never opens the door you picked or the door with the car.\n",
"\n",
"3. If you choose the door with the car, he chooses one of the other\n",
" doors at random.\n",
"\n",
"Under these assumptions, you are better off switching. \n",
"If you stick, you win $1/3$ of the time. If you switch, you win $2/3$ of the time.\n",
"\n",
"If you have not encountered this problem before, you might find that\n",
"answer surprising. You would not be alone; many people have the strong\n",
"intuition that it doesn't matter if you stick or switch. There are two\n",
"doors left, they reason, so the chance that the car is behind Door A is 50%. But that is wrong.\n",
"\n",
"To see why, it can help to use a Bayes table. We start with three\n",
"hypotheses: the car might be behind Door 1, 2, or 3. According to the\n",
"statement of the problem, the prior probability for each door is 1/3."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.923035Z",
"iopub.status.busy": "2021-04-16T19:35:10.922476Z",
"iopub.status.idle": "2021-04-16T19:35:10.925144Z",
"shell.execute_reply": "2021-04-16T19:35:10.925496Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

6 | 1/3 | 1/6 | 1/18 | 4/9 |

8 | 1/3 | 1/8 | 1/24 | 1/3 |

12 | 1/3 | 1/12 | 1/36 | 2/9 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior\n",
"Door 1 1/3\n",
"Door 2 1/3\n",
"Door 3 1/3"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table3 = pd.DataFrame(index=['Door 1', 'Door 2', 'Door 3'])\n",
"table3['prior'] = Fraction(1, 3)\n",
"table3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data is that Monty opened Door 3 and revealed a goat. So let's\n",
"consider the probability of the data under each hypothesis:\n",
"\n",
"* If the car is behind Door 1, Monty chooses Door 2 or 3 at random, so the probability he opens Door 3 is $1/2$.\n",
"\n",
"* If the car is behind Door 2, Monty has to open Door 3, so the probability of the data under this hypothesis is 1.\n",
"\n",
"* If the car is behind Door 3, Monty does not open it, so the probability of the data under this hypothesis is 0.\n",
"\n",
"Here are the likelihoods. "
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.931993Z",
"iopub.status.busy": "2021-04-16T19:35:10.931536Z",
"iopub.status.idle": "2021-04-16T19:35:10.933829Z",
"shell.execute_reply": "2021-04-16T19:35:10.934196Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | |
---|---|

Door 1 | 1/3 |

Door 2 | 1/3 |

Door 3 | 1/3 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood\n",
"Door 1 1/3 1/2\n",
"Door 2 1/3 1\n",
"Door 3 1/3 0"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table3['likelihood'] = Fraction(1, 2), 1, 0\n",
"table3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have priors and likelihoods, we can use `update` to compute the posterior probabilities."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.944379Z",
"iopub.status.busy": "2021-04-16T19:35:10.943730Z",
"iopub.status.idle": "2021-04-16T19:35:10.946709Z",
"shell.execute_reply": "2021-04-16T19:35:10.947151Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | |
---|---|---|

Door 1 | 1/3 | 1/2 |

Door 2 | 1/3 | 1 |

Door 3 | 1/3 | 0 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"Door 1 1/3 1/2 1/6 1/3\n",
"Door 2 1/3 1 1/3 2/3\n",
"Door 3 1/3 0 0 0"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"update(table3)\n",
"table3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After Monty opens Door 3, the posterior probability of Door 1 is $1/3$;\n",
"the posterior probability of Door 2 is $2/3$.\n",
"So you are better off switching from Door 1 to Door 2."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As this example shows, our intuition for probability is not always\n",
"reliable. \n",
"Bayes's Theorem can help by providing a divide-and-conquer strategy:\n",
"\n",
"1. First, write down the hypotheses and the data.\n",
"\n",
"2. Next, figure out the prior probabilities.\n",
"\n",
"3. Finally, compute the likelihood of the data under each hypothesis.\n",
"\n",
"The Bayes table does the rest."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"In this chapter we solved the Cookie Problem using Bayes's theorem explicitly and using a Bayes table.\n",
"There's no real difference between these methods, but the Bayes table can make it easier to compute the total probability of the data, especially for problems with more than two hypotheses.\n",
"\n",
"Then we solved the Dice Problem, which we will see again in the next chapter, and the Monty Hall problem, which you might hope you never see again.\n",
"\n",
"If the Monty Hall problem makes your head hurt, you are not alone. But I think it demonstrates the power of Bayes's Theorem as a divide-and-conquer strategy for solving tricky problems. And I hope it provides some insight into *why* the answer is what it is.\n",
"\n",
"When Monty opens a door, he provides information we can use to update our belief about the location of the car. Part of the information is obvious. If he opens Door 3, we know the car is not behind Door 3. But part of the information is more subtle. Opening Door 3 is more likely if the car is behind Door 2, and less likely if it is behind Door 1. So the data is evidence in favor of Door 2. We will come back to this notion of evidence in future chapters.\n",
"\n",
"In the next chapter we'll extend the Cookie Problem and the Dice Problem, and take the next step from basic probability to Bayesian statistics.\n",
"\n",
"But first, you might want to work on the exercises."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exercises"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:** Suppose you have two coins in a box.\n",
"One is a normal coin with heads on one side and tails on the other, and one is a trick coin with heads on both sides. You choose a coin at random and see that one of the sides is heads.\n",
"What is the probability that you chose the trick coin?"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.958218Z",
"iopub.status.busy": "2021-04-16T19:35:10.957565Z",
"iopub.status.idle": "2021-04-16T19:35:10.960351Z",
"shell.execute_reply": "2021-04-16T19:35:10.959987Z"
},
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

Door 1 | 1/3 | 1/2 | 1/6 | 1/3 |

Door 2 | 1/3 | 1 | 1/3 | 2/3 |

Door 3 | 1/3 | 0 | 0 | 0 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"Normal 0.5 0.5 0.25 0.333333\n",
"Trick 0.5 1.0 0.50 0.666667"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Solution\n",
"\n",
"table4 = pd.DataFrame(index=['Normal', 'Trick'])\n",
"table4['prior'] = 1/2\n",
"table4['likelihood'] = 1/2, 1\n",
"\n",
"update(table4)\n",
"table4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:** Suppose you meet someone and learn that they have two children.\n",
"You ask if either child is a girl and they say yes.\n",
"What is the probability that both children are girls?\n",
"\n",
"Hint: Start with four equally likely hypotheses."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.971798Z",
"iopub.status.busy": "2021-04-16T19:35:10.971166Z",
"iopub.status.idle": "2021-04-16T19:35:10.974088Z",
"shell.execute_reply": "2021-04-16T19:35:10.974691Z"
},
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

Normal | 0.5 | 0.5 | 0.25 | 0.333333 |

Trick | 0.5 | 1.0 | 0.50 | 0.666667 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"GG 0.25 1 0.25 0.333333\n",
"GB 0.25 1 0.25 0.333333\n",
"BG 0.25 1 0.25 0.333333\n",
"BB 0.25 0 0.00 0.000000"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Solution\n",
"\n",
"table5 = pd.DataFrame(index=['GG', 'GB', 'BG', 'BB'])\n",
"table5['prior'] = 1/4\n",
"table5['likelihood'] = 1, 1, 1, 0\n",
"\n",
"update(table5)\n",
"table5"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:** There are many variations of the [Monty Hall problem](https://en.wikipedia.org/wiki/Monty_Hall_problem). \n",
"For example, suppose Monty always chooses Door 2 if he can, and\n",
"only chooses Door 3 if he has to (because the car is behind Door 2).\n",
"\n",
"If you choose Door 1 and Monty opens Door 2, what is the probability the car is behind Door 3?\n",
"\n",
"If you choose Door 1 and Monty opens Door 3, what is the probability the car is behind Door 2?"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.986065Z",
"iopub.status.busy": "2021-04-16T19:35:10.985478Z",
"iopub.status.idle": "2021-04-16T19:35:10.988216Z",
"shell.execute_reply": "2021-04-16T19:35:10.988593Z"
},
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

GG | 0.25 | 1 | 0.25 | 0.333333 |

GB | 0.25 | 1 | 0.25 | 0.333333 |

BG | 0.25 | 1 | 0.25 | 0.333333 |

BB | 0.25 | 0 | 0.00 | 0.000000 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"Door 1 0.333333 1 0.333333 0.5\n",
"Door 2 0.333333 0 0.000000 0.0\n",
"Door 3 0.333333 1 0.333333 0.5"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Solution\n",
"\n",
"# If the car is behind Door 1, Monty would always open Door 2 \n",
"# If the car was behind Door 2, Monty would have opened Door 3\n",
"# If the car is behind Door 3, Monty would always open Door 2\n",
"\n",
"table6 = pd.DataFrame(index=['Door 1', 'Door 2', 'Door 3'])\n",
"table6['prior'] = 1/3\n",
"table6['likelihood'] = 1, 0, 1\n",
"\n",
"update(table6)\n",
"table6"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:10.998516Z",
"iopub.status.busy": "2021-04-16T19:35:10.998015Z",
"iopub.status.idle": "2021-04-16T19:35:11.000348Z",
"shell.execute_reply": "2021-04-16T19:35:11.000696Z"
},
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

Door 1 | 0.333333 | 1 | 0.333333 | 0.5 |

Door 2 | 0.333333 | 0 | 0.000000 | 0.0 |

Door 3 | 0.333333 | 1 | 0.333333 | 0.5 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"Door 1 0.333333 0 0.000000 0.0\n",
"Door 2 0.333333 1 0.333333 1.0\n",
"Door 3 0.333333 0 0.000000 0.0"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Solution\n",
"\n",
"# If the car is behind Door 1, Monty would have opened Door 2\n",
"# If the car is behind Door 2, Monty would always open Door 3\n",
"# If the car is behind Door 3, Monty would have opened Door 3\n",
"\n",
"table7 = pd.DataFrame(index=['Door 1', 'Door 2', 'Door 3'])\n",
"table7['prior'] = 1/3\n",
"table7['likelihood'] = 0, 1, 0\n",
"\n",
"update(table7)\n",
"table7"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise:** M&M's are small candy-coated chocolates that come in a variety of colors. \n",
"Mars, Inc., which makes M&M's, changes the mixture of colors from time to time.\n",
"In 1995, they introduced blue M&M's. \n",
"\n",
"* In 1994, the color mix in a bag of plain M&M's was 30\\% Brown, 20\\% Yellow, 20\\% Red, 10\\% Green, 10\\% Orange, 10\\% Tan. \n",
"\n",
"* In 1996, it was 24\\% Blue , 20\\% Green, 16\\% Orange, 14\\% Yellow, 13\\% Red, 13\\% Brown.\n",
"\n",
"Suppose a friend of mine has two bags of M&M's, and he tells me\n",
"that one is from 1994 and one from 1996. He won't tell me which is\n",
"which, but he gives me one M&M from each bag. One is yellow and\n",
"one is green. What is the probability that the yellow one came\n",
"from the 1994 bag?\n",
"\n",
"Hint: The trick to this question is to define the hypotheses and the data carefully."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"execution": {
"iopub.execute_input": "2021-04-16T19:35:11.011048Z",
"iopub.status.busy": "2021-04-16T19:35:11.010527Z",
"iopub.status.idle": "2021-04-16T19:35:11.013385Z",
"shell.execute_reply": "2021-04-16T19:35:11.012984Z"
},
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/html": [
"prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

Door 1 | 0.333333 | 0 | 0.000000 | 0.0 |

Door 2 | 0.333333 | 1 | 0.333333 | 1.0 |

Door 3 | 0.333333 | 0 | 0.000000 | 0.0 |

\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"

\n",
"

"
],
"text/plain": [
" prior likelihood unnorm posterior\n",
"A 0.5 0.040 0.020 0.740741\n",
"B 0.5 0.014 0.007 0.259259"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Solution\n",
"\n",
"# Hypotheses:\n",
"# A: yellow from 94, green from 96\n",
"# B: yellow from 96, green from 94\n",
"\n",
"table8 = pd.DataFrame(index=['A', 'B'])\n",
"table8['prior'] = 1/2\n",
"table8['likelihood'] = 0.2*0.2, 0.14*0.1\n",
"\n",
"update(table8)\n",
"table8"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"celltoolbar": "Tags",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
prior | likelihood | unnorm | posterior | |
---|---|---|---|---|

A | 0.5 | 0.040 | 0.020 | 0.740741 |

B | 0.5 | 0.014 | 0.007 | 0.259259 |