{ "cells": [ { "cell_type": "markdown", "id": "da3d4e67", "metadata": {}, "source": [ "Consider the eight schools model (https://www.jstor.org/stable/1164617) illustrating the effects of coaching on SAT scores.\n", "\n", "At each school the students were randomly assigned to either a coaching program or a control group. The possible SAT scores are in range 200-800, and the treatment effect is the difference in mean SAT scores between the coached and control groups.\n", "\n", "| School | Estimated treatment effect | Standard error of effect estimate |\n", "|--------|----------------------------|-----------------------------------|\n", "| A | 28 | 15 |\n", "| B | 8 | 10 |\n", "| C | -3 | 16 |\n", "| D | 7 | 11 |\n", "| E | -1 | 9 |\n", "| F | 1 | 11 |\n", "| G | 18 | 10 |\n", "| H | 12 | 18 |" ] }, { "cell_type": "markdown", "id": "86996843", "metadata": {}, "source": [ "Let's consider three models:\n", "1. **Model 1: No pooling** - Each school is analyzed independently, and the treatment effect is estimated separately for each school without sharing information across schools.\n", "2. **Model 2: Complete pooling** - All schools are combined into a single dataset, and a single treatment effect is estimated for all schools, ignoring the differences between them.\n", "3. **Model 3: Partial pooling (Hierarchical model)** - A hierarchical model is used to estimate the treatment effects for each school while allowing for some sharing of information across schools. This model assumes that the treatment effects for the schools are drawn from a common distribution, which allows for borrowing strength across schools while still accounting for individual differences." ] }, { "cell_type": "code", "execution_count": 1, "id": "45555c16", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pymc as pm\n", "import arviz as az\n", "import matplotlib.pyplot as plt\n", "\n", "y = np.array([28, 8, -3, 7, -1, 1, 18, 12])\n", "sigma = np.array([15, 10, 16, 11, 9, 11, 10, 18])\n", "N = len(y)" ] }, { "cell_type": "markdown", "id": "e1d20737", "metadata": {}, "source": [ "Pooled model" ] }, { "cell_type": "code", "execution_count": null, "id": "42e1f88a", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "a6137fc6", "metadata": {}, "source": [ "Hierarchical model:" ] }, { "cell_type": "code", "execution_count": null, "id": "68610e42", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "022961d5", "metadata": {}, "source": [ "What happens, when hyper-prior for variance goes to zero? What happens, when it goes to infinity?" ] }, { "cell_type": "markdown", "id": "0da82ada", "metadata": {}, "source": [ "Let's do model comparison." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.8" } }, "nbformat": 4, "nbformat_minor": 5 }