The following exercises are to familiarize you with the notation and manual calculations.

Exercise 1 (LN p. 10)

See the lecture notes for the exercise statement.

Exercise 2 — Normal model with discrete prior

Consider the following model:

\begin{aligned} \mu &\sim \pi, \\ x \mid \mu &\sim \mathcal{N}(\mu,1), \end{aligned}

where \pi(-2)=0.4 and \pi(2)=0.6.

Calculate

Solution (prior predictive density)

Likelihood: f(x\mid\mu) = \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2}\right)

Prior predictive density: \begin{aligned} f(x) &= \int f(x\mid\mu) \pi(\mu) d\mu \\ &= \sum_{\mu \in \{-2, 2\}} f(x\mid\mu) \pi(\mu) \\ &= 0.4 \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x+2)^2}{2}\right) + 0.6 \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x-2)^2}{2}\right). \end{aligned}

View the Desmos graph.

Solution (posterior distribution)

Likelihood:

f(x_1,...,x_n\mid\mu) = \prod_{i=1}^n f(x_i\mid\mu) = (2\pi)^{-n/2} \exp\left(-\frac{1}{2}\sum_{i=1}^n (x_i-\mu)^2\right).

Posterior distribution:

\begin{aligned} \pi(\mu \mid x_1,...,x_n) &= \frac{f(x_1,...,x_n \mid\mu) \pi(\mu)}{f(x_1,...,x_n)} \\ &= \frac{f(x_1,...,x_n \mid\mu) \pi(\mu)}{\sum_{\mu' \in \{-2, 2\}} f(x_1,...,x_n \mid\mu') \pi(\mu')} \\ &= \frac{f(x_1,...,x_n \mid\mu) \pi(\mu)}{0.4 \cdot f(x_1,...,x_n \mid -2) + 0.6 \cdot f(x_1,...,x_n \mid 2)}. \end{aligned}

Solution (posterior predictive density)

Posterior predictive density: \begin{aligned} f(x_{n} \mid x_1,...,x_{n-1}) &= \int f(x_{n} \mid \mu) \pi(\mu \mid x_1,...,x_{n-1}) d\mu \\ &= \sum_{\mu \in \{-2, 2\}} f(x_{n} \mid \mu) \pi(\mu \mid x_1,...,x_{n-1}) \\ &= \pi(-2 \mid x_1,...,x_{n-1}) \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x_{n}+2)^2}{2}\right) + \pi(2 \mid x_1,...,x_{n-1}) \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{(x_{n}-2)^2}{2}\right). \end{aligned}