Math 502: Statistical Inference

• 01/28: Notes, Example (2), (a)–(e); Textbook Exercises: 6.1, 6.3.
• 01/30: Notes, Example (4), (b) and (c). Use both methods and for each method try to use different representations (so that your answers are not unique). Textbook Exercises: 6.2, 6.5, 6.8, 6.9.

• 02/04 & 02/06: Textbook Exercises: 6.10, 6.11, 6.13, 6.14, 6.15, 6.18, 6.19, 6.20, 6.23, 6.25.
• 02/09:
  • Let $X_1, \dots, X_n$ be i.i.d. with density function defined as $f(x|\theta)=e^{-\lambda(x-\mu)}, x > \mu, \lambda > 0, \mu \in \mathbb{R}$. Prove that $(X_{(1)}, W)$ is the sufficient statistic of $\theta=(\mu,\lambda)$, where $W=\sum_{i=2}^n(X_{(i)}-X_{(1)})$.
  • Textbook Exercises: 7.1, 7.2, 7.6.

• 02/11: Textbook Exercises: 7.7, 7.8, 7.10, 7.11.
• 02/13:
  • Notes: Show that a Bayes estimator depends on the data through a sufficient statistic.
  • Notes: If $X_i$'s are i.i.d. given $\theta$, are they i.i.d. marginally? Why?
  • Textbook Exercises: 7.14, 7.22, 7.23 (read "conjugate prior" as "prior"), 7.24, 7.25.
• 02/16: Textbook Exercises: 7.9, 7.12, 7.50.

• 02/18: Textbook Exercises: 7.19, 7.20, 7.21.
• 02/20: Textbook Exercises: 7.37, 7.46, 7.49, 7.51, 7.52.
• 02/23: Textbook Exercises: 7.53, 7.57, 7.58.

• 02/27: Textbook Exercises: 7.59, 7.44, 7.48, 7.60, 7.63.
• 03/02: Textbook Exercises: 7.40, 7.65, 7.66.

• 03/06: Textbook Exercises: 8.1, 8.2, 8.3, 8.5, 8.6.
• 03/09: Textbook Exercises: 8.7, 8.8, 8.9.

• 03/11: Textbook Exercises: 8.12, 8.13, 8.15, 8.17, 8.20.
• 03/13: Textbook Exercises: 8.19, 8.21.
• 03/16:
  • Textbook Exercises: 8.25, 8.27.
  • In class, it was shown that if we remove $k>0$ from the necessity condition of the NPL, then when $k=0$, we must have $\beta_\phi(\theta_1)=1$ for the UMP level $\alpha$ test $\phi$. Complete this by arguing why in this case $\phi$ still must satisfy equation (2) in the NPL, that is, why it must be the case that $\phi=1$ when $f(x|\theta_1)>0$ wp1. [Hint: $\phi \le 1$. Proof should not exceed 2 lines.]

• 03/18: Textbook Exercises: 8.28, 8.29, 8.30, 8.33.
• 03/20: Textbook Exercises: 8.37, 8.38, 8.47.
• 03/23: Textbook Exercises: 9.1, 9.2, 9.3.
• Simulation Project: Submit R code and report the results properly.
  1. Use R to generate 10 observations from $N(1,4)$.
  2. Pretend you only know that data were from $N(\mu,4)$ without knowing $\mu$ and construct an 80% confidence interval for $\mu$.
  3. Repeat Steps 1 and 2 100 times. Count the proportion among trials where C.I. contains the true mean. What is its relation to the confidence coefficient?
  4. Repeat Steps 1–3, but pretend you know neither mean $\mu$ nor variance $\sigma^2$. Compare C.I. lengths between settings and comment.

• 03/25: Submit all code/output (preferably LaTeX). Default $1-\alpha = 0.95$. Textbook Exercises:
  • 9.4: Assume $n=10, m=15, \sigma_X^2=1, \sigma_Y^2=3$. Generate data and use numerical methods to provide a CI. Repeat 1000 times and report coverage count for true $\lambda=3$.
  • 9.6: Assume $X \sim \text{Bin}(n,p)$ observed with $n=50$ and $p=0.3$. Numerically provide CIs over 1000 trials and report coverage rate.
  • 9.12 & 9.13(b)
• 03/27: Textbook Exercises: 9.16, 9.17, 9.23. Find shortest CI using pivotal method for 9.17 and compare length with equal-tailed CI for $\alpha=0.05$.
• 03/30:
  • Textbook Exercise: 9.37
  • Let $X_1,\dots,X_n \sim \text{Cauchy}$ with $f(x)=[\pi (1+x^2)]^{-1}$. Calculate $\int_{-\infty}^\infty |x|f(x)dx$. Find $E[X_1]$. Can SLLN be applied? Simulate sample $\overline{X}_n$ for $n=100$ across 500 trials and plot sorted values.

• 03/27: Textbook Exercises: 10.1, 10.2
• Spring Break
• 04/17:
  • Let $W_n$ have mean $\mu$ and variance $C/n^\nu$ ($\nu>0$). Prove consistency with $\mu$.
  • Let $Y_n$ be $n$-th order statistic from $\text{Unif}(0,\theta)$. Prove $\sqrt{Y_n}$ is consistent with $\sqrt{\theta}$.
  • Find limiting distribution of $Z_n = n[1-F(Y_n)]$. Numerically verify for $F = \Phi$ (standard normal).
  • Provide a counterexample showing $X_n \Rightarrow X$ and $Y_n \Rightarrow Y \nRightarrow X_n+Y_n \Rightarrow X+Y$.
• 04/20:
  • Textbook Exercises: 10.4, 10.5, 10.6.
  • For $X \sim \text{Bin}(n,p)$, analyze $\hat{\tau}$ for $\tau(p)=1/(1-p)$ ($p \neq 1$).
  • For $X_1,\dots,X_n \sim \text{Unif}(0,\theta)$, find MLE, construct unbiased function of MLE, calculate variance, and compare to CRLB.

• 04/22: Textbook Exercises: 10.8, 10.19(a), 10.35.
• 04/24: Textbook Exercises: 10.31, 10.32, 10.33, 10.34, 10.36, 10.37.
• 04/27: In 10.36, derive two Wald statistics. For $n=25, \alpha=0.1, H_0: \beta=2$, compare power via 10,000 simulations when true $\beta=3$. Interpret results.

• 04/29:
  • Textbook Exercise: 10.38.
  • Find Fisher Information $I(\theta)$ for $X \sim \text{Pois}(\theta)$.
  • Derive large sample Z-test, Score test, and LRT for $H_0: \theta=2$ vs $H_a: \theta \neq 2$ with $X_i \sim \text{Pois}(\theta)$.
  • Simulate distribution of $-2\log(\lambda_n)$ using EDF and compare with $\chi^2(1)$ CDF for $n=5$ and $n=100$ (5000 simulations).
• 05/01:
  • Read Example 10.4.5; finish Exercises 10.40, 10.41, 10.47, 10.48.
  • For $\mathbf{X} \sim \text{Multinomial}(n, p_1,\dots,p_5)$, compare $H_0: p_1=p_2=p_5=0.01, p_3=0.5$ vs $H_1: H_0 \text{ false}$. Derive LRT for $n=1$ and $n=100$ at $\alpha=0.05$. Estimate Type II error probability $P(H_0|H_1)$ when $p_1=p_2=p_5, \mathbf{p_3=0.3}$.

R Reference Script (fig10.r)

myfun = function(n) {
  m = 1000
  x = rgamma(m, n, 1) / n  # m X's
  y = -2 * (n * log(x) + n * (1 - x))  # m \lambda's
  u = rchisq(m, 1)
  
  qqplot(y, u, main = paste("QQ plot, n =", n))
  lines(y, y)
  
  sy = sort(y)
  plot(sy, ppoints(sy), xlim = c(0.5, 2), ylim = c(0.4, 0.9), type = "l", lty = 1, main = paste("CDF, n =", n))
  lines(sy, pchisq(sy, 1), xlim = c(0.5, 2), ylim = c(0.4, 0.9), type = "l", lty = 2)
}

pdf("fig10.pdf", height = 9.0, width = 6.5)
par(mfrow = c(2, 2))
n = 1
myfun(n)
n = 100
myfun(n)
dev.off()