This is a course on advanced statistical inference for 3rd year students studying Mathematics and Statistics at Oxford.
Understanding how data can be interpreted in the context of a statistical model. Working knowledge and understanding of key-elements of model-based statistical inference, including awareness of similarities, relationships and differences between Bayesian and frequentist approaches.
Exponential families: Curved and linear exponential families; canonical parametrization; likelihood equations. Sufficiency: Factorization theorem; sufficiency in exponential families.
Frequentist estimation: unbiasedness; method of moments; the Cramer-Rao information inequality; Rao-Blackwell theorem, Lehmann-Scheffe Theorem and Rao-Blackwellization.
Statement of complete sufficiency for Exponential families.
The Bayesian paradigm: likelihood principal; subjective probability; prior to posterior analysis; asymptotic normality; conjugacy; examples from exponential families. Choice of prior distribution: proper and improper priors; Jeffreys and maximum entropy priors. Hierarchical Bayes models.
Computational techniques: Markov chain Monte Carlo methods; The Metropolis-
Hastings algorithm. Gibbs Sampling. Variational Bayesian methods. The EM
algorithm. Approximations to marginal likelihood : Laplace approximation and
Decision theory: risk function; Minimax rules, Bayes rules. Point estimators and admissability of Bayes rules. The James-Stein estimator, shrinkage estimators and Empirical Bayes. Hypothesis testing as decision problem.
PDF of all 16 lectures in 4up format : bs2a_4up.pdf
- P. H. Garthwaite, I. T. Jolliffe and Byron Jones, Statistical Inference, Second ed. Oxford University Press, 2002
- G.A.Young and R.L. Smith, Essentials of Statistical Inference, Cambridge University Press, 2005.
- T. Leonard and J.S.J. Hsu, Bayesian Methods, Cambridge University Press, 2005.
- D. R. Cox, Principles of Statistical Inference, Cambridge University Press, 2006
- H. Liero and S Zwanzig, Introduction to the Theory of Statistical Inference, CRC Press, 2012
- D. Barber, Bayes Reasoning and Machine Learning, Cambridge University Press,