This is a course on advanced statistical inference for 3rd year students studying Mathematics and Statistics at Oxford.

**Course aims **

Understanding how data can be interpreted in the context of a statistical model. Working knowledge and understanding of key-elements of model-based statistical inference, including awareness of similarities, relationships and differences between Bayesian and frequentist approaches.

### Synopsis

Exponential families: Curved and linear exponential families; canonical parametrization; likelihood equations. Sufficiency: Factorization theorem; sufficiency in exponential families.

Frequentist estimation: unbiasedness; method of moments; the Cramer-Rao information inequality; Rao-Blackwell theorem, Lehmann-Scheffe Theorem and Rao-Blackwellization.

Statement of complete sufficiency for Exponential families.

The Bayesian paradigm: likelihood principal; subjective probability; prior to posterior analysis; asymptotic normality; conjugacy; examples from exponential families. Choice of prior distribution: proper and improper priors; Jeffreys and maximum entropy priors. Hierarchical Bayes models.

Computational techniques: Markov chain Monte Carlo methods; The Metropolis-

Hastings algorithm. Gibbs Sampling. Variational Bayesian methods. The EM

algorithm. Approximations to marginal likelihood : Laplace approximation and

BIC.

Decision theory: risk function; Minimax rules, Bayes rules. Point estimators and admissability of Bayes rules. The James-Stein estimator, shrinkage estimators and Empirical Bayes. Hypothesis testing as decision problem.

### Lecture slides

PDF of all 16 lectures in 4up format : bs2a_4up.pdf

### Problem sheets

Class |
Sheet |

Week 3 |
ex1.pdf |

Week 4 |
ex2.pdf |

Week 5 |
ex3.pdf |

Week 6 |
ex4.pdf |

Week 7 |
ex5.pdf |

Week 8 |
ex6.pdf |

### Reading

- P. H. Garthwaite, I. T. Jolliffe and Byron Jones, Statistical Inference, Second ed. Oxford University Press, 2002
- G.A.Young and R.L. Smith, Essentials of Statistical Inference, Cambridge University Press, 2005.
- T. Leonard and J.S.J. Hsu, Bayesian Methods, Cambridge University Press, 2005.

### Further reading

- D. R. Cox, Principles of Statistical Inference, Cambridge University Press, 2006
- H. Liero and S Zwanzig, Introduction to the Theory of Statistical Inference, CRC Press, 2012
- D. Barber, Bayes Reasoning and Machine Learning, Cambridge University Press,

2012