Statistical Theory II
Stat
461-561
Course schedule
Three lectures per week (Monday, Wednesday and Friday from 11.00 to
12.00)
in LSK 301.
Office Hours: LSK 308c Monday 13.00-14.00 & Friday 13.00-14.00.
Objectives
To provide students an introduction to modern statistical inference.
Although
we will use sometimes the recommended textbook,
I wont follow it closely and will attempt to discuss 'modern'
topics. There will be a significant emphasis on computational
methods.
Announcements
- 561 Students: The final
project is due Thursday 24th April (no extension will be granted)
Quiz
Other notes
Handouts
- Lectures on Maximum Likelihood
Estimation, Method of Moments and
Pseudo-Likelihood Estimation Notes
Recommended reading for Weeks 1-2: section 6.4, section 7.1.1 to 7.2.2,
section 10.1.1 to 10.1.3
Advanced reading (for graduate students): D. Cox & N. Reid, A
note on pseudolikelihood constructed from marginal densities,
Biometrika, 2004 Pdf file
Varin, C. and Vidoni, P., Pairwise Likelihood Inference
for General State Space Models, Econometrics
Review, 2007 Pdf file
Suggested exercises for Week 1-2: 5.12, 5.13, 5.30, 7.1,
7.2, 7.9, 7.11, 7.19, 10.1, 10.3, 10.8
Solutions exercises pdf
- Lectures on the
Expectation-Maximization algorithm Notes1 Notes 2
Recommended reading for Weeks 3-4: section 7.2.4 and a nice tutorial
paper by J. Bilmes A gentle introduction to the
EM algorithm and its
applications to parameter estimation for Gaussian mixtures and Hidden
Markov models Pdf
file
Advanced reading (for graduate students): K. Lange, D.R. Hunter
& I.
Yang, Optimization Transfer Using Surrogate Objective Functions, JCGS,
2000 Pdf
file
Suggested exercises for Week 2-3: 7.20, 7.21, 7.27, 7.29, 7.30, 7.40,
7.46, 7.61
Solutions
exercises pdf
- Lectures on the Information
Criterion Notes
Notes strongly inspired from the recent book by Konishi & Kitagawa,
Information Criteria & Statistical Modelling, Springer-Verlag, 2007
Recommended reading for Weeks 6-7: chapter 8 and chapter 10, section
10.3
Advanced reading (for graduate students): Controlling
the False Discovery Rate: A Practical and Powerful Approach to Multiple
Testing. J. Roy Stat Soc, Ser B. 57:289-300.
Suggested exercises for Week 6-7: 8.3, 8.7a, 8.13a,b,c, 8.15 and
8.17
Solution exercises pdf
- Lectures on M-estimation
pdf
Recommended reading for Week 7: section 10.2 in textbook and the
following paper
Stefanski & Boos, The calculus of
M-estimation, The American
Statistician,. here
- Lectures on Bayesian Statistics
pdf
The C&B has a very short section on Bayesian statistics: read
chapter 7.
You should read the nice handouts 1 to 8 by Brani Vidakovic html
Suggested exercises for Week 7-8 pdf
Solution
exercises pdf pdf
Suggested
exercises pdf
Solution
exercises pdf
Tentative Topics
- Data reduction, likelihood
principle, sufficiency and equivariant principles.
- Point estimation: Maximum
Likelihood and Pseudo-Likelihood, Moments methods.
- The Expectation-Maximization
algorithms and applications.
- Markov chain Monte
Carlo and Importance Sampling Methods.
- Hypothesis Testing, Monte Carlo
tests.
- Nonparametric Statistics:
density estimation, wavelets etc.
Textbook
- G. Casella and R. L. Berger (2002). Statistical Inference, 2nd
Edition, Duxbury.
I will not follow very closely the textbook but this is a standard
reference that should prove useful.
We will also use
- R.V. Hogg et al., Introduction to Mathematical Statistics, 6th
edition, Prentice Hall.
- Andrew Gelman, John B. Carlin, Hal Stern and Donald B. Rubin,
Bayesian Data Analysis, Chapman&Hall/CRC, 2nd edition.
- Christian P. Robert, The Bayesian Choice, Springer, 2nd edition.
- Jun Shao, Mathematical Statistics, 2nd edition, Springer.
Grading
We will have 4 in-class exams instead of
homeworks. Of these 4 grades, the average of the 3 best
will be counted for 40% of your final grade.
461 students: The final exam will be
worth 60% of your grade. You have to obtain at least 40% on
the final exam to pass the course. There will be NO MAKE-UP exams.
561 students: The final project will be
worth 60% of your grade.