Statistical Computing - Monte Carlo Methods

      CPSC 535D

Course schedule

Two lectures per week (Tuesday and Thursday from 3.30 to 5.00) in  ICCS 238.
If you want to arrange a meeting, just send me an email  at the following address: arnaud at cs dot ubc dot ca

Announcements

Projects

Handouts

  Additional reading: Chapter 1 of the Bayesian Choice by C.P. Robert or the nice handouts h1 h2 by B. Vidakovic

Additional reading: Chapter 1 of the Bayesian Choice by C.P. Robert or h3 by B. Vidakovic Additional reading:
       - h6 by B. Vidakovic
       - R. Kass and A. Raftery, Bayes Factors, JASA, 1995 paper
       - R. Kass, Bayes Factors in Practice, The Statistician, 1992 here
       - M. Lavine and M.J. Schervish, Bayes Factors: What they are and what they are not, The American Statistician, 1999 here
      - Chapter 7 of the Bayesian Choice by C.P. Robert
      - J. Hoeting, D. Madigan, A. Raftery and C. Volinsky, Bayesian model averaging: A tutorial, Statistical Science, 1999 here
      - A. Raftery, D. Madigan and J. Hoeting,  Bayesian model averaging for linear regression models, JASA, 1997 here

Additional reading:
         - Section 3.1 and 3.2 of Monte Carlo Statistical Methods.

     Additional reading:
         - Chapter 2 of Monte Carlo Statistical Methods. 
         - Scale mixture of Gaussians, JRSS B, 1974 here: very useful  representation of non-Gaussian distributions as infinite mixture of Gaussians
         - W. Gilks and P. Wild, Adaptive rejection sampling for Gibbs sampling, Applied Statistics, 1992 here
         - B.D. Flury, Rejection sampling made easy, SIAM Review, 1990 here
      More advanced
          - A. Peterson and R. Kronmal, On mixture methods for the computer  generation of random variables, The American Statistician, 1982 here
          - J. Halton, Reject the rejection technique, J. Scientific Computing, 1992. (by the way please don't reject it)
         - A. Beskos and G. Roberts, Exact simulation of diffusions, Annals of Applied Proba, 2005. here
          Check Proposition 1 and its proof for a very clever and useful remark about rejection sampling.

Additional reading:

          - Chapter 3 of Monte Carlo Statistical Methods.

          -  Y. Chen, Another look at rejection sampling through importance sampling, Stat. Proba. Lett., 2005   here

          -   J. Geweke, Bayesian inference in econometric models using Monte Carlo integration, Econometrica, 1989  here
          -  H. Van Dijk, J. Hop, A. Louter, An Algorithm for the Computation of Posterior Moments and Densities Using Simple Importance Sampling, The Statistician, 1987 here
      Optional reading
          -   A. Owen and Y. Zhou, Safe and effective importance sampling, JASA, 2000   here

          - Chapter 11 of Robert & Casella
          - A.F.M. Smith, A.E. Gelfand, Bayesian Statistics without Tears: A Sampling Importance Resampling Perspective, The American Statistician, 1992 Pdf file here
          - A. Doucet, N. De Freitas and N.J. Gordon, An introduction to Sequential Monte Carlo, SMC in Practice, 2001 Ps file here
           - A. Kong, J.S. Liu and W.H. Wong, Sequential Imputations and Bayesian Missing Data Problems, JASA, 1994 Pdf file here
           - J.S. Liu and R. Chen, Sequential Monte Carlo methods for dynamic systems, JASA, 1998 Pdf file here


        Matlab code to generate fractal image code

  Additional reading:
          
            - D. Mackay, Introduction to Monte Carlo methods, here  
            - R. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods, Technical report, 1993 here
            
            - C. Andrieu, A. Doucet, N. De Freitas and M. Jordan, Markov chain Monte Carlo for Machine Learning, Machine Learning, 2003 here
            -  S. Brooks, Markov chain Monte Carlo Methods and Its Application, The Statistician, 1998 here
            - G. Casella and E.I. George, Explaining the Gibbs sampler. The American Statistician, 1992 here
            -  S. Chib and E. Greenberg, Understanding the Metropolis-Hastings algorithm, The American Statistician, 1995 here

             - Chapter 11 of Robert & Casella
             - P.J. Green, Transdimensional Markov chain Monte Carlo, Highly Structured Stochastic Systems, OUP, 2003 Pdf file here
             - S. Sisson, Trans-dimensional Markov chains: A decade of progress and future perspectives., JASA, 2005 Pdf file here
                - Gelman and Meng, Simulating normalizing constants: from importance sampling to bridge sampling to path sampling, Statistical Science, 1998. here
                 - Neal, Annealed importance sampling, Stat. Computing, 2001. here

             - Chapter 8 of Robert & Casella
             - C. Andrieu, L. Breyer & A. Doucet, Convergence of Simulated Annealing using Foster-Lyapunov Criteria, Journal Applied Probability, 2001. Pdf file here
             - Paul Damien, Jon Wakefield, Stephen Walker, Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables, JRSS B, 1999 Pdf file here
             - R. Neal, Sampling from Multimodal Distributions using Tempered Transitions, Statistics and Computing, 1996 Pdf file here
             - C. Geyer & E. Thompson, Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference, JASA, 1995 Pdf file here

            - M.I. Jordan Tutorial on Nonparametric Bayes, NIPS 2005 Ps file here
            - R. Neal MCMC for Dirichlet Process Mixture Models, JCGS, 2000. Pdf file here

Assignements

        Volatility data

       Probabilistic Nearest Neighbour paper here

Course contents

Textbook

We will also use
Grading

 This will be based on several assignments and a final project (exact weighting yet to be decided). The computational part of the
assignments will be done using the R statistical language or Matlab. If you don't know what these are, I urge you to familiarize yourself with them.
Note that R is open source and can be downloaded for free.

Some interesting links - other Bayesian computational courses