Seminar in Bayesian Statistical Methodology

Welcome to AMS280D for Winter 2014!!

 

 

Some useful information:

  • The coordinator of the seminar is Tatiana Xifara (e-mail: xifara@ams.ucsc.edu)
  • Meetings will be held every Friday at 1:30-3:00pm at BE358.
  • The topic for this quarter is "Advanced MCMC methods".

 


Meeting 1 (1/17/2014): At the first meeting, Brenda presented the paper of Robert and Gasella (2010) (a shorter version appears in the handbook of MCMC (2011)) on the history of MCMC, which served as a great introduction to the topic and a very useful list of references. Brenda's slides can be found here.

Comments: The paper mentions that Hastings (1970) warns against high rejection rates as indicative of a poor choice of transition matrix, but does not mention the opposite pitfall of low rejection rates, associated with a slow exploration of the target. This brings up the problem of the choice of the proposal distribution manually, through trail and error, which can be very time consuming epecially in high dimensions. An alternative approach is adaptive MCMC, which automatically "learns" better parameters values while the algorithm runs. 

 

Meeting 2 (1/24/2014): At this seminar, Analissa gave a great presentation on the adaptive Metropolis algorithm given in Haario (2001) and simulation examples from the algorithm. Her slides can be found here. She also coded the adaptive Metropolis algorithm for a multivariate normal, a higher dimensional multivariate normal and a bivariate banana shape.

Comments: Remember that adaptive algorithm uses the initial covariance matix C0 for the first, say, n0 iterations. But if this C0 is a bad choice and almost all proposals will be rejected, you would not have a sensible Cn for the adaptive part after the n>n0 iteration. So you need to ensure to use C0 for the first n0 accepted proposals and not iterations.

 

Meeting 3 (1/31/2014): At this seminar we saw the first part of more sophisticated examples of adaptive MCMC algorithms. Chelsea presented the "Examples of Adaptive MCMC" by Roberts and Rosenthal. Chelsea's slides can be found here. See also for the related software to the paper taken from Jeff Rosenthal's website.

Comments: The discussion on the adaptive MCMC will be completed with one more paper, as these two papers are to some extent complementary and also give different perspectives. 

 

Meeting 4 (2/7/2014): Maria presented a tutorial on Adaptive MCMC by Andrieu and Thoms (2008). With this presentation we completed our series on Adaptive MCMC algorithms. Maria's slides can me found here.

Comments: We will continue our discussion with the Reversible Jump MCMC algorithm proposed by Green in 1995. We will start with a tutorial rather than the paper that introduced RJMCMC.  

 

Meeting 5 (2/14/2014): Celeste presented a paper by Waagepetersen and Sorensen (2001), in which they give a tutorial on Reversible Jump MCMC with an application for QTL-mapping. Celeste's slides can be found here.

Comments: At the next meeting we will see simulation examples from Green (1995) and Richardson and Green (1997)

 

Meeting 6 (2/21/2014): This meeting focused on the implementation of Reversible Jump MCMC. Nick presented Richardson and Green (1997) along with examples and Sai presented an application from Green (1995). Here are Nick's slides and r code and also Sai's slides and r code

Comments: With these two presentations we completed the discussion on RJMCMC and we will now continue with a single series on SMC since the students are already familiar with these algorithms. 

 

Meeting 7 (2/28/2014): Pedro will present a turorial on SMC by Doucet and Johansen that is published in the Handbook of Nonlinear Filtering in 2009. His slides can be found here and his code here.

Comments: At the next meeting we will talk about HMC (Hamiltonian Monte Carlo). In short, HMC introduces auxiliary momentum variables with independent Gaussian proposals. Momentum variables receive alernate updates, from simple updates to Metropolis updates. Metropolis updates result in the proposal of a new state by computing a trajectory according to Hamiltonian dynamics, from physics. Hamiltonian dynamics is discretized with the leapfrog method. In the way, distant jumps can be proposes and random-walk behavior is avoided.  

 

Meeting 8 (3/7/2014): Robert will present a paper on HMC by Neal (2011) that is included in the Handbook of Markov Chain Monte Carlo. 

  

 

Instructors and Assistants