Markov Chains (Michaelmas 2019)

Time and Location: Tu-Th, 10-11am; Mill Lane Lecture Room 3

Markov Chains is an introductory course on Markov Chains, in Part IB of the Cambridge Tripos.


  • Definition and basic properties, the transition matrix. Calculation of n-step transition probabilities. Communicating classes, closed classes, absorption, irreducibility. Calculation of hitting probabilities and mean hitting times; survival probability for birth and death chains. Stopping times and statement of the strong Markov property.
  • Recurrence and transience; equivalence of transience and summability of n-step transition probabilities; equivalence of recurrence and certainty of return. Recurrence as a class property, relation with closed classes. Simple random walks in dimensions one, two and three.
  • Invariant distributions, statement of existence and uniqueness. Mean return time, positive recurrence; equivalence of positive recurrence and the existence of an invariant distribution. Convergence to equilibrium for irreducible, positive recurrent, aperiodic chains and proof by coupling. *Long-run proportion of time spent in given state*.
  • Time reversal, detailed balance, reversibility; random walk on a graph.

Lecture Notes

Example Sheets

The example sheets will be posted here during the course of the term.