# 2010/11 Undergraduate Module Catalogue

## MATH2750 Introduction to Markov Processes

### 10 creditsClass Size: 120

Module manager: Prof A. Veretennikov
Email: veretenn@maths.leeds.ac.uk

Taught: Semester 2 View Timetable

Year running 2010/11

### Pre-requisites

 MATH1715 Introduction to Probability

This module is approved as an Elective

### Module summary

A stochastic process refers to any quantity which changes randomly in time. The number of people in a queue, the capacity of a reservoir, the size of a population, are all examples from the real world. The linking model for all these examples is the simple random walk. The gambler's ruin problem is an example of a simple random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points. As a birth-death process this can be used to model population growth, the spread of diseases like AIDS, traffic flow, the queuing of students at a coffee bar, and so on.

### Objectives

To provide a simple introduction to stochastic processes.

On completion of this module, students should be able to:
(a) have an understanding of, and ability to solve, elementary problems of first passage time distributions
(b) understand about barriers in a random walk
(c) solve equilibrium distribution problems
(d) know the difference between an equilibrium distribution and a stationary distribution
(e) have a knowledge of Markov chains and elementary theory thereof
(f) learn about continuous time Markov process models
(g) have knowledge about the Poisson process
(h) extend the Poisson process model to other simple examples, and solve associated problems
(i) understand the role of forward and backward equations
(j) understand the use of simulation in modelling.

### Syllabus

1. Random walks: transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem.
2. Branching chain, probability of ultimate extinction.
3. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, irreducible Markov chains, stationary distribution, convergence to equilibrium.
4. Poisson process and its properties. Birth-and-death processes, queues.
5. Markov processes in continuous time with discrete state space: transition rates, forward and backward equations, stationary distribution.
6. Simulation of stochastic processes.

### Teaching methods

 Delivery type Number Length hours Student hours Workshop 10 1.00 10.00 Lecture 22 1.00 22.00 Practical 2 1.00 2.00 Private study hours 66.00 Total Contact hours 34.00 Total hours (100hr per 10 credits) 100.00

### Methods of assessment

Coursework
 Assessment type Notes % of formal assessment In-course Assessment . 20.00 Total percentage (Assessment Coursework) 20.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

Exams
 Exam type Exam duration % of formal assessment Standard exam (closed essays, MCQs etc) 2 hr 80.00 Total percentage (Assessment Exams) 80.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated