# 2008/09 Undergraduate Module Catalogue

## MATH2750 Introduction to Markov Processes

### 10 creditsClass Size: 200

**Module manager:** Professor A. Yu Veretennikov**Email:** veretenn@maths.leeds.ac.uk

**Taught:** Semester 2 (Jan to Jun) View Timetable

**Year running** 2008/09

### Pre-requisites

MATH1715 | Introduction to Probability |

**This module is approved as an Elective**

### Module summary

A stochastic process refers to any quantity which changes randomly in time. The number of people in a queue, the capacity of a reservoir, the size of a population, are all examples from the real world. The linking model for all these examples is the simple random walk. The gambler's ruin problem is an example of a simple random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points. As a birth-death process this can be used to model population growth, the spread of diseases like AIDS, traffic flow, the queuing of students at a coffee bar, and so on.### Objectives

To provide a simple introduction to stochastic processes.On completion of this module, students should be able to:

(a) have an understanding of, and ability to solve, elementary problems of first passage time distributions;

(b) understand about barriers in a random walk;

(c) solve equilibrium distribution problems;

(d) know the difference between an equilibrium distribution and a stationary distribution;

(e) have a knowledge of Markov chains and elementary theory thereof;

(f) learn about continuous time Markov process models;

(g) have knowledge about the Poisson process;

(h) extend the Poisson process model to other simple examples, and solve associated problems;

(i) understand the role of forward and backward equations;

(j) understand the use of simulation in modelling.

### Syllabus

1. Random walks: transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem.

2. Branching chain, probability of ultimate extinction.

3. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, irreducible Markov chains, stationary distribution, convergence to equilibrium.

4. Poisson process and its properties. Birth-and-death processes, queues.

5. Markov processes in continuous time with discrete state space: transition rates, forward and backward equations, stationary distribution.

6. Simulation of stochastic processes.

### Teaching methods

Due to COVID-19, teaching and assessment activities are being kept under review - see module enrolment pages for information

Delivery type | Number | Length hours | Student hours |

Example Class | 10 | 1.00 | 10.00 |

Lecture | 22 | 1.00 | 22.00 |

Practical | 2 | 1.00 | 2.00 |

Private study hours | 66.00 | ||

Total Contact hours | 34.00 | ||

Total hours (100hr per 10 credits) | 100.00 |

### Methods of assessment

Due to COVID-19, teaching and assessment activities are being kept under review - see module enrolment pages for information

**Coursework**

Assessment type | Notes | % of formal assessment |

Practical | . | 10.00 |

Total percentage (Assessment Coursework) | 10.00 |

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

**Exams**

Exam type | Exam duration | % of formal assessment |

Standard exam (closed essays, MCQs etc) | 2 hr | 90.00 |

Total percentage (Assessment Exams) | 90.00 |

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

### Reading list

The reading list is available from the Library websiteLast updated: 19/07/2010

## Browse Other Catalogues

- Undergraduate module catalogue
- Taught Postgraduate module catalogue
- Undergraduate programme catalogue
- Taught Postgraduate programme catalogue

Errors, omissions, failed links etc should be notified to the Catalogue Team.PROD