## MATH2750 Introduction to Markov Processes

### 10 creditsClass Size: 200

Module manager: Professor A. Yu Veretennikov
Email: veretenn@maths.leeds.ac.uk

Taught: Semester 2 (Jan to Jun) View Timetable

Year running 2008/09

### Pre-requisites

 MATH1715 Introduction to Probability

This module is approved as an Elective

### Module summary

A stochastic process refers to any quantity which changes randomly in time. The number of people in a queue, the capacity of a reservoir, the size of a population, are all examples from the real world. The linking model for all these examples is the simple random walk. The gambler's ruin problem is an example of a simple random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points. As a birth-death process this can be used to model population growth, the spread of diseases like AIDS, traffic flow, the queuing of students at a coffee bar, and so on.

### Objectives

To provide a simple introduction to stochastic processes.
On completion of this module, students should be able to:
(a) have an understanding of, and ability to solve, elementary problems of first passage time distributions;
(b) understand about barriers in a random walk;
(c) solve equilibrium distribution problems;
(d) know the difference between an equilibrium distribution and a stationary distribution;
(e) have a knowledge of Markov chains and elementary theory thereof;
(f) learn about continuous time Markov process models;
(g) have knowledge about the Poisson process;
(h) extend the Poisson process model to other simple examples, and solve associated problems;
(i) understand the role of forward and backward equations;
(j) understand the use of simulation in modelling.

### Syllabus

1. Random walks: transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem.
2. Branching chain, probability of ultimate extinction.
3. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, irreducible Markov chains, stationary distribution, convergence to equilibrium.
4. Poisson process and its properties. Birth-and-death processes, queues.
5. Markov processes in continuous time with discrete state space: transition rates, forward and backward equations, stationary distribution.
6. Simulation of stochastic processes.

### Teaching methods

Due to COVID-19, teaching and assessment activities are being kept under review - see module enrolment pages for information

 Delivery type Number Length hours Student hours Example Class 10 1.00 10.00 Lecture 22 1.00 22.00 Practical 2 1.00 2.00 Private study hours 66.00 Total Contact hours 34.00 Total hours (100hr per 10 credits) 100.00

### Methods of assessment

Due to COVID-19, teaching and assessment activities are being kept under review - see module enrolment pages for information

Coursework
 Assessment type Notes % of formal assessment Practical . 10.00 Total percentage (Assessment Coursework) 10.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

Exams
 Exam type Exam duration % of formal assessment Standard exam (closed essays, MCQs etc) 2 hr 90.00 Total percentage (Assessment Exams) 90.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated