2008/09 Undergraduate Module Catalogue
MATH2750 Introduction to Markov Processes
10 creditsClass Size: 200
Module manager: Professor A. Yu Veretennikov
Email: veretenn@maths.leeds.ac.uk
Taught: Semester 2 (Jan to Jun) View Timetable
Year running 2008/09
Pre-requisites
| MATH1715 | Introduction to Probability |
This module is approved as an Elective
Module summary
A stochastic process refers to any quantity which changes randomly in time. The number of people in a queue, the capacity of a reservoir, the size of a population, are all examples from the real world. The linking model for all these examples is the simple random walk. The gambler's ruin problem is an example of a simple random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points. As a birth-death process this can be used to model population growth, the spread of diseases like AIDS, traffic flow, the queuing of students at a coffee bar, and so on.Objectives
To provide a simple introduction to stochastic processes.On completion of this module, students should be able to:
(a) have an understanding of, and ability to solve, elementary problems of first passage time distributions;
(b) understand about barriers in a random walk;
(c) solve equilibrium distribution problems;
(d) know the difference between an equilibrium distribution and a stationary distribution;
(e) have a knowledge of Markov chains and elementary theory thereof;
(f) learn about continuous time Markov process models;
(g) have knowledge about the Poisson process;
(h) extend the Poisson process model to other simple examples, and solve associated problems;
(i) understand the role of forward and backward equations;
(j) understand the use of simulation in modelling.
Syllabus
1. Random walks: transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem.
2. Branching chain, probability of ultimate extinction.
3. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, irreducible Markov chains, stationary distribution, convergence to equilibrium.
4. Poisson process and its properties. Birth-and-death processes, queues.
5. Markov processes in continuous time with discrete state space: transition rates, forward and backward equations, stationary distribution.
6. Simulation of stochastic processes.
Teaching methods
| Delivery type | Number | Length hours | Student hours |
| Example Class | 10 | 1.00 | 10.00 |
| Lecture | 22 | 1.00 | 22.00 |
| Practical | 2 | 1.00 | 2.00 |
| Private study hours | 66.00 | ||
| Total Contact hours | 34.00 | ||
| Total hours (100hr per 10 credits) | 100.00 | ||
Methods of assessment
Coursework
| Assessment type | Notes | % of formal assessment |
| Practical | . | 10.00 |
| Total percentage (Assessment Coursework) | 10.00 | |
Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated
Exams
| Exam type | Exam duration | % of formal assessment |
| Standard exam (closed essays, MCQs etc) | 2 hr | 90.00 |
| Total percentage (Assessment Exams) | 90.00 | |
Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated
Reading list
The reading list is available from the Library websiteLast updated: 19/07/2010
Browse Other Catalogues
- Undergraduate module catalogue
- Taught Postgraduate module catalogue
- Undergraduate programme catalogue
- Taught Postgraduate programme catalogue
Errors, omissions, failed links etc should be notified to the Catalogue Team.PROD