2023/24 Undergraduate Module Catalogue

MATH2750 Introduction to Markov Processes

10 creditsClass Size: 190

Module manager: Dr Benjamin Lees
Email: B.T.Lees@leeds.ac.uk

Taught: Semester 2 (Jan to Jun) View Timetable

Year running 2023/24

Pre-requisite qualifications

[(MATH1710 and MATH1712) or MATH2700] or [LUBS1270 and LUBS1280], or equivalent.

This module is approved as a discovery module

Module summary

A stochastic process refers to any quantity which changes randomly in time. The capacity of a reservoir, an individual’s level of no claims discount, the number of insurance claims, the value of pension fund assets, and the size of a population, are all examples from the real world. The linking model for all these examples is the Markov process, which includes random walk, Markov chain and Markov jump processes. The gambler's ruin problem is a simple example of a random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points.

Objectives

The aim of this module is to study simple stochastic processes with applications including biological, financial and actuarial sciences through theoretical results and the use of statistical software for model fitting and simulation. For exemptions from actuarial exams, please see http://www.mathsstudents.leeds.ac.uk/careers-employment/exemption-from-professional-exams.html

Learning outcomes
On completion of this module, students should be able to:
(a) have knowledge of the key features of stochastic process;
(b) define the random walk and understand about barriers;
(c) solve ruin, first passage time and expected duration problems;
(d) define and classify Markov chain models;
(e) calculate long-term probability distributions for simple models;
(f) define continuous time Markov jump processes;
(g) have knowledge of the Poisson process;
(h) define simple time-inhomogeneous processes;
(i) use maximum likelihood estimation for transition probabilities and intensities;
(j) use statistical software for simulation.

Syllabus

1. Difference between deterministic and stochastic models. The role of models.
2. Definitions of stochastic processes, state space and time, mixed processes, the Markov property. Actuarial applications.
3. Random walks, transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem. Examples, eg financial indexes.
4. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, stationary distribution, convergence to equilibrium.
5. Application of Markov chain models, eg no-claims discount, sickness, marriage.
6. Estimation of probabilities, simulation and assessing goodness-of-fit.
7. Simple examples of time-inhomogeneous Markov chains.
8. Definition of continuous time processes.
9. Jump processes. Poisson process, inter-event times, Kolmogorov equations.
10. Time-inhomogeneous processes with generalisation of earlier examples (sickness, death, marriage models).
11. Use of maximum likelihood for estimating transition intensities.
12. Binomial and Poisson models of mortality. Maximum likelihood estimators for the probability of death.
13. Simulation of random walk, Markov chain and Markov jump processes with both time homogeneous and inhomogeneous examples.

Teaching methods

Delivery type	Number	Length hours	Student hours
Workshop	10	1.00	10.00
Lecture	22	1.00	22.00
Practical	2	1.00	2.00
Private study hours			66.00
Total Contact hours			34.00
Total hours (100hr per 10 credits)			100.00

Private study

Studying and revising of course material.
Completing of assignments and assessments.

Opportunities for Formative Feedback

Regular problem solving assignments

Methods of assessment

Coursework

Assessment type	Notes	% of formal assessment
In-course Assessment	.	15.00
Total percentage (Assessment Coursework)		15.00

There is no resit available for the coursework component of this module. If the module is failed, the coursework mark will be carried forward and added to the resit exam mark with the same weighting as listed above.

Exams

Exam type	Exam duration	% of formal assessment
Standard exam (closed essays, MCQs etc)	2 hr 00 mins	85.00
Total percentage (Assessment Exams)		85.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

Reading list

The reading list is available from the Library website

Last updated: 18/08/2023

Disclaimer

Browse Other Catalogues

• Undergraduate module catalogue
• Taught Postgraduate module catalogue
• Undergraduate programme catalogue
• Taught Postgraduate programme catalogue

Errors, omissions, failed links etc should be notified to the Catalogue Team.PROD