2019/20 Undergraduate Module Catalogue

MATH2750 Introduction to Markov Processes

10 creditsClass Size: 190

Module manager: Dr Matthew Aldridge
Email: m.aldridge@leeds.ac.uk

Taught: Semester 2 View Timetable

Year running 2019/20

Pre-requisite qualifications

[(MATH1710 and MATH1712) or MATH2700] or [LUBS1270 and LUBS1280], or equivalent.

This module is approved as a discovery module

Module summary

A stochastic process refers to any quantity which changes randomly in time. The capacity of a reservoir, an individual’s level of no claims discount, the number of insurance claims, the value of pension fund assets, and the size of a population, are all examples from the real world. The linking model for all these examples is the Markov process, which includes random walk, Markov chain and Markov jump processes. The gambler's ruin problem is a simple example of a random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points.

Objectives

The aim of this module is to study simple stochastic processes with applications including biological, financial and actuarial sciences through theoretical results and the use of statistical software for model fitting and simulation. For exemptions from actuarial exams, please see http://www.mathsstudents.leeds.ac.uk/careers-employment/exemption-from-professional-exams.html

Learning outcomes
On completion of this module, students should be able to:
(a) have knowledge of the key features of stochastic process;
(b) define the random walk and understand about barriers;
(c) solve ruin, first passage time and expected duration problems;
(d) define and classify Markov chain models;
(e) calculate long-term probability distributions for simple models;
(f) define continuous time Markov jump processes;
(g) have knowledge of the Poisson process;
(h) define simple time-inhomogeneous processes;
(i) use maximum likelihood estimation for transition probabilities and intensities;
(j) use statistical software for simulation.

Syllabus

1. Difference between deterministic and stochastic models. The role of models.
2. Definitions of stochastic processes, state space and time, mixed processes, the Markov property. Actuarial applications.
3. Random walks, transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem. Examples, eg financial indexes.
4. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, stationary distribution, convergence to equilibrium.
5. Application of Markov chain models, eg no-claims discount, sickness, marriage.
6. Estimation of probabilities, simulation and assessing goodness-of-fit.
7. Simple examples of time-inhomogeneous Markov chains.
8. Definition of continuous time processes.
9. Jump processes. Poisson process, inter-event times, Kolmogorov equations.
10. Time-inhomogeneous processes with generalisation of earlier examples (sickness, death, marriage models).
11. Use of maximum likelihood for estimating transition intensities.
12. Binomial and Poisson models of mortality. Maximum likelihood estimators for the probability of death.
13. Simulation of random walk, Markov chain and Markov jump processes with both time homogeneous and inhomogeneous examples.

Teaching methods

 Delivery type Number Length hours Student hours Workshop 10 1.00 10.00 Lecture 22 1.00 22.00 Practical 2 1.00 2.00 Private study hours 66.00 Total Contact hours 34.00 Total hours (100hr per 10 credits) 100.00

Private study

Studying and revising of course material.
Completing of assignments and assessments.

Opportunities for Formative Feedback

Regular problem solving assignments

Methods of assessment

Coursework
 Assessment type Notes % of formal assessment In-course Assessment . 15.00 Total percentage (Assessment Coursework) 15.00

There is no resit available for the coursework component of this module. If the module is failed, the coursework mark will be carried forward and added to the resit exam mark with the same weighting as listed above.

Exams
 Exam type Exam duration % of formal assessment Standard exam (closed essays, MCQs etc) 2 hr 00 mins 85.00 Total percentage (Assessment Exams) 85.00

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated