# 2021/22 Undergraduate Module Catalogue

## MATH2750 Introduction to Markov Processes

### 10 creditsClass Size: 190

**Module manager:** Dr Valentine Nlebedim**Email:** V.U.Nlebedim@leeds.ac.uk

**Taught:** Semester 2 (Jan to Jun) View Timetable

**Year running** 2021/22

### Pre-requisite qualifications

[(MATH1710 and MATH1712) or MATH2700] or [LUBS1270 and LUBS1280], or equivalent.**This module is approved as a discovery module**

### Module summary

A stochastic process refers to any quantity which changes randomly in time. The capacity of a reservoir, an individualâ€™s level of no claims discount, the number of insurance claims, the value of pension fund assets, and the size of a population, are all examples from the real world. The linking model for all these examples is the Markov process, which includes random walk, Markov chain and Markov jump processes. The gambler's ruin problem is a simple example of a random walk with two absorbing barriers. Replacing these absorbing barriers with reflecting barriers provides a model for reservoir capacity. With appropriate modifications the random walk can be extended to model stochastic processes which change over continuous time, not just at regularly spaced time points.### Objectives

The aim of this module is to study simple stochastic processes with applications including biological, financial and actuarial sciences through theoretical results and the use of statistical software for model fitting and simulation. For exemptions from actuarial exams, please see http://www.mathsstudents.leeds.ac.uk/careers-employment/exemption-from-professional-exams.html**Learning outcomes**

On completion of this module, students should be able to:

(a) have knowledge of the key features of stochastic process;

(b) define the random walk and understand about barriers;

(c) solve ruin, first passage time and expected duration problems;

(d) define and classify Markov chain models;

(e) calculate long-term probability distributions for simple models;

(f) define continuous time Markov jump processes;

(g) have knowledge of the Poisson process;

(h) define simple time-inhomogeneous processes;

(i) use maximum likelihood estimation for transition probabilities and intensities;

(j) use statistical software for simulation.

### Syllabus

1. Difference between deterministic and stochastic models. The role of models.

2. Definitions of stochastic processes, state space and time, mixed processes, the Markov property. Actuarial applications.

3. Random walks, transition probabilities, first passage time, recurrence, absorbing and reflecting barriers, gambler's ruin problem. Examples, eg financial indexes.

4. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, stationary distribution, convergence to equilibrium.

5. Application of Markov chain models, eg no-claims discount, sickness, marriage.

6. Estimation of probabilities, simulation and assessing goodness-of-fit.

7. Simple examples of time-inhomogeneous Markov chains.

8. Definition of continuous time processes.

9. Jump processes. Poisson process, inter-event times, Kolmogorov equations.

10. Time-inhomogeneous processes with generalisation of earlier examples (sickness, death, marriage models).

11. Use of maximum likelihood for estimating transition intensities.

12. Binomial and Poisson models of mortality. Maximum likelihood estimators for the probability of death.

13. Simulation of random walk, Markov chain and Markov jump processes with both time homogeneous and inhomogeneous examples.

### Teaching methods

Delivery type | Number | Length hours | Student hours |

Workshop | 10 | 1.00 | 10.00 |

Lecture | 11 | 1.00 | 11.00 |

Practical | 2 | 1.00 | 2.00 |

Private study hours | 77.00 | ||

Total Contact hours | 23.00 | ||

Total hours (100hr per 10 credits) | 100.00 |

### Private study

Studying and revising of course material.Completing of assignments and assessments.

### Opportunities for Formative Feedback

Regular problem solving assignments### Methods of assessment

**Coursework**

Assessment type | Notes | % of formal assessment |

In-course Assessment | . | 15.00 |

Total percentage (Assessment Coursework) | 15.00 |

There is no resit available for the coursework component of this module. If the module is failed, the coursework mark will be carried forward and added to the resit exam mark with the same weighting as listed above.

**Exams**

Exam type | Exam duration | % of formal assessment |

Standard exam (closed essays, MCQs etc) | 2 hr 00 mins | 85.00 |

Total percentage (Assessment Exams) | 85.00 |

Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated

### Reading list

The reading list is available from the Library websiteLast updated: 29/03/2022 15:20:27

## Browse Other Catalogues

- Undergraduate module catalogue
- Taught Postgraduate module catalogue
- Undergraduate programme catalogue
- Taught Postgraduate programme catalogue

Errors, omissions, failed links etc should be notified to the Catalogue Team.PROD