CS7DS2: Optimisation For Machine Learning 2021-22

Module descriptor

Lectures: Tues 3-4pm, Weds 3-4pm
Lecturer: Doug Leith
Teaching Assistants: Dilina Rajapakse, Chun Wei Ooi

The module will take a hands-on approach and aims to develop practical understanding of the how to select and use optimisation algorithms for learning/training in supervised machine learning.

• Introduction
• Gradients
• Gradient Descent
• Gradient Descent 2
• Line Search
• Polyak Step Size, Adagrad, RMSProp
• Momentum Methods: Heavy Ball
• Momentum Methods: Nesterov, Adam
• Stochastic Gradient Descent 1
• Stochastic Gradient Descent 2
• Newtons Method
• Derivative-Free Methods 1
• Derivative-Free Methods 2
• Projected Gradient Descent
• Penalty Methods
• Frank-Wolfe Method

• Algorithms for Optimization, Kochenderfer & Wheeler 2019