MULTIDISCIPLINARY OPTIMIZATION – AND INDUSTRIAL APPLICATIONS

Master Course, Mauritius, January 2023

Laurent Dumas (Versailles University, France)

Muhammad Zaid Dauhoo (University of Mauritius)

Optimization problems are encountered in many fields of engineering for which the associated cost function may be of various type : black box or explicit, with continuous or discrete variables, costly or not to compute, etc…

After a recall on first and second order descent methods, this course deals with various Derivative Free Optimization methods that have been recently developped, either local or global, deterministic or stochastic. It will be illustrated by various examples issued from industrial fields, usng Python.

 Outline

Session 1 : introdution, motivation. Notion of optimization with or without constraints.
Session 2 : cost function, gradient, descent methods (first and second order)

Session 3 : derivative free optimization, direct methods : pattern search
Session 4 : derivative free optimization, direct methods : Nelder Mead

Session 5 : derivative free optimization, metaheuristics : simulated annealing, evolution strategy
Session 6 : a real problem solving using Python

Documents:

Session 1, lecture notes Zaid Dauhoo

Exercise sessions:

Session 1 and 2 (first and second order methods : Exercices

Session 3 and 4 (direct methods) : Exercises

Session 5 and 6 (metaheuristics) : Exercises

 Python scripts:

Script 1 : first and second order methods, the Rosenbrock case (pdf)

Script 2 : penalisation method for the can problem (pdf)

Script 3 : pattern search for the Rosenbrock function (pdf)

Script 4 : Nelder Mead for the Rosenbrock function (pdf)

Script 5 : simulated annealing for the Rosenbrock function (pdf)

Script 6 : CMA-ES for the Rosenbrock function (pdf)

 Case study:

Script 7(Python) or script in Matlab: the epidemiological SIHR model + two sets of COVID data from March to June 2020 (pdf)

Each group will have to find the best fitted parameters with one of the algorithms below :

• Pattern Search

• Nelder Mead

• Simulated Annealing

• CMAES

and one of the data :

• Hospitalization in France

• Hospitalization in Rhone Alpes Auvergne (region of France)

and to present on a commented script its results.

Industrial applications :  slides.pdf

Bibliography :

• N. Gould, S. Leyffer, An introdution to algorithms for non linear optimization, online.

• Lecture notes (in french) Max Cerf (slides)

• A complete book on optimization (in french) : Max Cerf, Techniques d'optimisation, tome 1 : optimisation continue, EDP Sciences, 2022

• A reference book on DFO : A. Conn, K. Scheinberg and L. Vincente, Introduction to Derivative Free Optimization, SIAM, 2009.

• A review article on DFO : Journal Global Opt. 2013

• A presentation of Simulated Annealing (in french):  Interstices

• A presentation of CMA-ES : cmaesintro

• A list of optimization test functions : http://www.sfu.ca/~ssurjano/optimization.html