Tatu Linnala
Pareto front learning in multi-objective Bayesian optimization
Bayesian optimization (BO) is a popular machine learning technique for optimizing expensive black-box functions. It is widely used in many fields of science and engineering, for example, in materials structure and composition optimization tasks. Many BO applications involve multiple competing objectives, requiring decision-makers to seek optimal trade-offs that form the so-called Pareto front. Multi-objective optimization algorithms aim to approximate this front. A significant contribution of this work is to extend the Bayesian Optimization Structure Search (BOSS) code to support advanced multi-objective BO methods. The Hypervolume Improvement and three variants of the Expected Hypervolume Improvement acquisition function were implemented: an exact formulation for bi-objective cases and Monte Carlo-based approximations for scenarios with any number of objectives. Additionally, we propose adding an explicit exploration term to the Hypervolume Improvement acquisition function. We show that this can significantly boost the performance of the method in applications where exploration of the design space is required. Scalarization-based approaches and an extension of the single-objective ELCB acquisition function to multi-objective cases, which offer computationally cheaper solutions, were also included. The methods are benchmarked on six test cases, including five synthetic functions and a real-world problem based on lignin extraction data. The results show that the hypervolume-based methods provide the most accurate Pareto front predictions, but cheaper methods may sometimes be sufficient. This highlights the trade-offs between computational cost and accuracy, emphasizing the importance of selecting methods based on application-specific needs. This thesis provides guidelines for choosing appropriate multi-objective BO settings, which can all be accessed within the extended BOSS code. Thus, the BOSS code now provides an accessible, comprehensive toolkit for multi-objective optimization tasks.
2025