[edit]
MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts
Diederick Vermetten, Furong Ye, Thomas Bäck, Carola DoerrProceedings of the Second International Conference on Automated Machine Learning, PMLR 224:7/1-14, 2023.
Abstract
Extending a recent suggestion to generate new instances for numerical black-box optimization benchmarking by interpolating pairs of the well-established BBOB functions from the COmparing COntinuous Optimizers (COCO) platform, we propose in this work a further generalization that allows multiple affine combinations of the original instances and arbitrarily chosen locations of the global optima. We demonstrate that the MA-BBOB generator can help fill the instance space, while overall patterns in algorithm performance are preserved. By combining the landscape features of the problems with the performance data, we pose the question of whether these features are as useful for algorithm selection as previous studies have implied. MA-BBOB is built on the publicly available IOHprofiler platform, which facilitates standardized experimentation routines, provides access to the interactive IOHanalyzer module for performance analysis and visualization, and enables comparisons with the rich and growing data collection available for the (MA-)BBOB functions.
Cite this Paper
BibTeX
@InProceedings{pmlr-v224-vermetten23a, title = {MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts}, author = {Vermetten, Diederick and Ye, Furong and B\"ack, Thomas and Doerr, Carola}, booktitle = {Proceedings of the Second International Conference on Automated Machine Learning}, pages = {7/1--14}, year = {2023}, editor = {Faust, Aleksandra and Garnett, Roman and White, Colin and Hutter, Frank and Gardner, Jacob R.}, volume = {224}, series = {Proceedings of Machine Learning Research}, month = {12--15 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v224/vermetten23a/vermetten23a.pdf}, url = {https://proceedings.mlr.press/v224/vermetten23a.html}, abstract = {Extending a recent suggestion to generate new instances for numerical black-box optimization benchmarking by interpolating pairs of the well-established BBOB functions from the COmparing COntinuous Optimizers (COCO) platform, we propose in this work a further generalization that allows multiple affine combinations of the original instances and arbitrarily chosen locations of the global optima. We demonstrate that the MA-BBOB generator can help fill the instance space, while overall patterns in algorithm performance are preserved. By combining the landscape features of the problems with the performance data, we pose the question of whether these features are as useful for algorithm selection as previous studies have implied. MA-BBOB is built on the publicly available IOHprofiler platform, which facilitates standardized experimentation routines, provides access to the interactive IOHanalyzer module for performance analysis and visualization, and enables comparisons with the rich and growing data collection available for the (MA-)BBOB functions.}}
Endnote
%0 Conference Paper%T MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts%A Diederick Vermetten%A Furong Ye%A Thomas Bäck%A Carola Doerr%B Proceedings of the Second International Conference on Automated Machine Learning%C Proceedings of Machine Learning Research%D 2023%E Aleksandra Faust%E Roman Garnett%E Colin White%E Frank Hutter%E Jacob R. Gardner%F pmlr-v224-vermetten23a%I PMLR%P 7/1--14%U https://proceedings.mlr.press/v224/vermetten23a.html%V 224%X Extending a recent suggestion to generate new instances for numerical black-box optimization benchmarking by interpolating pairs of the well-established BBOB functions from the COmparing COntinuous Optimizers (COCO) platform, we propose in this work a further generalization that allows multiple affine combinations of the original instances and arbitrarily chosen locations of the global optima. We demonstrate that the MA-BBOB generator can help fill the instance space, while overall patterns in algorithm performance are preserved. By combining the landscape features of the problems with the performance data, we pose the question of whether these features are as useful for algorithm selection as previous studies have implied. MA-BBOB is built on the publicly available IOHprofiler platform, which facilitates standardized experimentation routines, provides access to the interactive IOHanalyzer module for performance analysis and visualization, and enables comparisons with the rich and growing data collection available for the (MA-)BBOB functions.
APA
Vermetten, D., Ye, F., Bäck, T. & Doerr, C.. (2023). MA-BBOB: Many-Affine Combinations of BBOB Functions for Evaluating AutoML Approaches in Noiseless Numerical Black-Box Optimization Contexts.Proceedings of the Second International Conference on Automated Machine Learning, inProceedings of Machine Learning Research 224:7/1-14 Available from https://proceedings.mlr.press/v224/vermetten23a.html.