v4.2.0

Compare

Choose a tag to compare

c-bata released this 20 Jan 07:13

v4.2.0

13ebffa

This commit was created on GitHub.com and signed with GitHub’sverified signature.

GPG key ID:B5690EEEBB952194

Verified

Learn about vigilant mode.

This is the release note ofv4.2.0. In conjunction with the Optuna release, OptunaHub 0.2.0 is released. Please refer tothe release note of OptunaHub 0.2.0 for more details.

Highlights of this release include:

🚀gRPC Storage Proxy for Scalable Hyperparameter Optimization
🤖 SMAC3: Support for New State-of-the-art Optimization Algorithm by AutoML.org (@automl)
📁 OptunaHub Now Supports Benchmark Functions
🧑‍💻 Gaussian Process-Based Bayesian Optimization with Inequality Constraints
🧑‍💻 c-TPE: Support Constrained TPESampler

Highlights

gRPC Storage Proxy for Scalable Hyperparameter Optimization

The gRPC storage proxy is a feature designed to support large-scale distributed optimization. As shown in the diagram below, gRPC storage proxy sits between the optimization workers and the database server, proxying the calls of Optuna’s storage APIs.

In large-scale distributed optimization settings where hundreds to thousands of workers are operating, placing a gRPC storage proxy for every few tens can significantly reduce the load on the RDB server which would otherwise be a single point of failure. The gRPC storage proxy enables sharing the cache about Optuna studies and trials, which can further mitigate load. Please refer tothe official documentation for further details on how to utilize gRPC storage proxy.

SMAC3: Random Forest-Based Bayesian Optimization Developed by AutoML.org

SMAC3 is a hyperparameter optimization framework developed byAutoML.org, one of the most influential AutoML research groups. The Optuna-compatible SMAC3 sampler is now available thanks to the contribution to OptunaHub by Difan Deng (@dengdifan), one of the core members of AutoML.org. We can now use the method widely used in AutoML research and real-world applications from Optuna.

# pip install optunahub smacimportoptunaimportoptunahubfromoptuna.distributionsimportFloatDistributiondefobjective(trial:optuna.Trial)->float:x=trial.suggest_float("x",-10,10)y=trial.suggest_float("y",-10,10)returnx**2+y**2smac_mod=optunahub.load_module("samplers/smac_sampler")n_trials=100sampler=smac_mod.SMACSampler(    {"x":FloatDistribution(-10,10),"y":FloatDistribution(-10,10)},n_trials=n_trials,)study=optuna.create_study(sampler=sampler)study.optimize(objective,n_trials=n_trials)

Please refer tohttps://hub.optuna.org/samplers/smac_sampler/ for more details.

OptunaHub Now Supports Benchmark Functions

Benchmarking the performance of optimization algorithms is an essential process indispensable to the research and development of algorithms. The newly added OptunaHub Benchmarks in the latest version v0.2.0 ofoptunahub is a new feature for Optuna users to conduct benchmarks conveniently.

# pip install optunahub>=4.2.0 scipy torchimportoptunaimportoptunahubbbob_mod=optunahub.load_module("benchmarks/bbob")smac_mod=optunahub.load_module("samplers/smac_sampler")sphere2d=bbob_mod.Problem(function_id=1,dimension=2)n_trials=100studies= []forstudy_name,samplerin [    ("random",optuna.samplers.RandomSampler(seed=1)),    ("tpe",optuna.samplers.TPESampler(seed=1)),    ("cmaes",optuna.samplers.CmaEsSampler(seed=1)),    ("smac",smac_mod.SMACSampler(sphere2d.search_space,n_trials,seed=1)),]:study=optuna.create_study(directions=sphere2d.directions,sampler=sampler,study_name=study_name)study.optimize(sphere2d,n_trials=n_trials)studies.append(study)optuna.visualization.plot_optimization_history(studies).show()

In the above sample code, we compare and display the performance of the four kinds of samplers using a two-dimensional Sphere function, which is part of a group of benchmark functions widely used in the black-box optimization research community known asBlackbox Optimization Benchmarking (BBOB).

Gaussian Process-Based Bayesian Optimization with Inequality Constraints

We worked on its extension and adaptedGPSampler to constrained optimization in Optuna v4.2.0 since Gaussian process-based Bayesian optimization is a very popular method in various research fields such as aircraft engineering and materials science. We show the basic usage below.

# pip install optuna>=4.2.0 scipy torchimportnumpyasnpimportoptunadefobjective(trial:optuna.Trial)->float:x=trial.suggest_float("x",0.0,2*np.pi)y=trial.suggest_float("y",0.0,2*np.pi)c=float(np.sin(x)*np.sin(y)+0.95)trial.set_user_attr("c",c)returnfloat(np.sin(x)+y)defconstraints(trial:optuna.trial.FrozenTrial)->tuple[float]:return (trial.user_attrs["c"],)sampler=optuna.samplers.GPSampler(constraints_func=constraints)study=optuna.create_study(sampler=sampler)study.optimize(objective,n_trials=50)

Please try outGPSampler for constrained optimization especially when only a small number of trials are available!

c-TPE: Support Constrained TPESampler

Although Optuna has supported constrained optimization forTPESampler, which is the default Optuna sampler, since v3.0.0, its algorithm design and performance comparison have not been verified academically. OptunaHub now supportsc-TPE, which is another constrained optimization method forTPESampler. Importantly, the algorithm design and its performance comparison are publicly reviewed to be accepted to IJCAI, a top-tier AI international conference. Please refer tohttps://hub.optuna.org/samplers/ctpe/ for details.

New Features

EnableGPSampler to support constraint functions (#5715)
Update output format options in CLI to include thevalue choice (#5822, thanks@iamarunbrahma!)
Add gRPC storage proxy server and client (#5852)

Enhancements

Introduce client-side cache inGrpcStorageProxy (#5872)

Bug Fixes

Fix CI (optuna/optuna-integration#185)
Fix ticks in Matplotlib contour plot (#5778, thanks@sulan!)
Adding check incli.py to handle an empty database (#5828, thanks@willdavidson05!)
Avoid the input validation fail in Wilcoxon signed ranked test for Scipy 1.15 (#5912)
Fix the default sampler ofload_study function (#5924)

Documentation

Update OptunaHub example in README (#5763)
Updatedistributions.rst to list deprecated distribution classes (#5764)
Remove deprecation comment forstep inIntLogUniformDistribution (#5767)
Update requirements for OptunaHub in README (#5768)
Use inline code rather than italic forstep (#5769)
Add notes toask_and_tell tutorial - batch optimization recommendations (#5817, thanks@SimonPop!)
Fix the explanation of returned values ofget_trial_params (#5820)
Introducesphinx-notfound-page for better 404 page (#5898)
Follow-up#5872: Update the docstring ofrun_grpc_proxy_server (#5914)
Modify doc-string of gRPC-related modules (#5916)

Examples

Adapt docker recipes to Python 3.11 (optuna/optuna-examples#292)
Add version constraint forwandb (optuna/optuna-examples#293)

Tests

Add unit tests forretry_history method inRetryFailedTrialCallback (#5865, thanks@iamarunbrahma!)
Add tests for value format in CLI (#5866)
Import grpc lazy to fix the CI (#5878)

Code Fixes

Fix annotations fordistributions.py (#5755, thanks@KannanShilen!)
Simplify type annotations fortests/visualization_tests/test_pareto_front.py (#5756, thanks@boringbyte!)
Fix type annotations fortest_hypervolume_history.py (#5760, thanks@boringbyte!)
Simplify type annotations fortests/test_cli.py (#5765, thanks@boringbyte!)
Simplify type annotations fortests/test_distributions.py (#5773, thanks@boringbyte!)
Simplify type annotations fortests/samplers_tests/test_qmc.py (#5775, thanks@boringbyte!)
Simplify type annotations fortests/sampler_tests/tpe_tests/test_sampler.py (#5779, thanks@boringbyte!)
Simplify type annotations fortests/samplers_tests/tpe_tests/test_multi_objective_sampler.py (#5781, thanks@boringbyte!)
Simplify type annotations fortests/samplers_tests/tpe_tests/test_parzen_estimator.py (#5782, thanks@boringbyte!)
Simplify type annotations fortests/storage_tests/journal_tests/test_journal.py (#5783, thanks@boringbyte!)
Simplify type annotations fortests/storage_tests/rdb_tests/create_db.py (#5784, thanks@boringbyte!)
Simplify type annotations fortests/storage_tests/rdb_tests/ (#5785, thanks@boringbyte!)
Simplify type annotations fortests/test_deprecated.py (#5786, thanks@boringbyte!)
Simplify type annotations fortests/test_convert_positional_args.py (#5787, thanks@boringbyte!)
Simplify type annotations fortests/importance_tests/test_init.py (#5790, thanks@boringbyte!)
Simplify type annotations fortests/storages_tests/test_storages.py (#5791, thanks@boringbyte!)
Simplify type annotations fortests/storages_tests/test_heartbeat.py (#5792, thanks@boringbyte!)
Simplify type annotationsoptuna/cli.py (#5793, thanks@willdavidson05!)
Simplify type annotations fortests/trial_tests/test_frozen.py (#5794, thanks@boringbyte!)
Simplify type annotations fortests/trial_tests/test_trial.py (#5795, thanks@boringbyte!)
Simplify type annotations fortests/trial_tests/test_trials.py (#5796, thanks@boringbyte!)
Refactor some funcs in NSGA-III (#5798)
Simplify type annotationsoptuna/_transform.py (#5799, thanks@JLX0!)
Make the assertion messages intest_trial.py readable (#5800)
Simplify type annotations fortests/pruners_tests/test_hyperband.py (#5801, thanks@boringbyte!)
Simplify type annotations fortests/pruners_tests/test_median.py (#5802, thanks@boringbyte!)
Simplify type annotations fortests/pruners_tests/test_patient.py (#5803, thanks@boringbyte!)
Usestudy.ask() in tests instead ofcreate_new_trial (#5807, thanks@unKnownNG!)
Simplify type annotations fortests/pruners_tests/test_percentile.py (#5808, thanks@boringbyte!)
Simplify type annotations fortests/pruners_tests/test_successive_halving.py (#5809, thanks@boringbyte!)
Simplify type annotations fortests/study_tests/test_optimize.py (#5810, thanks@boringbyte!)
Simplify type annotations fortests/hypervolume_tests/test_hssp.py (#5812, thanks@boringbyte!)
Refactor the MOTPE split (#5813)
Simplify type annotations foroptuna/_callbacks.py (#5818, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/_random.py (#5819, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/_gp/sampler.py (#5823, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/nsgaii/_crossovers/_sbx.py (#5824, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/nsgaii/_crossovers/_spx.py (#5825, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/nsgaii/_crossovers/_undx.py (#5826, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/nsgaii/_crossovers/_vsbx.py (#5827, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/nsgaii/_crossover.py (#5831, thanks@boringbyte!)
Simplify type annotations foroptuna/samplers/testing/threading.py (#5832, thanks@boringbyte!)
Simplify type annotations foroptuna/pruners/_patient.py (#5833, thanks@boringbyte!)
Usestudy.ask() intests/pruners_tests/test_percentile.py (#5834, thanks@fusawa-yugo!)
Simplify type annotations foroptuna/search_space/group_decomposed.py (#5836, thanks@boringbyte!)
Simplify type annotations foroptuna/search_space/intersection.py (#5837, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/journal/_base.py (#5838, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/journal/_redis.py (#5840, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/journal/_storage.py (#5841, thanks@boringbyte!)
Simplify type annotations foroptuna/study/_dataframe.py (#5842, thanks@boringbyte!)
Fix logger.warn() deprecation warning in GP module (#5843, thanks@iamarunbrahma!)
Simplify type annotations foroptuna/study/_optimize.py (#5844, thanks@boringbyte!)
Simplify type annotations foroptuna/study/_tell.py (#5845, thanks@boringbyte!)
Fixmypy errors due tonumpy 2.2.0 (#5848)
Simplify type annotations foroptuna/visualization/matplotlib/_contour.py (#5851, thanks@boringbyte!)
Refactor the fix for MyPy errors due to NumPy v2.2.0 (#5853)
Simplify type annotations foroptuna/visualization/matplotlib/_parallel_coordinate.py (#5854, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/matplotlib/_param_importances.py (#5855, thanks@boringbyte!)
Usestudy.ask() intests/pruners_tests/test_successive_halving.py (#5856, thanks@willdavidson05!)
Simplify type annotations foroptuna/visualization/matplotlib/_pareto_front.py (#5857, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/matplotlib/_rank.py (#5858, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/matplotlib/_slice.py (#5859, thanks@boringbyte!)
Refactor plot contour (#5867)
Refactor MOTPE weighting (#5871)
Simplify type annotations foroptuna/visualization/_utils.py (#5876, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/_contour.py (#5877, thanks@boringbyte!)
Simplify type annotations foroptuna/_gp/gp.py (#5879, thanks@boringbyte!)
Simplify type annotations foroptuna/study/_tell.py (#5880, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/_heartbeat.py (#5882, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/_rdb/storage.py (#5883, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/_rdb/alembic/versions/v3.0.0.a.py (#5884, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/_rdb/alembic/versions/v3.0.0.c.py (#5885, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/_rdb/alembic/versions/v3.0.0.d.py (#5886, thanks@boringbyte!)
Simplify type annotations foroptuna/_deprecated.py (#5887, thanks@boringbyte!)
Simplify type annotations foroptuna/_experimental.py (#5888, thanks@boringbyte!)
Simplify type annotations foroptuna/_imports.py (#5889, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/_slice.py (#5894, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/_parallel_coordinate.py (#5895, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/_rank.py (#5896, thanks@boringbyte!)
Simplify type annotations foroptuna/visualization/_param_importances.py (#5897, thanks@boringbyte!)
Simplify type annotations foroptuna/_callbacks.py (#5899, thanks@boringbyte!)
Simplify type annotations foroptuna/storages/journal/_file.py (#5900, thanks@boringbyte!)
Simplify type annotations fortests/storages_tests/test_with_server.py (#5901, thanks@boringbyte!)
Simplify type annotations fortests/test_multi_objective.py (#5902, thanks@boringbyte!)
Simplify type annotations fortests/artifacts_tests/test_gcs.py (#5903, thanks@boringbyte!)
Simplify type annotations fortests/samplers_tests/test_grid.py (#5904, thanks@boringbyte!)
Simplify type annotations foroptuna/study/_dataframe.py (#5905, thanks@boringbyte!)
Simplify type annotations fortests/visualization_tests/test_optimization_history.py (#5906, thanks@boringbyte!)
Simplify type annotations fortests/visualization_tests/test_intermediate_plot.py (#5907, thanks@boringbyte!)
Simplify type annotations for multiple test files intests/visualization_tests/ (#5908, thanks@boringbyte!)
Avoid the port number conflict in the copy study tests (#5913)
Simplify annotations intests/study_tests/test_study.py (#5923, thanks@sawa3030!)

Continuous Integration

Fix a GitHub action workflow for publishing to PyPI (optuna/optuna-integration#181)
Fix CI by adding a version constraint (optuna/optuna-integration#186)
Renametest_cache_is_invalidated and removeassert study._thread_local.cached_all_trials is None (#5733)
Use Python 3.12 for docs build CI (#5742)
Fix a GitHub action workflow for publishing to PyPI (#5759)
Install olderkaleido to fix CI errors (#5771)
Add Python 3.12 to the Docker image CI (#5789)
Movemypy related entries in setup.cfg topyproject.toml (#5861)