Repository files navigation

🔗Website| 📃Docs| ⚙️Install Guide| 📝Tutorial| 💡Examples|Twitter|LinkedIn|Medium

Optuna is an automatic hyperparameter optimization software framework, particularly designedfor machine learning. It features an imperative,define-by-run style user API. Thanks to ourdefine-by-run API, the code written with Optuna enjoys high modularity, and the user ofOptuna can dynamically construct the search spaces for the hyperparameters.

• Mar 11, 2025: A new article[Optuna v4.2] Gaussian Process-Based Sampler Can Now Handle Inequality Constraints has been published.
• Feb 17, 2025: A new articleSMAC3 Registered on OptunaHub has been published.
• Jan 22, 2025: A new articleOptunaHub Benchmarks: A New Feature to Use/Register Various Benchmark Problems has been published.
• Jan 20, 2025: Optuna 4.2.0 and OptunaHub 0.2.0 are out! Try the newest Optuna and OptunaHub! Check outthe release note for details.
• Jan 16, 2025: A new articleOverview of Python Free Threading (v3.13t) Support in Optuna has been published.
• Nov 12, 2024: We released Optuna 4.1 with new features, Python 3.13 support and much more! Check outthe release note for details.

Optuna has modern functionalities as follows:

• Lightweight, versatile, and platform agnostic architecture
  • Handle a wide variety of tasks with a simple installation that has few requirements.
• Pythonic search spaces
  • Define search spaces using familiar Python syntax including conditionals and loops.
• Efficient optimization algorithms
  • Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials.
• Easy parallelization
  • Scale studies to tens or hundreds of workers with little or no changes to the code.
• Quick visualization
  • Inspect optimization histories from a variety of plotting functions.

We use the termsstudy andtrial as follows:

• Study: optimization based on an objective function
• Trial: a single execution of the objective function

Please refer to the sample code below. The goal of astudy is to find out the optimal set ofhyperparameter values (e.g.,regressor andsvr_c) through multipletrials (e.g.,n_trials=100). Optuna is a framework designed for automation and acceleration ofoptimizationstudies.

import ...# Define an objective function to be minimized.defobjective(trial):# Invoke suggest methods of a Trial object to generate hyperparameters.regressor_name=trial.suggest_categorical('regressor', ['SVR','RandomForest'])ifregressor_name=='SVR':svr_c=trial.suggest_float('svr_c',1e-10,1e10,log=True)regressor_obj=sklearn.svm.SVR(C=svr_c)else:rf_max_depth=trial.suggest_int('rf_max_depth',2,32)regressor_obj=sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)X,y=sklearn.datasets.fetch_california_housing(return_X_y=True)X_train,X_val,y_train,y_val=sklearn.model_selection.train_test_split(X,y,random_state=0)regressor_obj.fit(X_train,y_train)y_pred=regressor_obj.predict(X_val)error=sklearn.metrics.mean_squared_error(y_val,y_pred)returnerror# An objective value linked with the Trial object.study=optuna.create_study()# Create a new study.study.optimize(objective,n_trials=100)# Invoke optimization of the objective function.

Optuna is available atthe Python Package Index and onAnaconda Cloud.

# PyPI$ pip install optuna

# Anaconda Cloud$ conda install -c conda-forge optuna

Optuna has integration features with various third-party libraries. Integrations can be found inoptuna/optuna-integration and the document is availablehere.

Optuna Dashboard is a real-time web dashboard for Optuna.You can check the optimization history, hyperparameter importance, etc. in graphs and tables.You don't need to create a Python script to callOptuna's visualization functions.Feature requests and bug reports are welcome!

optuna-dashboard can be installed via pip:

$ pip install optuna-dashboard

importoptunadefobjective(trial):x1=trial.suggest_float("x1",-100,100)x2=trial.suggest_float("x2",-100,100)returnx1**2+0.01*x2**2study=optuna.create_study(storage="sqlite:///db.sqlite3")# Create a new study with database.study.optimize(objective,n_trials=100)

# Run the study specified above$ python optimize_toy.py# Launch the dashboard based on the storage `sqlite:///db.sqlite3`$ optuna-dashboard sqlite:///db.sqlite3...Listening on http://localhost:8080/Hit Ctrl-C to quit.

OptunaHub is a feature-sharing platform for Optuna.You can use the registered features and publish your packages.

optunahub can be installed via pip:

$ pip install optunahub# Install AutoSampler dependencies (CPU only is sufficient for PyTorch)$ pip install cmaes scipy torch --extra-index-url https://download.pytorch.org/whl/cpu

You can load registered module withoptunahub.load_module.

importoptunaimportoptunahubdefobjective(trial:optuna.Trial)->float:x=trial.suggest_float("x",-5,5)y=trial.suggest_float("y",-5,5)returnx**2+y**2module=optunahub.load_module(package="samplers/auto_sampler")study=optuna.create_study(sampler=module.AutoSampler())study.optimize(objective,n_trials=10)print(study.best_trial.value,study.best_trial.params)

For more details, please refer tothe optunahub documentation.

You can publish your package viaoptunahub-registry.See theOptunaHub tutorial.

• GitHub Discussions for questions.
• GitHub Issues for bug reports and feature requests.

Any contributions to Optuna are more than welcome!

If you are new to Optuna, please check thegood first issues. They are relatively simple, well-defined, and often good starting points for you to get familiar with the contribution workflow and other developers.

If you already have contributed to Optuna, we recommend the othercontribution-welcome issues.

For general guidelines on how to contribute to the project, take a look atCONTRIBUTING.md.

If you use Optuna in one of your research projects, please citeour KDD paper "Optuna: A Next-generation Hyperparameter Optimization Framework":

@inproceedings{akiba2019optuna,title={{O}ptuna: A Next-Generation Hyperparameter Optimization Framework},author={Akiba, Takuya and Sano, Shotaro and Yanase, Toshihiko and Ohta, Takeru and Koyama, Masanori},booktitle={The 25th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},pages={2623--2631},year={2019}}

MIT License (seeLICENSE).

Optuna uses the codes from SciPy and fdlibm projects (seeLICENSE_THIRD_PARTY).