google/vizierPublic

NotificationsYou must be signed in to change notification settings
Fork102
Star1.6k

Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.

License

Apache-2.0 license

1.6k stars 102 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 1,200 Commits
.github/workflows		.github/workflows
demos		demos
docs		docs
vizier		vizier
.readthedocs.yml		.readthedocs.yml
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
build_protos.sh		build_protos.sh
requirements-algorithms.txt		requirements-algorithms.txt
requirements-benchmarks.txt		requirements-benchmarks.txt
requirements-jax.txt		requirements-jax.txt
requirements-test.txt		requirements-test.txt
requirements-tf.txt		requirements-tf.txt
requirements.txt		requirements.txt
run_tests.sh		run_tests.sh
setup.py		setup.py

Repository files navigation

Open Source Vizier: Reliable and Flexible Black-Box Optimization.

Google AI Blog|Getting Started|Documentation|Installation|Citing and Highlights

What is Open Source (OSS) Vizier?

OSS Vizier is a Python-based service for black-box optimization and research, based onGoogle Vizier, one of the first hyperparameter tuning services designed to work at scale.

OSS Vizier's distributed client-server system. Animation by Tom Small.

Getting Started

As a basic example for users, below shows how to tune a simple objective using all flat search space types:

fromvizier.serviceimportclientsfromvizier.serviceimportpyvizierasvz# Objective function to maximize.defevaluate(w:float,x:int,y:float,z:str)->float:returnw**2-y**2+x*ord(z)# Algorithm, search space, and metrics.study_config=vz.StudyConfig(algorithm='DEFAULT')study_config.search_space.root.add_float_param('w',0.0,5.0)study_config.search_space.root.add_int_param('x',-2,2)study_config.search_space.root.add_discrete_param('y', [0.3,7.2])study_config.search_space.root.add_categorical_param('z', ['a','g','k'])study_config.metric_information.append(vz.MetricInformation('metric_name',goal=vz.ObjectiveMetricGoal.MAXIMIZE))# Setup client and begin optimization. Vizier Service will be implicitly created.study=clients.Study.from_study_config(study_config,owner='my_name',study_id='example')foriinrange(10):suggestions=study.suggest(count=2)forsuggestioninsuggestions:params=suggestion.parametersobjective=evaluate(params['w'],params['x'],params['y'],params['z'])suggestion.complete(vz.Measurement({'metric_name':objective}))

Documentation

OSS Vizier's interface consists ofthree main APIs:

User API: Allows a user to optimize their blackbox objective and optionally setup a server for distributed multi-client settings.
Developer API: Defines abstractions and utilities for implementing new optimization algorithms for research and to be hosted in the service.
Benchmarking API: A wide collection of objective functions and methods to benchmark and compare algorithms.

Additionally, it containsadvanced API for:

Tensorflow Probability: For writing Bayesian Optimization algorithms using Tensorflow Probability and Flax.
PyGlove: For large-scale evolutionary experimentation and program search using OSS Vizier as a distributed backend.

Please see OSS Vizier'sReadTheDocs documentation for detailed information.

Installation

Quick start: For tuning objectives using our state-of-the-art JAX-based Bayesian Optimizer, run:

pip install google-vizier[jax]

Advanced Installation

Minimal installation: To install only the core service and client APIs fromrequirements.txt, run:

pip install google-vizier

Full installation: To support all algorithms and benchmarks, run:

pip install google-vizier[all]

Specific installation: If you only need a specific part "X" of OSS Vizier, run:

pip install google-vizier[X]

which installs add-ons fromrequirements-X.txt. Possible options:

requirements-jax.txt: Jax libraries shared by both algorithms and benchmarks.
requirements-tf.txt: Tensorflow libraries used by benchmarks.
requirements-algorithms.txt: Additional repositories (e.g. EvoJAX) for algorithms.
requirements-benchmarks.txt: Additional repositories (e.g. NASBENCH-201) for benchmarks.
requirements-test.txt: Libraries needed for testing code.

Developer installation: To install up to the latest commit, run:

pip install google-vizier-dev[X]

Check if all unit tests work by runningrun_tests.sh after a full installation. OSS Vizier requires Python 3.10+, while client-only packages require Python 3.8+.

Citing and Highlights

Citing Vizier: Please consider citing the appropriate paper(s):Algorithm,OSS Package, andGoogle System if you found any of them useful.

Highlights: We tracknotable users andmedia attention - let us know if OSS Vizier was helpful for your work.

Thanks!

@article{gaussian_process_bandit,author       ={Xingyou Song and                  Qiuyi Zhang and                  Chansoo Lee and                  Emily Fertig and                  Tzu-Kuo Huang and                  Lior Belenki and                  Greg Kochanski and                  Setareh Ariafar and                  Srinivas Vasudevan and                  Sagi Perel and                  Daniel Golovin},title        ={The Vizier Gaussian Process Bandit Algorithm},journal      ={Google DeepMind Technical Report},year         ={2024},eprinttype    ={arXiv},eprint       ={2408.11527},}@inproceedings{oss_vizier,author    ={Xingyou Song and               Sagi Perel and               Chansoo Lee and               Greg Kochanski and               Daniel Golovin},title     ={Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Black-box Optimization},booktitle ={Automated Machine Learning Conference, Systems Track (AutoML-Conf Systems)},year      ={2022},}@inproceedings{google_vizier,author    ={Daniel Golovin and               Benjamin Solnik and               Subhodeep Moitra and               Greg Kochanski and               John Karro and               D. Sculley},title     ={Google Vizier: {A} Service for Black-Box Optimization},booktitle ={Proceedings of the 23rd {ACM} {SIGKDD} International Conference on               Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13               - 17, 2017},pages     ={1487--1495},publisher ={{ACM}},year      ={2017},url       ={https://doi.org/10.1145/3097983.3098043},doi       ={10.1145/3097983.3098043},}