- Notifications
You must be signed in to change notification settings - Fork100
Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.
License
google/vizier
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Google AI Blog|Getting Started|Documentation|Installation|Citing and Highlights
OSS Vizier is a Python-based service for black-box optimization and research, based onGoogle Vizier, one of the first hyperparameter tuning services designed to work at scale.
OSS Vizier's distributed client-server system. Animation by Tom Small.
As a basic example for users, below shows how to tune a simple objective using all flat search space types:
fromvizier.serviceimportclientsfromvizier.serviceimportpyvizierasvz# Objective function to maximize.defevaluate(w:float,x:int,y:float,z:str)->float:returnw**2-y**2+x*ord(z)# Algorithm, search space, and metrics.study_config=vz.StudyConfig(algorithm='DEFAULT')study_config.search_space.root.add_float_param('w',0.0,5.0)study_config.search_space.root.add_int_param('x',-2,2)study_config.search_space.root.add_discrete_param('y', [0.3,7.2])study_config.search_space.root.add_categorical_param('z', ['a','g','k'])study_config.metric_information.append(vz.MetricInformation('metric_name',goal=vz.ObjectiveMetricGoal.MAXIMIZE))# Setup client and begin optimization. Vizier Service will be implicitly created.study=clients.Study.from_study_config(study_config,owner='my_name',study_id='example')foriinrange(10):suggestions=study.suggest(count=2)forsuggestioninsuggestions:params=suggestion.parametersobjective=evaluate(params['w'],params['x'],params['y'],params['z'])suggestion.complete(vz.Measurement({'metric_name':objective}))
OSS Vizier's interface consists ofthree main APIs:
- User API: Allows a user to optimize their blackbox objective and optionally setup a server for distributed multi-client settings.
- Developer API: Defines abstractions and utilities for implementing new optimization algorithms for research and to be hosted in the service.
- Benchmarking API: A wide collection of objective functions and methods to benchmark and compare algorithms.
Additionally, it containsadvanced API for:
- Tensorflow Probability: For writing Bayesian Optimization algorithms using Tensorflow Probability and Flax.
- PyGlove: For large-scale evolutionary experimentation and program search using OSS Vizier as a distributed backend.
Please see OSS Vizier'sReadTheDocs documentation for detailed information.
Quick start: For tuning objectives using our state-of-the-art JAX-based Bayesian Optimizer, run:
pip install google-vizier[jax]
Minimal installation: To install only the core service and client APIs fromrequirements.txt, run:
pip install google-vizier
Full installation: To support all algorithms and benchmarks, run:
pip install google-vizier[all]
Specific installation: If you only need a specific part "X" of OSS Vizier, run:
pip install google-vizier[X]
which installs add-ons fromrequirements-X.txt. Possible options:
requirements-jax.txt: Jax libraries shared by both algorithms and benchmarks.requirements-tf.txt: Tensorflow libraries used by benchmarks.requirements-algorithms.txt: Additional repositories (e.g. EvoJAX) for algorithms.requirements-benchmarks.txt: Additional repositories (e.g. NASBENCH-201) for benchmarks.requirements-test.txt: Libraries needed for testing code.
Developer installation: To install up to the latest commit, run:
pip install google-vizier-dev[X]
Check if all unit tests work by runningrun_tests.sh after a full installation. OSS Vizier requires Python 3.10+, while client-only packages require Python 3.8+.
Citing Vizier: Please consider citing the appropriate paper(s):Algorithm,OSS Package, andGoogle System if you found any of them useful.
Highlights: We tracknotable users andmedia attention - let us know if OSS Vizier was helpful for your work.
Thanks!
@article{gaussian_process_bandit,author ={Xingyou Song and Qiuyi Zhang and Chansoo Lee and Emily Fertig and Tzu-Kuo Huang and Lior Belenki and Greg Kochanski and Setareh Ariafar and Srinivas Vasudevan and Sagi Perel and Daniel Golovin},title ={The Vizier Gaussian Process Bandit Algorithm},journal ={Google DeepMind Technical Report},year ={2024},eprinttype ={arXiv},eprint ={2408.11527},}@inproceedings{oss_vizier,author ={Xingyou Song and Sagi Perel and Chansoo Lee and Greg Kochanski and Daniel Golovin},title ={Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Black-box Optimization},booktitle ={Automated Machine Learning Conference, Systems Track (AutoML-Conf Systems)},year ={2022},}@inproceedings{google_vizier,author ={Daniel Golovin and Benjamin Solnik and Subhodeep Moitra and Greg Kochanski and John Karro and D. Sculley},title ={Google Vizier: {A} Service for Black-Box Optimization},booktitle ={Proceedings of the 23rd {ACM} {SIGKDD} International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017},pages ={1487--1495},publisher ={{ACM}},year ={2017},url ={https://doi.org/10.1145/3097983.3098043},doi ={10.1145/3097983.3098043},}
About
Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.