Repository files navigation

Metaflow is a human-centric framework designed to help scientists and engineersbuild and manage real-life AI and ML systems. Serving teams of all sizes and scale, Metaflow streamlines the entire development lifecycle—from rapid prototyping in notebooks to reliable, maintainable production deployments—enabling teams to iterate quickly and deliver robust systems efficiently.

Originally developed atNetflix and now supported byOuterbounds, Metaflow is designed to boost the productivity for research and engineering teams working ona wide variety of projects, from classical statistics to state-of-the-art deep learning and foundation models. By unifying code, data, and compute at every stage, Metaflow ensures seamless, end-to-end management of real-world AI and ML systems.

Today, Metaflow powers thousands of AI and ML experiences across a diverse array of companies, large and small, including Amazon, Doordash, Dyson, Goldman Sachs, Ramp, andmany others. At Netflix alone, Metaflow supports over 3000 AI and ML projects, executes hundreds of millions of data-intensive high-performance compute jobs processing petabytes of data and manages tens of petabytes of models and artifacts for hundreds of users across its AI, ML, data science, and engineering teams.

Metaflow provides a simple and friendly pythonicAPI that covers foundational needs of AI and ML systems:

1. Rapid local prototyping,support for notebooks, and built-in support forexperiment tracking, versioning andvisualization.
2. Effortlessly scale horizontally and vertically in your cloud, utilizing both CPUs and GPUs, withfast data access for runningmassive embarrassingly parallel as well asgang-scheduled compute workloadsreliably andefficiently.
3. Easily manage dependencies anddeploy with one-click to highly available production orchestrators with built in support forreactive orchestration.

For full documentation, check out ourAPI Reference or see ourRelease Notes for the latest features and improvements.

Getting up and running is easy. If you don't know where to start,Metaflow sandbox will have you running and exploring in seconds.

To install Metaflow in your Python environment fromPyPI:

pip install metaflow

Alternatively, usingconda-forge:

conda install -c conda-forge metaflow

Once installed, a great way to get started is by following ourtutorial. It walks you through creating and running your first Metaflow flow step by step.

For more details on Metaflow’s features and best practices, check out:

• How Metaflow works
• Additional resources

If you need help, don’t hesitate to reach out on ourSlack community!

While you can get started with Metaflow easily on your laptop, the main benefits of Metaflow lie in its ability toscale out to external compute clustersand todeploy to production-grade workflow orchestrators. To benefit from these features, follow thisguide toconfigure Metaflow and the infrastructure behind it appropriately.

We'd love to hear from you. Join our communitySlack workspace!

We welcome contributions to Metaflow. Please see ourcontribution guide for more details.