Polars: Blazingly fast DataFrames in Rust, Python, Node.js, R, and SQL

Python

>>>importpolarsaspl>>>df=pl.DataFrame(...     {..."A": [1,2,3,4,5],..."fruits": ["banana","banana","apple","apple","banana"],..."B": [5,4,3,2,1],..."cars": ["beetle","audi","beetle","beetle","beetle"],...     }... )# embarrassingly parallel execution & very expressive query language>>>df.sort("fruits").select(..."fruits",..."cars",...pl.lit("fruits").alias("literal_string_fruits"),...pl.col("B").filter(pl.col("cars")=="beetle").sum(),...pl.col("A").filter(pl.col("B")>2).sum().over("cars").alias("sum_A_by_cars"),...pl.col("A").sum().over("fruits").alias("sum_A_by_fruits"),...pl.col("A").reverse().over("fruits").alias("rev_A_by_fruits"),...pl.col("A").sort_by("B").over("fruits").alias("sort_A_by_B_by_fruits"),... )shape: (5,8)┌──────────┬──────────┬──────────────┬─────┬─────────────┬─────────────┬─────────────┬─────────────┐│fruits   ┆cars     ┆literal_stri ┆B   ┆sum_A_by_ca ┆sum_A_by_fr ┆rev_A_by_fr ┆sort_A_by_B ││---      ┆---      ┆ng_fruits    ┆--- ┆rs          ┆uits        ┆uits        ┆_by_fruits  ││str      ┆str      ┆---          ┆i64 ┆---         ┆---         ┆---         ┆---         ││          ┆          ┆str          ┆     ┆i64         ┆i64         ┆i64         ┆i64         │╞══════════╪══════════╪══════════════╪═════╪═════════════╪═════════════╪═════════════╪═════════════╡│"apple"  ┆"beetle" ┆"fruits"     ┆11  ┆4           ┆7           ┆4           ┆4           ││"apple"  ┆"beetle" ┆"fruits"     ┆11  ┆4           ┆7           ┆3           ┆3           ││"banana" ┆"beetle" ┆"fruits"     ┆11  ┆4           ┆8           ┆5           ┆5           ││"banana" ┆"audi"   ┆"fruits"     ┆11  ┆2           ┆8           ┆2           ┆2           ││"banana" ┆"beetle" ┆"fruits"     ┆11  ┆4           ┆8           ┆1           ┆1           │└──────────┴──────────┴──────────────┴─────┴─────────────┴─────────────┴─────────────┴─────────────┘

SQL

>>>df=pl.scan_csv("docs/assets/data/iris.csv")>>>## OPTION 1>>># run SQL queries on frame-level>>>df.sql("""...SELECT species,...  AVG(sepal_length) AS avg_sepal_length...FROM self...GROUP BY species...""").collect()shape: (3,2)┌────────────┬──────────────────┐│species    ┆avg_sepal_length ││---        ┆---              ││str        ┆f64              │╞════════════╪══════════════════╡│Virginica  ┆6.588            ││Versicolor ┆5.936            ││Setosa     ┆5.006            │└────────────┴──────────────────┘>>>## OPTION 2>>># use pl.sql() to operate on the global context>>>df2=pl.LazyFrame({..."species": ["Setosa","Versicolor","Virginica"],..."blooming_season": ["Spring","Summer","Fall"]...})>>>pl.sql("""... SELECT df.species,...     AVG(df.sepal_length) AS avg_sepal_length,...     df2.blooming_season... FROM df... LEFT JOIN df2 ON df.species = df2.species... GROUP BY df.species, df2.blooming_season... """).collect()

# run an inline SQL query> polars -c"SELECT species, AVG(sepal_length) AS avg_sepal_length, AVG(sepal_width) AS avg_sepal_width FROM read_csv('docs/assets/data/iris.csv') GROUP BY species;"# run interactively> polarsPolars CLI v0.3.0Type .helpfor help.> SELECT species, AVG(sepal_length) AS avg_sepal_length, AVG(sepal_width) AS avg_sepal_width FROM read_csv('docs/assets/data/iris.csv') GROUP BY species;

Performance 🚀🚀

Blazingly fast

Lightweight

Handles larger-than-RAM data

Setup

Python

pip install polars

pip install'polars[all]'

pip install'polars[numpy,pandas,pyarrow]'

pl.show_versions()

Rust

polars = {git ="https://github.com/pola-rs/polars",rev ="<optional git tag>" }

Contributing

Python: compile Polars from source

Using custom Rust functions in Python

Going big...

Legacy

Sponsors