This month: a roundup of the summer’s Kedro news, some release updates, and our top picks from recent articles.

Kedro team news

Over the last few months, we’ve been happy to welcome some new team members to the Kedro and Kedro-Viz teams, who have also joined ourTechnical Steering Committee. WelcomeDmitry Sorokin, Jitendra Gundaniya, Laura Couto, Ravi Kumar Pilla, andVladimir Nikolic!

We are also pleased to announce a Kedro baby, delivered safely by one of the team, at the end of July!

Contributors news

We reworked the Kedro contributors guide in August, and moved it to theKedro wiki. There are loads of different ways to contribute to Kedro and if you want to get involved, we encourage you to look at thetable that introduces the Kedro contributor guide.

If you spot an article, podcast or video that discusses Kedro, you can also contribute by adding it to the “Awesome Kedro” repository, or letting us know onSlack.

There have been some amazing contributions in recent weeks, including thekedro-vineyard plugin for efficient intermediate sharing in Kedro pipelines,kedro-graphql for serving Kedro projects as a GraphQL API, andkedro-pandera to bring data validation to your Kedro projects.

Release news

August 2023 saw a set ofreleases to introduce Python 3.11 support across Kedro, Kedro-Viz and Kedro datasets.

Kedro version 0.18.13 included these major features and improvements:

• Added support for Python 3.11.

• Added newOmegaConfigLoader features: registering of custom resolvers throughCONFIG_LOADER_ARGS and support for global variables.

• Addedkedro catalog resolve CLI command that resolves dataset factories in the catalog with any explicit entries in the project pipeline.

• Simplified theconf folder structure for modular pipelines and updated kedro pipeline create and kedro catalog create accordingly.

• Made various updates to the Kedro project template and Kedro starters: use of OmegaConfigLoader, transition fromsetup.py topyproject.toml, and updated for the simplifiedconf structure.

Kedro Viz version 6.5 added support for Python 3.11, whileKedro Viz version 6.4 added two new features: feature hint cards to highlight key features of Kedro Viz and support for displaying dataset statistics in the metadata panel for further investigation.

Kedro Datasets version 1.7.0 addedpolars.GenericDataSet, a dataset backed bypolars, a lightning fast dataframe package built entirely using Rust.Kedro Datasets version 1.6.0 added support for Python 3.11.

Recently on the Kedro blog

In the last few weeks we’ve published the following on the Kedro blog:

• How to integrate Kedro and Databricks Connect

• How to use Databricks managed Delta tables in a Kedro project

• A new Kedro dataset for Spark Structured Streaming

• Collaborative experiment tracking in Kedro-Viz

• Get up to speed: How to build a custom Kedro runner

We’re always looking for collaborators to write about their experiences using Kedro, particularly if you’re working with Kedro datasets or converting an existing project to use Kedro. Get in touch with us on ourSlack workspace to tell us your story.

What we’ve learned

We really enjoyed reading more on Medium about theKedro Vineyard plugin, which is a cloud-native data manager, for data sharing using memory in data science pipelines on Kubernetes.

Quix published an interesting article called “Bridging the gap between data scientists and engineers in machine learning workflows” which is something we regularly discuss within the team.

We found asuper-interesting project about font recognition that uses Kedro.

And finally, we enjoyed reading more aboutdata streaming with Kedro over on the QuantumBlack Medium channel.

That’s it for this edition!