Description

Proposal to add support for Trino installations using dynamic catalogs to periodically refresh catalogs from the configuredCatalogStore. It allows Trino to periodically list the catalogs available in theCatalogStore and add/remove/update the available catalogs accordingly.

The objective of this proposed change is to allow administrators to externally manage dynamic catalogs and have Trino periodically sync to their configuration, rather than needing to push catalog changes to Trino throughCREATE/DROP CATALOG statements.

Catalog refreshes are disabled by default and can be enabled through thecatalog.refresh.enabled configuration property. The refresh interval defaults to 60 seconds and can be configured through thecatalog.refresh.refresh-interval

Example configuration:

catalog.management=dynamiccatalog.refresh.enabled=truecatalog.refresh.refresh-interval=60scatalog.store=my_catalog_store

I would love to hear feedback on whether this change is welcome and on the approach itself. I would also appreciate any recommendations on how to effectively test the change.

Additional context and related issues

Catalog refreshes are implemented in theCoordinatorDynamicCatalogManager::refreshCatalogs() method. The approach is to list the catalogs from the catalog store, compare them to the currently active catalogs and determine the catalogs that were added, removed or updated. With that information, update the catalog state accordingly.

Catalog refreshing is triggered by theCatalogRefreshTask, which works very similarly to the existingCatalogPruneTask.

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: