Add support for Catalog Providers

[philippemnoel]

What feature are you requesting?

For Iceberg and Delta Lake, we'd like to integrate with catalog metastores to facilitate fetching of credentials and other metadata. Unfortunately, this is primarily bottlenecked by DuckDB implementations. For instance, this is the PR for adding support for the AWS Glue catalog: duckdb/duckdb_iceberg#51

Once these are in, we can start adding support.

re: Delta Lake, their extension is more mature. It may be possible to do right away, but we haven't had any explicit request for Delta Lake yet.

Even though we are blocked on this, I'm opening up this tracking issue to collect interest as I've had many messages from interested users.

Why are you requesting this feature?

Integrate with where user's metadata is already located.

What is your proposed implementation for this feature?

Expose the catalog integration APIs from DuckDB

Full Name:

Philippe Noël

Affiliation:

ParadeDB