Mongodb to Postgres produces weird column names

[briankariuki]

dlt version

1.3.0

Describe the problem

As you can see in the image below the column should be named __is_new_cased_reported_from_initial_area

Another example. Note both investigation forms share the same schema in mongodb but there's an issue when mapped to postgres

Expected behavior

Proper column names should be in the destination table in postgres

Steps to reproduce

This uses the mongodb to postgres tutorial. My mongodb collection has alot nested entries most of which go two levels deep. Happy to share a dump for someone to reproduce this

Operating system

macOS

Runtime environment

Local

Python version

3.11

dlt data source

Mongodb

dlt destination

Postgres

Other deployment details

I'm using dagster to trigger the runs

Additional information

No response

[rudolfix]

@briankariuki unfortunately postgres has 64 characters identifier table limit. we are inserting a short tag when we shorten the name to fit into that limit. I'm afraid it cannot be avoided (without recompiling postgres with higher limit)

you could also condifer adding max table nesting level option to limit number of unnested tables. then maybe you do not generate such long names