[bug] snapshot_meta_column_names causing compilation error on second run

[graciegoheen]

Hi @graciegoheen not sure whether i understand it correctly. I have created a snapshot from scratch with adjusted columns using a dictionary:

    snapshot_meta_column_names = {
        "dbt_valid_from": "from_date",
        "dbt_valid_to": "to_date",
        "dbt_scd_id": "scd_key",
        "dbt_updated_at": "dbt_updated_at",
        "dbt_is_deleted": "active_flag"
    }

Snapshot has been created with that adjusted dbt column names.

Issue is if i am trying to run snapshot second time. Query is not even sent to databrick cluster- it fails during compilation (Compilation Error in snapshot snapshot__animals_scd2_1 (snapshots/snapshot__animals_scd2_1.sql)

Can you advice me how to avoid that?
Thank you

Originally posted by @czasoprzestrzenny in #10185 (comment)

[peterallenwebb]

@graciegoheen It looks like the databricks adapter has a customized snapshot implementation that needs to be updated to support custom meta-column names.

At a minimum, this line needs to be updated to pass the desired meta column name dictionary. Other supporting changes will probably also be needed.

[peterallenwebb]

Root cause is that dbt-databricks does not yet support custom meta column names. See: databricks/dbt-databricks#887