Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partition by should not quote the items #201

Open
Conq1 opened this issue Jul 11, 2023 · 5 comments · May be fixed by #246
Open

Partition by should not quote the items #201

Conq1 opened this issue Jul 11, 2023 · 5 comments · May be fixed by #246
Labels
enhancement New feature or request

Comments

@Conq1
Copy link

Conq1 commented Jul 11, 2023

Describe the enhancement requested

I would like to use the features of Iceberg partition transformation, but when adding them to "partition_by" in config they get quotes surrounding each item, meaning the create table will fail.

Justification for this enhancement

Partition transformation is a powerful Iceberg feature.

@Conq1
Copy link
Author

Conq1 commented Jul 11, 2023

For now I have created my own macro of config_cols.
Might be a better solution as config_cols is used multible other places.

New version without quoting:

{% macro config_cols(label, default_cols=none) %}
  {%- set cols = config.get(label | replace(" ", "_"), validator=validation.any[list, basestring]) or default_cols -%}
  {%- if cols is not none %}
    {%- if cols is string -%}
      {%- set cols = [cols] -%}
    {%- endif -%}
    {{ label }} ( {{ cols | join(', ') }}  )
  {%- endif %}
{%- endmacro -%}

@fabrice-etanchaud
Copy link

Hi @Conq1 , dremio usually accepts double quotes around object names (I just tested a partitioned table creation with partition by ("mycol")), are you sure the error is not coming from another problem, maybe a misnamed column ?

@Conq1
Copy link
Author

Conq1 commented Jul 13, 2023

Yes @fabrice-etanchaud . Problem is when you want to use partition-transformation like truncate(1, "mycol"):
https://docs.dremio.com/software/sql-reference/sql-commands/apache-iceberg-tables/apache-iceberg-create/
So quick fix for me was to remove the quotes, but a better fix maybe some other syntax to indicate when you want to use partition-transformation.

@fabrice-etanchaud
Copy link

Thank you @Conq1 , I understand. You are right, the solution may be to remove the systematic double quoting, and let the user decide.

@wabu
Copy link

wabu commented Nov 25, 2024

I'm facing the same issue. Any progress on implementing this?

@wabu wabu linked a pull request Nov 26, 2024 that will close this issue
1 task
@bcmeireles bcmeireles linked a pull request Dec 4, 2024 that will close this issue
1 task
@bcmeireles bcmeireles added the enhancement New feature or request label Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

Successfully merging a pull request may close this issue.

4 participants