You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SPARK-49098 introduced a SQL syntax to allow users to set table options on DSv2 write cases, but unfortunately, the options set by SQL are not propagated correctly to the underlying DSv2 WriteBuilder
We ideally should add tests for the new SQL syntax with options to make sure our writers handle it properly.
I think the only datasource.v2 writers we have is the delta lake one and I think we handle options there right but filing to double check as well.
The text was updated successfully, but these errors were encountered:
Describe the bug
[SPARK-50286][SQL] Correctly propagate SQL options to WriteBuilder
apache/spark@976f8875edd
SPARK-49098 introduced a SQL syntax to allow users to set table options on DSv2 write cases, but unfortunately, the options set by SQL are not propagated correctly to the underlying DSv2
WriteBuilder
We ideally should add tests for the new SQL syntax with options to make sure our writers handle it properly.
I think the only datasource.v2 writers we have is the delta lake one and I think we handle options there right but filing to double check as well.
The text was updated successfully, but these errors were encountered: