Spark: 4.0 snapshot support #11583
Spark: 4.0 snapshot support #11583
Conversation
| import org.apache.spark.sql.catalyst.util.truncatedString | ||
| import org.apache.spark.sql.connector.iceberg.catalog.Procedure | ||
|
|
||
| case class IcebergCall(procedure: Procedure, args: Seq[Expression]) extends LeafCommand { |
There was a problem hiding this comment.
Spark 4 already supports CALL syntax, and also brings the Call logical/physics plan node, instead of renaming it to IcebergCall, why not migrate the Iceberg procedures to the upstream procedure API?
There was a problem hiding this comment.
This PR is already very big. I want to make minimal changes to ensure that the current stored procedure continues to function, which is why I've renamed this to IcebergCall to avoid conflicts with Spark's CALL syntax. I think it probably better to migrate the Iceberg procedures in a follow-up PR.
|
#11257 is no longer valid? |
|
@manuzhang The old PR has some conflicting files. If I rebase, it will also pick up changes for Spark 3.5, which I don't want. So, I opened a new PR. |
No description provided.