Code

[CI] Do not upload dockerbuild#15

Code

Open

EnricoMi wants to merge 47 commits intomasterG-Research/spark:masterfrom ci-do-not-upload-dockerbuildG-Research/spark:ci-do-not-upload-dockerbuildCopy head branch name to clipboard

+2,128-611

Commits

Commits on Feb 28, 2025

• [SPARK-51316][PYTHON][FOLLOW-UP] Revert unrelated changes and mark mapInPandas/mapInArrow batched in byte size

  Show description for 5b45671
  HyukjinKwon

  committed
  5b45671

  Copy full SHA for 5b45671

• [SPARK-51351][SS] Do not materialize the output in Python worker for TWS

  Show description for 496fe7a
  HeartSaVioR

  committed
  496fe7a

  Copy full SHA for 496fe7a

Commits on Mar 1, 2025

• [SPARK-49960][SQL] Custom ExpressionEncoder support and TransformingEncoder fixes

  Show description for 50a328b
  

  chris-twiner

  authored and

  hvanhovell

  committed
  50a328b

  Copy full SHA for 50a328b

Commits on Mar 2, 2025

• [MINOR][DOCS] Fix code comments for Executor

  Show description for a33f232
  

  tomscut

  authored and

  HyukjinKwon

  committed
  a33f232

  Copy full SHA for a33f232

• [MINOR][TESTS][CONNECT] Fix the teardown function of test_connect_function.py

  Show description for d88db28
  

  sfc-gh-dyadav

  authored and

  HyukjinKwon

  committed
  d88db28

  Copy full SHA for d88db28

Commits on Mar 3, 2025

• [SPARK-51354][K8S][TEST][DOC] Fix sbt K8s integration test arg javaImageTag

  Show description for 4df5abc
  

  pan3793

  authored and

  dongjoon-hyun

  committed
  4df5abc

  Copy full SHA for 4df5abc

• [SPARK-51353][INFRA][BUILD] Retry dyn/closer.lua for mvn before falling back to archive.a.o

  Show description for ec03f1e
  yaooqinn

  committed
  ec03f1e

  Copy full SHA for ec03f1e

• Revert "[SPARK-51309][BUILD] Upgrade rocksdbjni to 9.10.0"

  Show description for e66e20a
  LuciferYang

  committed
  e66e20a

  Copy full SHA for e66e20a

• [SPARK-49489][SQL][HIVE] HMS client respects hive.thrift.client.maxmessage.size

  Show description for 2ea5621
  

  pan3793

  authored and

  LuciferYang

  committed
  2ea5621

  Copy full SHA for 2ea5621

• Revert "[SPARK-51353][INFRA][BUILD] Retry dyn/closer.lua for mvn before falling back to archive.a.o"

  Show description for ec44e29
  yaooqinn

  committed
  ec44e29

  Copy full SHA for ec44e29

• [SPARK-51369][SQL] Fix FOR cursor variable to work with mixed case names

  Show description for b4896c0
  

  davidm-db

  authored and

  cloud-fan

  committed
  b4896c0

  Copy full SHA for b4896c0

• [SPARK-51370] Code clarity improvements for INSERT INTO parsing codes in AstBuilder.scala

  Show description for 1ad7f31
  

  longvu-db

  authored and

  gengliangwang

  committed
  1ad7f31

  Copy full SHA for 1ad7f31

• [SPARK-51229][BUILD][CONNECT] Fix dependency:analyze goal on connect common

  Show description for 7653b40
  

  vrozov

  authored and

  hvanhovell

  committed
  7653b40

  Copy full SHA for 7653b40

• [SPARK-51374][CORE] Switch to Using java.util.Map in Logging APIs

  Show description for 6185916
  gengliangwang

  committed
  6185916

  Copy full SHA for 6185916

• [SPARK-51355][DOCS] Add a brief explanation for Spark Connect at PySpark Overview page

  Show description for 8299a14
  

  HyukjinKwon

  authored and

  dongjoon-hyun

  committed
  8299a14

  Copy full SHA for 8299a14

• [SPARK-50578][PYTHON][SS][FOLLOW-UP] Add the Python new package into setup.py

  Show description for 4b2ae48
  

  HyukjinKwon

  authored and

  dongjoon-hyun

  committed
  4b2ae48

  Copy full SHA for 4b2ae48

Commits on Mar 4, 2025

• [SPARK-51363][SQL] Desc As JSON` clustering column names

  Show description for 86c2140
  

  asl3

  authored and

  cloud-fan

  committed
  86c2140

  Copy full SHA for 86c2140

• [SPARK-51362][SQL] Change toJSON to use NextIterator API to eliminate adjacent record dependency

  Show description for cf1c3bf
  

  eason-yuchen-liu

  authored and

  HeartSaVioR

  committed
  cf1c3bf

  Copy full SHA for cf1c3bf

• [SPARK-51375][SQL][CONNECT] Suppress SparkConnect(Execution|Session)Manager.periodicMaintenance log messages

  Show description for d5a1754
  dongjoon-hyun

  committed
  d5a1754

  Copy full SHA for d5a1754

• [SPARK-51381][SQL][CONNECT] Show Session ID in Spark Connect Session page

  Show description for 4f97dcb
  dongjoon-hyun

  committed
  4f97dcb

  Copy full SHA for 4f97dcb

• [SPARK-50855][SS][CONNECT] Spark Connect Support for TransformWithState In Scala

  Show description for 3820ae0
  

  jingz-db

  authored and

  HeartSaVioR

  committed
  3820ae0

  Copy full SHA for 3820ae0

• [SPARK-45641][DOCS][FOLLOWUP] Add Started At description for web-ui.md

  Show description for 0ce096d
  

  tomscut

  authored and

  yaooqinn

  committed
  0ce096d

  Copy full SHA for 0ce096d

• [SPARK-51378][CORE] Apply JsonProtocol's accumulableExcludeList to ExecutorMetricsUpdate and TaskEndReason

  Show description for 62f7f14
  

  JoshRosen

  authored and

  HyukjinKwon

  committed
  62f7f14

  Copy full SHA for 62f7f14

• [SPARK-51368][SS][PYTHON][TEST] Disable Schema Evolution Suite in Connect TransformWithState for Python Suite

  Show description for 5756480
  

  jingz-db

  authored and

  HyukjinKwon

  committed
  5756480

  Copy full SHA for 5756480

• [SPARK-51373][SS] Removing extra copy for column family prefix from 'ReplyChangelog'

  Show description for c2f2be6
  

  ericm-db

  authored and

  HeartSaVioR

  committed
  c2f2be6

  Copy full SHA for c2f2be6

• [SPARK-51342][SQL] Add TimeType

  Show description for 14fff2a
  MaxGekk

  committed
  14fff2a

  Copy full SHA for 14fff2a

• [SPARK-50615][FOLLOWUP][SQL] Avoid dropping metadata in the push rule

  Show description for 8f2c066
  

  chenhao-db

  authored and

  cloud-fan

  committed
  8f2c066

  Copy full SHA for 8f2c066

• [SPARK-51029][INFRA][FOLLOWUP] Remove LICENSE and NOTICE for unused/duplicated hive deps

  Show description for 921ef95
  

  yaooqinn

  authored and

  dongjoon-hyun

  committed
  921ef95

  Copy full SHA for 921ef95

• [SPARK-51387][BUILD] Upgrade Netty to 4.1.119.Final

  Show description for 3d949af
  dongjoon-hyun

  committed
  3d949af

  Copy full SHA for 3d949af

• [SPARK-51367][BUILD] Upgrade slf4j to 2.0.17

  Show description for 8f0ac3b
  

  LuciferYang

  authored and

  dongjoon-hyun

  committed
  8f0ac3b

  Copy full SHA for 8f0ac3b

• [MINOR][CORE] Remove unused private[executor] var httpUrlConnectionTimeoutMillis from ExecutorClassLoader

  Show description for fd1f774
  

  LuciferYang

  authored and

  dongjoon-hyun

  committed
  fd1f774

  Copy full SHA for fd1f774

• [SPARK-51072][CORE] CallerContext to set Hadoop cloud audit context

  Show description for 229be37
  

  steveloughran

  authored and

  dongjoon-hyun

  committed
  229be37

  Copy full SHA for 229be37

• [SPARK-51307][SQL] locationUri in CatalogStorageFormat shall be decoded for display

  Show description for eb71443
  

  yaooqinn

  authored and

  dongjoon-hyun

  committed
  eb71443

  Copy full SHA for eb71443

• [SPARK-50855][CONNECT][TESTS][FOLLOWUP] Refactor TransformWithStateConnectSuite to run DROP TABLE IF EXISTS my_sink in beforeAll/afterEach

  Show description for 9cfc628
  

  LuciferYang

  authored and

  dongjoon-hyun

  committed
  9cfc628

  Copy full SHA for 9cfc628

Commits on Mar 5, 2025

• [SPARK-51314][DOCS][PS] Add proper note for distributed-sequence about indeterministic case

  Show description for f71291b
  itholic

  committed
  f71291b

  Copy full SHA for f71291b

• [SPARK-51383][PYTHON][CONNECT] Avoid making RPC calls if clients are already known as stopped

  Show description for 31cf3c3
  HyukjinKwon

  committed
  31cf3c3

  Copy full SHA for 31cf3c3

• Revert "[SPARK-51276][PYTHON] Enable spark.sql.execution.arrow.pyspark.enabled by default"

  Show description for 449d3cb
  HyukjinKwon

  committed
  449d3cb

  Copy full SHA for 449d3cb

• Revert "[SPARK-48516][PYTHON][FOLLOW-UP] Add a note in migration guide about Arrow-optimized Python UDF enabled by default"

  Show description for 1098697
  HyukjinKwon

  committed
  1098697

  Copy full SHA for 1098697

• Revert "[SPARK-48516][PYTHON][CONNECT] Turn on Arrow optimization for Python UDFs by default"

  Show description for eb48553
  HyukjinKwon

  committed
  eb48553

  Copy full SHA for eb48553

• [SPARK-51390][INFRA] Add more dependencies in LICENSE-binary for Spark 4.0.0 release

  Show description for 4be3aae
  

  cnauroth

  authored and

  yaooqinn

  committed
  4be3aae

  Copy full SHA for 4be3aae

• [MINOR] Remove unused imports

  Show description for 12f2967
  HyukjinKwon

  committed
  12f2967

  Copy full SHA for 12f2967

• [SPARK-51277][PYTHON][FOLLOW-UP] Remove 0-arg check in Spark Connect side as well

  Show description for 7fca7ae
  HyukjinKwon

  committed
  7fca7ae

  Copy full SHA for 7fca7ae

• [SPARK-51357][SQL] Preserve plan change logging level for views

  Show description for a6600af
  

  vladimirg-db

  authored and

  cloud-fan

  committed
  a6600af

  Copy full SHA for a6600af

• [SPARK-51385][SQL] Normalize out projection added in DeduplicateRelations for union child output deduplication

  Show description for c005a37
  

  vladimirg-db

  authored and

  cloud-fan

  committed
  c005a37

  Copy full SHA for c005a37

• [SPARK-51394][ML] Optimize out the additional shuffle in stats tests

  Show description for 26545d0
  zhengruifeng

  committed
  26545d0

  Copy full SHA for 26545d0

• [SPARK-51388][SQL] Improve SQL fragment propagation in to_timestamp and UNION

  Show description for 4b4bdcf
  

  vladimirg-db

  authored and

  cloud-fan

  committed
  4b4bdcf

  Copy full SHA for 4b4bdcf

• CI: Do not upload docker build file

  EnricoMi

  committed
  bde9fa9

  Copy full SHA for bde9fa9