HADOOP-19523. Upgrade to hadoop-thirdparty 1.4.0

[steveloughran]

Description of PR

Upgrade to the newly released 1.4.0 library

How was this patch tested?

Been running it locally for a week

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[cnauroth]

+1, pending resolution of a minor typo and successful pre-submit run.

[cnauroth]

Typo?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 50s		Maven dependency ordering for branch
+1 💚	mvninstall	18m 36s		trunk passed
+1 💚	compile	8m 17s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	7m 18s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	10m 31s		trunk passed
+1 💚	javadoc	5m 20s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	5m 1s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	29m 41s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	17m 35s		the patch passed
+1 💚	compile	8m 8s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	8m 8s		the patch passed
+1 💚	compile	7m 18s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	7m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	11m 35s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	5m 24s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	4m 59s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	31m 2s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	523m 38s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		690m 57s

Reason	Tests
Failed junit tests	hadoop.yarn.client.api.impl.TestYarnClient
	hadoop.yarn.client.api.impl.TestAMRMClientPlacementConstraints
	hadoop.yarn.client.api.impl.TestAMRMClient
	hadoop.yarn.client.api.impl.TestAMRMProxy
	hadoop.yarn.client.api.impl.TestNMClient
	hadoop.yarn.client.api.impl.TestOpportunisticContainerAllocationE2E

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/1/artifact/out/Dockerfile
GITHUB PR	#7560
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux cb9454d4fedf 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 0052185
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/1/testReport/
Max. process+thread count	3503 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[slfan1989]

These YARN unit test errors are not related to the upgrade to hadoop-thirdparty 1.4.0. I will continue to follow up.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 14s		Maven dependency ordering for branch
+1 💚	mvninstall	36m 34s		trunk passed
+1 💚	compile	17m 34s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	15m 20s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	21m 43s		trunk passed
+1 💚	javadoc	10m 14s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 40s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	56m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	35m 25s		the patch passed
+1 💚	compile	16m 46s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	16m 46s		the patch passed
+1 💚	compile	15m 8s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	15m 8s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 30s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	10m 10s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 56s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	57m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	506m 47s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		811m 29s

Reason	Tests
Unreaped Processes	root:2
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes
	hadoop.yarn.client.api.impl.TestOpportunisticContainerAllocationE2E
	hadoop.yarn.client.api.impl.TestAMRMProxy
	hadoop.yarn.client.api.impl.TestYarnClient
	hadoop.yarn.client.api.impl.TestNMClient
	hadoop.yarn.client.api.impl.TestAMRMClientPlacementConstraints
	hadoop.yarn.client.api.impl.TestAMRMClient

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/2/artifact/out/Dockerfile
GITHUB PR	#7560
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux c34323e1d98c 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 0052185
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Unreaped Processes Log	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/2/artifact/out/patch-unit-root-reaper.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/2/testReport/
Max. process+thread count	3134 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7560/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[steveloughran]

@cnauroth you happy with this?

[cnauroth]

@steveloughran , I think I'm still seeing a typo in the license file:

commons-beanutils:commons-beanutils:1.9.4j

+1 after deleting that trailing 'j'. (No need for another round of code review.)