[I-Build-Tests] Run Linux tests on 'ubuntu-2404' image agent

[HannesWell]

Follow-up for #2515

I have already replayed the current linux-java17 configuration to use the ubunut image for the smaller ui test-suite:
https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-linux-x86_64-java17/21/

Once that has passed I'll create a follow-up run with the complete all suite to see if all tests pass, then I think this change is save enough to apply it to all other configs as well.

[HannesWell]

The ubuntu-2404 image is the latest Ubuntu based image, but instead of using ubuntu-latest I think it we should make it a deliberate choice to upgrade to a new image once available in I-build tests.
This also helps to get more details about the image if necessary at a later point in time.

The available images are https://github.com/eclipse-cbi/jiro-agents/?tab=readme-ov-file#jiro-agents

[HannesWell]

I have already replayed the current linux-java17 configuration to use the ubunut image for the smaller ui test-suite:
https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-linux-x86_64-java17/21/

The UI test-suite passed. Started a complete run at
https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-linux-x86_64-java17/22/

[akurtakov]

Please post a list of failing tests (if any) as some of them could be caused by differences like dbus services active or similar.

[HannesWell]

Please post a list of failing tests (if any) as some of them could be caused by differences like dbus services active or similar.

The following three test-failures occurred for Linux-Java17 on an Ubuntu based image, but they are the same in last night's I-build were the old CentOS kubernetes config was still applied (see https://download.eclipse.org/eclipse/downloads/drops4/I20241105-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-linux-x86_64-java17_linux.gtk.x86_64_17.html).
So I assume it's not a regression.

 java.lang.AssertionError: 
Test timed out.
Expected true for the below, but have:
WindoOpenFired:true
VisibilityShowed:false
ChildCompleted:false
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_OpenWindow_Progress_Listener_ValidateEventOrder(Test_org_eclipse_swt_browser_Browser.java:891)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.test.TracingSuite.runChild(TracingSuite.java:287)
	Suppressed: java.lang.Throwable: Screenshot written to /home/jenkins/agent/workspace/AutomatedTests/ep434I-unit-linux-x86_64-java17/workarea/I20241105-1800/eclipse-testing/results/ep434I-unit-linux-x86_64-java17_linux.gtk.x86_64_17/org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_OpenWindow_Progress_Listener_ValidateEventOrder[browser flags: 0].png
		at org.eclipse.test.Screenshots$ScreenshotOnFailure.failed(Screenshots.java:41)
		... 1 more

java.lang.AssertionError: test timed out. Child's visibility Window listener didn't trigger
Test log:
openWindowListener fired
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_VisibilityWindowListener_eventSize(Test_org_eclipse_swt_browser_Browser.java:1384)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.test.TracingSuite.runChild(TracingSuite.java:287)
	Suppressed: java.lang.Throwable: Screenshot written to /home/jenkins/agent/workspace/AutomatedTests/ep434I-unit-linux-x86_64-java17/workarea/I20241105-1800/eclipse-testing/results/ep434I-unit-linux-x86_64-java17_linux.gtk.x86_64_17/org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_VisibilityWindowListener_eventSize[browser flags: 0].png
		at org.eclipse.test.Screenshots$ScreenshotOnFailure.failed(Screenshots.java:41)
		... 1 more

java.lang.AssertionError: 
Test timed out.
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_OpenWindowListener_open_ChildPopup(Test_org_eclipse_swt_browser_Browser.java:840)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.test.TracingSuite.runChild(TracingSuite.java:287)
	Suppressed: java.lang.Throwable: Screenshot written to /home/jenkins/agent/workspace/AutomatedTests/ep434I-unit-linux-x86_64-java17/workarea/I20241105-1800/eclipse-testing/results/ep434I-unit-linux-x86_64-java17_linux.gtk.x86_64_17/org.eclipse.swt.tests.junit.Test_org_eclipse_swt_browser_Browser.test_OpenWindowListener_open_ChildPopup[browser flags: 0].png
		at org.eclipse.test.Screenshots$ScreenshotOnFailure.failed(Screenshots.java:41)
		... 1 more

[akurtakov]

OK, these are the same that fail in swt verification build and should be looked before this release.

[HeikoKlare]

OK, these are the same that fail in swt verification build and should be looked before this release.

As a pointer, this is the issue documenting those test failures and the work on addressing them: eclipse-platform/eclipse.platform.swt#1564

[HannesWell]

OK, these are the same that fail in swt verification build and should be looked before this release.

Ok, then I think it's save to assume that we can continue the transition to the Ubuntu images. I'll submit this and regenerate the jobs so that tonight's I-build can run the tests on Ubunut for Linux.

OK, these are the same that fail in swt verification build and should be looked before this release.

As a pointer, this is the issue documenting those test failures and the work on addressing them: eclipse-platform/eclipse.platform.swt#1564

Thanks for the hint. What I found suspicuos is that it only fails for Java-17 and not 21 or 23. But in the latest I-build all swt tests passed. So was this just coincidence or was the issue fixed (I see the issue is still open)?