EL TCK Refactoring using Junit

[alwin-joseph]

Related Issue(s)
#1126

Describe the change

• Converted all standalone legacy EL Tests to Junit (No arquillian used).
• Used slf4j logging instead of TestUtil class
• Moved the tests from https://github.com/eclipse-ee4j/jakartaee-tck/tree/tckrefactor/src/com/sun/ts/tests/el to el/ folder
• Created glassfish runner for the same el tck

Pending:

• Use servlet & JSP containers to run these tests for platform vehicle.properties
• Refactor standalone EL signature tests
• Copy the test in Fix el issue 194 #1159 also.

[scottmarlow]

Thanks @alwin-joseph for working on EL TCK refactoring!

I have some general feedback that is not specific to EL. I think that the (EE 11 Platform TCK generated) Standalone TCKs will need to be completed before the Jakarta EE 11 Platform TCK changes are completed. The reason being that the relevant Standalone TCKs always need to be completed before the EE SPEC wave can be considered.

I'm mentioning ^ so that you can be mindful of the need and consider how it might apply to the EL refactoring work. IMO, we can separate the Standalone vs Full Platform aspects after the initial PR but I think we need to be stricter about addressing this for EE 11 so that we ensure that no changes are made to a TCK after the relevant EE Spec has been ratified without the SPEC team + implementations having a chance to validate that implementations still pass. The goal should be to produce a Standalone TCK that can be the basis for the Platform TCK tests (e.g. pending our better understanding how to do that with test vehicles or equivalent in the refactored EE 11 Platform TCK).

[alwin-joseph]

I see PR #1159 on master branch that adds a new EL test. Reminder to self that the same should be merged to tckrefactor too along with this PR or as a separate change.

[alwin-joseph]

Hi @starksm64

With respect to the vehicle implementation, would you have any comments for EL TCK refactoring in this PR.

1. Basically, the EL TCK in standalone mode is refactored in this PR and uses only JUnit. For legacy EL standalone TCK the tests did not run in any vehicle (

   platform-tck/install/el/other/vehicle.properties

   Line 52 in e3ab547

   com/sun/ts/tests/el = standalone

   ) and hence the changes in this PR does seem sufficient to me.

2. For platform TCK, the EL tests run in 2 vehicles (

   platform-tck/install/jakartaee/other/vehicle.properties

   Line 80 in e3ab547

   com/sun/ts/tests/el = servlet jsp

   ) servlet & jsp.
   Com.sun.ts.tests.common.vehicle.jsp/servlet has the legacy implementation of jsp/servlet vehicles (same folder with other vehicles). I believe the requirement will be to duplicate the current vehicle implementation or equivalent scenario using Arquillian.

3. I found that batch TCK created ejb vehicle and runner in the PR Add EJB runner to platform/Arquillian path as a separate failsafe execution batch-tck#40.
   Should we also follow the same approach? Any idea if arquillian already has extensions or possibilities to extend the test runs in custom vehicles or containers.

cc @scottmarlow @gurunrao

[scottmarlow]

Hi @starksm64

With respect to the vehicle implementation, would you have any comments for EL TCK refactoring in this PR.

1. Basically, the EL TCK in standalone mode is refactored in this PR and uses only JUnit. For legacy EL standalone TCK the tests did not run in any vehicle (https://github.com/jakartaee/platform-tck/blob/e3ab547f4308877e2178ef21609c0d8aac45f766/install/el/other/vehicle.properties#L52
   ) and hence the changes in this PR does seem sufficient to me.

2. For platform TCK, the EL tests run in 2 vehicles (https://github.com/jakartaee/platform-tck/blob/e3ab547f4308877e2178ef21609c0d8aac45f766/install/jakartaee/other/vehicle.properties#L80
   ) servlet & jsp.
   Com.sun.ts.tests.common.vehicle.jsp/servlet has the legacy implementation of jsp/servlet vehicles (same folder with other vehicles). I believe the requirement will be to duplicate the current vehicle implementation or equivalent scenario using Arquillian.

./webartifacts/servlet/src/main/resources/api/jakarta_servlet/servletcontext40/addJspFile.jsp is currently referenced from the refactored Servlet TCK but I don't see any references from the Servlet TCK to Servlet.*Vehicle or JSP.*Vehicle. So we might want to ask @olamy for input on whether we should do the same for other Web only container using Platform TCK tests. For the Standalone EL tests, we shouldn't reference the Web container.

3. I found that batch TCK created ejb vehicle and runner in the PR [Add EJB runner to platform/Arquillian path as a separate failsafe execution eclipse-ee4j/batch-tck#40](https://github.com/eclipse-ee4j/batch-tck/pull/40).

This is a good question as we currently have the problem of ratifying multiple Specifications before Core/Web Profile + Full Platform Specifications are complete.

   Should we also follow the same approach? 

IMO, it is better to not follow the same approach. We really should have separate source/maven artifacts for the Standalone (Java SE only tests) and the profile level TCK tests. I think that the profile level TCK tests can and should leverage the Standalone tests to avoid duplication (Profile TCK developers should update/adjust their tests accordingly to account for changes in the underlying Specification TCK tests that may occur prior to said underlying Specification being ratified).

Any idea if arquillian already has extensions or possibilities to extend the test runs in custom vehicles or containers.

We have a few choices on how we deal with each of the vehicles. For reference the different vehicles are still in https://github.com/jakartaee/platform-tck/tree/tckrefactor/common/src/main/java/com/sun/ts/tests/common/vehicle. I think we should walk through how each are used and consider if there is value in using them.

I haven't gotten to the Persistence tests yet but I will be looking at 5.4.14.3 Persistence Test Vehicles in https://github.com/jakartaee/platform-tck/blob/tckrefactor/user_guides/src/main/jbake/content/config.adoc (stateless3, stateful3, appmanaged, appmanagedNoTx, pmservlet, puservlet) vehicles.

cc @scottmarlow @gurunrao

[markt-asf]

This looks great. I managed to build the EL TCK locally without too much difficulty. javatest.jar was the only issue and that just needed to be installed at the right coordinates in my local repository.

I was able to configure the TCK to run against Tomcat's EL implementation easily. It was also simple to swap between the Jakarta EL API JAR and the EL API JAR provided by Tomcat. Everything passed in all combinations.

The tests report a total of 359 tests which I assume is one less than the 360 reported by the current TCK as the signatures are not tested. Is that correct?

Finally, by far the most impressive thing about this change is the time taken to run the TCK. The current TCK takes tens of minutes to complete. This refactored version takes less than 5s.

[scottmarlow]

@markt-asf I think the signature test work is still pending.

[alwin-joseph]

The tests report a total of 359 tests which I assume is one less than the 360 reported by the current TCK as the signatures are not tested. Is that correct?

Correct. The signature test work is pending (The total tests for standalone EL TCK are 349 +1 signature test).
Other pending works as mentioned in the description can be done in separate change(s).