Improvement: Extract datafile component, base/util, base/crypt projects

[ieugen]

• There are a LOT of circular dependencies !!!
• ~99% it is code shuffling - no behavior change
• I had to split some files to be able to shuffle code around: UtilValidateEmpty, UtilPropertiesRuntime, etc.
• A lot of code changes are related to CheckStyle
• We should consider spotbugs for code formatting via gradle

We can publish datafile component as for example:
org.apache.ofbiz/component-datafile/18.12.10

Appllications can consume this via reuglar maven dependency.
The dependency could be specified as having runtime / provided scope so when it gets pulled in OFBiz it will use the version available there (patch versions).

Also each component jar can be a Java 9 module - to encapsulate it's dependencies and avoid jar hell.

I have added a sample project that uses the crypto code from OFBiz lib to do crypto.
I could do a datafile example tool but that would take more time.
Fill in the blanks: ofbiz functionality in other apps and services.
Ease the integration at java level.
Open the borders of OFBiz to the outside developer world.
https://github.com/ieugen/ofbiz-tooling-demo

Also as exploratory work, making enity engine as a library (on top of this PR) takes ~ 670 additions and ~ 370 deletions. Work is not done yet, but close. See ieugen#3 .

[ieugen]

Also relates to https://issues.apache.org/jira/browse/OFBIZ-12308

[ieugen]

A glimpse of the cyclic dependencies in OFBiz

[sonarcloud]

Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarcloud]

Quality Gate passed

Issues
1 New issue

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[JacquesLeRoux]

We can neglet the CodeQl analysis. It reports 208 "RandomStringUtils.random(SECURE_RANDOM" and it's safe because of SECURE_RANDOM. Sincerely the tool is clumsy. I have also dismissed a lot (300+ ?) in the OOTB code.

[ieugen]

We can neglet the CodeQl analysis. It reports 208 "RandomStringUtils.random(SECURE_RANDOM" and it's safe because of SECURE_RANDOM. Sincerely the tool is clumsy. I have also dismissed a lot (300+ ?) in the OOTB code.

Other than this, did you manage to take a look at the code?

[JacquesLeRoux]

Hi Ieugen,

Not yet, I'm currently focusing on the new codeQL implementation.

I tried it few years ago for Java but it was not able to handle OFBiz, only JavaScript. It now works but still, like for JavaScript in the past, has a number of false alerts, like this one. I remember now, exactly 271 same cases. Fortunately GH is able to ease dismissing them, still 553 remaining, a lot of duplicate.

[JacquesLeRoux]

BTW, this is about whole OOTB OFBiz code. There is no other than this one in your PR.

[JacquesLeRoux]

For now I had only a cursory look at your work. But, as I said, I intend to have a deeper look in 2024.

[ieugen]

Thank you @JacquesLeRoux , It would be great.
Right now I paused my work on OFBiz.
IMO it does not worth investing more until I have a path forward.
I believe this PR opens the door for that path forward: more modular ofbiz, ability to develop tooling outside the codebase that can use parts of OFBiz.

Looking forward to your review and hopefully merge of this PR (in this form or another).
After that I will continue working on OFBiz.

[JacquesLeRoux]

Hi @ieugen,

While working on upgrading Apache Commons IO because of https://www.cve.org/CVERecord?id=CVE-2024-47554 (though we don't use org.apache.commons.io.input.XmlStreamReader so not a big deal) I was surprised to not find any Apache Commons IO Gradle dependency.

Then looking for "commons-io" I stumbled upon this PR. There is much conflicts and some recent ones. For instance dependencies are now in dependencies.gradle, and they have evolved with https://issues.apache.org/jira/browse/OFBIZ-13123.

Nevertheless, I have a question about Commons IO. I see that you used api 'commons-io:commons-io:2.15.1', why ?

Also are you still interested about pushing this PR (w/o conflicts of course)?

TIA

[ieugen]

Hi @JacquesLeRoux ,

I was surprised to not find any Apache Commons IO Gradle dependency.

The dependencies are brought in by transitive dependencies, that is why they are not "visible".
OFBiz does use the classes directly, see

ofbiz-framework/applications/content/src/main/java/org/apache/ofbiz/content/cms/ContentJsonEvents.java

Line 33 in f8ad232

import org.apache.commons.io.IOUtils;

I see that you used api 'commons-io:commons-io:2.15.1', why ?

I think that was the release available at that time https://commons.apache.org/proper/commons-io/changes-report.html

I am open to fixing and merging this PR.
I did start a process to try and merge it piece by piece here: #837

It's more slow and forces us to discuss and tackle issues one by one.
There are 2 options:

• fix this PR and merge it and then try to clean up the code later
• close this PR and continue with discussions on Implemented: Introduced base/util module (OFBIZ-12308) #837 and future PR's .

While not as clean, merging this PR would push things forward faster :) .

[mbrohl]

It's more slow and forces us to discuss and tackle issues one by one. There are 2 options:

* fix this PR and merge it and then try to clean up the code later

* close this PR and continue with discussions on [Implemented: Introduced base/util module (OFBIZ-12308) #837](https://github.com/apache/ofbiz-framework/pull/837) and future PR's .

While not as clean, merging this PR would push things forward faster :) .

I recommend to go with option 2. There is no need to hurry.

[JacquesLeRoux]

Thanks Eugen,

I agree with Michael, so close here.

I neglect the commons io update for now as there is no risk, trunk uses commons-io-2.16.1. Last one is commons-io-2.17.0 from 15 Sept. It's safe since 2.14.0;