Migrate devonfw-ide from bash to IDEasy with GraalVM

[alfeilex]

This is maybe the biggest epic we ever created:
The plan is to reimplement devonfw-ide with pure Java. As devonfw-ide wants to keep its per-requisites and requirements minimal using Java does not make sense in the first place, as you would need a JVM as per-requisite to run it. However with GraalVM it is possible to cross-compile Java code to a native image for a dedicated platform. This way we could create binaries for Windows (exe), MacOS, and Linux with GraalVM native image compiler and ship those as devonfw-ide releases in the future.

This has the following impact:

• • code gets much more maintainable (Java has JavaDoc, auto-completion, compile-checks, refactoring, inheritance, ability to return Strings or arbitrary objects, etc.)
• • code gets much more testable (Java comes with JUnit, AssertJ, etc.)
• • development speed for new features will increase massively (e.g. implementing a new commandlet to integrate a tool will become much easier)
• • existing Java-Code for configurator, migrator and ide-url structure and updater can be reused and integrated seamlessly
• • the dashboard UI could be re-implemented with JavaFx reusing all the Java-code allowing direct integration and generic UIs to invoke commands.
• • when using CLI libraries like pico-cli lots of things get much easier and auto-completion can be fully auto-generated
• • debugging the final product will become much harder as the native image will rather be a black box. Maybe only possible via logging and log-levels while currently end-users get the bash source-code and can read and directly edit it.
• • downloads will get platform specific while there is currently only one product download for all platforms. This also has impact on devon ide update scripts and has to be prepared in a bash-based released before the refactoring to Java took place.
• +/- downloads will IMHO become bigger as they contain parts of the rather complex JDK but this needs to be evaluated
• +/- performance changes unclear, maybe calling devon commands will be slower, needs to be evaluated

After all the pain with bash, we I believe that still makes a lot of sense to do this migration even though it will be a lot of effort.
In order to avoid that we invest too much effort and then notice currently unknown problems and obstacles @ahmedagdmoun will do an evaluation and PoC of this. Only if that is successful and brings a positive result, we will start with the actual migration in parallel stories by the regular dev-team.

Finally, we have to be aware that we can not entirely eliminate all shell scripts:
We will still need devon bash-script and devon.bat CMD script for integration into regular shells (e.g. setting the actual PATH what can not be done from Java). Therefore the "new java based devon CLI native image" needs to support some argument syntax to print out what to add to the PATH. Then the shell scripts can call this and update the PATH accordingly.

Also, we will use this migration as a chance to do a cleanup and will therefore create a new product with a name to be determined but around the term IDE (e.g. IDEasy). This will include the following aspects and changes:

• In general devonfw-ide and the new product can be installed together without interfering each other (even in the same IDE installation tenant).
• The new product should stay as compatible as possible regarding the settings git repository. It should therefore have some downward compatibility support to understand an existing settings git repository without any change but may show some warnings that legacy config was found with a link to a documentation how to modernize (without breaking devonfw-ide usage).
• The term devon and devonfw shall be stripped from the new product (it is not that devonfw is wrong or bad and we will keep our maven groupId and github organization still containing devonfw - it is mainly that average users were unable to understand the concepts behind and distinguish between devonfw with its best-practices about software development for according stacks and a specific tool devonfw-ide that was born in the same initiative but has little to nothing to do with the rest of devonfw. So I got requests saying "I installed devonfw but it is giving me the error 'java could not be installed'" while the next user was saying "I installed devonfw and it is giving me the error 'Failed to load ApplicationContext'". Another devonfw tool is CobiGen that did it smart from the start and does not carry devonfw in its name or config files and folders - we are just going to follow this approach):
  • ~/.devon -> ~/.ide
  • ~/scripts -> ~/.ide/scripts (see NightlyBuild broken #408)
  • ~/scripts/devon.bat -> ~/.ide/scripts/ide.bat (see NightlyBuild broken #408)
  • ~/.devon/devon -> ~/.ide/scripts/ide
  • devon.properties -> ide.properties (if no ide.properties was found in settings or conf it will fallback to devon.properties with compatibility support - also we will create a migration feature that can auto-generate ide.properties, etc. from the old config).
  • DEVON_IDE_HOME -> IDE_HOME
  • DEVON_OLD_PATH -> IDE_OLDE_PATH (if we still need this workaround)
  • DEVON_IDE_TOOLS -> IDE_TOOLS - will most probably be changed from array to comma separate list
  • DEVON_CREATE_START_SCRIPTS -> CREATE_START_SCRIPTS - will most probably be changed from array to comma separate list
  • DEVON_IDE_CUSTOM_TOOLS -> IMHO it would be much better to have a ide-custom-tools.json file instead of this env variable. Hacking all this into an environment variable and allowing to potentially override it in different devon.properties all does not make sense and is hard to maintain for end-users (IDE admins) as well as for the developers of our product.
  • DEVON_IDE_MIN_VERSION -> IDE_MIN_VERSION
• obviously from the above the CLI will also change
  • devon «tool» ... -> ide «tool» ...
  • devon «tool» setup -> ide install «tool»
  • devon ide create ... -> ide create ... (this means there can never be a tool call create)
  • devon ide update -> ide update (this means there can never be a tool call update)
  • devon ide update scripts -> ide update scripts
  • ... we will use this chance to also rethink the CLI and consider improvements ...
  • devon mvn version set 3.9.3 -> ide set-version mvn 3.9.3, also you can simply do ide install mvn 3.9.3 that will set the version to 3.9.3 and then install it.
• The plugin and workspace configuration of real IDEs (eclipse, intellij, vscode, android-studio, etc.) will remain compatible. We will even honor and emulate old properties such as DEVON_IDE_HOME that also work synonymous with IDE_HOME. Also we will not render warnings at the beginning but only after a longer time when we can consider devonfw-ide as deprecated and fully superseded by the new product
• IMHO we have to create a new git repository as replacement for ide-settings that is used by default in the new product. But maybe we can find a better option (or create a new legacy branch and ship out new devonfw-ide releases that use this new branch by default to stay compatible).
• The ide-urls repository shall stay 100% compatible.
• We have to decide if we might create a new repository for the new product or if we stay with the current approach to maintain both products within this same repo. However, for documentation, issues, and PRs a new repo would make it easier.

This epic will be implemented with the following stories:

• #1101
• #1191
• #1192
• #1193
• #1194
• #1195
• TODO Implement all currently existing commands in Java
• TODO more to come

Open questions:

• Currently ide set mvn version 3.9.3 sets the maven version to 3.9.3. Each commandlet must have a unique name that should also given an idea what the commandlet is about. Currently I named this commandlet set-version what seems to make sense. Also this would allow later to also have a ide set intellij edition ultimate that would be handled by a set-edition commandlet. As a result you would need to type ide help set-version to get the help of set-version commandlet and ide help set-edition to get the help of set-edition commandlet. Would it then be better to also change the command syntax to ide set-version mvn 3.9.3 and ide set-edition intellij ultimate? IMHO this would avoid some user-confusion.
• Should we already add changes to the bash edition of devonfw-ide to go away from .devon.software.version to .ide.software.version including and according migration to make our future easier for IDEasy? Currently the new code has to support both version files - should we then also write BOTH files for compatibility? Maybe easier if we just change this in devonfw-ide already. These files are just local and do not affect the settings git repo what avoids pain we had been facing with our latest cleanups (MVN vs MAVEN, etc.)
• IMHO it time has come to create the new repository and move the Java code to that repo. As nobody made another proposal the official product name would change from devonfw-ide to IDEasy what will also be the new repository name - I have already implemented the AsciiArt logo in the CLI. The CLI tool changes from devon to just ide as already described and decided. The top-level module will change from ide to cli. We will copy the documentation folder and adapt it over time to the new IDEasy. This all will make it much easier to do the github actions for the new product release incl. its documentation and for the URL updater that will only survive in the new repo. The ide module will then be removed from the ide repo. Here is the new repo: https://github.com/devonfw/IDEasy

[hohwille]

https://github.com/devonfw/IDEasy/blob/main/documentation/images/Design.png
https://github.com/devonfw/IDEasy/blob/main/documentation/images/Design.drawio