Add JVM/JNI target for local development #10
Comments
|
Sorry if this is a bit hand-wavey on implementation details. I was able to get some basic game logic running on the JVM and interfacing bidirectionally with Godot with the design described above. Working on cleaning up a local POC of this that I had built against MrAkakuy's project. It had some rough edges and is now pretty out of date with the updates you folks have been making (more so if the refactorings to use Poet go through), so not the most usable thing in current state. |
|
Just realized the kotlinpoet stuff is already in (for the most part?), so scratch that bit. Will aim for a minimal demo of the above to more clearly show what I'm picturing, but wanted to at least kick off a discussion here, and get a feeling for whether or not this is controversial. |
|
@JustinMullin That is a great idea overall! And exactly what @lassem asked for on discord. That said i personally think it's a great idea anyways because most users probably really only need the interaction with godot and thus writing the whole game logic in common would work for them. And for this majority of users your idea would really be helpful. To summarize: I really like your idea, but on an opt in basis. Also it would not have a high priority right now as we have to finish rewriting the code and improve the overall user experience first. Looking forward to your contribution :-) |
|
Starting from Kotlin 1.4, Jetbrain will be rewritting the whole Kotlin compiler ( a new unified one for JS/JVM/Native, so far the Native compiler has been a standalone tool). The speed up will be gradual so we can't expect lightspeed compilation time for Kotlin native in the coming months. So I have mixed feelings about this. But if this project exists, it's because we believe in Kotlin Native potential in the first place. |
|
As a developer writing a Godot game in C# who normally is primarily a Kotlin developer, the biggest thing keeping me from switching to this project at this very moment is the fact that Kotlin/Native takes an incredibly long time to compile, so easy iteration and testing is difficult when compared to C#. The ability to quickly test and run Kotlin code would make this invaluable. That said, the differences between K/N and Kotlin on JVM would make this difficult. For example, the fact that Singleton objects are frozen at runtime in K/N is fundamentally different from the JVM version, and would trip you up when compiling with JVM while developing the game (then switching and realizing these objects are frozen). That being said, I would still absolutely benefit from this feature. It might almost be preferable to have just a standard JNI/JNA link instead and do a JVM only version, instead of trying to wrangle K/N and JVM in the same project. I'd love to see what you can come up with. |
|
@JustinMullin If you are willing to work on this I would assign you this issue so we know that someone is working on it :-) |
|
Sure, absolutely. |
# Overview This implements annotation based Registration of Classes, Functions, Properties and Signals. Information on how to use the annotations are provided in the Registration readme. # Technical background: The annotation processing is done using a KotlinCompiler plugin I implemented in the old repo (`kotlin-compiler-plugin` and `kotlin-compiler-native-plugin` modules). To read the annotations on native targets we use the [MpApt project](https://github.com/Foso/MpApt). The annotations are received in a AnnotationProcessor (the `godot-annotation-processor` module). The Entry file is then generated using the `EntryGenerator` (inside the `entry-generator` module). The problem we had is that the compiler plugins are called too late in the compilation process for the compiler to pick up the newly generated `Entry.kt` file. Jetbrains goes around this problem by directly generating JVM Bytecode / IR (for native) and feeding that into the compiler (see the [Serializable compiler plugin](https://github.com/JetBrains/kotlin/tree/master/plugins/kotlin-serialization)). As I don't have the required know how to directly generate IR (nor do i find it useful as we cannot debug it conventionally) I opted to overcome this problem by adding a dummy target during build. As the actual compilation task is the shortest one the time overhead is negligible IMHO. During the compilation for this dummy target, the `Entry.kt` file is generated. So it's already present when we compile the real target. This dummy target logic is located in the `KotlinTargetPreset` (in the `godot-gradle-plugin` module). It ensures also only one dummy target is added even when building for multiple targets. # Summary of the steps: 1. During build a dummy target is added 2. The dummy trarget get's compiled 3. The kotlin compiler plugin calls the annotation processor with all required annotations 4. The annotation processor gathers all information required for the entry generator 5. The annotation processor calls the entry generator with all required informations 6. The entry generator generates the entry file 7. The actual target is compiled including the now present entry file 8. rest of the normal build pipeline follows... # Alternatives that i tried: - Normal annotation processing using `kapt`: The problem here is, that `kapt` only works for JVM targets. There is no `kapt` for native targets. Once #10 is implemented. My goal is to switch to normal `kapt` annotation processing as it's much faster. The migration should be almost seamless as i already use a annotation processor and we just need to remove the compiler plugin part and the dummy target. But for now this is sadly not possible. - Extracting the kotlin compiler frontend into a script and call it directly: This worked for the most part. The problem was that we needed to add all required sources of all dependencies to be parsed as well. Which is no problem for our own code but is a problem for dependencies the user might add. Furthermore the parsing took a long time (around 45-60s on a Dell XPS 15 with an i7-8750H). It also takes this long for the `godot-library` so it was not a problem with the script, more of the sheer amount of code that needed to be parsed. This and the fact that we cannot gather all the required source code for all the dependencies a user might add, lead me to the conclusion that i don't want to use this approach. # The next steps: There are many things we can optimize regarding annotation processing. But I didn't implement it yet as this is the basics on which we can build on. Some improvements i want to make: - Implementing some extension functions to make the life easier when working with signals So a user could call `connect(::methodInThisClass)` and this would automatically connect the signal he has overriden - Use code generation to provide the possibility to export `NodePath` from annotating a property of a type which derives from `Node`. Like this a user could write this code: ```kotlin @export lateinit var body: KinematicBody ``` instead of: ```kotlin @RegisterProperty(true, null) lateinit var body: KinematicBody @RegisterFunction override fun _ready() { body = KinematicBody2D from getNode(NodePath("Path/To/Node")) } ``` and then could wire up the node path in the editor, like when writing ``` export var pathToBody : NodePath onready var body: KinematicBody = get_node(pathToBody) ``` in GDScript. # Notes **Note:** I did not bother migrating the coroutines sample as it does not work at the moment anyways and we can do the migration together with the fix of it in #36. Commits: * Implement class constructor and destructor binding generation * Implement function binding generation * Fix constructor and destructor nullable types * Cleanup generated code by leveraging controlStatements and kotlinPoet string templation * Add internal function bridge generation and binding * Implement property binding * Implement signal registration * Change rpc mode annotations and add renaming todo's * Implement rpc modes registration * Change the way local builds are built This is done so we can hit global build without passing parameters or changing the build script to be able to build locally. This makes it easier for new contributors. If one want's to build only for his platform, he can still pass the `platform` argument. * Implement property type hint and string registration * Annotate all games, fix method binding and temporarily disable old entry generation * Remove old plugin configuration * Implement internal annotations * Implement dummy target for entry file generation and wire up all build tasks * Use internalAnnotations and fix some build problems * Remove classes.json files * Make internal annotations compatible with android and ios builds and everything for godot 3.2 * Cleanup annotations and remove unnecessary expect actuals * Enable annotations processing on jvm targets * Implement annotation processor sanity checks * Only build entry file for first target and fix entry file directory * Implement gdns file generation and remove dead code * Extract gdns file generation to separate class * Move the casts helper file to the utils package * Add some todos and comments * Add some javadoc * Remove unnecessary property sanity check function * Cleanup code and add internal annotations to travis * Fix travis build order * Edit documentation for new annotation based registration * Fix formatting * Update getting started readme * Fix source sets to configure on targeted builds * Make signal documentation easier to follow * Remove debug logs * Add android arm64 libkotlin * Fix toString method * Respect buildType passed as gradle argument * Fix gdnlib path to dylib on macos * Add new internal annotations module to compiling from source readme
|
Is there any progress on this feature? I assume given that slow compile speed is still an issue with kotlin native this can be a huge time safer during development? |
|
@fkrauthan This feature is not currently in development as a local-dev option for a Kotlin/Native solution, but efforts have turned to the https://github.com/utopia-rise/godot-jvm project which would use a JVM runtime at release as well as in dev, as Kotlin/Native isn't where it needs to be for runtime performance either, unfortunately. |
|
Ah so currently the plan is to concentrate on a JVM only project and reconsider this project (and kotlin native) when it is more mature? |
|
Currently kotlin native is not ready (see in readme). |
Describe the problem or limitation you are having in your project:
The Kotlin/Native compiler is very slow at present, resulting in fairly long turnaround for small game code changes using godot-kotlin during development.
Describe how this feature / enhancement will help you overcome this problem or limitation:
For local development, running game logic on the JVM rather than native code would allow for quick re-compile times, as well as some JVM niceties such as live class reloading at runtime and direct debugging+breakpoints into the Kotlin code. The Kotlin/Native implementation in godot-kotlin can then be used in the exported game to remove the dependency on the JRE.
Describe implementation detail for your proposal (in code), if possible:
expectdeclarations. Native implementations of these classes will acquireactualkeywords to define them as platform implementations, but otherwise remain unchanged.actualimplementations are created utilizingexternalmethods to call out to Godot APIs; implementations will be registered by the JNI later.actualimplementations rather than the Kotlin/Native compiler against the existing implementations. The output classes are bundled to a JAR and output into the Godot project path.externalmethods don't support pointer types, etc.Is there a reason why this should be in this project and not individually solved?:
Ideally a JVM+JNI implementation will be maintained in parallel to the Kotlin/Native version to ensure seamless transition between fast local development and export.
The text was updated successfully, but these errors were encountered: