Advanced vertex color features for polygon2d art

[insomniacUNDERSCORElemon]

Context

2D polygonal art is currently possible but not really viable due to the current workflow. There isn't even a painter currently, which I'm hoping gets added via core in 4.0 or some later 4.X release. reduz has talked about it before (in my issue 25120 mentioned below), but now @groud is working on editors.

Most of what I'm suggesting here I feel is fully capable as a core/general concept, although it is more advanced and I can understand that it might need to exist as an add-on at least to start with due to the (currently) niche status of polygonal art. That and even if it were commonly used, I could see this as a sort of "polygon2d+" add-on for those who actually need said features for the sake of cutting bloat (in the grand-scheme more than for the features on their own). Most features mentioned here I expect will need to be created by contributors anyways.

This is also related to godotengine/godot-proposals#1651 (allowing polygon2d to have normal/specular maps). I figured it might be better to create this for discussion rather than having it there.

Problem

Vertex colors need to be both faster/easier to create as well as giving more capability and modularity.

Suggestion

• a system for a vertex color palette/index, for a moderate amount of the same colors used on many vertices. Aside from a potential optimization, allows changing many colors (even in advanced composition) later very easily (change the color in the palette/index and every linked vertex changes, rather than the user needing to manually change each vertex)
• the ability to save and load vertex color maps. This could be made more powerful with some closest match/only close match adaption options
• the ability to work with vertex color maps (as a resource) via code and shaders (admittedly, this is likely something that would NEED to be core). For instance, swapping color maps, shaders using them for vertex flow maps etc
• vertex color layers for easier creation and editing. Allows you to put base colors on one layer and then add advanced effects (like realistic lighting) on top of that. Possibly layer blending/adjustments, but that isn't strictly needed
• an option to inherit color from the polygon's color attribute. Before 3.1 beta 2, this was done by vertices not included in the color map (see Vertex color breaks if number of colors mismatches number of polygon vertices godotengine/godot#25120 ). Probably best done via the palette/index system and simply linked. Aside from minor color additions... this is specifically for changing colors with the same vertex color map, such as alterations on one object/type of object (status, effects, or enchantments), a custom color, or something like multiplayer teams
• mentioned in the linked GIP thread already, but the ability to use vertex color maps for other features like normal and specular maps.

Less related, less core

• color interpolation of 2+ colors is a bit wonky when the user intends to have a quad face, particularly one that is elongated and/or trapezoidal. This (also an optical illusion with corners or concentric gradients) could probably be fixed with a shader
• an old vertex color trick (as used in vertex color skyboxes for the originals of Spyro and Homeworld 2) could be made more useful if there was a way to explicitly mark faces as solid colors and then having a process that automatically edits in (compiled into the game) the needed internal face loops into the meshes, allowing for things like animations/shaders etc to use the original polygon (as well as doing the vertex colors). (however, real face colors/hard-edge colors would probably be better if possible)

Both of those relate to this example:

Some discussion on the vertex color trick in this reddit thread, but there is an overlap of the information here.

---

Even less related, but more core

• the polygon editor itself could use some improvements, allowing for faster face creation (click/paint creation of tris/quads, edge flipping, and beautification) or simply showing the default generated mesh (and allowing generating the rest if you just need to explicitly make a few faces). This is not solely for polygonal art so might get added anyways
• an issue with the uv editor currently is that it won't open for polygons without a texture, despite the non-uv tabs (points/polygons/bones) aren't actually specific to textures and are indeed useful for untextured polygons
• I'm sure cool stuff can be done already with fragment shaders and polygons, but if there was a way to work in the context of polygons (by borders, faces, etc) it'd likely allow for more powerful effects
• the editors lack marquee/lasso select for easier editing, the floating editor is a bit uncomfortable (and the default placement in them is a bit odd and can make it hard to edit things). A dedicated editor might be nice

[insomniacUNDERSCORElemon]

@Xrayez, core and module are also applicable (some things I've described as core--like editor changes--might be considered module if I understand it correctly). Basically each piece whatever way it can be done, accepted obviously better.

---

2 sidenotes:

1. A (visual) example of the vertex color layers, (left-to-right) 2 layers (2 base colors, lighting) and then the composite(green background to show partial transparency):

Tedious enough (w/o vertex painting) to edit as composite (dozens of clicks for color presets to change all points of base color) which is one reason I didn't make it more advanced (I wanted as a design on the balloon in a 3rd color, would require an advanced of a mesh to look good). To support animations would be even more tedious to edit (as composite wouldn't work). (note, example lacks AA and the color from the balloon to the string is odd because I didn't use the mesh trick to fix it)

2. Related to the vertex color layers I've had thoughts for more features that'd be more plugin-worthy. Things like internal vertex and mesh generation (density, face type, deformation, etc options) and shape/line color influence to be used for lighting effects and color variation (for lighting effects, this may be redundant if support for vertex color normal maps are added).

[Xrayez]

@Xrayez, core and module are also applicable (some things I've described as core--like editor changes--might be considered module if I understand it correctly). Basically each piece whatever way it can be done, accepted obviously better.

I'm not sure what could be an objective criteria to assign labels like this... In theory almost everything is possible to implement via GDScript and/or C++. 🙂

2. Also some system of merging multiple polygons to create faces. Using the polygon overlap to define borders/faces and ignoring sections that spill out of the main polygon. So not exactly polygon CSG (likely possible w/it), but polygon CSG would be useful too. Also auto-generation if you just define a few internal faces, but if the existing editors were improved in ways I've suggested (auto-gen being one) then this might also be redundant.

Not sure if applicable, I've recently implemented boolean operators in 2D which is mostly inspired by CSG nodes in 3D, see goostengine/goost#39.

In there, PolyNode2D is similar to Polygon2D, with boolean operators... But no way to modify UVs currently (also not sure if needed since it may be possible to just convert nodes in the editor). Except that Polygon2D doesn't support internal holes, as far as I know.

[insomniacUNDERSCORElemon]

I'm not sure what could be an objective criteria to assign labels like this... In theory almost everything is possible to implement via GDScript and/or C++.

I'm no expert but I assume the criteria is based on suitability. Many things I've described likely can be done in some 3rd-party way, though they have varying suitability levels for how much they should be in core. Many things are suitable in idea, but just need to pass the "do enough people want this?" question (that's how I organized the sections). Which, with polygonal art being clunky/non-viable now is really a chicken-and-egg situation.

I mean if there were an amazing polygonal art suite (particularly if it could be installed to Godot where it'd be accessible to multiple projects by default/simple on-switch) I'd be all for that too. Things like the editors are definitely possible there, but I feel like the more technical stuff (again: shader context and vertex color normal/specular maps) likely wouldn't be viable outside of core.

There's also the problem of integration, something like the default UV editor would be mostly ignored/clutter if you're using the 3rd-party tools. Also compatibility if removed (aside from when results can be carried over and original pieces backed-up at least) but I guess that's a "it hurts when you do that? Don't do that then" issue.

Not sure if applicable, I've recently implemented boolean operators in 2D which is mostly inspired by CSG nodes in 3D

For me mentioning CSG in passing, yes.

Beyond that actually doesn't matter because I wasn't thinking properly, making the internal polygon editor more competent via autogen and quick tris/quads would be a significantly faster+easier workflow (esp. w/quick tri/quad brush mode, also using auto-gen after every-other-face is filled in or when only needing a few faces for details).

If you ignore quick tris/quads, the train of thought I had was that you'd save time if creating internal vertices and faces at the same time.