The future of release management and versioning

[Masgalor]

I recently opened two pull-requests to implement support for a addon manager called Nexus.
The meta issue for this effort can be found here #325 , but this is not subject of this discussion.

While discussing the details of the implementation @AsherGlick and I drifted into another discussion about how releases are handled and how the versioning scheme should look like. @AsherGlick gave a lengthy explanation of his thoughts here #326 (comment) , while I respect every decision made by the project owner, I disagree with some of the points made in that post, so I would like to have a discussion about this topic so we can hear more opinions on this matter.

Why do I even care about releases and version-numbers?
I already mentioned in other issues and discussions here, that this project lacks public visibility. Where ever I see people talk about Burrito, be it on reddit, the official forums or ingame, the subtitle is often something like it's dead anyway it's too complicated I know no-one who made it work. This needs to change! And to change it I see a need for more frequent releases (better release workflow) and easier access/installation (addon helper/manager).

As always I'm not here to talk empty words, but I'm down to work on PR's and workflows as long as there is a clear target everyone agrees to.

---

What I think we should discuss here.

1. The version format

Independent of the value that is represented what style should be used.
While there are several options like alphanumerical (6f6062b) alphanumerical.numerical.numerical (alpha1.4.0) numerical.numerical.numerical (1.4.0), I think this is not up for discussion, because every addon manager including arcdps chainloading expects three numbers with an optional 4th. This makes sense for a lot of people including me, with three numerals it is always possible to sort the versions historically without any other information. If we just put a commit hash in there no third party is able to tell what version is the most relevant without checking out the relevant branch and looking at HEAD or even worse walking down the whole tree until both hashes were found.

2. The version value

So where do we get the numbers from?
I assume here, we will continue with 3 to 4 numerals, if this is not the case the following thoughts might not be relevant.
There are three popular ways to fill the fields, SemVer 1.4.0, date & time 2024.08.10.15 (this is limited to one release per hour), unix timestamp 172.329.627.4 (this is limited to one release per second). The point here is always that the most left number changes the least and the most right number changes the most.

3. Release frequency and workflow

When do we publish a new release and how does that work.
At the moment a new release is only published if some big goal was accomplished, this is great because updating to a new release gives an experience like whoa all that great new stuff here amazing. But a lot of people are also looking at the commits made and at the release section with roughly one release per year and go like oh man a bug I've encountered was fixed, but until it is released I am no longer playing this game. Or as a friend of mine said, be aware strong language ahead, an alpha with barely one release per year is more than useless, it's dead.

As for the frequency I think we should publish a release every time we change/fix something that is notable to the user. The last commit to master for example 6f6062b would not warrant a release in my eyes, but 4185432 would definitely. Too frequent releases would bother people, because they need to move several files by hand to several different places.

For the workflow I agree that there should be no manual work necessary, but at the same time I don't see where you would do anything by hand. No matter how the version scheme looks all you do as a maintainer is go to the github release section hit new Release and put some number in the required tag-field. That's it, everything else is done by workflows and/or build-tools, and if that is not the case yet we need to work on those.

---

That's all from my side, I hope a lot of people tell us their opinion about those topics, and that we can get this project more of the appreciation it deserves.

[AsherGlick]

Great Writeup!

---

If we just put a commit hash in there no third party is able to tell what version is the most relevant without checking out the relevant branch and looking at HEAD

This is true for incremental numerical values too. In order to figure out what the newest version of a plugin is a third party would have to check out the git repo and search through git tags to identify the newest version. Any caching of the newest version number in the titles of releases, api endpoints, filenames, etc, could be mimicked by a hashed based version number too.

Currently, nexus goes to github and tries to identify what the newest release number is. Then it checks if it's version is less than that number and if it is then it updates. Using a commit hash instead means that that less than comparison becomes a not equals while all the other steps in an add-on manager's process stay identical.

The only advantage a numerical only version number gives you when querying newness is "is version x newer than version y". Users dont normally want to update to the next version, they want to update to the newest version.

However there is a good point to be made about addon managers primarily supporting numerical/semver and nothing else. This of course could be changed with a pr to said addon manager.

---

I am not opposed to semantic versioning from an ideological perspective. There are good non-update reasons to use other version systems

• semver, <major>.<minor>.<revision> typically means I can update to any <revision> without concern, any <minor> probably does not contain any breaking changes, and <major> could contain breaking changes.
• It is easier to say "Build one thousand thirty two" then "Build A zero D A four zero seven"

Using a hash is zero recurring work and also very little upfront work, hashes are automatically provisioned by git. An incremental solution requiring the same amount of recurring work, like some listed below, would be an acceptable end result.

---

No matter how the version scheme looks all you do as a maintainer is go to the github release section hit new Release and put some number in the required tag-field. That's it, everything else is done by workflows and/or build-tools, and if that is not the case yet we need to work on those.

We dont have any workflows that run on-release right now. We could though if it is the right decision. Right now the release process follows the pattern:

1. Push up or merge the newest commit
2. Let the automated build process build it
3. Create a new release and give it a number, which gives it a tag (human action)
4. Download the build artifacts from the build process (human action)
5. Upload the build artifacts to the release (human action)

Something to note in this process today is that the tag is added after the build process is triggered. So we would need to change this process no matter what for semver values to be read from tags at compile time.

My ideal release workflow is

1. Push up or merge the newest commit
2. A release is cut automatically

---

One solution to achieve both an incremental version count and not have any manual input is this is to ditch semver and just have a single incrementing number as the version number. It appears arcdps does this, and many other applications outside of the gw2 addon sphere do this too.

Obtaining this number without human interaction can be a bit tricky but there are a few options, and probably plenty more with a little bit of research.

• An external service that is queried automatically each time a release is cut, the service returns an incremental number each time it is queried.
  • This could be a bit annoying to setup initially because it would need to be hosted by a myself or reliable party (like github) and also be free from malicious actor abuse.
• Use github job id accessible in the github action github context
  • This could mean the versions baloon quickly, our current job id at the time of writing is 1035, personally that does not bother me and it is unlikely that number will eclipse an sint32 max of 2147483647
• Keep track of the current version number using automatically assigned tags
  • Every release workflow searches back through the tree to find the largest version tag, then increments that number by one and adds a new tag to the commit that is being turned into a release.

---

Too frequent releases would bother people, because they need to move several files by hand to several different places.

I think I will need to really think hard about how we can automate this best. I really want to move in the direction of fully automated releases but I agree that cutting a new release multiple times a day if there are say 2 merges is a bad idea. One thing we could do is automatically cut a release every Tuesday if there are changes since the last release.

I pick Tuesday only because that is the same day that gw2 updates.

If we need to do an emergency release for a bug or a crash or something we can add a way to run the workfow manually somehow. Maybe via the github ui but there are also more programmatic options that can be included in runbooks too.

This could still create a release that had very few user facing changes in it though, I personally am ok with that.

---

it's dead anyway
I know no-one who made it work
alpha with barely one release per year is more than useless, it's dead

This type of attitude is pretty common unfortunately, but far from the majority. I see it especially inside video game spheres. It can be hard to ignore them because belittling comments hurt, but these comments usually come people that have little to no experience creating or running projects like this themselves, even if they are programmers. I often see things like "this project needs" or "you must do x". Just remember to treat them with respect because the fact that they are upset means that they care enough to be upset and could one day be proponents of the project.

it's too complicated

If they were talking about window manager complexity then: yep it is, I hope to fix that one day but it is way more complicated than automatic version updates.

---

As always I'm not here to talk empty words, but I'm down to work on PR's and workflows as long as there is a clear target everyone agrees to.

A great attitude.

[AsherGlick]

Thanks @Technetium1, I think I agree with this. If we ship a broken update by accident, which will only happen if something is broken that merges into main/master, then we can always push a new release. The folks who would be effected would be the ones who downloaded a new version between the bugged launch and the fix. Those people could also always downgrade to the version prior to whatever broke until a fix is made.

[Masgalor]

When talking about SemVer being easier to sort than hashes I was thinking about a scenario where no third party is involved.
If I would provide you with two files and no other context, you could tell me what file is newer if SemVer is used, you could not if hashes were used.
But I have to admit this argument is squishy at best if you have to contact the github api to ask for the newest version, because the commit hash is part of the response anyway.

However there is a good point to be made about addon managers primarily supporting numerical/semver and nothing else.
This of course could be changed with a pr to said addon manager.

Do you realistically see this happen?
I mean not only writing said pr, but also getting it merged, and this potentially not only with one addon manager but with several projects. I really think this efforts are not justified for the little benefit hashes might have (admittedly opinionated).

---

For reasons why I do not like automated releases.

• A project might have a lot of commits to the master branch that do not influence the built binaries.
  For example if we would implement new workflows or if we polish the readme.
  We might end up with several releases (based on the criteria triggering the release) that do nothing for the user.
• We have to keep the master branch in a good state (is able to build) all the time.
  Maybe this is more or less a personal thing, but I like to have the option to merge unstable features into master for testing or merge multi-part-prs one by one over some time frame while breaking master in between. I am aware this can also be accomplished with feature-branches.
• Automatically created release-notes suck. I think release-notes are an important part of every release and should reflect the efforts put into the software, some bot collecting pr/commit titles is not that great. It is also hard to communicate eventually occurring breaking changes without a manual statement.
• People might not report a broken version but silently drop burrito instead.
  I think a lot of people are rather skeptical about burrito and a faulty release might be bad for the image of it. If the first impression is a broken release, the user is lost.

I see how several arguments are based on my personal tendency to perfection, I will let them here anyway. ;)

[AsherGlick]

Do you realistically see this happen?

If we decide that a hash based version is the best course of action: Yes, of course. But it does seem like we are leaning away from the hash based solution right now.

A project might have a lot of commits to the master branch that do not influence the built binaries.

Good point, we might be able to theoretically handle that too by comparing build hashes of the release. If the file hashes are the same then nothing inside the binary changed and we dont need to cut a new release. Practically, this can only be done after the workflow builds the project so if we go with the automated Tuesday release workflow idea we will want some way to prevent the Tuesday release workflow from rebuilding the same non-hash-changing commits over and over again if there are no new commits on top of that. A minor detail with many good solutions.

We have to keep the master branch in a good state (is able to build) all the time.

This is generally ideal because we do have users, such as myself, who run burrito off of master. Or kling who runs off of xml_converter

I like to have the option to merge unstable features into master for testing

This can still be done if the features are placed behind feature flags in the settings file

Automatically created release-notes suck

Agreed.

People might not report a broken version but silently drop burrito instead.

I am not concerned about this right now. I am not a megacorp who is trying to amass as many users as possible and it is not for me to dictate that everyone must use burrito. The solution to getting bug reports from silent users is telemetry which is a different can of worms.

[Masgalor]

If we decide that a hash based version is the best course of action: Yes, of course. But it does seem like we are leaning away from the hash based solution right now.

This is very optimistic as we do not know if any addon manager/loader would even consider our PR to add hashes as a version mechanism. If I were the project owner of e.g. nexus and I would already support over a dozen addons (with identical version scheme) and you would ask me to add another (entirely different) variant just to satisfy one addon I would decline your PR.

But this is pure speculation from my side, maybe they are more open minded as I think.

[AsherGlick]

Started wtih
https://discord.com/channels/410828272679518241/917239829664788571/1234496589490294918

Found this
https://github.com/RaidcoreGG/Nexus/blob/2b0bb2358076382c7e44e81f392d883808e86caa/src/Loader/AddonDefinition.h#L26-L27

UpdateProvider is defined in
https://github.com/RaidcoreGG/Nexus/blob/2b0bb2358076382c7e44e81f392d883808e86caa/src/Loader/EUpdateProvider.h
Which has these two options as well as others
EUpdateProvider::Direct
EUpdateProvider::Self

Which is used in
https://github.com/RaidcoreGG/Nexus/blob/2b0bb2358076382c7e44e81f392d883808e86caa/src/Services/Updater/Updater.cpp#L249-L282

UpdateDirect()
https://github.com/RaidcoreGG/Nexus/blob/2b0bb2358076382c7e44e81f392d883808e86caa/src/Services/Updater/Updater.cpp#L514

• downloads from a target and expects a .md5sum file

UpdateSelf()
https://github.com/RaidcoreGG/Nexus/blob/2b0bb2358076382c7e44e81f392d883808e86caa/src/Services/Updater/Updater.cpp#L580

• downloads from an addon provided url, does not expect a .md5sum

and we can set those here
https://github.com/AsherGlick/Burrito/pull/326/files#diff-169c2cbb436c209759ed4b2c9c81a98e91e57efb7e64249745d7ffe604a253dcR429-R430

to trigger manual updates based on anything we want without a pr to nexus

---

from arcdps https://www.deltaconnected.com/arcdps/api/README.txt

optional extension module exports:

const wchar_t*     get_update_url()
      no parameters

      expected for the extension to check for an update, and to return null, or a wchar_t* string, eg. www.mydomain.com/myfile.dll (443/HTTPS only).
      if a string is returned the module is freelibrary'd, replaced with the file from url, and re-loaded.

which tells me that it functions in a similar way

---

Would they accept a pr, probably, do they need to even, probably not.

---

However there is a good point to be made about addon managers primarily supporting numerical/semver and nothing else

I am no longer convinced that addon managers only support semver, they seem to be quite flexible.

[klingbolt]

I agree that our release schedule could be improved. Most of the work I have been doing is on the back end and your support for the project is inspiring for finishing this work. Once the xml converter is fully finished, I think Burrito will be in a great state for additional front end features.

My ideal update structure would be one where the program checks its version on launch, prompts the user to update, and updates without any additional work done by the user. Many updates are going to be small changes and bug fixes.
Question: Does anyone happen to know how BlishHud handles updates?

I agree with your statement that the version number should be in the format that is supported my mod managers. I find Symver the easiest human readable that conveys something about the state of the project as well. I think the biggest thing to gain from manual releases is publishing a change log of the updates that have been made. Perhaps we could do those for larger changes or critical bug fixes.

[Masgalor]

BlishHud runs a self-update-service, which does what you are suggesting for automated updates.

I am honestly a bit puzzled about the fact that we, as a linux native software, do not use packages (including flatpak) to distribute our software. If something is solved on linux it is packaging, no?

[Masgalor]

For the distribution of burrito itself. This was not part of my initial post, but it came up a few times already.
I said this before, and I say it again, this software is exclusive to linux, every distribution has a working package management solution, why not use them?

Building packages is by far no witchcraft I am packaging software for rpm & deb distributions for years (including burrito) . @Technetium1 seems to be involved in Arch, so we only need someone who can help us with flatpak and we got the whole ecosystem covered, including steam deck.

[AsherGlick]

I think this is a bit off topic from the original discussion point. Your thread from last year is probably a better place to continue that discussion.

[AsherGlick]

Here is what I am thinking we do.

There are two types of releases, "weekly" and "major".

Weekly - Every Tuesday. No release note, or a list of PRs merged in since the last release. Updates the <minor> version of the semver.

• 1.1.0
• 1.2.0
• 1.3.0

Major - Likely infrequent. Triggered manually. Contains manually written release notes. Updates the <major> version of the semver.

• 1.x.x
• 2.x.x
• 3.x.x

Version numbers are tracked via git tags. Tags will use the pattern release-<major>.<minor>.<revision>.

Tasks

• New Weekly workflow

  • Runs every Tuesday
  • Runs the workflow to build the project

• Weekly workflow automatic version tags

  • Determine the current release number by looking at the most recent commit with a tag in the format
    • /release-(?P<major>[0-9]+)\.(?P<minor>[0-9]+)\.(?P<revision>[0-9]+)/
  • Puts a new tag on the current commit with the format
    • release-{major}.{minor+1}.0

• Don't run weekly workflow if there are no new commits

  • Detect if there are any commits that have been added since the previous release, and does not run if there are not

• Weekly workflow releases

  • Weekly workflow creates a new GitHub release based on the tag it made

• Ignore non user facing releases

  • Weekly workflow compares newly built files against previous weekly's files
  • If they are the same no release is cut
  • Maybe some indicator or tag is moved to the most recent commit when this happens so the next workflow only checks for new commits between this one and that one instead of between the last cut build and that one

• Automatic Descriptions (even though they are less good then manual ones)

  • Weekly workflow finds and includes the names and links of any PRs, that were newly included, in that build in the description

• New "Major" workflow that is run manually using workflow_dispatch

  • It takes in the description of the release as a parameter
  • The major workflow finds the last tag with the format
    • /release-(?P<major>[0-9]+)\.(?P<minor>[0-9]+)\.(?P<revision>[0-9]+)/
  • Then puts a new tag on the current commit with the format
    • release-{major+1}.0.0
  • builds the code
  • creates a new release
  • uploads the newly build code to the release
  • sets the description based on the parameter

@Masgalor @klingbolt what do you think about this? It is likely one or both will be writing this code assuming the xml_converter stuff wraps up soon freeing kling, while my next goal is the vm graphics stack testing harness.

[Technetium1]

This seems to correctly follow semantic versioning, lgtm
https://semver.org/

[AsherGlick]

Just to confirm, We would then use the revision for manual releases to address bug fixes and the like correct?

Good point, we probably will need a second manual process that is pretty identical to the "major" release but updates the "build" number manually instead of the major number. Ill be sure to add that in when I put this into a metaissue and make all the sub issues now that everyone who participated here seems to like this idea going forwards.

[AsherGlick]

This seems to correctly follow semantic versioning, lgtm https://semver.org/

This will look like semver but it wont actually be semver. In semver the "major" number is the only time that breaking changes are allowed, but we might have breaking changes in a "minor" release because they get merged in at some point. I more see it as "stable"."probably_stable"."bugfix" then semver. The minor/weekly releases could maybe even be considered prereleases of the next major

[AsherGlick]

This is now tracked in the issue #333

[Technetium1]

@AsherGlick if minors can still [accidentally] have breaking changes, it would be good to mention in readme.