Support for Deterministic Builds in Redot

[benscodeworks]

Describe the project you are working on

I've been exploring the Redot repository and am eager to contribute where I can. However, I find it concerning that our current build pipelines are not entirely deterministic. To address this, I propose implementing Docker files for our GitHub pipelines. These files would provide a consistent and reproducible build environment, making it easier for developers to identify and install the necessary dependencies for Redot development.

This approach would not only ensure we are all using the same packages where possible but also lay the foundation for supporting additional operating systems in the future. While Ubuntu is currently the only Linux distribution in our pipeline, I believe we can make its build process more comprehensive and standardized as a starting point. Over time, this can serve as a model for extending support to other operating systems and further enhancing our development workflows.

Describe the problem or limitation you are having in your project

To ensure consistency, reliability, and reproducibility in the build and testing pipeline for Redot, this proposal suggests transitioning to deterministic build practices. Specifically, I recommend adopting pre-configured build images that include all necessary dependencies for building Redot, thereby reducing inconsistencies caused by environment-specific factors such as package manager configurations.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Currently, Redot's builds rely on operating system package managers to install dependencies. For example, the Linux build for Redot uses Ubuntu 22.04 as a base image, installing additional dependencies via the Ubuntu package manager. While this approach works in principle, it introduces variability in the build process due to:

1. Differences in dependency versions across package manager repositories.
2. Potential updates or changes in package configurations over time.
3. Incompatibilities or subtle differences between dependencies on other Linux distributions.

These factors, albeit minor, can result in inconsistencies between builds, complicate testing, and reduce confidence in the reproducibility of results. For a project involving contributions from a diverse and distributed community, such variability can impact both productivity and quality.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Proposal

To address these concerns, I propose the following measures:

1. Adopt Pre-configured Build Images

• Create and maintain standardized build images that include all dependencies required to build Redot.
• These images should be designed for deterministic builds, ensuring that the same inputs consistently produce the same outputs.

2. Expand Support Beyond Ubuntu

• While Ubuntu 22.04 is currently used as a base, the community benefits from supporting additional Linux distributions by applying the same pre-configured build image methodology.
• Examples include Fedora, Arch Linux, and Debian-based distributions, which cater to different user preferences.

3. Integrate Build Images in CI/CD Pipelines

• Update the CI/CD pipeline to utilize these pre-configured images for all builds and tests.
• This ensures that both developers and automated systems operate on identical environments, reducing variability and debugging effort.

4. Version Control for Build Images

• Establish a clear versioning system for build images, enabling traceability and facilitating updates when dependencies need to be upgraded.
• Maintain previous versions of build images to support long-term branches or legacy builds.

5. Documentation and Community Onboarding

• Provide clear documentation on how contributors can pull and use these build images locally to ensure consistency with the official builds.
• Encourage the community to adopt these practices to further enhance collaboration and quality.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not Applicable

Is there a reason why this should be core and not an add-on in the asset library?

The reason this should be part of the core rather than an add-on in the asset library is that deterministic builds directly impact the foundation of the project's development and distribution processes. Ensuring consistent and reproducible builds is essential for maintaining the integrity and reliability of Redot as a whole. This approach benefits not just developers but also the end users who rely on predictable and stable releases.

Making deterministic builds a core feature aligns with the principles of quality assurance and reduces variability in development, testing, and deployment environments. An add-on, while potentially helpful for some use cases, wouldn't address these systemic needs comprehensively. By integrating this solution into the core, we create a strong, unified baseline for all contributors and pipelines, enabling a more seamless and scalable path for future enhancements, including supporting additional operating systems.