Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and improve the nix flake #512

Merged
merged 12 commits into from
Mar 29, 2024
Merged

Fix and improve the nix flake #512

merged 12 commits into from
Mar 29, 2024

Conversation

haraldh
Copy link
Contributor

@haraldh haraldh commented Mar 27, 2024

  • fix(flake): public assets were moved to kitsune/assets

  • feat(flake): use mold linker

  • fix(flake): fix build dependency for openssl
    Native deps need the tools which run on the build host (when cross
    compiling). Normal build deps are the libs linked to the binary.

    OPENSSL_NO_VENDOR ensures that openssl-sys uses the system lib.

    Remove the rest of the workarounds for openssl.

  • fix(flake): proper mkYarnPackage

  • feat(flake): add overlay test
    to test overlay.nix

  • fix(flake): skip more tests

  • feat(flake): use the crane nix lib
    https://crane.dev/

    enables caching of artifacts and potentially reducing build time.

    Even End to End(E2E) testing could be added:
    https://crane.dev/examples/end-to-end-testing.html

  • ci(flake): add nix github action

@haraldh haraldh changed the title Nix flake Improve the nix flake Mar 27, 2024
@haraldh haraldh changed the title Improve the nix flake Fix and improve the nix flake Mar 27, 2024
@haraldh haraldh marked this pull request as draft March 27, 2024 19:27
@haraldh haraldh force-pushed the nix_flake branch 2 times, most recently from 4d980cc to 35b5eda Compare March 27, 2024 19:33
@haraldh haraldh marked this pull request as ready for review March 27, 2024 19:34
@haraldh haraldh marked this pull request as draft March 27, 2024 20:12
haraldh added 4 commits March 27, 2024 21:13
Signed-off-by: Harald Hoyer <[email protected]>
Native deps need the tools which run on the build host (when cross
compiling). Normal build deps are the libs linked to the binary.

OPENSSL_NO_VENDOR ensures that openssl-sys uses the system lib.

Remove the rest of the workarounds for openssl.

Signed-off-by: Harald Hoyer <[email protected]>
@haraldh haraldh force-pushed the nix_flake branch 2 times, most recently from 6d625d1 to 9e69e96 Compare March 27, 2024 20:15
@haraldh haraldh marked this pull request as ready for review March 27, 2024 20:18
haraldh added 2 commits March 27, 2024 21:39
Signed-off-by: Harald Hoyer <[email protected]>
Signed-off-by: Harald Hoyer <[email protected]>
@haraldh
Copy link
Contributor Author

haraldh commented Mar 27, 2024

Want me to disable cargo test in the "Nix checks" action (nix build)? There is a cargo nextest already, and you probably don't want to edit flake.nix constantly to disable tests, which need docker

@aumetra
Copy link
Member

aumetra commented Mar 27, 2024

Yeah, I think testing inside the nix build isn't really needed since we have our CI to ensure buildability.

@haraldh
Copy link
Contributor Author

haraldh commented Mar 27, 2024

Done as a separate commit, so it is easy to revert.

flake.nix Outdated Show resolved Hide resolved
@aumetra
Copy link
Member

aumetra commented Mar 27, 2024

Thank you for introducing the crane library, I saw a bit about it but never really looked into it too much.
About the Nix build though, is it doing a full release build? The timings seem like it. Can we downgrade that to a debug build in CI maybe?

haraldh added 3 commits March 28, 2024 11:54
https://crane.dev/

enables caching of artifacts and potentially reducing build time.

Even End to End(E2E) testing could be added:
https://crane.dev/examples/end-to-end-testing.html

Signed-off-by: Harald Hoyer <[email protected]>
Left as a separate commit, so it is easy to revert.

Signed-off-by: Harald Hoyer <[email protected]>
@haraldh
Copy link
Contributor Author

haraldh commented Mar 28, 2024

I had a misconfig in the crane calls, so it did not use the cached dependency build... Fixed that. Additionally the last commit changes the default to the dev profile, which is the debug mode. This although populates the github cache with lots more payload, because the debug artifacts are way bigger.

@haraldh
Copy link
Contributor Author

haraldh commented Mar 28, 2024

Don't know why the build has failed.. no logs available. Maybe it ran out of RAM??

@aumetra
Copy link
Member

aumetra commented Mar 28, 2024

Just gonna rerun it, maybe it works now? If not, then I'll have a look.
Yeah, I'd assume either RAM or disk space. Either way it's not good. I really need to start profiling the compilation and figure out where all the RAM is going

@haraldh
Copy link
Contributor Author

haraldh commented Mar 28, 2024

I had to change the nix build TMPDIR for my system, because it builds in /tmp and this is a tmpfs

@haraldh
Copy link
Contributor Author

haraldh commented Mar 28, 2024

kitsune-cli> installing /build/source/target/debug/kitsune-cli
kitsune-cli> cp: error copying '/build/source/target/debug/kitsune-cli' to '/nix/store/yjnadx1arhqmg9yj4y955w8ar3x8hnyr-kitsune-cli-0.0.1-pre.6/bin/kitsune-cli': No space left on device
error (ignored): error: cannot unlink '/tmp/nix-build-kitsune-0.0.1-pre.6.drv-0/source/target/debug/deps': Directory not empty
error: writing to file: No space left on device

Seems like a 2 debug builds in parallel is too big :)

haraldh added 2 commits March 28, 2024 13:30
normal flake produces release code still.

Signed-off-by: Harald Hoyer <[email protected]>
and skip running nixci.

Signed-off-by: Harald Hoyer <[email protected]>
@haraldh haraldh requested a review from aumetra March 28, 2024 15:08
@aumetra
Copy link
Member

aumetra commented Mar 29, 2024

Thank you for working on this! This looks really good

@aumetra aumetra merged commit d68c5e3 into kitsune-soc:main Mar 29, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants