{shoko-webui,shoko,nixos/shoko}: init

[diniamo]

Adds a NixOS module for https://github.com/ShokoAnime/ShokoServer, along with derivations for the module's dependencies.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[diniamo]

The formatting check fails for the generated dependencies file, which I don't believe should be formatted.

[gepbird]

cc @NixOS/nix-formatting, the deps.nix file generated by nuget-to-nix is not formatted by default

[diniamo]

To be fair, I like its formatting way more. But I guess it should be formatted then?

[dasJ]

For the nuget thing: what about:

diff --git a/pkgs/build-support/dotnet/add-nuget-deps/default.nix b/pkgs/build-support/dotnet/add-nuget-deps/default.nix
index bf3df815af8a..12810e092d03 100644
--- a/pkgs/build-support/dotnet/add-nuget-deps/default.nix
+++ b/pkgs/build-support/dotnet/add-nuget-deps/default.nix
@@ -8,6 +8,7 @@
   cacert,
   fetchNupkg,
   callPackage,
+  nixfmt-rfc-style,
 }:

 {
@@ -67,6 +68,7 @@ attrs
             isExecutable = true;
             inherit cacert;
             nugetToNix = nuget-to-nix;
+            nixfmt = nixfmt-rfc-style;
           };

           defaultDepsFile =
diff --git a/pkgs/build-support/dotnet/add-nuget-deps/fetch-deps.sh b/pkgs/build-support/dotnet/add-nuget-deps/fetch-deps.sh
index af560b96fcd5..47731b29cd59 100644
--- a/pkgs/build-support/dotnet/add-nuget-deps/fetch-deps.sh
+++ b/pkgs/build-support/dotnet/add-nuget-deps/fetch-deps.sh
@@ -7,5 +7,6 @@ genericBuild
     @nugetToNix@/bin/nuget-to-nix "${NUGET_PACKAGES%/}"
 ) > deps.nix

+@nixfmt@/bin/nixfmt deps.nix
 mv deps.nix "$1"
 echo "Succesfully wrote lockfile to $1"

[diniamo]

The nuget formatting thing isn't in the scope of this PR. Although it would be better to generate formatted code, instead of formatting it after.

[dasJ]

Sorry, that was unclear. I was suggesting that solution to the formatting team you pinged, not to you for this very PR ;)

[revam]

Just a note that you will need to provide your own tmdb api key in the setting when building from source outside our CD pipeline, and the version number and release channel are currently not set while building the binary in this patch set. See the dockerfile in the server repo, or the GH action(s) in the server repo for how to include them, if desired.

[diniamo]

The test fails because Shoko is trying to access its settings file (/var/lib/shoko/settings-server.json), but fails. Not sure how I can fix that.

[gepbird]

nixosTests.shoko is hanging, the relevant error there:

The server failed to start: System.UnauthorizedAccessException: Access to the path '/var/lib/shoko/settings-server.json' is denied.

Also adding an update script would be nice, for the webui package passthru.updateSript = nix-update-script { }; should be sufficient, the shoko package uses dotnet which doesn't have a clean solution yet, but this should work:

  passthru.updateScript = _experimental-update-script-combinators.sequence [
    (gitUpdater { }).command
    (finalAttrs.passthru.fetch-deps)
  ];

[diniamo]

That should be most of the stuff, except

• the test fail, which I have no clue why it exists, nor how to fix it
• I'd rather not add a hacky update script to the main program

[diniamo]

Updated to 5.1.0 and 2.2.0

[gepbird]

the test fail, which I have no clue why it exists, nor how to fix it

Explanation of how can you debug it

This could be useful to run commands in the VM without having to rebuild it every time: https://wiki.nixos.org/wiki/NixOS_VM_tests#Connecting_to_an_interactive_VM_via_SSH
After copying the ssh config from that wiki into nodes.machines in the shoko nixos test, for some reason I can't connect to it when the machine is ran by nixbld (this happens when you do nix-build -A nixosTests.shoko or the same with nix build ...). To fix that I can build the VM, and run it as my user with $($(nix-build -A nixosTests.shoko.driver)/bin/nixos-test-driver). Then I can get a shell inside the VM with ssh root@localhost -p 2000.

Inside the VM, the shoko service throws Access to the path '/var/lib/shoko/settings-server.json' is denied.. This file doesn't exists, there are only webui files:

[root@machine:~]# find -L /var/lib/shoko
/var/lib/shoko
/var/lib/shoko/webui
/var/lib/shoko/webui/favicon.ico
/var/lib/shoko/webui/version.json
/var/lib/shoko/webui/assets
/var/lib/shoko/webui/assets/sora-latin-100-normal-wz_LKC25.woff2
/var/lib/shoko/webui/assets/sora-latin-400-normal-DZtuNFrm.woff2
/var/lib/shoko/webui/assets/sora-latin-200-normal-CSO-6fll.woff2
/var/lib/shoko/webui/assets/sora-latin-500-normal-D6ntt6OU.woff2
/var/lib/shoko/webui/assets/sora-latin-300-normal-sH7t8am-.woff2
/var/lib/shoko/webui/assets/sora-latin-600-normal-C1m6bnBo.woff2
/var/lib/shoko/webui/assets/sora-latin-700-normal-CcRYJWjj.woff2
/var/lib/shoko/webui/assets/sora-latin-800-normal-B4rKiZhk.woff2
/var/lib/shoko/webui/assets/OnePiece-CCVDVjBa.png
/var/lib/shoko/webui/assets/sora-latin-100-normal-Db8iwt7l.woff
/var/lib/shoko/webui/assets/sora-latin-200-normal-E31gOQMh.woff
/var/lib/shoko/webui/assets/sora-latin-500-normal-CmPS9DAJ.woff
/var/lib/shoko/webui/assets/sora-latin-600-normal-DC-VVm69.woff
/var/lib/shoko/webui/assets/sora-latin-700-normal-7BwG48_h.woff
/var/lib/shoko/webui/assets/sora-latin-800-normal-PDlJt_sS.woff
/var/lib/shoko/webui/assets/sora-latin-400-normal-woMVPSgH.woff
/var/lib/shoko/webui/assets/monaco-editor-CWFJhXPe.css
/var/lib/shoko/webui/assets/index-CjvHA5ht.css
/var/lib/shoko/webui/assets/sora-latin-300-normal-_gcayfTt.woff
/var/lib/shoko/webui/assets/shoko_mascot-BxR7aIJM.png
/var/lib/shoko/webui/assets/@monaco-editor
/var/lib/shoko/webui/assets/@monaco-editor/react-CmuKTs2V.js
/var/lib/shoko/webui/assets/@monaco-editor/react-CmuKTs2V.js.map
/var/lib/shoko/webui/assets/common-0jV1spn5.js
/var/lib/shoko/webui/assets/monaco-editor-gpLA3-Rd.js
/var/lib/shoko/webui/assets/vendor-DsPoBL21.js
/var/lib/shoko/webui/assets/index-s1tCB-Bw.js
/var/lib/shoko/webui/assets/RenamerEditor-BgClqkFS.js
/var/lib/shoko/webui/assets/javascript-BrYQgcVz.js
/var/lib/shoko/webui/assets/lua-nf6ki56Z.js
/var/lib/shoko/webui/assets/typescript-Bu_7SVrm.js
/var/lib/shoko/webui/assets/yaml-CqiVQN8T.js
/var/lib/shoko/webui/assets/typescript-Bu_7SVrm.js.map
/var/lib/shoko/webui/assets/yaml-CqiVQN8T.js.map
/var/lib/shoko/webui/assets/lua-nf6ki56Z.js.map
/var/lib/shoko/webui/assets/RenamerEditor-BgClqkFS.js.map
/var/lib/shoko/webui/assets/javascript-BrYQgcVz.js.map
/var/lib/shoko/webui/assets/common-0jV1spn5.js.map
/var/lib/shoko/webui/assets/index-s1tCB-Bw.js.map
/var/lib/shoko/webui/assets/vendor-DsPoBL21.js.map
/var/lib/shoko/webui/assets/monaco-editor-gpLA3-Rd.js.map
/var/lib/shoko/webui/assets/editor.worker-oRlJJsnX.js.map
/var/lib/shoko/webui/assets/editor.worker-oRlJJsnX.js
/var/lib/shoko/webui/index.html

Since the file doesn't exist and dotnet throws an access denied error, most likely the parent directory has permission issues:

[root@machine:/]# stat /var/lib/shoko/
  File: /var/lib/shoko/
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: 253,0	Inode: 168         Links: 2
Access: (0700/drwx------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-01-20 11:03:11.662780450 +0000
Modify: 2025-01-20 11:01:03.098375196 +0000
Change: 2025-01-20 11:01:03.098375196 +0000
 Birth: 2025-01-20 11:01:03.085373543 +0000

Conclusion/TLDR

/var/lib/shoko is only accessible by root, when the service is running with the shoko user and group correctly. To fix this, you can set the user and group for the directory:

diff --git a/nixos/modules/services/misc/shoko.nix b/nixos/modules/services/misc/shoko.nix
index de000171f828..ebc7ed82e19d 100644
--- a/nixos/modules/services/misc/shoko.nix
+++ b/nixos/modules/services/misc/shoko.nix
@@ -70,7 +70,7 @@ in
     #    imperative plugins as well), then symlink the specified plugins.
     system.activationScripts.shoko =
       ''
-        install -dm700 '${cfg.dataDir}'
+        install -dm700 -o shoko -g shoko '${cfg.dataDir}'
       ''
       + optionalString (cfg.webui != null) ''
         rm -rf '${cfg.dataDir}/webui'

[diniamo]

Yeah, I've found it since, but the fix won't be that simple since I gotta redo some stuff anyway

[diniamo]

Alright, that should do it. @gepbird I'd appreciate some testing, if you have the time.

[gepbird]

Test works now and changes LGTM, thanks!

I was a bit bothered by making filesystem changes in systemd.activationScript and tried to replace it with systemd.tmpfiles.rules and systemd.services.shoko.serviceConfig.ExecStartPre but failed. Your solution with StateDirectory and preStart is much cleaner, nicely done.