Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pinchflat: init at v2025.1.14 #364135

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

charludo
Copy link

@charludo charludo commented Dec 11, 2024

Pinchflat is a self-hosted app for downloading YouTube content built using yt-dlp. You set up rules for how to download content from YouTube channels or playlists and it'll do the rest, periodically checking for new content. It also tags media in a Plex, Jellyfin, and Kodi compatible manner, and integrates with Sponsorblock.

This PR adds the package itself, as well as a NixOS module for configuring a server install of pinchflat.
This resolves #351127.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation 8.has: changelog 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Dec 11, 2024
@NixOSInfra NixOSInfra added the 12. first-time contribution This PR is the author's first one; please be gentle! label Dec 11, 2024
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes labels Dec 12, 2024
@charludo charludo force-pushed the init-pinchflat branch 2 times, most recently from 8613a7b to cdad1a1 Compare December 16, 2024 12:33
@ob7
Copy link
Contributor

ob7 commented Dec 16, 2024

I enabled the pinchflat service and it seems to work well, I'm archiving a channel now. However, I haven't reviewed the code, just testing if it works or not.

I do wonder if there is a way to automate the secrets file part, as that takes some additional manual set up that isn't clear if you never used pinchflat before. Effectively I needed to install elixir, and generate the key by running an interactive iex session and running :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64) then saving the output to the secrets file. Just a suggestion if possible for usability is to generate the secrets file if user doesn't supply one. Just a personal suggestion, not something that would effect merging AFAIK.

Other observations are the systemd log for the service is very verbose, and perhaps this is due to pinchflat itself I don't know. I do notice you have LOG_LEVEL = "info" as an example one could use for extraConfig. Perhaps enable a less verbose log level by default?

Again just my observations from an initial install/setup user perspective, not a review of the code or service implementation etc. In a nutshell, it seems to install and function as intended, nice work!

One other suggestion is perhaps note that the services.pinchflat.port is where user accesses the web gui, ie localhost:8945 by default, it may not be clear in the docs that that is where the user accesses pinchflat from.

This is a pretty cool tool though, nice job. Another idea is make it so channels to archive can be supplied in the configuration.nix, similar to how the ollama service allows ppl to declare what models they want downloaded by passing them to services.ollama.loadModels.

@ob7
Copy link
Contributor

ob7 commented Dec 17, 2024

Note I ran into an error when setting extraConfig:

error: attribute 'toString' missing
at /home/nixos/nixpkgs/nixos/modules/services/misc/pinchflat.nix:131:70:
   130|           "PHX_SERVER=true"
   131|         ] ++ lib.attrValues (lib.mapAttrs (name: value: name + "=" + lib.toString value) cfg.extraConfig);
      |                                                                      ^
   132|         EnvironmentFile = lib.optional (cfg.secretsFile != null) cfg.secretsFile;

I changed the value via

services.pinchflat.extraConfig = {
  LOG_LEVEL = "info";
}

as suggested in man configuration.nix

However I was able to make this work by modifying nixos/modules/services/misc/pinchflat.nix line 131
from

] ++ lib.attrValues (lib.mapAttrs (name: value: name + "=" + lib.toString value) cfg.extraConfig); 

to

] ++ lib.attrValues (lib.mapAttrs (name: value: name + "=" + builtins.toString value) cfg.extraConfig); 

@ob7
Copy link
Contributor

ob7 commented Dec 17, 2024

Another observation, services.pinchflat.mediaDir and services.pinchflat.dataDir depend on each other. If one changes dataDir without changing mediaDir, downloads break as default mediaDir is a subdirectory of dataDir. This is only apparent if one is checking systemd logs, as everything appears to be running fine in GUI other than the lack of growth in downloads/library size. Maybe just set it to $dataDir/media by default?... ie:

config = mkIf cfg.enable {
  services.pinchflat.mediaDir = lib.mkDefault "${cfg.dataDir}/media";

and user can still change the value if they please.

As it currently stands, if user changes dataDir and does not update mediaDir, mediaDir will no longer exist as a valid path, as the service does not create /var/lib/pinchflat/media in the absense of /var/lib/pinchflat.

@charludo
Copy link
Author

charludo commented Dec 17, 2024

However I was able to make this work by modifying nixos/modules/services/misc/pinchflat.nix line 131 from

] ++ lib.attrValues (lib.mapAttrs (name: value: name + "=" + lib.toString value) cfg.extraConfig); 

to

] ++ lib.attrValues (lib.mapAttrs (name: value: name + "=" + builtins.toString value) cfg.extraConfig); 

Oops, yes, thank you for catching this!

@charludo
Copy link
Author

Another observation, services.pinchflat.mediaDir and services.pinchflat.dataDir depend on each other. If one changes dataDir without changing mediaDir, downloads break as default mediaDir is a subdirectory of dataDir. This is only apparent if one is checking systemd logs, as everything appears to be running fine in GUI other than the lack of growth in downloads/library size. Maybe just set it to $dataDir/media by default?.

Absolutely, good call. Should have done that from the get-go.

@charludo
Copy link
Author

I do wonder if there is a way to automate the secrets file part, as that takes some additional manual set up that isn't clear if you never used pinchflat before. Effectively I needed to install elixir, and generate the key by running an interactive iex session and running :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64) then saving the output to the secrets file. Just a suggestion if possible for usability is to generate the secrets file if user doesn't supply one. Just a personal suggestion, not something that would effect merging AFAIK.

I will add a note in the docs for the option. You can use any string longer than 64 bytes, it doesn't appear to matter if they were generated like you / the docs describe.

But I also just double-checked and it looks like adding a RUN_CONTEXT=selfhosted env var would make pinchflat not check for a user-supplied secret, so I'll add an option for that.

Other observations are the systemd log for the service is very verbose, and perhaps this is due to pinchflat itself I don't know. I do notice you have LOG_LEVEL = "info" as an example one could use for extraConfig. Perhaps enable a less verbose log level by default?

Agreed, it's very verbose.

Another idea is make it so channels to archive can be supplied in the configuration.nix, similar to how the ollama service allows ppl to declare what models they want downloaded by passing them to services.ollama.loadModels.

I am afraid this one is not possible. The reason this can be done for e.g. ollama is that setting a my:modelName there simply executes ollama pull my:modelName on restart of the service, if that model is missing.
The extraConfig option added in this PR lets you set any and all of Pinchflat's settings, but pre-configuring the channels to download simply isn't one of them.

I checked the pinchflat repo, and it does not appear that such a feature request has been made in the past. But I honestly don't know if it even could be added at a reasonable amount of effort. Because it's not just the channels you need to define, you also need to define all the other download settings, as well as a media profile with all of its settings. These then need to be read in to the database, as well as have a mechanism to detect when old entries were removed, and so on.

I think this is just one of those services - like sonarr, jellyfin,... - that are sufficiently complex to configure that a fully stateless solution isn't feasible :(

But hey, thanks for your review! 🎉 I'll push the requested changes shortly.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 31, 2024
@charludo charludo changed the title pinchflat: init at v2024.12.10 pinchflat: init at v2025.1.3 Jan 11, 2025
@nix-owners nix-owners bot requested a review from r-vdp January 11, 2025 13:21
nixos/modules/misc/ids.nix Outdated Show resolved Hide resolved
@charludo charludo force-pushed the init-pinchflat branch 2 times, most recently from 33e1793 to 68c79ea Compare January 12, 2025 07:23
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 12, 2025
@charludo charludo requested a review from r-vdp January 15, 2025 09:20
@charludo charludo changed the title pinchflat: init at v2025.1.3 pinchflat: init at v2025.1.14 Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog 8.has: documentation This PR adds or changes documentation 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 11.by: package-maintainer This PR was created by the maintainer of the package it changes 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Package request: pinchflat
5 participants