nextcloud: add recognize app #333545

beardhatcode · 2024-08-09T22:17:55Z

Description of changes

Cleans up the way nextcloud(28|29|30)Packages etc is defined. And adding a package for recognize with support for libtensorflow (darwin suport and GPU stuff is for another PR by someone who has those kinds of machines).

On my x86 linux it now says:

✔️ The machine learning models have been downloaded successfully.

✔️ Node.js v20.14.0 binary was installed successfully.

✔️ Libtensorflow was loaded successfully into Node.js.

And it has

✔️ Face recognition is working.
✔️ Object recognition is working.
✔️ Audio recognition is working.
✔️ Video recognition is working.

old intro

Alters the way nextcloud28Packages etc is defined to allow adding custom packages.

In this case to add the recognize app which downloads the ML models at runtime (which is not compatible with the read only filesystem). Additionally, I also patch in the version of nodejs (in a very dirty way, for now). I tested it and it says

✔️ The machine learning models have been downloaded successfully.

and

✔️ Node.js v20.14.0 binary was installed successfully.

but also

⚠️ Could not load libtensorflow in Node.js. You can try to manually install libtensorflow or run in WASM mode.

This is mostly a request for comments on of how a thing like this can be done. So, @Ma27, @dotlambda, @onny, and @pyrox0 (the last 4 editors of the Nextcloud packages folder), what do you think?

Things done

Add a 👍 reaction to pull requests you find important.

teutat3s · 2024-08-15T07:50:49Z

Related PR: #211631

pkgs/servers/nextcloud/packages/default.nix

pkgs/servers/nextcloud/packages/adapted/nextcloud-app-recognize.nix

SuperSandro2000 · 2024-08-18T12:16:33Z

This means we don't need something like https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L172-L183 but still https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L210-L217 ?

Ma27

This means we don't need something like https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L172-L183 but still https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L210-L217 ?

To be clear: I'm very, very 👎 on doing any build steps at runtime.

I expect the JS code to be built correctly and to have the settings patched by this package to point to other Nix packages (I.e. node, ffmpeg etc.). I looked through those super-questionable[0] "migration" "steps"[1] and I really don't want to rely on any of that.

[0] to say the least
[1] https://github.com/nextcloud/recognize/blob/main/lib/Migration/InstallDeps.php

pkgs/servers/nextcloud/packages/default.nix

beardhatcode · 2024-08-18T14:55:41Z

This means we don't need something like https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L172-L183 but still https://github.com/NuschtOS/nixos-modules/blob/da5286bc062adee0e0aaf2bd3b784b477c623422/modules/nextcloud.nix#L210-L217 ?

I was planning to wrap the nodejs to add those packages so they do not need to be in the module. This way it "just works" if you enable this. But I'm still figuring out how to do this nicely.

onny · 2024-08-27T17:41:55Z

I'm also going to add a third party app to the package set #328634

SuperSandro2000 · 2024-08-28T09:59:05Z

To be clear: I'm very, very 👎 on doing any build steps at runtime.

I don't use declerative nextcloud apps, so I must do it like this.

So we want to patch the app before in nextcloudApps.

Ma27 · 2024-08-28T16:55:08Z

I don't use declerative nextcloud apps, so I must do it like this.

That's OK, my comment was strictly about what I'd like to see (or not see) in nixpkgs.

So we want to patch the app before in nextcloudApps.

Most likely yes.

onny · 2024-09-11T08:24:16Z

We managed to package third-party apps in #328634

So you can add your derivation into pkgs/servers/nextcloud/packages/thirdparty.nix

beardhatcode · 2024-09-24T19:54:04Z

We managed to package third-party apps in #328634

So you can add your derivation into pkgs/servers/nextcloud/packages/thirdparty.nix

Rebased on that. This app is not really third-party however.

Unfortunately, I'm still unable to get the tfjs stuff to compile under nix as this thing seems to insist on downloading stuff. So I'm still stuck on:

Could not load libtensorflow in Node.js. You can try to manually install libtensorflow or run in WASM mode.

If anyone knows how to fix it, that would be much appreciated.

Ma27 · 2024-09-24T20:44:01Z

This app is not really third-party however.

Yeah, we should probably change the naming.
What I'm wondering is, would it be possible to track the version/src via nc4nix and only do overrides in this file?

beardhatcode · 2024-09-25T08:03:28Z

Yea, I think that should be possible. I'd like to first get the tfjs node thing working. Because atm, what I have looks horrible, and it does not even work:

{
  stdenv,
  fetchurl,
  lib,
  nodejs,
  python311,
  python311Packages,

  util-linux, # needed?
  ...
}:
stdenv.mkDerivation rec {
  pname = "nextcloud-app-recognise";
  version = "6.0.1";

  srcs = [
    (fetchurl {
      inherit version;
      url = "https://github.com/nextcloud/recognize/releases/download/v${version}/recognize-${version}.tar.gz";
      hash = "sha256-HQHfcQY2N6INLfaLDWbWn0SSzvkSZqL0rPl2koqej6c=";
    })

    (fetchurl {
      inherit version;
      url = "https://github.com/nextcloud/recognize/archive/refs/tags/v${version}.tar.gz";
      hash = "sha256-4iJVKz/HfzIUouiOK/NUkJUm4+p9meVptQwJ4OG1JaU=";
    })
    (fetchurl {
      inherit version;
      url = "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-2.9.1.tar.gz";
      hash = "sha256-f1ENJUbj214QsdEZRjaJAD1YeEKJKtPJW8pRz4KCAXM=";
    })
    (fetchurl {
      inherit version;
      url = "https://nodejs.org/download/release/v20.15.1/node-v20.15.1-headers.tar.gz";
      hash = "sha256-jCMFxt9dFFJeBxHw2jgpVgCYffTCcQxzjAFACGKhdrQ=";
    })

  ];

  unpackPhase = ''
    tar -xzpf "${builtins.elemAt srcs 0}" recognize;
    tar -xzpf "${builtins.elemAt srcs 1}" recognize-${version}/models;
    mv recognize-${version}/models recognize
    #cd recognize/node_modules/@tensorflow/tfjs-node/deps
    #tar -xzpf "${builtins.elemAt srcs 2}"
    #cd -
    cd recognize/node_modules/@tensorflow/tfjs-node/lib
    tar -xzpf "${builtins.elemAt srcs 3}"
    cd -

    # recognize/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.so.2.9.1

  '';

  buildInputs = [
    nodejs
    nodejs.pkgs.node-pre-gyp
    nodejs.pkgs.node-gyp
    python311
    util-linux
  ];
  buildPhase = ''
    mkdir tmphome
    export HOME="$(realpath tmphome)"
    sed -i "/'node_binary'/s:'""':'${nodejs}/bin/node':" recognize/lib/Service/SettingsService.php
    cd recognize
    set -x
    #ln -sf -t node_modules/@tensorflow/tfjs-node/deps ${python311Packages.tensorflow}/*
    ls node_modules/@tensorflow/tfjs-node/deps
    export CPPFLAGS="-I${nodejs}/include/node -Ideps/include -I${python311Packages.tensorflow}/lib/python3.11/site-packages/tensorflow/include"
    cd node_modules/@tensorflow/tfjs-node
    echo "===="
    ls deps/include
    echo "===="
    node-pre-gyp install --prefer-offline --build-from-source --nodedir=${nodejs}/include/node
    cd -
    #npm rebuild --prefer-offline @tensorflow/tfjs-node --build-addon-from-source
    ${nodejs}/bin/node src/test_libtensorflow.js
    set +x
    exit 1
    cd -
  '';
  installPhase = ''
    set -x
    approot="$(dirname $(dirname $(find -path '*/appinfo/info.xml' | head -n 1)))"
    if [ -d "$approot" ];
    then
      mv "$approot/" $out
      chmod -R a-w $out
    fi
  '';

  meta = with lib; {
    license = licenses.agpl3Only;
    maintainers = with maintainers; [ beardhatcode ];
    longDescription = ''
      Nextcloud app that does Smart media tagging and face recognition with on-premises machine learning models.
      This app goes through your media collection and adds fitting tags, automatically categorizing your photos and music.
    '';
    homepage = "https://apps.nextcloud.com/apps/recognize";
  };
}

teutat3s · 2024-09-25T17:15:11Z

@beardhatcode I assume you already saw the hacky workaround in this comment? If not, I figured it might be useful to make the node tfjs thing build.

beardhatcode · 2024-09-26T07:53:43Z

@beardhatcode I assume you already saw the hacky workaround in this comment? If not, I figured it might be useful to make the node tfjs thing build.

Yes I did, but I would like something reproducible. It must be possible.

beardhatcode · 2024-11-22T14:01:44Z

Finally got it working, now to clean it up and rebase on master

pkgs/servers/nextcloud/packages/apps/recognize.nix

beardhatcode · 2024-11-24T15:39:02Z

@Ma27 and @SuperSandro2000 , for me, this PR is complete (unless there is something I missed).

Ma27 · 2024-11-30T13:18:29Z

Skimmed over it last week and it looks good so far.
Just need to find some time to try it out :)

beardhatcode · 2024-12-07T14:07:13Z

Has been working for me on my server for a couple of weeks now

beardhatcode · 2024-12-09T12:55:01Z

I added support for older Nextclouds as well now, see https://apps.nextcloud.com/apps/recognize#downloads.
And I tested all these version with a couple of files.

britter · 2024-12-11T09:06:34Z

Can't wait to see this merged!

pkgs/servers/nextcloud/packages/apps/recognize.nix

SuperSandro2000 · 2024-12-13T14:45:56Z

@Ma27 we want to merge this?

fixed

dotlambda

meta.changelog and meta.description are missing

dotlambda · 2024-12-17T16:58:21Z

pkgs/servers/nextcloud/packages/apps/recognize.nix

+    test "$(grep "get[a-zA-Z]*('node_binary'" recognize/lib/**/*.php | wc -l)" -gt 0
+    substituteInPlace recognize/lib/**/*.php \
+      --replace-quiet "\$this->settingsService->getSetting('node_binary')" "'${nodejs}/bin/node'" \
+      --replace-quiet "\$this->config->getAppValueString('node_binary', '""')" "'${nodejs}/bin/node'" \


Suggested change

--replace-quiet "\$this->config->getAppValueString('node_binary', '""')" "'${nodejs}/bin/node'" \

--replace-quiet "\$this->config->getAppValueString('node_binary', '""')" "'${lib.getExe nodejs}'" \

dotlambda · 2024-12-17T16:58:46Z

pkgs/servers/nextcloud/packages/apps/recognize.nix

+    # Skip trying to install it... (less warnings in the log)
+    sed  -i '/public function run/areturn ; //skip' recognize/lib/Migration/InstallDeps.php
+
+    ln -s ${ffmpeg}/bin/ffmpeg recognize/node_modules/ffmpeg-static/ffmpeg


Suggested change

ln -s ${ffmpeg}/bin/ffmpeg recognize/node_modules/ffmpeg-static/ffmpeg

ln -s ${lib.getExe ffmpeg} recognize/node_modules/ffmpeg-static/ffmpeg

dotlambda · 2024-12-17T16:59:55Z

pkgs/servers/nextcloud/packages/apps/recognize.nix

+    nodejs.pkgs.node-pre-gyp
+    nodejs.pkgs.node-gyp


These are only aliases for top-level attributes.

Suggested change

nodejs.pkgs.node-pre-gyp

nodejs.pkgs.node-gyp

node-pre-gyp

node-gyp

dotlambda · 2024-12-17T17:00:15Z

pkgs/servers/nextcloud/packages/apps/recognize.nix

+    cd recognize
+
+    # Install tfjs dependency
+    export CPPFLAGS="-I${nodejs}/include/node -Ideps/include"


Suggested change

export CPPFLAGS="-I${nodejs}/include/node -Ideps/include"

export CPPFLAGS="-I${lib.getDev nodejs}/include/node -Ideps/include"

teutat3s · 2024-12-18T18:05:43Z

pkgs/servers/nextcloud/packages/apps/recognize.nix

+in
+stdenv.mkDerivation rec {
+
+  pname = "nextcloud-app-recognise";


Typo snuck in here:

pname = "nextcloud-app-recognize";

teutat3s · 2024-12-18T18:06:53Z

Would really like to backport this to 24.11.

nixpkgs-ci · 2024-12-18T18:07:37Z

Successfully created backport PR for release-24.11:

[Backport release-24.11] nextcloud: add recognize app #366241

Ma27

error:
       … while evaluating an expression to select 'drvPath' on it
         at «internal»:1:552:
       … while calling the 'derivationStrict' builtin
         at «internal»:1:208:
       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attribute 'node-pre-gyp' missing
       at /home/ma27/Projects/nixpkgs/pkgs/servers/nextcloud/packages/apps/recognize.nix:100:17:
           99|     nodejs
          100|     nodejs.pkgs.node-pre-gyp
             |                 ^
          101|     nodejs.pkgs.node-gyp

I'm sorry that it takes a while until I get to things sometimes, but did nobody even bother to evaluate this against latest master before merging?

EDIT: OK to be fair, this is a deprecated alias which makes me wonder why CI didn't complain. My money is on missing recurseIntoAttrs somewhere. But still, this needs to change.

beardhatcode · 2024-12-24T02:22:23Z

I ran it on the latest master @Ma27 . Or at least that was what I thought I did, but maybe not. I did the following:

I rebased this PR on master, then I created and ran a vm for each of nextcloud{28,29,29} and I uploaded photos and videos and checked that it did what it should do. I also placed it on my server but that one is nixos-unstable.

Should I have done something additionally, or could you tell me how you bumped into this issue?

For future reference, #366247 should contain a fix for the alias thing.

beardhatcode force-pushed the feat/ncapps-recognise branch 3 times, most recently from 2c18298 to d37b446 Compare August 13, 2024 19:31

beardhatcode mentioned this pull request Aug 18, 2024

nextcloud25Packages.apps.recognize: init at 3.3.6 #211631

Closed

13 tasks

SuperSandro2000 reviewed Aug 18, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Aug 18, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/adapted/nextcloud-app-recognize.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Aug 18, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/adapted/nextcloud-app-recognize.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Aug 18, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/adapted/nextcloud-app-recognize.nix Outdated Show resolved Hide resolved

Ma27 previously requested changes Aug 18, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/default.nix Outdated Show resolved Hide resolved

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 10, 2024

beardhatcode force-pushed the feat/ncapps-recognise branch from d37b446 to c7ada99 Compare September 24, 2024 18:51

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 24, 2024

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 1, 2024

beardhatcode force-pushed the feat/ncapps-recognise branch 2 times, most recently from 31abb08 to 5043312 Compare November 22, 2024 13:59

Ma27 reviewed Nov 24, 2024

View reviewed changes

pkgs/servers/nextcloud/packages/apps/recognize.nix Show resolved Hide resolved

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 24, 2024

nextcloud30Packages.apps.recognize: 8.1.1->8.2.0

ed5608b

teutat3s requested a review from Ma27 December 9, 2024 11:42

nextcloudXXPackages.apps.recognize: suport nextcloud 29 and 28

0479e6e

SuperSandro2000 reviewed Dec 11, 2024

View reviewed changes

beardhatcode added 3 commits December 11, 2024 22:17

nextcloud30Packages.apps.recognize: patchPhase -> postPatch

115d3a6

nextcloudXXPackages.apps.recognize: use node from path

67a6a59

nextcloudXXPackages.apps.recognize: use python3 rather than 311

4a87aa2

beardhatcode requested a review from SuperSandro2000 December 12, 2024 10:27

nextcloud30Packages.apps.recognize: s/buildInputs/nativeBuildInputs/

7c9903f

SuperSandro2000 approved these changes Dec 13, 2024

View reviewed changes

wegank added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Dec 15, 2024

SuperSandro2000 merged commit 65aef64 into NixOS:master Dec 16, 2024
37 checks passed

dotlambda reviewed Dec 17, 2024

View reviewed changes

teutat3s reviewed Dec 18, 2024

View reviewed changes

teutat3s added the backport release-24.11 Backport PR automatically label Dec 18, 2024

nixpkgs-ci bot mentioned this pull request Dec 18, 2024

[Backport release-24.11] nextcloud: add recognize app #366241

Merged

1 task

teutat3s mentioned this pull request Dec 18, 2024

nextcloudXXPackages.apps.recognize: use lib.{getExe,getDev}, fix typo #366247

Merged

13 tasks

Ma27 reviewed Dec 22, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nextcloud: add recognize app #333545

nextcloud: add recognize app #333545

beardhatcode commented Aug 9, 2024 •

edited

Loading

teutat3s commented Aug 15, 2024

SuperSandro2000 commented Aug 18, 2024

Ma27 left a comment

beardhatcode commented Aug 18, 2024

onny commented Aug 27, 2024

SuperSandro2000 commented Aug 28, 2024

Ma27 commented Aug 28, 2024

onny commented Sep 11, 2024

beardhatcode commented Sep 24, 2024 •

edited

Loading

Ma27 commented Sep 24, 2024

beardhatcode commented Sep 25, 2024

teutat3s commented Sep 25, 2024

beardhatcode commented Sep 26, 2024

beardhatcode commented Nov 22, 2024

beardhatcode commented Nov 24, 2024

Ma27 commented Nov 30, 2024

beardhatcode commented Dec 7, 2024

beardhatcode commented Dec 9, 2024 •

edited

Loading

britter commented Dec 11, 2024

SuperSandro2000 commented Dec 13, 2024

dotlambda left a comment

dotlambda Dec 17, 2024

dotlambda Dec 17, 2024

dotlambda Dec 17, 2024

dotlambda Dec 17, 2024

teutat3s Dec 18, 2024

teutat3s commented Dec 18, 2024

nixpkgs-ci bot commented Dec 18, 2024

Ma27 left a comment •

edited

Loading

beardhatcode commented Dec 24, 2024 •

edited

Loading

	--replace-quiet "\$this->config->getAppValueString('node_binary', '""')" "'${nodejs}/bin/node'" \
	--replace-quiet "\$this->config->getAppValueString('node_binary', '""')" "'${lib.getExe nodejs}'" \

	ln -s ${ffmpeg}/bin/ffmpeg recognize/node_modules/ffmpeg-static/ffmpeg
	ln -s ${lib.getExe ffmpeg} recognize/node_modules/ffmpeg-static/ffmpeg

	export CPPFLAGS="-I${nodejs}/include/node -Ideps/include"
	export CPPFLAGS="-I${lib.getDev nodejs}/include/node -Ideps/include"

nextcloud: add recognize app #333545

nextcloud: add recognize app #333545

Conversation

beardhatcode commented Aug 9, 2024 • edited Loading

Description of changes

Things done

teutat3s commented Aug 15, 2024

SuperSandro2000 commented Aug 18, 2024

Ma27 left a comment

Choose a reason for hiding this comment

beardhatcode commented Aug 18, 2024

onny commented Aug 27, 2024

SuperSandro2000 commented Aug 28, 2024

Ma27 commented Aug 28, 2024

onny commented Sep 11, 2024

beardhatcode commented Sep 24, 2024 • edited Loading

Ma27 commented Sep 24, 2024

beardhatcode commented Sep 25, 2024

teutat3s commented Sep 25, 2024

beardhatcode commented Sep 26, 2024

beardhatcode commented Nov 22, 2024

beardhatcode commented Nov 24, 2024

Ma27 commented Nov 30, 2024

beardhatcode commented Dec 7, 2024

beardhatcode commented Dec 9, 2024 • edited Loading

britter commented Dec 11, 2024

SuperSandro2000 commented Dec 13, 2024

dotlambda left a comment

Choose a reason for hiding this comment

dotlambda Dec 17, 2024

Choose a reason for hiding this comment

dotlambda Dec 17, 2024

Choose a reason for hiding this comment

dotlambda Dec 17, 2024

Choose a reason for hiding this comment

dotlambda Dec 17, 2024

Choose a reason for hiding this comment

teutat3s Dec 18, 2024

Choose a reason for hiding this comment

teutat3s commented Dec 18, 2024

nixpkgs-ci bot commented Dec 18, 2024

Ma27 left a comment • edited Loading

Choose a reason for hiding this comment

beardhatcode commented Dec 24, 2024 • edited Loading

beardhatcode commented Aug 9, 2024 •

edited

Loading

beardhatcode commented Sep 24, 2024 •

edited

Loading

beardhatcode commented Dec 9, 2024 •

edited

Loading

Ma27 left a comment •

edited

Loading

beardhatcode commented Dec 24, 2024 •

edited

Loading