File with ID ... doesn't exist anymore, skipping it for files that exists

[priyankub]

Hey, Thanks for reporting issues back to Nextcloud Face Recognition. Please, try to complete this report in detail so we can help you easier. 😄

Make sure you read all the documentation, and the FAQ, and that the issue has not been reported before. 😉

Expected behaviour

Process files that were not marked stale, because they exist

Actual behaviour

File do not exist anymore, skipping it...

Steps to reproduce

1. Start nextcloud
2. Let it scan

More details in background log section

Server configuration

• Operating system:
  Unraid

• Pdlib version:
  ... Nextcloud AIO does not support

• How is DLib installed: Make sure it is working correctly with this tool

• How is PDlib installed: Make sure it is working correctly with this tool

• PHP version:
  8.2.25

• Web server:
  Apache - standard AIO

• Database:
  Postgres

• Nextcloud version:

Client configuration

• Browser:
  Chrome
• Operating system:
  Mac

Logs

Background task log with debug.

sudo -u apache php occ -vvv face:background_job

        File with ID 5778421 doesn't exist anymore, skipping it
        File with ID 477886 doesn't exist anymore, skipping it
        File with ID 4056430 doesn't exist anymore, skipping it
        File with ID 2352789 doesn't exist anymore, skipping it
        File with ID 489257 doesn't exist anymore, skipping it
        File with ID 1403850 doesn't exist anymore, skipping it
        File with ID 471549 doesn't exist anymore, skipping it
        File with ID 3825816 doesn't exist anymore, skipping it

nextcloud_database=> SELECT fileid, storage, path FROM oc_filecache WHERE fileid IN (2352789, 489257, 1403850, 471549, 3825816);
 fileid  | storage |                                           path                                           
---------+---------+------------------------------------------------------------------------------------------
  471549 |       7 | files/<redacted> stuff/Photos/Manipal/Hanging bridge/IMG-20161204-WA0007.jpg
  489257 |       7 | files/<redacted> stuff/<redacted> Camera/DSC_0377.jpg
 1403850 |       7 | files/<redacted> stuff/Photos/Bangalore feat Bosch/Bangalore Jam/Jam in my room blr/DSC_0450.JPG
 2352789 |       7 | files/<redacted> stuff/Iphoneautoupload/21-03-11 17-42-51 3858.heic
 3825816 |       7 | files/<redacted> stuff/Photos/United States/TexasA&M/<redacted> RA treat/IMG_0687.jpg
(5 rows)

nextcloud_database=> exit
could not save history to file "/var/lib/postgresql/.psql_history": Read-only file system
dc6bd59f1662:/$ exit
exit
root@Param:~# ls -l /mnt/user/nextcloud/<redacted>/files/<redacted>\ stuff/Photos/Manipal/Hanging\ bridge/IMG-20161204-WA0007.jpg
-rwxr-xr-x 1 sshd root 416898 Aug 19  2017 /mnt/user/nextcloud/<redacted>/files/<redacted>\ stuff/Photos/Manipal/Hanging\ bridge/IMG-20161204-WA0007.jpg*

Web server error log

Web server error log

Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log

Insert your Nextcloud log here

Browser log

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

[priyankub]

If helpful:

 docker exec -it -u www-data nextcloud-aio-nextcloud php occ face:stats
+------------+--------+-----------+-------+----------+---------+
| User       | Images | Processed | Faces | Clusters | Persons |
+------------+--------+-----------+-------+----------+---------+
| admin      | 9      | 9         | 47    | 47       | 0       |
| user1     | 14416  | 14374     | 9009  | 4693     | 21      |
| user2      | 3902   | 3902      | 7817  | 3222     | 10      |
| user3     | 0      | 0         | 0     | 0        | 0       |
| user4 | 17961  | 4601      | 7209  | 3280     | 0       |
| user5  | 2623   | 279       | 351   | 0        | 0       |
| user6   | 7793   | 7791      | 5505  | 1091     | 70      |
+------------+--------+-----------+-------+----------+---------+

I love this app! I primarily use and support Photoprism for all my photos needs (>300k photos), but really like to be able to handle most use cases in Nextcloud where it is easier to share and access control etc.
I will be happy to work on features to be able to offload processing to a container with GPU access for users with more capable systems...

[Oski19831]

Hi @priyankub
Is there a reason why your webserver cant write to the directory or the file... your ls command show that the owner is sshd:root and your webserver runs as apache. The permission is only for the owner read and write. for Group and other userers ther is only read permission. I don´t know if this can fix the problem. But in my opinion this is a Problem for Nextcloud.

[priyankub]

Thanks for checking!

I really wish it was that simple, but likely not. The file is owned by 33:0. In my server, 33:0 is sshd and root:

getent passwd 33
getent passwd 0
sshd:x:33:33:sshd:/:/bin/false
root:x:0:0:Console and webGui login account:/root:/bin/bash

After exec into the docker, that is www-data and root, which is what Nextcloud recommends file permissions should be. Inside nextcloud container:

root@Param:~# docker exec -it -u www-data nextcloud-aio-nextcloud bash
4cfd10f6908c:/var/www/html$ getent passwd 33
www-data:x:33:33:Linux User,,,:/home/www-data:/sbin/nologin
4cfd10f6908c:/var/www/html$ getent passwd 0
root:x:0:0:root:/root:/bin/sh
4cfd10f6908c:/var/www/html$ ls -l /mnt/ncdata/<redacted>/files/<redacted>\ stuff/Photos/Manipal/Hanging\ bridge/IMG-20161204-WA0007.jpg 
-rwxr-xr-x 1 www-data root 416898 Aug 19  2017 '/mnt/ncdata/<redacted>/files/<redacted> stuff/Photos/Manipal/Hanging bridge/IMG-20161204-WA0007.jpg'
4cfd10f6908c:/var/www/html$ 

[Oski19831]

but you wrote in your first Post that u execute sudo -u apache .... or is this a mistake?

[priyankub]

but you wrote in your first Post that u execute sudo -u apache .... or is this a mistake?

That's a remnant of the issue template left there as is

[deftmartian]

@priyankub. Did you sort this out? I'm seeing a similar issue. My permissions for ncdata are 750 as recommended. Error says file doesn't exist, but when db is checked, it does exist.

What is interesting is that it's not for all files. I'm scanning all my photos, which are in a group folder. One subfolder doesn't have any issues, the rest do. The permissions, and owner are the same.