Externally deleted files uploaded again (because external app isn't really deleting them)

[simon-eller]

⚠️ Before posting ⚠️

• This is a bug, not a question or an enhancement.
• I've searched for similar issues and didn't find a duplicate.
• I've written a clear and descriptive title for this issue, not just "Bug" or "Crash".
• I agree to follow Nextcloud's Code of Conduct.

Steps to reproduce

Configure auto upload in the Nextcloud app for the camera folder of the smartphone
Shoot a photo and wait until it's uploaded on the Nextcoud server
Delete the photo in the Samsung gallery app
Go to your Nextloud app and click on "show hidden files" option
Now you can see that the photo was uploaded twice

Expected behaviour

If the photo was deleted on the smartphone, nothing should actually happen with the current options on the Nextcloud itself.

Whether the photo should then also be deleted on the Nextcloud has already been discussed in a number of other issues.

Actual behaviour

If you delete a photo in Samsung Gallery, it will not be deleted directly. Instead, the file is renamed and only really deleted after 30 days (because of the recycle bin function).

Original file name: 20220717_1234.jpg
Deleted file name: .trashed--20220717_1234.jpg

The deleted file will no longer appear in Samsung Gallery, but it is still there.
The problem now is that Nextcloud thinks it is a new file and uploads this file as well.

The photo is now available twice on Nextcloud, except that the deleted version is not displayed directly because the file is classified as a hidden file by the dot at the beginning of the name.

The second version is only displayed as soon as you select the "Show hidden files" option.

Android version

12

Device brand and model

Samsung Galaxy A51 SM-A515F/DSN

Stock or custom OS?

Stock

Nextcloud android app version

3.21.0

Nextcloud server version

24.0.3

Using a reverse proxy?

No

Android logs

No response

Server error logs

No response

Additional information

The Nextcloud app should recognize if it is a file with the name something like .trashed<...>.jpg and not synchronize it to the server.

[AlvaroBrey]

This should be treated as an enhancement, I think. We're not responsible for the behaviour of a particular photos app. We could try to detect this but who is to say that the next version of the Samsung Gallery will not rename them differently?

The only thing that would make sense to me here would be to add a setting to the autoupload folder to avoid uploading hidden files (possibly on by default).

[t3pfaffe]

This should be treated as an enhancement, I think. We're not responsible for the behaviour of a particular photos app.

@AlvaroBrey

This is the trash behavior of Google's default "Files" app as well on my Pixel 7. If it's the behavior of Android's default file browser as well as Samsung's, I'm pretty sure it can be treated as a standard Android behavior going forward. I agree it should be generalized to allow for ignoring custom file extensions, but you can't write this off as just a peculiarity of some niche photos app. This is just as much a bug as a feature enhancement.

[joshtrichards]

I presume you don't want to use the client's existing setting that enables delete the on-device photos after they're auto-uploaded?

[pulsejet]

I want to quickly bump this; this seems to be default behavior even when Photos are deleted using the native APIs or Google Photos. Is there any issue in simply excluding files with name matching ^.trashed-?

[qugebert]

Hi,
I encountered this problem for the first time today. What surprises me a bit is that in the general settings, 'Show hidden files' is disabled, yet these files, which should be considered hidden, are still being uploaded.
Would it make sense to generalize the problem to the extent of adding an option 'Ignore hidden files' (files starting with .) to the settings of Auto-Upload?

[ofrias]

Same problem here on a Google Pixel 6. The standard photos app renames deleted files as ".trashed..." and they are uploaded by Nextcloud app. This should not happen.
Nextcloud app should not upload hidden files (those starting by a dot).

[idle-user]

There should be an "do not upload hidden files" or "ignore hidden files" flag. It's frustrating that they're being uploaded.

As others have mentioned, the standard method these file cleaners work is to mark the files for deletion by putting ".trashed-" prefixed to the files.

But the app reads these as new files and begins the upload process.

I also noticed some videos that are processing try to upload if the timing is right. They appear as ".pending-". This would solve that too

[calando]

I would second the request, but for a slightly different reason: in my setup, each generation of a .trashed_* file leads to a subsequent upload error (that means, I do not have duplicated photos on my NC instance, but I get lots of error notifications from the NC app on my phone).

No idea why the upload of these .trashed_* files does not work, though (NC instance is latest release version, runs behind a reverse proxy)

[joshtrichards]

Instead, the file is renamed and only really deleted after 30 days (because of the recycle bin function).

As a workaround, try using the option for your auto-upload profiles to only upload while charging.

[ofrias]

As a workaround, try using the option for your auto-upload profiles to only upload while charging.

In my case, I am already using the "only upload while charging" setting but this bug breaks my use case. I want to be able to delete the photos that I don't want to keep using Google Pixel default camera/gallery application. Then, when I plug the power cable, I want Nextcloud app to upload the new photos (but not the ones that I deleted). Since the Pixel renames to hidden files instead of deleting, and Nextcloud upload all hidden files, all photos end up being uploaded, even the ones deleted/renamed in the phone.

[ghaberek]

I was getting fed up with this so as a work-around, I've got Termux installed and wrote a Bash script to purge the .trashed files. In my case they're typically found in either DCIM/Camera or Pictures/Screenshots so those are the only folders I'm targeting.

#!/data/data/com.termux/files/usr/bin/bash
for folder in 'dcim/Camera' 'pictures/Screenshots'; do
  find $HOME/storage/$folder/ -type f -name '.trashed-*.jpg' -print -delete 2>/dev/null
done

(Note the storage folders use lower-case names in Termux, e.g. storage/dcim.)

Edit: I transposed this by hand and was missing the single quotes between the two folder names on line 2.

[Aohzan]

In addition, .pending files are uploaded too

[zeugmatis]

Bumping this, very aggravating when cleaning out galleries on the phone, it re-uploads useless dotfiles doubling space and also bandwidth usage. There should be an option to ignore dotfiles.

[bioxz]

I recently switched from a Pixel 4 to a Pixel 7 and now I get a lot of upload failures due to .pending files ("Local file not found") and . trashed files ("File could not be copied to local storage"), someone else above had the same issue. But in the end the outcome (failure or duplicate) does not matter. This is with the default photos applications on Android. While I do not think that the behavior of that photos app is "correct" (those files should go into cache and trash folders, not just given a different name in place), it's unlikely that someone here will be able to change that behavior.

Some kind of regex blacklist feature would be great to solve this and similar issues. Could be per upload folder or system wide. Alternatively just being able to avoid specific files (e.g. "ignore hidden files") would be sufficient here as well.

[DecaTec]

Same problem here.

As a workaround, you can use the "file access control" app and define a flow in order to deny access to these files:
Filename matches /^\.trashed-.+$/i

This prevents uploads of these deleted photos, but it's still very ugly, because every time, you'll get an error in the Android app telling you that the access was denied while uploading these files.

At least these files do not end up being uploaded to your cloud...

Keep in mind to delete all .trashed-* files before activating this rule, otherwise you'll won't be able to delete this files afterwards in the cloud.

[akmail]

wondering why this is still open? thousands of users on a android uploading deleted files to a cloud sinnce 2+ years??? Guys, this should be a very high priority just because the effort is low and the benefit is huge. My opinion. Uploading photos is the most used feature of Nextcloud ever. Hope the developers realise that.

[joshtrichards]

If you use a Custom Folder (rather than the auto detected folders) for Auto Upload, there is an Exclude hidden files/folders option. See #12323.

[ofrias]

Thanks. I have created a Feature Request asking for the exact same setting to be implemented for autodetected folders: #13648

[Kloppi313]

I just wanted to add, that I use a standard Google Pixel 8 and I am affected.
The standard foto app works with the behaviour of renaming trashed files to .trashed-NAME
As a Google Phone uses Google Android in some kind of reference, this Issue should get some attention/prio.

As of now I keep getting errors ".trashed-NAME could not be moved to local Nextcloud..." as Android System Message

[joshtrichards]

Oddly, on my Pixel 7 (stock) this never happens with my Camera auto-uploads. I see the .trashed* file in DCIM/Camera on the device when I delete an image, but I've yet to encounter it getting uploaded by the client. I wonder what is different. 🤔

[Hunweibyn]

Having same problem on Xiaomi Poco x3 pro:
If I'm recording video with wifi connected, it tries to upload .pending files with error "file not exist" if I stop recording before .pending upload.

And it uploads .trashed images when I delete photo from native Xiaomi gallery.

So there is a lot of garbage files on upload folder.

[SilverNicktail]

I've just noticed that this is why my Nextcloud folder keeps getting bigger no matter how many times I delete photos from it. As with others, I'm using a Pixel-series phone with the standard "Photos" app.

This is functionality I use constantly. I love Nextcloud, and I'm a dev with some Android experience. If I can find some time over Christmas I might look into patching this in. Doesn't seem too difficult a requirement as a standalone ticket, even for someone with too many projects ;-)

[0ranki]

I'm created a draft PR (#14384) to address this, basically using the same hard-coded method as is used for .thumbnail files and .thumbdata directories.

I've tried on a Pixel 7a and a OnePlus 12, both with Google Photos. Don't know the deciding factor, but either the .trashed file were uploaded to the server or they caused an error. I think it might be because the Pixel phone had the F-Droid version, so also the "Access to all files" permission, while the OnePlus had the GPlay version, which can't have that permission anymore (#14349)

If anyone has the capacity to build and try it out, please do.

[akmail]

thanks for taking over, @0ranki !
just some thoughts (and I really have no clue if they make sense):

• what about blocking any files starting with a dot (hidden files) from android app?
• what about server declining any hidden files if coming from any app?
  appreciate your help!

[0ranki]

@akmail Ignoring all hidden files hard-coded might be a bit drastic, at least I can imagine someone wanting to sync those. Files with prefix like .trashed or .pending are more likely result from an external application, so I'd stick with specific prefixes for now.

Best option would be to make the list of prefixes to ignore configurable by the user, even better if per-folder. I'm just not that experienced in Android, Java or the codebase so implementing even a global setting may take a good while.. I'm interested to try doing it, though, as long as I find time for it.

[joshtrichards]

Fixed in #14384