Add support for btrfs

[luigir-it]

Hi.
Today I did some changes on how I handle trash. I decided to create a @Trash subvolume and symlink ~/.local/share/Trash to it.
There aren't any issue with Dolphin (KDE gui file manager), everything works as before: I can trash files, folders, restore them...

But the utility I was using before (trash-cli) stopped working. So I tried everything else I could find on the AUR, and none of them did work.
I came across rmw, and it looks simple and straightforward, but has the same issue: with WASTE=$HOME/.local/share/Trash in the config file, deleting files yield this result:
:error: while trying to move (rename) haha -> /home/luigi/.local/share/Trash/files/haha_195333-240326 -- remove_to_waste:L362 rename(): Invalid cross-device link

This Invalid cross-device link is the same output error of trash-cli and similar utilities.

Is there a way to fix it? Like I said, Dolphin works fine with this new setup.

[andy5995]

Thanks for asking about this @luigir-it . I'll have to give the matter some thought. It would mostly just be a matter of adding copy and delete functionality when the source and target are on different filesystems. Although there are some extra things I'd have to be careful of (See the 'Implementation notes' at https://specifications.freedesktop.org/trash-spec/trashspec-latest.html).

Here's a bit of an explanation from ChatGPT why Dolphin works. I'm pretty sure it's true that Dolphin is doing a copy/delete instead of rename, because rename can't rename files across file systems (So rename and mv are essentially the same, a very quick operation).

Yes, Dolphin, the file manager for KDE, behaves similarly to other file managers when it comes to handling files that are "trashed" from a different filesystem than the one where the trash folder resides. When you move a file to the trash, if the file is on the same filesystem as the trash folder, it is simply moved to the trash directory, which is a fast operation. However, if the file is on a different filesystem, Dolphin cannot directly move the file to the trash folder.

In this case, Dolphin performs a copy-and-delete operation: it copies the file to the trash folder on the filesystem where the trash resides, and then deletes the original file from the source filesystem. This ensures that the file appears in the trash and can be restored if needed, but it can be a slower process than a simple move, especially for large files, because the file data must be physically copied to a different filesystem.

Consider when using 'mv' at the command line when the source and target are on the same filesystem:

When using the mv command to move a file or directory within the same filesystem, the actual data blocks on the disk are not moved or copied. Instead, the operation involves updating the file system's metadata to reflect the change in the file's or directory's location. This makes moving files or directories within the same filesystem a very fast operation, as it doesn't involve physically copying the data.

Here's a simplified explanation of what happens:

1. Update Directory Entries: The file system updates the directory entries to remove the reference to the file or directory from its original location and add a reference in the new location.

2. Inode Update: The inode (the data structure that stores information about the file, such as its size, permissions, and disk block locations) remains the same, but the file system updates any metadata that points to the file's location, such as the file's path.

3. No Data Movement: Since the actual data blocks on the disk are not moved, the operation is quick, regardless of the file's size.

In contrast, if the source and target are on different filesystems, mv effectively becomes a copy-and-delete operation: the file is copied to the target filesystem, and then the original file is deleted from the source filesystem. This process can take significantly longer, especially for large files, because the data must be physically copied to a different location on the disk.

[luigir-it]

So it's not that trivial, since you have to keep original permissions.
Anyway, thanks for getting back at me. Have a good day.

[andy5995]

@luigir-it Right now I don't think I'll be adding support for moving files. It's not a feature I would use personally because I prefer to avoid the extra wear-and-tear on my drive.

It's possible I might change my mind though and add it anyway, but I don't know when that would happen.

One option I could suggest to you so that you could use rmw or trash-cli is to have a script that moves the files.

1. Remove the symlink to the trash folder on the other drive
2. Have a script that periodically moves files from your trash folder to the trash folder on your other drive
3. Add the trash folder on your other drive to the rmw configuration file (or trash-cli conf if it has one)

If you have the trash folder on your other filesystem in the rmw config file, it will still purge (permanently delete) any expired files based on when they were first trashed.

So while I can suggest that, I won't recommend it. The only problem I can think of is if you run rmw (or trash-cli) at the same time your script is moving files. There's the potential for them to get confused as they read the .trashinfo files and it's possible the corresponding file might not exist yet if it's being moved.

[ccoVeille]

@luigir-it did you try using a mount --bind instead of a symlink.

So you would have two mount, the btrfs drive AND the bind.

mount --bind sometimes helps solving issue like that.

https://www.baeldung.com/linux/bind-mounts

I hope it would help solving your issue.

[luigir-it]

I just tried it, no luck. The error message is slightly different:

sudo mount --bind --mkdir @Trash/ Trash
touch hey
rm hey
 :'hey' not ReMoved:
No WASTE folder defined in '/home/luigi/.config/rmwrc' that resides on the same filesystem.
0 items were removed to the waste folder

[luigir-it]

It's not a feature I would use personally because I prefer to avoid the extra wear-and-tear on my drive.

That's totally understandable. And I also think it's reasonable to have a Trash directory per filesystem to avoid unnecessary read/write operations.

However, btrfs subvolumes are not exactly different filesystems. In my case, the @Trash subvolume resides on the same disk where the files I want to delete are located. To be more precise, the files are inside the @home subvolume, and @Trash is a nested subvolume (inside @home). The logic behind this is that when I take snapshots of @home, I'd like to exclude trashed files, so that I can export "clean" snapshots (btrfs send/receive) elsewhere for backups.

This means that mv should, in theory, just update metadata, not perform copy-and-delete operation. I confirmed this by:

1. Creating a large file: dd if=/dev/urandom of=10GB.bin bs=64M count=128 iflag=fullblock [8GiB] (it takes few seconds, even on a fast NVMe SSD)
2. Manually moving it to the @Trash subvolume: mv 10GB.bin .local/share/@Trash
   The move is instantaneous and iotop don't report any significant read/write activity.

[andy5995]

It's not a feature I would use personally because I prefer to avoid the extra wear-and-tear on my drive.

That's totally understandable. And I also think it's reasonable to have a Trash directory per filesystem to avoid unnecessary read/write operations.

I agree. I hope we can figure out a reasonable solution.

However, btrfs subvolumes are not exactly different filesystems. In my case, the @Trash subvolume resides on the same disk where the files I want to delete are located.

I see. That explains why rmw is attempting the rename (move). Normally if it sees the target file and Trash folder are on different devices, you'd get the message: "No WASTE folder defined in '/home/luigi/.config/rmwrc' that resides on the same filesystem."

So rmw is detecting they are on the same device (using 'dev_t') but rename() is failing. ChatGPT says:

Based on the discussion you referred to, the rename() function in C might be experiencing problems when moving files across different Btrfs subvolumes due to specific Btrfs properties, even though they are technically on the same filesystem. This could relate to how Btrfs handles subvolume IDs and their relationships within the filesystem's structure.

In Btrfs, each subvolume has a unique ID and is part of a flat namespace within the filesystem, but they are managed like separate filesystems when it comes to certain operations. This can sometimes result in the EXDEV error (cross-device link error) with rename(), especially if the subvolumes are not properly recognized as being on the same physical device. This issue can be misleading because rename() should theoretically work within the same filesystem.

A potential workaround for this issue involves manually copying the file to the new location and then deleting the original, as the rename() function cannot handle what it perceives as cross-filesystem moves in this context. Additionally, ensuring that subvolumes are mounted with correct options and that the filesystem understands their relationship might help. For instance, correctly using mount points and ensuring that the filesystem treats these subvolumes as part of the same device can mitigate such issues.

Further reading and more detailed explanations about how Btrfs handles these scenarios can be found in resources such as the Btrfs documentation and community discussions on platforms like ArchWiki and Ubuntu documentation. These resources provide valuable insights into the peculiarities of Btrfs handling, especially in complex configurations involving nested subvolumes and snapshots.

Because in your case 'mv' isn't using the fallback copy/delete method, there should be a way to do this with rmw.

Can you provide a few lines on how I'd replicate your setup on my system so I can do some testing?

[luigir-it]

Normally if it sees the target file and Trash folder are on different devices, you'd get the message: "No WASTE folder defined in '/home/luigi/.config/rmwrc' that resides on the same filesystem."

Note that when I used the bind mount instead symlink as suggested by @ccoVeille, the error I got was

rm hey
:'hey' not ReMoved:
No WASTE folder defined in '/home/luigi/.config/rmwrc' that resides on the same filesystem.
0 items were removed to the waste folder

But the filesystem structure is of course the same

[andy5995]

Noted. Thanks.

[andy5995]

2. Manually moving it to the `@Trash` subvolume: `mv 10GB.bin .local/share/@Trash`

@luigir-it Are you using the GNU Coreutils 'mv'? I can look for clues here.

[luigir-it]

Yes it's the standard GNU Coreutils (version 9.5-1 on arch linux)

[andy5995]

Ok. Btw, you can use '-v' with 'mv' to tell you what's being done next time you try a test like that (which means you won't have to create a 10GB file first).

[andy5995]

I'm getting different results than you when I try a 'mv'.

Here's what I did:

• Formatted a partition as btrfs
• Added it to fstab /dev/sda5 /mnt/btrfs_part auto user,defaults,noatime 0 2)
• Mounted it
• Changed to it and created a subvolume called 'Waste'
• Created a file called 'hello'

$ mv -v hello Waste/
copied 'hello' -> 'Waste/hello'
removed 'hello'

And stat shows different device ids for the mounted partition and the subvolume on that partition:

[andy@prometheus mnt]$ stat btrfs_part
  File: btrfs_part
  Size: 52        	Blocks: 32         IO Block: 4096   directory
Device: 0,37	Inode: 256         Links: 1
Access: (1777/drwxrwxrwt)  Uid: (    0/    root)   Gid: (    0/    root)

[andy@prometheus mnt]$ stat btrfs_part/Waste
  File: btrfs_part/Waste
  Size: 18        	Blocks: 0          IO Block: 4096   directory
Device: 0,93	Inode: 256         Links: 1
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    andy)   Gid: ( 1000/    andy)

[luigir-it]

Hi. You're right here. I tried moving the same 8GiB file across different subvolumes using mv -v and it indeed indicated "copied/removed". However, when I move a file within the same subvolume, it simply shows "renamed".

Nevertheless, during the movement of the 8GiB file, there was no spike in IO usage, and the move occurred instantly.

[andy5995]

@luigir-it I get the same results. I opened ticket #451. You may want to subscribe to it.

[luigir-it]

If you want to replicate my setup:

1. This is the fstab (I removed unnecessary entries)

# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
#
/dev/mapper/luks-nvme0n1 /btrfs_toplevel          btrfs   subvolid=5,defaults,noatime,compress=zstd 0 0
/dev/mapper/luks-nvme0n1 /home                    btrfs   subvol=/@home,defaults,noatime,compress=zstd 0 0

2. This is the Trash subvolume

> sudo btrfs subvolume list -o /home
ID 1046 gen 210893 top level 257 path @home/luigi/.local/share/@Trash

you first need to create @home, then cd into $HOME/.local/share/ and then

> sudo btrfs subvolume create @Trash
> ln -s @Trash Trash

3. This is the rmwrc config

WASTE=$HOME/.local/share/Trash
expire_age = 7

[luigir-it]

If you need anything else let me know

[andy5995]

Closing. See issue #451 for further discussion.