Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty data object transfer between storage tiers #266

Open
ll4strw opened this issue Mar 20, 2024 · 6 comments
Open

Empty data object transfer between storage tiers #266

ll4strw opened this issue Mar 20, 2024 · 6 comments
Labels
Milestone

Comments

@ll4strw
Copy link

ll4strw commented Mar 20, 2024

I have two resources with the following settings and a rule that executes periodically the tiering plugin

$ imeta ls -R laptopResc  
AVUs defined for resource laptopResc:
attribute: irods::storage_tiering::group
value: groupie
units: 0
----
attribute: irods::storage_tiering::time
value: 15
units:

$ imeta ls -R laptopResc1
AVUs defined for resource laptopResc1:
attribute: irods::storage_tiering::group
value: groupie
units: 1

Upon itouching an empty data object into laptopResc, this file stays there forever and does not get moved to laptopResc1. On the other hand, any other data object, which contains data, honors the tiering rules

$ itouch test
$ itouch test33
$ echo "hello" >> test1 && iput test1
$ echo "hello" >> test2 && iput test2
$ ils -l
/laptopZone/home/rods:
  rods              0 laptopResc            0 2024-03-20.09:29 & test
  rods              1 laptopResc1           11 2024-03-20.10:00 & test1
  rods              1 laptopResc1           11 2024-03-20.10:09 & test2
  rods              0 laptopResc            0 2024-03-20.10:10 & test33

However, if I have the following local directory made of an empty file (test33) and a data file (file1)

$ ls -l TEST33
total 4
-rw-rw-r-- 1 irods irods 5 Mar 20 10:15 file1
-rw-rw-r-- 1 irods irods 0 Mar 20 10:15 test333

and I copy it over to iRODS

irsync -R laptopResc -r TEST33 i:TEST33

then the empty file DOES get moved to the second storage tier

$ ils -l /laptopZone/home/rods/TEST33
/laptopZone/home/rods/TEST33:
  rods              1 laptopResc1            5 2024-03-20.10:18 & file1
  rods              1 laptopResc1            0 2024-03-20.10:18 & test333

Finally, by adding again an empty data object to the TEST33 collection results in a missed transfer to the socond storage tier again

$ itouch TEST33/empty_file
$ ils -l /laptopZone/home/rods/TEST33
/laptopZone/home/rods/TEST33:
  rods              0 laptopResc            0 2024-03-20.10:22 & empty_file
  rods              1 laptopResc1            5 2024-03-20.10:18 & file1
  rods              1 laptopResc1            0 2024-03-20.10:18 & test333

Is this the expected behavior? If so, why to treat differently an empty file generated using itouch and one generated via irsync? Thanks in advance, L.

@trel
Copy link
Member

trel commented Mar 20, 2024

Definitely not expected behavior.

Which version of iRODS is this?

@trel trel added the bug label Mar 20, 2024
@ll4strw
Copy link
Author

ll4strw commented Mar 20, 2024

Ubuntu 22.04.3 LTS

irods-server/jammy,now 4.3.1-0~jammy amd64
irods-icommands/jammy,now 4.3.1-0~jammy amd64
irods-rule-engine-plugin-unified-storage-tiering/jammy,now 4.3.1.1-0~jammy amd64

@trel
Copy link
Member

trel commented Mar 20, 2024

Got it - we'll work to reproduce what you're seeing.

@korydraughn
Copy link
Collaborator

The plugin does not respond to itouch because the plugin doesn't know about the PEPs for the touch API.

We'll get that resolved.

@korydraughn korydraughn added this to the 4.3.1.2 milestone Mar 20, 2024
@alanking
Copy link
Contributor

Subset of #200, I believe (as seen here: #200 (comment))

@trel
Copy link
Member

trel commented Mar 20, 2024

ah, indeed.

@korydraughn korydraughn removed this from the 4.3.1.2 milestone Apr 26, 2024
@alanking alanking added this to the 4.3.3.1 milestone Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

4 participants