I can't clean files that don't belong to me using the -c parameter

[PPX888]

workerID 23
bucket 11
I had some files on my server that didn't belong to me, and I added the -c parameter to the storage node startup configuration, but the number and size of the files didn't change, and the storage service became unstable. After the -c parameter is removed, the server recovers.
I also tried using storage-node util:cleanup, but it didn't work either.

nano /root/joystream/docker-compose.yml

  storage:
    image: joystream/storage-node:${COLOSSUS_VERSION}
    container_name: storage
    restart: unless-stopped
    volumes:
      - ${DATA_FOLDER}:/data
      - ${KEY_FOLDER}:/keystore
      - ${LOG_FOLDER}:/logs
      - ${ENDPOINT}:/joystream/storage-node/runner.sh:ro
    working_dir: /joystream/storage-node
    ports:
      - 3333:3333
    env_file:
      - .env
    environment:
      ENABLE_TELEMETRY: ${ENABLE_TELEMETRY:-no}
      OTEL_RESOURCE_ATTRIBUTES: service.name=colossus,deployment.environment=production
      NODE_ENV: ${NODE_ENV}
    entrypoint: ['/joystream/storage-node/runner.sh']
    command: [
      'server',
      '--worker=${COLOSSUS_1_WORKER_ID}',
      '--port=3333',
      '--uploads=/data',
      **'-c',**
      '--sync',
      '--syncInterval=10',
      '--syncWorkersNumber 20',
      '--storageSquidEndpoint ${STORAGESQUIDENDPOINT}',
      '--apiUrl=${JOYSTREAM_NODE_WS}',
docker logs -f storage --tail 100

root@js-fb:~/joystream-cli# yarn run storage-node util:cleanup -b 11 -w 23 -d /data --password=X -k /root/keys/storage-role-key.json
yarn run v1.22.19
$ /root/joystream-cli/node_modules/.bin/storage-node util:cleanup -b 11 -w 23 -d /data --password=X -k /root/keys/storage-role-key.json
2024-07-28 11:17:37:1737 info: Adding key from /root/keys/storage-role-key.json
2024-07-28 11:17:38:1738 info: Initializing runtime connection to: ws://localhost:9944
2024-07-28 11:17:38:1738 info: Waiting for chain to be synced before proceeding.
2024-07-28 11:17:38:1738 info: Cleanup...
2024-07-28 11:17:38:1738 info: Started cleanup service...
2024-07-28 11:17:39:1739 debug: Sync - getting all storage buckets: offset = 0, limit = 1000
2024-07-28 11:18:15:1815 debug: Cleanup - pruning 0 obsolete objects
2024-07-28 11:18:15:1815 info: Cleanup ended.
Done in 254.21s.
root@js-fb:~/joystream-cli# du -sh /data
67T     /data
root@js-fb:~/joystream-cli# 

[Lezek123]

added the -c parameter to the storage node startup configuration, but the number and size of the files didn't change

By default --cleanupInterval is set to 360 minutes (6 hours), that's why you may not have seen any results immediately.

the storage service became unstable

If by that you mean warnings during sync like:

2024-10-15 11:43:51:4351 warn: Sync - fetching data error for https://sieemmastorage.com/storage/api/v1/files/2908270: Error: Invalid file hash. Expected: gW5etidbCBn5zeQY6ZgJ8gcn2kQjUmPY3nc16b3dpQRRsJ - real: gW9T5gSXhgQB5LaqmnJrLVVpHawhM7youzHnuY6HyGBDy3
2024-10-15 11:43:51:4351 warn: Cleaning up file /data/temp/016096f3-a0d6-4ff1-8f49-9c81f32729ce

They are not related to cleanup. They occur because some data objects you're trying to sync are missing on other nodes. Invalid file hash is actually a misleading message here, it should probably say Data object not found.
Some objects seem not to be stored by any SP right now, even though they exist on chain, like 2908605 for example.

I also tried using storage-node util:cleanup, but it didn't work either.

This was an actual bug and should be fixed by #5191

[PPX888]

added the -c parameter to the storage node startup configuration, but the number and size of the files didn't change

By default --cleanupInterval is set to 360 minutes (6 hours), that's why you may not have seen any results immediately.

the storage service became unstable

If by that you mean warnings during sync like:

2024-10-15 11:43:51:4351 warn: Sync - fetching data error for https://sieemmastorage.com/storage/api/v1/files/2908270: Error: Invalid file hash. Expected: gW5etidbCBn5zeQY6ZgJ8gcn2kQjUmPY3nc16b3dpQRRsJ - real: gW9T5gSXhgQB5LaqmnJrLVVpHawhM7youzHnuY6HyGBDy3
2024-10-15 11:43:51:4351 warn: Cleaning up file /data/temp/016096f3-a0d6-4ff1-8f49-9c81f32729ce

They are not related to cleanup. They occur because some data objects you're trying to sync are missing on other nodes. Invalid file hash is actually a misleading message here, it should probably say Data object not found. Some objects seem not to be stored by any SP right now, even though they exist on chain, like 2908605 for example.

I also tried using storage-node util:cleanup, but it didn't work either.

This was an actual bug and should be fixed by #5191

I ran it with the -c parameter for several days, but there was no change in the available space on the disk.

[Lezek123]

Could you confirm whether #5191 fixed those issues for you?