This is related to #61 which introduced the caching.

What if you want to keep making queries to the cached metadata? In that case we can't just remove it before exiting. What is the definition of "fus has finished using this metadata"?
@ignatenkobrain any opinion on this?

Of course one possible solution is to add an option --clear-cache-on-exit for those who are running only one query on the metadata.

The issue is that everytime the repository metadata changes, fus downloads new set of metadata and previously cached metadata is stored on fs forever. For me, this is 1GB of metadata every day. I generate 3 composes using Pungi every hour as CI.

I also think that when executing fus in Pungi, the repository passed to fus is actually different on every pungi run. @lubomir probably knows that for sure.

It is OK for me to remove metadata older than 1 day by that cron job, but I'm afraid that this will have to be sooner or later be done by any fus (or at least pungi) user and therefore I was thinking it might be better to address this in fus directly.

Well, cache exists for a reason. If we would remove it, we could just not create it in the first place :)

what I think we should do is encode filenames in cache by the name of repo and not by url. @r4f4 could you make such patch?

@ignatenkobrain but we never used URL. The current layout of the cache is:

~/.cache/fus/$(repo name used in cmdline)/repodata/repomd.xml
~/.cache/fus/$(repo name used in cmdline)/repodata/$(repo-checksum)-{primary,comps,filelists}.xml.gz
~/.cache/fus/$(repo name used in cmdline)/$(repo-checksum).solv

So your suggestion is to use $(repo name used in cmdline) instead of $(repo-checksum)?

yes

Ok, will do. Thanks.

Pungi reuses the same repo name, but there are no guarantees they will point to similar repos (particular in @hanzz's use case in ODCS). Caching by repo name instead of checksum should help, since that should result in the files being overwritten instead of always adding a new ones.

That's how the cache directory looks for me currently:

https://paste.fedoraproject.org/paste/Sr41jYTwmLnJnUGzjDLOfg

As I understand the proposed change, the files in the repodata subdir will use lookaside-X or repo-X name and thus there won't be that many of them.

@lubomir exactly. If that helps, I'll implement it today.

That would help me a lot.

FYI for the Pungi use case this issue is no longer a problem since it uses a temporary directory as a cache and only shares it between runs with the same repos. Afterwards it gets cleaned by Pungi.