Cache poetry itself

[KapJI]

• Cache pip use base to not reinstall poetry on every job.
• Fix for macos which was using python from runner instead of provided.

[leikoilja]

looks quite complex 😅
why do we not just use good old pip? it's slower - yes, but get's the job done in a lot simpler fashion with less dependencies to manage.
rye/uv is also an option

[KapJI]

I guess at this point it's no longer about usefulness but rather about learning how to make CI the most efficient 😅

I don't think we need pipx here actually, I can try to remove it. This is still WIP.

uv looks interesting, we can try to use it instead of poetry.

[KapJI]

Good enough now for this PR. Other improvements will be done in subsequent PRs.

Windows build job is 30% faster now (51s->37s)