requirements.lock is not removed when deploying per rsync

[ctheune]

No description provided.

[ctheune]

Alright. So the problem here was that we were iterating over the local top-level entries so we can blacklist some of them. However, this caused us to not delete-sync top-level elements that did not exist any longer because we never added them as sync targets.

I've rewritten the affected code to do a single rsync step and use a subclass leveraging the built-in filter to provide the blacklist.

[ctheune]

I have to undo this fix for because it has a major impact on how it combines with ignoring some files: pytest-dev/execnet#134

This can lead to major issues because this will consistently remove the work directory in rsync-based deployments.

[ctheune]

I wonder whether we could do something related to #235 that would fix this, too?

[sallner]

I am still confused with the title and the actual aim of this ticket. Why should the requirements.lock be removed in any case when deploying? I thought this would pin the version to be consistent? I do understand that there might be a reason to have untracked files to be removed on the remote part (deployment) but is this the issue which should be solved here?

[ctheune]

The problem is in this situation:

1. you deploy using rsync while requirements.txt and requirements.lock both exist
2. you update your requirements.txt
3. you decide to delete the requirements.lock
4. you deploy and the remote and still has the requirements.lock with old settings

I wonder whether we should decide that the requirements.lock must always exist and never do unclean environments at all? Not sure which drawbacks that would have.

[sallner]

I wonder whether we should decide that the requirements.txt must always exist and never do unclean environments at all? Not sure which drawbacks that would have.

I think you mean requirements.lock here, didn't you? In my opinion, the lock-file is a hard requirement for a reproducible deployment. We sometimes have version ranges in requirements.txt like batou>=2.2 which would lead to different deployments without a lock file over time. This is even more the case for dependencies, which are probably not even shown during installation.

I would vote for an error in case the lock-file is not present. At least in my workflow, it is not often, that I have to change the batou requirements very often. Is there a use-case where this is different?

[ctheune]

Yeah, I meant requirements.lock. Fixed it in my comment.

[sallner]

It is implemented in flyingcircusio/appenv#31 but has to be copied to this project after successful merge.