feat: git Access, AccessMethod through BlobAccess

[jakobmoellerdev]

What this PR does / why we need it:

This is an extremely early (WIP) draft of implementing ctf archives in directory format tracked via git repositories, ultimately serving me as a playground to get familiar with OCM and also to play around with its feature set. So we do not need to merge this if there is no interest or disagreement on the implementation.

Not only does this allow using git repositories for storing blobs, it also attempts to add git downloader functionality for a generic git access method based on in memory cloning and I plan to add a OCM implementation on it as well.

Currently the implementation is written so that any modification to the CTF in the work directory also triggers a dedicated commit, and any lookup triggers a refres/pull cycle on git.

For testing I use file:// URLs which makes it quite easy to verify E2E in the test suites.

TODO:

• Implement Access Credential mapping
• Implement Git Downloader
• Make Git Client thread-safe, optionally in-memory and allow more customization (currently hard bount to one target ref that I am creating or syncing against, see tests for examples)
• Expand Git Options - Authentication support for repo and client

Which issue(s) this PR fixes:

As this is my own "experimentation" issue, it doesnt necessarily fix anything, it might as well be moved to a plugin for example or be scrapped alltogether

[Skarlso]

Hello.

So this work happens to be implementing this issue open-component-model/ocm-project#239.

I'll assign it to you then.

[mandelsoft]

Hi @jakobmoellerdev, concerning your idea of git backed CTF repositories, if I understand your approach correctly,
every change will create a new commit. I think this approach misses the need for a semantical bracket for a set of changes.
It is more like creating a commit for every changed line in a source file, which does not look like a good idea.
A useful bracket (which is already supported) could be the open/close of a CTF. The typical workflow is to open a CTF, apply some changes (for example with the ocm add cv command) and then close it again. So a more useful point in time to create a new commit should be the Close operation of a CTF.

[jakobmoellerdev]

Hi @mandelsoft, thanks for taking a look.

I think the semantical equivalent to a git push would be to close the repository. In your description that would be

1. Open CTF -> Git Clone / Checkout
2. Apply Changes (e.g. add cv) -> Write/Update Files in CTF, Commit atomically
3. Close CTF -> Git Push

Of course we can change the flow to

1. Open CTF -> Git Clone / Checkout
2. Apply Changes (e.g. add cv) -> Write/Update Files in CTF,
3. Close CTF -> Git Commit all changes, Git Push

If you are concerned with that. In my view they are equivalent and the atomic commits are much more transparent when a change was made. However Im totally fine with your take as well, it shouldnt be too hard to change.

I think we could change it so that there is one commit that has all the current commit messages in the commit body. Then we will have 1 commit. WDYT?

[jakobmoellerdev]

this is ready for a first pass review. We might change the way the commits are generated (see discussion above) but generally i would still like some input as its my first take on a larger change in this codebase

[mandelsoft]

Great, that you implemented a general git access, also,

But unfortunately this should include more extensions to be implemented consistently:

• blob accesses
• access method
• input types (for the CLI based composition environment)
• the convenience functions in the elements package. (I personally would prefer to get rid of them)

For the other accesses, we have implemented the blob access and then based the others on this common implementation.

[jakobmoellerdev]

I implemented the blob access and the access method based on that blob access now. For the input types and the elements, thats still missing, still need a pointer

[mandelsoft]

see comment for downloader.

[jakobmoellerdev]

This was changed to incorporate tar.gz archives instead. I still need to incorporate the path filtering

[jakobmoellerdev]

Just a quick update on this PR: I didnt get around to it yet, but I will probably remove the repo code because its unlikely to be used (as it all started for me to get into the OCM codebase). The AccessMethod etc can stay in and are still valid though.