Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(umoci)!: umoci's copy buffer set to 1 MiB #537

Merged
merged 7 commits into from
Nov 8, 2023

Conversation

smoser
Copy link
Contributor

@smoser smoser commented Nov 8, 2023

project-stacker/umoci@89a7ea1

Use updated umoci (project-stacker's fork) to use a 1 MiB copy buffer. This change brings stacker inline with compression parameters used in containers/image libraries and makes them compatible.

test(gzip): add unit tests for gzip

(cherry picked from commit 589a648)

What type of PR is this?

Which issue does this PR fix:

What does this PR do / Why do we need it:

If an issue # is not available please add repro steps and logs showing the issue:

Testing done on this change:

Automation added to e2e:

Will this break upgrades or downgrades?

Does this PR introduce any user-facing change?:


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@smoser smoser force-pushed the fix/0.40.1-umoci-buffer branch 8 times, most recently from 0c74667 to a252e4b Compare November 8, 2023 19:26
rchincha and others added 7 commits November 8, 2023 12:12
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Scott Moser <[email protected]>
anuvu/squashfs will not currently build with master from
squashfs-tools-ng.  squashfs-tools-ng has just moved forward with
breaking changes to their api.

We need to build and use a specific version.

Change the squashfs-tools-ng both on the host
(.github/workflows/build.yaml) and in the stacker
build.yaml file

Signed-off-by: Scott Moser <[email protected]>
This step is just not necessary, we do not use the 'squashfs' binary
for anything.

Signed-off-by: Scott Moser <[email protected]>
This branch has not been built for some time.  In that time, alpine
container has moved forward, and has caused bitrot.  The changes
here are all present in main branch's build.yaml.

Signed-off-by: Scott Moser <[email protected]>
Mostly a cherry-pick of 3134d84
But we do not need the po4a change here as we have an older
version of xz.

Signed-off-by: Scott Moser <[email protected]>
cherry-pick commments:
> below, is original upstream commit message.
> Things that have been adjusted for this branch are:
> * There is no 're-enable' here as it was not disabled here
> * The version to compare against is v0.40.1, the last released
>   version on this branch.

Commit 9671d8f disabled the caching.bats test
"test can read previous version's cache".

This re-enables that test after updating it to use a specific value
(v1.0.0-rc4) as the "previous version".

The change to use a specific version and use its released binary
improves the test. That improves the test for the following
reasons:

 * This should be faster - we don't have to build stacker twice.
 * Ultimately testing against the last released version is probably what
   you want... that is the cache that your users have, not the last
   development commit.
 * Building stacker from another version is both slow and hard to
   guarantee. This removes the assumption that the current build
   environment can build another version of stacker.
 * There was assumed knowledge about both the current and other stacker
   build systems in the 'make LXC_BRANCH=$(grep ...)' command.
 * The prior version of the test used "current/main" as "previous", but
   current/main is a moving target.  Re-running a checkout
   at a future point in time may fail as current/main has moved forward.
   (Think about using 'git bisect' to diagnose a past regression)
 * Even when working as intended, a series of PRs could still break
   re-use of the cache.  Given PRs A, B, C that all are merged to
   master.  B is tested against A, C is tested against B, but
   nothing tested C against A.

The change here *does* mean the following things are true, which are
possibly undesirable:

 * There is a requirement to update the old version string.
   v1.0.0-rc4 is an arbitrary value.
 * The old binary must run in the current build environment.  This
   may be hard to guarantee.  I'm honestly not sure if static binary
   will be more reliable than building and executing a specific old
   version of source.

(cherry picked from commit b4f7294)
Signed-off-by: Scott Moser <[email protected]>
project-stacker/umoci@89a7ea1

Use updated umoci (project-stacker's fork) to use a 1 MiB copy buffer.
This change brings stacker inline with compression parameters used in
containers/image libraries and makes them compatible.

* test(gzip): add unit tests for gzip

(cherry picked from commit 589a648)

Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Scott Moser <[email protected]>
@smoser smoser force-pushed the fix/0.40.1-umoci-buffer branch from a252e4b to 33c80df Compare November 8, 2023 20:13
Copy link
Contributor

@rchincha rchincha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rchincha rchincha merged commit 4170fc0 into project-stacker:rel-0.40.x Nov 8, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants