Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use cache built charms in bundle tests #662

Conversation

NohaIhab
Copy link
Contributor

@NohaIhab NohaIhab commented Jan 27, 2025

Closes #654

This PR deprecates the --no-build option, that was used to deploy the charms from Charmhub and test them locally due to it not being used by the team.

Summary

  • adds step to download charms built with cache before running bundle integration tests
  • adds --charms-path parameter to pass to integration tests
  • refactors the bundle integration test files to use the downloaded charms

Note to reviewer:
the single charm integration tests are failing due to canonical/kubeflow-ci#152, this should be fixed by #663

@NohaIhab NohaIhab marked this pull request as draft January 27, 2025 11:49
@NohaIhab NohaIhab marked this pull request as ready for review January 28, 2025 10:58
Copy link
Contributor

@DnPlas DnPlas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NohaIhab ! Tiny comments, other than that lgtm.

tests/integration/test_kfp_functional_v1.py Outdated Show resolved Hide resolved
tests/integration/test_kfp_functional_v1.py Show resolved Hide resolved
Comment on lines +90 to +103
update_charm_context(context, charm, cached_charm)
# Otherwise build the charms with ops_test
else:
charms_to_build = {
charm: Path(CHARM_PATH_TEMPLATE.format(basedir=str(basedir), charm=charm))
for charm in KFP_CHARMS
}
log.info(f"Building charms for: {charms_to_build}")
built_charms = await ops_test.build_charms(*charms_to_build.values())
log.info(f"Built charms: {built_charms}")

for charm, charm_file in built_charms.items():
charm_resources = get_resources_from_charm_file(charm_file)
context.update([(f"{charm.replace('-', '_')}_resources", charm_resources)])
context.update([(f"{charm.replace('-', '_')}", charm_file)])

# Build charms sequentially
for charm_name, charm_path in charms_to_build.items():
log.info(f" Building charm {charm_name}")
built_charm = await ops_test.build_charm(charm_path)
update_charm_context(context, charm_name, built_charm)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The local build will now be different from the CI because it's using charmcraft, as opposed to the CI using charmcraftcache. We could approach this by telling developers to pack the charms with charmcraftcache locally before running the tests and passing --charms-path, although I can see the prerequisite being a bit annoying. WDYT @DnPlas ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have this information on the integration tests README.md. I wouldn't consider it a pre-requisite, but an option:

## Testing locally

Option A - Build charms with charmcraft

<test instructions and a tiny explanation of what's happening>

Option B - Build charms with ccc

<test instructions and a tiny explanation of what's happening>
<tox -ve integration -- --charms-path >

Copy link
Contributor

@DnPlas DnPlas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NohaIhab !

@NohaIhab NohaIhab merged commit 6fefbc4 into KF-6684-refactor-ci-with-build Jan 29, 2025
91 checks passed
@NohaIhab NohaIhab deleted the KF-6761-feat-use-cached-charms-bundle-tests branch January 29, 2025 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants