Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Zuul] Increase the resources for OCP4.12 jobs #604

Closed
wants to merge 12 commits into from

Conversation

elfiesmelfie
Copy link
Collaborator

@elfiesmelfie elfiesmelfie commented May 31, 2024

Copy link

Zuul encountered a syntax error while parsing its configuration in the
repo infrawatch/service-telemetry-operator on branch master. The error was:

mapping values are not allowed in this context
in "infrawatch/service-telemetry-operator/.zuul.yaml@master", line 195, column 46

1 similar comment
Copy link

Zuul encountered a syntax error while parsing its configuration in the
repo infrawatch/service-telemetry-operator on branch master. The error was:

mapping values are not allowed in this context
in "infrawatch/service-telemetry-operator/.zuul.yaml@master", line 195, column 46

@elfiesmelfie elfiesmelfie force-pushed the efoley-zuul-bump-412-crc branch from 1af0e89 to fd6c814 Compare May 31, 2024 12:08
Copy link

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/service-telemetry-operator for 604,fd6c81477a8c2416592eaa646fd8e45287fbfae2

@elfiesmelfie elfiesmelfie force-pushed the efoley-zuul-bump-412-crc branch from fd6c814 to dadc358 Compare May 31, 2024 12:11
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/82f58a84897d43f79c0385fb229365c3

stf-crc-ocp_412-local_build FAILURE in 20m 05s
stf-crc-ocp_412-local_build-3xl RETRY_LIMIT in 18m 59s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 55m 04s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 22m 58s

Debug information:
Project <ProjectConfig github.com/infrawatch/service-telemetry-operator source: infrawatch/service-telemetry-operator/.zuul.yaml@master {ImpliedBranchMatcher:master}> matched
Project <ProjectConfig github.com/infrawatch/service-telemetry-operator source: infrawatch/service-telemetry-operator/[email protected] {ImpliedBranchMatcher:stable-1.5}> did not match
Freezing job graph
Freezing job stf-crc-ocp_412-local_build
Collecting job variants for stf-crc-ocp_412-local_build
Variant <Job stf-crc-ocp_412-local_build branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#141> matched
Collecting job variants for stf-crc-local_build
Variant <Job stf-crc-local_build branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#109> matched
Collecting job variants for stf-base
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#72> matched
Collecting job variants for stf-base-2node
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#18> matched
Collecting job variants for podified-multinode-edpm-deployment-crc
Variant <Job podified-multinode-edpm-deployment-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/edpm_multinode.yaml@main#247> matched
Collecting job variants for cifmw-podified-multinode-edpm-base-crc
Variant <Job cifmw-podified-multinode-edpm-base-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main#121> matched
Collecting job variants for base-extracted-crc
Variant <Job base-extracted-crc branches: None source: config/zuul.d/_jobs-crc.yaml@master#55> matched
Collecting job variants for base-minimal
Variant <Job base-minimal branches: None source: config/zuul.d/jobs.yaml@master#24> matched
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#19> did not match
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#69> did not match
Variant <Job stf-crc-local_build branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#106> did not match
Variant <Job stf-crc-ocp_412-local_build branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#138> did not match
Pipeline variant <Job stf-crc-ocp_412-local_build branches: None source: infrawatch/service-telemetry-operator/.zuul.yaml@master#266> matched
Freezing job stf-crc-ocp_412-local_build-3xl
Collecting job variants for stf-crc-ocp_412-local_build-3xl
Variant <Job stf-crc-ocp_412-local_build-3xl branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#174> matched
Collecting job variants for stf-crc-ocp_412-local_build
Variant <Job stf-crc-ocp_412-local_build branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#141> matched
Collecting job variants for stf-crc-local_build
Variant <Job stf-crc-local_build branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#109> matched
Collecting job variants for stf-base
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#72> matched
Collecting job variants for stf-base-2node
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#18> matched
Collecting job variants for podified-multinode-edpm-deployment-crc
Variant <Job podified-multinode-edpm-deployment-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/edpm_multinode.yaml@main#247> matched
Collecting job variants for cifmw-podified-multinode-edpm-base-crc
Variant <Job cifmw-podified-multinode-edpm-base-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main#121> matched
Collecting job variants for base-extracted-crc
Variant <Job base-extracted-crc branches: None source: config/zuul.d/_jobs-crc.yaml@master#55> matched
Collecting job variants for base-minimal
Variant <Job base-minimal branches: None source: config/zuul.d/jobs.yaml@master#24> matched
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#19> did not match
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#69> did not match
Variant <Job stf-crc-local_build branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#106> did not match
Variant <Job stf-crc-ocp_412-local_build branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#138> did not match
Pipeline variant <Job stf-crc-ocp_412-local_build-3xl branches: None source: infrawatch/service-telemetry-operator/.zuul.yaml@master#266> matched
Freezing job stf-crc-ocp_412-local_build-index_deploy
Collecting job variants for stf-crc-ocp_412-local_build-index_deploy
Variant <Job stf-crc-ocp_412-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#155> matched
Collecting job variants for stf-crc-local_build-index_deploy
Variant <Job stf-crc-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#118> matched
Collecting job variants for stf-base
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#72> matched
Collecting job variants for stf-base-2node
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#18> matched
Collecting job variants for podified-multinode-edpm-deployment-crc
Variant <Job podified-multinode-edpm-deployment-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/edpm_multinode.yaml@main#247> matched
Collecting job variants for cifmw-podified-multinode-edpm-base-crc
Variant <Job cifmw-podified-multinode-edpm-base-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main#121> matched
Collecting job variants for base-extracted-crc
Variant <Job base-extracted-crc branches: None source: config/zuul.d/_jobs-crc.yaml@master#55> matched
Collecting job variants for base-minimal
Variant <Job base-minimal branches: None source: config/zuul.d/jobs.yaml@master#24> matched
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#19> did not match
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#69> did not match
Variant <Job stf-crc-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#115> did not match
Variant <Job stf-crc-ocp_412-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#152> did not match
Pipeline variant <Job stf-crc-ocp_412-local_build-index_deploy branches: None source: infrawatch/service-telemetry-operator/.zuul.yaml@master#266> matched
Freezing job stf-crc-ocp_412-local_build-index_deploy-3xl
Collecting job variants for stf-crc-ocp_412-local_build-index_deploy-3xl
Variant <Job stf-crc-ocp_412-local_build-index_deploy-3xl branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#215> matched
Collecting job variants for stf-crc-ocp_412-local_build-index_deploy
Variant <Job stf-crc-ocp_412-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#155> matched
Collecting job variants for stf-crc-local_build-index_deploy
Variant <Job stf-crc-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#118> matched
Collecting job variants for stf-base
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#72> matched
Collecting job variants for stf-base-2node
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:master}} source: infrawatch/service-telemetry-operator/.zuul.yaml@master#18> matched
Collecting job variants for podified-multinode-edpm-deployment-crc
Variant <Job podified-multinode-edpm-deployment-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/edpm_multinode.yaml@main#247> matched
Collecting job variants for cifmw-podified-multinode-edpm-base-crc
Variant <Job cifmw-podified-multinode-edpm-base-crc branches: None source: openstack-k8s-operators/ci-framework/zuul.d/base.yaml@main#121> matched
Collecting job variants for base-extracted-crc
Variant <Job base-extracted-crc branches: None source: config/zuul.d/_jobs-crc.yaml@master#55> matched
Collecting job variants for base-minimal
Variant <Job base-minimal branches: None source: config/zuul.d/jobs.yaml@master#24> matched
Variant <Job stf-base-2node branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#19> did not match
Variant <Job stf-base branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#69> did not match
Variant <Job stf-crc-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#115> did not match
Variant <Job stf-crc-ocp_412-local_build-index_deploy branches: {MatchAny:{ImpliedBranchMatcher:stable-1.5}} source: infrawatch/service-telemetry-operator/[email protected]#152> did not match
Pipeline variant <Job stf-crc-ocp_412-local_build-index_deploy-3xl branches: None source: infrawatch/service-telemetry-operator/.zuul.yaml@master#266> matched

.zuul.yaml Outdated Show resolved Hide resolved
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/bdcfb626e6cf4352b469197f27e81ac7

stf-crc-ocp_412-local_build FAILURE in 22m 46s
stf-crc-ocp_412-local_build-3xl FAILURE in 22m 21s
✔️ stf-crc-ocp_412-local_build-index_deploy SUCCESS in 45m 45s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 20m 15s

Copy link

Zuul encountered a syntax error while parsing its configuration in the
repo infrawatch/service-telemetry-operator on branch master. The
error was:

Job stf-crc-ocp_412-local_build-6xl not defined

The error appears in the following project stanza:

project:
name: infrawatch/service-telemetry-operator
#templates:
# - stf-crc-jobs
periodic:
jobs:
- stf-crc-ocp_412-nightly_bundles
- stf-crc-ocp_414-nightly_bundles
github-check:
jobs:
- stf-crc-ocp_412-local_build
- stf-crc-ocp_412-local_build-3xl
- stf-crc-ocp_412-local_build-4xl
- stf-crc-ocp_412-local_build-5xl
- stf-crc-ocp_412-local_build-6xl
- stf-crc-ocp_412-local_build-index_deploy
- stf-crc-ocp_412-local_build-index_deploy-3xl
- stf-crc-ocp_412-local_build-index_deploy-4xl
- stf-crc-ocp_412-local_build-index_deploy-5xl
- stf-crc-ocp_412-local_build-index_deploy-6xl

in "infrawatch/service-telemetry-operator/.zuul.yaml@master", line 266, column 3

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/845f63d59733448bbee7faedd9255e85

stf-crc-ocp_412-local_build FAILURE in 26m 58s
stf-crc-ocp_412-local_build-3xl FAILURE in 20m 55s
stf-crc-ocp_412-local_build-4xl FAILURE in 27m 17s
stf-crc-ocp_412-local_build-5xl FAILURE in 27m 38s
stf-crc-ocp_412-local_build-6xl FAILURE in 28m 13s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 20m 41s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 27m 06s
stf-crc-ocp_412-local_build-index_deploy-4xl FAILURE in 25m 50s
stf-crc-ocp_412-local_build-index_deploy-5xl FAILURE in 26m 44s
stf-crc-ocp_412-local_build-index_deploy-6xl FAILURE in 23m 42s

@danpawlik
Copy link
Contributor

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/0122cec274ed41ed81f0d7cb7902eeee

stf-crc-ocp_412-local_build FAILURE in 28m 56s
stf-crc-ocp_412-local_build-3xl FAILURE in 27m 51s
stf-crc-ocp_412-local_build-4xl FAILURE in 26m 48s
stf-crc-ocp_412-local_build-5xl FAILURE in 28m 02s
stf-crc-ocp_412-local_build-6xl FAILURE in 28m 37s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 29m 02s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 27m 52s
stf-crc-ocp_412-local_build-index_deploy-4xl FAILURE in 28m 09s
stf-crc-ocp_412-local_build-index_deploy-5xl FAILURE in 25m 59s
stf-crc-ocp_412-local_build-index_deploy-6xl FAILURE in 28m 35s

@elfiesmelfie
Copy link
Collaborator Author

elfiesmelfie commented Jun 4, 2024

recheck

I added a Depends-On for sg-core change

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/fcf12aba9b80404798fe782e4bbcc0d5

stf-crc-ocp_412-local_build FAILURE in 32m 58s
stf-crc-ocp_412-local_build-3xl FAILURE in 32m 36s
stf-crc-ocp_412-local_build-4xl FAILURE in 31m 25s
stf-crc-ocp_412-local_build-5xl FAILURE in 30m 28s
stf-crc-ocp_412-local_build-6xl FAILURE in 30m 44s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 32m 10s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 32m 57s
stf-crc-ocp_412-local_build-index_deploy-4xl FAILURE in 31m 04s
stf-crc-ocp_412-local_build-index_deploy-5xl FAILURE in 34m 12s
stf-crc-ocp_412-local_build-index_deploy-6xl FAILURE in 32m 17s

@danpawlik
Copy link
Contributor

recheck

@danpawlik
Copy link
Contributor

checking if job would be running on different cloud provider.

@danpawlik
Copy link
Contributor

danpawlik commented Jun 5, 2024

Suggest to squash commits and add into the commit message: Depends-On: https://github.com/infrawatch/sg-core/pull/135. Would be easier later to read git log

.zuul.yaml Outdated
nodeset:
nodes:
- name: controller
label: cloud-centos-9-stream-tripleo-vexxhost
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why it does not start CI jobs on other cloud provider.
Consider to change to cloud-centos-9-stream-tripleo so it will not force to spawn instances on Vexxhost.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to be hitting getting RETRY_LIMIT errors since I made this change

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/ea763561b88c4900851925bac7851256

stf-crc-ocp_412-local_build FAILURE in 30m 07s
stf-crc-ocp_412-local_build-3xl FAILURE in 31m 21s
stf-crc-ocp_412-local_build-4xl FAILURE in 31m 05s
stf-crc-ocp_412-local_build-5xl FAILURE in 31m 06s
stf-crc-ocp_412-local_build-6xl FAILURE in 32m 01s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 32m 56s
stf-crc-ocp_412-local_build-index_deploy-3xl FAILURE in 31m 06s
stf-crc-ocp_412-local_build-index_deploy-4xl FAILURE in 32m 36s
stf-crc-ocp_412-local_build-index_deploy-5xl FAILURE in 31m 20s
stf-crc-ocp_412-local_build-index_deploy-6xl FAILURE in 32m 00s

Copy link

Zuul encountered a syntax error while parsing its configuration in the
repo infrawatch/service-telemetry-operator on branch stable-1.5. The
error was:

Nodeset stf-crc_extracted-ocp412 does not match existing definition in branch master

The error appears in the following nodeset stanza:

nodeset:
name: stf-crc_extracted-ocp412
nodes:
- name: controller
label: cloud-centos-9-stream-tripleo-vexxhost
- name: crc
label: coreos-crc-extracted-2-19-0-xxl

in "infrawatch/service-telemetry-operator/[email protected]", line 2, column 3

@elfiesmelfie elfiesmelfie force-pushed the efoley-zuul-bump-412-crc branch from c01e0ed to 2b05669 Compare June 5, 2024 12:05
Copy link

This change depends on a change that failed to merge.

Change infrawatch/prometheus-webhook-snmp#22 is needed.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/357354a203884a63b9143778ef9ec7af

✔️ stf-crc-ocp_412-local_build SUCCESS in 35m 52s
stf-crc-ocp_412-local_build-index_deploy POST_FAILURE in 36m 43s

@elfiesmelfie
Copy link
Collaborator Author

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/695681b92b734db18ba86fa99ec31708

stf-crc-ocp_412-local_build RETRY_LIMIT in 10s
stf-crc-ocp_412-local_build-index_deploy RETRY_LIMIT in 10s

@elfiesmelfie
Copy link
Collaborator Author

recheck

@elfiesmelfie elfiesmelfie force-pushed the efoley-zuul-bump-412-crc branch from 649f33b to b14b546 Compare June 5, 2024 17:23
@elfiesmelfie elfiesmelfie force-pushed the efoley-zuul-bump-412-crc branch from b14b546 to e091546 Compare June 5, 2024 18:12
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/b4e50c8e4ac44c67963735dd3dda7eb3

✔️ stf-crc-ocp_412-local_build SUCCESS in 34m 33s
stf-crc-ocp_412-local_build-3xl RETRY_LIMIT in 3m 50s
stf-crc-ocp_412-local_build-4xl RETRY_LIMIT in 3m 48s
✔️ stf-crc-ocp_412-local_build-index_deploy SUCCESS in 37m 16s
stf-crc-ocp_412-local_build-index_deploy-3xl NODE_FAILURE Node request 100-0007243577 failed in 0s
stf-crc-ocp_412-local_build-index_deploy-4xl NODE_FAILURE Node request 100-0007243578 failed in 0s

@csibbitt
Copy link
Collaborator

csibbitt commented Jun 5, 2024

It seems that the CRC image rebuild solved our problems and more resources are not required after all. Can we just abandon this change?

@elfiesmelfie
Copy link
Collaborator Author

It seems that the CRC image rebuild solved our problems and more resources are not required after all. Can we just abandon this change?

That makes sense. I'm mildly concerned about the "larger" jobs failing, but we're dropping OCP4.12 support once 4.16 is available, so it is not worth looking into.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants