From 221e3827ce3819ba4a4747deb4958aae171ad9bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Mon, 26 Jun 2023 13:41:14 -0700 Subject: [PATCH 1/3] Refactor groovy script to include batching. The last several years, we've added batching to the this template script when executing it on the official build farms but never actually updated it here. This change includes what is essentially, a rewrite of the batch process example script to more closely match the one I've been using the last year or so. I someday hope to get even more meta about this and create a management job on Jenkins that runs variations of this script on templated job types in order to remove the need for batching and the chance for a typo when copy-pasting and modifying the example script. --- doc/ongoing_operations.rst | 68 +++++++++++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 15 deletions(-) diff --git a/doc/ongoing_operations.rst b/doc/ongoing_operations.rst index 5ef294b97..682bf5dde 100644 --- a/doc/ongoing_operations.rst +++ b/doc/ongoing_operations.rst @@ -81,34 +81,72 @@ Perform action on a set of jobs ------------------------------- Sometimes you want to do bulk actions like disable or delete all jobs of a specific distro or just one target. -We recommend running a Groovy scripts using the script console. +We recommend running a Groovy script using the script console. + +You can change the distro letter or add or remove prefixes from the list in order to reduce the scope of targeted jobs. + +Some operations take time and an HTTP timeout creates performance issues with dangling script runs. +To prevent them the example script uses a ``BATCH_SIZE`` constant to control how many jobs to change before returning. +You can leave the default, fairly conservative batch size or increase that constant steadily until results are no longer instantaneous and then adjusting back down. +Running the script repeatedly until there are no longer any remaining jobs to process will work as long as non-destructive changes, like enabling or disabling jobs, are properly skipped. The following Groovy script is a good starting point for various actions: .. code-block:: groovy - import hudson.model.Cause + DISTRO_LETTER = "F" + PREFIXES = ["ci_", "dev_", "pr_", "rel", "src_", "bin_"].collect({pre -> DISTRO_LETTER + pre}) - for (p in Jenkins.instance.allItems) { - if ( - p.name.startsWith("PREFIX1__") || - p.name.startsWith("PREFIX2__") || - ... || - p.name.startsWith("PREFIXn__")) - { - println(p.name) + def starts_with_any_prefix(name) { + for (prefix in PREFIXES) { + if (name.startsWith(prefix)) { + return true + } + } + return false + } - // p.disable() - // p.enable() - // p.scheduleBuild(new Cause.UserIdCause()) + /* Some operations take time and an http timeout + * creates performance issues with dangling script + * runs. + * To prevent them use a batch size that returns + * fairly instant results and run the script multiple + * times + */ + BATCH_SIZE = 100 - // p.delete() + count = 0 + for (job in Jenkins.get().getItems({job -> starts_with_any_prefix(job.name)})) + { + if (count >= BATCH_SIZE) + { + println("Reached ${BATCH_SIZE} limit before processing ${job.name}.") + break } + + /* Disable a job if it is not already disabled */ + // if (job.isDisabled()) { continue } + // job.disable() + + /* Enable a job if it is currently disabled */ + // if (!job.isDisabled()) { continue } + // job.enable() + + /* Delete a job! This action is irreversable! */ + // job.delete() + + /* Increase count for batch processing. */ + count++ + } + + if (count <= BATCH_SIZE) + { + println("Completed execution of the last batch.") } This script will print only the matched job names. -You can uncomment any of the actions to disable, enable, trigger or delete these projects. +You can uncomment any of the actions to disable, enable, or delete these projects. To run a Groovy script: From 1505d62e2a9500da051cc222802332729b5ac9b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Wed, 5 Jul 2023 13:41:11 -0700 Subject: [PATCH 2/3] Fix shadwowed variable name. Groovy does not allow a closure variable to shadow another one. Co-authored-by: Dharini Dutia --- doc/ongoing_operations.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/ongoing_operations.rst b/doc/ongoing_operations.rst index 682bf5dde..a00169120 100644 --- a/doc/ongoing_operations.rst +++ b/doc/ongoing_operations.rst @@ -117,7 +117,7 @@ The following Groovy script is a good starting point for various actions: BATCH_SIZE = 100 count = 0 - for (job in Jenkins.get().getItems({job -> starts_with_any_prefix(job.name)})) + for (job in Jenkins.get().getItems({j -> starts_with_any_prefix(j.name)})) { if (count >= BATCH_SIZE) { From 5a80a00ddb8459f78d8fd3fe2c002487ba3d3523 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Wed, 26 Jul 2023 09:48:26 -0700 Subject: [PATCH 3/3] Print job name on each run as a progress indicator. Co-authored-by: Dharini Dutia --- doc/ongoing_operations.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/ongoing_operations.rst b/doc/ongoing_operations.rst index a00169120..50c2bdb26 100644 --- a/doc/ongoing_operations.rst +++ b/doc/ongoing_operations.rst @@ -136,6 +136,8 @@ The following Groovy script is a good starting point for various actions: /* Delete a job! This action is irreversable! */ // job.delete() + println(job.name) + /* Increase count for batch processing. */ count++ }