Making changes to support the Jenkins CD flow

[TooDissing]

Adding requested changes mentioned here: jenkins-infra/repository-permissions-updater#3746. To enable the CD process for this repo.

Testing done

Submitter checklist

Preview Give feedback

1. Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
2. Ensure that the pull request title represents the desired changelog entry
3. Please describe what you did
4. Link to relevant issues in GitHub or Jira
5. Link to relevant pull requests, esp. upstream and downstream changes
6. Ensure you have provided tests - that demonstrates feature works or fixes the issue