infra-team-sync-2022-11-08

🎥 Meeting Recording
📆 Shared Calendar
💬 community.jenkins.io & IRC Chat Room #jenkins-infra
📧 Google Group (mailing list) jenkins-infra

Attendees 👥

• @dduportal (Damien Duportal)
• @lemeurherve (Hervé Le Meur)
• @smerle33 (Stéphane Merle)

Announcements 📢

1. Weekly:

• Base release is OK
• Container image now available

2. 11th of November is banking day in part of the EU

Upcoming Calendar 📆

• Next Weekly: 2022-11-15 (maybe?)
• Next LTS: 2022-11-30
• Next Security Release: 2022-11-15
• Next major event: FOSDEM (3-4 February 2023 in Brussels)

Notes 📖

• Done:

  • Sign Up
  • Login
  • Missing datadog metrics for the prodpublick8s AKS cluster
  • [From Platform SIG] Remove PPC64 mentions on ci.jenkins.io
  • Artifact downloads failed on agent using repo cache
  • Access to npm namespace
  • Archive a few Jira components of NotMyFault’s plugins
  • Windows ACI 11 agent broken: no git found
  • [INFRA-2939] Request permission elevation on oracle-cloud-infrastructure-compute-plugin-developers
  • Archive TobiX’s Jira plugin components
  • https://twitter.com/jenkins_release is many weeks behind
  • ci.jenkins.io does not have enough GH permissions to check trust on jenkins-infra/jenkins-codeql
  • Hosting team onboarding
  • Upgrade to Kubernetes 1.23
  • Not receiving reset account password email
  • Crowdin localization for lockable-resources
  • update center json returning 404
  • password

• Work In Progress:

  • https://ci.jenkins.io/job/Infra/job/stories/ is not handling PRs
    • Missing permissions
    • New Job location on ci.jenkins.io
    • WiP: merge pipelines + packer image adding missing tools
    • Damien and Stephane
  • Login Issue
    • TBD
  • forget password
    • TBD
  • Windows agents on ci.jenkins.io disconnect prematurely
    • No incident
    • But label windows to be remove and/or acceptance test to be updated
  • Node.js missing on agent
    • Issue ASDF + NodeJS and the env.
    • Should we use official NodeJS?
    • Alternatives: NVM
    • => install node from official installer, first on docker-builder, then on packer images
  • Publish pipeline-steps-doc-generator and backend-extension-indexer artifacts to some kind of storage
    • Done, but need to enable CI for public visibility (same as stories)
    • Damien and Stephane
  • Artifact caching proxy unable to deliver git-changelist-maven-extension
    • 2 issues:
      • one is from maven central (and not ACP)
      • second comes from the azure’s ACP (HTTP/504). Checked in datadog: sounds like network issues on prodpublick8s (IP overlapping).
    • Let’s prioritize the privatek8S + publick8s fixups.
  • Add an help desk template for account recovery issues
    • Herve will open a draft open for feedbacks
  • Backlog of already built items in queue on ci.jenkins.io
    • Waiting for the 16th
  • (Re) Introduce an artifact caching proxy for ci.jenkins.io
    • 3 ACPs are running
    • We get datadog metrics and logs for all of them
    • Herve watching it, but priority on “Azure IP overlap” solving
    • Improvement on the pipeline-library:
      • Define “available” ACPs as env vars (feature flag)
      • Run an healthcheck before a build, to fallback to JFrog if current ACP is down
      • (and some minor fixes ;))
  • Migrate Keycloak database to Azure
    • New database created in Azure (on the current “public” postgres instance)
    • WiP: initial backup/restore
  • [INFRA-2754] Realign repo.jenkins-ci.org mission
    • Migration to be defined, old instance to be kept one month
    • Bandwidth decrease: Mark and Damien to write and hack to find a proper path
    • Work on HA LDAP (at least read-only replica) to be started
  • AKS: add cluster privatek8s
    • Create a new cluster with terraform using a managed identity instead of a service principal
    • Migrate services from temp-privatek8s to this new cluster
    • Migrate services from prodpublick8s to this new private cluster
      • infra, release
      • monitoring
      • bots (only need egress)
  • (proposal) Add external-dns to the Kubernetes cluster with an ingress (and a loadbalancer)
    • Go back to the backlog!