fix(dockerfiles): fix the eof problem of centos for ci cd builder ima…

…ges (#341) ### **User description** Signed-off-by: wuhuizuo <[email protected]> ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Updated CentOS 7 repository URLs across multiple Dockerfiles due to CentOS 7 reaching EOL. - Removed `centos-release-scl` from package installations where applicable. - Updated base image versions from `v1.8.0` to `v1.9.0` across multiple Dockerfiles. - Added `unzip` package installation in `dockerfiles/cd/builders/tikv/Dockerfile`. - Updated Go version to `1.21.12`, ORAS tool to `1.2.0`, and Git tool to `2.45.2` in `dockerfiles/ci/base/Dockerfile`. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>3 files</summary><table> <tr> <td> <details> <summary><strong>skaffold.yaml</strong><dd><code>Update Skaffold template tag version</code>                                          </dd></summary> <hr> dockerfiles/bases/skaffold.yaml - Updated `template` tag from `v1.8.0-fips` to `v1.9.0-fips`. </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-acf8f8ffe238d9298c752e4419514358c4ddd51fd1df7a5bf765f1b062920eb7">+1/-1</a>      </td> </tr> <tr> <td> <details> <summary><strong>fips.Dockerfile</strong><dd><code>Update TiKV base image version</code>                                                      </dd></summary> <hr> dockerfiles/bases/tikv-base/fips.Dockerfile - Updated base image version from `v1.8.0` to `v1.9.0`. </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-6e24fa17a22270af67a366a253d7a7907f2fbcef749a9954184d0c9750fd3a5e">+1/-1</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Update Go, ORAS, and Git versions</code>                                                </dd></summary> <hr> dockerfiles/ci/base/Dockerfile <li>Updated Go version from <code>1.21.10</code> to <code>1.21.12</code>.<br> <li> Updated ORAS tool to use version <code>1.2.0</code> from Bitnami.<br> <li> Updated Git tool to use version <code>2.45.2</code> from Bitnami.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-83674330ea927f554d17eb158462495449cc6fb2865d9874c1e8b1acd6441025">+5/-34</a>    </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>8 files</summary><table> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/ng-monitoring/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Removed <code>centos-release-scl</code> from package installation.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-ecaf493a7b859f878916d155747fa536a0d10fe0e3870c9843685c2acf8dda9d">+6/-2</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/pd/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Removed <code>centos-release-scl</code> from package installation.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-0de530f1b12622877887d3eade5a9e27a722105609d6d57ea825041f5656f934">+6/-2</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/tidb-dashboard/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-bed7eec8462976cce3381568b5199d178d8f8166c4b90009ccd8e470c48afa23">+5/-1</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/tidb/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Removed <code>centos-release-scl</code> from package installation.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-1845e6cdfe6203806bce900f004783e8ef2f9691e27e81e34d1f42e9d2e3a64f">+6/-2</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/tiflash/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-e02766e1fc3c8a5c45039033aa80fa8001e65c3b11e3b2fd80904f47cffbbf9e">+5/-1</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues and update base image</code>                        </dd></summary> <hr> dockerfiles/cd/builders/tiflow/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Removed <code>centos-release-scl</code> from package installation.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-86df9347a078bec997d51dff6a2475dfe77275d00dd9bd63f5d484ce41503161">+7/-3</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues, add unzip, and update base image</code></dd></summary> <hr> dockerfiles/cd/builders/tikv/Dockerfile <li>Added commands to update CentOS 7 repository URLs due to EOL.<br> <li> Removed <code>centos-release-scl</code> from package installation.<br> <li> Added <code>unzip</code> package installation.<br> <li> Updated base image version from <code>v1.8.0</code> to <code>v1.9.0</code>.<br> </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-da122fa590e9e91011c5c155781b8794076994c39bfba2a06f16f2b36fa01471">+8/-3</a>      </td> </tr> <tr> <td> <details> <summary><strong>Dockerfile</strong><dd><code>Fix CentOS 7 EOL issues</code>                                                                    </dd></summary> <hr> dockerfiles/ci/release-build-base/Dockerfile - Added commands to update CentOS 7 repository URLs due to EOL. </details> </td> <td><a href="https://github.com/PingCAP-QE/artifacts/pull/341/files#diff-062684ee95fd2f99424d58836e5a399ce2919269e5be11c56ecaaf71b46d14fb">+7/-1</a>      </td> </tr> </table></details></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: >Comment `/help` on the PR to get a list of all available PR-Agent tools and their descriptions --------- Signed-off-by: wuhuizuo <[email protected]>
PingCAP-QE · Jul 3, 2024 · f60f629 · f60f629
1 parent b84541a
commit f60f629
Show file tree

Hide file tree

Showing 11 changed files with 57 additions and 51 deletions.
diff --git a/dockerfiles/bases/skaffold.yaml b/dockerfiles/bases/skaffold.yaml
@@ -118,7 +118,7 @@ build:
         cache: {}
   tagPolicy:
     customTemplate:
-      template: "v1.8.0-fips"
+      template: "v1.9.0-fips"
   cluster:
     concurrency: 0
     randomDockerConfigSecret: false

diff --git a/dockerfiles/bases/tikv-base/fips.Dockerfile b/dockerfiles/bases/tikv-base/fips.Dockerfile
@@ -1,4 +1,4 @@
-ARG PINGCAP_BASE=ghcr.io/pingcap-qe/bases/pingcap-base:v1.8.0
+ARG PINGCAP_BASE=ghcr.io/pingcap-qe/bases/pingcap-base:v1.9.0
 FROM $PINGCAP_BASE
 # wget is requested by operator
 RUN dnf install -y tzdata wget openssl && dnf clean all

diff --git a/dockerfiles/cd/builders/ng-monitoring/Dockerfile b/dockerfiles/cd/builders/ng-monitoring/Dockerfile
@@ -8,10 +8,14 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 # install packages.
 RUN yum update --nogpgcheck -y && \
-    yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    yum install --nogpgcheck -y epel-release deltarpm && \
     yum update --nogpgcheck -y && \
     yum groupinstall --nogpgcheck -y "Development Tools" && \
     yum install -y wget && \
@@ -33,7 +37,7 @@ RUN GOPROXY=${GOPROXY} make default -C /ws
 RUN /ws/bin/ng-monitoring-server -V
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/ng-monitoring-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/ng-monitoring-base:v1.9.0
 
 COPY --from=building /ws/bin/ng-monitoring-server /ng-monitoring-server
 

diff --git a/dockerfiles/cd/builders/pd/Dockerfile b/dockerfiles/cd/builders/pd/Dockerfile
@@ -7,10 +7,14 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 # install packages.
 RUN yum update --nogpgcheck -y && \
-    yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    yum install --nogpgcheck -y epel-release deltarpm && \
     yum update --nogpgcheck -y && \
     yum groupinstall --nogpgcheck -y "Development Tools" && \
     yum install --nogpgcheck -y which && \
@@ -34,7 +38,7 @@ RUN /pd/bin/pd-ctl -V
 RUN /pd/bin/pd-recover -V
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/pd-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/pd-base:v1.9.0
 
 COPY --from=building /pd/bin/pd-server  /pd-server
 COPY --from=building /pd/bin/pd-ctl     /pd-ctl

diff --git a/dockerfiles/cd/builders/tidb-dashboard/Dockerfile b/dockerfiles/cd/builders/tidb-dashboard/Dockerfile
@@ -8,6 +8,10 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 # install packages.
 RUN yum -y update && \
@@ -36,7 +40,7 @@ RUN GOPROXY=${GOPROXY} make package -C /ws
 RUN /ws/bin/tidb-dashboard -V
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/pd-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/pd-base:v1.9.0
 
 COPY --from=building /ws/bin/tidb-dashboard /tidb-dashboard
 

diff --git a/dockerfiles/cd/builders/tidb/Dockerfile b/dockerfiles/cd/builders/tidb/Dockerfile
@@ -8,10 +8,14 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 # install packages.
 RUN yum update --nogpgcheck -y && \
-    yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    yum install --nogpgcheck -y epel-release deltarpm && \
     yum update --nogpgcheck -y && \
     yum groupinstall --nogpgcheck -y "Development Tools" && \
     yum clean all
@@ -32,7 +36,7 @@ RUN GOPROXY=${GOPROXY} make server -C /tidb
 RUN /tidb/bin/tidb-server -V
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/tidb-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/tidb-base:v1.9.0
 
 COPY --from=building /tidb/bin/tidb-server /tidb-server
 

diff --git a/dockerfiles/cd/builders/tiflash/Dockerfile b/dockerfiles/cd/builders/tiflash/Dockerfile
@@ -7,6 +7,10 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 LABEL org.opencontainers.image.authors "[email protected]"
 LABEL org.opencontainers.image.description "binary builder for tiflash"
@@ -37,7 +41,7 @@ RUN mkdir output && mv release-centos7-llvm/tiflash output/tiflash
 RUN output/tiflash/tiflash version
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/tiflash-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/tiflash-base:v1.9.0
 
 ENV LD_LIBRARY_PATH /tiflash
 COPY --from=building /tiflash/output/tiflash /tiflash

diff --git a/dockerfiles/cd/builders/tiflow/Dockerfile b/dockerfiles/cd/builders/tiflow/Dockerfile
@@ -7,10 +7,14 @@
 
 ########### stage: Builder - cdc
 FROM centos:7.9.2009 as builder-cdc
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 # install packages.
 RUN yum update --nogpgcheck -y && \
-    yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    yum install --nogpgcheck -y epel-release deltarpm && \
     yum update --nogpgcheck -y && \
     yum groupinstall --nogpgcheck -y "Development Tools" && \
     yum clean all
@@ -64,14 +68,14 @@ RUN /tiflow/bin/dm-syncer -V
 RUN /tiflow/bin/dmctl -V
 
 ########### stage: Final image - cdc
-FROM ghcr.io/pingcap-qe/bases/tools-base:v1.8.0 as final-cdc
+FROM ghcr.io/pingcap-qe/bases/tools-base:v1.9.0 as final-cdc
 
 COPY --from=building-cdc /tiflow/bin/cdc /cdc
 EXPOSE 8300
 CMD ["/cdc"]
 
 ########### stage: Final image - dm
-FROM ghcr.io/pingcap-qe/bases/tools-base:v1.8.0 as final-dm
+FROM ghcr.io/pingcap-qe/bases/tools-base:v1.9.0 as final-dm
 
 COPY --from=building-dm /tiflow/bin/dm-master /dm-master
 COPY --from=building-dm /tiflow/bin/dm-worker /dm-worker

diff --git a/dockerfiles/cd/builders/tikv/Dockerfile b/dockerfiles/cd/builders/tikv/Dockerfile
@@ -7,12 +7,17 @@
 
 ########### stage: Builder
 FROM centos:7.9.2009 as builder
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
+
 # install packages.
 ARG DEVTOOLSET_VER=8
 RUN yum update --nogpgcheck -y && \
-    yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    yum install --nogpgcheck -y epel-release deltarpm && \
     yum update --nogpgcheck -y && \
-    yum install -y git devtoolset-${DEVTOOLSET_VER} perl cmake3 && \
+    yum install -y git devtoolset-${DEVTOOLSET_VER} perl cmake3 unzip && \
     yum clean all && \
     ln -s /usr/bin/cmake3 /usr/bin/cmake
 ENV DEVTOOLSET_VER ${DEVTOOLSET_VER}
@@ -38,7 +43,7 @@ RUN --mount=type=cache,target=/tikv/target \
 RUN /tikv/bin/tikv-server --version
 
 ########### stage: Final image
-FROM ghcr.io/pingcap-qe/bases/tikv-base:v1.8.0
+FROM ghcr.io/pingcap-qe/bases/tikv-base:v1.9.0
 
 ENV MALLOC_CONF="prof:true,prof_active:false"
 COPY --from=building /tikv/bin/tikv-server  /tikv-server

diff --git a/dockerfiles/ci/base/Dockerfile b/dockerfiles/ci/base/Dockerfile
@@ -1,6 +1,6 @@
 # Base image
 # renovate: datasource=docker depName=golang
-ARG GOLANG_VERSION=1.21.10
+ARG GOLANG_VERSION=1.21.12
 FROM golang:${GOLANG_VERSION}
 
 LABEL go-version "${GOLANG_VERSION}"
@@ -31,13 +31,7 @@ RUN OS=linux; ARCH=$([ "$(arch)" = "x86_64" ] && echo amd64 || echo arm64); \
 
 # oras tool
 # renovate: datasource=github-tags depName=oras-project/oras
-ARG ORAS_VERSION=v1.1.0
-RUN OS=linux; ARCH=$([ "$(arch)" = "x86_64" ] && echo amd64 || echo arm64); \
-    curl -LO "https://github.com/oras-project/oras/releases/download/${ORAS_VERSION}/oras_${ORAS_VERSION#?}_${OS}_${ARCH}.tar.gz" && \
-    mkdir -p oras-install/ && \
-    tar -zxf oras_*_${OS}_${ARCH}.tar.gz -C oras-install/ && \
-    install oras-install/oras /usr/local/bin/ && \
-    rm -rf oras_*_${OS}_${ARCH}.tar.gz oras-install/
+COPY --from=bitnami/oras:1.2.0 /oras /usr/local/bin/oras
 
 # codecov tool
 # renovate: datasource=github-tags depName=codecov/uploader
@@ -73,30 +67,7 @@ RUN curl -sL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - && \
     apt-get clean && \
     npm install -g yarn pnpm
 
-
-# update  some tools versions to fix security issues
-# update git to 2.39.4
-RUN apt-get remove git -y
-RUN apt-get update && \
-    apt-get install -y \
-    wget \
-    build-essential \
-    libssl-dev \
-    libcurl4-gnutls-dev \
-    libexpat1-dev \
-    gettext \
-    unzip \
-    zlib1g-dev \
-    autoconf \
-    dh-autoreconf
+# Upgrade Git tool 
 # renovate: datasource=github-tags depName=git/git
-ARG GIT_VERSION=2.39.5
-RUN wget https://github.com/git/git/archive/refs/tags/v$GIT_VERSION.zip -O git.zip && \
-    unzip git.zip && \
-    cd git-$GIT_VERSION && \
-    make configure && \
-    ./configure --prefix=/usr/local && \
-    make all && \
-    make install && \
-    cd .. && \
-    rm -rf git-$GIT_VERSION git.zip
+COPY --from=bitnami/git:2.45.2 /opt/bitnami/git/bin/git /usr/bin/git 
+
diff --git a/dockerfiles/ci/release-build-base/Dockerfile b/dockerfiles/ci/release-build-base/Dockerfile
@@ -1,6 +1,10 @@
 # Base image
 # renovate: datasource=docker depName=centos
 FROM centos:7.9.2009
+# CentOS 7 has reached EOL
+RUN sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo \
+    && sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
 
 LABEL org.opencontainers.image.authors "[email protected]"
 LABEL org.opencontainers.image.description "CD Base image for building"
@@ -9,6 +13,9 @@ LABEL org.opencontainers.image.source = "https://github.com/PingCAP-QE/artifacts
 # install packages.
 RUN yum update --nogpgcheck -y && \
     yum install --nogpgcheck -y epel-release centos-release-scl deltarpm && \
+    sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo && \
+    sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo && \
+    sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo && \
     yum update --nogpgcheck -y && \
     yum groupinstall --nogpgcheck -y "Development Tools" && \
     yum install --nogpgcheck -y \
@@ -130,7 +137,6 @@ RUN curl -sSL https://github.com/pingcap/tiflash/raw/${TIFLASH_TAG}/release-cent
     source ./install_ccache.sh && \
     install_ccache ${CCACHE_VERSION}
 
-# Git, update to 2.40.2 to fix the security issue
 # renovate: datasource=github-tags depName=git/git
 ARG GIT_VERSION=2.40.3
 RUN wget https://github.com/git/git/archive/refs/tags/v$GIT_VERSION.zip -O git.zip && \