Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OPSEXP-2801: Add Search service image build support #53

Merged
merged 31 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
30bb19d
alfresco-search - add basic bake config
alxgomz Sep 24, 2024
a7bc90c
alfresco-search - add solr artifact
alxgomz Sep 25, 2024
9e98e46
alfresco-search - add Dockerfile
alxgomz Sep 25, 2024
e02450a
add Solr entrypoint script
alxgomz Sep 30, 2024
aeb48c8
add replication support through entrypoint
alxgomz Sep 30, 2024
a7f26c9
add a community compose file
alxgomz Sep 30, 2024
03ee753
add a community test
alxgomz Sep 30, 2024
8309e5c
add search build in default all
alxgomz Sep 30, 2024
064625d
fix community docker compose test
alxgomz Oct 1, 2024
4e156de
remove not yet implemented sync-service - tbd in OPSEXP-2869
alxgomz Oct 1, 2024
b28d8f3
remove not yet implemented adw - tbd in OPSEXP-2868
alxgomz Oct 1, 2024
0063f82
allow both 401 & 403 for solr api protection
alxgomz Oct 1, 2024
17d2d6c
add search api test
alxgomz Oct 1, 2024
5f5a138
typo
alxgomz Oct 2, 2024
8738c05
add solr directory factory settings
alxgomz Oct 2, 2024
d6d38dc
add search service image readme file
alxgomz Oct 2, 2024
c163806
Revert "add search api test"
alxgomz Oct 2, 2024
27f3d7d
use regular comunity image while waiting for OPSEXP-2843
alxgomz Oct 2, 2024
f6b0d21
Revert "remove not yet implemented adw - tbd in OPSEXP-2868"
alxgomz Oct 2, 2024
520fa8a
partial revert of 4e156de
alxgomz Oct 2, 2024
f4165e9
always save logs as artifacts
alxgomz Oct 2, 2024
d7976ad
test compose using a matrix
alxgomz Oct 2, 2024
9deea81
test with dbp-chart builtools action fix
alxgomz Oct 2, 2024
08553c2
community compose fixup
alxgomz Oct 2, 2024
b54c85d
[skip ci] update search service readme
alxgomz Oct 3, 2024
ac64283
actions bumps
alxgomz Oct 3, 2024
6820e83
review comments
alxgomz Oct 3, 2024
33659fd
review comments
alxgomz Oct 3, 2024
f191614
remove unnecessary ENV
alxgomz Oct 3, 2024
e689765
review comments
alxgomz Oct 3, 2024
544672d
review comments
alxgomz Oct 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 22 additions & 14 deletions .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
permissions:
contents: write
steps:
- uses: Alfresco/alfresco-build-tools/.github/actions/pre-commit@v6.0.0
- uses: Alfresco/alfresco-build-tools/.github/actions/pre-commit@v7.1.1
with:
auto-commit: "true"

Expand All @@ -53,7 +53,7 @@ jobs:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- uses: Alfresco/alfresco-build-tools/.github/actions/free-hosted-runner-disk-space@v5.34.0
- uses: Alfresco/alfresco-build-tools/.github/actions/free-hosted-runner-disk-space@v7.1.1

- name: Setup nexus authentication
run: |
Expand All @@ -62,7 +62,7 @@ jobs:
echo "password ${{ secrets.NEXUS_PASSWORD }}" >> ~/.netrc

- name: Restore packages artifacts
uses: actions/cache/restore@v4
uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
id: artifacts-cache
with:
key: ${{ runner.os }}-packages-${{ hashFiles('**/artifacts.json') }}
Expand All @@ -79,7 +79,7 @@ jobs:

- name: Save packages artifacts
id: cache-primes-save
uses: actions/cache/save@v4
uses: actions/cache/save@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
key: ${{ steps.artifacts-cache.outputs.cache-primary-key }}
path: |
Expand Down Expand Up @@ -111,7 +111,7 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}

- name: Collect Workflow Telemetry
uses: catchpoint/workflow-telemetry-action@v2
uses: catchpoint/workflow-telemetry-action@94c3c3d9567a0205de6da68a76c428ce4e769af1 # v2.0.0
with:
comment_on_pr: false

Expand All @@ -127,6 +127,10 @@ jobs:
name: compose-test
needs: build-test
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
edition: [enterprise, community]
steps:
- name: Log in to the Container registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
Expand All @@ -137,17 +141,21 @@ jobs:

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1

- name: Verify docker-compose
uses: Alfresco/alfresco-build-tools/.github/actions/dbp-charts/[email protected]

- name: Verify docker-compose (${{ matrix.edition }})
id: verify_compose
uses: Alfresco/alfresco-build-tools/.github/actions/dbp-charts/[email protected]
timeout-minutes: 10
with:
compose_file_path: test/docker-compose.yml
compose_file_path: test/${{ matrix.edition }}-docker-compose.yml
quay_username: ${{ secrets.QUAY_USERNAME }}
quay_password: ${{ secrets.QUAY_PASSWORD }}

- uses: Alfresco/alfresco-build-tools/.github/actions/[email protected]

- name: Save containers logs (${{ matrix.edition }})
if: always() && steps.verify_compose.outcome != 'skipped'
uses: Alfresco/alfresco-build-tools/.github/actions/[email protected]
with:
output-archive-name: ${{ matrix.edition }}-logs

helm-test:
strategy:
Expand All @@ -159,7 +167,7 @@ jobs:
runs-on: ${{ matrix.arch == 'arm64' && 'alfrescoARM-ubuntu2404-16G-4CPU' || 'alfrescoPub-ubuntu2204-16G-4CPU' }}
steps:
- name: Setup cluster
uses: Alfresco/alfresco-build-tools/.github/actions/setup-kind@v5.35.0
uses: Alfresco/alfresco-build-tools/.github/actions/setup-kind@v7.1.1
with:
ingress-nginx-ref: controller-v1.8.2
metrics: "true"
Expand Down Expand Up @@ -192,10 +200,10 @@ jobs:
--type=kubernetes.io/dockerconfigjson

- name: Checkout acs-deployment sources
uses: actions/checkout@v4
uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
with:
repository: Alfresco/acs-deployment
ref: ${{ env.ACS_CHART_VERSION }}
ref: ${{ env.ACS_CHART_VERSION }}
path: github-source

- name: Setup helm repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-make.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
make-test:
runs-on: ubuntu-latest
steps:
- uses: Alfresco/alfresco-build-tools/.github/actions/free-hosted-runner-disk-space@v5.34.0
- uses: Alfresco/alfresco-build-tools/.github/actions/free-hosted-runner-disk-space@v7.1.1

- name: Setup nexus authentication
run: |
Expand Down
37 changes: 36 additions & 1 deletion docker-bake.hcl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
group "default" {
targets = ["content_service", "enterprise-search", "ats", "tengines", "connectors"]
targets = ["content_service", "enterprise-search", "ats", "tengines", "connectors", "search_service"]
}

group "content_service" {
Expand Down Expand Up @@ -576,3 +576,38 @@ target "share" {
output = ["type=docker"]
platforms = split(",", "${TARGETARCH}")
}

variable "ALFRESCO_SOLR_DIST_DIR" {
default = "/opt/alfresco-search-services"
}

variable "ALFRESCO_SOLR_USER_NAME" {
default = "solr"
}

variable "ALFRESCO_SOLR_USER_ID" {
default = "33007"
}

target "search_service" {
context = "./search/service"
dockerfile = "Dockerfile"
inherits = ["java_base"]
contexts = {
java_base = "target:java_base"
}
args = {
ALFRESCO_SOLR_GROUP_NAME = "${ALFRESCO_GROUP_NAME}"
ALFRESCO_SOLR_GROUP_ID = "${ALFRESCO_GROUP_ID}"
ALFRESCO_SOLR_USER_NAME = "${ALFRESCO_SOLR_USER_NAME}"
ALFRESCO_SOLR_USER_ID = "${ALFRESCO_SOLR_USER_ID}"
ALFRESCO_SOLR_DIST_DIR = "${ALFRESCO_SOLR_DIST_DIR}"
}
labels = {
"org.opencontainers.image.title" = "${PRODUCT_LINE} Search Service (Solr)"
"org.opencontainers.image.description" = "Alfresco Search Service (Solr)"
}
tags = ["${REGISTRY}/${REGISTRY_NAMESPACE}/alfresco-search-service:${TAG}"]
output = ["type=docker"]
platforms = split(",", "${TARGETARCH}")
}
2 changes: 1 addition & 1 deletion repository/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ docker buildx bake repository

### Alfresco repository configuration

All preperties you would normally add in the alfresco-global.properties file can
All properties you would normally add in the alfresco-global.properties file can
be added in the `JAVA_OPTS` environment variable to the container.

For example, to set the database URL, you can use the following environment
Expand Down
66 changes: 66 additions & 0 deletions search/service/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
ARG DISTRIB_NAME
ARG DISTRIB_MAJOR

FROM java_base as solr_build

ARG ALFRESCO_SOLR_DIST_DIR

USER root

COPY distribution /tmp/

RUN yum install -y xmlstarlet unzip && \
unzip /tmp/*.zip -d $(dirname ${ALFRESCO_SOLR_DIST_DIR}) && \
mkdir -p ${ALFRESCO_SOLR_DIST_DIR}/data && \
mv ${ALFRESCO_SOLR_DIST_DIR}/solrhome/alfrescoModels ${ALFRESCO_SOLR_DIST_DIR}/data/ && \
for i in ${ALFRESCO_SOLR_DIST_DIR}/solrhome/templates/*/conf/solrconfig.xml; do \
xmlstarlet ed -L \
-d '//comment()' \
-s '/config/requestHandler[@name="/replication"]' -t elem -n lst \
-a '/config/requestHandler[@name="/replication"]/lst[last()]' -t attr -n name -v '${solr.replication.master.role:notMaster}' \
-s '/config/requestHandler[@name="/replication"]/lst[last()]' -t elem -n str -v '${solr.replication.replicate.after:commit,startup}' \
-a '/config/requestHandler[@name="/replication"]/lst[last()]/str[last()]' -t attr -n name -v replicateAfter \
-s '/config/requestHandler[@name="/replication"]/lst[last()]' -t elem -n str -v '${solr.replication.conf.files:schema.xml,stopwords.txt}' \
-a '/config/requestHandler[@name="/replication"]/lst[last()]/str[last()]' -t attr -n name -v confFiles \
-s '/config/requestHandler[@name="/replication"]' -t elem -n 'lst' \
-a '/config/requestHandler[@name="/replication"]/lst[last()]' -t attr -n name -v '${solr.replication.slave.role:notSlave}' \
-s '/config/requestHandler[@name="/replication"]/lst[last()]' -t elem -n str -v '${solr.replication.master.url:https://primary.domain.tld/solr/corename/replication}' \
-a '/config/requestHandler[@name="/replication"]/lst[last()]/str[last()]' -t attr -n name -v masterUrl \
-s '/config/requestHandler[@name="/replication"]/lst[last()]' -t elem -n str -v '${solr.replication.poll.interval:00:00:30}' \
-a '/config/requestHandler[@name="/replication"]/lst[last()]/str[last()]' -t attr -n name -v pollInterval \
$i; \
done

FROM java_base AS solr-rhlike

ARG ALFRESCO_SOLR_USER_ID
ARG ALFRESCO_SOLR_USER_NAME
ARG ALFRESCO_SOLR_GROUP_ID
ARG ALFRESCO_SOLR_GROUP_NAME
ARG ALFRESCO_SOLR_DIST_DIR

ENV ALFRESCO_TEMPLATE=rerank

USER root

COPY --chown=:${ALFRESCO_SOLR_GROUP_ID} --from=solr_build ${ALFRESCO_SOLR_DIST_DIR} ${ALFRESCO_SOLR_DIST_DIR}
COPY entrypoint.sh /

WORKDIR ${ALFRESCO_SOLR_DIST_DIR}

RUN groupadd -g ${ALFRESCO_SOLR_GROUP_ID} ${ALFRESCO_SOLR_GROUP_NAME} && \
useradd -u ${ALFRESCO_SOLR_USER_ID} -g ${ALFRESCO_SOLR_GROUP_NAME} ${ALFRESCO_SOLR_USER_NAME} && \
yum install gettext -y && \
yum clean all && \
rm -rf /var/cache/yum && \
chmod g+w -R {logs,data,solrhome} && \
chmod +x /entrypoint.sh


VOLUME ["${ALFRESCO_SOLR_DIST_DIR}/data","${ALFRESCO_SOLR_DIST_DIR}/solrhome","${ALFRESCO_SOLR_DIST_DIR}/keystores"]

EXPOSE 8983

USER ${ALFRESCO_SOLR_USER_NAME}

CMD ["/entrypoint.sh"]
Loading
Loading