[Bug]: CI/CD - Build step takes up to 14min - Caching not working

[niklasr22]

Current Behavior

The Test build step of the .github/workflows/build.yml workflow takes up to 14min to complete.
It seems like the caching that was implemented in #353 does not work as docker never mentions that any of the build steps is read from the cache. I think it might be that the cache directories created in Create cache directory and Clean up cache are only created inside the filesystem of the container that runs the job but not stored in a volume on the host.

Expected Behavior

The pipeline step successfully reads/writes layers from/into the docker cache.

How to reproduce the issue

No response

[ll7]

We should consult @JulianTrommer regarding the cashing.

[niklasr22]

Maybe, in addition, we could also add a paths filter to the entire workflow. We could filter for changes in the code directories (maybe others as well). This would prevent this potentially long-running step from being executed even though only documentation files have changed.

[ll7]

paf/.github/workflows/build.yml

Lines 45 to 106 in df9452d

	- name: Create cache directory
	run: |
	mkdir -p ./.buildx-cache/cache
	mkdir ./.buildx-cache/cache-new
	mkdir ./.buildx-cache/cache/test
	mkdir ./.buildx-cache/cache/latest
	mkdir ./.buildx-cache/cache-new/test
	mkdir ./.buildx-cache/cache-new/latest
	- name: Test build
	if: github.event_name == 'pull_request'
	uses: docker/build-push-action@v3
	with:
	context: .
	file: ./build/docker/agent/Dockerfile
	load: true
	tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}:test
	cache-from: type=local,src=./.buildx-cache/cache/test/
	cache-to: type=local,dest=./.buildx-cache/cache-new/test/,mode=max
	build-args: |
	USERNAME=paf
	USER_UID=1000
	USER_GID=1000
	- name: Build and push Docker image
	id: build
	if: github.event_name != 'pull_request'
	uses: docker/build-push-action@v3
	with:
	context: .
	file: ./build/docker/agent/Dockerfile
	push: true
	# tag 'latest' and version on push to main, otherwise use the commit hash
	tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
	cache-from: type=local,src=./.buildx-cache/cache/latest/
	cache-to: type=local,dest=./.buildx-cache/cache-new/latest/,mode=max
	build-args: |
	USERNAME=paf
	USER_UID=1000
	USER_GID=1000
	- name: Save PR ID
	if: github.event_name == 'pull_request'
	env:
	PR_ID: ${{ github.event.number }}
	run: |
	mkdir -p ./pr
	echo $PR_ID > ./pr/pr_id
	- name: Upload PR ID
	if: github.event_name == 'pull_request'
	uses: actions/upload-artifact@v4
	with:
	name: pr_id
	path: pr/
	retention-days: 1
	- name: Clean up cache
	run: |
	rm -rf ./.buildx-cache/cache/test
	rm -rf ./.buildx-cache/cache/latest
	mv ./.buildx-cache/cache-new ./.buildx-cache/cache

[ll7]

We are now down to 8 minutes and run three build jobs in parallel, thanks to @JulianTrommer .
How important do you think this issue is now? What would be a build time that is good enough?