arm builds

[brandon-dacrib]

Would it be possible to publish arm builds for k8dash. Right now the image is x86 only and I want to use it on my raspberry pi 4 cluster. I know how to do this manually and would be willing to investigate how to automate it. https://engineering.docker.com/2019/04/multi-arch-images/

[kimdiallo]

I've had some trouble with node-sass, but for the moment it works well on my Pi 3 cluster.

https://hub.docker.com/r/kimdiallo/rpi_k8dash

But arm builds directly from here would be very welcome.

[herbrandson]

Thanks for the leg work @kimdiallo. Any chance you'd like to provide a PR?

[danielsatanik]

@kimdiallo I was just able to deploy your rpi version, unfortunately when I expose it with an Ingress it asks for Basic Auth. Did you set up some kind of standard user-name or password? Can this be disabled for login with a token?

[kimdiallo]

@danielsatanik I don't think that depends on the build. Try to follow these instructions[1] and start the pod again.

[1]https://github.com/herbrandson/k8dash#service-account-token

[kimdiallo]

@herbrandson Sorry, i have never noticed your request. So far, unfortunately, this has been very makeshift and built directly on a real Pi. I think with a dockerfile for a reasonable cross build would be the better option. If there is no great hurry, I would be happy to help.

[danielsatanik]

@danielsatanik I don't think that depends on the build. Try to follow these instructions[1] and start the pod again.

[1]https://github.com/herbrandson/k8dash#service-account-token

@kimdiallo I followed the instructions but unfortunately it still asks for authorization. Something else you can think of?

[kimdiallo]

@danielsatanik The only difference that comes to my mind is, that i run this as a NodePort, not with ingress...

[herbrandson]

@herbrandson Sorry, i have never noticed your request. So far, unfortunately, this has been very makeshift and built directly on a real Pi. I think with a dockerfile for a reasonable cross build would be the better option. If there is no great hurry, I would be happy to help.

There's no hurry, so that would be great if you can!

[danielsatanik]

@danielsatanik The only difference that comes to my mind is, that i run this as a NodePort, not with ingress...

@kimdiallo I tried nodeport as well and the same happens. I couldn't build it with the official Dockerfile myself, so I was wondering if you used a custom one, adapted by yourself for arm, could you share it with me? I don't want to imply anything, but maybe you set some ENV variable or something that enables authentication, could that have happened? are you using it with authentication or did it work on your end, not prompting for username and password?

Edit: I noticed that when I use nodeport it actually does perform different. It still asks for the basic authentication first, but when one cancels that, instead of showing an api error it just continues normally to the dashboard somehow. I'm still curious though why it would work with nodeport and not with ingress and why it is not possible to completly disable the authentication request.

[thiscantbeserious]

Just in case I'm already on it since I'm dying for a good dashboard to get started with my little multi-arch-cluster (amd64/armhf/arm64) that I've just recently setup with k3sup.

I'm using Docker 19.03 with buildx as outlined here: https://mirailabs.io/blog/multiarch-docker-with-buildx/

I'm also throwing out node-sass (js-wrapper ontop of c++ code that needs to compiled for every single target-platform / -arch) replacing it with sass (pure js-implementation of Google's Dart-Compiler which is battle-tested and stable enought).

node-sass increases the build time more then 3-fold on each subsequent build and also requires python 2.7 and gcc dependencies ontop - for little to no gain while compiling scss to css (just so much pain given its unmanagable dependencies and the many possible side-effects that can happen while building native binaries during npm-install).

I'll track my progress here (currently not up-to-date just for future reference):

https://github.com/thiscantbeserious/k8dash

and here:

https://hub.docker.com/r/youcantbeserious/multiarch-k8dash

amd64, arm64, arm (armhf, armv7 and armv6) and possibly 386/x86 are on my list if they all work properly with buildx. I can test and verify them all myself - but I'll try to integrate travis to verify it via CI if possible (sadly no 32-bit support tought).

https://docs.travis-ci.com/user/multi-cpu-architectures/

I'll send a PR once I'm done (no worries if anyone is faster then me).

[herbrandson]

Thanks @thiscantbeserious! Super excited to see how it goes :)

[thiscantbeserious]

well, just to keep you updated:

https://github.com/ayufan-rock64/linux-build/blob/master/recipes/binfmt-misc.md

Make sure you're using a kernel above 4.8 if you're going for the binfmt_misc route - otherwise you won't be able to run (a good way to verify if binfmt_misc is actually working or not):

docker run --rm -it arm64v8/busybox echo Hello World

[thiscantbeserious]

There you go @herbrandson - I successfully published an multiarch-image for armv6, armv7, arm64 and amd64.

https://hub.docker.com/r/youcantbeserious/multiarch-k8dash/tags

kubectl get all -n k8dash
NAME                          READY   STATUS    RESTARTS   AGE
pod/k8dash-5b46db4c7f-sg86m   1/1     Running   0          4h10m
pod/k8dash-5b46db4c7f-qk82r   1/1     Running   0          62s

NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/k8dash   ClusterIP   10.43.74.205   <none>        4654/TCP   4h10m

NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/k8dash   2/2     2            2           4h10m

NAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/k8dash-5b46db4c7f   2         2         2       4h10m

[thiscantbeserious]

So I'm close to finalizing my adjustments, after some trendemous cross-platform testing and walzing trough the docs.

I added a Travis config, which will build automatically for each commit and tag it with the revision and latest tag, pushes both to Docker Hub automatically and in the end also does a Vulnerability-Check with trivy.

Trivy also scans the base-container, not just npm and provides an additional layer of security - which discovered multiple high risk exploits in the alpine-image that's been previously used, so I also switched that out with node:lts-alpine instead.

Travis will require you to setup the Repository variables HUB_USER and HUB_TOKEN (the API-Token you should generate on Docker-Hub, NOT your Password).

Example Travis log:
https://travis-ci.com/thiscantbeserious/k8dash/builds/149215576

Tags on Docker Hub:
https://hub.docker.com/r/youcantbeserious/multiarch-k8dash/tags

[thiscantbeserious]

Ping @herbrandson PR is up for Review 🙉 - please also test if everything is still working in the Dashboard itself - I might have discovered an Express bug, sometimes not serving webpack-bundles requiring a refresh of the Page (or did I introduce it?) Edit: Nevermind I messed with the underlying storage of a few nodes while they were running Pods - which resulted in InvalidDiskCapacity - once I scaled down/up again everything is working flawlessly again.

[herbrandson]

Thanks for this @thiscantbeserious ! I'll work on reviewing this weekend.

[ghost]

Hello,
I enjoy using Skooner on x86_64, but I have trouble on my raspberry pi 4.
I get this :
standard_init_linux.go:228: exec user process caused: exec format error

I think the image is not OK for ARM Rpi.
Are there some updates to be able to run skooner on ARM archs ? (or the image tags to use ?

I think the pull request I did #215 should still fix that.

Regards,

[moll-re]

Am I missing something in order to run on arm64? When pulling this on a raspberry pi 4 (dpkg reports aarch64) I still encouner an exec format error. This happens when applying the deployment described in https://raw.githubusercontent.com/skooner-k8s/skooner/master/kubernetes-skooner.yaml
Do I need to specify a special tag for the image?

[sprior]

@moll-re Did you figure out how to do this? I'm just trying a Raspberry Pi install of Skooner and having the same issue.

[sprior]

I found this commit which removed arm64 and arm7 image builds but provides no explanation as to why this was done: 76f6a1c