By CVS, I mean "Container Virtual Servers", is a POC(Prove Of Concept) project implementing kernal space load balancer based on netfilter framework and ebpf MAPs, solely used for container environment.
- Random access data structure with O(1) time complexity selecting real ip, compared with O(log(n)) for iptables
- No need to add vip to a dummy interface to make load balancer work in a container environment, as k8s/docker swarm does to make use of ipvs.
- ebpf MAPs are used as in-kernel data structure storing vip-rip bindings.
- netfilter framework nat PREROUTING/OUTPUT hook to make DNAT
- build-essential to build
- kernel 5+ (My test run only on latest kernel 5.19)
gcc ./cmd/main.c -o cvsadm
cd module && make
cd module && insmod cvs.ko
rmmod cvs
# add new vip-rip bindings
cvsadm push 10.10.0.1 192.168.0.2
cvsadm push 10.10.0.1 192.168.0.3
cvsadm push 10.10.0.1 192.168.0.4
# show vip-rip bindings for specific vip
cvsadm list 10.10.0.1
# remove the last rip for specific vip
csvadm pop 10.10.0.1
# Then 192.168.0.2/3/4 would be access via 10.10.0.1 on host or inside another container
The kernel module, when receive a package with destination ip address set to 10.10.0.1(vip), it would randomly select a rip, and change the destination ip to the selected rip.
- To access ebpf MAP inside another kernel module, current implementation actually uses a
hackcopied from ebpf helpers, it should be changed to a public interface supplied by ebpf when available. - Lots of other stuff...