If I wish to avoid committing generated BPF object files, am I out of luck?

[mejedi]

Having to commit generated binary files to a repository is a bit troubling.

I realise that go:generate is an integral part of Golang. But still, binary files merge pretty badly. Any 2 simultaneous changes to C code will result in a merge conflict.

I wonder if the alternative model with bpf object files being generated on the fly is supported. Can I find an example somewhere?

[lmb]

I remember discussing this at some point, but I can't seem to find where. Essentially, bpf2go is written with the assumption that you will check in the .o and the .go. If you don't want to do that I'd say you shouldn't commit either file: the .o depends on which compiler you are using, what the build environment looks like, etc. This in turn means the .go also depends on the build environment. Checking in the .go but skipping the .o sounds like it's inviting trouble.

[ti-mo]

It's even a little more complicated. Technically, both the .go and the .o files are the output of compiling a .c file. I'd say if you check in the .go files, then you should also check in the .o file, since any changes in the .c can cause changes in both .go and .o.

Alternatively, just run bpf2go in CI or in your project's makefile or build automation, triggered by any changes in the corresponding .c. The case you mention, merging two .c files, will always require a recompilation.

[mejedi]

I wonder if the alternative model with bpf object files being generated on the fly is supported. Can I find an example somewhere?

I have figured the bits and pieces. Please let me know if the setup below can be improved.

.gitignore
Makefile
cmd/test/bpf/xdp.c
cmd/test/main.go
go.mod
go.sum

This is a simple eBPF-enabled tool to play with the setup (./cmd/test). I don't want to ever commit generated files, therefore there are a bunch of lines in .gitignore:

*_bpfe[lb].o
*_bpfe[lb].go
*_bpfe[lb].go.d

As go build won't invoke bpf2go on its own, I have a wrapper Makefile.

build: cmd/test/dp_bpfel.go
	go build -tags bpf ./cmd/test

cmd/test/dp_bpfel.go:
	BPF2GO_MAKEBASE=${CURDIR} go generate -tags bpf ./cmd/test

-include cmd/test/*.go.d

If someone attempts to build or run ./cmd/test the "normal Go way", I'd like for the things to behave predictably. After all, files might not have been generated yet, leading to confusing errors. I use bpf build tag to exclude bpf-dependant parts from the build by default.

Finally, here's my main.go:

//go:build bpf
package main

import (
	_ "github.com/cilium/ebpf"
)

//go:generate go run github.com/cilium/ebpf/cmd/bpf2go -makebase $BPF2GO_MAKEBASE -tags bpf Dp bpf/xdp.c

func main() {
}

The tool should only be built via Makefile, therefore //go:build bpf line. I use bpf2go's -tags feature to have it in generated files as well:

$ go build ./cmd/test
package example.com/m/cmd/test: build constraints exclude all Go files in /home/nickz/try-cilium-ebpf/cmd/test

Finally, I use -makebase feature to automatically discover dependencies in the C land and have make rebuild stuff as needed. The generated dependency list is included in the Makefile (-include cmd/test/*.go.d). Please note the use of $BPF2GO_MAKEBASE variable (the directory where Makefile lives). I could have used ../.. instead, but then moving packages around would become tricky.

Could we maybe make bpf2go recognize BPF2GO_MAKEBASE out of the box as an alternative to -makebase flag?

[ti-mo]

If someone attempts to build or run ./cmd/test the "normal Go way", I'd like for the things to behave predictably.

What would you consider predictable behaviour when trying to go run an incomplete Go program? build constraints exclude all Go files reads a little cryptic to me as well.

Re BPF2GO_MAKEBASE: certainly, feel free to send a patch! I'm surprised we're not doing this already. @lmb

[mejedi]

@ti-mo here's a PR #1047

Maybe predictable is not the best choice of a word, I'd prefer ergonomic.

Ideally, there should be a message telling that one is not supposed to use go build but rather the other way. I doubt if it is possible. Spewing out compile errors is the absolute worst. "Build constraints exclude all Go files" is indeed cryptic, but at least consistent (and googleable).