pkg: Create generic AttributesStore

[metalmatze]

Rather than creating many implementations of the AttributesStore per type, let's use the new Go generic types.

Starting to use this code as a library I was wondering if this was possible. Turns out it is.

[lquerel]

@metalmatze I had thought about that at one point but I had a bad experience with Go generics in terms of performance. It might have changed (or my memory is bad), but if I remember correctly, Go generics aren't completely zero-cost. Since these attribute stores are on the hot path, I think it would be good to verify the performance impact with the command trace_producer_simu in the tools directory. If the difference is negligible between the current implementation and the generic based implementation, then this simplification would be more than welcome.

[metalmatze]

I'm having trouble understanding how the trace_producer_simu works.
Using the tools/trace_gen tool:

go run tools/trace_gen/main.go -output generated.json -format json 

Now, there's a generated.json file that can be passed to the tools/trace_producer_simu.

go run ./tools/trace_producer_simu  -batch-size=10,100,500,1000,2500,5000,1000 --max-batches-per-stream=10,100,500,1000,2500,5000,10000 generated.json 

If I read the source code correctly, the first tool generates a file with batchSize 20. We may need many of these files.

Here's the data of my run: https://gist.github.com/metalmatze/70ce17f8818c2fb8579d1c424c3bd6f4

Is trace_producer_simu even relevant? That tool seems to measure compression but not the performance of the code itself.

[lquerel]

@metalmatze Sorry, reviewing a PR on Thanksgiving evening before going to bed isn't the best idea in the world... On the substance, I still think we need to verify the performance impact, but it wasn't the right tool :-)
Still in the tools directory, there's a metrics_benchmark tool that allows comparing compression rates between OTLP and OTEL Arrow. In your context, the important thing isn't the compression rate (which should remain unchanged) but the timings for each phase in the OTEL Arrow column. There's a data file in the data directory data/multivariate-metrics.pb that you can pass to this tool.

[metalmatze]

Happy Thanksgiving! 🦃

Here are the results: https://gist.github.com/metalmatze/f6b635f261c4e1a017f143c4e9232d55
That was way easier and makes a lot more sense to me.

As you said, there are some performance hits, but you need to decide how impactful they are. I'm totally fine closing this PR if you think there's too much overhead.

[lquerel]

The performance difference seems small enough to justify the gain in maintainability. I think this improvement can be included in the next release.
Adding @jmacd for visibility.

[jmacd]

Thanks, @metalmatze.