How to use curl for remote write?

[huy7070]

Hello. Is there an example of how to use the /api/v1/push endpoint to send metrics data via curl? I am able to authenticate with the endpoint, but have not been able to get the correct body payload. Thanks.

[pstibrany]

To make remote write work from curl, you need to:

1. prepare remote-write request. This is snappy-compressed WriteRequest protobuf message from this proto file: https://github.com/prometheus/prometheus/blob/main/prompb/remote.proto
2. send it to /api/v1/push with "Content-Type: application/x-protobuf" header.

So this works:

curl -v -X POST -H 'X-Scope-OrgID: anonymous' -H "Content-Type: application/x-protobuf" http://localhost:9009/api/v1/push --data-binary @/tmp/req

> POST /api/v1/push HTTP/1.1
> Host: localhost:9009
> User-Agent: curl/7.79.1
> Accept: */*
> X-Scope-OrgID: anonymous
> Content-Type: application/x-protobuf
> Content-Length: 1
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.6
< Date: Tue, 07 Jun 2022 08:25:49 GMT
< Content-Length: 0
< Connection: keep-alive
<

Where /tmp/req is single-byte file with \000 byte. This is empty WriteRequest, so it's not very useful, but for illustration.

[huy7070]

@pstibrany Is there a simple way to convert to a "snappy-compressed WriteRequest protobuf message"?

[pstibrany]

Short of writing some code that does it for you, I don't think there is an easy way. It may be easier to write data in different format (eg. recently merged OTLP support).

[huy7070]

@pstibrany I see. Is there documentation or examples on using OTLP? That's a new endpoint?

[pracucci]

We're still writing the OTLP doc, but you can get a preview here:
#2463

[SquireUK]

I appreciate this is an old thread, but I've come across this as I'm wanting to create custom SLI metrics using python, and write them into Grafana Cloud - The above curl works with the zero byte send, but I'm hitting Wiretype issues when trying to send snappy compressed payloads.

Did anyone get this working with a native python approach?