How to get streaming output?

[failable]

The equivalent option for -N in curl? Thanks.

[fabricereix]

Hi @failable,
Hurl does not currently support streaming.
We haven't really looked at it so far. It might not fit well with our request/response model.
What is your use case? For testing or fetching data? Have you examples with other tools?

[failable]

Hello I just want to use hurl as a drop in replacement of curl. I use curl as a basic command line tools for testing streaming LLM requests.

Examples: https://docs.together.ai/docs/inference-rest#streaming-tokens

Parse the SSE data and perform validation on it to check if the request working correctly.I currently use Python for this.

[fabricereix]

Thanks for your example.
We can see if we can make it work with Hurl.

POST "https://api.together.xyz/v1/chat/completions
Authorization: Bearer {{TOGETHER_API_KEY}}
{
  "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
  "messages": [{"role": "user", "content": "Q: Tell me fun things to do in NYC"}],
  "temperature": 0.8, "max_tokens": 25, "stream_tokens": true
}

If Hurl is hanging on this request, it won't be able to execute any successive request in the Hurl file.
We should maybe add an additional explicit option to allow that.

[failable]

Sorry for the super late reply. Just try and I get this output

user@macos:~ $ hurl test.hurl
data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" A","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":330,"role":"assistant","content":" A","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":":","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":28747,"role":"assistant","content":":","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" Sure","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":12875,"role":"assistant","content":" Sure","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":",","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":28725,"role":"assistant","content":",","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" there","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":736,"role":"assistant","content":" there","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" are","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":460,"role":"assistant","content":" are","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" countless","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":23387,"role":"assistant","content":" countless","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" fun","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":746,"role":"assistant","content":" fun","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" things","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":1722,"role":"assistant","content":" things","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" to","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":298,"role":"assistant","content":" to","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" do","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":511,"role":"assistant","content":" do","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" in","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":297,"role":"assistant","content":" in","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" New","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":1450,"role":"assistant","content":" New","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" York","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":2726,"role":"assistant","content":" York","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" City","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":3805,"role":"assistant","content":" City","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":"!","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":28808,"role":"assistant","content":"!","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" Here","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":4003,"role":"assistant","content":" Here","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" are","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":460,"role":"assistant","content":" are","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" just","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":776,"role":"assistant","content":" just","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" a","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":264,"role":"assistant","content":" a","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" few","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":1664,"role":"assistant","content":" few","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":" ideas","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":5766,"role":"assistant","content":" ideas","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":":","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":28747,"role":"assistant","content":":","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":"\n","logprobs":null,"finish_reason":null,"seed":null,"delta":{"token_id":13,"role":"assistant","content":"\n","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":null}

data: {"id":"8a4abacacd564aa6-TPE","object":"chat.completion.chunk","created":1721224527,"choices":[{"index":0,"text":"\n","logprobs":null,"finish_reason":"length","seed":11441488877205000000,"delta":{"token_id":13,"role":"assistant","content":"\n","tool_calls":null}}],"model":"mistralai/Mixtral-8x7B-Instruct-v0.1","usage":{"prompt_tokens":20,"completion_tokens":25,"total_tokens":45}}

data: [DONE]