structured LLM output in labeling

[frrad]

Split off from #18

It would be nicer to force the model output to adhere to a schema.

• OpenAI has structured outputs https://openai.com/index/introducing-structured-outputs-in-the-api/
• It seems like ollama has "JSON Mode" https://github.com/ollama/ollama/blob/main/docs/api.md#json-mode which can guarantee that the output is some valid JSON. It seems like it doesn't guarantee adherence to a particular schema though.
• This issue Ollama Product Stance on Grammar Feature / Outstanding PRs ollama/ollama#6237 makes it seem like particular-schema support is coming to ollama potentially soon. Apparently they already support general grammars internally and it's just a matter of them doing the plumbing to expose more fine-grained control.

[xerootg]

You mean adding system messages like "you're the best in the world at json, to the point that all you are capable of replying to prompts with is well formed json" might be unneeded soon? Amazing!