Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threads endpoint remapping #32

Merged
merged 32 commits into from
Feb 10, 2025
Merged

Conversation

BoHomola
Copy link
Contributor

@BoHomola BoHomola commented Feb 5, 2025

Implemented and mapped most of the endpoints for Threads (thread, messages, run, run steps)

https://platform.openai.com/docs/api-reference/threads
https://platform.openai.com/docs/api-reference/messages
https://platform.openai.com/docs/api-reference/runs
https://platform.openai.com/docs/api-reference/run-steps

Still missing: Submit tool outputs to run, more complex demo cases (currently only file info extraction is showcased).

LlmTornado/Common/ImageFile.cs Outdated Show resolved Hide resolved
LlmTornado/Common/ImageUrl.cs Outdated Show resolved Hide resolved
/// <summary>
/// The content of the message.
/// </summary>
[JsonProperty("content")]
public string Content { get; }
// public IReadOnlyList<MessageContent> Content { get; }
public required string Content { get; set; } //TODO: open ai supports also array of MessageContent object, but the text object differs from create request and from response object
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be resolved in a follow-up pr? See ChatRequest for a reference - we give users both fields to use, if the implementing IEnumerable<> one is filled it has precedence when serializing the request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a request and won't be de-serialized, I left only the array (since it support text content as well) and if a message with a single string content needs to be created, for the simplicity I overloaded a constructor, that automatically constructs the array with text content. What do you think?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a few ctor overloads in 3c79c73, which should solve this

LlmTornado/Threads/MessageAnnotation.cs Outdated Show resolved Hide resolved
LlmTornado/Threads/MessageContent.cs Outdated Show resolved Hide resolved
/// <summary>
/// For now, this is always going to be an empty object.
/// For now, this is always going to be an empty object. TODO: When OpenAI finished implementation, map it here
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memento to check this out

@lofcz
Copy link
Owner

lofcz commented Feb 6, 2025

Tried running all the threads tests:

  • DeleteMessage, DeleteRun failed with 404 / 405, though they work fine when run standalone
  • ListRunSteps, RetrieveRunStep failed on JSON deserialization (again, they work fine when run standalone)
Test failed with exception: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'LlmTornado.Threads.ToolCalls' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.

Some nits:

  • CreateThreadAndRunRequest should accept ChatModel (we might specialize into AssistantModel later, possibly generating the code), there are also some nullability issues in the file.
  • SubmitToolOutputsRequest has some nullability issues.
  • In ThreadsEndpoint RestDataOrException<HttpResponseMessage> was sometimes not propagated correctly, fixed in f202c34
  • FileSearchTool seems finished by OpenAI? So instead of object, ToolFileSearchConfig should be used? (in public sealed class FileSearchToolCall : ToolCall).
  • In ToolFileSearchConfig shouldn't it be max_num_results instead of max_num_result? (https://platform.openai.com/docs/api-reference/runs/createRun)
  • RankingOptions.Ranker should be en enum with auto, default_2024_08_21 (https://platform.openai.com/docs/assistants/tools/file-search#customizing-file-search-settings).
  • There are still some System.Text.Json.Serialization usings, we should get rid of them.
  • RequiredAction.Type should be an enum.

Overall, this looks really good and consistent! Looking forward to more tests, streaming, and wrapping this up. Let me know when this is ready.

@BoHomola BoHomola force-pushed the threads_endpoint_remapping branch from 18aef4e to 4b7bd81 Compare February 6, 2025 12:21
@lofcz
Copy link
Owner

lofcz commented Feb 8, 2025

Pushed a few changes, and optimized Assistants streaming. To wrap this up:

  • Please add missing XML docs, currently many public APIs introduced are undocumented.
  • When possible, prefer FrozenDictionary lookup over patterns like ToolChoiceType? toolType = JsonConvert.DeserializeObject<ToolChoiceType>($"\"{reader.Value}\""); (see changes to ThreadsEndpoint in 27a4d77 for a reference) - this reduces allocations & increases throughput significantly, given we incur the penalty on each event received.
  • Add tests invoking code_interpreter, file_search to increase coverage.

@BoHomola BoHomola marked this pull request as ready for review February 10, 2025 10:54
@lofcz
Copy link
Owner

lofcz commented Feb 10, 2025

Thanks for working on this! :shipit:

@lofcz lofcz merged commit d9d2e4a into lofcz:master Feb 10, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants