TTFX time of request is high. MVP #1194
Comments
|
I really think there needs to be a way to speed this up, because literally any POST request anyone make with HTTP.jl has a 6 seconds TTFX. 🤔 |
|
Just checked, literally calling curl (Cmd) to make the request has faster TTFX: 🤔 |
That seems wholly unsurprising, or? |
|
True. Both |
|
I wonder if we could add precompile step just to precompile this IMO quite common usecase of HTTP.jl. Are there reasons we are not doing it already? Want to keep precompilation faster? Are there too much edge cases? and it would blow up compilation too much? I would prefer at least one way we could have low TTFX for a POST request with HTTP.jl. |
|
I guess part of the challenge is that the precompilation should still be effective when done offline. |
|
One thing I've done in the past is run a package in SnoopCompile's |
|
Yeah this has been a notoriously tricky problem due to offline challenges. I think the approach @JamesWrigley mentioned is the right way to go, though it's still a bit of work and needs ongoing maintenance. I'm actually a decent way through an entire rewrite of the package that uses the aws https://github.com/awslabs/aws-c-http library under the hood. I've been working through the challeneges of getting the C-interfacing right, as well as balancing what would end up being breaking vs. keeping API compatibility. Hopefully have something people can see/try soon. But I mention because the rewrite is much more precompile friendly. |
|
If you want @quinnj I can run some LLM calls just to make a list of functionality differences. :) Also maybe if you have in mind what should be done, we can actually run some agents to propose you some solutions, and then you only need to check if the changes are optimal. In the current state of LLMs these agents won't invent new things, but can connect the dots pretty well. |
|
For reference, with Sorted by duration. Note a bunch are recompiles. |
|
where is |
|
julia master |
|
There are some recompiles sure but the main stuff is just the big HTTP functions? |
|
Just checked the suggested CurlHTTP, also switched to julia 1.11 (previously was on 1.10.6): curl_post(url, body, method=CurlHTTP.POST) = begin
curl = CurlEasy(; url, method)
databuffer = UInt8[]
response, status, error = curl_execute(curl, body, ) do d
isa(d, Array{UInt8}) && append!(databuffer, d)
end
response_body = String(databuffer)
end
(method, url, serialized_body) = (:POST, "http://localhost:9000", "{\"content\":\"Hello\"}", )
@time response = curl_post(url, serialized_body)
@time response = curl_post(url, serialized_body)
@time response = curl_post(url, serialized_body)
@time response = curl_post(url, serialized_body)
0.262022 seconds (294.45 k allocations: 15.009 MiB, 99.40% compilation time)
0.000631 seconds (4.80 k allocations: 270.227 KiB)
0.000456 seconds (475 allocations: 22.867 KiB)
0.000409 seconds (80 allocations: 3.570 KiB)
@time HTTP.request(method, url; body=serialized_body)
@time HTTP.request(method, url; body=serialized_body)
7.492825 seconds (13.31 M allocations: 664.436 MiB, 1.76% gc time, 113.52% compilation time)
0.000679 seconds (209 allocations: 10.148 KiB)Warm run timings are probably shouldn't be compared, I think even localhost might have some kind of latency. I am a little bit amazed, by how fast CurlHTTP.jl TTFX is. I am guessing HTTP.jl has a lot more features, although for a simple usecase it should be more lightweight and extras should be more of an optional thing? I hope if I used CurlHTTP in its simplest form. |
Why? It just calls into a statically compiled library. Why would there be any significant latency in that? |
|
Yeah, true. A lot less challenges need to be solved for precompilation. I wonder how much it loses with needing to call a lib. 🤔 |
|
I made a proposal at #1195 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I would think this should run in 0.1seconds even on first call (not 6 seconds), because it is one of the most common way we construct a POST request.
An example server for testing:
What is going on ? Am I missing something? I see so many packages using this very same line, and literally everywhere these long compilations are happening!
The text was updated successfully, but these errors were encountered: