-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
improve fake network write() efficiency #1178
base: master
Are you sure you want to change the base?
Commits on Nov 4, 2024
-
- new: class EthernetPacketEncoder, replaces all dynamically allocated write-buffers with a single, static buffer - new: class Uint8Stream, designed to replace TCPConnection.send_buffer[] with a dynamically growing ringbuffer - new: added internet checksum to synthesized UDP packets (required by mTCP) in function write_udp() - replaced internet checksum calculation with more efficient algorithm from RFC1071 (chapter 4.1) - increased TCP chunk size in TCPConnection.pump() from 500 to 1460 bytes (max. TCP payload size) - added function to efficiently copy an array into a DataView using TypedArray.set(), replaces a bunch of for-loops - added function to efficiently encode a string into a DataView using TextEncoder.encodeInto() - refactored all write_...()-functions to use EthernetPacketEncoder - added several named constants for ethernet/ipv4/udp/tcp-related offsets - removed several apparently unused "return true" statements
Configuration menu - View commit details
-
Copy full SHA for 5af221d - Browse repository at this point
Copy the full SHA 5af221dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3c28d30 - Browse repository at this point
Copy the full SHA 3c28d30View commit details -
fixed FIN flag in generated TCP packets
TCP FIN flag marks the sender's last package. This patch delays sending FIN in TCPConnection.close() until the send buffer is drained.
Configuration menu - View commit details
-
Copy full SHA for c46e34b - Browse repository at this point
Copy the full SHA c46e34bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ed7bf7a - Browse repository at this point
Copy the full SHA ed7bf7aView commit details
Commits on Nov 5, 2024
-
several improvements to GrowableRingbuffer (former class Uint8Stream)
- Renamed Uint8Stream to GrowableRingbuffer - Minimum capacity now guaranteed to be 16, and maximum (if given) to be greater or equal to minumum - Changed peek(dst_array, length) to peek(dst_array) - Removed misleading method resize(), moved code into write() - Now throws an Exception on capacity overflow error instead of console.error() - Removed debug code that is no longer needed When we have a capacity overflow the situation is as bad as if the Uint8Array constructor had failed with out-of-memory, which throws a RangeError.
Configuration menu - View commit details
-
Copy full SHA for dd3e314 - Browse repository at this point
Copy the full SHA dd3e314View commit details -
fixed finding unused dynamic TCP port
- use actual dynamic TCP port range from RFC6335 - throw an Exception when pool of dynamic ports is exhausted
Configuration menu - View commit details
-
Copy full SHA for c1c52b8 - Browse repository at this point
Copy the full SHA c1c52b8View commit details -
removed class EthernetPacketEncoder
- removed class EthernetPacketEncoder and its global single instance ethernet_encoder - moved encoder buffer ownership to adapter (WispNetworkAdapter and FetchNetworkAdapter) - made view_{setArray,setString,setInetChecksum} global functions - changed prototype of all write_...(spec, ...) functions to uniform write_...(spec, out) - removed function make_packet(spec), added new function adapter_receive(adapter, spec) - replaced all calls of form "A.receive(make_packet(S))" with "adapter_receive(A, S)" (also in WispNetworkAdapter) Note that before this patch function make_packet() was always called in the form: adapter.receive(make_packet(spec)); This patch binds the lifetime and scope of the encoder buffer from class EthernetPacketEncoder to the WispNetworkAdapter/FetchNetworkAdapter instance "adapter".
Configuration menu - View commit details
-
Copy full SHA for bf877f8 - Browse repository at this point
Copy the full SHA bf877f8View commit details -
- removed unnecessary comments around TCP_STATE_... - renamed TCPConnection.send_stream to send_buffer - calculate package length in write_udp() and write_tcp() along the same structure as in all other write_...() functions
Configuration menu - View commit details
-
Copy full SHA for d8136c1 - Browse repository at this point
Copy the full SHA d8136c1View commit details
Commits on Nov 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bd91c53 - Browse repository at this point
Copy the full SHA bd91c53View commit details -
Configuration menu - View commit details
-
Copy full SHA for f8af3c6 - Browse repository at this point
Copy the full SHA f8af3c6View commit details
Commits on Nov 10, 2024
-
improve TCP state mechanics support
For the most part, this patch adds graceful TCP connection shutdown behaviour to class TCPConnection in fake_network.js. Network-intense guest applications like "apt" run noticeably smoother with support for graceful TCP shutdown. Changes in detail: - completed SYN-related state mechanics (TCP connection establishment) - added FIN-related state mechanics (TCP connection shutdown) - reordered and regrouped TCP packet handling steps in TCPConnection.process() - reduced minimum IPv4 packet size from 46 to 40 bytes to reduce noise in logs (why is this even checked) - added explicit detection of TCP Keep-Alive packets to reduce noise in logs - added dbg_log() calls to trace TCP connection state (could be removed if unwanted, or maybe use smth. like LOG_TCP instead of LOG_FETCH?) - removed TCPConnection.seq_history as it was not used anywhere As should be expected, the changes related to TCP connection state only affect class TCPConnection internally and are mostly concentrated in its methods process() and pump(). A few TCP connection states are left unimplemented, reasons: - CLOSING: simultaneous close from both ends (deemed too unlikely) - TIME_WAIT: wait seconds to minutes after sending final ACK before entering CLOSED state (pointless in our specific case and would require a Timer) - LISTEN: are server sockets even supported? Both WISP and FETCH do not support them. Also, TCPConnection.connect() seems not to be called from anywhere (it used to be called from fake_tcp_connect() which was deleted in the previous patcH). As before, fake_network.js does not use any Timers (i.e. setTimeout()), the code is purely driven by outside callers. That means that there are no checks for lost packets (or resends), the internal network is assumed to be ideal (as was the case before). Even though I fixed and removed most of the TODOs I originally found in fake_network.js, I've added several new ones that are left to be discussed. Tested with V86 network providers WispNetworkAdapter and FetchNetworkAdapter, and with V86 network devices ne2k (FreeDOS) and virtio (Debian 12 32-bit).
Configuration menu - View commit details
-
Copy full SHA for 7ddf24b - Browse repository at this point
Copy the full SHA 7ddf24bView commit details
Commits on Nov 11, 2024
-
suggested improvements and a bugfix
- removed orphaned function siptolong() - refactored view_setInetChecksum() to calc_inet_checksum() - renamed view_setString() into view_set_string() and simplified it - renamed view_setArray() into view_set_array() - refactored adapter_receive() to make_packet() - bugfix: send FIN after receiving last ACK instead of tagging it to the last packet we send, fixes a corner case when TCPConnection.close() is called with empty send_buffer but with a yet to be acknowledged package in transit. Send a FIN packet in TCPConnection.close() only if both the send_buffer is empty and there is no package in transit, else delay that until we receive the last ACK.
Configuration menu - View commit details
-
Copy full SHA for 72e8aec - Browse repository at this point
Copy the full SHA 72e8aecView commit details -
Configuration menu - View commit details
-
Copy full SHA for 51204e2 - Browse repository at this point
Copy the full SHA 51204e2View commit details
Commits on Nov 17, 2024
-
added more TCP state mechanics
See https://en.wikipedia.org/wiki/File:Tcp_state_diagram_fixed_new.svg for a full TCP state diagram, all states (except TIME_WAIT) and state transitions below ESTABLISHED are now properly implemented in class TCPConnection. - fixed misconceptions regarding TCP close states and transitions - fixed incorrect behaviour when guest closes a TCP connection - added state CLOSING to state machine - added support to pass passive close up to WispNetworkAdapter - commented out all calls to dbg_log() that are part of this PR This improves interoperability by further completing the TCP state machine model in fake_network.js.
Configuration menu - View commit details
-
Copy full SHA for 4004a4a - Browse repository at this point
Copy the full SHA 4004a4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7cbd89c - Browse repository at this point
Copy the full SHA 7cbd89cView commit details
Commits on Nov 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c5557a6 - Browse repository at this point
Copy the full SHA c5557a6View commit details -
added method TCPConnection.writev([ Uint8Array, ... ])
Optimizes writing multiple buffers to the TCPConnection at once.
Configuration menu - View commit details
-
Copy full SHA for 00ad28d - Browse repository at this point
Copy the full SHA 00ad28dView commit details -
use ReadableStream in fetch() response body handling
- replaced blocking call to fetch() in on_data_http() with async loop, response body chunks are now forwarded as soon as they arrive - commented out old code path in on_data_http() (for fallback) - commented out FetchNetworkAdapter.fetch() (for fallback), merged into on_data_http() - removed code to make type checking happy for data argument in on_data_http(), fixed JSC declaration instead
Configuration menu - View commit details
-
Copy full SHA for b3721ef - Browse repository at this point
Copy the full SHA b3721efView commit details
Commits on Nov 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 65ceac6 - Browse repository at this point
Copy the full SHA 65ceac6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ad738a - Browse repository at this point
Copy the full SHA 6ad738aView commit details
Commits on Nov 21, 2024
-
reactivated FetchNetworkAdapter.fetch()
Even though this method is currently only used ín a test, keep this function for possible future use.
Configuration menu - View commit details
-
Copy full SHA for 68b9b2c - Browse repository at this point
Copy the full SHA 68b9b2cView commit details -
removed method TCPConnection.on_passive_close()
Even though this method is conceptually correct, it is useless because neither WISP nor fetch() support closing only our half of the TCP connection.
Configuration menu - View commit details
-
Copy full SHA for b06c291 - Browse repository at this point
Copy the full SHA b06c291View commit details
Commits on Nov 22, 2024
-
added TCPConnection.on_shutdown() and TCPConnection.on_close()
- both methods must be overloaded by the network adapter, just like TCPConnection.on_data() - on_shutdown() informs the network adapter that the client has closed its half of the connection (FIN) - on_close() informs the network adapter that the client has fully closed the connection (RST) - added call to on_shutdown() when we receive a TCP packet with active FIN flag from guest - added call to on_close() when we receive a TCP packet with active RST flag from guest - added calls to on_close() when we have to tear down the connection in fake_network.js - added implementation of on_shutdown() and on_close() in wisp_network.js The default implementation of these methods is to do nothing. These methods do not apply to fetch-based networking, fetch() only supports HTTP and it doesn't matter if the client closes its end of the connection after it has sent its HTTP request. Hence FetchNetworkAdapter does not override these methods. Note that WISP currently only supports close(), as a workaround shutdown() is implemented like close() which might be incorrect (missing support for half-closed TCP connections).
Configuration menu - View commit details
-
Copy full SHA for 5ec28a2 - Browse repository at this point
Copy the full SHA 5ec28a2View commit details