Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

network: Fetch based browser networking #1061

Merged
merged 11 commits into from
Jul 24, 2024
Merged

network: Fetch based browser networking #1061

merged 11 commits into from
Jul 24, 2024

Conversation

basicer
Copy link
Contributor

@basicer basicer commented Jun 20, 2024

Emulate a networking stack, intercept HTTP requests, serve them with fetch(). Enable by setting networking_proxy=fetch

Somewhat fixes #198

Just enough networking for http to work for package managers and net-boot. The networking stack just reflects the mac address of packets it receives, so shouldn't be bothered by mac address changes. Services include:

  • ARP replies by establishing the gateways IP
  • DHCP server sending gateway, dns, and client-ip
    • Gateway: 192.168.86.1
    • Client: 192.168.86.100
    • Netmask: 255.255.255.0
  • UDP DNS server, all addresses resolve to: 192.168.87.1
  • NTP Server, giving out the current time from Date()
  • Replies to ICMP pings to any IP address
  • UDP echo server on port 8

Limitations:

  • Request are limited by CORS rules, but CORS-proxies can work around this.
  • Redirects are handled transparently by fetch. The upside is sites that force an http->https upgrade appear to the VM to work over http.

Future Work:

  • Allow HTTP requests from browser to be served by VM.
  • HTTPS support (maybe with help of a rust based TLS stack in WASM)
  • Support for POSTing binary data

@SuperMaxusa
Copy link
Contributor

  • UDP DNS server, all addresses resolve to: 192.168.87.1

How about use a DNS over HTTPS (https://datatracker.ietf.org/doc/html/rfc8484) or this not suitable for this case?

@basicer
Copy link
Contributor Author

basicer commented Jun 22, 2024 via email

Copy link
Owner

@copy copy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this very cool.

From my first look:

  • Some parts are rough performance-wise. This doesn't block a merge, but some TODO(perf) comments would be nice
  • I think this should be exposed as an experimental option in index.html
  • This needs a test (nodejs has fetch now). See tests/devices/virtio_console.js
    for an example to script a Linux guest through the serial console. Just a simple udhcpcd and curl should be sufficient for now.
  • Some documentation should be added to docs/networking.md

Allow HTTP requests from browser to be served by VM.

That would be really cool.

HTTPS support (maybe with help of a rust based TLS stack in WASM)

Could be interesting, but it could also increase the size of v86.wasm significantly. I'd suggest sticking to http for this PR.

After a rebase on master the tests should be passing.

tools/fetch_network_harness.js Outdated Show resolved Hide resolved
src/browser/fetch_network.js Outdated Show resolved Hide resolved
src/browser/fetch_network.js Outdated Show resolved Hide resolved
@basicer basicer force-pushed the fetch-net branch 8 times, most recently from 35ea6ca to 95c0684 Compare July 7, 2024 08:56
Copy link
Owner

@copy copy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks mostly good now. I've left a few comments on the test.

Before merging, could you rebase on master, run eslint src tests gen lib examples tools --fix and fix the remaining issues manually?

tests/devices/fetch_network.js Outdated Show resolved Hide resolved
tests/devices/fetch_network.js Outdated Show resolved Hide resolved
tests/devices/fetch_network.js Outdated Show resolved Hide resolved
tests/devices/fetch_network.js Outdated Show resolved Hide resolved
tests/devices/fetch_network.js Show resolved Hide resolved
tools/fetch_network_harness.js Outdated Show resolved Hide resolved
Emulate a networking stack, intercept HTTP requests, serve
them with fetch().  Enable by setting networking_proxy=fetch

Somewhat fixes copy#198

Just enough networking for http to work for package managers and
net-boot. The networking stack just reflects the macaddress of
packets it receives, so shouldnt be bothered by mac address
changes. Services include:

- ARP replies by establishing the gateways IP
- DHCP server sending gateway, dns, and client-ip
  - Gateway: 192.168.86.1
  - Client: 192.168.86.100
  - Netmask: 255.255.255.0
- UDP DNS server, all addresses resolve to: 192.168.87.1
- NTP Server, giving out the current time from Date()
- Replies to ICMP pings to any IP address
- UDP echo server on port 8

Limitations:
- Request are limited by CORS rules, but CORS-proxies can work around
  this.
- Redirects are handled transparently by fetch. The upside is sites that
  force an http->https upgrade appear to the VM to work over http.

Future Work:
- Allow HTTP requests from browser to be served by VM.
- HTTPS support (maybe with help of a rust based TLS stack in WASM)
- Support for POSTing binary data
- Add code to establishing TCP connection to VM
- Cleanup
@basicer basicer force-pushed the fetch-net branch 2 times, most recently from 8bd658c to 8df028f Compare July 16, 2024 09:12
@copy copy merged commit 92c936d into copy:master Jul 24, 2024
2 checks passed
@copy
Copy link
Owner

copy commented Jul 24, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use networking without the websocket proxy?
3 participants