Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyodide/Wasm Support #1273

Closed
dataphysician opened this issue Feb 1, 2025 · 2 comments · Fixed by #1277
Closed

Pyodide/Wasm Support #1273

dataphysician opened this issue Feb 1, 2025 · 2 comments · Fixed by #1277
Labels
triage label for issues that need to be triaged.

Comments

@dataphysician
Copy link

Issue
Hamilton is well-suited for in-browser execution via Pyodide/WebAssembly (Wasm), but its reliance on multiprocessing is a blocker since WebAssembly currently does not support traditional multiprocessing.

Request

  • Identify areas where multiprocessing is used and explore alternatives within Pyodide’s single-threaded constraints.
  • Assess any other Pyodide-specific adjustments needed for smoother operations.

Expected Outcome
Hamilton running in a Pyodide/Wasm environment, potentially with an alternative parallel execution model. If feasible, outline steps toward a production-ready WASM build.

(Note: Posted this in Burr thinking it was Hamilton. 🤷‍♂️)

@skrawcz skrawcz added the triage label for issues that need to be triaged. label Feb 3, 2025
@skrawcz
Copy link
Collaborator

skrawcz commented Feb 3, 2025

@dataphysician thanks!

So currently https://github.com/DAGWorks-Inc/hamilton/tree/try-hamilton has some changes that make hamilton installable.

Ideas:

  1. Figure out if we can determine whether we're in a WASM environment and not import multiprocessing and a few other things.
  2. Create new package sf-hamilton-wasm that is a parallel branch we maintain and update specifically for wasm use.

My preference is for (1) but I haven't explored it at all.

@skrawcz skrawcz linked a pull request Feb 11, 2025 that will close this issue
7 tasks
@skrawcz
Copy link
Collaborator

skrawcz commented Feb 11, 2025

Change should go out this week to enable default package to work under wasm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage label for issues that need to be triaged.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants