Implement Thread Pooling #31
Assignees
Labels
enhancement
New feature or request
Comments
chrisdickinson
added a commit
that referenced
this issue
Nov 27, 2023
`createPlugin` now accepts two additional manifest types (`response` and `module`) as well
as `Response` and `WebAssembly.Module`. There are four goals here:
1. Allow us to target the Cloudflare Workers platform. CF Workers only support
loading Wasm via `import` statements; these resolve to `WebAssembly.Module`
objects, which means we need to allow users to pass `Module`s in addition
to our other types.
2. Play nicely with V8's [Wasm caching][1]; in particular V8 will use metadata
from the `Response` to build a key for caching the results of Wasm compilation.
3. This sets us up to implement [Wasm linking][2] by allowing us to introspect
plugin modules imports and exports before instantiation.
4. And finally, resolving to modules instead of arraybuffers allows us to add
[hooks for observe-sdk][3] (especially in advance of adding [thread pooling][4]).
Because Bun lacks support for `WebAssembly.compileStreaming` and
`Response.clone()`, we provide an alternate implementation for converting a
response to a module and its metadata.
One caveat is that there's no way to get the source bytes of a
`WebAssembly.Module`, so `{module}` cannot be used with `{hash}` in a
`Manifest`.
Fixes #9
[1]: https://v8.dev/blog/wasm-code-caching#stream
[2]: #29
[3]: #3
[4]: #31
chrisdickinson
added a commit
that referenced
this issue
Nov 27, 2023
`createPlugin` now accepts two additional manifest types (`response` and `module`) as well
as `Response` and `WebAssembly.Module`. There are four goals here:
1. Allow us to target the Cloudflare Workers platform. CF Workers only support
loading Wasm via `import` statements; these resolve to `WebAssembly.Module`
objects, which means we need to allow users to pass `Module`s in addition
to our other types.
2. Play nicely with V8's [Wasm caching][1]; in particular V8 will use metadata
from the `Response` to build a key for caching the results of Wasm compilation.
3. This sets us up to implement [Wasm linking][2] by allowing us to introspect
plugin modules imports and exports before instantiation.
4. And finally, resolving to modules instead of arraybuffers allows us to add
[hooks for observe-sdk][3] (especially in advance of adding [thread pooling][4]).
Because Bun lacks support for `WebAssembly.compileStreaming` and
`Response.clone()`, we provide an alternate implementation for converting a
response to a module and its metadata.
One caveat is that there's no way to get the source bytes of a
`WebAssembly.Module`, so `{module}` cannot be used with `{hash}` in a
`Manifest`.
Fixes #9
[1]: https://v8.dev/blog/wasm-code-caching#stream
[2]: #29
[3]: #3
[4]: #31
chrisdickinson
added a commit
that referenced
this issue
Nov 28, 2023
`createPlugin` now accepts two additional manifest types (`response` and `module`) as well
as `Response` and `WebAssembly.Module`. There are four goals here:
1. Allow us to target the Cloudflare Workers platform. CF Workers only support
loading Wasm via `import` statements; these resolve to `WebAssembly.Module`
objects, which means we need to allow users to pass `Module`s in addition
to our other types.
2. Play nicely with V8's [Wasm caching][1]; in particular V8 will use metadata
from the `Response` to build a key for caching the results of Wasm compilation.
3. This sets us up to implement [Wasm linking][2] by allowing us to introspect
plugin modules imports and exports before instantiation.
4. And finally, resolving to modules instead of arraybuffers allows us to add
[hooks for observe-sdk][3] (especially in advance of adding [thread pooling][4]).
Because Bun lacks support for `WebAssembly.compileStreaming` and
`Response.clone()`, we provide an alternate implementation for converting a
response to a module and its metadata.
One caveat is that there's no way to get the source bytes of a
`WebAssembly.Module`, so `{module}` cannot be used with `{hash}` in a
`Manifest`.
Fixes #9
[1]: https://v8.dev/blog/wasm-code-caching#stream
[2]: #29
[3]: #3
[4]: #31
|
Postpone until after 1.0 and broaden scope to all supported Extism SDKs |
|
Probably related: extism/extism#696 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Background plugins are not pooled at the time of writing. Because calls into a plugin "consume" the backing thread for the duration of execution, many use cases will require a pooled approach. We'll want to make sure we integrate well with
AsyncResourcein order to integrate well with o11y instrumentation. Node has examples for this; we'll want to see if other platform targets provide polyfills or alternatives.The text was updated successfully, but these errors were encountered: