Skip to content

Commit

Permalink
Cloudflare worker support (#133)
Browse files Browse the repository at this point in the history
* working wrangler demo

* improve worker example code quality

* add helpers for cloudflare support

* document cloudflare, deno, esmodule

* deno smoketest file

* doc cleanup

* expand loader docs

* test minimal example in CI

* rebuild docs

* document minimal size
  • Loading branch information
justjake authored Dec 29, 2023
1 parent 2482607 commit 0c30c6f
Show file tree
Hide file tree
Showing 152 changed files with 5,863 additions and 1,145 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,9 @@ jobs:
- name: Build tarballs
run: yarn tarball && du -h build/tar/*

- name: Test release with NodeJS minimal ESModule example
run: ./scripts/smoketest-node-minimal.ts

- name: Test release with NodeJS/Typescript example
run: ./scripts/smoketest-node.ts

Expand Down
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
ts/generated
quickjs
.yarn
.*
doc
json-generator-dot-com-1024-rows.json
dist
Expand Down
149 changes: 125 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,13 @@ main()
- [Async module loader](#async-module-loader)
- [Async on host, sync in QuickJS](#async-on-host-sync-in-quickjs)
- [Testing your code](#testing-your-code)
- [Using in the browser without a build step](#using-in-the-browser-without-a-build-step)
- [quickjs-emscripten-core, variants, and advanced packaging](#quickjs-emscripten-core-variants-and-advanced-packaging)
- [Packaging](#packaging)
- [Reducing package size](#reducing-package-size)
- [WebAssembly loading](#webassembly-loading)
- [Using in the browser without a build step](#using-in-the-browser-without-a-build-step)
- [Debugging](#debugging)
- [Supported Platforms](#supported-platforms)
- [More Documentation](#more-documentation)
- [Requirements](#requirements)
- [Background](#background)
- [Status \& Roadmap](#status--roadmap)
- [Related](#related)
Expand Down Expand Up @@ -515,7 +517,105 @@ For more testing examples, please explore the typescript source of [quickjs-emsc
[debug_sync]: https://github.com/justjake/quickjs-emscripten/blob/main/doc/quickjs-emscripten/exports.md#debug_sync
[testquickjswasmmodule]: https://github.com/justjake/quickjs-emscripten/blob/main/doc/quickjs-emscripten/classes/TestQuickJSWASMModule.md

### Using in the browser without a build step
### Packaging

The main `quickjs-emscripten` package includes several build variants of the WebAssembly module:

- `RELEASE...` build variants should be used in production. They offer better performance and smaller file size compared to `DEBUG...` build variants.
- `RELEASE_SYNC`: This is the default variant used when you don't explicitly provide one. It offers the fastest performance and smallest file size.
- `RELEASE_ASYNC`: The default variant if you need [asyncify][] magic, which comes at a performance cost. See the asyncify docs for details.
- `DEBUG...` build variants can be helpful during development and testing. They include source maps and assertions for catching bugs in your code. We recommend running your tests with _both_ a debug build variant and the release build variant you'll use in production.
- `DEBUG_SYNC`: Instrumented to detect memory leaks, in addition to assertions and source maps.
- `DEBUG_ASYNC`: An [asyncify][] variant with source maps.

To use a variant, call `newQuickJSWASMModule` or `newQuickJSAsyncWASMModule` with the variant object. These functions return a promise that resolves to a [QuickJSWASMModule](./doc/quickjs-emscripten/classes/QuickJSWASMModule.md), the same as `getQuickJS`.

```typescript
import {
newQuickJSWASMModule,
newQuickJSAsyncWASMModule,
RELEASE_SYNC,
DEBUG_SYNC,
RELEASE_ASYNC,
DEBUG_ASYNC,
} from "quickjs-emscripten"

const QuickJSReleaseSync = await newQuickJSWASMModule(RELEASE_SYNC)
const QuickJSDebugSync = await newQuickJSWASMModule(DEBUG_SYNC)
const QuickJSReleaseAsync = await newQuickJSAsyncWASMModule(RELEASE_ASYNC)
const QuickJSDebugAsync = await newQuickJSAsyncWASMModule(DEBUG_ASYNC)

for (const quickjs of [
QuickJSReleaseSync,
QuickJSDebugSync,
QuickJSReleaseAsync,
QuickJSDebugAsync,
]) {
const vm = quickjs.newContext()
const result = vm.unwrapResult(vm.evalCode("1 + 1")).consume(vm.getNumber)
console.log(result)
vm.dispose()
quickjs.dispose()
}
```

#### Reducing package size

Including 4 different copies of the WebAssembly module in the main package gives it an install size of [about 9.04mb](https://packagephobia.com/result?p=quickjs-emscripten). If you're building a CLI package or library of your own, or otherwise don't need to include 4 different variants in your `node_modules`, you can switch to the `quickjs-emscripten-core` package, which contains only the Javascript code for this library, and install one (or more) variants a-la-carte as separate packages.

The most minimal setup would be to install `quickjs-emscripten-core` and `@jitl/quickjs-wasmfile-release-sync` (1.3mb total):

```bash
yarn add quickjs-emscripten-core @jitl/quickjs-wasmfile-release-sync
du -h node_modules
# 640K node_modules/@jitl/quickjs-wasmfile-release-sync
# 80K node_modules/@jitl/quickjs-ffi-types
# 588K node_modules/quickjs-emscripten-core
# 1.3M node_modules
```

Then, you can use quickjs-emscripten-core's `newQuickJSWASMModuleFromVariant` to create a QuickJS module (see [the minimal example][minimal]):

```typescript
// src/quickjs.mjs
import { newQuickJSWASMModuleFromVariant } from "quickjs-emscripten-core"
import RELEASE_SYNC from "@jitl/quickjs-wasmfile-release-sync"
export const QuickJS = await newQuickJSWASMModuleFromVariant(RELEASE_SYNC)

// src/app.mjs
import { QuickJS } from "./quickjs.mjs"
console.log(QuickJS.evalCode("1 + 1"))
```

See the [documentation of quickjs-emscripten-core][core] for more details and the list of variant packages.

[core]: https://github.com/justjake/quickjs-emscripten/blob/main/doc/quickjs-emscripten-core/README.md

#### WebAssembly loading

To run QuickJS, we need to load a WebAssembly module into the host Javascript runtime's memory (usually as an ArrayBuffer or TypedArray) and [compile it](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/instantiate_static) to a [WebAssembly.Module](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Module). This means we need to find the file path or URI of the WebAssembly module, and then read it using an API like `fetch` (browser) or `fs.readFile` (NodeJS). `quickjs-emscripten` tries to handle this automatically using patterns like `new URL('./local-path', import.meta.url)` that work in the browser or are handled automatically by bundlers, or `__dirname` in NodeJS, but you may need to configure this manually if these don't work in your environment, or you want more control about how the WebAssembly module is loaded.

To customize the loading of an existing variant, create a new variant with your loading settings using `newVariant`, passing [CustomizeVariantOptions][newVariant]. For example, you need to customize loading in Cloudflare Workers (see [the full example][cloudflare]).

```typescript
import { newQuickJSWASMModule, DEBUG_SYNC as baseVariant, newVariant } from "quickjs-emscripten"
import cloudflareWasmModule from "./DEBUG_SYNC.wasm"
import cloudflareWasmModuleSourceMap from "./DEBUG_SYNC.wasm.map.txt"

/**
* We need to make a new variant that directly passes the imported WebAssembly.Module
* to Emscripten. Normally we'd load the wasm file as bytes from a URL, but
* that's forbidden in Cloudflare workers.
*/
const cloudflareVariant = newVariant(baseVariant, {
wasmModule: cloudflareWasmModule,
wasmSourceMapData: cloudflareWasmModuleSourceMap,
})
```

[newVariant]: https://github.com/justjake/quickjs-emscripten/blob/main/doc/quickjs-emscripten/interfaces/CustomizeVariantOptions.md

#### Using in the browser without a build step

You can use quickjs-emscripten directly from an HTML file in two ways:

Expand Down Expand Up @@ -548,16 +648,6 @@ You can use quickjs-emscripten directly from an HTML file in two ways:
</script>
```

### quickjs-emscripten-core, variants, and advanced packaging

Them main `quickjs-emscripten` package includes several build variants of the WebAssembly module.
If these variants are too large for you, you can instead use the `quickjs-emscripten-core` package,
and manually select your own build variant.

See the [documentation of quickjs-emscripten-core][core] for more details.

[core]: https://github.com/justjake/quickjs-emscripten/blob/main/doc/quickjs-emscripten-core/README.md

### Debugging

- Switch to a DEBUG build variant of the WebAssembly module to see debug log messages from the C part of this library:
Expand Down Expand Up @@ -595,24 +685,35 @@ See the [documentation of quickjs-emscripten-core][core] for more details.

[setDebugMode]: doc/quickjs-emscripten/exports.md#setdebugmode

### More Documentation

[Github] | [NPM] | [API Documentation][api] | [Variants][core] | [Examples][tests]

### Requirements
### Supported Platforms

`quickjs-emscripten` and related packages should work in any environment that supports ES2020.

- NodeJS: requires v16.0.0 or later for WebAssembly compatibility. Tested with node@18.
- We estimate support for the following browsers:
- Browsers: we estimate support for the following browser versions. See the [global-iife][iife] and [esmodule][esm-html] HTML examples.
- Chrome 63+
- Edge 79+
- Safari 11.1+
- Firefox 58+
- NodeJS: requires v16.0.0 or later for WebAssembly compatibility. Tested with node@18. See the [node-typescript][tsx-example] and [node-minimal][minimal] examples.
- Typescript: tested with [email protected] and [email protected]. See the [node-typescript example][tsx-example].
- Vite: tested with [email protected]. See the [Vite/Vue example][vite].
- Create react app: tested with [email protected]. See the [create-react-app example][cra].
- Webpack: tested with [email protected] via create-react-app.
- Vite: tested with [email protected].
- Typescript: tested with [email protected] and [email protected].
- Create react app: tested with [email protected].
- Cloudflare Workers: tested with [email protected]. See the [Cloudflare Workers example][cloudflare].
- Deno: tested with deno 1.39.1. See the [Deno example][deno].

[iife]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/global-iife.html
[esm-html]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/esmodule.html
[deno]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/deno
[vite]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/vite-vue
[cra]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/create-react-app
[cloudflare]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/cloudflare-workers
[tsx-example]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/node-typescript
[minimal]: https://github.com/justjake/quickjs-emscripten/blob/main/examples/node-minimal

### More Documentation

[Github] | [NPM] | [API Documentation][api] | [Variants][core] | [Examples][tests]

## Background

Expand Down
42 changes: 22 additions & 20 deletions doc/@jitl/quickjs-ffi-types/exports.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,14 @@

- [EmscriptenModule](interfaces/EmscriptenModule.md)
- [EmscriptenModuleLoader](interfaces/EmscriptenModuleLoader.md)
- [EmscriptenModuleLoaderOptions](interfaces/EmscriptenModuleLoaderOptions.md)
- [QuickJSAsyncEmscriptenModule](interfaces/QuickJSAsyncEmscriptenModule.md)
- [QuickJSAsyncFFI](interfaces/QuickJSAsyncFFI.md)
- [QuickJSAsyncVariant](interfaces/QuickJSAsyncVariant.md)
- [QuickJSEmscriptenModule](interfaces/QuickJSEmscriptenModule.md)
- [QuickJSFFI](interfaces/QuickJSFFI.md)
- [QuickJSSyncVariant](interfaces/QuickJSSyncVariant.md)
- [SourceMapData](interfaces/SourceMapData.md)

## Type Aliases

Expand All @@ -55,7 +57,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:77](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L77)
[packages/quickjs-ffi-types/src/ffi-types.ts:77](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L77)

***

Expand All @@ -65,7 +67,7 @@ for the Emscripten stack.
#### Source

[variant-types.ts:50](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/variant-types.ts#L50)
[packages/quickjs-ffi-types/src/variant-types.ts:50](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/variant-types.ts#L50)

***

Expand All @@ -75,7 +77,7 @@ for the Emscripten stack.
#### Source

[emscripten-types.ts:166](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/emscripten-types.ts#L166)
[packages/quickjs-ffi-types/src/emscripten-types.ts:242](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/emscripten-types.ts#L242)

***

Expand All @@ -88,7 +90,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:89](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L89)
[packages/quickjs-ffi-types/src/ffi-types.ts:89](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L89)

***

Expand All @@ -100,7 +102,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:19](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L19)
[packages/quickjs-ffi-types/src/ffi-types.ts:19](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L19)

***

Expand All @@ -112,7 +114,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:24](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L24)
[packages/quickjs-ffi-types/src/ffi-types.ts:24](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L24)

***

Expand All @@ -124,7 +126,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:29](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L29)
[packages/quickjs-ffi-types/src/ffi-types.ts:29](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L29)

***

Expand All @@ -136,7 +138,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:14](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L14)
[packages/quickjs-ffi-types/src/ffi-types.ts:14](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L14)

***

Expand All @@ -149,7 +151,7 @@ See JSValueConst and StaticJSValue.

#### Source

[ffi-types.ts:41](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L41)
[packages/quickjs-ffi-types/src/ffi-types.ts:41](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L41)

***

Expand All @@ -161,7 +163,7 @@ Used internally for Javascript-to-C function calls.

#### Source

[ffi-types.ts:51](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L51)
[packages/quickjs-ffi-types/src/ffi-types.ts:51](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L51)

***

Expand All @@ -174,7 +176,7 @@ See JSValue.

#### Source

[ffi-types.ts:35](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L35)
[packages/quickjs-ffi-types/src/ffi-types.ts:35](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L35)

***

Expand All @@ -186,7 +188,7 @@ Used internally for Javascript-to-C function calls.

#### Source

[ffi-types.ts:46](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L46)
[packages/quickjs-ffi-types/src/ffi-types.ts:46](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L46)

***

Expand All @@ -198,7 +200,7 @@ Opaque pointer that was allocated by js_malloc.

#### Source

[ffi-types.ts:94](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L94)
[packages/quickjs-ffi-types/src/ffi-types.ts:94](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L94)

***

Expand All @@ -211,7 +213,7 @@ for the Emscripten stack.

#### Source

[ffi-types.ts:83](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L83)
[packages/quickjs-ffi-types/src/ffi-types.ts:83](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L83)

***

Expand All @@ -223,7 +225,7 @@ Used internally for C-to-Javascript function calls.

#### Source

[ffi-types.ts:61](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L61)
[packages/quickjs-ffi-types/src/ffi-types.ts:61](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L61)

***

Expand All @@ -235,7 +237,7 @@ Used internally for C-to-Javascript interrupt handlers.

#### Source

[ffi-types.ts:66](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L66)
[packages/quickjs-ffi-types/src/ffi-types.ts:66](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L66)

***

Expand All @@ -247,7 +249,7 @@ Used internally for C-to-Javascript module loading.

#### Source

[ffi-types.ts:71](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L71)
[packages/quickjs-ffi-types/src/ffi-types.ts:71](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L71)

***

Expand All @@ -257,7 +259,7 @@ Used internally for C-to-Javascript module loading.
#### Source

[variant-types.ts:49](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/variant-types.ts#L49)
[packages/quickjs-ffi-types/src/variant-types.ts:49](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/variant-types.ts#L49)

## Variables

Expand Down Expand Up @@ -325,7 +327,7 @@ module code

#### Source

[ffi-types.ts:99](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L99)
[packages/quickjs-ffi-types/src/ffi-types.ts:99](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L99)

## Functions

Expand Down Expand Up @@ -358,7 +360,7 @@ module code
#### Source

[ffi-types.ts:106](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L106)
[packages/quickjs-ffi-types/src/ffi-types.ts:106](https://github.com/justjake/quickjs-emscripten/blob/main/packages/quickjs-ffi-types/src/ffi-types.ts#L106)

***

Expand Down
Loading

0 comments on commit 0c30c6f

Please sign in to comment.