diff --git a/README.md b/README.md index 2536fc7..e120758 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # crunches :muscle: -The (as of the time of writing this) smallest **and** fastest JavaScript value serialization library in the wild. **2.3 kB** gzipped. Efficiently encode and decode your values to and from `ArrayBuffer`s. Integrates very well with WebSockets. +The (as of the time of writing this) smallest **and** fastest JavaScript value serialization library in the wild. **2.64 kB** gzipped; **0 dependencies**. Efficiently encode and decode your values to and from `ArrayBuffer`s. Integrates very well with WebSockets. ## Example @@ -70,7 +70,7 @@ In this example, the size of payload is only **22 bytes**. `JSON.stringify` woul [SchemaPack](https://github.com/phretaddin/schemapack/tree/master) (huge respect from and inspiration for this library! :heart:) is great for packing objects into Node buffers. Over time, this approach has become outdated in favor of modern standards like `ArrayBuffer`. -It is also frequently desirable to preallocate and reuse buffers for performance reasons. SchemaPack always allocates new buffers when encoding. The performance hit is generally less than the naive case since Node is good about buffer pooling, but performance degrades in the browser (and doesn't exist on any other platform). Buffer reuse is the Correct Way™. +It is also frequently desirable to preallocate and reuse buffers for performance reasons. SchemaPack always allocates new buffers when encoding. The performance hit is generally less than the naive case since Node is good about buffer pooling, but performance degrades in the browser (and doesn't exist on any other platform). Buffer reuse is the Correct Way™. We also apply even more [optimizations of buffers and arrays](#buffers-and-arrays). I also wanted an implementation that does amazing things like [boolean coalescence](#boolean-coalescence) and [optional fields](#optional-fields) (also with [coalescence](#optional-field-coalescence)) as well as supporting more even more types like `Map`s, `Set`s, `Date`s, etc. @@ -205,28 +205,28 @@ Just set a key on the `Codecs` object and go. Too easy! ## Schema types -| Type Name | Bytes | Range of Values | -|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------| -| bool | 1 (worst case, see [boolean coalescence](#boolean-coalescence)) | Truthy values are coerced to `true`; falsy values to `false` | -| int8 | 1 | -128 to 127 | -| uint8 | 1 | 0 to 255 | -| int16 | 2 | -32,768 to 32,767 | -| uint16 | 2 | 0 to 65,535 | -| int32 | 4 | -2,147,483,648 to 2,147,483,647 | -| uint32 | 4 | 0 to 4,294,967,295 | -| float32 | 4 | 3.4E +/- 38 (7 digits) | -| float64 | 8 | 1.7E +/- 308 (15 digits) | -| string | 32-bit length prefix followed by the [encoded](https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder/encodeInto) string bytes | Any string | -| buffer | 32-bit length prefix followed by the bytes of the buffer | Any `ArrayBuffer`

**NOTE:** For performance, a `DataView` of the buffer is returned when decoding to avoid buffer copying. | -| varuint |
sizeminmax
10127
212816,383
316,3842,097,151
42,097,152268,435,455
5268,435,45634,359,738,367
634,359,738,3684,398,046,511,103
74,398,046,511,104562,949,953,421,311
| 0 to 562,949,953,421,311 | -| varint |
sizeminmax
1-6463
2-8,1928,191
3-1,048,5761,048,575
4-134,217,728134,217,727
5-17,179,869,18417,179,869,183
6-2,199,023,255,5522,199,023,255,551
7-281,474,976,710,656281,474,976,710,655
| -281,474,976,710,656 to 281,474,976,710,655 | -| date | Same as `string` above after calling [`toIsoString`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString) | Value is coerced to `Date` e.g. `new Date(value).toIsoString()` | +| Type Name | Bytes | Range of Values | +|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------| +| bool | 1 (worst case, see [boolean coalescence](#boolean-coalescence)) | Truthy values are coerced to `true`; falsy values to `false` | +| int8 | 1 | -128 to 127 | +| uint8 | 1 | 0 to 255 | +| int16 | 2 | -32,768 to 32,767 | +| uint16 | 2 | 0 to 65,535 | +| int32 | 4 | -2,147,483,648 to 2,147,483,647 | +| uint32 | 4 | 0 to 4,294,967,295 | +| float32 | 4 | 3.4E +/- 38 (7 digits) | +| float64 | 8 | 1.7E +/- 308 (15 digits) | +| string | 32-bit length prefix followed by the [encoded](https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder/encodeInto) string bytes | Any string | +| buffer | 32-bit length prefix followed by the bytes of the buffer | Any `ArrayBuffer`. Decodes to a `DataView`.

See: [buffers and arrays](#buffers-and-arrays). | +| varuint |
sizeminmax
10127
212816,383
316,3842,097,151
42,097,152268,435,455
5268,435,45634,359,738,367
634,359,738,3684,398,046,511,103
74,398,046,511,104562,949,953,421,311
| 0 to 562,949,953,421,311 | +| varint |
sizeminmax
1-6463
2-8,1928,191
3-1,048,5761,048,575
4-134,217,728134,217,727
5-17,179,869,18417,179,869,183
6-2,199,023,255,5522,199,023,255,551
7-281,474,976,710,656281,474,976,710,655
| -281,474,976,710,656 to 281,474,976,710,655 | +| date | Same as `string` above after calling [`toIsoString`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString) | Value is coerced to `Date` e.g. `new Date(value).toIsoString()` | ### Aggregate types #### `object` -> Requires a `properties` key to define the properties on the object. Supports [`optional` fields](#optional-fields). Booleans are [coalesced](#boolean-coalescence). +Requires a `properties` key to define the properties on the object. Supports [`optional` fields](#optional-fields). Booleans are [coalesced](#boolean-coalescence). Example: @@ -247,7 +247,7 @@ console.log(schema.size({foo: 32})); #### `array` -> Requires an `element` key to define the structure of the array elements. Encodes a 32-bit prefix followed by the contents of the array. +Requires an `element` key to define the structure of the array elements. Encodes a 32-bit prefix followed by the contents of the array. ```js const schema = new Schema({ @@ -259,9 +259,11 @@ const schema = new Schema({ console.log(schema.size([1, 2, 3])); ``` +[Arrays of number types decode to the corresponding `TypedArray`](#buffers-and-arrays). + #### `map` -> Requires a `key` and `value` key to define the structure of the map. Any [iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols) will be coerced as [entries](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/Map#iterable). Encoded as an array of entries. Decodes to a native `Map` object. +Requires a `key` and `value` key to define the structure of the map. Any [iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols) will be coerced as [entries](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/Map#iterable). Encoded as an array of entries. Decodes to a native `Map` object. ```js const schema = new Schema({ @@ -282,7 +284,7 @@ console.log(schema.size([[32, 'sup'], [64, 'hi']])); #### `set` -> Requires an `element` key to define the structure of the map. Any [iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols) will be coerced. Encoded as an array. Decodes to a native `Set` object. +Requires an `element` key to define the structure of the map. Any [iterable](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols) will be coerced. Encoded as an array. Decodes to a native `Set` object. ```js const schema = new Schema({ @@ -317,6 +319,25 @@ Defining schema blueprints are slightly more verbose than SchemaPack. The tradeo SchemaPack's `varint` types only work up to $2^{30}-1$ whereas `crunches` uses mathematical transformations (instead of bitwise) to allow numbers up to [`Number.MAX_SAFE_INTEGER`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER). In practice, due to sacrificing space for the length flags, this is $2^{48}-1$. Same goes for `varuint`: $2^{31}-1$ vs. $2^{49}-1$. +### Buffers and arrays + +A massive performance gain is achieved by copy-free buffer decoding. In other words, a buffer value is not copied out of the binary from which it is decoded; a `DataView` is created over the encoded binary and the `DataView` is returned. Decoding a 1024-byte buffer is ***10x faster*** on the machine used to benchmark. The gains increase even more as the size of the buffer increases. + +A similar performance gain is also used for arrays. The fast path is used for arrays of the following types: + +- `int8` +- `uint8` +- `int16` +- `uint16` +- `int32` +- `uint32` +- `float32` +- `float64` + +Instead of copying the data from the buffer, a [`TypedArray`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Typed_arrays#typed_array_views) is created over the encoded binary and returned instead. The same optimization is applied for encoding. This optimization nets a gain of roughly **1.5x** speed for encoding and ***50x*** speed for decoding on the machine used to benchmark. + +**NOTE:** `float64` arrays are padded with an extra 4 bytes after the length prefix to satisfy the required 8-byte alignment. + # TODO - Fixed-length arrays @@ -332,7 +353,7 @@ SchemaPack's `varint` types only work up to $2^{30}-1$ whereas `crunches` uses m **A**: 'cuz you gotta crunch those flabby AB(`ArrayBuffer`)s! **Q**: Why no TypeScript support? -**A**: Feel free to create an issue. +**A**: Feel free to contribute typing! -**Q**: This library isn't **always** faster than SchemaPack! -**A**: That's true, at least for now. SchemaPack does a lot of black magic and compiles codecs on-the-fly. However, it is consistently faster when using larger aggregate types even in its current state. +**Q**: How fast is it, overall? +**A**: Benchmarks are generally dubious in my opinion, but the `benchmark.js` script included in the repository runs 50,000 iterations of both SchemaPack and `crunches` encoding and decoding a schema. SchemaPack validation is disabled, to be as fair as possible. On the machine used to benchmark, `crunches` runs consistently **3-4x faster** than SchemaPack.