Skip to content

Commit

Permalink
chore(example): websocket server example
Browse files Browse the repository at this point in the history
Adds an example project in ./example that demonstrates hosting an
instance of Hold This as a WebSocket server. Using multiple clients to
read and write data, giving a starting point for implementation.
  • Loading branch information
gregdaynes committed May 24, 2024
1 parent 28b98be commit 6d11edf
Show file tree
Hide file tree
Showing 6 changed files with 269 additions and 10 deletions.
92 changes: 82 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,44 @@ Hold This
A simple key-value store that uses SQLite as the backend.
It is designed to be used in a single-threaded synchronous environment.

[Getting Started](#getting-started)
[Examples](#examples)
- [No Config](#no-config-setup)
- [File Backed](#file-backed-store)
- [WebSocket Server](#websocket-server)
- [Data Serialization](#data-serialization)
- [TTL / Expiring Records](#ttl-%2F-expiring-records)
- [Turbo Mode](#turbo-mode)
- [Bulk Insertion](#bulk-insertion)
- [Buffered Insertion](#buffered-insertion)


Getting Started
---------------

```js
npm install --save hold-this
```
1. Add dependency to project

npm install --save hold-this

2. Create instance of hold-this

import Hold from 'hold-this'

const holder = Hold()

3. Write data to store

holder.set('accounts', 'account-123:user-123:name', 'Alice')

### Use in-memory store
4. Read data from store

holder.get('accounts', 'account-123:*:name')


Examples
--------

### No Config Setup

```js
import hold from 'hold-this'
Expand All @@ -24,10 +54,7 @@ console.log(holder.get('accounts', 'account-123:*:name'))
// => [['account-123:user-123:name', 'Alice'], ['account-123:user-456:name', 'Bob']]
```

Other Examples
--------------

### File based store
### File backed store

Pass an object with a key `location` and a path to a file. This will be the filepath that hold-this utilizes to write to disk.

Expand Down Expand Up @@ -75,6 +102,46 @@ const holder = holder({ location: './holder.sqlite', enableWAL: false })

_Performed on Macbook Pro M1 with 16 GB Memory_


### WebSocket Server

Because Hold This is based on SQLite3, it does not support a native network connection.
However we can achieve a networked, multi-connection instance by wrapping Hold This in a WebSocket Server.

>[!CAUTION]
>This is not a production ready example. Security, and failure modes must be considered, but are outside the scope of the example.
```js
/* global WebSocket */
import { WebSocketServer } from 'ws'
import Hold from 'hold-this'

const server = new WebSocketServer(3000)
server.holder = Hold()

server.on('connection', function connection (ws) {
ws.on('error', console.error)

ws.on('message', function message (payload) {
const message = JSON.parse(payload)

const { id, cmd } = message
const { topic, key, value, options } = message.data

const data = server.holder[cmd](topic, key, value, options)

ws.send(JSON.stringify({ id, cmd, data }))
})

ws.send('connected')
})

export default server
```

_The complete example including client and benchmarks can be found in [example/](example/)_


### Bind Topic / Shorthand

Calling `.bind('myTopic')` on your hold-this instance, will return a modified instance that has topic already defined on set/get methods.
Expand All @@ -90,7 +157,8 @@ console.log(holder.get('account-123:*:name'))
// => [['account-123:user-123:name', 'Alice'], ['account-123:user-456:name', 'Bob']]
```

### Serialization

### Data Serialization

When passing the value with `.set`, if the value is not a string, the data will be serialized with `serialize-javascript` and then stored.
Passing an options object like `{ isJSON: true }`, with a proper JSON object, will signal to the serializer to use a faster mechanism.
Expand All @@ -106,7 +174,8 @@ console.log(holder.get('account-123:*:name'))
// => [['account-123:user-123:name', { firstName: 'Alice' }], ['account-123:user-456:name', 'Bob']]
```

### TTTL / Expiring Records

### TTL / Expiring Records

When setting a record, specifying in a options object `{ ttl: 1000 }` will set a date in the future where the record will no longer be retrievable.
Note: TTL value is set in milliseconds.
Expand Down Expand Up @@ -135,6 +204,7 @@ holder.set('accounts', 'account-123:user-123:name', 'Alice', { ttl: 1000 })
holder.clean()
```


### Turbo Mode

If speed of insertion is a priority, turbo mode can be enabled.
Expand All @@ -157,6 +227,7 @@ console.log(holder.get('accounts', 'account-123:*:name'))
// => [['account-123:user-123:name', 'Alice']]
```


### Bulk Insertion

Bulk insertion leverages transactions to insert a batch of records, prepared ahead of time.
Expand Down Expand Up @@ -187,6 +258,7 @@ console.log(holder.get('bulk', 'key:*'))
_Using Turbo Mode_
_Performed on Macbook Pro M1 with 16 GB Memory_


### Buffered Insertion

Like bulk insertion, buffered insertion uses transactions, but handles _everything_ for you.
Expand Down
1 change: 1 addition & 0 deletions example/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
22.1.0
26 changes: 26 additions & 0 deletions example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Hold This Example: WebSocket
============================

This is an example application that runs Hold This behind a WebSocket server, and has multiple clients connect, write and read to the same instance.

>[!CAUTION]
>This is not a production ready example. Security, and failure modes must be considered, but are outside the scope of the example.
>[!NOTE]
>This example depends on the package `ws` to provide the WebSocket Server
>
>Requires Node 22.0.0
>- WebSocket Client
>- Promise.withResolvers
>- import.meta.filename
Getting Started
---------------

1. Install dependency

npm install

2. Run Example

npm start
31 changes: 31 additions & 0 deletions example/bench.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { Bench } from 'tinybench'
import { Client, Server } from './index.js'
import { randomUUID } from 'node:crypto'

// disable console.info when running benchmarks
console.info = function () {}

const bench = new Bench()

const server = Server()
const client = await Client()

server.holder.set('socket', 'foo', 'bar')

bench
.add('GET through WebSocket', async () => {
const id = randomUUID()

await new Promise((resolve) => {
client.messages.once(`get:${id}`, resolve)
client.send(JSON.stringify({ id, cmd: 'get', data: { topic: 'socket', key: 'foo' } }))
})
})
.add('GET local connection', () => {
server.holder.get('socket', 'foo')
})

await bench.run()
console.table(bench.table())
client.close()
server.close()
111 changes: 111 additions & 0 deletions example/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/* global WebSocket */
import { randomUUID } from 'node:crypto'
import EventEmitter from 'node:events'
import { WebSocketServer } from 'ws'
import Hold from 'hold-this'

/**
* Creates a WebSocket server decorated with an instance of the data store
*
* The server is dependent on WebSocketServer provided by the ws package.
*
* @param {Object} options - server options
* @param {number} options.port - port number to listen on
* @returns {WebSocketServer}
* @example
* const server = Server()
*/
export function Server ({ port = 3000 } = {}) {
const server = new WebSocketServer({ port })
server.holder = Hold()

server.on('connection', function connection (ws) {
ws.on('error', console.error)

ws.on('message', function message (payload) {
const message = JSON.parse(payload)
console.info('Server Rx', message)

const { id, cmd, data: { topic, key, value, options } } = message
const data = server.holder[cmd](topic, key, value, options)

ws.send(JSON.stringify({ id, cmd, data }))
})

ws.send('connected')
})

return server
}

/**
* Creates a WebSocket client that connects to the server.
* The client is decorated with an EventEmitter to handle receiving response messages from the
* server initialized by a previous request. This is done by listening to the `get:${id}` event.
*
* Client uses the WebSocket API provided by Node 22.0.0.
* @param {Object} options - client options
* @param {string} options.url - server url to connect to
* @returns {Promise<WebSocket>}
* @example
* const client = await Client()
* client.send(JSON.stringify({ id: '123', cmd: 'get', data: { topic: 'socket', key
*/
export function Client ({ name, url = 'ws://localhost:3000' } = {}) {
const client = new WebSocket(url)
client.messages = new EventEmitter()

client.post = async function send (cmd, data) {
const request = Promise.withResolvers()
const id = randomUUID()
client.send(JSON.stringify({ id, cmd, data }))
client.messages.on(`${cmd}:${id}`, (data) => request.resolve(data))

return request.promise
}

return new Promise(function (resolve, reject) {
client.addEventListener('error', (err) => {
console.error(err)

// If error on connection, reject the promise to make the client unavailable
return reject(err)
})

client.addEventListener('message', (message) => {
// Handle connection message, resolving the promise when connected, enabling
// the await Client() initialization api.
if (message.data === 'connected') {
console.info(`Client[${name}] connected`)
return resolve(client)
}

const { id, cmd, data } = JSON.parse(message.data)
console.info(`Client[${name}] Rx`, id, data)

client.messages.emit(`${cmd}:${id}`, data)
})
})
}

// When this file is run directly from node (main module)
// setup a server and two clients, and run a test demonstrating
// the use of the Server and Client functions.
if (process.argv[1] === import.meta.filename) {
const server = Server()
const A = await Client({ name: 'A' })
const B = await Client({ name: 'B' })

await A.post('set', { topic: 'socket', key: 'a', value: 'a' })
await B.post('set', { topic: 'socket', key: 'b', value: 'b' })

await Promise.all([
A.post('get', { topic: 'socket', key: 'a' }),
B.post('get', { topic: 'socket', key: 'b' })
])
.then(() => {
A.close()
B.close()
server.close()
})
}
18 changes: 18 additions & 0 deletions example/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"name": "hold-this-example-socket-server",
"version": "1.0.0",
"description": "Demonstrates how to use hold-this over a network with multiple clients",
"main": "index.js",
"type": "module",
"scripts": {
"start": "node index.js",
"bench": "node bench.js"
},
"keywords": [],
"author": "Gregory Daynes <[email protected]>",
"license": "ISC",
"dependencies": {
"hold-this": "file:../",
"ws": "^8.17.0"
}
}

0 comments on commit 6d11edf

Please sign in to comment.