Worker streaming (#3)

squash streaming work

README.md

@@ -1,25 +1,29 @@
-# webnect *...now with [live demo](https://turbocrime.github.io/webnect/)!*
+# webnect
+
+**(try [live demo](https://turbocrime.github.io/webnect/))**
 
 this is a webusb driver for microsoft's xbox360 kinect.
 
-![webnect](https://github.com/turbocrime/webnect/assets/134443988/1bfbb58f-4a5a-4276-8cde-b80c7d91b63a)
+<https://github.com/turbocrime/webnect/assets/134443988/40879425-cd91-4bae-a14f-633dc0f8d88c>
 
 chrome only :'C mozzy dont do webusb
 
 i have never written a usb driver before, nor even a typescript library, so critique is welcome.
 
-currently, this driver only supports "Xbox NUI Motor" PID `0x02b0` and "Xbox NUI Camera" PID `0x02ae` devices, labelled "Model 1414", because that's what i found at goodwill. i believe there's a few externally-identical models of "kinect", some with dramatic hardware revisions. if your device doesn't work with this, please verify that it works at all, and then send me the details.
+this driver at least works with "Xbox NUI Motor" PID `0x02b0` and "Xbox NUI Camera" PID `0x02ae` devices, labelled "Model 1414", because that's what i found at goodwill. i there may be a few externally-identical models of "kinect", some with dramatic hardware revisions. if your device doesn't work with this, please verify that it works at all, and then send me the details.
 
 ## what
 
 the kinect is an early consumer depth sensor based on structured light projection, plus some other goodies. it was released in 2010 as a gamer thing and nobody cares about it anymore except me
 
-they're fun, and they go for like $5 now. plus they're usb2, so i can drive them with throwaway SBCs like an rpi3. maybe not with this driver, but that's how i got familiar. i been using them for video synth input, and interactive generative art installations (yes i do parties, hmu) but it's the kind of thing you really gotta see in person
+they're fun, and they go for like $5 now. plus they're usb2, so i can drive them with throwaway SBCs like an rpi3. maybe not with this driver, but that's how i got familiar. i been using them for video synth input, and interactive generative art installations
 
 a webusb driver lets more folks see it in person :) after i rewrite everything :)
 
 ## ware
 
+original kinect only.
+
 ### Xbox NUI Motor
 
 * accelerometer
@@ -28,10 +32,11 @@ a webusb driver lets more folks see it in person :) after i rewrite everything :
 
 ### Xbox NUI Camera
 
-* depth camera 11bit only
-* thats it for now
-* no ir
-* no visible
+* depth 11bpp, 10bpp
+* visible 8bpp bayer
+* infrared 10bpp
+
+visible and infrared stream from the same endpoint, so you can only have one at a time.
 
 ## why
 
@@ -41,11 +46,13 @@ whatever. its the future and webusb is real
 
 ## how
 
-available on npm as [`@webnect/webnect`](https://www.npmjs.com/package/@webnect/webnect)
+go dig your kinect out of the closet. plug it in. open <https://turbocrime.github.io/webnect>
+
+## diy
 
-for local demos, clone the repo.
+available on npm as [`@webnect/webnect`](https://www.npmjs.com/package/@webnect/webnect)
 
-go dig your kinect out of the closet. plug it in. run
+or for a local demo, clone this repo. run
 
 ```sh
 $ pnpm install
@@ -79,24 +86,24 @@ devices? : {
 
 pass a boolean indicating your desire to request acquisition, or pass a USBDevice if you have already one already acquired.
 
-## it dont work
+### um its broekn
 
 if you see an empty device selection modal, you probably have the wrong model kinect. you can check your usb devices with `lsusb` on linux or on `system_profiler SPUSBDataType` on macos
 
 if you see glitchy stream output, haha nice. cool
 
-## bad parts
+### bad parts
 
 a single kinect is technically three devices in a trenchcoat. afaict there's no way to associate them, because webusb won't expose bus position. it doesn't matter; you probably only plugged in one kinect anyway.
 
 also, typescript aint exactly the optimal language for bitmath or destructuring binary data
 
-## the future
+## way
 
-the mathy parts are an obvious candidate for webgpu acceleration. do NOT send a patch i wanna do it
+the mathy parts are an obvious candidate for assemblyscript and webgpu acceleration. do NOT send a patch i wanna do it
 
-i should probably learn how to actually use canvas and arraybuffers
+i should probably learn how to actually use canvas and streams
 
-probably going after ir video next, then bayer/yuv, and then audio device and firmware stuff.
+probably going after pose features next, maybe registration of visible light to depth frames. and then audio device and firmware stuff.
 
 someday.... kinect2?

demo/index.html

@@ -4,7 +4,7 @@
 <head>
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-  <title>kinect WebUSB demo</title>
+  <title>webnect demo</title>
 </head>
 
 <body>
@@ -19,6 +19,10 @@ <h2><a href="https://github.com/turbocrime/webnect">webnect is a WebUSB driver f
     <p>when your kinect is plugged in, activate it with the UI below</p>
     <img src="https://github.com/turbocrime/webnect/assets/134443988/141bb8b5-fdf4-45a4-b58e-2f00b590fae8"
       alt="an image of the appropriate kinect">
+    <video autoplay loop>
+      <source src="https://github.com/turbocrime/webnect/assets/134443988/40879425-cd91-4bae-a14f-633dc0f8d88c">
+      </source>
+    </video>
     <p>Firefox, Safari are not supported ;_; only chrom</p>
     <p>i have only tried this with my kinect. you can try it on yours if you like</p>
     <fieldset id="requestDevices">
@@ -50,12 +54,16 @@ <h3>
     <legend>
       <h2>Camera</h2>
     </legend>
-    <button id="fsDepth">Fullscreen</button>
-    <label>
-      <input type="checkbox" id="depthStream"></input>Depth Stream
-    </label>
+    <button id="videoFsBtn">FullscreenVideo</button>
+    <select id="videoMode">
+      <option value="depth">Depth</option>
+      <option value="visible">Visible</option>
+      <option value="ir">Infrared</option>
+    </select>
+    <div>FPS: <span id="videoFps"></span></div>
+    <label><input type="checkbox" id="flipCb"></input>Flip</label>
     <div class="canvasBack">
-      <canvas id="depthCanvas" width="640" height="480"></canvas>
+      <canvas id="videoCanvas" width="640" height="480"></canvas>
     </div>
   </fieldset>
 

demo/package.json

@@ -9,7 +9,7 @@
 		"preview": "vite preview"
 	},
 	"dependencies": {
-		"@webnect/webnect": "workspace:../"
+		"@webnect/webnect/": "workspace:../"
 	},
 	"devDependencies": {
 		"@types/node": "^20.4.4",