~ Presenting ~
with support from Glitch.com, the STUDIO at CMU, the School of AI and you!
A wrapper library around web-based computer vision models for the purpose of interacting with the web handsfree
Powered by Jeeliz Weboji
<!DOCTYPE html>
<head>
<!-- Require dependencies, which adds Handsfree to global namespace -->
<link
rel="stylesheet"
href="https://unpkg.com/[email protected]/dist/handsfreejs/handsfree.css"
/>
<script src="https://unpkg.com/[email protected]/dist/handsfreejs/handsfree.js"></script>
</head>
<body>
<script>
// Create a new instance. Use one instance for each camera
const handsfree = new Handsfree({})
// Create a simple plugin that displays pointer values on every frame
// When using only 1 instance, handsfree === context
Handsfree.use('consoleLogger', (pointer, context) => {
console.log(pointer, context.head.rotation, context.head.morphs)
})
// Start tracking
handsfree.start()
</script>
</body>
- Getting Started: Learn how to setup Handsfree.js and access the
handsfree.pointerproperties - Controlling a YouTube 360 video: Learn how to access and use
handsfree.head.rotationto control a 360 video
The rest of this document is for running Handsfree.js.org and the Handsfree.js library locally.
👆 See the above wiki links for how to actually use Handsfree.js in your own app 👆
This codebase is currently broken into two parts:
- The library itself, located in
/src/assets/handsfree/handsfree.js- Handsfree.js.org, which is everything else
This really should be two separate repositories, but for now just know that the library itself starts in
/src/assets/handsfree/handsfree.js
To run this project locally you'll need NodeJS and the Yarn package manager.
After downloading this project repo, you'll then need to install dependencies by running yarn in the project's root directory. Then you'll have the following commands available:
# Start a local dev environment on localhost:8080
yarn start
# Build for production into /dist/sandbox/
yarn build
# Deploy to handsfree.js.org
yarn deploy
# Create handsfree.js in /dist/handsfreejs/
yarn bundleRunning yarn deploy will commit everything inside of /dist to the gh-pages branch of the repository set in package.json:deploy.repo using the package.json:deploy.url custom domain. This lets you quickly deploy this repository to GitHub Pages!
When you run yarn start, yarn build, or yarn deploy what you're actually doing is running or building the development environment. To create a single handsfree.js script for use within your own projects, do the following:
- Install Parcel on your system globally with:
yarn global add parcel-bundler - Run
npm bundle - The files will be built into
/dist/handsfree/
Handsfree.js is available for free and commercial use under Apache 2.0.
- Jeeliz Weboji (Apache License 2.0) - Face and head pose estimation
A very special thanks goes out to @Golan for inviting me out to his studio, The STUDIO for Creative Inquiry at Carnegie Mellon during the Spring of 2019. It was during this residency that I was encouraged to begin integrating Handsfree.js into different libraries and where I had a chance to use Handsfree.js with a real UR5 robot!
Another special thanks goes out to @AnilDash for sponsoring me during Winter 2018. Also a thank you to The School of AI for the 2018 Fellowship. And a very special thanks to Jess Holbrook from Google Pair for driving all the way (way) out to meet me and helping kickstart this all with a new computer!
Thanks also to everyone who's supported me on Patreon, GoFundMe, and through Twitter over the months (and almost years!). And thanks everyone else for believing in this project 👋