“Unsinn” is a creative coding experiment that fuses machine learning, embodied interaction, and human language. Utilizing p5.js with Google’s open-source machine learning solution MediaPipe, the piece aims to explore the imaginative possibilities of the German language. As a non-native speaker of German, I am particularly fascinated by prefixes, which can drastically alter the meaning of a word in German. Through a random selection of prefixes and verbs, I invite participants to interact with both hands to generate nonsensical and often surprising word combinations. The result is a playful, hands-on experience of the German language.
Bring both hands up so they are visible on screen. One of your hands will display a German prefix, and the other hand a German verb. Based on the distance between the index and thumb finger points on each hand, you can manipulate the size of the text to make it larger or smaller. The text will follow your hands across the screen. If you touch your fingers together, a different, randomly-generated prefix or verb will appear. You can mix and match the text to create new, nonsensical word combinations.
- Live website: https://catalinaescalona.github.io/unsinn/
- Video recording of the piece in action: https://www.youtube.com/watch?v=RqJH2aEKnjM
- MediaPipe documentation: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker
- Starter code from Nahuel Gerth: https://github.com/NahuelGerthVK/bodytracking-p5-mediapipe/tree/main/04_resize_letter