Tangibles

preconditions: experience with visioning; visioning libraries; a bit character recognition; Java; a bit experience with design

context: OpenRoberta is a learning platform. It uses a graphical programming language (based on blockly) and has code generators/loader for many robots and embedded systems used in education. Browse through https:lab.open-roberta.org . Use the simulator and write some simple programs

problem: the lab should support group work in a class room. Traditional interfaces (laptop, keyboard, mouse) are not well-suited for this.

idea: programmming blocks made of cardboard (or wood, ...) are available on a desk. Students grouped around this desk can put these parts together to make up a program, discuss this, modify and so on (so the program is tangible). After they have finished, one takes a photo using a tablet or smartphone. The image is analyzed (server-side), the program is detected and extracted, shown in a browser, compiled and sent to a robot to execute it. Compilation and sending is already available, of course, and not part of the project

task: analyse the graphical building blocks of the lab and design your set of cardboard blocks.

Take an image and use image processing software to generate the blocks and chain them into the program. It is your choice to use traditional technology or have trained a neural network using pictures of cardboards or ... .

If this sounds too easy: some blocks can contain hand-written numbers or text. Do character recognition for that.

note: the project is a nice combination of design and programming. Experience shows, that this project can be more complex than you may expect especially if you don't have much knowledge about visioning (image preprocessing, segmentation).

contact: reinhard.budde AT iais.fraunhofer.de