Open Source Sensory Substitution Device

The “Unfolding Space” project deals with sensory substitution: A phenomenon by which the function of one missing or dysfunctional sensory modality is replaced (substituted) by stimulating another one. The outcome is a prototype that  projects a 3D picture, generated by a special camera, as vibration patterns on the back of the hand.

The blind can therefore use their tactile modality of the hand to actually see.

Science

This project started as bachelor thesis. It is therefore based on scientific research and can support new findings in the area of cognitive processes.

Design

The prototype is easy to use and learn as a result of using methods of interaction design and rapid prototyping.

Open Source

The project is released under CC-BY licence.
Therefore, the usage and further development of this material is not only allowed but encouraged!

Low Price

Due to production costs of less than 500 Euro it is cheaper than most of the available sensory substitution devices.

Video Teaser

Information

Basic Functionality

A 3D generated picture from a depth camera is haptically projected on the back of the hand by using vibration patterns. The location of a vibration depicts an object’s relative position in space, the strength of the vibration represents its distance.

Scientific Background

The theoretical background of this project is called Sensory Substitution. A phenomenon by which the function of one missing or faulty sensory modality is replaced (substituted) by stimulating another one – in this case the tactile modality. At the beginning of the substitution process, this new stimulation has to be actively interpreted by analysing the tactile stimulus. But after some training the new visual-like Input becomes implicit and gets processed subconsciously. Users begin to see the space in front of them.

Motivation

These scientific efforts started more than 50 years ago, but even today there are almost no blind people using substitution processes to handle the absence of their visual system. All attempts to develop a device for the broad market failed.

Process

Many projects failed on a practical implementation. An analysis revealed that despite the elaborate technology used by them, design and usability issues were often not taken into place. Therefore I followed an open and iterative rapid-prototyping approach to quickly work out strengths and potentials and to identify limitations of the hardware and algorithms. Even though my theoretical work about Sensory Plasiticity had already predicted many aspects, it was this process of prototyping that lead me to a functioning device that quick.

Awards and Nominations

Winner of the "Kölner Design Preis / Toby E. Rodes Award"

In 2018 the „Kölner Design Preis“ was awarded for the 11th time. For the first time it was commended by the Neumann-Hug SA – the preserver of the Toby E. Rodes estate. The awards go to final papers of the cologne design schools and are very much desired under the gradates.

Only in German – Sorry

„Mit ‚Unfolding Space‘ zeichnet die Jury ein Projekt aus, das hinsichtlich Idee, Entwurf und Umsetzung vorbildlich und bestechend ist. Jakob Kilian präsentiert mit dem Handschuh nicht nur einen voll funktionsfähigen Prototypen, sondern zeigt auch auf, wie die Kosten für ein solches Produkt zukünftig um ein Vielfaches reduziert werden können“

For more information go to the Kölner Design Preis website

Golden Pin Concept Award Mark + amongst the 7 finalists

Established in 1981, Taiwan’s Golden Pin Design Award is the most influential design award in the global Huaren market.

The Golden Pin Concept Design Award invites products and projects that have not yet entered production or are not yet on the market. The winners of the annual award are granted the right to use the coveted Design Mark. The Best Design winner receives a certificate, award trophy, and a cash prize. The Golden Pin Concept Design Award
gathers together innovative design concepts from all around the globe, and creates a platform for future innovation in design.

Final selection of the “Best Design Winners” will take place in November 2019.

For more information go to the Golden Pin Award website

"Special Recognition" at the Lucky Strike Junior Designer Award 2018

“With the Lucky Strike Junior Designer Award the Raymond Loewy Foundation is fostering the new generation of designers in Germany since 1992. With this award that comes with 10,000 Euros, the Foundation significantly contributes to design training. The Foundation’s continuous efforts contribute to raising the performance and quality levels at German design universities, support transparency of design training and develop guidelines for university design courses that bring results. Business and industry alike consider receiving an award as part of the Lucky Strike Junior Designer Award a ticket to the design profession. The award provides the new generation of designers a public forum to build networks in the design industry. This ensures the recognition of the broad as well as the industry-specific public.

The award cerenomy took place at saturday, the 24th of November 2018 at “stilwerk”, Hamburg. Together with 20 other young Designers I received a “special recognition” for the project. More information on the Lucky Strike Awards website.

Press Releases

In English:
Other Languages:

Material & Updates

The whole project is published under CC-BY license (attribution) in order to inspire other designers, developers or commercial providers to work within this field and to make Sensory Substitution available to as many people as possible. In this section you will find an overview over the materials. Feel free to contact me, if you need anything else or if you have any questions regarding my work!

Thesis and Project in General

BA Thesis Paper

The BA thesis paper including a theoretical part about "sensory plasticity" and the documentation of the practical part. Available in German only.

Text Package (1 MB)

This package contains text material such as documentary texts and short descriptions in both German and English.

Picture Package (140MB)

This package contains high resolution pictures of the prototype and the process, which can freely be used with attribution.

Technical Details and Project Progress

Code on Github.com

The current version of the prototype already runs on a mini computer (Raspberry Pi) worn on the upper arm. The code processes the 3D data and drives the motors. It is available under an Open Source License on Github.com

Files, Logs and Instructions on Hackaday.io

Project logs documenting the progress of the project, schematics, instructions, files for 3D printing and the manufacturing of the PCB and a lot more can be found on Hackaday.io

Contact

I am BA in Integrated Design (Köln International School of Design) and working as a freelancer.

Feel free to contact me via Email, via Hackaday.io, via Github.com or just visit my portfolio page.

Der Autor und Macher des Projekts Jakob Kilian im Porträt
Jakob Kilian
Interaction Designer
Eine Hand steckt Kabel zur Verbindung eines elektrischen Geräts