Open Source Sensory Substitution Device
The “Unfolding Space” project deals with sensory substitution: A phenomenon by which the function of one missing or dysfunctional sensory modality is replaced (substituted) by stimulating another one. The outcome is a prototype that projects a 3D picture, generated by a special camera, as vibration patterns on the back of the hand.
The blind can therefore use their tactile modality of the hand to actually see.
Science
This project started as bachelor thesis. It is therefore based on scientific research and can support new findings in the area of cognitive processes.
Design
The prototype is easy to use and learn as a result of using methods of interaction design and rapid prototyping.
Open Source
The project is released under CC-BY licence.
Therefore, the usage and further development of this material is not only allowed but encouraged!
Low Price
Due to production costs of less than 500 Euro it is cheaper than most of the available sensory substitution devices.
Information
Basic Functionality
A 3D generated picture from a depth camera is haptically projected on the back of the hand by using vibration patterns. The location of a vibration depicts an object’s relative position in space, the strength of the vibration represents its distance.
Scientific Background
The theoretical background of this project is called Sensory Substitution. A phenomenon by which the function of one missing or faulty sensory modality is replaced (substituted) by stimulating another one – in this case the tactile modality. At the beginning of the substitution process, this new stimulation has to be actively interpreted by analysing the tactile stimulus. But after some training the new visual-like Input becomes implicit and gets processed subconsciously. Users begin to see the space in front of them.
Motivation
These scientific efforts started more than 50 years ago, but even today there are almost no blind people using substitution processes to handle the absence of their visual system. All attempts to develop a device for the broad market failed.
Process
Many projects failed on a practical implementation. An analysis revealed that despite the elaborate technology used by them, design and usability issues were often not taken into place. Therefore I followed an open and iterative rapid-prototyping approach to quickly work out strengths and potentials and to identify limitations of the hardware and algorithms. Even though my theoretical work about Sensory Plasiticity had already predicted many aspects, it was this process of prototyping that lead me to a functioning device that quick.
Awards and Nominations
Winner of the "Kölner Design Preis / Toby E. Rodes Award" 2018
“In 2018 the „Kölner Design Preis“ was awarded for the 11th time. For the first time it was commended by the Neumann-Hug SA – the preserver of the Toby E. Rodes estate. The awards go to final papers of the cologne design schools and are very much desired under the gradates.”
In German only, sorry…
„Mit ‚Unfolding Space‘ zeichnet die Jury ein Projekt aus, das hinsichtlich Idee, Entwurf und Umsetzung vorbildlich und bestechend ist. Jakob Kilian präsentiert mit dem Handschuh nicht nur einen voll funktionsfähigen Prototypen, sondern zeigt auch auf, wie die Kosten für ein solches Produkt zukünftig um ein Vielfaches reduziert werden können“
In 2018 the project won the Cologne Design Prize which honours the best graduation projects of Cologne based Design Schools on a yearly basis.
For more information go to the Kölner Design Preis website
HackadayPrize 2019 - Honorable Mention (Best Concept)
“A worldwide hardware design contest focused on product development.”
The HackadayPrize annually honours a few Open Source projects uploaded to the hacker and maker community hackaday.io. Condition to enter the prize is a good documentation, a video and upload of all the files and material you need to build a copy of the presented device.
At the award ceremony in November 2019 the project received a “Honorable Mention” in the category “Best Concept”.
For more information about other winning projects and the prize go to the HackadayPrize website.
Golden Pin Concept Design Award 2019 - Design Mark Winner
“Established in 1981, Taiwan’s Golden Pin Design Award is the most influential design award in the global Huaren market.
The Golden Pin Concept Design Award invites products and projects that have not yet entered production or are not yet on the market. The winners of the annual award are granted the right to use the coveted Design Mark. The Best Design winner receives a certificate, award trophy, and a cash prize. The Golden Pin Concept Design Award gathers together innovative design concepts from all around the globe, and creates a platform for future innovation in design.”
At the Golden Pin Concept Design Award the project won a “Design Mark” and was nominated for the “Best Design Winners”. The award ceremony took place in December 2019.
For more information go to the Golden Pin Award website
Lucky Strike Junior Designer Award 2018 – Special Recognition
“With the Lucky Strike Junior Designer Award the Raymond Loewy Foundation is fostering the new generation of designers in Germany since 1992. With this award that comes with 10,000 Euros, the Foundation significantly contributes to design training. The Foundation’s continuous efforts contribute to raising the performance and quality levels at German design universities, support transparency of design training and develop guidelines for university design courses that bring results. Business and industry alike consider receiving an award as part of the Lucky Strike Junior Designer Award a ticket to the design profession. The award provides the new generation of designers a public forum to build networks in the design industry. This ensures the recognition of the broad as well as the industry-specific public.”
The award cerenomy took place at saturday, the 24th of November 2018 at “stilwerk”, Hamburg. Together with 20 other works of young Designers the project received a “special recognition”.
More information on the Lucky Strike Awards website.
Press Releases
In English:
- 14.09.19 – Designboom: Unfolding space by jakob kilian lets the visually impaired ‘see’ with their hands
- 14.09.19 – Hackster.io: Unfolding Space Allows the Blind to See with Their Hands
Other Languages:
- [german] 26.10.18 – Deutschlandfunk: Kölner Design Preis | Auszeichung für „sehenden“ Handschuh
- [german] 26.10.18 – Deutschlandfunk Nova: Student entwickelt sehenden Handschuh
- [german] 26.10.18 – Informationsdienst Wissenschaft: Kölner Design Preis 2018: Erster und dritter Platz für KISD-Absolventen
- [german] 26.10.18 – WDR Kultur: Kölner Designpreis für Sehhilfe
- [german] 26.10.18 – MeineSüdstadt: Kölner Design Preis geht in die Südstadt
- [german] 05.11.18 – barrierefrei bauen:Kölner Designpreis geht an “Unfolding Space”
- [german] 17.11.18 – Koelncampus: Ein Handschuh für mehr Inklusion
- [german] 01.12.18 – handicap.life: Mit den Händen sehen – das Projekt Unfolding Space
- [german] 18.03.19 – Beitrag in der Sendung “Lokalzeit” des WDR
- [chinese] 18.09.19 – 36kr.com: 戴上这个手套，视障人士就能「用手看见」了
- [chinese] 25.09.19 – ifeng.com: 创新前沿 | “橙汁杯子橙皮做”等三个创新案例
- [arabic] 26.09.19 – gulf365.co:فيديو-صور| “jakob kilian” يتيح لضعاف البصر رؤية الأشياء بأيديهم
Thanks for Support
Unfolding Space started as a Bachelor’s thesis back in 2018 but is still actively developed mainly by me myself. Anyway I got so much support over the years: I talked to a lot of people and got so much good advices and suggestions. I got support for code questions, for the production of documentation materials and videos, for the writing of research applications and finally I got sponsored with hardware by producing companies.
I would like to thank all of the supporters of the project, specifically:
Tim Becker, Matthias Krauß, Eduard Paal, Johanna Warchol, Marco Reitano, Brandon Gilles, Niklas Isselburg, Dmitry Kuznetsov, Sebastian Miller, Sören Stirn, Ronnie Balcazar, …
Hardware sponsoring was granted by:
Material & Updates
The whole project is published under CC-BY license (attribution) in order to inspire other designers, developers or commercial providers to work within this field and to make Sensory Substitution available to as many people as possible. In this section you will find an overview over the materials. Feel free to contact me, if you need anything else or if you have any questions regarding my work!
This work is licensed under a Creative Commons Attribution 4.0 International License.
Thesis and Project in General
BA Thesis Paper
The BA thesis paper including a theoretical part about "sensory plasticity" and the documentation of the practical part. Available in German only.
Text Package (1 MB)
This package contains text material such as documentary texts and short descriptions in both German and English.
Picture Package (140MB)
This package contains high resolution pictures of the prototype and the process, which can freely be used with attribution.
Technical Details and Project Progress
Code on Github.com
The current version of the prototype already runs on a mini computer (Raspberry Pi) worn on the upper arm. The code processes the 3D data and drives the motors. It is available under an Open Source License on Github.com
Files, Logs and Instructions on Hackaday.io
Project logs documenting the progress of the project, schematics, instructions, files for 3D printing and the manufacturing of the PCB and a lot more can be found on Hackaday.io
Contact
I am BA in Integrated Design (Köln International School of Design) and working as a freelancer.
Feel free to contact me via Email, via Hackaday.io, via Github.com or just visit my portfolio page.
