Sensing Color Images through Touch and Sound

SoundView is  an experimental vision substitution system  for the blind. Visual  images are  mapped onto  a virtual  surface with  a fine-grained color dependent roughness texture.  The user explores an image by moving a pointer device over the  image which creates sounds.  The current prototype uses a Wacom graphics tablet as a pointer device. The pointer acts like a virtual gramophone needle,  and the sound produced depends on the motion as well as on the  color of the area explored.  User studies were performed  on sighted  people to  test their  ability to  observe simple shapes with this device.

The design of SoundView was described at the 2003 ICAD conference. Conference paper.

The results of user studies with SoundView can be found in this preprint.

A demo (Windows, Linux, Mac OS/X) is available. If you don't have a graphics tablet and pen you can use a mouse too, though this is not a very good pointing device. Download SoundView demo.

Kees van den Doel