Multisensory user interface

A project led by Ph.D. Assistant professor Marjatta Kangassalo and Professor Roope Raisamo is developing a learning environment that can be used by both normally seeing and vision-impaired children. Until now, vision-impaired children have been at a disadvantage compared to normally seeing children.

Teaching programs developed for general use have not been of any help because they rely heavily on pictorial and text-based forms of expression. New user interface technologies now allow these programs to be used by the seeing impaired. Kangassalo and Raisamo’s project is a part of the Proactive Computing Research Program (PROACT) funded by the Academy of Finland.
 
Teaching programs developed in the project are mainly directed at preschool and elementary school children. So-called proactive intelligent agents monitor a child’s computer use, and aim to provide support and guidance when needed. Multi-sensory user interface technologies, especially touch and voice feedback, enable computer use and learning even when the user can’t see the applications. When a child is vision-impaired or normally seeing, the visual feedback supports the information provided by touch and voice feedback. The learning applications deal with natural and astronomical phenomena.

Children have tested the learning applications

Visually impaired children have been involved throughout the whole project. They have tested the developed applications, and participated in more widespread user tests during different phases of the project. Normally seeing children have also tested the developed solutions. Exploratory learning in a teaching environment is based on the premise that children can freely explore and investigate the contents of the learning application. Proactive intelligent agents thus do not continually steer the child into a given direction; rather, they are exhibited when needed. The purpose of the agents is to support the child’s exploratory learning, and the development of conceptual thinking by encouraging questions and the investigation of phenomena.
 
The functionality of the learning environment has been studied in both laboratory settings and in children’s natural educational surroundings – elementary schools. The primary goal has been to enable visually impaired and blind children to use learning applications, and to provide tools with which they can take part in activities with normally seeing children. Both of these goals have, with regard to their central intentions, been achieved.
 
Source: Suomen Akatemia (Academy of Finland)