Musical Vision: an interactive bio-inspired sonification tool to convert images into music

Antonio Polo, Xavier Sevillano*

*Autor corresponent d’aquest treball

Producció científica: Article en revista indexadaArticleAvaluat per experts

13 Cites (Scopus)

Resum

Musical Vision is a highly flexible, interactive and bio-inspired sonification tool that translates color images into harmonic polyphonic music by mimicking the human visual system in terms of its field of vision and photosensitive sensors. Putting the user at the center of the sonification process, Musical Vision allows the interactive design of fully configurable mappings between the color space and the MIDI instruments and audio pitch spaces to tailor the music rendering results to the application needs. Moreover, Musical Vision incorporates a harmonizer capable of introducing the necessary modifications to create melodies using harmonic chords. Above all else, Musical Vision is an extremely flexible system that the user can interactively configure to convert an image into either a few seconds or a several minutes long musical piece. Thus, it can be used, for instance, with trans-artistic purposes like the conversion of a painting into music, for augmenting vision with music, or for learning musical skills such as sol-fa. To evaluate the proposed sonification tool, we conducted a pilot user study, in which twelve volunteers were tested to interpret images containing geometric patterns from music rendered by Musical Vision. Results show that even those users with no musical education background were able to achieve nearly 70% accuracy in multiple choice tests after less than 25 min training. Moreover, users with some musical education were capable of accurately “drawing by ear” the images from no other stimuli than the sonifications.

Idioma originalAnglès
Pàgines (de-a)231-243
Nombre de pàgines13
RevistaJournal on Multimodal User Interfaces
Volum13
Número3
DOIs
Estat de la publicacióPublicada - 1 de set. 2019

Fingerprint

Navegar pels temes de recerca de 'Musical Vision: an interactive bio-inspired sonification tool to convert images into music'. Junts formen un fingerprint únic.

Com citar-ho