Musical Vision: an interactive bio-inspired sonification tool to convert images into music

Antonio Polo, Xavier Sevillano

Producción científica: Artículo en revista indizadaArtículorevisión exhaustiva

12 Citas (Scopus)

Resumen

Musical Vision is a highly flexible, interactive and bio-inspired sonification tool that translates color images into harmonic polyphonic music by mimicking the human visual system in terms of its field of vision and photosensitive sensors. Putting the user at the center of the sonification process, Musical Vision allows the interactive design of fully configurable mappings between the color space and the MIDI instruments and audio pitch spaces to tailor the music rendering results to the application needs. Moreover, Musical Vision incorporates a harmonizer capable of introducing the necessary modifications to create melodies using harmonic chords. Above all else, Musical Vision is an extremely flexible system that the user can interactively configure to convert an image into either a few seconds or a several minutes long musical piece. Thus, it can be used, for instance, with trans-artistic purposes like the conversion of a painting into music, for augmenting vision with music, or for learning musical skills such as sol-fa. To evaluate the proposed sonification tool, we conducted a pilot user study, in which twelve volunteers were tested to interpret images containing geometric patterns from music rendered by Musical Vision. Results show that even those users with no musical education background were able to achieve nearly 70% accuracy in multiple choice tests after less than 25 min training. Moreover, users with some musical education were capable of accurately “drawing by ear” the images from no other stimuli than the sonifications.

Idioma originalInglés
Páginas (desde-hasta)231-243
Número de páginas13
PublicaciónJournal on Multimodal User Interfaces
Volumen13
N.º3
DOI
EstadoPublicada - 1 sept 2019

Huella

Profundice en los temas de investigación de 'Musical Vision: an interactive bio-inspired sonification tool to convert images into music'. En conjunto forman una huella única.

Citar esto