A Study of Material Sonification in Touchscreen Devices

In: arXiv preprint arXiv:1807.01106 (July 2018)
 

Abstract

Even in the digital age, designers largely rely on physical material samples to illustrate their products, as existing visual representations fail to sufficiently reproduce the look and feel of real world materials. Here, we investigate the use of interactive material sonification as an additional sensory modality for communicating well-established material qualities like softness, pleasantness or value. We developed a custom application for touchscreen devices that receives tactile input and translate it into material rubbing sound using granular synthesis. We used this system to perform a psychophysical study, in which the ability of the user to rate subjective material qualities is evaluated, with the actual material samples serving as reference stimulus. Our experimental results indicate that the considered audio cues do not significantly contribute to the perception of material qualities but are able to increase the level of immersion when interacting with digital samples.

Keywords: Digital Material Appearance, multisensory perception, sonification

Full paper version of the poster published at ISS '18

Images

Download Paper

Download Paper

Bibtex

@ARTICLE{Martin2018:arxiv,
    author = {Mart{\'i}n, Rodrigo and Weinmann, Michael and Hullin, Matthias B.},
     title = {A Study of Material Sonification in Touchscreen Devices},
   journal = {arXiv preprint arXiv:1807.01106},
      year = {2018},
     month = jul,
  keywords = {Digital Material Appearance, multisensory perception, sonification},
  abstract = {Even in the digital age, designers largely rely on physical material samples to illustrate their
              products, as existing visual representations fail to sufficiently reproduce the look and feel of
              real world materials. Here, we investigate the use of interactive material sonification as an
              additional sensory modality for communicating well-established material qualities like softness,
              pleasantness or value. We developed a custom application for touchscreen devices that receives
              tactile input and translate it into material rubbing sound using granular synthesis. We used this
              system to perform a psychophysical study, in which the ability of the user to rate subjective
              material qualities is evaluated, with the actual material samples serving as reference stimulus. Our
              experimental results indicate that the considered audio cues do not significantly contribute to the
              perception of material qualities but are able to increase the level of immersion when interacting
              with digital samples.},
       url = {http://arxiv.org/abs/1807.01106}
}