A Study of Material Sonification in Touchscreen Devices

In proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces, Tokyo, Japan, pages 305-310, ACM, 2018
 

Abstract

Even in the digital age, designers largely rely on physical material samples to illustrate their products, as existing visual representations fail to sufficiently reproduce the look and feel of real world materials. Here, we investigate the use of interactive material sonification as an additional sensory modality for communicating well-established material qualities like softness, pleasantness or value. We developed a custom application for touchscreen devices that receives tactile input and translate it into material rubbing sound using granular synthesis. We used this system to perform a psychophysical study, in which the ability of the user to rate subjective material qualities is evaluated, with the actual material samples serving as reference stimulus.

Keywords: Digital Material Appearance, multisensory perception, sonification

Images

Download Paper

Download Paper

Bibtex

@INPROCEEDINGS{Martin2018:iss,
     author = {Mart{\'i}n, Rodrigo and Weinmann, Michael and Hullin, Matthias B.},
      pages = {305--310},
      title = {A Study of Material Sonification in Touchscreen Devices},
    journal = {arXiv preprint arXiv:1807.01106},
  booktitle = {the 2018 ACM International Conference on Interactive Surfaces and Spaces},
     series = {ISS '18},
       year = {2018},
  publisher = {ACM},
   location = {Tokyo, Japan},
    address = {New York, NY, USA},
       note = {Best Poster Award},
   keywords = {Digital Material Appearance, multisensory perception, sonification},
   abstract = {Even in the digital age, designers largely rely on physical material samples to illustrate their
               products, as existing visual representations fail to sufficiently reproduce the look and feel of
               real world materials. Here, we investigate the use of interactive material sonification as an
               additional sensory modality for communicating well-established material qualities like softness,
               pleasantness or value. We developed a custom application for touchscreen devices that receives
               tactile input and translate it into material rubbing sound using granular synthesis. We used this
               system to perform a psychophysical study, in which the ability of the user to rate subjective
               material qualities is evaluated, with the actual material samples serving as reference stimulus.},
       isbn = {978-1-4503-5694-7},
        url = {http://doi.acm.org/10.1145/3279778.3281455},
        doi = {10.1145/3279778.3281455}
}