Evaluating the Effects of Material Sonification in Tactile Devices

Submitted at 6th July 2018, July 2018
 

Abstract

Since the integration of internet of things technologies in our daily lives, there has been an increasing demand for new ways to interact with commodities in the domain of e-commerce, that go beyond passive product visualization. When considering materials from retailing stores, the utilization of audio cues has proven to be a simple but effective way to enhance not only the estimation of well-established physical qualities (e.g. roughness, flexibility), but also affective properties (e.g. pleasantness, value), which have an important leverage in the user decision for or against a product. In this paper, we propose to investigate augmenting visual representations of leather and fabric materials with touch-related audio feedback generated when rubbing the fingertip against the samples. For this purpose, we developed an interactive material sonification system for tactile devices that allows evaluating the impact of such audio cues on the human perception of materials by means of a psychophysical study based on rating scales for a set of relevant physical and affective material qualities. Our experimental results indicate that the evaluated touch-related audio cues do not significantly contribute to the perception of these attributes for the considered materials. In light of these findings, we suggest complementary directions of research in interactive material sonification, which may lead to profitable results.

Keywords: Digital Material Appearance, multisensory perception., sonification

Images

Download Paper

Download Paper

Bibtex

@UNPUBLISHED{Martin2018:b,
    author = {Mart{\'i}n, Rodrigo and Weinmann, Michael and Hullin, Matthias B.},
     title = {Evaluating the Effects of Material Sonification in Tactile Devices},
      year = {2018},
     month = jul,
      note = {Submitted at 6th July 2018},
  keywords = {Digital Material Appearance, multisensory perception., sonification},
  abstract = {Since the integration of internet of things technologies in our daily lives, there has been an
              increasing demand for new ways to interact with commodities in the domain of e-commerce, that go
              beyond passive product visualization. When considering materials from retailing stores, the
              utilization of audio cues has proven to be a simple but effective way to enhance not only the
              estimation of well-established physical qualities (e.g. roughness, flexibility), but also affective
              properties (e.g. pleasantness, value), which have an important leverage in the user decision for or
              against a product. In this paper, we propose to investigate augmenting visual representations of
              leather and fabric materials with touch-related audio feedback generated when rubbing the fingertip
              against the samples. For this purpose, we developed an interactive material sonification system for
              tactile devices that allows evaluating the impact of such audio cues on the human perception of
              materials by means of a psychophysical study based on rating scales for a set of relevant physical
              and affective material qualities. Our experimental results indicate that the evaluated touch-related
              audio cues do not significantly contribute to the perception of these attributes for the considered
              materials. In light of these findings, we suggest complementary directions of research in
              interactive material sonification, which may lead to profitable results.},
       url = {http://arxiv.org/abs/1807.01106}
}