Audiovisual semantic congruency effect with onomatopoeia

Book chapter


Fidalgo, A., Takahashi, Kohshe, Murata, Aiko and Watanabe, Katsumi 2018. Audiovisual semantic congruency effect with onomatopoeia. in: Proceedings of 10th International Conference on Knowledge and Smart Technology (KST) 2018 Institute of Electrical and Electronics Engineers (IEEE). pp. 254 -259
AuthorsFidalgo, A., Takahashi, Kohshe, Murata, Aiko and Watanabe, Katsumi
Abstract

It has been reported that when a congruent natural sound precedes briefly presented visual stimuli, it promotes performance in psychophysics detection tasks. Onomatopoeias refer to words that phonetically mimic or suggest actual sounds. Onomatopoeic words are a form of sound symbolism and are frequently used in Japanese language. In this study, we examined whether the presentation of spoken Japanese onomatopoeia to Japanese native-speakers results in visual detection sensitivity changes. Results indicate that when onomatopoeias are presented 227 ms before a visual stimulus, they have a modulatory audiovisual effect. This effect is closer to the results observed with natural sounds than spoken words, with d’ being lower for onomatopoeias when compared with natural sounds. Such suggests that Japanese spoken onomatopoeias may be processed in a manner that is closer to natural sounds than spoken words and points to behavioral consequences of sound symbolism.

Book titleProceedings of 10th International Conference on Knowledge and Smart Technology (KST) 2018
Page range254 -259
Year2018
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Publication dates
Online09 Aug 2018
Publication process dates
Deposited15 Jan 2018
Accepted08 Dec 2017
Accepted08 Dec 2017
EventIEEE 10th International Conference on Knowledge and Smart Technology (KST) 2018
ISBN978-1-5386-4015-9
978-1-5386-4016-6
Digital Object Identifier (DOI)doi:10.1109/KST.2018.8426115
Web address (URL)https://doi.org/10.1109/KST.2018.8426115
Additional information

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Permalink -

https://repository.uel.ac.uk/item/84727

Accepted author manuscript

  • 8
    total views
  • 0
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Related outputs

Social Media's Impact on Creative Thought
Robinson, Lauren, Streuli, Janine and Fidalgo, A. 2019. Social Media's Impact on Creative Thought. 2019 11th International Conference on Knowledge and Smart Technology (KST).
Zero-Hour Contracts and Stress in UK Domiciliary Care Workers
Ravalier, Jermaine M, Morton, Rheanna, Russell, Lauren and Fidalgo, A. 2018. Zero-Hour Contracts and Stress in UK Domiciliary Care Workers. Health and Social Care in the Community. 27 (2), pp. 348-355.
The influence of zero-hours contracts on care worker well-being
Ravalier, J. M., Fidalgo, A., Morton, R. and Russell, L. 2017. The influence of zero-hours contracts on care worker well-being. Occupational Medicine. 67 (5), pp. 344-349.
Pain expressiveness and altruistic behavior
Williams, Amanda C de C, Gallagher, Elizabeth, Fidalgo, A. and Bentley, Peter J 2015. Pain expressiveness and altruistic behavior. PAIN. 157 (3), pp. 759-768.