Audiovisual semantic congruency effect with onomatopoeia

Book chapter


Fidalgo, A., Takahashi, Kohshe, Murata, Aiko and Watanabe, Katsumi 2018. Audiovisual semantic congruency effect with onomatopoeia. in: Proceedings of 10th International Conference on Knowledge and Smart Technology (KST) 2018 IEEE. pp. 254 -259
AuthorsFidalgo, A., Takahashi, Kohshe, Murata, Aiko and Watanabe, Katsumi
Abstract

It has been reported that when a congruent natural sound precedes briefly presented visual stimuli, it promotes performance in psychophysics detection tasks. Onomatopoeias refer to words that phonetically mimic or suggest actual sounds. Onomatopoeic words are a form of sound symbolism and are frequently used in Japanese language. In this study, we examined whether the presentation of spoken Japanese onomatopoeia to Japanese native-speakers results in visual detection sensitivity changes. Results indicate that when onomatopoeias are presented 227 ms before a visual stimulus, they have a modulatory audiovisual effect. This effect is closer to the results observed with natural sounds than spoken words, with d’ being lower for onomatopoeias when compared with natural sounds. Such suggests that Japanese spoken onomatopoeias may be processed in a manner that is closer to natural sounds than spoken words and points to behavioral consequences of sound symbolism.

Book titleProceedings of 10th International Conference on Knowledge and Smart Technology (KST) 2018
Page range254 -259
Year2018
PublisherIEEE
Publication dates
Online09 Aug 2018
Publication process dates
Deposited15 Jan 2018
Accepted08 Dec 2017
Accepted08 Dec 2017
EventIEEE 10th International Conference on Knowledge and Smart Technology (KST) 2018
ISBN978-1-5386-4015-9
978-1-5386-4016-6
Digital Object Identifier (DOI)https://doi.org/10.1109/KST.2018.8426115
Web address (URL)https://doi.org/10.1109/KST.2018.8426115
Additional information

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Accepted author manuscript
Permalink -

https://repository.uel.ac.uk/item/84727

Download files

  • 94
    total views
  • 88
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Export as

Related outputs

Acute impact of a national lockdown during the COVID-19 pandemic on wellbeing outcomes among individuals with chronic pain
Zambelli, Z., Fidalgo, A. R., Halstead, E. J. and Dimitriou, D. 2022. Acute impact of a national lockdown during the COVID-19 pandemic on wellbeing outcomes among individuals with chronic pain. Journal of Health Psychology. 27 (5), pp. 1099-1110. https://doi.org/10.1177/1359105321995962
Good Sleep Quality Improves the Relationship Between Pain and Depression Among Individuals With Chronic Pain
Zambelli, Z., Halstead, E. J., Fidalgo, A. and Dimitriou, D. 2021. Good Sleep Quality Improves the Relationship Between Pain and Depression Among Individuals With Chronic Pain. Frontiers in Psychology. 12 (Art. 668930). https://doi.org/10.3389/fpsyg.2021.668930
Social Media's Impact on Creative Thought
Robinson, Lauren, Streuli, Janine and Fidalgo, A. 2019. Social Media's Impact on Creative Thought. 2019 11th International Conference on Knowledge and Smart Technology (KST). https://doi.org/10.1109/KST.2019.8687632
Zero-Hour Contracts and Stress in UK Domiciliary Care Workers
Ravalier, Jermaine M, Morton, Rheanna, Russell, Lauren and Fidalgo, A. 2018. Zero-Hour Contracts and Stress in UK Domiciliary Care Workers. Health and Social Care in the Community. 27 (2), pp. 348-355. https://doi.org/10.1111/hsc.12652
The influence of zero-hours contracts on care worker well-being
Ravalier, J. M., Fidalgo, A., Morton, R. and Russell, L. 2017. The influence of zero-hours contracts on care worker well-being. Occupational Medicine. 67 (5), pp. 344-349. https://doi.org/10.1093/occmed/kqx043
Pain expressiveness and altruistic behavior
Williams, Amanda C de C, Gallagher, Elizabeth, Fidalgo, A. and Bentley, Peter J 2015. Pain expressiveness and altruistic behavior. PAIN. 157 (3), pp. 759-768. https://doi.org/10.1097/j.pain.0000000000000443