TY - BOOK

T1 - Combining speech with sight and touch

T2 - investigating the benefits of audio-visual and audio-tactile speech on speech intelligibility and cortical speech-envelope tracking accuracy

AU - O'Hanlon, Brandon

PY - 2025

Y1 - 2025

N2 - Speech plays a critical role in communication in our everyday lives. Understanding how speech is processed in the brain, from mechanoelectrical transduction in the ear to processing in the subcortical and cortical regions of the auditory pathway, provides insight into how we may improve speech perception during difficult listening conditions which may hinder us. As discussed in Chapter 1, multisensory integration is one such method in which we may restore intelligibility to speech-in-noise. This thesis aims to contribute to the current multisensory speech integration literature through new investigations into audio-visual and audio-tactile speech and their benefits to cortical speech-envelope tracking in the auditory cortex and speech intelligibility.Chapter 2 reassessed the audio-visual benefits to speech perception in-noise when phoneme stimuli are selected from different visually distinct viseme categories. It found that, when visemes are considered, the benefits of audio-visual integration were reduced compared to previous literature, with some phonemes providing more benefit to intelligibility than others. Chapters 3 and 4 explored potential benefits of both bottom-up and top-down short-term audio-tactile training on cortical speech-envelope tracking accuracy and speech intelligibility. Within these chapters, findings indicated an initial enhancement on neural tracking accuracy but with no associated benefit to speech intelligibility. This unexpected outcome was further investigated with Chapter 5, which returned to audio-visual speech to examine the potential role of neural tracking in the prediction of oncoming speech.In all, this thesis provided evidence of multisensory benefits to speech perception, especially in difficult listening conditions. The thesis also contributes to the growingliterature around neural tracking in the brain, with further evidence to suggest that tracking accuracy is not intrinsically linked to intelligibility. Chapter 6 discusses these findings, alongside future directions, to finalise this thesis.

AB - Speech plays a critical role in communication in our everyday lives. Understanding how speech is processed in the brain, from mechanoelectrical transduction in the ear to processing in the subcortical and cortical regions of the auditory pathway, provides insight into how we may improve speech perception during difficult listening conditions which may hinder us. As discussed in Chapter 1, multisensory integration is one such method in which we may restore intelligibility to speech-in-noise. This thesis aims to contribute to the current multisensory speech integration literature through new investigations into audio-visual and audio-tactile speech and their benefits to cortical speech-envelope tracking in the auditory cortex and speech intelligibility.Chapter 2 reassessed the audio-visual benefits to speech perception in-noise when phoneme stimuli are selected from different visually distinct viseme categories. It found that, when visemes are considered, the benefits of audio-visual integration were reduced compared to previous literature, with some phonemes providing more benefit to intelligibility than others. Chapters 3 and 4 explored potential benefits of both bottom-up and top-down short-term audio-tactile training on cortical speech-envelope tracking accuracy and speech intelligibility. Within these chapters, findings indicated an initial enhancement on neural tracking accuracy but with no associated benefit to speech intelligibility. This unexpected outcome was further investigated with Chapter 5, which returned to audio-visual speech to examine the potential role of neural tracking in the prediction of oncoming speech.In all, this thesis provided evidence of multisensory benefits to speech perception, especially in difficult listening conditions. The thesis also contributes to the growingliterature around neural tracking in the brain, with further evidence to suggest that tracking accuracy is not intrinsically linked to intelligibility. Chapter 6 discusses these findings, alongside future directions, to finalise this thesis.

U2 - 10.17635/lancaster/thesis/2769

DO - 10.17635/lancaster/thesis/2769

M3 - Doctoral Thesis

PB - Lancaster University

ER -