Final published version
Licence: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - On the similarities of representations in artificial and brain neural networks for speech recognition
AU - Wingfield, Cai
AU - Zhang, Chao
AU - Devereux, Barry
AU - Fonteneau, Elisabeth
AU - Thwaites, Andrew
AU - Liu, Xunying
AU - Woodland, Phil
AU - Marslen-Wilson, William
AU - Su, Li
PY - 2022/12/21
Y1 - 2022/12/21
N2 - Introduction: In recent years, machines powered by deep learning have achieved near-human levels of performance in speech recognition. The fields of artificial intelligence and cognitive neuroscience have finally reached a similar level of performance, despite their huge differences in implementation, and so deep learning models can—in principle—serve as candidates for mechanistic models of the human auditory system. Methods: Utilizing high-performance automatic speech recognition systems, and advanced non-invasive human neuroimaging technology such as magnetoencephalography and multivariate pattern-information analysis, the current study aimed to relate machine-learned representations of speech to recorded human brain representations of the same speech. Results: In one direction, we found a quasi-hierarchical functional organization in human auditory cortex qualitatively matched with the hidden layers of deep artificial neural networks trained as part of an automatic speech recognizer. In the reverse direction, we modified the hidden layer organization of the artificial neural network based on neural activation patterns in human brains. The result was a substantial improvement in word recognition accuracy and learned speech representations. Discussion: We have demonstrated that artificial and brain neural networks can be mutually informative in the domain of speech recognition.
AB - Introduction: In recent years, machines powered by deep learning have achieved near-human levels of performance in speech recognition. The fields of artificial intelligence and cognitive neuroscience have finally reached a similar level of performance, despite their huge differences in implementation, and so deep learning models can—in principle—serve as candidates for mechanistic models of the human auditory system. Methods: Utilizing high-performance automatic speech recognition systems, and advanced non-invasive human neuroimaging technology such as magnetoencephalography and multivariate pattern-information analysis, the current study aimed to relate machine-learned representations of speech to recorded human brain representations of the same speech. Results: In one direction, we found a quasi-hierarchical functional organization in human auditory cortex qualitatively matched with the hidden layers of deep artificial neural networks trained as part of an automatic speech recognizer. In the reverse direction, we modified the hidden layer organization of the artificial neural network based on neural activation patterns in human brains. The result was a substantial improvement in word recognition accuracy and learned speech representations. Discussion: We have demonstrated that artificial and brain neural networks can be mutually informative in the domain of speech recognition.
KW - Neuroscience
KW - automatic speech recognition
KW - deep neural network
KW - representational similarity analysis
KW - auditory cortex
KW - speech recognition
U2 - 10.3389/fncom.2022.1057439
DO - 10.3389/fncom.2022.1057439
M3 - Journal article
VL - 16
JO - Frontiers in Computational Neuroscience
JF - Frontiers in Computational Neuroscience
SN - 1662-5188
M1 - 1057439
ER -