Bridging Language in Machines with Language in the Brain

Several major innovations in artificial intelligence (AI) (e.g. convolutional neural networks, experience replay) are based on findings about the brain. However, the underlying brain findings took many years to first consolidate and many more to transfer to AI. Moreover, these findings were made using invasive methods in non human species. For brain functions that are uniquely human, such as understanding complex language, there is no suitable animal that can serve as a model organism and thus a mechanistic understanding is that much farther away.

In this dissertation, we present a data-driven framework that circumvents these limitations by establishing a direct connection between brain recordings of people comprehending language and natural language processing (NLP) computer systems. We present evidence that this connection can be beneficial for both neurolinguistics and NLP. Specifically, we show that this framework can utilize recent successes in neural networks for NLP to enable scientific discovery about context- and task-dependent meaning composition in the brain, and we present the first evidence that brain activity measurements of people reading can be used to improve the generalization performance of a popular deep neural network language model. These investigations also contribute advances in cognitive modeling that may be useful beyond the study of language. In short, this dissertation involves multidisciplinary investigations and makes contributions to cognitive neuroscience, neurolinguistics, and natural language processing.