Denitsa Saynova defends her doctoral thesis, “Fact and Ideology in the Machine: Modelling Knowledge and Belief in Neural Models from Text” on September 17 at Chalmers University of Technology.
This thesis explores questions of knowledge, language, and neural network models. Motivated by an increasing need for insight into complex political and social science phenomena, we study how methods within natural language processing (NLP) can help us gain such insight. With a particular focus on a model’s knowledge, how it is structured, and how we can access and assess it, we study two important aspects of NLP models.
First, we investigate their capabilities and limitations, focusing on how they can capture political and social signals. We use embedding models to capture and reveal distinctions in policy and ideology in Swedish political parties, discussing the strengths and drawbacks of the approach. We also investigate the presence of more complex social knowledge in large pre-trained language models. We prompt models to produce synthetic samples of responses to social science experiments and access if effects calculated from the synthetic data can be used to predict a study’s replicability. A central limitation we find in these studies is the lack of robustness, which we explore in depth by studying what influences model consistency in a more simplified setting, namely, recalling facts.
Second, we aim to bridge the gap between the model and the domain expert by developing and improving interpretability insights of model behaviour. We develop a method for aggregating class-level explanations for a text classifier and demonstrate its utility in the context of Swedish political texts. We also develop the understanding of how models store and access factual information. We propose a taxonomy of possible language model behaviours for fact completion and, based on our novel testing data set, examine internal knowledge structures using established mechanistic interpretability methods.
