Countering Bias in Al Methods in the Social Sciences

Artificial intelligence (Al) techniques for text have resulted in some of the most spectacular Al showcases recently. How does this shift affect research in applied fields such as political or social science? What are the new lines of research enabled by these new technologies, and conversely what are the limitations?

In this project, we will investigate text-based Al techniques in the type of research method called causal inference, where researchers investigate cause-and-effect questions such as “What is the effect of an IMF program on poverty?” or “What is the effect of changing the party leader on a party’s polling?” Recently, researchers have proposed methods for exploiting Al when carrying out such investigations, allowing them to utilize text as a research material that might otherwise be difficult to use. For instance, for the investigations mentioned above we might consider texts describing IMF programs, or publications by the parties.

However, previous research by the participants in this project and others has shown that a naive use of text­based Al risks introducing biases skewing the estimates of causal effects. The consequences for society of drawing incorrect conclusions about the effects of policies are obvious and may lead to the implementation of harmful policies.

In the project, we will investigate the challenges of bias when applying text-based Al in causal inference applied to the social and political sciences. We will investigate to what extent these problems can be measured, so that we could know if we are on thin ice when applying the new methods. Furthermore, we will try to “correct” the biases and make the text-based Al more robust when applied in research. On the applied side, we are going to apply these Al methods for the questions exemplified above: effects of IMF programs on poverty, and effects of political events on polling.