Tacit knowledge for explaining how large language models work

In this talk, I will explore how we can use tacit knowledge to explain the behavior of current large language models. In particular, I argue that large language models might learn something beyond just predicting the next word during training. One way to determine what these models learn is to look at their model internals. However, due to the black box problem it is not straightforward to interpret these parameters. Therefore, I propose that we can conceptualize waht LLMs learn as tacit knowledge as proposed by Martin Davies. This account of tacit knowledge introduces a particular causal structure that would support attribution of some kind of knowledge. In the talk, I first introduce the account in more detail and discuss some potential issues when applying this to connectionist neural networks. Then, I argue that we can nevertheless apply this account for explaining current LLMs and illustrate this using an example, namely ROME by Meng and colleagues (2022). I end with some recommendations for further research.

Slides can be found here.