Understanding Large Language Models via a TCS Lens

Large language models have transformed text generation and are already being used across domains ranging from information retrieval to scientific discovery. This progress has sparked a growing body of theoretical work in the TCS community that aims to understand the mathematical foundations of language generation and modern language models.

The theoretical study of language is not new: it connects to classic ideas from computation, information theory, and formal language theory. Today, researchers are bringing tools from learning theory, automata theory, computational complexity, communication complexity, and algorithm design, to bear on the modern landscape of LLMs.

The goal is twofold: to give STOC attendees an accessible entry point into these emerging theoretical frameworks, and to connect researchers who are approaching the foundations of LLMs from different angles.