This paper explores why language models exhibit less surprisal than humans when processing syntactically ambiguous sentences. It tests the hypothesis that LMs can simultaneously consider a greater number of sentence interpretations using Recurrent Neural Network Grammars.
The article describes how the Novellum interactive fiction player overcame the challenge of streaming LLM responses that combine dialogue with structured metadata. To avoid raw tag display or long loading times, they developed a structured stream parser that emits dialogue instantly while processing other tags appropriately.
This content, titled "Parsing Inside-Out," likely delves into specific methodologies or perspectives on parsing techniques. It explores their foundational role and applications within artificial intelligence, particularly concerning language understanding and data interpretation.