Dialogue distillery: Crafting interpolable, interpretable, and introspectable dialogue from LLMs

We present the third iteration of Chirpy Cardinal, an open-domain dialogue agent developed for the Alexa Prize Socialbot Grand Challenge 5 (SGC5) competition. In 2023, while pure large language model (LLM) chatbots have superior breadth, flexibility, and fluency compared to traditional dialogue trees, they have high latency and are not factual, controllable, or maintainable, limiting their effectiveness as dialogue systems. To address these weaknesses while preserving the strengths of a neural system, we propose a chatbot architecture that involves distilling a language model into a symbolic structure. Toward this end, we propose the following innovations: (a) CAMEL , a domain-specific language and graphical interface for chatbot development, (b) Goldfinch , which automatically generates nodes in a conversational graph using an LLM, and (c) Kingfisher , which reduces latency and enables use of larger neural models by predicting future user utterances and precomputing responses to them. In deployment, we find that these methods substantially improve conversation quality. We release our open-source code and hope it serves as a useful toolbox for custom chatbot development.