Panda: Performance debugging for databases using LLM agents

Debugging a performance issue in databases is notoriously hard. Wouldn’t it be convenient if there exists an oracle or a co-pilot for every database system which users can query in natural language (NL) — ‘what’s wrong?’, or even better— ‘How to fix it?’. Large Language Models (LLMs), like ChatGPT, seem to be a natural surrogate to this oracle given their ability to answer a wide range of questions by efficiently encoding vast amount of knowledge for e.g., a major chunk of the internet. However, prompting ChatGPT with database performance queries often results in ‘technically correct’ but highly ‘vague’ or ‘generic’ recommendations typically rendered useless and untrustworthy by experienced Database Engineers (DBEs).

In this work we propose Panda, a framework to provide context grounding to pre-trained LLMs in order to generate more ‘useful’ and ‘in-context’ troubleshooting recommendations. Panda draws inspiration from the way experienced DBEs perform debugging, and puts a system in place with necessary components required to robustly deploy pre-trained LLMs in production for debugging. The 4 key components of Panda are: (1) Grounding; (2) Verification; (3) Affordance; and (4) Feedback. We describe the necessity and usefulness of each component and how they communicate internally to transform a given pre-trained LLM into generating in-context, actionable, useful and accurate recommendation for debugging a given database system.