Large databases need small, open-weight language models
Key system optimizations required to efficiently deploy open-weight models within an LM-DB system.
Language model systems built around proprietary APIs often operate on a token-based cost model. In the context of large databases, running LM-enhanced relational operators through commercial APIs becomes prohibitively expensive, hindering both thorough research and practical deployment. In this paper we demonstrate that quantized, open-weight models running locally on just 16GB of VRAM can match or exceed the accuracy of closed-source counterparts at lower latency and a fraction of the price, challenging the prevailing assumption that closed-source LM APIs are necessary for effective LM-database integration. We present and analyze the key system optimizations required to efficiently deploy these open-weight models within an LM-DB system. By integrating these local models into the BlendSQL framework, we demonstrate a 342x reduction in overall costs and 4.8x reduction in latency.
Latest publications
QDBO: A real-time quantum-augmented database system optimizer
A white-box quantum-augmented database optimizer that directly leverages low-level quantum sampling on a quantum annealer.
VLDBRECAP: Regression evaluation for continual adaptation of prompts
A benchmark that measures continual-learning phenomena at the constraint level for prompt-level adaptation methods.
EMNLPMEMGUARD: Preventing memory contamination in long-term memory-augmented large language models
A type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.
EMNLP