Large databases need small, open-weight language models

Language model systems built around proprietary APIs often operate on a token-based cost model. In the context of large databases, running LM-enhanced relational operators through commercial APIs becomes prohibitively expensive, hindering both thorough research and practical deployment. In this paper we demonstrate that quantized, open-weight models running locally on just 16GB of VRAM can match or exceed the accuracy of closed-source counterparts at lower latency and a fraction of the price, challenging the prevailing assumption that closed-source LM APIs are necessary for effective LM-database integration. We present and analyze the key system optimizations required to efficiently deploy these open-weight models within an LM-DB system. By integrating these local models into the BlendSQL framework, we demonstrate a 342x reduction in overall costs and 4.8x reduction in latency.