Play by the type rules: inferring constraints for small language models in declarative programs

Integrating language model powered operators in declarative query languages allows for the combination of cheap and interpretable functions with powerful, generalizable reasoning. However, in order to benefit from the optimized execution of a database query language like SQL, generated outputs must align with the rules enforced by both type checkers and database contents. Current approaches address this challenge with orchestrations consisting of many LLM-based post-processing calls to ensure alignment between generated outputs and database values, introducing performance bottlenecks. We perform a study on the ability of various sized open-source language models to both parse and execute functions within a query language based on SQL, showing that small language models can excel as function executors over hybrid data sources. Then, we propose an efficient solution to enforce the well-typedness of language model functions, demonstrating 7% accuracy improvement on a multi-hop question answering dataset.