Influence functions for efficient data selection in reasoning
A proposal to define reasoning data quality using influence functions.
Fine-tuning large language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive datasets. Yet, what constitutes “quality” remains ill-defined. Existing reasoning methods rely on indirect heuristics such as problem difficulty or trace length, while instruction tuning has explored a broader range of automated selection strategies—but rarely in the context of reasoning. We propose to define reasoning data quality using influence functions, which measure the causal effect of individual CoT examples on downstream accuracy, and introduce influence-based pruning, which consistently outperforms perplexity and embedding-based baselines on math reasoning within a model family.
Latest publications
Leveraging parameter space symmetries for reasoning skill transfer in LLMs
Utilizing an alignment-first strategy to transfer advanced reasoning skills to a non-reasoning model.
NeurIPSGPO: learning from critical steps to improve LLM reasoning
A novel fine-tuning strategy designed to improve LLM multi-step reasoning capabilities by focusing on pivotal moments.
NeurIPSViCrit: a verifiable reinforcement learning proxy task for visual perception in VLMs
An RL proxy task that trains VLMs to localize synthetic hallucinations injected into human-written captions.
NeurIPS