SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for efficient financial LLM adaptation

Large language models (LLMs) adapted to financial domains often suffer from catastrophic forgetting of general reasoning capabilities essential for customer interactions and complex financial analysis. We introduce Selective Parameter Evaluation and Restoration via Model Merging (SPEAR-MM), a practical framework that preserves critical general skills while enabling domain adaptation. Our method approximates layer-wise impact on external benchmarks through post-hoc analysis, then selectively freezes or restores transformer layers via spherical interpolation merging. Applied to LLaMA-3.1-8B for financial tasks, SPEAR-MM maintains performance on public benchmarks (ARC, GSM8K, MMLU-Pro, GPQA) within 1\% of baseline while achieving substantial improvements on financial customer support QA and conversation summarization. The approach reduces computational requirements by 60\% compared to full continual pretraining, making it particularly valuable for resource-constrained financial institutions. Our framework provides a tunable trade-off between knowledge retention and domain adaptation, crucial for safety-critical financial applications.