Beyond Text: Inferring Fedspeak Policy Stance via Multimodal Cues and Agentic Historical Memory

The interpretation of Federal Reserve communications (Fedspeak) is critical for financial forecasting and market stability. While recent Large Language Model (LLM) based approaches have achieved significant progress by incorporating economic reasoning paths, they predominantly rely on textual data, overlooking the rich paralinguistic cues present in press conferences. Furthermore, existing models often lack dynamic access to historical context and domain-specific knowledge. In this paper, we propose a novel multimodal alignment and retrieval-augmented framework for policy stance analysis of the Federal Open Market Committee (FOMC). Our approach introduces a robust pipeline to align video timestamps with textual transcripts, enabling the extraction of audio-modality features such as pitch, rhythm, and energy. We further enhance the model via a multi-stage data augmentation process that injects structured domain knowledge via a financial knowledge graph, utilizes an agentic retrieval system for historical consistency, and incorporates audio modality to capture non-verbal signals. By fine-tuning a foundation model on this augmented dataset, our method significantly outperforms text-only baselines, achieving state-of-the-art accuracy on the FOMC benchmark as demonstrated by extensive experiments.