Cross-Lingual Semantic Alignment in Large Language Models via Context-Aware Training

This paper introduces Context-Aware Cross-Modal Alignment Training (CACMAT), a novel multi-stage training paradigm to enhance translation capabilities of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs). Current LLM translation models often struggle with contextual nuances and cross-lingual semantic alignment. CACMAT addresses this by incorporating three stages: secondary pre-training on target language monolingual data, continual pre-training with a contextual contrastive loss using Interlinear Text Format (ITF) data to improve cross-lingual alignment, and supervised fine-tuning on parallel translation datasets. Experiments on FLORES-200 and WMT datasets demonstrate that CACMAT significantly outperforms baseline models and achieves competitive results against state-of-the-art systems, as validated by both BLEU scores and human evaluations. Ablation studies confirm the crucial role of the contextual contrastive alignment stage. The results highlight CACMAT as an effective approach for improving translation quality by explicitly enhancing cross-lingual and cross-modal semantic alignment in LLMs and LVLMs.