From Aligned Embeddings to Instruction-Following Perception: A Survey of Vision–Language Model Development

Recognizing an object from a text label, describing an unfamiliar scene, and following an instruction about a small image region require different forms of visual-language correspondence. The development of vision-language models can be understood through the effort to support these increasingly varied outputs. This survey follows the transition from contrastive image-text representations to captioning and grounding systems, and then to instruction-tuned multimodal language models. Within this account, we examine how pretraining objectives, web-data curation, visual connectors, and adaptation methods affect the tasks a model can perform. Particular attention is given to novel object captioning and dense prediction: both expose information that can be lost when image-level alignment is used as the starting point for a more specific task. Relative contrastive alignment [Fan et al., 2023] and uncertainty-guided cross-head co-training [Fan et al., 2022] offer complementary case studies of supervision quality. Evaluation and hallucination are considered alongside the capabilities they are meant to measure. The review brings together architectural comparisons, task-oriented evidence, and unresolved questions about visual resolution, grounding, and benchmark validity. Its central concern is how a system’s training signal and representation support the visual claims it ultimately makes.