Language-Driven Image Restoration and Semantic-Aware Quality Assessment: A Survey

1. Introduction

Image restoration (IR), as a fundamental problem in low-level computer vision, aims to recover high-quality images from degraded counterparts. Over the past decades, numerous IR methods have been extensively studied for a wide range of tasks, including denoising [1,2,3], deraining [4,5,6], dehazing [7,8,9], desnowing [10,11,12], deblurring [13,14,15], low-light enhancement (LLIE) [16,17,18], super-resolution [19,20,21]. Beyond these tasks, IR techniques have also been widely applied to domain-specific scenarios, such as underwater enhancement [22,23,24], medical IR [25,26].

IR methodologies have evolved from model-driven approaches based on handcrafted priors to data-driven paradigms powered by deep neural networks, including CNNs [27], Transformers [28], and more recent architectures [29]. In parallel, restoration frameworks have progressed from task-specific designs, where each degradation type is modeled independently [5,9,30], to unified frameworks such as all-in-one (AiO) restoration, which aim to handle multiple degradation types within a single model [31,32,33,34].

Despite these advances, existing methods remain largely constrained by predefined degradation assumptions and are typically optimized with pixel-level supervision, limiting their ability to generalize to complex or user-dependent restoration scenarios. These limitations motivate the exploration of new paradigms beyond conventional restoration frameworks.

Recent breakthroughs in multimodal large language models (MLLMs) and vision–language models (VLMs) have opened new opportunities to address these challenges. By encoding rich semantic priors and exhibiting strong reasoning capabilities beyond conventional visual representations, MLLMs and VLMs have been increasingly introduced into IR pipelines [34,35,36,37]. Rather than directly performing pixel-level reconstruction, these models are typically employed as auxiliary components to provide high-level information, such as degradation interpretation, semantic guidance, and adaptive control. This emerging paradigm fundamentally reshapes IR from a purely visual mapping problem into a multimodal, semantically informed, and interactive framework.

This paradigm shift introduces a critical challenge in image quality assessment (IQA). Traditional IQA metrics [38,39,40] are effective for measuring pixel-level fidelity or perceptual similarity. However, they are not designed to assess semantic alignment or instruction consistency, leading to a mismatch between the restoration targets and the evaluation criteria. Recent language-driven IQA methods [41,42,43,44,45,46,47] address this gap by leveraging multimodal representations to assess semantic coherence and cross-modal consistency.

Although integrating language for IR and IQA has developed rapidly [34,37,48], a comprehensive survey of this emerging field remains absent, to the best of our knowledge. Existing surveys on IR primarily focus on architectural designs [49] or task-specific learning strategies [4,31,50], while recent reviews on multimodal models [51,52] seldom address low-level vision problems in depth.

In this survey, we provide the first systematic review of VLM-/MLLM-based image restoration and language-driven IQA. Figure 1 summarizes the taxonomy of this survey in a hierarchically structured way. We review advances in language-driven restoration frameworks, summarize representative model designs and training strategies, and discuss open challenges and potential future research directions.

The main contributions of this survey are summarized as follows:

• We introduce a unified conceptual framework that interprets language-driven IR as an interaction-centric paradigm, revealing how language models impact restoration behavior beyond architectural modifications.
• We systematically categorize VLM-/MLLM-based IR methods through a functional role-oriented taxonomy, clarifying distinct interaction mechanisms including representation-level integration, optimization-level coupling, decision-driven control, and cross-level paradigms.
• We critically analyze VLM-/MLLM-based IQA, highlighting its conceptual distinctions from conventional fidelity metrics and clarifying challenges related to evaluation reliability, calibration stability, and semantic bias.
• We summarize the restoration datasets used in VLM-/MLLM-based frameworks and analyze their limitations and emerging requirements from a language-driven perspective, emphasizing the need for semantically enriched, language-aware benchmarks. We also provide comparisons between conventional frameworks and VLM-/MLLM-based methods across different settings.
• We investigate open challenges posed by language-integrated restoration systems and outline promising directions for future research that bridge multimodal reasoning, visual perception, and restoration optimization.

The rest of the work is organized as follows. Section 2 introduces the necessary preliminaries, including IR fundamentals, multimodal language models, and IQA, providing an overview of the core concepts relevant to this work. Section 3 reviews representative VLM-/MLLM-based IR approaches, and the analysis is guided by the proposed interaction-centric taxonomy with a focus on their model architectures and key technical innovations. Section 4 summarizes commonly used datasets for different IR tasks and discusses corresponding evaluation metrics, covering both traditional criteria and recent VLM-/MLLM-based assessment methods. Finally, Section 5 analyzes open challenges in current studies and outlines potential solutions and future research directions.

2. Background

In this section, we introduce the fundamental concepts and taxonomy of VLM-/MLLM-based image restoration, aiming to provide a structured background for understanding recent language-driven restoration frameworks.

2.1. Image Restoration

Image restoration aims to recover a clean image $\mathcal{I}\left(x\right)$ from its degraded observation $\mathcal{D}\left(x\right)$ by mitigating various types of distortions:

$$\mathcal{D}\left(x\right)=H\left(\mathcal{I}\right(x\left)\right)+\mathcal{N},$$

(1)

where $H(·)$ denotes the degradation operator and $\mathcal{N}$ represents additive noise.

Figure 2. Examples of different degradation types and domains. The images correspond to the following open-source datasets: (a) PolyU [53], (b) LHP [54], (c) NH-HAZE [55], (d) RealSnow10K [56], (e) DPDblur [14], (f) LSRW [57], (g) DIV4K50 [58], (h) Squid [59], (i) FastMRI [60], and (j) MiO100 [61].

Figure 2. Examples of different degradation types and domains. The images correspond to the following open-source datasets: (a) PolyU [53], (b) LHP [54], (c) NH-HAZE [55], (d) RealSnow10K [56], (e) DPDblur [14], (f) LSRW [57], (g) DIV4K50 [58], (h) Squid [59], (i) FastMRI [60], and (j) MiO100 [61].

Existing IR methods can be broadly categorized into task-specific models and unified frameworks. Task-specific IR methods [8,17] are trained to handle a single degradation type. While such models often achieve strong performance within their targeted tasks, their specialization inherently restricts generalization to unseen or mixed degradations. This limitation motivates the development of general IR frameworks [62,63], which employ unified architectures capable of addressing multiple restoration tasks. Nevertheless, these approaches typically still rely on task-wise training or fine-tuning procedures. More recently, AiO restoration frameworks [32,33,64,65] have attracted increasing attention. By enabling multiple restoration tasks within a single model without explicit retraining, AiO methods aim to further improve generalization. This is often achieved through degradation-aware representations, prompt-based mechanisms, or multi-branch architectures that explicitly distinguish different degradation types.

Despite these advances, conventional deep learning–based restoration paradigms remain largely constrained by predefined task formulations and limited adaptability to diverse restoration requirements. To address these limitations, recent studies have started to explore the use of language models for IR [34,36,66,67,68]. Leveraging their strong semantic understanding, reasoning capability, and cross-modal alignment, language-driven approaches enable more flexible and interactive restoration pipelines, such as instruction-guided or context-aware restoration. This emerging paradigm marks a shift from task-centric restoration models toward more general, adaptive, and semantically informed IR frameworks.

2.2. Multimodal Language Models

Recent studies on multimodal language models involve multiple closely related model families, including large language models (LLMs), vision-language models (VLMs), and multimodal large language models (MLLMs). These models differ in their input modalities and output objectives, yet collectively form the foundation of language-driven IR methods reviewed in this survey. While capability boundaries between VLMs and MLLMs continue to develop, we adopt a functional distinction based on their dominant usage patterns in restoration frameworks.

Large language models. LLMs are foundation models trained on large-scale text datasets and operate exclusively on language inputs and outputs. Representative models include GPT [69], LLaMA [70], Qwen [71], and DeepSeek [72]. These models have achieved remarkable progress in natural language understanding, reasoning, and generation, demonstrating strong generalization across a wide range of language-centric tasks. LLMs typically do not process images directly. Instead, they receive visual information as textual descriptions, such as captions and image attribute descriptions.

Vision-language models. VLMs focus on aligning visual and textual representations through cross-modal embedding spaces [73,74,75]. By jointly modeling visual perception and language semantics, VLMs support cross-modal understanding and semantic grounding. Typical VLMs include CLIP-like [73,75] and ALIGN-like [74] architectures. While recent VLM models demonstrate limited reasoning ability, they can process images directly and are typically optimized for representation alignment.

Multimodal large language models. Building upon the success of LLMs, recent studies have extended LLMs to multimodal settings by giving them the capability to process non-textual inputs [76,77,78,79,80]. In particular, by integrating visual perception modules, MLLMs can directly adopt images as input and perform high-level reasoning. At the same time, these models preserve the ability to follow instructions and make stepwise decisions, making them suitable for tasks that need both image understanding and logical decision-making.

In this survey, we adopt the term MLLM to denote language-centric foundation models equipped with visual perception modules, regardless of whether they are implemented via explicit multimodal pretraining or modular integration. When referring to language-driven IR methods without emphasizing a specific model category, we adopt the umbrella term VLM-/MLLM-based IR.

2.3. Image Quality Assessment

IQA plays a critical role in image processing by enabling the evaluation and optimization of visual content quality [81]. Although subjective assessment based on the human visual system (HVS) is generally regarded as the most reliable criterion, it is costly and time-consuming, motivating extensive research into objective IQA metrics.

Depending on the availability of reference images, IQA methods are broadly categorized into full-reference (FR-IQA) and no-reference (NR-IQA). FR-IQA methods compare distorted images with their high-quality references using metrics such as PSNR [39], SSIM [38], and MSE, as well as learning-based perceptual metrics [40,82,83,84,85]. In contrast, NR-IQA predicts image quality directly from distorted inputs without reference images. Early methods rely on handcrafted natural scene statistics, such as BRISQUE [86], NIQE [87], and PIQE [88], while recent approaches adopt deep learning to model complex perceptual patterns [89,90,91,92,93].

Most recently, VLM-/MLLM-based IQA methods [41,42,45,46,47,94,95,96] have emerged as a new paradigm. By leveraging multimodal reasoning capabilities and foundational knowledge, these approaches aim to narrow the gap between objective metrics and subjective human assessment. Some methods evaluate image quality using natural languages [96,97,98], while others directly produce quantitative quality scores [45,95,99].

2.4. Relevant Surveys

A number of surveys have reviewed IR from different perspectives. Many of these works organize the literature according to specific degradation types or application domains [1,4,7,13,16,19,22,31,50]. Along the recent trend of AiO restoration, Jiang et al. [31] provided a systematic overview of AiO restoration frameworks. Beyond task-oriented surveys, several studies have examined IR from a model-centric perspective. For example, Su et al. [49] reviewed deep learning architectures widely adopted in IR, while Li et al. [100] offered an in-depth review of diffusion-based IR methods. Despite these efforts, the rapid emergence of VLM-/MLLM-based IR methodologies has not yet been systematically reviewed. Differently, in this work, we present a comprehensive review of VLM-/MLLM-based IR from the following three aspects: 1) recent advances in VLM-/MLLM-based IR methods, 2) datasets and VLM-/MLLM-assisted evaluation protocols, and 3) benchmarking and comparative evaluation of VLM-/MLLM-based approaches.

3. Methodology

This section first categorizes VLM-/MLLM-based method prototypes and then analyzes each category in detail through representative approaches. Figure 3 summarizes the representative paradigms of existing VLM-/MLLM-based IR methods.

3.1. Overview of Language-Driven Restoration and Taxonomy Definition

Recent advances in MLLMs and VLMs have significantly influenced the design of IR systems. Rather than solely improving restoration architectures, an increasing number of approaches leverage language-driven semantic priors, cross-modal reasoning, and high-level decision-making to enhance restoration robustness, flexibility, and controllability. As a result, clarifying the roles of language models and how they interact with restoration pipelines becomes critical for understanding existing frameworks and guiding future research.

Existing surveys on IR mainly organize the literature from architecture-driven (e.g., CNN or transformer) [49] or task-driven [4,31] perspectives. However, these categorizations become insufficient for language-driven restoration systems, where the primary methodological distinctions arise from interaction mechanisms between language models and restoration networks. Specifically, language-driven restoration systems reshape restoration behavior by modifying information flow structures, supervision signals, control mechanisms, and optimization objectives. Therefore, to provide a structured understanding of this rapidly growing body of work, we adopt an interaction-centric taxonomy that categorizes methods according to the functional role of language models within restoration pipelines. The definition is described below:

Based on this interface characterization, we distinguish different paradigms according to the dominant interaction type:

• Representation-Level Coupling: LM outputs modify forward feature representations without altering the optimization objective or execution logic. This includes embedding replacement and semantic conditioning.
• Optimization-Level Coupling: LM outputs reshape the training objective by defining differentiable supervision signals or scalar reward functions, thereby altering optimization dynamics.
• Decision-Level Coupling: LM outputs regulate the execution structure of the restoration pipeline, such as task decomposition, module scheduling, or control policies.
• Cross-Level Coupling: Systems that simultaneously integrate multiple interaction depths within a unified framework, combining representation, optimization, and/or decision mechanisms.

On the other hand, from the perspective of model families, existing approaches are naturally organized into VLM-based, MLLM-based, and hybrid VLM–MLLM frameworks. This organization reflects practical architectural distinctions and facilitates systematic presentation. However, the model family alone does not fully explain how the introduction of language information influences restoration behavior. Within each family, language models may intervene at different depths of the restoration pipeline, from representation modulation to decision-level orchestration. Therefore, the interaction-centric taxonomy introduced above serves as a complementary analytical dimension that abstracts the coupling mechanism between language reasoning and pixel reconstruction, independent of whether VLMs or MLLMs are employed. In other words, the remaining parts are organized by model families for clarity, while the interaction-centric taxonomy provides a unified theoretical interpretation across these categories by revealing how semantic information affects representation, optimization, and decision-making processes.

VLM-based image restoration operates mainly at the representation and supervision levels of interaction depth. These methods exploit pretrained VLMs, such as CLIP [73], to extract semantically aligned embeddings from textual and visual inputs and integrate them into conventional restoration networks. Specifically, the representation-level coupling includes: (a) VLM-embedded image restoration and (b) VLM-guided image restoration. In contrast, (c) VLM-supervised image restoration corresponds to optimization-level coupling.

MLLM-based image restoration extends IR frameworks by introducing multimodal large language models as external agents. MLLM-based systems often not only modulate at representations but also intervene at optimization or decision levels, enabling higher-level control and policy adjustment. Accordingly, existing MLLM-based approaches can be grouped into three categories: (d) MLLM-guided image restoration, (e) control-based agentic image restoration, and (f) reward-driven agentic image restoration.

Cross-level hybrid systems combine multiple interaction depths within a unified framework. Two representative formats are: (g) cascaded language-guided image restoration, representing a hybrid paradigm that hierarchically combines MLLMs and VLMs within a unified pipeline. The second one is (h) role decoupled image restoration, where VLMs and MLLMs operate as independent but complementary modules.

Although the proposed interaction-centric framework provides a principled interpretation of language-driven restoration systems, practical implementations may exhibit hybrid characteristics. To reduce ambiguity, we clarify the categorization criteria based on the primary functional influence of language models. Figure 4 illustrates these paradigms along a temporal axis, highlighting the rapid development and variety of language-driven IR approaches since 2023. In the following subsections, we review each paradigm in detail, summarizing representative methods, core design improvements, as well as their respective strengths and limitations.

3.2. VLM-Based Image Restoration

VLM-Embedded Image Restoration. As one subtype of the representation-based interaction, VLM-Embedded IR directly integrates representations from pretrained VLMs into restoration backbones, typically by replacing the visual encoding stage. In this paradigm, VLMs act as fixed feature extractors that inject robust and semantically aligned priors into conventional IR networks. During CLIP pretraining, visual representations are optimized to align with text semantics, which are inherently insensitive to high-frequency details. As a result, CLIP features tend to emphasize semantic content while being less sensitive to degradation-specific perturbations, thereby improving robustness.

CLIPDenoising [101] incorporates a frozen CLIP [102] visual encoder together with a learnable denoising decoder. By leveraging the distortion-invariant and content-aware properties of CLIP’s dense visual features, the method demonstrates enhanced out-of-distribution (OOD) generalization under unseen degradation conditions.

VLM-Guided Image Restoration. In this paradigm, VLM-derived signals, such as semantic cues or degradation-aware representations, are incorporated into restoration networks through feature modulation, conditioning mechanisms, or representation alignment.

For example, CLIP-RPN [35] leverages CLIP features to characterize rain patterns and intensities, enabling adaptive deraining via dynamic cross-attention modulation. Similarly, LDP [103] utilizes text-aligned embeddings to describe blur attributes, allowing restoration models to adjust internal feature representations. In structured medical imaging, LEDA [104] enforces consistency across perceptual representations and semantic tokens guided by a language-informed codebook, while LangMamba [105] integrates language-aligned features into a Mamba-based backbone for efficient CT denoising. In addition, CLIP-SR [106] couples linguistic guidance with image feature processing to refine structure and textures for super-resolution. In AiO multi-degradation restoration, UniProcessor [107], VL-Unet [67], MdaIF [108], and AllRestorer [109] exploit textual priors to identify and handle mixed degradations. In hyperspectral IR, MP-HSIR [110] combines spectral prompts with language-visual prompts, where spectral representations provide low-rank priors and language guidance conveys degradation semantics. PromptHSI [111] further represents degradation factors through prompts to realize a universal hyperspectral restoration model.

Beyond feed-forward restoration networks, VLM guidance has been widely integrated into diffusion-based frameworks. In these approaches, VLM-derived embeddings or perceptual priors serve as conditioning signals that constrain the diffusion process. Concretely, DA-CLIP [112] explores controllable adaptation of VLM representations for multi-task restoration. Methods such as AutoDIR [113], SPIRE [114], and MPerceiver [115] encode degradation descriptors into language-aligned embeddings for diffusion conditioning. TPGDiff [116] further extends this idea by incorporating VLM-derived degradation priors into a hierarchical multi-prior formulation, constraining the diffusion trajectory at multiple stages. VLM-SSR [117] introduces CLIP-based semantic features to guide real-world super-resolution diffusion.

VLM-guided strategies have also been explored in adversarial learning settings. By conditioning discriminators on VLM-derived representations, methods such as NeRCo [118] and SeD [119] introduce semantic-aware discriminative signals that complement adversarial objectives.

VLM-Supervised Image Restoration. VLM-Supervised IR employs VLMs as perceptual supervisors to guide the training of restoration networks by constructing semantic-aware supervision, typically in the form of differentiable perceptual losses. By measuring the similarity between restored images and textual descriptions of desirable visual attributes, restoration models are optimized to produce outputs that are semantically aligned with target concepts.

This paradigm is particularly effective in scenarios where paired ground-truth data are scarce or unavailable. A representative application is underwater image enhancement [120], where a FLIP [121]-pretrained text encoder is used to encode high-level quality concepts through textual prompts, and the resulting similarity scores are incorporated as perceptual loss terms alongside traditional reconstruction objectives. Furthermore, Perceive-IR [122] enables fine-grained quality control through CLIP-aligned prompt learning and difficulty-adaptive supervision. However, since the supervision from VLMs is typically global, this paradigm may be less sensitive to fine-grained local artifacts, and its effectiveness depends on prompt design and the expressive capacity of the underlying VLM.

Methodological Synthesis. Despite leveraging different coupling mechanisms, VLM-based IR methods exhibit several shared methodological characteristics. Usually, VLMs act as semantic regularizers in many restoration frameworks. By leveraging language-aligned representations that emphasize content-level invariance, these models introduce features that are inherently less sensitive to degradation-specific perturbations. This property improves restoration robustness, particularly under distribution shifts and unseen degradations. As a result, most frameworks treat pretrained VLMs as fixed modules, leveraging their generalizable cross-modal representations without task-specific retraining.

3.3. MLLM-Based Image Restoration

MLLM-Guided Image Restoration. MLLM-guided IR refers to a class of methods that use MLLMs to interpret images or assess perceptual quality and, based on the resulting outputs, generate auxiliary guidance signals for IR networks. Typical guidance includes instruction interpretation, perceptual attribute assessment, or preference feedback that conditions restoration models in a soft and interpretable manner.

Representative works include instruction-driven restoration methods such as InstructIR [34], which uses natural language instructions to model a unified all-in-one restoration network across multiple degradation types. VLM-IMI [123] further explores instruction-based guidance by generating textural descriptions from the input images and user prompts. Beyond instruction following, perceptual guidance has also been investigated. For example, GPP-LLIE [124] introduces a generative perceptual prior for low-light enhancement by extracting high-level perceptual attributes from LLaVA [44]. Additionally, SnowMaster [56] leverages MLLM-derived perceptual preference feedback to rank candidate desnowing results and uses the selected outputs as pseudo-labels to support semi-supervised training on real-world snowy images.

Agentic Restoration. In these frameworks, MLLMs operate as high-level decision-making modules that regulate restoration behavior through planning or optimization mechanisms. Instead of directly performing pixel reconstruction, language models govern how restoration actions are selected, scheduled, or evaluated. From a system perspective, existing approaches can be broadly categorized into control-driven and optimization-driven paradigms.

Control-Driven Image Restoration. In contrast to guidance-based approaches that provide external semantic cues or conditioning signals, control-driven restoration assigns MLLMs the role of central controllers responsible for structuring the restoration procedure. The restoration problem is interpreted as a sequential decision process in which degradations are analyzed, tasks are decomposed, and restoration operations are scheduled. Language models regulate execution logic, while task-specific restoration networks remain responsible for pixel-level recovery.

Early works such as RestoreAgent [125] and AgenticIR [37] demonstrate the feasibility of this paradigm by enabling MLLMs to autonomously determine restoration tasks and execution sequences. Subsequent works focus on improving planning stability and computational efficiency. Q-Agent [126] employs chain-of-thought [127] (CoT) reasoning to decompose multi-degradation perception and adopts quality-driven greedy planning based on no-reference image quality assessment (IQA), effectively mitigating unnecessary rollbacks. To enhance scalability and robustness, several studies extend MLLM-planned restoration to multi-agent systems. MAIR [128] consists of a scheduler–expert hierarchy to decouple degradation perception and restoration execution, while HybridAgent [129] adopts collaborative agents to balance planning accuracy and computational efficiency. Specialized frameworks such as 4KAgent [58] further adapt the planning paradigm to ultra-high-definition (UHD) IR. System-level frameworks, including Clarity ChatGPT [130] and JarvisIR [36], emphasize interactive refinement and robustness in real-world environments.

Reward-Driven Image Restoration. Reward-driven restoration as an optimization-level coupling treats language models primarily as evaluators rather than controllers. Instead of structuring restoration procedures, MLLMs define perceptual objectives that guide policy learning. Restoration behavior is shaped through reward signals, preference feedback, or evaluator-based scoring, shifting the learning target from explicit reconstruction supervision toward perceptual optimization.

MOERL [131] formulates IR as a reinforcement learning (RL) problem, where a Mixture-of-Experts (MoE) model is dynamically refined based on perceptual rewards. A reward-driven policy is learned to adaptively route restoration actions without requiring explicit degradation labels, enabling robust handling of complex and mixed weather conditions. In contrast, SimpleCall [132] adopts a label-free decision process in which a lightweight agent is trained to sequentially invoke restoration tools using MLLM-derived perceptual feedback. By optimizing restoration policies through evaluator-based rewards, the framework enables effective restoration without ground-truth supervision.

Methodological Synthesis. MLLM-based restoration paradigms extend coupling from representation modules to optimization and decision levels. Specifically, rather than serving as auxiliary feature providers, these models participate in degradation interpretation, task decomposition, and restoration planning. Across guided, planned, and agentic formulations, MLLMs operate at a higher semantic level, representing a structural reformulation of restoration pipelines rather than conventional multimodal enhancement.

3.4. Cross-Level Hybrid Systems

Cascaded Language-Guided Image Restoration. Cascaded language-guided IR adopts a multi-stage pipeline in which language understanding and visual restoration are decomposed into hierarchically connected modules. In this paradigm, high-level semantic understanding is first performed by MLLMs, and the resulting linguistic representations are subsequently propagated to VLM components to bridge language semantics with visual embedding spaces. The transformed semantic information is then utilized to support downstream pixel-level restoration.

For instance, LLMRA [68] employs an MLLM [133] to perform high-level degradation understanding. The generated language descriptions are then mapped into an embedding space via a pretrained CLIP text encoder, forming structured degradation representations that condition a restoration network. Similarly, PromptFix [134] utilizes a combination of LLaVA [44] together with a pretrained CLIP visual encoder to extract semantic cues, which are used as auxiliary prompts to guide an instruction-driven diffusion restoration model. While both LLMRA and PromptFix adopt cascaded language-guided pipelines, they differ in the functional criticality of language guidance. In LLMRA, language-generated degradation representations constitute a structural prerequisite for restoration, whereas in PromptFix, language guidance from the MLLM serves as an auxiliary enhancement that complements a generative prior.

Role-Decoupled Image Restoration. In contrast to cascaded language-guided pipelines that integrate VLMs and MLLMs into a hierarchical semantic processing chain, the role-decoupled interaction pattern employs them as parallel and functionally independent modules. In this paradigm, VLMs and MLLMs are utilized for different roles, such as task identification or perceptual assessment, and jointly support the restoration process.

Following this paradigm, ClearAIR [66] employs a VLM-based task identifier [112] to recognize degradation types and determine appropriate restoration branches, while an MLLM-based IQA module [45] provides global perceptual quality scores that guide restoration refinement. Such designs allow different language models to specialize in heterogeneous subtasks, improving system interpretability while avoiding error accumulation caused by hierarchical semantic dependencies.

Methodological Synthesis. Mixed systems demonstrate cross-level coupling and typically exploit functional complementarity, where VLMs provide stable representation-level priors, and MLLMs contribute higher-level reasoning capabilities. Such designs mitigate limitations of single-model systems while promoting modular restoration architectures. This transition suggests that language-driven restoration may evolve toward heterogeneous multimodal systems, in which restoration behavior emerges from coordinated interactions among specialized semantic and visual components.

4. Experiments

In this section, we summarize representative datasets widely used in language-driven IR frameworks across different tasks. We then review commonly adopted IQA metrics, covering both classical and recent VLM-/MLLM-based evaluation paradigms. Together, these datasets and metrics provide a comprehensive view of current benchmarks and evaluation practices for language-driven IR.

4.1. Datasets

Existing VLM-/MLLM-based IR studies commonly adopt subsets of established restoration benchmarks, as summarized in Table 1. These datasets vary substantially in scale, spatial resolution, scene diversity, and data-acquisition protocols, spanning both real-world captures and synthetically generated degradations. While synthetic datasets offer controllability and scalability, they may inadequately capture the ambiguity, complexity, and stochastic characteristics of real-world degradations. This discrepancy becomes particularly critical in language-driven frameworks, where degraded understanding relies on semantic representations that are inherently sensitive to contextual variations.

On the other hand, recent language-driven restoration frameworks increasingly require datasets that explicitly model image-language interactions. However, as shown in Figure 5, most datasets consist primarily of degraded-clean image pairs while rarely providing structured linguistic annotations describing degradation characteristics or perceptual attributes. The absence of structured language annotations may introduce inconsistencies when training or evaluating language-driven restoration models. To bridge this gap, PromptFix [134] constructs a large-scale instruction-following dataset that contains approximately 1.01 million input-goal-instruction triplets across diverse low-level tasks. Furthermore, in safety-critical applications such as autonomous driving, CleanBench [36] defines an instruction sample as a triplet, consisting of a user instruction, a degraded image, and a response. Such datasets reflect a paradigm shift in which restoration is formulated not only as pixel reconstruction but also as a language-conditioned generation problem. Despite these initial explorations, restoration datasets with captions are still underexplored.

4.2. Evaluation Metrics

Existing IQA metrics are summarized in Table 2, including FR-IQA, NR-IQA, and other evaluation paradigms.

Conventional IQA metrics. Image restoration performance is traditionally evaluated using IQA metrics that measure fidelity or perceptual similarity between restored images and references. Full-reference IQA metrics, such as PSNR [39], SSIM [38], FSIM [184], and MAE, quantify pixel-level or structural consistency with ground-truth images. Learning-based metrics (e.g., LPIPS [40], DISTS [82], CKDN [83], and AHIQ [84]) assess perceptual similarity in deep feature spaces and exhibit improved correlation with human judgments. When reference images are unavailable, no-reference IQA metrics are commonly adopted, ranging from hand-crafted statistical measures (e.g., BRISQUE [86], NIQE [87], and PIQE [88]) to learning-based models such as MUSIQ [89] and MANIQA [91]. In addition, distribution-based metrics like FID [185] are often used to evaluate feature distribution alignment between restored and real images, particularly in generative restoration scenarios.

VLM-based IQA. VLMs have recently been widely employed for NR-IQA, leveraging multimodal representations learned through large-scale image–text pretraining, such as CLIP [102]. Unlike conventional IQA approaches that rely on explicit distortion modeling or perceptual regression networks, VLM-based methods exploit the implicit perceptual priors encoded within cross-modal alignment spaces.

Early studies, such as CLIP-IQA [41], demonstrate that quality perception can be formulated as a prompt-driven similarity-comparison problem using antonym prompt pairs (e.g., good vs. bad). Subsequent works extend this principle in various directions. QualiClip [43] proposes a self-supervised, opinion-unaware approach via quality-aware image–text ranking to learn distortion-sensitive representations. PromptIQA [193] further enables requirement-adaptive assessment by incorporating a small set of image–score pairs as prompts. In addition, GRMP-IQA [192] improves data efficiency through meta-prompt learning with gradient regularization, and ATTIQA [194] addresses annotation scarcity by introducing attribute-aware pretraining that leverages VLMs to generate quality-related attribute pseudo-labels. In summary, these approaches highlight that perceptual quality can be effectively inferred from language-aligned representations, often exhibiting strong cross-dataset generalization without explicit mean opinion scores (MOS) supervision. In special domains, SCUIA [201] explicitly models semantic context via contrastive learning for underwater IQA.

Beyond prompt-based mechanisms, LIQE [191] introduces a vision–language correspondence framework that jointly models semantics, distortions, and quality, whereas UNIQA [190] proposes a unified multimodal pretraining strategy bridging image quality and aesthetic assessment. Meanwhile, SFD [195] explores semantic feature discrimination as a proxy for quality estimation, demonstrating that perceptual quality can be approximated via semantic consistency and feature-space discrimination.

In summary, VLM-based IQA treats IR as a multimodal alignment problem, offering advantages in training efficiency and generalization, yet remaining constrained by prompt dependency, semantic bias, and imperfect calibration with human subjective judgments.

MLLM-based IQA. MLLM-based IQA naturally supports both language-based quality representation and quantitative scoring paradigms. Description-oriented approaches formulate IQA as a language-grounded reasoning task, where quality is expressed through structured linguistic outputs that improve interpretability and human alignment. Representative frameworks like DepictQA [96] and DepictQA-Wild [198] formulate IQA as descriptive and comparative language-based quality understanding to achieve human-like assessment. IQAGPT [199] proposes a caption-driven pipeline to generate quality scores and natural-language explanations. Beyond global quality understanding, SEAGULL [97] highlights region-aware quality reasoning through SAM-guided [215] feature modeling, while Q-Ground [98] introduces a detailed visual quality analysis through visual grounding. Additionally, Co-Instruct [205] explores an open-ended comparative reasoning framework, facilitating MLLMs to produce quality comparisons with detailed reasoning.

Score-based MLLM-IQA methods focus on predicting continuous quality scores while leveraging reasoning mechanisms for robustness. In particular, Q-Align [46] discretizes subjective scores to a one-hot label to emulate the human judgment process. In contrast, DeQA-Score [45] improves quality prediction by modeling score distributions as soft labels, and Q-Scorer [99] explicitly adapts multimodal representations for direct scalar quality prediction. In addition, Dog-IQA [196] introduces a standard-guided discrete scoring mechanism combined with mix-grained global–local quality aggregation. To address label efficiency, LEAF [202] decouples perceptual knowledge from MOS calibration via distillation.

On the other hand, the comparison-derived scoring framework Compare2Score [95] achieves quality scoring by inferring continuous scores from adaptive pairwise comparisons. Q-Insight [197] integrates reinforcement learning with preference modeling, whereas Q-Ponder [203] introduces explicit joint optimization objectives for both scoring and reasoning. Furthermore, RALI [208] treats reinforcement learning-based IQA as a reasoning-driven representation compression process and proposes a lightweight framework that preserves scoring accuracy and cross-domain generalization without requiring LLM inference. Reliability-aware modeling is explored by Q-Hawkeye [200], incorporating uncertainty-aware policy optimization. In domain-specific settings, CAP-IQA [204] integrates text priors and image-specific context for task-aware quality scoring, demonstrating reliable performance.

In parallel, Q-Instruct [94] and Q-Bench [206,207] focus on improving and evaluating low-level perceptual reasoning capabilities of multimodal language models, underscoring the importance of instruction tuning and benchmarking. Additionally, the AgenticIQA framework [48] represents a paradigm that treats perceptual quality assessment as a structured decision-making process rather than a single-pass prediction task. While MLLM-based IQA offers enhanced interpretability, flexibility, and perceptual reasoning capacity, challenges remain in score calibration, reasoning consistency, and computational efficiency.

Comparison: Conventional vs. VLM-/MLLM-based Evaluation. Conventional IQA metrics and language-driven evaluation reflect different yet complementary assessment paradigms. Classical full-reference measures (e.g., PSNR, SSIM) quantify pixel fidelity and structural consistency, providing deterministic and reproducible criteria. These metrics remain essential for benchmarking reconstruction accuracy and optimization stability.

By contrast, VLM-/MLLM-based evaluation estimates quality through multimodal similarity modeling, preference reasoning, and language-conditioned quality interpretation. Instead of directly measuring pixel fidelity, these approaches estimate quality through mechanisms such as prompt-driven image–text alignment, pairwise comparison, distribution-aware score modeling, and reasoning-guided scoring. Such evaluation strategies are particularly relevant for generative restoration, mixed degradations, and no-reference scenarios. Unlike conventional metrics, language-driven approaches may favor semantic plausibility or contextual consistency.

Importantly, language-driven evaluation should not be interpreted as a direct replacement for conventional IQA metrics, but rather as a complementary perceptual assessment mechanism. Conventional metrics ensure numerical stability and comparability, whereas language-driven evaluators capture complementary perceptual and semantic cues. Recent studies [37,114,132] increasingly adopt hybrid evaluation protocols that combine deterministic fidelity measures with language-driven perceptual assessment to achieve more comprehensive performance evaluation.

Evaluation Criteria and Auxiliary Paradigms. In addition to direct quality prediction metrics, correlation-based criteria such as PLCC, SRCC, and KRCC [209] are widely adopted to measure the consistency between predicted scores and human subjective ratings. These statistics do not evaluate image quality directly but instead quantify the reliability of IQA models with respect to human perception. Additionally, text-based metrics such as BLEU-N [211] and ROUGE-L [212] are often used to evaluate the semantic fidelity of generated descriptions and reference captions, especially in instruction-following or explanation-based restoration frameworks.

Furthermore, several studies adopt task-oriented evaluation paradigms that assess image quality indirectly through downstream performance [98]. In this setting, restored images are treated as inputs to high-level vision systems, such as detection, segmentation, or recognition models, where improvements in task performance serve as proxies for quality enhancement. The underlying assumption is that degradations often impair feature extraction and perception reliability, and thus higher downstream accuracy may indicate improved visual quality. Nevertheless, this assumption is not universally valid. Prior studies [216] have shown that perceptual quality and downstream performance are not strictly causally linked. Despite these limitations, task-driven evaluation remains particularly relevant in application-critical domains, including medical imaging and autonomous driving, where restoration quality is ultimately defined by its functional impact on subsequent perception or decision-making processes.

Metrics for Language-driven IQA Ability. Leveraging the strong semantic understanding capabilities of MLLMs and VLMs for IQA has emerged as a promising and rapidly evolving research direction. Usually, correlation-based metrics, such as PLCC and SRCC, are widely used, which quantify agreement between predicted scores and human subjective ratings. However, the systematic evaluation of the IQA capability of such models remains limited. 2AFC [217] represents an early attempt by proposing consistency, accuracy, and correlation metrics to analyze judgment robustness and alignment with human opinion scores. Despite this initial effort, the evaluation of VLM-/MLLM-based IQA remains largely underexplored, leaving substantial room for further investigation.

4.3. Experimental Results

To better understand the effectiveness of different design choices, we report experimental results of several VLM-/MLLM-based all-in-one restoration frameworks under three commonly used experimental settings. Table 3 presents results on three degradations: deraining, denoising ($\sigma =15,25,50$), and dehazing. Meanwhile, Table 4 extends the evaluation to five degradations by additionally including deblurring and LLIE. Table 5 further reports performance on the single LLIE task. All results are reported in terms of PSNR and SSIM.

Under the three-degradation setting (Table 3), language-driven restoration frameworks exhibit comparable overall performance. For instance, VLU-Net [67] achieves the highest deraining PSNR (38.93 dB), whereas ClearAIR [66] exhibits more consistent cross-task behavior, maintaining stable performance across deraining, denoising, and dehazing. A similar trend is observed in the five-degradation setting (Table 4), where ClearAIR achieves the best average performance compared to other language-driven approaches. In contrast, on the single-task LLIE benchmark (Table 5), task-specific non-language-driven methods such as RetinexFormer [30] (25.16 dB) and RetinexDiff++ [218] (24.67 dB) outperform most language-driven models in terms of PSNR.

These observations suggest that, although language-driven frameworks enhance flexibility and generalization in multi-degradation scenarios, their advantages in specialized tasks remain limited. Moreover, gains in pixel-level fidelity (e.g., PSNR) remain limited. This is because language-driven methods focus more on semantic alignment and user intent than on distortion minimization. This leads to a mismatch between current evaluation protocols and the goals of language-driven restoration. Moreover, language-driven IQA metrics are rarely included in existing benchmarks, highlighting the need for evaluation frameworks that jointly consider fidelity, semantic correctness, and user-oriented restoration quality.

5. Discussion and Open Challenges

Although the integration of MLLMs and VLMs has driven the development of IR frameworks, it has introduced new capabilities while simultaneously giving rise to additional challenges. In this section, we analyze key open challenges in VLM-/MLLM-based methods, focusing on generalization, computational efficiency, cross-paradigm trade-offs, evaluation reliability, dataset design, high-dimensional representations, and trustworthiness. We further discuss potential research directions that may help address these challenges and inform future developments in language-driven restoration systems.

5.1. Generalization and Robustness

In real-world scenarios, degradations are often complex, mixed, or poorly defined, making generalization and robustness persistent challenges for IR, even when incorporating semantic priors and degradation-aware guidance via MLLMs and VLMs. Specifically, while VLM- or MLLM-derived representations provide high-level semantic context, they do not fully eliminate sensitivity to degradation variations. Second, language-driven frameworks frequently rely on textual descriptions, prompt formulations, or semantic interpretations to characterize degradation types [68,123,134]. Such dependencies may lead to inconsistent restoration behaviors due to linguistic variability and prompt sensitivity. Moreover, language models themselves may exhibit hallucinations and domain biases inherited from large-scale pretraining, which can affect degradation understanding and guidance reliability.

One potential solution involves prompt-invariant modeling, such as template-based or structured prompts, to reduce instability caused by diverse linguistic formulations. Another direction is uncertainty-aware restoration, where degradation cues are accompanied by confidence estimates to improve robustness against hallucination-induced errors. Despite these emerging strategies, developing principled mechanisms for handling degradation ambiguity, prompt variability, and language-model uncertainty remains an important challenge for future research.

5.2. Computational Efficiency

While VLM-/MLLM-based IR frameworks introduce enhanced semantic awareness and perceptual reasoning capabilities, computational efficiency remains a central challenge. The computational burden arises from multiple sources. First, pretrained VLMs and MLLMs typically contain a large number of parameters, making semantic feature extraction and cross-modal reasoning inherently expensive. Additional conditioning and reasoning mechanisms further increase memory consumption. Moreover, agentic paradigms often require multiple reasoning iterations, candidate evaluations, or tool invocation loops, leading to high latency and inference cost.

Recent studies have explored several strategies to mitigate these efficiency bottlenecks. For example, policy optimization [132] and dynamic routing mechanisms [126] have been applied to reduce unnecessary reasoning and model invocation steps. Hybrid designs [129] further improve efficiency by restricting expensive language-driven computation to critical stages. Despite these advances, computational efficiency remains a key challenge for practical deployment. Future research may focus on developing compact multimodal models tailored for restoration tasks as well as reducing redundancy in cross-modal representations through mechanisms such as knowledge distillation or token reduction. In addition, resolution-adaptive architectures present a promising direction, enabling language-driven reasoning to operate at coarse semantic scales while preserving high-resolution reconstruction within lightweight visual backbones.

5.3. Cross-Paradigm Trade-Offs

Different restoration paradigms rely on distinct design assumptions that introduce structural trade-offs. Language-driven supervision signals may conflict with low-level reconstruction constraints. Specifically, perceptual losses or evaluator-based rewards derived from VLMs/MLLMs typically operate at a global semantic level and may exhibit limited sensitivity to local structural artifacts. As a result, optimization may favor high-level consistency while under-constraining fine-grained visual fidelity. Moreover, paradigms also differ in balancing capability and system complexity. MLLM-centered frameworks enable flexible degradation interpretation and dynamic control but incur increased deployment overhead, whereas VLM-guided approaches preserve relatively structural simplicity with more limited decision-level flexibility.

5.4. Evaluation Reliability

In contrast to traditional evaluation metrics, language-driven IQA models do not always produce numerically stable or strictly consistent scores. Instead, they often operate in a semantic assessment space, where quality judgments are expressed through preferences, rankings, or linguistic interpretations [96,198]. Recently, several studies [45,99] have explored MLLMs as quantitative quality scorers. While such models demonstrate promising alignment with human perception in many scenarios, their evaluation behavior may exhibit variability and uncertainty. In particular, assessment results can be sensitive to prompt formulations, where small variations in phrasing or textual context may lead to inconsistent quality predictions. This prompt sensitivity introduces challenges for reproducibility and comparability across evaluation settings. Moreover, score calibration remains a nontrivial challenge, as language models are not explicitly optimized for metric-level numerical stability. This variability becomes particularly critical when language-based IQA outputs are directly used for model selection or benchmark comparisons.

The coexistence of conventional IQA metrics and language-driven evaluators introduces new considerations regarding evaluation reliability. Future research may explore standardized prompting protocols, confidence-aware evaluation mechanisms, and hybrid assessment frameworks that combine deterministic metrics with language-driven quality reasoning. Nevertheless, ensuring stable and comparable evaluation remains an open problem.

5.5. Dataset Design for VLM-/MLLM-Based IR

Numerous datasets have been proposed for various restoration tasks. However, several limitations still remain in existing datasets. Current restoration benchmarks rarely incorporate structured linguistic annotations describing degradation characteristics or perceptual attributes. While MLLMs and VLMs benefit from large-scale pretraining and strong semantic understanding, they may still exhibit failure modes in complex or ambiguous scenarios. Consequently, language-driven restoration frameworks often rely on synthetic prompts [110] or automatically generated descriptions [56,68], which can introduce semantic inconsistencies. This highlights the importance of reliable textual supervision for improving model stability and performance. Moreover, most restoration datasets are constructed under predefined degradation models, which may not adequately capture the complexity or compositional nature of real-world degradations. Such limitations may restrict the robustness and generalization capability of language-integrated restoration systems.

Future dataset construction may benefit from multi-level annotations beyond pixel-level ground truth, including degradation semantics, perceptual quality descriptions, and task-oriented linguistic guidance. Furthermore, developing standardized degradation annotation protocols represents a critical research direction for improving cross-modal consistency while reducing annotation ambiguity. Furthermore, incorporating diverse, composed, and open-world degradations may enhance model robustness under realistic conditions.

5.6. Leveraging Multimodal Data and High-Dimensional Representations

MLLMs and VLMs enable IR frameworks to incorporate textual priors and cross-modal semantics, opening new opportunities to exploit richer data modalities. However, existing studies mainly focus on RGB images, while other sensing modalities remain largely underexplored, including infrared imagery, event-based data, and depth measurements [227,228]. Integrating heterogeneous sensory inputs may improve the interpretation of degradation, structural reasoning, and contextual consistency. Therefore, extending language-driven restoration frameworks to these modalities presents a promising research direction.

Beyond static images, applying VLM-/MLLM-based restoration paradigms to high-dimensional data, such as videos, represents another important direction for future research. Video restoration inherently requires not only accurate frame-level reconstruction but also temporal consistency across frames. In this context, language-driven reasoning mechanisms may provide complementary benefits by scene-level coherence. For example, MLLMs may assist in interpreting dynamic degradation patterns [229] or evaluating video quality to support invoking tools.

5.7. Ethics and Trustworthiness

Ethical considerations and trustworthiness have become increasingly relevant in VLM-/MLLM-based IR frameworks. Certain approaches rely on online inference services, which require transmitting visual data for non-local processing [130]. Such designs may raise privacy and data security concerns, particularly in sensitive scenarios including medical imaging, unmanned aerial vehicle (UAV), and autonomous driving.

Beyond data privacy, the reliability of language-driven components introduces additional challenges. MLLMs are known to exhibit hallucinations, reasoning inconsistencies, and biases inherited from large-scale training data. When integrated into restoration pipelines, these factors may result in incorrect degradation interpretation, unstable guidance signals, or semantically inconsistent restoration outcomes. Unlike conventional restoration errors, failures caused by MLLMs or VLMs may be less predictable and more difficult to attribute.

Future research may investigate privacy-preserving multimodal restoration frameworks and robustness against hallucination-induced errors. In addition, the development of locally deployable multimodal models [72] represents a promising direction for mitigating privacy concerns while preserving semantic reasoning capabilities.

6. Conclusions

In this survey, we provided a structured review of language-driven IR frameworks and IQA methods based on MLLMs and VLMs. We introduced a unified taxonomy that characterizes how language models interact with restoration pipelines, encompassing paradigms such as representation-level, optimization-level, design-level, and hybrid coupling. Through this perspective, we analyzed representative methodologies and architectural designs, highlighting the evolving role of language information in IR. In addition, we reviewed emerging VLM-/MLLM-based IQA approaches and discussed their distinctions from conventional evaluation metrics. We also summarized widely used restoration datasets and compared state-of-the-art VLM-/MLLM-based methods across different restoration tasks.

The rapid progress of language-integrated restoration systems also raises fundamental challenges, including cross-paradigm trade-offs, generalization under complex degradations, computational efficiency, evaluation reliability, dataset construction, and trustworthiness. These challenges highlight that language-driven restoration is not merely an architectural extension of traditional frameworks, but rather a paradigm shift that raises new methodological and system-level questions. We further anticipate future research directions: exploring uncertainty-aware restoration mechanisms, more efficient semantic-guided architectures, and scalable annotation strategies for image–text paired datasets. Extending language-driven restoration paradigms to heterogeneous modalities and high-dimensional data also presents promising opportunities.

As language and vision models continue to develop, their integration is expected to play an increasingly important role in bridging low-level visual restoration with high-level semantic understanding. Overall, this survey aims to provide a comprehensive reference for understanding the evolving design principles of language-driven IR and to encourage further investigation into this rapidly developing research direction.