A Comprehensive Survey of Multimodal LLMs for Scientific Discovery

1. Introduction

Recent breakthroughs in artificial intelligence (AI) have been driven by foundation models—large-scale neural networks trained on broad data that can be adapted to diverse tasks [55,132]. In particular, large language models (LLMs) based on the Transformer architecture [162] have achieved remarkable proficiency in natural language processing, exhibiting emergent abilities such as few-shot learning [5,14,81,175,176] and human-aligned dialogue generation [48,133,237]. However, these advances remain confined to text-based inputs and outputs, whereas scientific problems are inherently multimodal—spanning modalities such as clinical text, biomedical images, genomic sequences, molecular graphs and protein structures, among others [35,86,108,119]. This has catalyzed a new generation of multimodal large language models (MLLMs) designed to bridge diverse data modalities and enable more comprehensive reasoning.

MLLMs extend language modeling beyond text, enabling AI systems to ingest and generate diverse data types such as images, audio, and structured scientific representations [98,181,201]. Early examples like Flamingo [5] and Kosmos-1 [70] showed that LLMs can be adapted or trained to jointly reason over visual and textual inputs, while open-source efforts such as MiniGPT-4 [233] and LLaVA [87] align vision encoders with LLMs, marking a shift from text-only AI towards generalist multimodal agents. This multimodal trend is especially impactful in science, where tasks often integrate multiple modalities. Biomedical models such as BioMedGPT [119] unify protein sequences, molecular structures, and textual knowledge for drug discovery. In genomics, systems like Geneverse [113] and GeneChat [35] connect DNA sequences with biomedical knowledge. In materials science, multimodal AI can parse literature and microstructure images jointly to propose new materials or predict properties [4,11,15,136]. Across these domains, MLLMs act as engines that fuse language with domain-specific modalities, enabling holistic analysis and accelerating discovery (Figure 1).

Given this rapid progress, there is a pressing need to systematically survey MLLMs in science. Existing surveys mainly focus on general-purpose LLMs (e.g., [223]) or on narrower multimodal techniques (e.g., [201]). Domain-specific reviews exist for biology or biomedicine [60,106,157,165,167,185,215,218,226,228], but no prior work offers a unified overview across natural language, biomedical imaging, molecular data, genomics, and material science (Table 2).

• Our Contributions.

To fill this gap, we present ${\mathrm{S}}^3-\mathrm{Bench}$, a comprehensive study of MLLMs for scientific discovery. Our contributions are threefold: (1) We present the first comprehensive survey work of MLLMs across major scientific domains—including drug discovery, protein engineering, genomics, materials science, and biomedicine—highlighting representative model architectures, domain-specific adaptations, and benchmark datasets;(2) We synthesize emerging directions, including diffusion-based LLMs and multimodal diffusion-based LLMs, and outline open challenges for future research (Section 8); and (3) we conduct benchmarking experiments on selected open-source MLLMs, evaluating their performance on highly significant tasks such as molecular property prediction and protein function prediction (Section 9).

In summary, MLLMs are rapidly evolving and hold immense promise for advancing scientific discovery, by consolidating progress across diverse modalities and domains and by providing empirical benchmark results, this survey aims to serve as both a reference and a foundation for future work. The paper is organized as follows: Section 3, Section 4, Section 5, and Section 6 review domain-specific developments of MLLMs in small molecules, proteins, genomics, and materials, respectively. We also discuss emerging topics and future directions in Section 8.

Figure 2. Overview of our ${\mathrm{S}}^3$-Bench, highlighting four major components discussed in the paper and presenting the key modalities and their corresponding applications in this field.

Figure 2. Overview of our ${\mathrm{S}}^3$-Bench, highlighting four major components discussed in the paper and presenting the key modalities and their corresponding applications in this field.

2. General Overview for LLMs and MLLMs

In this section, we aim to provide readers with a coherent background framework by reviewing the foundational components and architectural innovations of LLMs and their multimodal counterparts (MLLMs). By systematically discussing their core components, training paradigms, multi-modal extensions, we establish a clear understanding of how these models function. We also present a high-level overview of the framework for the LLMs and MLLMs in Figure 7. This overview sets the stage for the the main paper, where we turn to the specific applications of MLLMs in scientific domains.

• Core Components of LLMs.

The backbone of modern LLMs is the Transformer architecture [162], which revolutionized natural language processing by introducing self-attention mechanisms. At the input stage, text is first processed into tokens through a tokenizer. Depending on the domain, these tokens may correspond to words, subwords, or characters, while specialized tokenizers are designed for structured domains such as DNA sequences or chemical molecules. Each token is then mapped into a dense vector representation by the embedding layer, where positional embeddings (absolute or relative type) inject sequence order information into the otherwise permutation-invariant architecture. The central component of LLMs consists of stacked Transformer blocks. Based on the original Transformer architecture, three mainstream LLM architectures have emerged: encoder-only, represented by the BERT [42] family; decoder-only, exemplified by LLaMA [96]; and encoder-decoder, represented by models such as GLM [37]. Specifically, each block (often referred to as an LM layer) contains multi-head self-attention layers, feed-forward networks, normalization steps, and residual connections, which together enable the model to capture long-range dependencies across large contexts. After that, models employ different pretraining tasks to acquire their language understanding capabilities. For instance, encoder-only models are typically trained with Masked Language Modeling (MLM), decoder-only models with Next Token Prediction (NTP), and encoder-decoder models with permutation-based tasks. In recent years, it has been observed that fine-tuning large models after large-scale pretraining effectively bridges the gap between the next-word prediction objective of LLMs and the users’ objective of having LLMs followhumaninstructions [133,147]. Finally, the model is equipped with an output layer: generative models project hidden representations to vocabulary probabilities, while encoder-based models connect to task-specific heads for classification, retrieval, or regression. These components collectively determine the expressive power and adaptability of LLMs across tasks.

Table 1. Comparison of coverage of recent survey papers on LLMs/MLLMs across different domains.

Survey	Protein	Drug & Samll Molecule	Gene	Material	Biomedicine	Target Multimodal	Benchmarking
Our Survey	✓	✓	✓	✓	✓	✓	✓
LLMs/MLLMs for Science
[218]	✓	✓	✓	✓
[216]	✓	✓	✓			✓
[69]	✓	✓	✓	✓	✓
[20]		✓			✓	✓
LLMs/MLLMs for Biomedicine
[186]					✓
[200]					✓
[164]		✓			✓
[228]					✓
[16]	✓		✓		✓
[226]					✓
[106]					✓
[60]					✓
[185]					✓
[167]					✓
[165]					✓
[157]					✓

Table 1. Comparison of coverage of recent survey papers on LLMs/MLLMs across different domains.

Survey	Protein	Drug & Samll Molecule	Gene	Material	Biomedicine	Target Multimodal	Benchmarking
Our Survey	✓	✓	✓	✓	✓	✓	✓
LLMs/MLLMs for Science
[218]	✓	✓	✓	✓
[216]	✓	✓	✓			✓
[69]	✓	✓	✓	✓	✓
[20]		✓			✓	✓
LLMs/MLLMs for Biomedicine
[186]					✓
[200]					✓
[164]		✓			✓
[228]					✓
[16]	✓		✓		✓
[226]					✓
[106]					✓
[60]					✓
[185]					✓
[167]					✓
[165]					✓
[157]					✓

• Training Objectives and Techniques.

The objectives used in training LLMs directly shape their behavior and suitability for downstream tasks. Autoregressive models, exemplified by the GPT family [137], learn to predict the next token in a sequence, which makes them particularly effective for text generation. In contrast, masked language modeling (MLM), popularized by BERT [34], involves randomly masking tokens and training the model to recover them, producing strong bidirectional representations useful for understanding tasks. Other approaches, such as XLNet [197], introduce permutation-based objectives to combine the strengths of both autoregressive and masked methods. Beyond these pretraining objectives, finetuning strategies are used for models to better perform on downstream tasks or align better with human preferences. alignment with human preferences has become increasingly important. Instruction tuning and reinforcement learning with human feedback (RLHF) represent major advances that allow models to follow instructions more reliably and produce outputs that align with user intent. By training LLMs on a dataset consisting of instruction and output pairs or using reinforcement learning with human feedback, instruction tuning bridges the gap between the next-word prediction objective and users’ objective of having LLMs adhere to human instructions [133,147]. These techniques have been critical to the deployment of interactive models like ChatGPT and GPT-4.

• Multimodal Large Language Models (MLLMs).

While LLMs excel in language tasks, many real-world applications demand reasoning across multiple modalities such as text, images, audio, or structured scientific data. MLLMs extend LLMs by introducing architectures capable of integrating heterogeneous inputs. Typically, they first leverage modality-specific encoders which are aligned with the text modality via contrastive learning to transform non-textual modalities into language-aligned embeddings, such as pretrained CLIP visual encoder [87]. Textual inputs are processed in a manner similar to LLMs. These embeddings may be then projected into the language space through a projection layer or a perceiver module, followed by the adoption of various fusion strategies to integrate information across modalities. Early-fusion approaches combine embeddings from different modalities at the input stage, often through direct concatenation [233]. In contrast, late-fusion architectures encode each modality independently and combine their outputs only at the reasoning or decision stage. The strategy has become less common as LLM capabilities have advanced. More sophisticated Fusion strategy can occur in the mid stage. For example, cross-attention architectures allow one modality to attend to features from another, exemplified by models such as Flamingo [5] and BLIP-2 [89], which achieve strong results in vision-language tasks. To address the prohibitive cost of retraining entire LLMs for multimodal tasks, adapter-based techniques such as LoRA [67] introduce lightweight, trainable components into frozen models. These advances make MLLMs more efficient and practical for specialized multimodal scenarios.

Figure 3. The overview of the architecture for LLMs and MLLMs. (a) presents the encoding strategies by which heterogeneous modalities are transformed into unified representations suitable for processing by LLMs and MLLMs. MLLMs may project or inject the representations into the language embedding space via projection layers or perceivers (b) illustrates three major LLM paradigms (encoder-only, encoder-decoder, and decoder-only) with their pretraining and supervised fine-tuning stage. Beyond these, additional refinement approaches, including Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF), may be employed. LLMs serve as the foundation of MLLMs which fuse multi-modal embeddings to generate the final output.

Figure 3. The overview of the architecture for LLMs and MLLMs. (a) presents the encoding strategies by which heterogeneous modalities are transformed into unified representations suitable for processing by LLMs and MLLMs. MLLMs may project or inject the representations into the language embedding space via projection layers or perceivers (b) illustrates three major LLM paradigms (encoder-only, encoder-decoder, and decoder-only) with their pretraining and supervised fine-tuning stage. Beyond these, additional refinement approaches, including Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF), may be employed. LLMs serve as the foundation of MLLMs which fuse multi-modal embeddings to generate the final output.

• Pretraining Datasets and Modalities.

The performance of LLMs and MLLMs is intimately tied to the scale and diversity of their pretraining datasets. For text, models typically rely on large and diverse corpora such as Wikipedia, Common Crawl, PubMed, and patent databases. In the multimodal domain, paired datasets such as LAION-5B [146] provide billions of image-text pairs for training vision-language systems. Scientific and technical applications require more specialized resources. Biological sequence data (e.g., UniProt), molecular graphs (e.g., ChEMBL), and crystallographic structures are increasingly integrated into pretraining. Moreover, structured ontologies and knowledge graphs such as the Gene Ontology (GO) or UMLS are used to augment factual reasoning and reduce hallucinations. The combination of unstructured and structured data creates rich environments for pretraining models capable of bridging multiple domains.

• Common Use Cases Across Domains.

The versatility of LLMs and MLLMs is reflected in their broad range of use cases. One major paradigm is zero- or few-shot inference, where models solve novel tasks with little to no labeled data by leveraging their pretraining knowledge. When higher domain specificity is needed, fine-tuning can adapt general-purpose LLMs to specialized applications such as drug discovery, clinical prediction, or materials design. Increasingly, LLMs are being used as tool-augmented systems. By integrating with external APIs, databases, or scientific engines such as AlphaFold DB, models can dynamically expand their capabilities beyond what is encoded in their parameters. A further evolution of this idea is the emergence of agent-based workflows, where models orchestrate multi-step reasoning, execute code, and autonomously coordinate experiments or data analysis pipelines.

3. MLLMs for Molecule Science and Drug Design

Multimodal large language models (MLLMs) are transforming molecular science and drug discovery by combining different chemical representations such as SMILES (1D) [177], SELFIES (1D) [83], molecular graphs (2D) [40] and geometric structure (3D) [49]. They improve key tasks including property prediction, molecular generation, reaction planning, and synthesis optimization, thus accelerating the discovery of novel compounds. In this section, we review recent progress along four directions: (1) LLMs for molecular representation and design, focusing on SMILES- and graph-based embeddings as well as generative models; (2) MLLMs for 1D and 2D tasks, where string and graph/image representations are fused; (3) MLLMs with 3D integration, which enhance structural understanding and retrosynthesis; and (4) chemistry-focused agents and specific applications, covering tool-augmented systems, puzzle-style reasoning, and reaction optimization. Table A1, Table A6, Table A7 and Figure 4 summarize models, datasets, and the research landscape. We also present the benchmarking results of molecular property prediction in Section 9.

3.1. LLMs for Molecule Representation and Design

While our work centers on multimodal LLMs, we also include an overview of LLMs for molecular science to give readers a comprehensive understanding of progress in this field. LLMs are advancing molecular science by learning from diverse chemical representations [179], including the aforementioned 1D, 2D, and 3D data. Transformer models such as ChemBERTa [30] and MolBERT [43] yield rich embeddings that improve property, drug-target, and drug-drug interaction prediction [62,74]. For de novo design, models like MolGPT [9], ChatMol [209], and ChatDrug [114] generate valid and novel compounds via conditional generation, reinforcement learning, or molecular editing [28]. LLMs further support multi-objective optimization and iterative refinement with expert or oracle feedback [184]. In reaction prediction and synthesis, the Molecular Transformer excels in forward and retrosynthetic tasks [102], while multimodal and instruction-following models bridge chemical language with experimental reasoning [156]. Overall, LLMs are emerging as powerful engines for molecular discovery, optimization, and synthesis.

Table 2. Comparison of coverage of recent survey papers on LLMs/MLLMs across different domains.

Survey	Protein	Drug & Samll Molecule	Gene	Material	Biomedicine	Target Multimodal	Benchmarking
Our Survey	✓	✓	✓	✓	✓	✓	✓
LLMs/MLLMs for Science
[218]	✓	✓	✓	✓
[216]	✓	✓	✓
[69]	✓	✓	✓	✓	✓
[20]		✓			✓
LLMs/MLLMs for Biomedicine
[186]					✓
[200]					✓	✓
[164]		✓			✓
[228]					✓
[16]	✓		✓		✓
[226]					✓
[106]					✓
[60]					✓
[185]					✓
[167]					✓
[165]					✓
[157]					✓

Table 2. Comparison of coverage of recent survey papers on LLMs/MLLMs across different domains.

Survey	Protein	Drug & Samll Molecule	Gene	Material	Biomedicine	Target Multimodal	Benchmarking
Our Survey	✓	✓	✓	✓	✓	✓	✓
LLMs/MLLMs for Science
[218]	✓	✓	✓	✓
[216]	✓	✓	✓
[69]	✓	✓	✓	✓	✓
[20]		✓			✓
LLMs/MLLMs for Biomedicine
[186]					✓
[200]					✓	✓
[164]		✓			✓
[228]					✓
[16]	✓		✓		✓
[226]					✓
[106]					✓
[60]					✓
[185]					✓
[167]					✓
[165]					✓
[157]					✓

3.2. MLLMs for 1D and 2D Molecular Tasks

Recent advances in molecular AI highlight a fundamental paradigm shift from single-modality models toward deeply integrated MLLMs, particularly focusing on the fusion of 1D (e.g., SMILES, SELFIES) and 2D (e.g., molecular graphs, structure images) representations [10,18,25,33,66,74,84,85,90,107,117,118,120,143,160,211]. This shift is motivated by the realization that 1D string representations provide scalability and access to abundant chemical databases, but alone cannot capture the rich spatial, topological, and functional information encoded in 2D modalities. Early progress in the field centered around models leveraging 1D molecular strings, but these were soon recognized as insufficient for tasks demanding a nuanced understanding of molecular connectivity and spatial arrangement. Addressing this, recent works such as MolPROP [143] pioneered the fusion of pretrained language models with GNN-based graph encoders, achieving significant gains in property prediction. This line of research has since been extended by LLM-MPP [74], Mol-LLM [85], and related models such as ${\mathrm{M}}^3\mathrm{LLM}$ [66], which employ advanced architectural innovations such as cross-attention between SMILES, molecular graphs, and textual descriptions, large-scale instruction tuning, and multi-level graph feature integration, resulting in strong and generalizable performance across property prediction, reaction, and generation tasks. Modular and adapter-based approaches, including MolX [84] and ChemLML [33], make it possible to flexibly combine graph encoders with LLMs and rapidly adapt to new tasks with minimal parameter overhead. Meanwhile, tokenizer-based solutions like UniMoT [211] unify 1D and 2D information at the token level, enabling seamless molecule-to-text and text-to-molecule generation. Beyond graph representations, vision-enhanced models such as ChemVLM [90], GIT-Mol [107], and Mol2Lang-VLM [160] incorporate 2D structure images alongside textual and graph modalities, further boosting captioning and molecular understanding. On the system level, frameworks like ModuLM [25] and nach0 [117] generalize the multimodal paradigm by supporting arbitrary combinations of 1D, 2D, and even 3D encoders, while InstructMol [18] and BioMedGPT [120] demonstrate the value of multi-stage instruction tuning and domain-specific integration for high-stakes biomedical applications. Importantly, domain-specialized models such as BioGPT [118] represent a milestone in biomedical molecular research. Pre-trained on large-scale PubMed literature, BioGPT achieves state-of-the-art results in biomedical text generation and knowledge extraction, accelerating automated molecular discovery from unstructured data. Collectively, these studies demonstrate that fusing 1D and 2D modalities not only consistently improves accuracy and generalizability for property prediction, generation, and retrosynthesis tasks, but also lowers the barrier for extending models to new modalities and domains. As such, the evolution from 1D-only to 1D&2D-fused MLLMs marks a major leap for molecular AI, setting a new foundation for interpretable, robust, and transferable molecular representation learning in chemistry, biology, and drug discovery.

3.3. MLLMs with 3D Geometry Integration for Molecular Tasks

Recent advances in MLLMs with 3D geometry integration can be broadly categorized by their target molecular tasks. For representation learning and property prediction, MolBind [188] aligns scientific language, 2D molecular graphs, 3D conformations, and protein pockets into a unified representation space via contrastive learning, enabling cross-modal retrieval and zero-shot molecular property prediction. Similarly, ModuLM [25] provides a modular framework that flexibly combines 1D, 2D, and 3D encoders with diverse LLM backbones, facilitating benchmarking and adaptation across a wide range of molecular tasks. For reaction modeling, RetroInText [78] integrates 3D geometry, 2D molecular graphs, and in-context reaction text to enhance multi-step retrosynthesis, particularly for long and complex synthetic routes. For materials and polymer science, PolyLLMem [217] couples Llama3-based SMILES embeddings with Uni-Mol 3D embeddings through a gated fusion mechanism, demonstrating strong performance in polymer property prediction under limited-data scenarios. Overall, these approaches reflect a growing trend toward fully multimodal MLLMs that combine complementary molecular representations (1D, 2D, and 3D) to achieve improved accuracy, interpretability, and generalizability across chemical and biological domains

3.4. MLLMs for Chemistry-Focused Agents and Special Applications

(1) Chemistry-Focused Agents. Recent work has introduced chemistry-focused agents that couple MLLMs with domain-specific tools to automate molecular data processing and reasoning [12,76,154,204,207]. Examples include ChatMolData [207], which integrates modules for literature mining, structure handling, and database operations; ChemCrow [12] and ChemToolAgent [204], which enhance LLMs for synthesis planning and property prediction; and ChemAgent [154] and ChemThinker [76], which introduce memory or multi-agent designs for more accurate and interpretable reasoning. (2) Puzzle and Reaction Condition Recommendation. Beyond standard benchmarks, chemistry also involves expert-level reasoning tasks that require integrating diverse data sources. Puzzle-style problems [1,17,46,128,238], such as structure elucidation from spectroscopic clues, test the limits of MLLMs; MolPuzzle [57] shows that while models like GPT-4o handle simple cases, they still lag behind human experts. Similarly, tasks such as reaction condition recommendation and synthesis optimization demand advanced reasoning. MM-RCR [219] exemplifies progress here by unifying textual, graph, and SMILES data, achieving state-of-the-art results and strong generalization. Overall, MLLMs are moving from unimodal to fused 1D/2D/3D, agent-augmented systems that boost property prediction, generation, retrosynthesis, and condition recommendation. We believe key hurdles remain in rigorous reasoning, interpretability/reproducibility, and closed-loop experimental and safety integration.

4. MLLMs for Protein Science

As protein-related tasks increasingly involve diverse data modalities, including natural language descriptions (1D), amino acid sequences (1D), protein graph (2D), and protein geometric structures (3D), MLLMs have emerged as a powerful framework for integrating these heterogeneous sources of information [58,108,230]. Unlike unimodal models, MLLMs can jointly reason across multiple biological representations, enabling more expressive learning and flexible interaction with biological data. In this section, we review recent advances in MLLMs across three major categories: (1) we examine models that integrate protein sequences with textual information, supporting tasks such as protein captioning, design, and function prediction. (2) we discuss models that incorporate geometric representations alongside sequence and text, enabling structure-aware learning for enhanced prediction and generation. (3) we highlight MLLMs developed for specialized tasks, including protein–protein and free-text-based biological translation. Table A2, Table A8, Table A9 and Figure 5 summarize models, datasets, and the research landscape. We also present the benchmarking results of protein function prediction in Section 9.

4.1. LLMs for Protein Science

We likewise begin by providing an overview of LLMs in protein science for readers to contextualize the broader advances in this domain. Large language models have revolutionized protein science, enabling efficient and scalable solutions for major challenges in protein property prediction, function annotation, structure prediction, and protein engineering [6,41,77,123,142]. In property prediction, models such as UniRep [6] and ProtTrans [41] leverage large-scale pretraining to achieve state-of-the-art accuracy on tasks including stability, solubility, and fluorescence. For function annotation, transformer-based models like ESM-1b [142], MSA Transformer [140], TCR-BERT [182], and ProteinBERT [13] have significantly improved label prediction, enzyme classification, and TCR-antigen binding. In structure prediction, advances such as AlphaFold2 [77], ESMFold [101], and ESM-IF [65] have enabled end-to-end and inverse folding, approaching experimental-level 3D accuracy. Models like GearNet [221], SaProt [152], and OntoProtein [214] integrate structural knowledge and ontologies, further enhancing performance on structure-aware tasks. For protein engineering and generation, ProGen [123], ProtGPT2 [45], and ProGen2 [130] apply autoregressive and conditional generation to produce novel, functional, and diverse proteins. Specialized models such as IgLM [149] and PALM-H3 [59] address antibody and virus-specific design. Collectively, these advances establish Protein LLMs as powerful engines for biological discovery and rational protein design, expanding the reach of AI-driven protein science [13,77,101,123,142].

4.2. MLLMs for Protein Sequence–Language Integration

Recent advancements in MLLMs that integrate protein sequences with textual descriptions have led to significant progress in protein-related tasks [22,36,71,94,108,116,119,122,134,135,155,171,174,212,224,227,230,236]. ProteinDT [108] combines protein sequences with textual prompts for protein design, achieving high accuracy in generating novel proteins. ProtT3 [116] excels in generating text descriptions from protein sequences using a Q-Former encoder, specifically targeting protein captioning and QA tasks. ProtCLIP [227] enhances protein function prediction by integrating protein sequences with textual knowledge graphs, further improving prediction accuracy. BioMedGPT [119] expands this by incorporating both protein sequences and textual knowledge for biomedical question answering, enabling improved understanding and reasoning in the biomedical domain. PROTLLM [236] and ProLLaMA [122] bridge protein sequence understanding and generation tasks, with ProLLaMA excelling in multi-task learning, particularly in protein structure and function prediction. InstructProtein [171] aligns protein sequences with natural language through knowledge-guided instructions, improving task handling.

Other models such as DrugGPT [94] and ESM-AA [224] target drug design and molecular modeling, tackling ligand generation and protein interaction analysis. BioT5 [135] and BioT5+ [134] integrate molecular properties with text for multi-task protein understanding. OntoProtein [212] fuses Gene Ontology with sequences to improve function prediction (e.g., GO-CC/GO-BP). Galactica [155] trains on a curated scientific corpus for multimodal reasoning, outperforming GPT-3 on LaTeX and PubMedQA. For multimodal protein tasks, BioBRIDGE [174] links unimodal biomedical models via knowledge graphs to predict drug–target and protein–protein interactions. xTrimoPGLM [22] unifies protein understanding and generation, achieving state-of-the-art results. ProteinChat [71] conditions on sequences and text prompts to describe protein functions in free-form and classification settings. LLaPA [230] combines sequences, PPI networks, and instructions for multi-label PPI and multi-protein affinity prediction. Lastly, MProt-DPO [36] employs Direct Preference Optimization to surpass the ExaFLOPS barrier in protein design, improving efficiency. Collectively, these models showcase the power of MLLMs that couple sequences with text for protein design, function prediction, and interaction analysis.

4.3. MLLMs for Protein Structure–Sequence–Language Integration

Given the critical role of geometric information in understanding protein behavior, recent research has increasingly focused on integrating structural modalities into MLLMs [47,58,92,99,144,153,163,168,187,190,231,235]. Several representative models—including ESM3 [58], DPLM2 [168], FoldToken [47], ProTokens [99], Saprot [153], and ProSST [92]—incorporate protein structural information using various tokenization strategies. Compared to other models, ESM3 [58] incorporates additional functional tokens designed to support specific protein function design tasks. DPLM2 [168] leverages a GVP-based encoder and an IPA-based decoder to learn structural tokens, fine-tuned from DPLM [169], and achieves strong performance in generative tasks. ProTokens [99] employs an SE(3)-invariant transformer to obtain latent structural representations, which are then quantized into discrete tokens that capture structural features. FoldToken [47], identifies the limitations of classical quantization approaches and proposes three custom-designed quantizers, whose effectiveness is validated through experimental evaluation. Saprot [153] constructs structure-aware tokens with the aid of Foldseek [161] and performs well across various downstream tasks. ProSST [92] differs from previous models by constructing a local structure codebook that captures contextual information beyond individual residues and introducing a sequence–structure disentangled attention mechanism, which is validated through ablation studies.

Beyond tokenization-based approaches, other MLLMs integrate structural information primarily through encoders and align the resulting representations with corresponding sequences or textual data. Models such as ProtChatGPT [163], ProteinGPT [190], STELLA [187], InstructBioMol [235], Evolla [231], and ProseLM [144] exemplify this strategy. The overall architectures of ProtChatGPT [163], STELLA [187], InstructBioMol [235], and ProteinGPT [190] are similar, as they all utilize protein structure encoders. However, ProtChatGPT uniquely incorporates a second protein structure encoder to enhance structural feature extraction, while InstructBioMol adds an additional molecular encoder to integrate molecular information. ProseLM [144] employs a causal encoder that integrates structural and functional contexts, successfully designing a PD-1 binder with a binding affinity of 2.2 nM. Evolla [231] also integrates structural information through protein encoders; however, its distinguishing feature is the use of Direct Preference Optimization (DPO) [138] as a post-pretraining method. The model is primarily designed for protein-related question answering tasks.

4.4. MLLMs for Protein Interactions and Specialized Applications

Understanding protein–protein interactions (PPIs) [131] is critical for elucidating protein function, and several MLLMs have been developed for this task. LLaPA [230] integrates protein and graph encoders with a language model in a multimodal fusion framework, while BioBRIDGE [174] links diverse biological modalities through a knowledge graph, both achieving strong PPI performance. Although BioT5 [135] and BioT5+ [134] were not explicitly designed for interaction prediction, they still perform competitively on PPI benchmarks. Beyond interaction tasks, multimodal translation is another emerging direction: MolBind [189] supports protein-related zero-shot cross-modal retrieval, and BioTranslator [192] converts free-text descriptions into biological representations across modalities, enabling more flexible interaction with scientific data.

Collectively, these advances highlight the growing potential of MLLMs to unify heterogeneous protein modalities, enabling more accurate prediction, versatile design, and broader applications in protein science.

5. MLLMs for Genomics and Gene

MLLMs and LLMs are rapidly advancing genomics by enabling tasks such as sequence modeling, gene function prediction, functional annotation, and knowledge retrieval. Compared to traditional computational approaches, these models offer greater flexibility, interpretability, and the ability to integrate heterogeneous biological data [26,68,75]. In this section, we review recent progress from two perspectives. First, we introduce LLMs for genomics, covering their applications in molecular and drug design, functional annotation, gene and variant prioritization, regulatory network modeling, and sequence-level protein or gene tasks. Second, we focus on MLLMs for genomics and gene function prediction, highlighting how multimodal integration of sequences, biological data, and language enables richer reasoning, interpretable predictions, and generalist genomic analysis. Table A3, Table A10, Table A11 and Figure 6 summarize models, datasets, and the research landscape.

5.1. LLMs for Genomics

LLMs are rapidly transforming bioinformatics and genomics, with applications spanning molecular and drug design, functional annotation, gene and variant prioritization, regulatory network modeling, sequence analysis, and synthetic data generation [21,26,64,68,75,159]. In molecular design, models such as GexMolGen [26] align gene expression features with chemical structures to enable gene-guided de novo molecule generation. For functional annotation and knowledge retrieval, LLMs are evaluated on summarizing gene sets [68], discovering gene–disease associations [21], and augmenting biomedical search with APIs [75], while GeneTuring [64] provides systematic benchmarks. In gene and variant prioritization, LLM-based approaches [93,95,159] integrate literature, biological data, and phenotypes to rank causative genes, with automated pipelines supported by API-driven workflows [79,80]. For network modeling, LLMs aid cancer driver gene discovery [208] and reconstruct regulatory networks from single-cell and multi-omics data [170]. In sequence-level tasks, models like ProGen [124] generate functional proteins, while others annotate genes and structures directly from sequence data [3,38,105,148,234]. Beyond these, LLMs support antimicrobial resistance prediction [202], variant effect modeling [61], and even generate synthetic training data for fine-tuning and benchmarking [125]. Together, these studies highlight the broad and transformative role of LLMs in genomics, offering new levels of automation, accuracy, and creativity for precision medicine.

5.2. MLLMs for Genomics and Gene Function Prediction

The integration of MLLMs into genomics has introduced a transformative paradigm for gene function prediction, gene expression modeling, and broader biological tasks [10,35,63,113,126,141]. Traditional methods based on sequence homology, ontology classification, or narrow supervised models often lack flexibility and interpretability. In contrast, MLLMs enable free-form reasoning and cross-modal understanding. For example, GeneChat [35] reframes gene function prediction as a language generation task, combining DNABERT-2 [232] as a gene encoder with Vicuna-13B [29] as a decoder to produce rich natural-language descriptions from raw DNA input. Extending this idea, Geneverse [113] provides a suite of open-source models tailored to genomic and proteomic data, demonstrating strong results in gene/protein function summarization and spatial transcriptomics. ChatNT [141], built on the Nucleotide Transformer [31], supports unified instruction-based inference across DNA, RNA, and protein tasks, making advanced analyses more accessible. Other methods, such as GTA [63] and GeneBERT [126], further improve regulatory modeling by aligning sequence features with language embeddings or leveraging multimodal pretraining. Despite ongoing challenges—such as limited annotations and multimodal heterogeneity—these advances highlight the potential of MLLMs as generalist, interpretable, and conversational engines for genomics and molecular biology [10].

6. MLLMs for Material Science

The use of MLLMs in materials science is still at an early stage but shows strong potential. By integrating text (1D), images (2D), and geometric structural data (3D), these models promise to accelerate material discovery, property prediction, and design optimization [4,11,15,136]. In this section, we review progress from two angles: (1) we discuss LLMs for material discovery, highlighting their role in crystal structure generation, property prediction, and inverse design. (2) we turn to MLLMs for material discovery, where multimodal fusion of textual, visual, and structural representations further enhances property estimation, data extraction, and design pipelines. Table A4 and Figure 6 summarize models and the research landscape.

6.1. LLMs for Material Discovery

Recent advancements show that LLMs can significantly aid materials discovery by generating crystal structures, predicting properties, and supporting inverse design [7,24,32,54,56,72,104,151,172,194,195]. CrystaLLM [7] autoregressively generates CIF sequences to produce plausible crystal structures. MatterGPT [24] targets properties such as formation energy and band gap and enables multi-property inverse design, demonstrating control over both lattice-insensitive and lattice-sensitive attributes [24]. LLMatDesign [72] provides an agentic, iterative framework where LLMs propose material modifications, while domain-aware prompt engineering further boosts property prediction [104]. FlowLLM [151] couples LLMs with Riemannian Flow Matching to refine representations and generate stable, novel materials. CrystaltextLLM [56] fine-tunes LLMs by encoding atomistic data as text and using energy calculations for stability prediction. [32] demonstrate ChatGPT’s ability to suggest compositions and processing routes, accelerating design. GenMS [195] combines language conditioning with diffusion to generate low-energy crystal structures, and Mat2Seq [194] offers SE(3)- and periodic-invariant crystal sequences for robust LM generation. Finally, studies on material selection show that prompt-refined LLMs can assist decisions by comparing expert recommendations [54]. Collectively, these advances expand the searchable chemical space and strengthen data-driven materials design.

6.2. MLLMs for Material Discovery

The integration of MLLMs into materials science is advancing rapidly for discovery and property prediction [4,11,15,136]. A key direction is multimodal fusion of text, images, and molecular representations; for example, LLM-Fusion [11] flexibly ingests SMILES/SELFIES/fingerprints to enhance property prediction over unimodal baselines. Cephalo [15] applies vision–language integration to bio-inspired materials, combining images and text from documents and experiments for property estimation and design optimization. MaCBench [4] identifies current limitations—especially spatial reasoning and cross-modal synthesis—highlighting the need for stronger multimodal reasoning. Recent work also targets automatic extraction of materials data from literature and visual content to enable scalable prediction [136]. Overall, these multimodal approaches are poised to transform materials discovery by enabling robust, data-driven design pipelines for both research and industrial applications.

7. MLLMs Bridging Molecular Science and Biomedicine

The biomedical field encompasses a vast array of disciplines, from fundamental biological research to complex clinical applications [164], and naturally involves a variety of data modalities, amog which analyses of molecules, proteins, genes, and cells play a crucial role. MLLMs have opened new possibilities for integrating heterogeneous biomedical data, enabling not only multi-molecular data fusion [97,113] but also the combination of microscopic-level data(e.g., molecular or cellular information) with macroscopic-level data such as pathology images [100,193], offering valuable insights into disease machanisms and improving diagnostic accuracy. In this section, we primarily focus on the recent surge of studies employing MLLMs to integrate molecular science with biomedicine,along with their methodological approaches. Table A5 summarizes the models discussed in this section. Based on existing advancements, we discuss the limitations identified and outline future directions for further integrating molecular science into biomedicine.

7.1. LLMs for Biomedicine

Genomic, epigenetic, and transcriptomic analyses such as gene pathway finding, gene expression analysis, and so on, greatly facilitate our understanding of biological processes and mechanisms in both normal organism development and disease [173]. These sequences modalities are escpecially suitable for LLMs to process. Some methods [2,173] integrates domain knowledge and study context into LLMs to enable gene analysis at different levels of granularity. Specifically, [173] focuses on gene set enrichment analysis to explicitly consider gene interactions and regulatory relationships within gene sets, while [2] aims to infer gene regulatory networks (GRNs). Together, these approaches facilitate the characterization of caner-related pathways and the elucidation of disease mechanisms, ultimately aiding the idendification of effective treatments. In more recent applications, GenoMAS [103] orchestrating six specialized LLM agents, each contributing complementary strengths to a shared analytic canvas, is applied to gene expression analysis which exposes biologically plausible gene-phenotype associations corroborated by the literature.

7.2. MLLMs for Cross Modal Tasks

With the advent of MLLMs, it has become possible to analyze biomedical problems from multiple perspectives — not only at the macroscopic level (e.g., images and audio) but also at the molecular level. Unlike traditional multimodal fusion approaches [19,127,145], which rely on human-designed summarization, MLLMs can autonomously provide highly interpretable insights and handle cross-modal tasks such as visual question answering and report generation.

(1) Multi-omics Fusion Models. Combining omics data into biomedical research has achieved some success [39]. Current research primarily focuses on developing methods to effectively harmonize diverse omics modalities [200]. One line of research leverages the intrinsic capability of MLLMs to directly fuse heterogeneous omics data, such as genes, molecules, and proteins. Geneverse [113] fine-tunes LLaVA by incorporating protein structural information, gene expression profiles, and functional descriptions as inputs. BioMedGPT [119] further integrates a broader range of biomedical modalities with different encoders, unifies the feature spaces of molecules, proteins, and natural language through encoding and alignment. Another line of research first transforms different modalities into a shared representation before feeding them into MLLMs. LLaMA-Gene [96] trains a single BPE (Byte Pair Encoding) tokenizer to encode genes, proteins, and natural language sequences without additional markers and further converts gene-related task data into a unified format for instruction fine-tuning, constructing a unified model for diverse gene tasks. Collectively, these works support downstream applications such as protein identification and marker gene discovery with the potential to greatly accelerate the discovery of new drugs and therapeutic targets.

(1) Richer Multimodal Fusion in Biomedicine. At the same time, beyond exploring modality fusion within a specific domain or dimension, there have been growing efforts to integrate a broader range of modalities. For example, multi-omics data are fused with cell even organ type data, offering more subtle information about the condition. OmniCellTOSG [210] encodes textual annotations with an LLM and leverages a graph neural network (GNN) to capture the topology of signaling(TOSG) networks labeled with annotations like organ, cell subtype, and quantitative gene and protein data. By integrating these two representations, it constructs patient-specific single-cell TOSG maps, thereby enabling precise cell classification, cancer cell state prediction, and other clinically relevant tasks transforming research in life sciences, healthcare, and precision medicine. SpaLLM [91] combines LLM representations from single-cell transcriptomics with spatially resolved multi-omics data (e.g., RNA, chromatin accessibility, proteins), enabling precise identification of functionally specialized cell types, providing essential molecular and spatial references for disease diagnosis. Recently, another popular direction in MLLM-based research has been to leverage spatial transcriptomics (ST) technologies, which provide both molecular signatures and the spatial localization of cells within tissues. ST-ALign [100] leverages ST technology to achieve fine-grained alignment between histological morphology and molecular features, including image–gene alignment at both the spot and niche levels, following by an Attention-Based Fusion Network used to fuse visual and genetic features. Extending spatial transcriptomics to pathology, mSTAR and spEMO [112,193] integrate microscopic slides, macroscopic reports, and gene expression via multi-level alignment into a pathology foundation model, enabling tasks such as diagnosis, molecule prediction, survival analysis, and report generation. Furthermore, spEMO introduces the novel task of multimodal alignment, offering a new perspective to evaluate information retrieval ability and guide the development of future pathology foundation models.

7.3. Outlook

Although MLLMs have begun to explore the integration of multiple modalities, current progress remains at an early stage. For instance, while some models [91,96,113] have been trained on multi-omics data simultaneously, few are capable of jointly processing image-based data, largely due to the weak consistency across such heterogeneous modalities. integrating more diverse data types thus remains challenging. A few models, such as [112,193], have attempted to combine pathological images with genomic information for disease diagnosis, but such approaches are still limited. There remains a clear need for more comprehensive methods that effectively integrate diverse multimodal data in the future. A promising direction for sustainable progress is to curate large-scale, comprehensive multimodal benchmarks and datasets to facilitate the development of future methods.

8. Emerging Hot Topics and Future Directions

In this section, we (1) examine several emerging hot topics, with a particular focus on diffusion-based paradigms that are reshaping large language models and their multimodal extensions, and (2) discuss future directions in scientific applications of MLLMs, covering domain-specific challenges and opportunities across molecular science, protein modeling, materials discovery, and genomics.

8.1. Emerging Hot Topics

The rapid progress of large language models has spurred a new wave of research into alternative training and decoding paradigms, as well as extensions to multimodal understanding and generation. In this section, we highlight two directions that have recently gained considerable momentum. The first is diffusion large language models (dLLMs), which replace the conventional autoregressive decoding strategy with an iterative mask–denoise process and have shown promising advances in reasoning, controllability, and efficiency. The second is diffusion multimodal large language models (dMLLMs), which extend this paradigm to vision, audio, and other modalities, enabling more flexible cross-modal reasoning and structured generation. Together, these emerging topics illustrate how diffusion-based methods are shaping the future landscape of language and multimodal modeling.

8.1.1. Diffusion Large Language Models

dLLMs replace the traditional left-to-right next-token prediction paradigm with a mask-and-denoise process over discrete tokens. Instead of generating text sequentially with unidirectional attention, dLLMs begin from a heavily masked (or absorbed) sequence and iteratively denoise it using bidirectional attention. This design enables parallel decoding of many tokens at once, providing explicit trade-offs between quality, latency, and controllability through adjustable steps and scheduling [51,115,150,205,225]. Compared with autoregressive (AR) models, which suffer from rigidity in mid-sequence editing and lack global structural control, diffusion-based decoding offers greater flexibility and coherence.

Figure 7. The comparison between predominant Autoregressive language models and Diffusion language models. In autoregressive models, the model generates text sequentially from left to right using “next token prediction,” and the generated length is unrestricted. In contrast, diffusion language models generate text by randomly masking and predicting masked tokens, which are not constrained by spatial position but typically produce sequences of fixed length.

Figure 7. The comparison between predominant Autoregressive language models and Diffusion language models. In autoregressive models, the model generates text sequentially from left to right using “next token prediction,” and the generated length is unrestricted. In contrast, diffusion language models generate text by randomly masking and predicting masked tokens, which are not constrained by spatial position but typically produce sequences of fixed length.

(1) Core Mechanics. The forward process in dLLMs typically applies random masking or absorbing states, while the reverse process learns to reconstruct clean tokens from noisy inputs. Recent advances, such as reparameterized discrete diffusion (RDM), reduce training variance and enable confidence-aware decoding by prioritizing high-confidence tokens during generation [225]. Training objectives span from NLL-equivalent token prediction to reweighting strategies at the token or sequence level. For example, multi-granularity diffusion (MGDM) emphasizes difficult tokens and subgoals to enhance complex reasoning [198]. At inference, specialized schedulers such as dilated unmasking explicitly minimize conditional entropy in each round, thereby reducing the number of iterations [121].

(2) Scaling Strategies. Two main approaches have emerged for scaling dLLMs. The first is training from scratch, exemplified by LLaDA, which pre-trains an 8B-parameter diffusion LLM on 2.3T tokens and demonstrates competitive or superior performance to comparable AR baselines, particularly on reversal-style tasks that reveal AR brittleness [129]. The second strategy adapts pretrained AR models by gradually relaxing the causal mask and shifting prediction targets, yielding variants such as DiffuGPT & DiffuLLaMA that achieve strong zero/few-shot and fill-in-the-middle abilities with significantly reduced training cost [50].

(3) Capabilities and Directions. Diffusion decoding has opened new research avenues across multiple fronts: (i) Reasoning and planning. Diffusion-of-Thought supports parallelized chain-of-thought and multi-step self-correction [199], while MGDM reports substantial improvements on tasks such as Countdown, Sudoku, and SAT [198]. Recent work like d1 combines supervised fine-tuning with a diffusion-compatible policy-gradient method (diffu-GRPO), further improving math, logic, and coding performance [222]. (ii) Program synthesis and structured generation. DiffuCoder introduces analysis tools for “AR-ness” of dLLMs and a coupled-GRPO RL procedure, matching or beating similar-sized AR coders on several leaderboards [52]. For controllable outputs (JSON/tables), the S3 scaffolding method uses schema templates and null tokens to achieve high structural validity without retraining [191]. (iii) Seq2Seq and one-step generation. DiffuSeq extends diffusion to conditional text generation [51]. DLM-One distills iterative denoising into a single forward pass via score-based distillation—reporting up to 500× speedups on classic Seq2Seq tasks at near-teacher quality [23]. (iv) Systems & efficiency. At inference, dilated unmasking reduces rounds from $O\left(B\right)$ to roughly $O(logB)$ per block [121]; Fast-dLLM adds block-wise KV caching plus confidence-gated parallel decoding, reporting up to 27.6× speedups with minimal accuracy loss [180]. Block diffusion interleaves AR across blocks with diffusion within blocks, closing perplexity gaps while preserving parallelism [8]. (v) Industrial interest. Google DeepMind’s Gemini Diffusion signals growing product-level exploration of text diffusion [53].

(4) Safety Outlook. The novel dynamics of dLLMs introduce distinct safety challenges. Parallel decoding and mask-aware mechanisms create new attack surfaces, and recent jailbreak methods such as PAD and DIJA achieve high success rates across multiple diffusion models [178,220]. These results suggest that AR-based defenses cannot be directly applied, underscoring the need for diffusion-native alignment and guardrails.

(5) Takeaway. dLLMs combine parallelism, global coherence, and fine-grained controllability, positioning them as a promising alternative—and in some regimes, a superior paradigm—to autoregressive models [205]. With both training-from-scratch and AR-adaptation paths maturing, and with rapidly improving inference-time efficiency, dLLMs are evolving from niche prototypes to competitive large-scale systems.

(6) Open Problems and Future Directions. Key challenges remain: (i) establishing theoretical foundations for scheduling, convergence, and optimality; (ii) developing scalable diffusion-native alignment and RLHF methods [222]; (iii) hybridizing diffusion with AR, retrieval, and external tools [8,198]; (iv) designing standardized evaluation protocols for latency–quality trade-offs and structural validity; (v) advancing security via mask-aware defenses and robust red-teaming [178,220]; and (vi) optimizing serving systems for KV-cache consistency, adaptive decoding, and distributed/edge deployment [121,180].

8.1.2. Diffusion Multi-Modal Large Language Models

dMLLMs are also attracting increasing attention in the multimodal domain. Compared to autoregressive approaches, iterative mask–denoise refinement provides global context modeling, parallel token prediction, and natural support for structure priors (e.g., layouts, JSON schemas) as well as fill-in-the-middle editing. These properties make diffusion particularly suitable for vision–language, audio–language, and other structured multimodal tasks, while offering explicit quality–latency trade-offs through the choice of denoising steps [205].

(1) Representative Models. Several recent systems demonstrate the potential of diffusion in multimodal scenarios. (i) Vision–language. Llada-v extends LLaDA with visual instruction tuning while retaining diffusion-style parallel decoding, enabling visual question answering and multimodal dialogue [203]. Dimple adopts a two-stage training paradigm: an initial AR phase aligns vision and text representations and supports instruction following, after which diffusion decoding is reinstated to recover parallelism and structural control. At inference, Dimple incorporates confident decoding and explicit structure priors (e.g., JSON length control), achieving state-of-the-art results with fewer denoising steps (often less than one-third of the response length) [206]. (ii) Audio–language. DIFFA freezes Whisper and a diffusion LLM backbone, training only lightweight dual adapters (semantic and acoustic). This adapter-based design yields strong performance across multiple audio–language benchmarks at modest data and compute cost, highlighting the efficiency of multimodal diffusion tuning [229]. (iii) Broader ecosystem. Beyond academic prototypes, Gemini Diffusion illustrates early integration of diffusion-style generation into large-scale product pipelines, signaling practical interest in retrieval- and tool-augmented multimodal agents [53].

(2) Capabilities and Engineering Patterns. Diffusion multimodal models inherit many of the strengths of their text-only counterparts. (i) Controllability and structure. By conditioning on scaffolds such as schemas or layouts, these models substantially reduce format errors and hallucination in chart/table reasoning and structured generation; S3-style prompting can be readily reused in multimodal contexts [191,206]. (ii) Throughput and latency. Inference accelerations developed for dLLMs, including KV-cache reuse, confidence-gated parallel decoding, and dilated scheduling, transfer cleanly to vision and audio modalities [121,180]. (iv) Applications. Iterative refinement proves beneficial for fact-faithful summarization (Arg-LLaDA) and for constrained scientific design/optimization where diffusion acts as a constrained sampler over feasible manifolds [82,88]. Other applications include controllable user-facing content generation such as poll/question generation with attribute control [27].

(3) Risks and Challenges. Despite these advances, several challenges remain open. (i) Security. Mask-aware, parallel denoising can amplify multimodal jailbreak attacks, including cross-modal prompt mixing and masked injection; diffusion-native safeguards are still underdeveloped [178,220]. (ii) Long-context efficiency. Processing long videos or extended speech raises issues of memory and cache consistency across denoising steps, requiring more principled architectural solutions [121,180]. (iii) Data and alignment. High-quality multimodal instruction data remain scarce; balancing frozen-backbone adapters (e.g., DIFFA) with full-parameter training (e.g., Dimple) is still an open question for efficient scaling [206,229].

(4) Future Directions. Promising research avenues include: (i) designing unified diffusion agents that couple vision, audio, and text with retrieval and tool use; (ii) developing verifiable generation under hard structure/layout constraints; (iii) scalable alignment via multimodal preference modeling and reinforcement learning for diffusion; (iv) building diffusion-native defenses and safety benchmarks; and (v) systems co-design for efficient step-adaptive serving, block-wise diffusion, and distributed or edge inference [8,121,180,191].

9. Selected Benchmarking Evaluation

9.1. Molecular Property Prediction

Experiment setting. We evaluate on the MoleculeNet benchmark [183], which comprises three single-modal binary classification datasets for assessing the expressiveness of pretrained molecular representation methods. Performance is reported as the area under the receiver operating characteristic curve (AUROC).

Benchmarking Models. We identify several MLLMs, including InstructMol [18], MoleculeSTM (Graph) [110], MoleculeSTM (Smiles) [110], GIT-Mol [107], Token-Mol [166], and M3LLM [66], which target the downstream task of molecular property prediction. For non-MLLM models, we adopt the results reported in the InstructMol paper [18]. Since the model weights of InstructMol, M3LLM, and GIT-Mol are not publicly available, we rely on the reported results of InstructMol from the original paper, while M3LLM and GIT-Mol are excluded from our evaluation. For the remaining models, we rerun the experiments ourselves.

Analysis. Overall, as show in Table 3, the results show that MLLM-based models achieve competitive performance in molecular property prediction, but they generally lag behind strong specialist models such as Uni-Mol and MolFM. Among the evaluated MLLMs, Token-Mol and MoleculeSTM (Smiles/Graph) consistently perform comparably, while other generalist LLM-based methods (e.g., Galactica and Vicuna variants) exhibit significantly weaker performance across all tasks. InstructMol demonstrates strong results as reported in the original paper, though its lack of released weights prevents direct reproducibility. Notably, Token-Mol achieves results that are on par with MoleculeSTM, indicating that specialized adaptation of MLLMs can substantially narrow the performance gap with task-specific molecular models.

9.2. Protein Property Prediction

Experiment setting. We evaluate models on the TAPE benchmark [139] to assess their capability in protein property prediction across six tasks: secondary structure(SS) prediction, contact prediction, homology prediction, fluorescence prediction and stability prediction. Secondary structure and homology prediction are multi-label classification tasks with accuracy used as the evaluation metric. Contact prediction is performed using the precision of the top $L/2$ predicted contacts, where L denotes the sequence length, focusing on medium- and long-range interactions. Fluorescence prediction aims to predict the logarithm of a protein’s fluorescence intensity, while stability prediction estimates a proxy for protein stability. Both tasks are evaluated using Spearman’s rank correlation coefficient($\rho$). Benchmarking Models. We identify OntoProtein [212], ProtBERT [?], and ProteinDT [108]. For non-MLLM models, we adopt the results reported in the ProteinDT [108].

Analysis. As shown in Table 4, traditional baselines such as TAPE Transformer, and MSA Transformer perform moderately, while specialist models like ProtBERT and OntoProtein achieve stronger results. ProteinDT further improve performance across most tasks.

10. Conclusion

This work provides a comprehensive overview of recent advances in MLLMs for science, highlighting representative architectures, datasets, and benchmarks, as well as their emerging applications in science. Beyond cataloging progress, we also emphasize the growing role of diffusion-based LLMs in multimodal generation and reasoning. Looking ahead, MLLMs hold the potential to reshape the way scientists explore and integrate diverse data sources. Continued progress will require addressing open challenges in factual reliability, modality-specific reasoning, interpretability, and ethical deployment. By synthesizing current advances and pointing toward future directions, this work aims to serve as both a reference and a foundation for further research in multimodal scientific AI.