Generative AI for Drug Discovery and Protein Design: The Next Frontier in AI-Driven Molecular Science

1. Introduction

Drug discovery is traditionally costly, slow, and failure-prone (1). Preclinical discovery takes over five years, consuming one-third of total costs (2). With fewer than 10% of candidates succeeding, R&D expenditure per new drug exceeds $2 billion, mainly due to failures (3,4). Many failures stem from safety/efficacy issues emerging late (5). AI has recently accelerated in silico modeling across the pipeline, improving QSAR-based virtual screening and ML-driven protein engineering (6).

Historically, rule-based de novo drug design (e.g., LUDI, PRO_LIGAND) explored limited chemical space due to human bias (6,7,8). Generative AI overcomes this by learning molecular patterns and creating novel compounds (10). Unlike classical methods that recombine known motifs, it explores uncharted chemical space (11). Given an estimated >10⁶⁰ drug-like molecules, AI efficiently samples viable candidates via chemical manifolds (12). AI designs millions of molecules in the time it takes for manual design, optimizing multiple properties simultaneously (13). Recent pipelines enhance synthetic feasibility and drug-likeness.

AI-driven discovery integrates generative models with computational chemistry, transitioning from empirical screening to rational design (14). Mid-20th-century drug discovery relied on trial-and-error; later, structure-based (X-ray) and ligand-based (pharmacophore) methods emerged (15). The 2000s saw docking and QSAR improvements (16). AI now creates novel molecules, with the first AI-designed drug (DSP-1181) entering trials in 2020 (17).

In AI workflows, generative models design de novo molecules, filtered via predictive models (binding affinity, ADMET) (18,19). Top hits undergo docking, synthesis planning, and wet-lab validation. Retrosynthesis AI suggests lab synthesis routes, while experimental feedback refines models (20). AI continuously self-improves, navigating chemical space intelligently. Future directions include autonomous discovery, quantum computing, and regulatory frameworks (21).

2. Deep Generative Models: Core Architectures

Deep generative models enable de novo molecular design by learning statistical patterns in chemical datasets to generate novel, valid compounds. The five principal architectures and optimization strategies are illustrated in Figure 1, providing a conceptual overview of their roles in molecular generation

2.1. Variational Autoencoders (VAEs)

A VAE comprises an encoder that compresses a molecule (typically represented as a SMILES string or molecular graph) into a continuous latent vector, and a decoder that reconstructs the molecule from this vector (22). Trained on large compound datasets, the encoder maps structurally similar molecules to proximate points in latent space, effectively learning a continuous chemical manifold (23). The training objective is to maximize data reconstruction likelihood while enforcing the latent vectors to follow a smooth, multi-dimensional Gaussian distribution using Kullback–Leibler (KL) regularization, enabling meaningful interpolation (24). The model optimizes the evidence lower bound (ELBO):

$$L_{VAE}=E_{q_{\mathrm{\varnothing }}\left({\displaystyle \frac{z}{x}}\right)}\left[\ln p_{\theta }\left({\displaystyle \frac{x}{z}}\right)\right]-D_{KL}(q_{\mathrm{\varnothing }}({\displaystyle \frac{z}{x}})||p\left(z\right))$$

Where $q_{\varnothing }\left({\displaystyle \frac{z}{x}}\right)$ is the encoder (approximate posterior), and $p_{\theta }\left({\displaystyle \frac{x}{z}}\right)$ is the decoder (likelihood). Once trained, a random vector $z ~ p\left(z\right)$ can be decoded into a novel molecule. Demonstrated for drug-like molecules by Gómez-Bombarelli et al. (2016–2017), VAEs enabled direct optimization in latent space to search for molecules with improved properties (25). The continuous latent space allows smooth interpolation between compounds (chemical morphing) to explore analogs (26). However, early SMILES-based VAEs often generated invalid or implausible structures (27). To address this, chemically informed decoders like Junction-Tree VAE were introduced, generating molecules as a tree of substructures, ensuring valency constraints are satisfied (28). VAEs have also been extended to 3D molecular conformations (23).

VAEs provide a principled framework grounded in Bayesian inference, ensuring training stability and interpretability (29). Despite limitations in output validity and reconstruction bias, VAEs are frequently combined with latent space optimization techniques such as Bayesian optimization or gradient-based methods to identify latent vectors yielding molecules with desirable properties (30). While their abstract latent space can hinder direct property optimization, VAEs remain foundational in molecular generation (31).

2.2. Generative Adversarial Networks (GANs)

GANs approach generation differently. Instead of modeling data likelihood explicitly, GANs set up a two-player game between a generator and a discriminator (32). The generator creates molecules from random noise, while the discriminator distinguishes real from generated samples. Adversarial training forces the generator to produce realistic molecules (33). Applied to molecules in 2018 (e.g., ORGAN by Insilico Medicine), GANs used an RNN generator to output SMILES strings and a discriminator that rewarded drug-like outputs (34). Conditional GANs guide generation towards desired properties (e.g., target binding) by conditioning on context (35).

Figure 1. Architectures of deep generative models and latent space optimization in molecular design. (A) Variational Autoencoders (VAEs), (B) Generative Adversarial Networks (GANs), (C) Transformer-based models, and (D) Denoising Diffusion Models generate molecules using distinct mechanisms. (E) Latent space optimization explores continuous chemical manifolds to design molecules with desired properties.

Figure 1. Architectures of deep generative models and latent space optimization in molecular design. (A) Variational Autoencoders (VAEs), (B) Generative Adversarial Networks (GANs), (C) Transformer-based models, and (D) Denoising Diffusion Models generate molecules using distinct mechanisms. (E) Latent space optimization explores continuous chemical manifolds to design molecules with desired properties.

GANs face challenges in molecular domains due to discrete outputs: the generator’s character-sequence output is non-differentiable. Solutions include policy gradient reinforcement learning and differentiable relaxations. Insilico’s Adversarial Threshold Neural Computer integrated GANs with reinforcement learning, using a differentiable neural computer as the generator and providing external rewards based on pharmacological properties (36). This hybrid generated a high percentage of valid, unique, and property-optimized molecules, while also incorporating synthesizability constraints (37). MolGAN, another milestone, generated molecular graphs (atom and bond matrices) directly. It achieved nearly 100% validity, improved synthetic accessibility, and solubility profiles compared to ORGAN (38).

Despite these advances, GANs may suffer from mode collapse and training instability. Their learned distribution might not cover the full chemical space (39). However, conditional GANs remain powerful for generating analogs of lead compounds (40). Overall, GANs introduce adversarial learning into molecular design, emphasizing realistic outputs and targeted objectives, though maintaining output diversity and validity requires care.

2.3. Transformer-Based Models

Transformer-based models, originally developed for NLP, are also applied to molecular science (41). Their self-attention mechanism captures long-range dependencies in sequences. For small molecules (SMILES or SELFIES) and protein sequences, transformers treat chemistry as a language. For example, ChemBERTa, trained on millions of SMILES strings using masked token prediction, produces rich, label-free molecular embeddings suitable for downstream tasks such as property prediction or generative modelling (42,43).

Transformers can generate molecules token-by-token (GPT-style), learning chemical syntax analogous to grammar. Such models can be conditioned to bias outputs toward specific property profiles (44). In lead optimization, transformers have proposed structural analogs based on known series (45). Protein language models (e.g., ProGen with 1.2B parameters trained on ~280M protein sequences) treat amino acid sequences like sentences. ProGen has generated functional enzymes with catalytic activity comparable to natural lysozymes, despite ~30% sequence identity. X-ray crystallography confirmed correct folds and active-site geometries (46,47). Transformers also support sequence-to-sequence tasks like codon optimization or property prediction (48).

These models bridge sequence and structure, enabling protein and molecule generation via attention-based encoding of complex dependencies. They leverage large unlabeled datasets for self-supervised learning, yielding representations useful for property prediction, structure generation, and analog design (49,50).

2.4. Denoising Diffusion Models (DDPMs)

DDPMs represent the latest generative modeling wave. They iteratively corrupt data with Gaussian noise and learn to reverse this process. A forward Markov chain adds noise over T steps until the sample becomes pure noise. A neural network then learns to reverse the corruption by predicting denoised data at each step. Training minimizes a reweighted variational bound, typically reducing to the loss between predicted and true noise, equivalent to learning the score function $\nabla \log p\left(x_t\right)$ (51).

Generation begins from random noise and progressively reconstructs data, enabling generation in the original space (e.g., 3D coordinates of atoms) rather than latent space. This supports highly diverse and high-quality outputs (52,53). Diffusion models have been applied to 2D molecular graphs, 3D conformations, and protein structures (54). For instance, graph diffusion models like GeoDiff (55) and RFdiffusion (56) add noise to adjacency and node feature matrices or 3D coordinates and reconstruct valid molecular structures, preserving symmetry and chemical rules. DiffDock (57), using SE(3)-equivariant diffusion, generates ligand poses in binding sites by diffusing atomic positions.

Mathematically, as $T\to \infty$, the model can approximate any data distribution, offering theoretical guarantees absent in VAEs or GANs (58). Though generation is slow due to multiple neural evaluations, recent innovations like DDIMs have reduced required steps. Diffusion models enable unconditional generation with high validity and conditional generation guided by context (e.g., pharmacophores, protein pockets). RFdiffusion can be prompted with a protein backbone motif to generate a full structure incorporating it, resulting in functional de novo binders (54,56).

VAEs, GANs, transformers, and diffusion models each offer a distinct lens on learning and sampling chemical space (59). VAEs provide continuous latent embeddings and stable training. GANs deliver adversarial realism and property-driven design (60). Transformers model long-range dependencies in molecular/protein sequences, leveraging large datasets. Diffusion models refine samples from noise with high fidelity, especially in complex structured outputs. Modern workflows often combine models: e.g., using a transformer or VAE to generate candidates, then refining with a Diffusion model, or using a GAN to further optimize properties. Hybrid architectures (e.g., Diffusion models using transformers, or VAEs with GAN-style discriminators) are increasingly common (60–64).

2.5. Theoretical Considerations in Chemical Space Exploration

2.5.1. Latent Space Optimization and Chemical Manifolds

Generative models operate in high-dimensional chemical spaces and must ensure output validity and synthesizability, while efficiently identifying rare, high-quality candidates (14). VAEs and certain autoregressive models learn latent chemical manifolds, where distances correspond to structural similarity. In theory, optimizing in this latent space (via Bayesian or gradient methods) enables design of potent analogs near known actives (65–67). Yet, not all latent directions map to valid molecules. Some lead off the learned manifold, producing invalid or strange outputs (68). Techniques like property-conditioned latent spaces and validity filters help mitigate this (69). Alternatively, methods like PASITHEA invert differentiable property predictors to optimize input molecules directly (70).

2.5.2. Validity and Synthesizability Constraints

A well-trained generative model maps continuous latent vectors to discrete molecular space (71). Ensuring this mapping is smooth and chemically realistic remains an open challenge (72). Validity and synthesizability are essential. Early SMILES generators often violated valency rules (73). Modern models use graph-based construction, fragment-based methods (e.g., junction trees), or validity filters, achieving >95% valid molecules (28). Synthesizability remains difficult to measure. Some models use synthetic accessibility scores or predicted retrosynthesis steps as proxies (74). Reinforcement learning agents can be rewarded for generating molecules requiring fewer synthesis steps (75). TRACER, a conditional transformer, generates both molecules and plausible reaction paths using learned transformations, ensuring synthetic feasibility (76).

2.5.3. High-Dimensional Chemical Space

Exploring high-dimensional chemical space is computationally demanding. A typical drug-like molecule has 20–70 heavy atoms, leading to enormous combinatorial possibilities (77). Generative models, trained on bioactive molecules, bias outputs toward favorable motifs (78). From a theoretical view, this is importance sampling: the model learns a distribution q(x) focused on regions where the true utility distribution p(x) is high.

Techniques like reinforcement learning (RL) and Monte Carlo tree search (MCTS) efficiently guide search (79). In RL, the model acts as an agent adding atoms or groups, receiving rewards for desirable properties (e.g., potency, low toxicity), enabling targeted exploration (80). For example, a DDR1 kinase inhibitor was discovered within 21 days using RL-guided generative models (81). Genetic algorithms (GAs), which evolve molecular populations using crossover and mutation, also explore chemical space (82). Modern GAs use neural networks to bias mutations or select crossover points (83).

Exploration balances exploitation (refining known scaffolds) with discovering novel ones. This balance is adjustable via model hyperparameters (e.g., softmax temperature, diffusion noise variance) (84). Some models explicitly incorporate exploration-exploitation tradeoffs, using strategies like Thompson sampling or multi-objective optimization (85).

Generative models must also contend with the curse of dimensionality: high-dimensional property landscapes are complex, with many local optima (86). Generative models, trained on real data, implicitly learn some of this structure. But they rely on property predictors or experiments to evaluate novel molecules. Thus, the best strategy is integrating generation, prediction, and optimization. This closed loop, used in Bayesian optimization and active learning, iteratively improves candidates with fewer evaluations (73,78,87,88).

Navigating chemical space with AI requires smooth, chemically realistic mappings, valid and synthesizable outputs, and strategic exploration (89). Embedding generative models within predictive-evaluative frameworks enables discovery of novel bioactives that would be infeasible by brute force (90). Future models may guarantee validity and bound synthetic complexity, further expanding the reach of AI-driven drug design (91).

3. Generative AI for Molecular Structure Prediction and Optimization

With the foundations in place, we turn to the practical applications of generative AI in designing new molecular structures and optimizing them for drug-like properties. These applications fall into two categories: small molecule design (drug-like compounds) and macromolecule/protein design (biologics, enzymes, antibodies). Generative models are used in conjunction with other AI techniques (self-supervised pretraining or reinforcement learning) to achieve specific goals, such as improving a lead compound’s potency or inventing a protein that binds a given target (14). This section discusses molecular optimization (Section 3.1) and protein design (Section 3.2), highlighting representative methods.

With the foundations in place, we turn to the practical applications of generative AI in designing new molecular structures and optimizing them for drug-like properties (Figure 2).

3.1. AI-Driven Small Molecule Design

Designing small-molecule drugs is a multi-objective optimization problem: achieving potency against the target while satisfying other criteria (selectivity, pharmacokinetics, safety, etc.). Three approaches in AI-driven molecule design are: self-supervised learning of molecular representations, reinforcement learning for goal-directed optimization, and graph-based generative models.

3.1.1. Self-Supervised Learning for Molecular Representations

A critical aspect of molecular optimization is having a rich representation of molecules. Self-supervised learning (SSL) trains models on large chemical databases using tasks like predicting masked atoms or contrastive learning between molecule augmentations (92–94). Models such as ChemBERTa were trained to learn chemical context through a masked token prediction task on millions of SMILES. The resulting models can predict properties or initialize generative tasks (42). For example, transformer models pre-trained to predict missing atoms can generate complete molecules from partial fragments (95). Denoising autoencoders, trained to reconstruct a molecule from a corrupted version, can propose modifications to lead compounds, such as filling in missing parts (96).

Figure 2. Generative AI strategies for molecular and protein design. (A–C) Approaches for small molecule optimization using self-supervised learning (ChemBERTa), reinforcement learning (ReLeaSE), and graph-based models (DeepScaffold). (D–F) Protein design methods, including diffusion models (RFdiffusion), large language models for sequence generation, and applications in antibody and enzyme engineering.

Figure 2. Generative AI strategies for molecular and protein design. (A–C) Approaches for small molecule optimization using self-supervised learning (ChemBERTa), reinforcement learning (ReLeaSE), and graph-based models (DeepScaffold). (D–F) Protein design methods, including diffusion models (RFdiffusion), large language models for sequence generation, and applications in antibody and enzyme engineering.

3.1.2. Reinforcement Learning (RL) for Molecular Optimization:

Generative models can be coupled with reinforcement learning (RL) to optimize objectives (97). Generation of a molecule is treated as a sequential decision process, where a generative model (e.g., an RNN or transformer) chooses actions like which atom to add at each step. A reward function reflects design goals, e.g., high reward for molecules predicted to bind a target and low reward for those likely to be toxic (14,28). ReLeaSE (Reinforcement Learning for Structural Evolution) uses two neural networks—one for generation and one for predicting properties like biological activity (98). RL algorithms like policy gradients or Q-learning guide the model towards molecules with better scores. RL can discover non-intuitive modifications to improve a molecule’s profile (99,100).

3.1.3. Graph-Based Generative Models:

Since molecules are naturally graphs, generative models often use graph neural networks (GNNs). In graph grammar-based approaches, molecules are built by adding atoms or larger fragments to a partial graph (101,102). DeepScaffold uses CNN-based GNNs to add substituents to a predefined scaffold. These models ensure that molecules remain valid and incorporate medicinal chemistry rules (103). Some models allow scaffold hopping, generating a different core structure that still satisfies activity requirements (104). Combined with reinforcement learning, graph-based design is particularly useful in lead optimization, generating analogs of a given lead compound that improve properties (98).

3.2. AI-Driven Protein Design

Designing proteins with specific structures or functions is a grand challenge, and AI is revolutionizing this process. Unlike small molecules, proteins are large macromolecules with complex folding patterns and vast design spaces. Generative AI tackles these problems using techniques like diffusion models for protein structures, large language models for protein sequences, and specialized models for antibody or enzyme design.

3.2.1. Diffusion Models for Protein Folding & Stability Prediction:

Diffusion models, like RFdiffusion, treat the 3D coordinates of a protein backbone as data to be diffused. Starting from a random initial backbone, the model iteratively refines it into a physically plausible structure (56,105). RFdiffusion generates novel protein structures that are computationally predicted to be stable and experimentally verified to fold and function. This model excels at designing symmetric protein assemblies and enzyme active site scaffolds. The success rate of generating foldable proteins was significantly higher than prior methods. Diffusion models incorporate physical and evolutionary constraints, learning rules of protein folding that guide the generation process, leading to more stable and functional designs.

3.2.2. LLMs for De Novo Protein Sequence Generation:

Sequence-based generative models, particularly large language models (LLMs), have opened new pathways in protein engineering (106,107). Trained on large sequence databases (e.g., Uniprot), these models capture evolutionary patterns like motifs and domains that relate to protein function. LLMs can generate protein variants and rank them by their likelihood of being functional. Sampling in high-probability regions produces novel proteins that might not exist in nature. LLMs complement structure-first methods by generating sequences that can be predicted to fold into desired structures using tools like AlphaFold. This reduces the need for extensive wet-lab testing by filtering out likely failures in silico (107–111).

3.2.3. Antibody and Enzyme Design Using AI:

Two significant application areas are antibody design and enzyme design, where generative AI proves highly effective.

• Antibody Design: AI can design antibodies by generating complementarity-determining region (CDR) sequences likely to bind a target antigen or by generating 3D conformations of antibody loops that complement antigen surfaces (112). DiffAb, a diffusion model, generates antibody structures conditioned on the 3D structure of the target antigen’s epitope, effectively growing an antibody loop to fit into the epitope pocket (113). The success of AbSci’s model in creating functional antibodies in silico indicates that these methods can produce viable therapeutic candidates (114).
• Enzyme and Biocatalyst Design: Enzymes catalyze chemical reactions, and AI is transforming enzyme design by improving active site modeling and exploring backbone arrangements. RFdiffusion has been used to design enzyme active sites, with some designs showing promising activity. AI can also optimize existing enzymes by proposing mutations that stabilize them or alter their substrate scope. Generative models can propose multi-enzyme pathways for synthetic routes, offering a new approach to metabolic network design (115–117).

In both antibody and enzyme design, integrating experimental feedback accelerates the process. AI-generated designs are tested through high-throughput experiments, and the resulting data refines the models. This experiment-AI loop is becoming more efficient with automated laboratories that integrate robotics and AI for real-time analysis (118,119).

4. Computational Strategies for AI-Guided Drug–Target Interactions

A critical aspect of drug design is not just proposing molecules, but understanding and predicting how those molecules will interact with biological targets (proteins, nucleic acids, etc.). This involves docking (predicting the binding pose of a ligand in a protein’s active site), scoring and predicting binding affinity, and efficiently searching through large libraries for those interactions (virtual screening). Traditional computational chemistry methods like molecular docking programs and physics-based scoring functions have been standard for decades, but they have limitations in accuracy and speed. AI-driven approaches are now enhancing or outright replacing these steps: diffusion models are redefining molecular docking, deep neural networks are predicting binding affinities with high accuracy, and generative models are enabling ultra-large virtual screens by focusing on the most promising candidates. In this section, we explore how generative AI and related models contribute to drug–target interaction prediction, covering DiffDock and modern docking, binding affinity prediction, and large-scale virtual screening.

4.1. DiffDock and Beyond: AI in Molecular Docking

Molecular docking is the computational prediction of a ligand’s preferred orientation (pose) and position when bound to a target protein, typically an early step in in silico drug screening. Classical docking programs (AutoDock, DOCK, Glide, etc.) use physics-inspired scoring functions to evaluate many possible poses but often treat the protein as rigid and use approximations that can mis-rank good binders. Enter AI: methods like DiffDock have reframed docking as a generative modeling problem. DiffDock uses a 3D diffusion model to generate candidate ligand poses in a given protein binding site. It starts with random orientations and positions, then iteratively “denoises” its translation and rotation, guided by a learned scoring potential to bring the ligand toward likely binding modes.

DiffDock doesn’t use a traditional scoring function; instead, it was trained on a large dataset of known protein–ligand complexes, learning an implicit representation of shape complementarity and interactions. On standard benchmarks, DiffDock significantly outperformed traditional docking tools. For example, at a 2 Å RMSD threshold, DiffDock placed ~22% of predictions within that range, more than double the success rate of traditional methods, which often hovered around ~10%. It also maintained strong performance on difficult cases where other methods failed. This success is attributed to DiffDock’s ability to implicitly account for protein flexibility, learning a distribution of likely poses that might correspond to slight protein side-chain movements, something rigid docking struggles with.

DiffDock includes a confidence model that estimates the reliability of its predicted pose. This confidence score correlates well with pose accuracy, helping to prioritize high-confidence predictions. DiffDock also provides speed combined with accuracy, enabling high-throughput docking campaigns. Additionally, as a generative model, DiffDock produces multiple plausible poses, reflecting possible binding modes or tautomeric states of the ligand, giving medicinal chemists a richer view of ligand binding. The DiffDock approach, along with AI-enhanced virtual screening and affinity prediction strategies, is illustrated in Figure 3.

Beyond DiffDock, other AI methods like EquiBind (120), a one-shot GNN-based method, also show promise, though DiffDock’s diffusion approach is more accurate for many targets. Another extension, DiffDock-PP (121), applies diffusion to protein–protein docking, which is a more complex scenario of two flexible bodies coming together, and has shown promising results.

AI in docking also integrates with scoring refinement. Once DiffDock places a ligand, one can use a brief physics-based minimization or a neural network rescoring model to refine the pose and binding score, further improving accuracy. This synergy of AI-guided generation and traditional force-field refinement can deliver both speed and data-driven insights, plus fine-detail adjustment from physics. DiffDock’s ability to learn a statistical potential for interactions from data also implicitly captures difficult-to-model effects like entropy and solvation, helping it outperform hand-crafted scoring functions (122,123).

What does this mean for drug discovery? In practice, DiffDock accelerates drug target identification. Researchers can now screen a library of compounds by docking them with DiffDock to a target, triaging huge libraries in a day. DiffDock also supports polypharmacology studies, screening a drug against many proteins in silico to predict off-targets or new uses (124). The model can help elucidate mechanisms of action for novel phenotypic screening hits by docking them to panels of protein structures.

Figure 3. AI-driven strategies for drug–target interaction prediction. (A) DiffDock uses diffusion models for pose generation and refinement. (B) AI-enhanced virtual screening accelerates compound prioritization via deep learning and optimized docking. (C) AI-based models, such as GNNs, outperform traditional scoring in binding affinity prediction.

Figure 3. AI-driven strategies for drug–target interaction prediction. (A) DiffDock uses diffusion models for pose generation and refinement. (B) AI-enhanced virtual screening accelerates compound prioritization via deep learning and optimized docking. (C) AI-based models, such as GNNs, outperform traditional scoring in binding affinity prediction.

4.2. Protein–Ligand Binding Affinity Prediction

Accurately predicting the binding affinity (e.g., K_d or IC₅₀) of a small molecule to its target protein is crucial for lead optimization. Traditional methods like scoring functions or physics-based free energy calculations can be unreliable or slow. AI-driven models now provide data-driven predictions that can quickly estimate binding affinities with high accuracy.

End-to-end deep learning models have been developed that take protein–ligand complexes as input (either 3D coordinates or interaction lists) and output binding affinities or scores. These models often use 3D convolutional neural networks or graph neural networks (GNNs) that treat the protein–ligand pair as a combined graph (125,126). GNN models represent protein residues and ligand atoms as nodes, with edges representing interactions (contacts, hydrogen bonds, etc.), learning to predict affinity (127,128). Some deep models have achieved a Pearson correlation of ~0.8 on benchmarks like PDBBind (129), significantly outperforming traditional methods.

Quantum + AI hybrid models are also emerging, combining quantum mechanics with AI to improve binding predictions, particularly in cases where electronic effects or polarization are critical. These hybrid models use quantum mechanical descriptors as inputs to machine learning models or even quantum circuits to represent parts of the model, potentially improving predictions of subtle electronic interactions. While quantum methods are still in the early stages, quantum + AI combinations are showing promise for more accurate binding affinity predictions (130–133).

Binding affinity prediction also embraces multi-task and multi-modal learning. A single model can be trained to predict not just affinity, but also other experimental readouts like activity in a cell assay or entropy of binding. This allows the model to be more robust through shared representations. Additionally, coupling binding prediction with generative design is powerful: generative models propose analogs, and deep affinity predictors quickly estimate their potency, enabling thousands of designs to be tested in seconds (134,135).

4.3. Large-Scale Virtual Screening

Virtual screening (VS) evaluates large compound libraries to identify hits for a target, often using docking or pharmacophore matching. With AI and improved prediction models, VS is evolving to handle ultra-large libraries and AI-guided combinatorial library generation.

Recent efforts have led to the creation of vast purchasable libraries, like Enamine REAL, containing >1 billion compounds. Screening these with traditional docking is impractical, but deep learning models can predict docking scores or binding likelihood for all compounds, rapidly reducing the library size to a manageable set for further evaluation.

Another approach uses similarity in latent space: if one has a known active ligand, one can encode the compounds into a learned embedding space and do a nearest-neighbor search to find those most similar in relevant ways, faster than traditional docking. AI can also generate focused libraries on the fly, sampling virtual compounds biased toward predicted binders and screening them for efficacy, blurring the line between virtual screening and de novo design. This approach, demonstrated during the COVID-19 pandemic, has the potential to vastly increase the scale of virtual screening.

AI-guided combinatorial chemistry further enhances screening by intelligently selecting which combinations of building blocks to synthesize. AI models evaluate subsets of possible products, learning which parts contribute to desired activity, and pruning the search space to focus on promising combinations.

5. AI-Driven Synthesis Planning and Retrosynthesis

Designing a promising molecule is only half the battle – one must also be able to make that molecule efficiently. Retrosynthesis planning is the process of identifying a sequence of chemical reactions to synthesize a target molecule from available starting materials. Historically tackled by expert chemists and rule-based software (like E.J. Corey’s LHASA or Synthia), AI is now playing a major role in retrosynthesis and synthesis planning, offering data-driven predictions and creative route suggestions. Key contributions of AI include predicting feasible reactions, using reinforcement learning to navigate possible routes, and employing Bayesian optimization to propose optimal reaction conditions or pathways (Figure 4).

Predicting synthetic feasibility: AI helps evaluate a molecule’s structure and suggests possible retrosynthetic disconnections. Transformer models and GNNs trained on millions of reactions can predict reaction patterns (20). For instance, IBM’s RXN for Chemistry uses a sequence-to-sequence transformer to predict reactants given a product. These models output multiple disconnections, which can be recursively applied to break the molecule down stepwise (136). AI retrosynthesis produces a retrosynthetic tree or network of possible routes, each step predicted with a confidence score. Early deep learning models, like RetroTransformer, have achieved success rates comparable to expert chemists and sometimes uncover routes human chemists might overlook (137).

Figure 4. AI-driven synthesis planning pipeline for retrosynthesis and reaction optimization. The process begins with a target molecule (top left), where AI models predict retrosynthetic disconnections. Transformer models and graph neural networks (GN004Es) are trained on reaction databases to identify viable bond disconnections, yielding confidence scores for each prediction. Monte Carlo Tree Search (MCTS) is then employed to optimize synthetic pathways by evaluating and pruning possible routes. After selecting an optimal pathway, AI-based Bayesian optimization algorithms identify optimal reaction conditions to maximize yield. The entire process culminates in an experimentally feasible and optimized synthesis route.

Figure 4. AI-driven synthesis planning pipeline for retrosynthesis and reaction optimization. The process begins with a target molecule (top left), where AI models predict retrosynthetic disconnections. Transformer models and graph neural networks (GN004Es) are trained on reaction databases to identify viable bond disconnections, yielding confidence scores for each prediction. Monte Carlo Tree Search (MCTS) is then employed to optimize synthetic pathways by evaluating and pruning possible routes. After selecting an optimal pathway, AI-based Bayesian optimization algorithms identify optimal reaction conditions to maximize yield. The entire process culminates in an experimentally feasible and optimized synthesis route.

However, AI doesn’t fully replace human planning; it acts as a powerful assistant. The model proposes several routes, and a chemist reviews and refines them. A limitation is that AI models are trained on known reaction data, making it difficult for them to suggest truly novel chemistry (138).

Reinforcement Learning for Retrosynthesis: The space of possible synthetic routes is vast, resembling a game with many reactions as possible moves. AI uses methods like Monte Carlo Tree Search (MCTS) guided by learned policies to explore the retrosynthesis tree efficiently (139,140). Deep reinforcement learning (RL) has been applied, with an RL agent proposing retrosynthesis steps and receiving rewards when reaching purchasable building blocks. This approach minimizes the number of steps, rediscovering many known strategies (141). AI-guided search prunes unlikely paths, making it more efficient than traditional rule-based programs. A challenge is ensuring that the predicted steps are not only theoretically plausible but also practically executable.

Reaction condition optimization: Once a route is chosen, AI/ML techniques like Bayesian optimization automate reaction condition optimization. Bayesian optimization treats reaction yield as a function of conditions and selects which conditions to try next. A cost-aware Bayesian optimizer can factor in the time/resource cost of experiments, focusing on cost-effective routes (142–146).

Integration of synthesis planning in design: Generative models can guide design toward more synthesizable regions of chemical space in real-time. Combined design-synthesis optimization frameworks, like TRACER and Syn-MolOpt, optimize both molecular properties and synthetic accessibility (147). For example, a complex molecule predicted to be difficult to synthesize can be deprioritized in favor of a more synthesizable alternative, ensuring a balance between potency and ease of synthesis (148).

AI-driven synthesis planning is narrowing the gap between the molecules we can design and synthesize. By predicting synthesis pathways and optimizing reaction conditions, generative pipelines focus on candidates that are both innovative and realizable. Reinforcement learning and search algorithms enable retrosynthesis tools to handle complex targets. This fusion of design and synthesis planning accelerates the drug discovery cycle and minimizes the risk of pursuing infeasible designs.

6. AI for Pharmacokinetics and Toxicity Prediction

While potency and synthesizability are crucial, a successful drug must also possess suitable pharmacokinetic (PK) and safety profiles. This includes absorption (can it get into the bloodstream?), distribution (does it reach the target tissue?), metabolism (is it broken down too quickly or into toxic metabolites?), excretion (can it be eliminated from the body?), and toxicity (does it harm cells or organs, or cause side effects?). These properties are encapsulated in the acronym ADME/Tox. Generative AI models and predictive machine learning are being used to evaluate and optimize these factors early in the design process, aiming to produce drug candidates that are not only effective but also drug-like and safe.

Figure 5. AI-driven strategies for optimizing pharmacokinetics, toxicity, and personalized drug discovery. (A) AI predicts ADME/Tox properties to guide early drug optimization. (B) Generative models balance potency with ADME/toxicity profiles. (C) AI leverages multi-omics data for patient-specific drug design in precision medicine.

Figure 5. AI-driven strategies for optimizing pharmacokinetics, toxicity, and personalized drug discovery. (A) AI predicts ADME/Tox properties to guide early drug optimization. (B) Generative models balance potency with ADME/toxicity profiles. (C) AI leverages multi-omics data for patient-specific drug design in precision medicine.

6.1. ADME/Tox Predictions

Drug-likeness constraints: Medicinal chemists apply rules (like Lipinski’s Rule of 5) to ensure oral bioavailability. AI can learn nuanced drug-likeness patterns from large datasets of known drugs and failed compounds. Models like neural networks and ensemble methods (random forests, gradient boosting) distinguish drug vs non-drug molecules, capturing subtle features. Generative models incorporate drug-likeness as part of their scoring function, co-optimizing for favorable ADME properties.

Absorption and distribution: AI models predict permeability, solubility, and plasma protein binding. Deep learning regression models predict Caco-2 cell permeability or blood-brain barrier penetration based on molecular structure. Models for human intestinal absorption can classify compounds as high vs low absorption, guiding early elimination of very polar compounds.

Metabolism and elimination: AI methods (MetPred, RS-WebPredictor) predict metabolic stability and sites of transformation (e.g., CYP450 enzymes). More advanced models predict metabolite structures using sequence-to-sequence learning. Models predict P450 inhibition to avoid drug-drug interactions, penalizing molecules likely to inhibit major isoforms like CYP3A4.

Toxicity and off-target effects: AI predicts various toxicities:

• In vitro cytotoxicity using Tox21 challenge data.
• Organ toxicity (hepatotoxicity, cardiotoxicity), including hERG channel inhibition, predicted by ML models.
• Genotoxicity and carcinogenicity predictions using Ames test data or animal studies.
• Reactive functional group alerts: AI identifies substructures causing nonspecific reactivity or toxicity, learning broader patterns of reactivity beyond known PAINS.

In practice, AI-driven ADMET tools are applied in lead optimization, predicting properties like logP, solubility, permeability, clearance, and hERG risk. Multi-parameter optimization (MPO) frameworks balance potency and ADMET properties. AI helps navigate trade-offs; for instance, improving solubility might reduce CNS toxicity but also lower permeability. AI proposes modifications to improve one property without overly harming others (149,150). By identifying ADME/Tox issues early, AI saves time and cost by avoiding failure due to pharmacokinetic issues or toxicity.

Predicting off-target interactions: AI models trained on bioactivity databases predict unwanted off-target interactions, guiding generative design. Multi-task neural networks like prOCTOR predict activity across multiple off-targets, enabling in silico “safety pharmacology” panels. Generative design can penalize compounds with high affinity for undesirable anti-targets, actively minimizing off-target effects (151).

Modern AI-driven drug design optimizes multi-factor properties (potency, ADME, toxicity), ensuring compounds have a balanced profile. This approach embodies “fail fast, fail cheap” by identifying potential failures early, reducing costly animal studies.

6.2. Personalized Drug Discovery

AI is paving the way for precision medicine, tailoring drug discovery to individual patient data (e.g., genomic, multi-omic). Unlike traditional drug discovery, AI in personalized medicine aims to design drugs for specific subpopulations or individual patients. Generative AI can leverage multi-omics datasets (genomics, transcriptomics, proteomics) to discover novel therapeutic strategies or patient-specific drug candidates.

In oncology, AI models design molecules targeting mutant proteins without affecting normal variants. For example, AI could propose a drug combination for a tumor with specific oncogene dependencies. These applications are illustrated in Figure 5, which highlights AI-driven strategies for ADME/Tox prediction, multi-parameter optimization, and personalized drug design. Generative AI optimizes drugs to reverse a disease-specific expression signature in transcriptomic-driven drug design. AI models predict gene expression changes based on structure and optimize accordingly.

Multi-omics-based generation: AI analyzes rich patient data to identify novel targets or pathways. For example, AI might stabilize an atypical protein conformation in a tumor, thereby blocking its function. AI can also design personalized vaccines, creating neoantigens optimized for an individual’s HLA type. This was successfully demonstrated with AI-generated therapeutic vaccines tailored to a patient’s tumor mutations.

Generative AI also aids in rare disease drug discovery. For example, AI could design a pharmacological chaperone for a unique pathogenic mutation. Moreover, AI suggests drug repurposing for patients with specific gene expression signatures, identifying existing drugs with profiles opposite to the disease state.

While personalized generative drug design is emerging, AI can integrate patient data to suggest personalized therapeutic molecules. This could lead to AI-designed drugs tested on a patient’s cells or organoids, with the potential for rapid, precise treatments. While challenges exist, AI in precision drug discovery promises the long-envisioned goal of “the right drug for the right patient at the right time.”

7. Experimental Validation and AI-Augmented Pipelines

No matter how powerful our in silico methods are, experimental validation is the ultimate proving ground for any AI-designed molecule or protein (Figure 6). In this section, we discuss how AI-designed candidates are being validated in the lab (and some notable success stories), as well as how experiments themselves are becoming more integrated with AI (creating a closed-loop discovery pipeline). We cover wet lab validation of AI-designed drugs (7.1) and how AI assists in protein engineering and biotechnology (7.2), including real-world examples where AI-designed proteins have been synthesized and tested.

7.1. Wet Lab Validation: Case Studies of AI-Designed Drugs and Challenges in Translation

Over the past few years, we’ve seen AI-designed molecules advancing into experimental and clinical stages. A landmark in 2020 was the first fully AI-designed drug (DSP-1181 for OCD, designed by Exscientia) entering Phase I clinical trials (152). This small molecule, optimized for activity on a GPCR target, went from concept to clinic in 12 months, instead of the usual 4–5 years. Similarly, Insilico Medicine’s AI-discovered drug for idiopathic pulmonary fibrosis entered Phase I trials in 2022, reducing time and cost compared to traditional programs.

Another exciting case is AbSci’s 2023 AI-designed de novo antibody, which was synthesized and confirmed to bind and neutralize its target. The FDA also granted Orphan Drug Designation to an Insilico-designed drug for a rare disease in 2023, further validating AI’s role in drug development (153).

Figure 6. AI-augmented pipelines in drug discovery and biotechnology. (A) AI accelerates drug discovery through molecular design, high-throughput screening, and iterative validation. (B) AI enables de novo protein design, enzyme engineering, and synthetic biology applications, enhancing experimental efficiency and precision.

Figure 6. AI-augmented pipelines in drug discovery and biotechnology. (A) AI accelerates drug discovery through molecular design, high-throughput screening, and iterative validation. (B) AI enables de novo protein design, enzyme engineering, and synthetic biology applications, enhancing experimental efficiency and precision.

However, not all AI-designed candidates succeed. Some molecules have failed to meet efficacy endpoints or faced unforeseen issues, such as one report where AI-derived molecules did not outperform traditional leads. These instances highlight that while AI expedites clinical candidate development, rigorous experimental validation is essential. AI predictions can be wrong, as compounds predicted to be non-toxic may show toxicity due to overlooked factors, like rare metabolic byproducts.

To mitigate risks, AI-driven projects adopt a fail-fast approach: generating multiple top candidates, testing them in vitro, and iterating. For instance, if AI yields five candidates with similar profiles, all might be tested for potency, solubility, metabolic stability, and toxicity (e.g., hERG patch-clamp assay). Insilico’s fibrosis drug underwent ~6 AI design iterations, testing dozens of compounds, before identifying the clinical candidate.

AI augmenting experiments: AI also aids in experiment planning and analysis. In high-throughput screening, AI can detect patterns in assay readouts, identifying hits that work via desired mechanisms and distinguishing false positives. In robotics and automation, AI directs experiments like flow chemistry setups to optimize reaction conditions, updating the model in real-time. In microfluidics, AI designs experiments, executes them, and analyzes the data with minimal human intervention (154–158).

Challenges in translation: A major issue is the predictive gap. AI models may fail to account for real-world variables, such as molecule instability or dynamic protein structures. Verifying binding through biophysical methods like X-ray crystallography is crucial. Some AI-designed ligands have matched their predicted binding poses with targets, reinforcing confidence in the design (159–163).

Chemical novelty vs synthetic familiarity is another challenge. AI sometimes proposes novel structures that present synthetic difficulties or unexpected reactivity. Medicinal chemists often apply a “chemical intuition filter” to make these designs more practical.

Despite these challenges, each successful case of an AI-designed drug reaching clinical trials validates the approach. By 2024, over 15 AI-designed molecules were in clinical trials, suggesting that in the next decade, many new clinical candidates may involve AI (164).

Experimental validation is essential for testing AI-designed solutions. Proof-of-concept that AI-designed molecules can become real drugs and proteins function as intended marks a significant achievement. The challenges are addressed through iterative testing and improved models, and with advances in AI and laboratory automation, the gap between design and validation will continue to narrow.

7.2. Protein Engineering in Biotechnology: AI-Augmented Enzyme and Pathway Design

Generative AI is profoundly impacting protein engineering and biotechnology. AI is being used to design industrial enzymes, optimize metabolic pathways, and create synthetic biological parts. AI-designed enzymes and proteins, as discussed in previous sections, are having significant applications.

Enzyme design and metabolic engineering: AI is enabling the de novo design of enzymes with functions not found in nature. For example, a de novo enzyme was designed to hydrolyze organophosphates, showing measurable activity in breaking them down—useful in bioremediation. AI-designed enzymes often function without experimental optimization, which was rare with traditional methods (165,166).

In metabolic pathway engineering, AI identifies enzyme variants that improve pathway efficiency, specificity, or by-product formation. For example, AI may suggest enzyme variants for a rate-limiting step or redesign enzymes to improve specificity.

Synthetic biology and novel protein functions: AI is also used to design transcription factors, DNA-binding proteins, and self-assembling peptides. RFdiffusion, for instance, was used to design symmetric nanocages, confirmed by electron microscopy. These can be applied in drug delivery, vaccines, or biomaterials. AI can also design multi-enzyme complexes that streamline metabolic flux by channeling intermediates, reducing the need for separate enzymes (167–169).

From AI design to biotech product: While AI-designed proteins still require substantial lab work, the models help reduce the number of variants to test. AI aids in designing proteins that fluoresce at specific wavelengths, expanding cell biology imaging capabilities.

Real-world impact: AI is helping design solutions in environmental, industrial, agricultural, and medical applications. For example, AI has assisted in designing enzymes to degrade plastic waste, replacing traditional catalysts in pharmaceutical synthesis, and creating pest-resistant proteins in agriculture. AI is also enhancing therapeutics by designing proteins with fewer side effects by altering their surfaces to avoid undesired interactions.

In the future, AI-augmented protein engineering will enable the rapid creation of custom enzymes or proteins on demand. Early results, such as AI-designed proteins binding insulin receptors or enzymes accelerating novel reactions, indicate that AI will revolutionize biotechnology by enabling tailored solutions for various challenges.

8. Future Perspectives: AI-Designed Medicines & Autonomous Discovery

AI has initiated a shift in molecular science, promising more integration into drug and protein discovery, potentially leading to fully autonomous discovery pipelines. These systems could generate hypotheses, test them (virtually or physically via robotics), learn from outcomes, and improve with minimal human intervention. Existing components, like generative models proposing molecules and automated labs synthesizing and testing them, hint at what broader autonomous discovery could achieve. For example, a project used a flow chemistry robot and an AI planner to autonomously synthesize and test hundreds of analogs, improving target activity ten-fold without human chemists deciding each step (170).

A particularly exciting prospect is self-driving AI that not only designs molecules but refines itself by learning from outcomes. An AI could design a drug, test it, adjust its parameters, and generate new hypotheses. These AI agents could handle data crunching and routine decision-making, leaving scientists to focus on higher-level strategy and creative insights.

8.1. Fusion with Quantum Computing

AI combined with quantum computing could revolutionize drug design by solving quantum mechanical problems that classical computers struggle with, like binding free energies or reaction pathways. Quantum machine learning algorithms could operate in chemical Hilbert space, enabling simulations of large biomolecules or materials beyond classical reach. Companies are exploring quantum-enhanced generative models (like quantum GANs for molecules), which may improve proposal quality and diversity. Quantum algorithms might also generate reaction pathways, aiding retrosynthesis. While practical quantum computing is emerging, it promises breakthroughs in complex systems like large drug-targets (171–173).

8.2. Ethics and Regulation of AI-Designed Drugs

As AI plays a larger role, ethical and regulatory questions arise. A key concern is accountability if an AI-designed drug causes adverse effects. Regulators might require additional validation steps and transparency in AI model decisions. Research is ongoing to make AI models interpretable, e.g., highlighting molecular substructures linked to low toxicity. Moreover, AI must avoid generating harmful compounds; an example showed an AI generative model could design chemical weapons if misdirected. Safeguards, such as filtering toxic outputs, are necessary. The FDA has approved AI-designed drugs for trials, and regulators may soon require AI design methodology in submission dossiers, ensuring AI’s role is validated with empirical evidence for safety and efficacy (174–178).

8.3. Personalized Drug Design Ethics

In personalized drug design, regulators must address N-of-1 trials or adaptive trial designs. Equity concerns will arise, ensuring AI-designed therapies are accessible globally, not just to wealthy individuals. Automating the design process and reducing costs will make personalized treatments more accessible (179).

The future of AI in molecular science promises transformative advances. AI will not replace humans but work alongside them, expanding creativity and accelerating innovation in drug discovery and protein engineering. With careful oversight and regulation, AI will help create cures at unprecedented speeds, addressing unmet medical needs, including for rare diseases and personalized therapies (180).

9. Conclusions

Generative AI has revolutionized drug discovery and protein design, shifting from rule-based, labor-intensive methods to AI-driven processes. Deep generative models, including VAEs, GANs, transformers, and diffusion models, enable the creation of novel molecular structures and protein sequences with desired properties. This addresses challenges in early-stage drug discovery: navigating vast chemical space, optimizing multiple parameters, and overcoming human bias.

AI-designed molecules have advanced from models to clinical trials, and AI-generated proteins now perform valuable functions. AI optimizes for multiple metrics simultaneously, producing balanced candidates less likely to fail. The future holds autonomous discovery systems where AI designs molecules and controls robotic experimentation, compressing the time from target identification to preclinical candidate.

However, ethical and regulatory challenges remain. AI can generate harmful molecules, requiring safeguards and human oversight. Regulatory bodies must adapt, evaluating AI-designed drugs with predictive modeling results and ensuring safety and efficacy.

Generative AI is transforming molecular science, uniting computational chemistry, structural biology, and systems biology. Advances in deep generative models, AI-guided docking like DiffDock, and diffusion models for protein design demonstrate rapid field progress. AI promises more effective, personalized medicines, biotech solutions, and faster responses to emerging health threats. Responsible integration will enhance the discovery of cures and engineered biomolecules.