Hardware Design and Verification with Large Language Models: A Literature Survey, Challenges, and Open Issues

1. Introduction

1.1. Introduction to LLMs

Large Language Models (LLMs) such as OpenAI’s GPT-4 1, Google’s Gemini 2, Google’s Bidirectional Encoder Representations from Transformers (BERT) 3 and Denoising Autoencoder from Transformer (BART) 4 which is a transformer-based model introduced by Facebook, are at the forefront of Artificial Intelligence (AI) research, revolutionizing how machines understand and generate human language. These models process extensive datasets covering a wide spectrum of human discourse, enabling them to perform complex tasks including translation, summarization, conversation, and creative content generation [1,2,3,4,5]. Recent advancements in this field, driven by innovative model architectures, refined training methodologies, and expanded data processing capabilities, have significantly enhanced the ability of these models to deliver nuanced and contextually relevant outputs. This evolution reflects a growing sophistication in AI’s approach to Natural Language Processing (NLP), positioning LLMs as crucial tools in both academic research and practical applications, transforming interactions between humans and machines [5,6,7].

This evolution can be traced back to early statistical language models like n-gram models, which simply predicted word sequences based on the frequencies of previous sequences observed in a dataset. Although these models provided a foundational approach for text prediction, their limited ability to perceive broader contextual cues restricted their application to basic tasks [8,9,10,11,12]. The advent of neural network-based models, especially Recurrent Neural Networks (RNN), represented a significant advancement, offering the ability to retain information over longer text sequences and thus, managing more complex dialogues and text structures [13,14,15]. Although RNNs made advancements, they continued to struggle with scalability and long-term dependency issues, leading to the creation of Transformer models. These models introduced an innovative self-attention mechanism, allowing simultaneous processing of different sentence segments to enhance relevance and contextuality in text interpretation. This breakthrough underpins modern LLMs, which are pre-trained in extensive web-text data and subsequently fine-tuned for specific tasks, enabling them to generate nuanced, stylistically diverse, and seemingly authentic human text [16,17,18,19,20,21].

Moreover, with the advent of highly sophisticated models, LLMs have become an indispensable domain for both academic research and practical applications. These models necessitate thorough evaluations to fully understand their potential risks and impacts, both at task-specific and societal levels. In recent years, significant efforts have been invested in assessing LLMs from multiple perspectives, enhancing their applicability and effectiveness. The adaptability and deep comprehension abilities of LLMs have led to their extensive deployment across numerous AI domains. They are utilized not only in fundamental NLP tasks but also in complex scenarios involving autonomous agents and multimodal systems that integrate textual data with other data forms [22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39]. The utility of LLMs spans several domains, including healthcare [40,41,42,43,44,45], education [46,47,48,49,50], law [51,52,53,54,55,56,57,58], finance [59,60,61,62,63], and sciences [64,65,66,67,68,69], where they substantially improve data analysis and decision-making processes. This wide-ranging application underscores the transformative impact of LLMs on both technological innovation and societal functions.

In particular, within domains like hardware design and verification, LLMs enhance productivity and innovation by automating and optimizing various stages of the design process. These models can assist engineers in generating design specifications, suggesting improvements, and even creating initial design drafts. By leveraging vast amounts of data and advanced algorithms, LLMs can identify patterns and propose design optimizations that might not be immediately apparent to human designers. This capability helps in reducing time-to-market and ensuring that hardware designs are both efficient and innovative [70].

In hardware design, verification is a critical step in the lifecycle of hardware development. Verification ensures that the hardware performs as intended and meets all specified requirements before going into production. Traditionally, this process has been time-consuming and prone to human error. LLMs can automate much of the verification process by generating test cases, simulating hardware behavior, and identifying potential faults or discrepancies. They can analyze a large amount of verification data to predict potential issues and provide solutions, thus enhancing the reliability and accuracy of the hardware verification process. This not only speeds up the verification process, but also ensures a higher quality of the final product [71].

In addition to automation, LLMs facilitate better communication and collaboration among hardware design and verification teams. By providing a common platform where designers, engineers, and verification experts can interact with the model, LLMs help in bridging the gap between different teams. This collaborative approach ensures that all aspects of the hardware design and verification are aligned and that any issues are identified and addressed early in the process. Furthermore, LLMs can serve as knowledge repositories, offering solutions based on previous designs and verifications, thus ensuring that best practices are followed and past mistakes are not repeated [72].

Another significant benefit of LLMs in hardware design and verification is their ability to handle complex and high-dimensional data. Modern hardware designs are increasingly complex with numerous components and interdependencies. LLMs can manage this complexity by analyzing and processing large datasets to extract meaningful insights. They can model intricate relationships between different hardware components and predict how changes in one part of the design could impact the overall system. This holistic understanding is crucial for creating robust and reliable hardware systems [73].

In conclusion, integration of LLMs in hardware design and verification not only fosters innovation, but also ensures the development of cutting-edge hardware technologies [74,75]. This survey aims to explore the transformative role of LLMs in this domain, highlighting key contributions, addressing challenges, and discussing open issues that continue to shape this dynamic landscape. The goal is to provide a comprehensive overview that not only informs, but also inspires continued research and application of LLMs to improve hardware design and verification processes. We hope that this study will contribute to a better understanding and use of LLMs. In summary, the contributions of this paper can be summarized as follows:

• Identification of Core Applications: We detail the fundamental ways in which LLMs are currently applied in hardware design, debugging, and verification, providing a solid foundation to understand their impact.
• Analysis of Challenges: This paper presents a critical analysis of the inherent challenges in applying LLMs to hardware design, such as data scarcity, the need for specialized training, and integration with existing tools.
• Future Directions and Open Issues: We outline potential future applications of LLMs in hardware design and verification and discuss methodological improvements to bridge the identified gaps.

The remainder of the paper is organized as follows. Section 2 reviews the literature on the application of LLMs in hardware design and verification. Section 3 discusses the challenges associated with training and adapting LLMs for specific hardware design tasks. Section 4 explores open issues in the field and proposes areas for further investigation. Finally, Section 5 presents the conclusion of the study. The organization of the paper is depicted in Figure 1.

1.2. A Brief History of LLMs

The evolution of LLMs represents a crucial aspect of the broader development in AI. This progression begins with the earliest models and extends through to the sophisticated systems that today significantly influence computational linguistics and AI applications. A very brief history of LLMs is shown by Figure 2.

Initially, LLMs operated on rule-based systems established in the mid-20^th century, which were limited by strict linguistic rules and struggled to adapt to the variability of natural language. These systems, while foundational, offered limited utility for complex language tasks due to their inability to capture nuanced linguistic patterns [76]. The transition to statistical models like n-grams and Hidden Markov Models (HMMs) [77] in the late 20^th century marked a pivotal enhancement. These models introduced a statistical approach to language processing, utilizing probabilities derived from large text corpora to predict language patterns. This shift allowed better handling of larger datasets, significantly improving real-world language processing capabilities. Despite these advancements, these models continued to struggle with deep contextual and semantic understanding, which later developments in algorithmic technology aimed to address [8,9,10,11,12].

By the early 2000s, the integration of advanced neural networks, specifically RNN [13,14,15] and Long Short-Term Memory (LSTM) networks [78], brought about substantial improvements in modeling sequential data. Additionally, the emergence of word embedding technologies like Word2Vec and GloVe advanced LLM capabilities by mapping words into dense vector spaces, capturing complex semantic and syntactic relationships more effectively. Despite these innovations, the increasing complexity of neural networks raised new challenges, particularly in model interpretability and the computational resources required [14,79,80].

The mid-2010s marked another significant advancement with the introduction of deep learning-based neural language models, notably the Recurrent Neural Network Language Model (RNNLM) in 2010, designed to effectively capture textual dependencies [81,82]. This development improved the generation of text that was more natural and contextually informed. However, these models also faced limitations such as restricted memory capacity and extensive training demands [83]. In 2015, Google’s breakthrough with the Neural Machine Translation (GNMT) model utilized deep learning to significantly enhance machine translation, moving away from traditional rule-based and statistical techniques towards a more robust neural approach. This development not only improved translation accuracy but also addressed complex NLP challenges with greater efficacy [84,85].

A major breakthrough occurred in 2017 with the development of the Transformer model, which abandoned the sequential processing limitations of previous models in favor of self-attention mechanisms [16,17,86]. This innovation allowed for the parallel processing of words, drastically increasing efficiency and enhancing the model’s ability to manage long-range dependencies. The Transformer architecture facilitated the creation of more sophisticated models such as BERT, which utilized bidirectional processing to achieve a deep understanding of text context, greatly improving performance across a multitude of NLP tasks [18,19,87]. Following BERT, models such as RoBERTa, T5, and DistilBERT have been tailored to meet the diverse requirements of various domains, illustrating the adaptability and expansiveness of LLM applications [88].

Subsequently, the introduction of OpenAI’s GPT series further pushed the boundaries of what LLMs could achieve. Starting with GPT-1 and evolving through GPT-3, these models demonstrated exceptional capabilities in generating coherent and contextually relevant text across various applications. GPT-3, in particular, with its wide array of parameters, showcased the potential of LLMs to perform complex language tasks such as translation, question-answering and creative writing with minimal specific tuning. The advent of GPT-4 further broadened these capabilities by incorporating multimodal applications that process both text and images, thus significantly expanding the scope of LLMs. Recent developments, including enhancements in GPT-4 and the introduction of innovative models such as DALL-E 3, have continued this trend, emphasizing efficiency in fine-tuning, and enhancing capabilities in creative AI fields, demonstrating the versatility and depth of current models [19,21,87,89,90,91,92].

This progression from statistical models to today’s advanced, multimodal, and domain-specific systems illustrates the dynamic and ongoing nature of LLMs development. These continual innovations not only advance the technology but also significantly impact the fields of AI and computational linguistics. Innovations such as sparse attention mechanisms, more efficient training algorithms, and the use of specialized hardware like Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs) have enabled researchers to build increasingly larger and more powerful models. Moreover, efforts to improve model interpretability, reduce bias, and ensure ethical use are increasingly becoming central to the field.

In summary, the history of LLMs is a story of rapid progress driven by breakthroughs in Machine Learning (ML) and neural network architectures. From early statistical models to the transformative impact of the Transformer architecture and the rise of models like GPT-4, LLMs have evolved dramatically, reshaping our understanding of language and AI. As research continues, LLMs are poised to become even more integral to technological innovation and human-computer interaction in the years to come.

1.3. Survey Papers on the Application of LLMs in Different Areas

Due to the success of LLMs on various tasks and their increasing integration into AI research, examining the extensive survey literature on these models is essential. A significant number of surveys [5,93,94,95,96,97,98,99,100,101,102,103], provide detailed insights into the application of LLMs across different fields, demonstrating their advancements and wide-ranging uses. By analyzing these surveys, primarily published in 2022 and 2024, our aim is to gain a deeper understanding of LLMs’ applications, and evaluate their potential impact across various sectors.

In this context, the work by Huang et al. [93] present a comprehensive survey of reasoning in LLMs, focusing on the current methodologies and techniques to enhance and evaluate reasoning capabilities in these models. The paper provides an in-depth review of various reasoning types, including deductive, inductive, and abductive reasoning, and discusses how these reasoning forms can be applied in LLMs. It also explores key methods used to elicit reasoning, such as fully supervised fine-tuning, prompting, and techniques like "chain of thought" prompting, which encourages models to generate reasoning steps explicitly. The authors review benchmarks and evaluation methods to assess reasoning abilities and analyze recent findings in this rapidly evolving field. Despite the advancements in reasoning with LLMs, the paper points out the limitations of current models, emphasizing that it remains unclear whether LLMs truly possess reasoning abilities or are merely following heuristics. Huang et al. conclude by offering insights into future directions, suggesting that better benchmarks and more robust reasoning techniques are needed to push the boundaries of LLMs’ reasoning capabilities. This survey serves as an essential resource for researchers looking to understand the nuances of reasoning in LLMs and guide future research in this critical area. The authors review benchmarks and evaluation methods to assess reasoning abilities and analyze recent findings in this rapidly evolving field. Despite the advancements in reasoning with LLMs, the paper points out the limitations of current models, emphasizing that it remains unclear whether LLMs truly possess reasoning abilities or are merely following heuristics. Huang et al. conclude by offering insights into future directions, suggesting that better benchmarks and more robust reasoning techniques are needed to push the boundaries of LLMs’ reasoning capabilities. This survey serves as an essential resource for researchers looking to understand the nuances of reasoning in LLMs and guide future research in this critical area. The authors review benchmarks and evaluation methods to assess reasoning abilities and analyze recent findings in this rapidly evolving field. Despite the advancements in reasoning with LLMs, the paper points out the limitations of current models, emphasizing that it remains unclear whether LLMs truly possess reasoning abilities or are merely following heuristics. Huang et al. conclude by offering insights into future directions, suggesting that better benchmarks and more robust reasoning techniques are needed to push the boundaries of LLMs’ reasoning capabilities. This survey serves as an essential resource for researchers looking to understand the nuances of reasoning in LLMs and guide future research in this critical area. The authors review benchmarks and evaluation methods to assess reasoning abilities and analyze recent findings in this rapidly evolving field. Despite the advancements in reasoning with LLMs, the paper points out the limitations of current models, emphasizing that it remains unclear whether LLMs truly possess reasoning abilities or are merely following heuristics. Huang et al. conclude by offering insights into future directions, suggesting that better benchmarks and more robust reasoning techniques are needed to push the boundaries of LLMs’ reasoning capabilities. This survey is an essential resource for researchers looking to understand the nuances of reasoning in LLMs and guide future research in this critical area.

In the study by Xi et al. [94], the authors comprehensively survey the rise and potential of LLM-based agents. The paper traces the evolution of AI agents, with a particular focus on LLMs as foundational components. It explores the conceptual framework of LLM-based agents, which consists of three main parts: brain, perception, and action. The survey discusses how LLMs, particularly transformer models, have been leveraged to enhance various agent capabilities, such as knowledge processing, reasoning, and decision-making, allowing them to interact effectively with their environments. Furthermore, the paper delves into the real-world applications of LLM-based agents across different sectors, including single-agent and multi-agent systems, as well as human-agent cooperation scenarios. It also highlights how LLM-based agents can exhibit behaviors akin to social phenomena when placed in societies of multiple agents. In addition, the study examines the ethical, security, and trustworthiness challenges posed by these agents, stressing the need for robust evaluation frameworks to ensure their responsible deployment. Finally, the authors present future research directions, particularly around scaling LLM-based agents, improving their capabilities in real-world settings, and addressing open problems related to their generalization and adaptability.

In [95], the authors present a detailed review of LLMs’ evolution, applications, and challenges. The paper highlights the architecture and training methods of LLMs, particularly focusing on transformer-based models, and emphasizes their significant contributions across a range of sectors, including medicine, education, finance, and engineering. It also explores both the potential and limitations of LLMs, addressing ethical concerns such as biases, the need for vast computational resources, and issues of model interpretability. Furthermore, the survey delves into emerging trends, including efforts to improve model robustness and fairness, while anticipating future directions for research and development in the field. This comprehensive analysis serves as a valuable resource for researchers and practitioners, offering insights into the current state and future prospects of LLM technologies.

Naveed et al. [96] provide a comprehensive overview of LLMs, focusing on their architectural design, training methodologies, and diverse applications across various domains. The paper delves deeply into transformer models and their role in advancing NLP tasks. It also highlights the challenges associated with LLM deployment, including ethical concerns, computational resource demands, and the complexity of training these models. Additionally, the survey explores the impact of LLMs on different sectors such as healthcare, engineering, and social sciences, and identifies potential research directions for the future. This review serves as a key resource for researchers and practitioners looking to understand the current landscape of LLM development and deployment.

Fan et al. [97] present a comprehensive bibliometric analysis of over 5,000 publications on LLMs spanning from 2017 to 2023. This study aims to provide a detailed map of the progression and trends in LLM research, offering valuable insights for researchers, practitioners, and policymakers. The analysis delves into key developments in LLM algorithms and explores their applications across a wide range of fields, including NLP, medicine, engineering, and the social sciences. Additionally, the paper reveals the dynamic and fast-paced evolution of LLM research, highlighting the core algorithms that have driven advancements and examining how LLMs have been applied in diverse domains. By tracing these developments, the study underscores the substantial impact LLMs have had on both scientific research and technological innovation and provides a roadmap for future research in the field.

The study by Zhao et al. [5] offers an extensive survey of the evolution and impact of LLMs within AI and NLP. It traces the development from early statistical and neural language models to modern pre-trained language models (PLMs) with vast parameter sets. The paper highlights the unique capabilities that emerge as LLMs scale, such as in-context learning and instruction-following, which distinguish them from smaller models. A significant portion of the survey is dedicated to the contributions of LLMs, including their role in advancing AI applications like ChatGPT. Organized around four key areas—pre-training, adaptation tuning, utilization, and capacity evaluation—the study offers a comprehensive analysis of current evaluation techniques and benchmarks, while also identifying future research directions for enhancing LLMs and exploring their full potential.

In the study by Raiaan et al. [98], the authors conduct a comprehensive review of LLMs, focusing on their architecture, particularly transformer-based models, and their role in advancing NLP tasks such as text generation, translation, and question answering. The paper explores the historical development of LLMs, beginning with early neural network-based models, and examines the evolution of architectures like transformers, which have significantly enhanced the capabilities of LLMs. It discusses key aspects such as training methods, datasets, and the implementation of LLMs across various domains including healthcare, education, and business.

In another study, Minaee et al. [99] survey on LLMs illustrates an insightful analysis of the rise and development of LLMs, focusing on key models like GPT, LLaMA, and PaLM. The paper offers a comprehensive analysis of their architectures, training methodologies, and the scaling laws that underpin their performance in natural language tasks. Additionally, the survey examines key advancements in LLM development techniques, evaluates commonly used training datasets, and compares the effectiveness of different models through benchmark testing. Importantly, the study explores the emergent abilities of LLMs—such as in-context learning and multi-step reasoning—that differentiate them from smaller models, while also addressing real-world applications, current limitations, and potential future research directions.

In [100], the authors provide an in-depth analysis of the methodologies and technological advancements in the training and inference phases of LLMs. The paper explores various aspects of LLM development, including data preprocessing, model architecture, pre-training tasks, and fine-tuning strategies. Additionally, it covers the deployment of LLMs, with a particular emphasis on cost-efficient training, model compression, and the optimization of computational resources. The review concludes by discussing future trends and potential developments in LLM technology, making it a valuable resource for understanding the current and future landscape of LLM research and deployment.

Cui et al. [101] present a comprehensive survey on the role of Multimodal Large Language Models (MLLMs) in advancing autonomous driving technologies. The paper systematically explores the evolution and integration of LLMs with vision foundation models, focusing on their potential to enhance perception, decision-making, and control in autonomous vehicles. It reviews current methodologies and real-world applications of MLLMs in the context of autonomous driving, including insights from the 1st WACV Workshop on Large Language and Vision Models for Autonomous Driving (LLVM-AD). The study highlights emerging research trends, key challenges, and innovative approaches to improving autonomous driving systems through MLLM technology, emphasizing the importance of multimodal learning for the future of autonomous vehicles. Additionally, it stresses the need for further research to address critical issues like safety, data processing, and real-time decision-making in the deployment of these models.

Chang et al. [102] thoroughly explores the essential practices for assessing the performance and applicability of LLMs. It systematically reviews evaluation methodologies focusing on what aspects of LLMs to evaluate, where these evaluations should occur, and the best practices on how to conduct them. The paper explores evaluations across various domains including NLP, reasoning, medical applications, ethics, and education, among others. It also highlights successful and unsuccessful case studies in LLM applications, providing a critical insight into future challenges that might arise in LLM evaluation and stressing the need for a discipline-specific approach to effectively support the ongoing development of these models.

Kachris [103] provides a comprehensive analysis of the various hardware solutions designed to optimize the performance and efficiency of LLMs. The paper explores a wide variety of accelerators, including GPUs, FPGAs, and Application-Specific Integrated Circuit (ASIC)s, providing a detailed discussion of their architectures, performance, and energy efficiency metrics. It focuses on the significant computational demands of LLMs, particularly in both training and inference, and evaluates how these accelerators help meet these demands. The survey also highlights the trade-offs in performance and energy consumption, making it a valuable resource for those seeking to optimize hardware solutions for LLM deployment in data centers and edge computing.

Table 1 provides a comparison between various review papers, categorizing them based on critical features such as LLM models, APIs, datasets, domain-specific LLMs, ML-based comparisons, taxonomies, architectures, performance, hardware specifications for testing and training, and configurations.

1.4. How LLM facilitate Hardware Design and Verification?

By automating repetitive tasks, providing intelligent suggestions, and facilitating better communication and documentation, LLMs significantly improve the efficiency and effectiveness of hardware design and verification processes. They enable engineers to focus on higher-level problem-solving and innovation, thereby accelerating the development cycle and improving the quality of hardware products. A comprehensive list of these tasks are listed on Table 2.

2. Literature Review

2.1. Overview of LLMs in Hardware Design

LLMs have become a transformative tool in the field of hardware design and verification, bringing significant advancements in efficiency and accuracy. These models, powered by sophisticated AI and NLP capabilities, can analyze and interpret vast amounts of documentation, code, and design specifications, which accelerate the initial phases of hardware design. Using LLMs, engineers can automate the generation of design documents, ensuring consistency and reducing human error. This automation not only speeds up the design process but also enables the exploration of more complex and innovative designs, as the model can provide insights and suggestions based on a wide array of previous designs and industry standards.

In the realm of hardware verification, LLMs play a crucial role in improving the robustness and reliability of hardware systems. Verification is a critical step that ensures that the designed hardware functions correctly under all specified conditions. LLMs can help generate comprehensive test cases, identifying potential edge cases that could be overlooked by human designers. In addition, they can analyze the results of these tests more efficiently, highlighting discrepancies and providing detailed diagnostics that can pinpoint the root causes of failures. This capability significantly reduces the time and resources required for verification, allowing quicker iterations and more reliable hardware products. As a result, the integration of LLMs into hardware design and verification workflows is increasingly essential to maintain a competitive advantage in the fast-paced tech industry.

These studies collectively illustrate the transformative potential of LLMs in hardware design and verification, offering new methodologies that enhance efficiency, accuracy, and innovation in the field. As technology continues to evolve, further research and development will likely uncover even more applications and benefits, solidifying the role of LLMs as a crucial tool in modern hardware engineering.

2.2. Different Categories of LLMs for Hardware Design and Verification

To the best of our knowledge, all articles in the literature can be categorized into six categories, as depicted in Figure 3. It should be noted that, although some papers could belong to two or more categories due to their approaches, we have decided to assign them to the category that most closely aligns with the majority of their content. Initially, we evaluated 62 articles. After the first round of review, 54 articles were selected for our survey. Hardware Design and Hardware Security were the most prominent categories, representing 48% and 18% of the articles, respectively. In the following sections, each category will be discussed in detail together with all surveyed papers.

2.2.1. Hardware Design

Hardware design using HDLs is evolving with the integration of LLMs such as OpenAI’s ChatGPT, Google Gemini, and Microsoft Copilot. These LLMs assist in modifying and generating HDL code by accepting design specifications as input prompts, streamlining the development process. In Figure 4, we demonstrate an HDL design for a shift register, where the input prompt describes the desired behavior and adjustments. The results of this input prompt processed by different LLMs are presented in Figure 5, Figure 6 and Figure 7, showing the outputs of ChatGPT 4o, Gemini, and Copilot, respectively. These figures illustrate the varying approaches taken by each model to interpret and modify the HDL design.

This paper [104] presents a comprehensive framework for evaluating the performance of hardware setups used in the inference of LLMs. The framework aims to measure critical performance metrics such as latency, throughput, energy efficiency, and resource utilization to provide a detailed assessment of hardware capabilities. By standardizing these measurements, the framework facilitates consistent and comparable evaluations across different hardware platforms, enabling researchers and engineers to identify the most efficient configurations for LLM inference. The proposed framework addresses the growing complexity and computational demands of LLMs by offering a robust tool for hardware benchmarking. It integrates various testing scenarios to capture the diverse workloads LLMs handle during inference. This allows for a nuanced understanding of how different hardware components contribute to overall performance. Ultimately, the framework aims to guide the development of more efficient hardware solutions tailored to the specific needs of LLMs, promoting advancements in both hardware design and LLM deployment strategies.

This paper [105] investigates the application of LLMs in optimizing and designing VHDL (VHSIC Hardware Description Language (HDL)) code, a critical aspect of digital circuit design. The study explores how LLMs can automate the generation of efficient VHDL code, potentially reducing the time and effort required in the design process. Through a series of experiments, the authors demonstrate the capability of LLMs to provide high-quality code suggestions and optimizations, which can enhance the performance and reliability of digital circuits. The paper also discusses the challenges associated with integrating LLMs into the VHDL design workflow. It highlights issues such as the need for domain-specific training data and the importance of understanding the context and constraints of hardware design. By addressing these challenges, the study provides insights into how LLMs can be effectively utilized to support and streamline VHDL code development, paving the way for more automated and intelligent design processes in digital electronics.

AutoChip [106] introduces a novel method to automate the generation of HDL code by using feedback from LLMs. The approach involves an iterative process where LLMs provide suggestions and improvements on initial HDL code drafts, leading to refined and optimized final versions. This method significantly reduces the need for manual intervention, making the design process more efficient and accessible, especially for complex hardware projects. The paper outlines the technical details of implementing AutoChip, including training LLMs on domain-specific datasets and the integration of feedback loops into the design workflow. Case studies demonstrate the effectiveness of AutoChip in producing high-quality HDL code with minimal human oversight. By automating routine and complex coding tasks, AutoChip has the potential to revolutionize the field of hardware design, enabling faster prototyping and more innovative solutions in hardware development.

This study [107] focuses on benchmarking various LLMs to evaluate their performance in generating Verilog Register Transfer Level (RTL) code. The paper compares the accuracy, efficiency, and complexity handling capabilities of different models, providing a comprehensive assessment of their suitability for automated hardware design tasks. By establishing standardized benchmarks, the research offers valuable insight into the strengths and limitations of each model in the context of the generation of Verilog code RTL. The findings highlight significant differences in model performance, emphasizing the importance of selecting the right LLM for specific hardware design tasks. The paper also discusses the potential for further improving LLMs by incorporating more specialized training data and refining model architectures. Through detailed comparisons and practical examples, this study contributes to the ongoing effort to enhance the role of LLMs in automating and optimizing the hardware design process.

Chip-Chat [108] explores the emerging field of using conversational glsai, particularly LLMs, in hardware design. The paper discusses the potential benefits of natural language interfaces, such as increased accessibility and collaboration, by enabling designers to interact with design tools through simple conversational commands. This approach could democratize hardware design, making it more accessible to non-experts and fostering innovation through diverse contributions. However, the paper also highlights significant challenges in this domain, including the current limitations of LLMs in understanding complex hardware design concepts and the need for precise and unambiguous communication in technical contexts. The authors propose potential solutions, such as improving model training with domain-specific data and developing more sophisticated interaction protocols. By addressing these challenges, the paper aims to pave the way for more effective integration of conversational AI in hardware design, potentially transforming how engineers and designers approach complex projects.

ChipGPT [109] examines the current state of using LLMs for natural language-based hardware design, assessing their capabilities and identifying existing gaps. The paper evaluates various LLMs in terms of their ability to understand and generate hardware design code from natural language descriptions. It highlights the potential of these models to streamline the design process by enabling more intuitive and accessible interactions between designers and design tools. Despite the promising potential, the paper identifies several challenges that need to be addressed to achieve seamless natural language hardware design. These include improving the models’ understanding of technical jargon and design constraints, enhancing the accuracy and efficiency of code generation, and ensuring robust handling of complex design scenarios. By outlining these challenges and suggesting areas for future research, the study provides a roadmap for advancing the integration of NLP in hardware design workflows.

This paper [110] explores the application of LLMs to detect code segments that can benefit from hardware acceleration. The study presents techniques for identifying computation-intensive parts of code that, when offloaded to specialized hardware, can significantly improve overall system performance. The authors demonstrate the effectiveness of LLMs in pinpointing these critical segments and providing recommendations for hardware acceleration. The research also discusses the practical implementation of this approach, discussing how LLMs can be integrated into existing development workflows to automatically suggest and optimize code for hardware acceleration. The paper highlights the potential performance gains and efficiency improvements achievable through this method, making a strong case for the use of LLMs to optimize software for better hardware utilization. By leveraging LLMs for this purpose, developers can achieve more efficient and powerful computing solutions.

CreativEval [111] introduces a novel approach to evaluating the creativity of hardware design code generated by LLMs. The paper proposes specific metrics and benchmarks to assess the originality, efficiency, and practicality of the generated code, emphasizing the importance of creativity in hardware design. By focusing on these aspects, the study aims to determine how well LLMs can innovate within the constraints of hardware development. The findings suggest that while LLMs are capable of producing creative solutions, there are limitations to their ability to fully replicate human ingenuity in hardware design. The paper discusses the potential for improving LLMs through more diverse and comprehensive training data, as well as refining the evaluation metrics to better capture the nuances of creative design. Through this evaluation framework, CreativEval contributes to the understanding of LLM capabilities in generating novel and effective hardware design solutions.

Designing Silicon Brains using LLM [112] explores the use of ChatGPT, a type of LLMs, for designing a spiking neuron array, which is a type of neuromorphic hardware that mimics the behavior of biological neurons. The authors demonstrate how ChatGPT can generate detailed and accurate descriptions of the neuron array, including its architecture and functionality, thereby aiding in the design process. This approach leverages the model’s ability to understand and articulate complex technical concepts in natural language. The study showcases the potential of using LLMs for designing advanced neuromorphic systems, highlighting the benefits of automated description and specification generation. By providing a structured and comprehensive design output, ChatGPT can significantly streamline the development process of spiking neuron arrays. The paper also discusses the challenges and future directions for integrating LLMs into neuromorphic hardware design, emphasizing the need for further refinement and domain-specific training to enhance the accuracy and utility of the generated descriptions.

Digital ASIC Design with Ongoing LLMs [113] explores the application of ongoing LLMs in the design of ASICs. The paper provides an overview of current methodologies and strategies for incorporating LLMs into various stages of the ASIC design process, from initial specification to final implementation. It highlights the potential of LLMs to automate routine tasks, enhance design accuracy, and reduce development time. The authors also discuss the prospects and challenges of using LLMs in digital ASIC design, including the need for specialized training data, the integration of LLMs into existing design workflows, and the importance of maintaining design integrity and performance. By addressing these issues, the study offers valuable insights into the future of ASIC design, suggesting that LLMs could play a significant role in advancing the field through increased automation and intelligent design support.

This paper [114] reviews recent advancements in the development of efficient algorithms and hardware architectures specifically tailored for NLP tasks. The authors discuss various techniques for optimizing NLP algorithms to improve performance and reduce computational requirements. These optimizations are crucial for handling large-scale data and complex models typical of modern NLP applications, including LLMs. The study also explores the design of specialized hardware that can support efficient NLP. This includes discussing hardware accelerators, such as GPUs and TPUs, and their role in enhancing the performance of NLP tasks. By combining algorithmic improvements with hardware advancements, the paper outlines a comprehensive approach to achieving high efficiency and scalability in NLP applications. This integrated approach is essential for meeting the growing demands of NLP and leveraging the full potential of LLMs.

GPT4AIGChip [115] investigates the use of LLMs to automate the design of AI accelerators. The paper presents methodologies for leveraging LLMs to generate design specifications and optimize the architecture of AI accelerators. By automating these processes, the study aims to streamline the development of specialized hardware for AI tasks, reducing both the time and costs associated with traditional design methods. The authors provide detailed case studies demonstrating the effectiveness of LLMs in producing high-quality design outputs for AI accelerators. They highlight the potential for LLMs to not only accelerate the design process but also to innovate and improve upon existing architectures. The paper discusses future directions for this research, including the integration of more advanced LLMs and the development of more sophisticated automation tools. By advancing the use of LLMs in hardware design, GPT4AIGChip contributes to the ongoing evolution of AI hardware development.

Hardware Phi-1.5B [116] introduces a LLM trained specifically on hardware-related data, demonstrating its capability to encode domain-specific knowledge. The paper discusses the model’s architecture and training process, emphasizing the importance of specialized datasets for achieving high performance in hardware design tasks. The authors showcase various applications of Hardware Phi-1.5B, including code generation, optimization, and troubleshooting in hardware development. The study highlights the advantages of using domain-specific LLMs over general-purpose models, particularly in terms of accuracy and relevance of the generated content. By tailoring the model to the hardware domain, Hardware Phi-1.5B is able to provide more precise and contextually appropriate outputs, which can significantly enhance the efficiency and effectiveness of hardware design processes. The paper concludes with a discussion of future research directions and potential improvements to further leverage domain-specific LLMs in hardware engineering.

Hardware-Aware Transformers (HAT) [117] introduces a novel approach to designing LLMs that consider hardware constraints during model training and inference. The paper details the development of transformers that are optimized for specific hardware configurations, aiming to improve performance and reduce resource consumption. This hardware-aware design is particularly important for deploying NLP tasks on various platforms, including edge devices and specialized accelerators. The study presents experimental results demonstrating the efficiency gains achieved by HAT models compared to traditional transformers. The authors highlight significant reductions in latency and energy usage, making these models more suitable for real-world applications where resource constraints are a critical factor. By focusing on the co-design of hardware and software, HAT offers a promising solution for enhancing the performance and scalability of NLP tasks in diverse deployment environments.

This paper [118] proposes a post-processing technique to improve the quality of hardware design code generated by LLMs. The approach involves applying search algorithms to refine and optimize the initial outputs from LLMs, ensuring higher quality and more reliable code generation. The authors detail the implementation of this technique and provide experimental results demonstrating its effectiveness in enhancing code quality. The study highlights the limitations of current LLM-generated code, such as inaccuracies and inefficiencies, and shows how post-LLM search can address these issues. By iteratively refining the code, the proposed method can significantly improve the final output, making it more suitable for practical hardware design applications. This approach underscores the potential for combining LLM capabilities with additional optimization techniques to achieve superior results in automated code generation.

OliVe [119] introduces a quantization technique designed to accelerate LLMs by focusing on hardware-friendly implementations. The concept of outlier-victim pair quantization is presented as a method to reduce the computational load and improve inference speed on hardware platforms. This technique targets specific outliers in the data, which typically require more resources, and optimizes their representation to enhance overall model efficiency. The paper provides a detailed analysis of the quantization process and its impact on model performance. Experimental results demonstrate significant improvements in inference speed and resource utilization without compromising the accuracy of the LLMs. By making LLMs more hardware-friendly, OliVe offers a practical solution for deploying these models in environments with limited computational resources, such as mobile devices and edge computing platforms.

RTLCoder [120] presents a specialized model designed to outperform GPT-3.5 in generating RTL code. The paper highlights the use of an open-source dataset and a lightweight model architecture to achieve superior results in RTL code generation tasks. The authors provide a comprehensive comparison of RTLCoder’s performance against GPT-3.5, demonstrating significant improvements in accuracy, efficiency, and code quality. The study also discusses the advantages of using a focused dataset and a streamlined model for specific applications, such as hardware design. By tailoring the model to the unique requirements of RTL code generation, RTLCoder can provide more relevant and high-quality outputs, reducing the need for extensive manual corrections. This approach underscores the potential for developing specialized models that can outperform general-purpose LLMs in targeted tasks.

RTLLM [121] introduces an open-source benchmark specifically designed to evaluate the performance of LLMs in generating RTL code. The benchmark provides a standardized set of tasks and metrics to facilitate consistent and comprehensive assessments of LLM capabilities in RTL generation. By offering a common evaluation framework, RTLLM aims to drive improvements and innovation in the use of LLMs for hardware design. The paper details the creation of the benchmark, including the selection of tasks, the development of evaluation criteria, and the compilation of relevant datasets. The authors present initial results from using RTLLM to evaluate various LLMs, highlighting strengths and areas for improvement. By providing an open-source tool for benchmarking, RTLLM encourages collaboration and transparency in the development and assessment of LLMs for RTL code generation, fostering advancements in this emerging field.

This paper [122] discusses the co-design approach to selecting features for language models, and balancing software and hardware requirements to achieve optimal performance. The authors highlight the importance of considering both aspects in the development of LLMs, as hardware constraints can significantly impact model efficiency and scalability. By integrating software and hardware design, the study aims to create more efficient and effective LLMs. The paper presents case studies and experimental results demonstrating the benefits of the co-design approach. These examples show how tailored features can enhance model performance on specific hardware platforms, reducing latency and resource consumption. The authors argue that this integrated approach is essential for developing LLMs that can meet the growing demands of real-world applications, offering a path forward for more sustainable and scalable NLP solutions.

SlowLLM [123] investigates the feasibility and performance of running LLMs on consumer-grade hardware. The paper addresses the challenges and limitations of deploying LLMs on less powerful devices, such as personal computers and smartphones, which often lack the computational resources of specialized hardware. The authors propose solutions to enhance performance, such as model compression and optimization techniques tailored for consumer hardware. The study provides experimental results showcasing the performance of various LLMs on consumer devices, highlighting both successes and areas for improvement. The findings demonstrate that, while there are significant challenges, it is possible to achieve acceptable performance levels with appropriate optimizations. By exploring these possibilities, SlowLLM contributes to making advanced NLP capabilities more accessible to a broader audience, potentially expanding the applications and impact of LLMs in everyday technology use.

SpecLLM [124] investigates the use of LLMs for generating and reviewing Very-Large Scale Integration (VLSI) design specifications. The paper evaluates the ability of LLMs to produce accurate and comprehensive specifications, which are crucial for the development of complex integrated circuits. The authors present methods for training LLMs on domain-specific data to enhance their understanding and performance in VLSI design tasks. The study provides experimental results demonstrating the effectiveness of LLMs in generating VLSI design specifications, highlighting their potential to streamline the design process and reduce errors. The authors discuss the challenges and future directions for improving the integration of LLMs in VLSI design, including the need for more sophisticated training techniques and better handling of technical jargon. By exploring these possibilities, SpecLLM contributes to the ongoing efforts to enhance the role of LLMs in the field of hardware design.

ZipLM [125] introduces a structured pruning technique designed to improve the inference efficiency of LLMs. The paper details the development of pruning methods that selectively remove less important components of the model, reducing its size and computational requirements without significantly impacting performance. This inference-aware approach ensures that the pruned models remain effective for their intended tasks while benefiting from enhanced efficiency. The study presents experimental results demonstrating the effectiveness of ZipLM in reducing model size and improving inference speed. The authors highlight the potential applications of this technique in environments with limited computational resources, such as edge devices and mobile platforms. By focusing on structured pruning, ZipLM offers a practical solution for deploying LLMs more efficiently, enabling broader accessibility and application of these powerful models in various real-world scenarios.

This paper [126] explores the application of advanced language models (ALMs), such as GPT-3.5 and GPT-4, in the realm of electronic hardware design, particularly focusing on Verilog programming. Verilog is a HDL used for designing and modeling digital systems. The study introduces the VeriPPA framework, which utilizes ALMs to generate and refine Verilog code. This framework incorporates a two-stage refinement process to enhance both the syntactic and functional correctness of the generated code and to align it with key performance metrics—Power, Performance, and Area (PPA). This iterative approach involves leveraging diagnostic feedback from simulators to identify and correct errors systematically, akin to human problem-solving techniques. The methodology begins with ALMs generating initial Verilog code, which is then refined through the VeriRectify process. This process uses error diagnostics from simulators to guide the correction of syntactic and functional issues, ensuring the generated code meets specific correctness criteria. Following this, the code undergoes a PPA optimization stage where its power consumption, performance, and area efficiency are evaluated and further refined if necessary. This dual-stage approach significantly improves the quality of Verilog code, achieving an 81.37% linguistic accuracy and a 62.0% operational efficacy in programming synthesis, surpassing existing techniques. The study highlights the potential of ALMs in automating and improving the hardware design process, making it more accessible and efficient for those with limited expertise in chip design.

This paper [127] reviews and proposes various strategies to accelerate and optimize LLM, addressing the computational and memory challenges associated with their deployment. It covers algorithmic improvements, including early exiting and parallel decoding, and introduces hardware-specific optimizations through LLM-hardware co-design. The paper presents frameworks like Medusa for parallel decoding, achieving speedups up to 2.8x, and SnapKV for memory efficiency improvements. It also explores High Level Synthesis (HLS) applications with frameworks like ScaleHLS and HIDA, which convert LLM architectures into hardware accelerators. These advances improve LLM performance in real-time applications, such as NLP and EDA, while reducing energy consumption and improving efficiency.

From English to ASIC [128] explores the application of LLMs in automating hardware design using HDLs like Verilog for ASICs. The authors focus on improving the precision of LLM-generated HDL code by fine-tuning Mistral 7B and addressing challenges such as syntax errors and the scarcity of quality training datasets. By creating a labeled Verilog dataset and applying advanced optimization techniques such as LoRA and DeepSpeed ZeRO, the paper shows significant improvements in code generation accuracy, with up to a 20% increase in pass@1 metrics. The paper’s contributions include optimizing memory usage and inference speed, making LLMs more practical for EDA in hardware development. Table 3 provides an overview of all the papers discussed in the hardware design subsection for comparison.

2.2.2. Hardware/Software Codesign

This paper [129] explores co-design strategies to improve training speed and scalability of deep learning recommendation models. The authors emphasize the integration of software and hardware design to achieve significant performance gains, addressing the computational intensity and resource demands of training large models. The study presents various techniques for optimizing algorithms and hardware architectures, ensuring efficient utilization of resources. The paper showcases experimental results demonstrating the effectiveness of co-design strategies in accelerating model training. These results highlight improvements in training times and scalability, making it feasible to handle larger datasets and more complex models. By focusing on the co-design approach, the study provides valuable insights into achieving faster and more scalable training processes, which are essential for the ongoing advancement of deep learning recommendation systems.

This paper [130] explores the integration of software and hardware design principles to optimize the performance of LLMs. It focuses on the co-design approach, which synchronizes the development of software algorithms and hardware architecture to achieve efficient processing and better utilization of resources. This synergy is essential in managing the computational demands of LLMs, which require significant processing power and memory bandwidth. The authors discuss various strategies for optimizing both hardware (such as specialized accelerators and memory hierarchies) and software (such as algorithmic improvements and efficient coding practices) to enhance the performance and efficiency of LLMs. Furthermore, the paper delves into the application of this co-design methodology in design verification processes. Design verification, a critical phase in the development of digital systems, benefits from the enhanced capabilities of co-designed LLMs. By leveraging optimized LLMs, verification tools can process complex datasets and simulations more effectively, leading to more accurate and faster verification results. The integration of co-designed LLMs into verification workflows helps in identifying design flaws early, reducing the time and cost associated with the design and development of hardware systems. The paper highlights case studies and experimental results that demonstrate the practical benefits and improvements achieved through the software/hardware co-design approach in real-world verification scenarios.

This paper [131] investigates the potential of leveraging LLMs in the co-design process of software and hardware, specifically for designing Compute-in-Memory (CiM) Deep Neural Network (DNN) accelerators. It explores how LLMs can be utilized to enhance the co-design process by providing advanced capabilities in generating design solutions and optimizing both software and hardware components concurrently. The authors emphasize the importance of LLMs in automating and improving the design workflow, leading to more efficient and effective development of CiM DNN accelerators. The paper presents a detailed case study demonstrating the application of LLMs in the co-design of CiM DNN accelerators. Through this case study, the authors illustrate how LLMs can aid in identifying optimal design configurations and addressing complex design challenges. The study shows that LLMs can significantly reduce the time and effort required for design iterations and verification, thereby accelerating the overall development process. The findings suggest that integrating LLMs into the co-design framework can result in substantial performance gains and resource savings, highlighting the viability and benefits of using LLMs in the co-design of advanced hardware accelerators.

2.2.3. Hardware Accelerators

This paper [132] presents a comprehensive dataset specifically designed to facilitate the generation of AI accelerators driven by LLMs. The dataset includes a diverse range of hardware design benchmarks, synthesis results, and performance metrics. The goal is to provide a robust foundation for training LLMs to understand and optimize hardware accelerators effectively. By offering detailed annotations and a variety of design scenarios, the dataset aims to enhance the ability of LLMs to generate efficient and optimized hardware designs. The authors detail the structure of the dataset, which covers various aspects of hardware accelerator design, including computational kernels, memory hierarchies, and interconnect architectures. They also discuss the potential applications of the dataset in training LLMs for tasks such as design space exploration, performance prediction, and design optimization. The paper demonstrates the utility of the dataset through several case studies, showing how LLMs can leverage the provided data to generate and optimize hardware accelerators with significant improvements in performance and efficiency.

This study [110] explores the use of LLMs to detect and optimize code for hardware acceleration. The authors propose a methodology where LLMs analyze software codebases to identify sections that can benefit from hardware acceleration. The LLMs are trained to recognize patterns and structures within the code that are amenable to acceleration, such as loops and parallelizable tasks. The paper presents an evaluation of this methodology using several open-source projects, demonstrating that LLMs can effectively identify and suggest optimizations for hardware acceleration. The authors also highlight the potential for integrating this approach into existing development workflows, allowing for seamless detection and acceleration of critical code sections. The study concludes by discussing the challenges and future directions for improving the accuracy and applicability of LLM-driven code detection for hardware acceleration.

Gen-acceleration [133] focuses on the pioneering efforts to use LLMs for the automatic generation of hardware accelerators. The paper outlines a novel framework where LLMs are employed to translate high-level design specifications directly into hardware accelerator designs. The approach leverages the NLP capabilities of LLMs to understand and interpret complex design requirements and convert them into efficient hardware architectures. The authors provide a detailed analysis of the framework, including its architecture, training process, and performance evaluation. They demonstrate the effectiveness of the approach through multiple case studies, showing that LLMs can generate hardware accelerators that meet or exceed the performance of manually designed counterparts. The paper also discusses the potential for this technology to revolutionize the hardware design process, making it more accessible and efficient.

NeuPIMs [134] introduces a heterogeneous acceleration framework that combines Neural Processing Units (NPUs) with Processing-In-Memory (PIM) technologies to enhance the performance of batched LLM inferencing. The paper discusses the architectural innovations that enable this combination, focusing on how NPUs and PIM can work together to overcome the memory bandwidth limitations and computational bottlenecks typically associated with LLM inferencing. The authors provide a comprehensive evaluation of the NeuPIMs framework, highlighting its performance benefits across various LLM benchmarks. They demonstrate significant improvements in throughput and energy efficiency compared to traditional GPU-based solutions. The paper also delves into the technical details of the NPU-PIM integration, including the data flow, memory management, and synchronization mechanisms that enable efficient batched inferencing.

The paper [135] presents the LCM (LLM-focused Hybrid Scratch-Pad Memory (SPM)-cache) architecture designed for multi-core AI accelerators to address the growing computational demands of LLMs. By integrating a hybrid system combining SPM and a shared cache, this architecture provides shorter compilation times and better memory management, particularly for LLMs using mixed-precision quantization. The proposed system utilizes a Tensor Management Unit (TMU) for efficient cache handling and employs innovative hardware prefetching and dead block prediction strategies to mitigate memory access issues. The system outperforms conventional SPM-based architectures, showing up to 50.5% performance improvements in specific scenarios. Table 4 provides an overview of all the papers discussed in the hardware accelerators subsection for comparison.

2.2.4. Hardware Security

DIVAS (Distributed Intelligence for Verification and Security) [136] introduces a comprehensive framework utilizing LLMs to improve the security of System on Chip (SoC) designs. DIVAS integrates multiple security analysis tools and LLMs to provide an end-to-end solution for detecting vulnerabilities and enforcing security policies in SoC designs. The framework automates the identification of security threats and applies policy-based protection mechanisms, with the aim of streamlined and fortifying the security analysis process. DIVAS employs LLMs for various tasks such as vulnerability assessment, anomaly detection, and generating mitigation strategies. The authors detail the system architecture, including its integration with existing SoC design tools and the use of LLMs to interpret and analyze complex security data. The experimental results demonstrate the effectiveness of DIVAS in identifying security vulnerabilities and implementing policies, leading to a significant improvement in SoC security. The paper concludes with a discussion of potential improvements, such as the incorporation of real-time monitoring and refinement LLMs to handle a wider range of security scenarios.

This paper [137] addresses the potential pitfalls of relying on LLMs for hardware specification, particularly in the context of security. The authors argue that, while LLMs can accelerate the specification process, they often generate specifications that are syntactically correct but semantically flawed, leading to security vulnerabilities. The paper presents case studies where LLM-generated specifications resulted in security issues, emphasizing the need for formal methods to verify these specifications. The authors propose a hybrid approach that combines LLMs with formal verification techniques to ensure the correctness and security of hardware specifications. They present a framework that uses LLMs to generate initial specifications, followed by formal verification tools to validate and correct these specifications. Experimental results show that this approach significantly reduces the incidence of security flaws compared to using LLMs alone. The paper concludes with a discussion on the limitations of current LLMs in understanding complex security requirements and the importance of integrating formal methods to achieve reliable and secure hardware designs.

This paper [138] also explores the use of LLMs to identify and fix security bugs in hardware designs. The authors describe a system where LLMs analyzes hardware code to detect potential security vulnerabilities and suggest fixes. This approach aims to automate the bug-fixing process, reducing the time and expertise required to secure hardware designs. The paper details the methodology for training LLMs on hardware security datasets, the types of bugs the system can identify, and the accuracy of the suggested fixes. Experimental results indicate that LLMs can effectively identify and propose solutions for a wide range of security bugs, though their effectiveness varies with the complexity of the bugs. The authors discuss the limitations of LLMs in understanding intricate hardware interactions and suggest future work to improve the robustness of the models and expand their applicability to more complex security scenarios.

This paper [139] examines the potential misuse of general-purpose LLMs in designing hardware Trojans, malicious circuits embedded in hardware designs. The authors demonstrate how LLMs, typically used for benign purposes, can be repurposed to create sophisticated Trojans that are difficult to detect. The paper presents a series of experiments where LLMs are used to generate Trojan designs and assesses their effectiveness and stealthiness. The authors highlight the risks posed by the accessibility of powerful LLMs and the need for robust detection mechanisms. They propose countermeasures, including enhanced verification processes and the development of specialized LLMs trained to recognize and flag suspicious patterns in hardware designs. The paper concludes by discussing the ethical implications of LLMs in hardware design and the importance of proactive measures to prevent their misuse in creating security threats.

LLM for SoC Security [140] explores the transformative impact of LLMs on the security of SoC designs. The paper argues that LLMs represent a significant advancement in the ability to analyze and secure SoC architectures. The authors describe various applications of LLMs in SoC security, including vulnerability detection, threat modeling, and automated patch generation. The paper provides detailed case studies demonstrating the effectiveness of LLMs in enhancing SoC security. Experimental results show that LLMs can identify vulnerabilities more efficiently and accurately than traditional methods. The authors discuss the challenges in integrating LLMs with existing security workflows and propose solutions to address these challenges. The paper concludes by highlighting the potential for LLMs to redefine SoC security practices and the need for continued research to fully realize their potential.

This paper [141] discusses the dual role of LLMs in chip design: as tools for improving design efficiency and as potential sources of security risks. The authors examine the capabilities of LLMs in automating various aspects of chip design, including specification, verification, and optimization. They also highlight the security risks associated with LLM-generated designs, such as the inadvertent introduction of vulnerabilities and the potential for malicious use. The authors propose a framework for building trust in LLM-generated designs by incorporating rigorous security checks and validation processes. They present case studies where LLMs successfully aided in chip design and instances where they introduced security risks. The paper concludes with recommendations for developing secure LLM workflows, including enhanced training protocols and collaboration between LLM developers and security experts to mitigate risks and ensure the reliability of LLM-assisted chip designs.

This paper [142] investigates the use of LLMs to assist in fixing security bugs in hardware code. The authors present a system where developers can interactively prompt LLMs to identify and correct security vulnerabilities in hardware designs. The approach aims to leverage the language understanding capabilities of LLMs to enhance the efficiency and accuracy of bug fixing. The paper details the interactive prompting system, the types of security bugs it can address, and the effectiveness of the LLM-generated fixes. Experimental results show that the system can significantly reduce the time required to identify and fix security bugs, though the success rate varies with the complexity of the bugs. The authors discuss the limitations of current LLMs in handling complex hardware security scenarios and suggest future research directions to improve the system’s robustness and expand its capabilities.

This paper [143] explores the use of LLMs to generate security assertions for hardware designs. Security assertions are critical for verifying that hardware operates as intended without vulnerabilities. The authors propose a framework where LLMs are trained to understand the hardware specifications and automatically generate the corresponding security assertions. The paper demonstrates the effectiveness of this approach through multiple case studies, showing that LLM-generated assertions can identify security issues early in the design process. The authors also discuss the challenges of training LLMs to understand complex hardware specifications and the need for continuous improvement in the training datasets. The study concludes that LLMs has a significant potential to enhance the security verification process by providing accurate and automated security assertions.

This paper [145] explores how LLMs can be leveraged to enhance SoC security by automating the HDL generation process, bug fixing, and security verification. It highlights the potential of LLMs such as GPT, ChatGPT, and BERT to assist in SoC security through NLP and contextual reasoning. The authors emphasize the role of LLMs in automating the detection of security vulnerabilities across the complex design process of SoCs, offering automated fixes and providing a framework for generating secure HDL code. The paper focuses on the importance of policies and asserts that LLMs can not only identify vulnerabilities but also automatically generate assertions and security policies to mitigate these risks. The paper illustrates several use cases, such as detecting vulnerabilities from HDL documentation and using LLMs to map security weaknesses to known vulnerabilities like Common Weakness Enumerations (CWEs). It also outlines the challenges of applying LLMs to hardware design, such as token limitations and incomplete context processing. The paper concludes by identifying current limitations and suggests directions for improving LLM integration in SoC security.

SoCureLLM [144] introduces an innovative LLM-based framework that addresses the limitations of contemporary hardware security verification techniques, especially their adaptability and scalability for large-scale SoC designs. The framework leverages LLMs’ advanced reasoning and pattern recognition abilities to detect vulnerabilities in large SoC designs and generate comprehensive security policies. Unlike previous methods that focus on smaller designs, SoCureLLM overcomes challenges related to token limitations and memory constraints, effectively partitioning designs into smaller code snippets and analyzing them contextually. In evaluations, SoCureLLM successfully detected 76.47% of security bugs on vulnerable RISC-V SoCs and formulated 84 novel security policies. The framework introduces automation in both the detection and the generation of security policies based on threat models, significantly reducing manual effort. The authors emphasize the importance of automated security policy databases to address SoC vulnerabilities and enhance presilicon verification. The paper concludes by highlighting the scalability and effectiveness of SoCureLLM in verifying large designs while proposing potential refinements for future iterations. Table 5 compares all the paper discussed in this subsection.

2.2.5. Hardware Debugging

In Figure 8, we demonstrate an HDL debugging for a shift register, where the input prompt describes the desired behavior and adjustments. The results of this input prompt processed by different LLMs are presented in Figure 9, Figure 10, and Figure 11, showcasing the outputs from ChatGPT 4o, Gemini, and Copilot, respectively. These figures illustrate the varying approaches taken by each model to interpret and modify the HDL design.

HDLdebugger [146] presents a novel approach to debugging HDL code using LLMs. The authors introduce a system that leverages the capabilities of LLMs to automate and streamline the debugging process for HDL, which includes languages like Verilog and VHDL. The system is designed to identify syntax and semantic errors, suggest corrections, and provide explanations for detected issues, thereby reducing the manual effort and expertise required in traditional debugging methods. The paper details the architecture of HDL debugger, which includes modules for parsing HDL code, generating debugging suggestions, and integrating user feedback to refine its outputs. The authors provide experimental results showing that HDL debugger can effectively identify and correct a wide range of common errors in HDL code, significantly improving debugging efficiency. The study concludes by discussing potential enhancements, such as expanding the system’s knowledge base and incorporating more advanced ML techniques to handle more complex debugging scenarios.

LLM4SecHW [147] explores the use of a domain-specific LLM for hardware debugging, particularly focusing on security-related issues. The paper introduces a specialized LLM trained on hardware design and security datasets to assist in identifying and fixing security vulnerabilities in hardware designs. The system aims to enhance the debugging process by providing targeted suggestions and fixes for security bugs, which are often more challenging to detect and resolve than general errors. The authors describe the development of LLM4SecHW, including the data collection, training process, and the integration of the model into existing hardware debugging workflows. Experimental results demonstrate that LLM4SecHW is effective in detecting and resolving a variety of security-related issues in hardware designs, outperforming general-purpose LLMs in both accuracy and relevance. The paper concludes with a discussion on the limitations of the current model and future directions for improving its performance, such as expanding the training dataset and refining the model’s understanding of complex security scenarios. Table 6 compares the papers discussed in this subsection.

2.2.6. Hardware Verification

In Figure 12, we demonstrate an HDL verification for a shift register, where the input prompt describes the desired behavior and adjustments. The results of this input prompt processed by different LLMs are presented in Figure 13, Figure 14, and Figure 15, showcasing the outputs from ChatGPT 4o, Gemini, and Copilot, respectively. These figures illustrate the varying approaches taken by each model to interpret and modify the HDL design.

AssertLLM [148] focuses on utilizing multiple LLMs to generate and evaluate hardware verification assertions directly from the design specifications. It presents a novel approach where various LLMs are used to understand the design specifications and subsequently generate appropriate verification assertions that ensure the correctness of hardware designs. This methodology aims to streamline the process of hardware verification by automating the generation of assertions, which traditionally requires significant manual effort and expertise. The authors detail the framework architecture, which includes parsing design specifications, employing multiple LLMs for assertion generation, and using evaluation metrics to assess the quality and accuracy of the generated assertions. The paper showcases experimental results indicating that the multi-LLM approach produces assertions that are both accurate and relevant, reducing the overall time and effort required for hardware verification. The study concludes by discussing potential improvements and future directions, such as refining LLM training datasets and integrating the framework with existing verification tools.

This paper [149] explores the application of LLMs to assist in the formal verification of RTL designs. It introduces methods for leveraging LLMs to automatically generate properties and invariants that are crucial for formal verification processes. The goal is to improve the efficiency and accuracy of formal verification by reducing the dependency on manual property generation, which can be time consuming and error-prone. The authors present a detailed workflow where LLMs are used to interpret the RTL code and produce formal properties that can be verified using formal methods. They provide experimental results that demonstrate LLM-generated properties can significantly augment the formal verification process, catching more design errors, and reducing verification time. The paper also discusses the limitations of current LLMs in understanding complex RTL semantics and suggests future research directions to improve LLM capabilities in formal verification tasks.

This paper [150] addresses the optimization of design verification processes using ML techniques, presenting an open source solution for the hardware design community. The authors outline a framework that integrates ML algorithms to predict verification outcomes, identify potential design issues early, and prioritize verification tasks. This approach aims to enhance the efficiency and effectiveness of the design verification process by leveraging data-driven insights. The framework’s implementation is discussed in detail, including the selection of ML models, training datasets, and integration with existing verification workflows. Experimental results are provided to demonstrate the improvements in verification coverage and reduction in verification time achieved by the ML-enhanced framework. The paper concludes with a discussion on the open-source nature of the solution, encouraging community collaboration and further development, as well as outlining potential future enhancements such as incorporating more sophisticated ML models and expanding the dataset.

VerilogEval [151] focuses on assessing the performance of LLMs in generating Verilog code, which is critical for hardware design. The paper provides a comprehensive evaluation framework to benchmark various LLMs’ ability to generate accurate and efficient Verilog code from high-level descriptions. The authors detail the criteria used for the evaluation, including code accuracy, readability, and resource efficiency. The study presents extensive experimental results comparing different LLMs on a series of Verilog code generation tasks. These results highlight the strengths and weaknesses of each model, providing insights into how well they can support hardware designers in automating code generation. The paper concludes by discussing the implications of these findings for future research and development, suggesting ways to improve LLM training for better performance in hardware design applications, such as using more domain-specific datasets and refining model architectures.

This paper [152] proposes a method for enhancing verification productivity by translating natural language descriptions into System Verilog assertions with a circuit-aware approach. The authors introduce a system that leverages NLP techniques to interpret design specifications and automatically generate corresponding verification assertions. This approach aims to bridge the gap between high-level design intents and low-level verification tasks, making the verification process more intuitive and efficient. The system’s architecture is described in detail, including the use of NLP models to understand natural language inputs and generate accurate System Verilog assertions that are contextually aware of the circuit design. Experimental results are presented to demonstrate the effectiveness of the proposed method, showing significant improvements in the speed and accuracy of assertion generation compared to manual methods. The paper concludes with discussions on the potential impact of this approach on the verification industry and future research directions to further refine the translation process.

ChipNeMo [153] explores the adaptation of LLMs specifically for the domain of chip design. The paper presents techniques for fine-tuning LLMs on chip design-related datasets to enhance their performance in tasks such as code generation, optimization, and verification. The goal is to create domain-adapted models that can understand and generate chip design content more effectively than general-purpose LLMs. The authors describe the process of collecting and curating domain-specific datasets, the fine-tuning methodology, and the evaluation metrics used to assess the performance of the adapted models. Experimental results show that ChipNeMo outperforms general-purpose LLMs in various chip design tasks, demonstrating the benefits of domain adaptation. The paper concludes with a discussion on the broader implications of domain-adapted LLMs for the chip design industry and potential future research directions, including the integration of these models into existing design workflows and further refinement of adaptation techniques.

LLM4DV [154] investigates the use of LLMs to generate test stimuli for hardware verification. The paper presents methods for training LLMs to understand hardware design specifications and produce relevant test cases that can be used to verify the correctness and performance of hardware designs. This approach aims to automate the test generation process, reducing the time and effort required for comprehensive hardware verification. The authors provide a detailed description of the training process, including the selection of training data, model architecture, and the generation of test stimuli. Experimental results demonstrate that LLM4DV-generated test cases are effective in identifying design issues and achieving high coverage in verification tasks. The paper discusses the potential for further enhancing test generation capabilities by incorporating more sophisticated LLMs and integrating the approach with existing verification frameworks. It concludes by highlighting the benefits of using LLMs in hardware verification, such as increased efficiency and reduced manual effort.

This paper [155] explores the use of LLMs to assist in generating hardware assertions, which are critical for verifying the correct behavior of hardware designs. The authors present a framework that uses LLMs to interpret design documentation and automatically produce assertions that can be used in formal verification processes. The goal is to streamline the assertion generation process and ensure comprehensive coverage of design specifications. The framework architecture is described, including the use of LLMs to parse design documents and generate assertions, and evaluation methods to assess the quality and precision of these assertions. Experimental results indicate that the LLM-assisted approach produces high-quality assertions that can significantly improve verification efficiency. The paper concludes with a discussion on the limitations of current LLMs in understanding complex hardware designs and potential future research directions to improve the capabilities of LLMs in generating assertions, such as incorporating more detailed training data and refining model architectures. Table 7 compares all the papers discussed in this subsection.

2.3. Use Cases and Success Stories

LLMs have seen promising applications in real-world hardware design projects, particularly in areas such as Design for Manufacturing (DfM) and embedded system development. LLMs have been integrated into DfM processes to help generate, optimize, and modify designs based on manufacturing constraints. For example, in a study, an LLM was used to modify a parametric design of parts like LEGO bricks and CNC machined components, offering improvements in manufacturability and efficiency. LLM could even suggest optimal manufacturing techniques and improve design geometry to streamline production [156].

In the realm of hardware design, LLMs have also been employed to generate HDL code such as Verilog. For instance, LLMs can be used to write, annotate, and debug hardware code, potentially improving design efficiency and reducing errors. While still in experimental stages, these systems show potential in automating parts of the hardware development process [157].

LLMs have shown promise in generating code for embedded systems, particularly in automating the development of C/C++ code for hardware platforms. While their use is still evolving in this field, LLMs offer potential to make hardware programming more approachable and assist in debugging and improving low-level code [158].

LLMs, such as GPT-3.5 and GPT-4, have been applied to generate and refine Verilog code used in hardware design. A process called "VeriRectify" allows the LLM to take error diagnostics from simulators (e.g., iverilog) and iteratively refine the generated Verilog code, significantly improving its correctness and performance. This approach has led to successful code generation for complex hardware designs like multipliers, incorporating both error correction and PPA checks, ensuring optimal designs [151,159].

LLMs are also being explored for validating system configurations. The "Ciri" framework leverages models like GPT-4 to detect misconfigurations in large-scale systems (e.g., HBase, ZooKeeper). Ciri uses prompt engineering and few-shot learning to improve accuracy, achieving promising F1 scores in identifying configuration errors. While it excels at catching certain types of misconfigurations, challenges remain, such as handling dependency violations [160]

3. Challenges

The most important challenges of LLMs in hardware design and verification are shown in Figure 16.

3.1. Training Challenges

Training LLMs for hardware design and verification faces substantial challenges primarily due to the nature of the data and the computational demands involved. Hardware design and verification data are highly specialized, often proprietary, and not readily available in public repositories. This scarcity of high-quality, annotated data makes it difficult to train effective models. Moreover, the data’s heterogeneity, encompassing HDL code, schematic diagrams, and natural language documentation, complicates the integration into a cohesive training set. Additionally, the technical expertise required for labeling such data further complicates and prolongs the preparation process. The resource-intensive nature of training LLMs, which demands powerful GPUs or TPUs, significant memory, and extensive storage, is another major hurdle. The energy consumption associated with training these models is also considerable, raising sustainability concerns.

The domain-specific nature of hardware design and verification adds another layer of complexity. LLMs must understand and accurately interpret the intricate language and concepts specific to this field, ensuring that generated outputs are both syntactically correct and semantically meaningful. Errors in generated code can lead to significant functional issues, necessitating robust verification and validation processes. The black-box nature of LLMs presents interpretability challenges, making it difficult to trace and understand their decision-making processes. This lack of transparency can hinder trust and acceptance among hardware engineers, who require clear explanations for the model’s outputs. Integrating LLMs into existing workflows without causing disruptions, ensuring compatibility with various industry tools, and providing adequate training and support for users are critical for successful implementation. Addressing these challenges requires a multidisciplinary approach, continuous collaboration, and advances in both ML and hardware design methodologies.

To overcome data scarcity and quality issues, collaboration between academia, industry, and research institutions is crucial. Establishing partnerships can facilitate the sharing of proprietary data under strict confidentiality agreements, expanding the pool of available training data. Additionally, crowdsourcing initiatives and competitions can be organized to gather more annotated data. Developing advanced data augmentation techniques can help generate synthetic data that closely mimics real-world hardware design scenarios, enhancing the diversity and richness of the training datasets. Implementing semi-supervised or self-supervised learning methods can also leverage unlabeled data to improve model performance.

3.2. Adaptation to Hardware-Specific Vocabulary

Adapting LLMs to hardware-specific vocabulary for hardware design and verification involves enhancing the models’ ability to understand and generate text that accurately reflects the specialized terminology and concepts used in this field. This process begins with the careful curation of domain-specific corpora, which include HDLs like VHDL and Verilog, technical documentation, research papers, and industry standards. These corpora provide the foundational data that the models need to learn the syntax, semantics, and context of hardware-related terms. Fine-tuning pre-trained LLMs on these specialized datasets enables the models to grasp the nuances of hardware design language, ensuring that they can generate syntactically correct and semantically meaningful code. Additionally, incorporating glossaries and thesauri specific to hardware design during the training phase can help models better understand and use technical terms accurately.

Furthermore, integrating feedback loops with domain experts during the training and evaluation phases is crucial for refining the models’ understanding of hardware-specific vocabulary. Experts can review and provide insights on the models’ outputs, identifying areas where the terminology might be used incorrectly or where the context may be misunderstood. This feedback can be used to iteratively improve the models, making them more adept at handling the intricate and precise language of hardware design. Leveraging transfer learning techniques, where models trained on general data are subsequently fine-tuned on hardware-specific data, also enhances their adaptability. This approach allows the models to retain their broad language understanding while becoming proficient in the specialized vocabulary of hardware design and verification. By continuously refining the training process and incorporating expert feedback, LLMs can become powerful tools for assisting in hardware design and verification tasks, offering precise and reliable language understanding tailored to the domain.

3.3. Explainability and Interpretability

Explainability and interpretability are critical issues when applying LLMs to hardware design and verification, given the need for transparency, traceability, and trust in the model’s outputs. In hardware design, even minor errors can lead to significant functional flaws, making it essential for engineers to understand how and why an LLM arrived at a particular decision or generated specific code. The black-box nature of LLMs, where the internal workings are opaque and difficult to interpret, poses a significant challenge. This lack of transparency can hinder engineers’ ability to trust the model’s outputs, as they cannot easily verify the correctness or understand the rationale behind specific decisions. Moreover, in the highly specialized field of hardware design, the contextual nuances and technical details are crucial, and any misinterpretation can have far-reaching consequences.

To address these issues, several strategies can be employed to enhance the explainability and interpretability of LLMs in hardware design and verification. One approach is the use of attention mechanisms and visualization tools that highlight which parts of the input data the model focused on when generating its outputs. This can provide engineers with insights into the model’s decision-making process and help them understand how specific terms and concepts were interpreted. Another strategy is incorporating explainable AI (XAI) techniques, which aim to make the outputs of ML models more transparent and interpretable. For instance, generating intermediate representations or step-by-step explanations can help bridge the gap between the model’s complex computations and the human engineer’s understanding. Additionally, involving domain experts in the training and validation process can ensure that the model’s explanations align with the technical realities of hardware design. By continuously refining the model based on expert feedback and focusing on transparency, LLMs can become more reliable and trustworthy tools in the hardware design and verification process.

3.4. Integration with Existing Design Tools

Integrating LLMs with existing design tools in hardware design and verification presents several challenges due to the specialized and complex nature of these tools. Hardware design workflows rely heavily on sophisticated software such as EDA tools, simulation environments, and formal verification systems. These tools have been optimized over years to handle specific tasks efficiently, and introducing LLMs into this ecosystem requires ensuring seamless interoperability. One significant issue is compatibility: the LLMs must be able to interact with various file formats, data structures, and APIs used by these tools. This necessitates the development of robust interfaces and integration layers that can bridge the gap between the general-purpose nature of LLMs and the specialized requirements of hardware design tools.

Additionally, the integration process must ensure that the introduction of LLMs does not disrupt existing workflows. Hardware engineers are accustomed to certain processes and toolchains that have been fine-tuned for productivity and reliability. Introducing LLMs should enhance these workflows by providing added value, such as automating repetitive tasks, offering code suggestions, and assisting in error detection, without requiring significant changes to established practices. This requires careful design of user interfaces and interaction models, ensuring that the outputs from LLMs are presented in a way that is intuitive and easily actionable for engineers. Furthermore, extensive testing and validation are needed to ensure that the LLMs’ outputs are reliable and accurate, thereby gaining the trust of the engineers. Training sessions and comprehensive documentation will also be necessary to help users understand and effectively utilize the new capabilities brought by LLMs. By addressing these integration challenges thoughtfully, LLMs can be successfully incorporated into the hardware design and verification ecosystem, enhancing efficiency and innovation.

4. Open Issues

4.1. Unexplored Applications

LLMs have the potential to significantly improve a variety of sub-domains by automating and optimizing complex tasks. LLMs can streamline the HLS process by converting high-level code into RTL code and optimizing for performance, area, and power. Similarly, in HDL Generation, LLMs can automate the generation of Verilog, VHDL, or SystemVerilog based on functional specifications.

Moreover, LLMs can enhance Component Integration by automating the management of hardware module interactions, while in Design Optimization, they can suggest optimal configurations that balance performance and power trade-offs. In areas like FSM design and Design Space Exploration (DSE), LLMs can assist in optimizing state transitions, identifying unreachable states, and exploring multiple design alternatives for better performance.

LLMs are also helpful in power-conscious tasks like Power-Aware Design, providing suggestions for power-saving techniques, and in Timing Analysis and Optimization, where they assist in meeting timing constraints through clock tree and propagation optimizations. Other physical design tasks, such as floor planning and physical layout, benefit from the ability of LLMs to optimize component placement and routing for improved performance and reduced congestion.

The role of LLMs extends to Low-Power Design Techniques, Hardware Accelerators, and Clock Tree Synthesis (CTS), where they help designers balance power with performance, create architectures for AI accelerators, and optimize clock distribution networks. LLMs can also influence Chip Architecture Design by proposing architectural suggestions and improving data flow.

In advanced design areas such as ASIC Design, Fault-Tolerant Design, and Verification Plan Generation, LLMs provide automated support for optimizing integrated circuits, generating redundancy schemes, and creating comprehensive verification plans. For detailed tasks and applications of LLMs in each sub-domain, refer to Table 8, which offers a structured overview of how LLMs contribute to enhancing various aspects of hardware design.

4.2. Research Gaps

The current research landscape for LLMs in hardware design reveals several significant gaps that limit their application in critical areas such as formal verification, design optimization, and security. Despite the potential of LLMs, their integration with formal methods remains largely absent, posing risks to safety-critical systems. Moreover, the lack of contextual understanding in optimizing complex design trade-offs between PPA hampers multi-objective optimization efforts. Security vulnerabilities specific to hardware designs are also underexplored, leaving hardware systems at risk. The shortage of specialized datasets further compounds the challenge, limiting LLMs’ effectiveness in hardware-specific tasks like circuit design and verification. As hardware designs scale in complexity, managing large systems like SoCs becomes increasingly challenging for LLMs, and areas such as analog/mixed-signal design, hardware/software codesign, and post-silicon validation remain underdeveloped. Table 9 provided details on the specific gaps in each of these areas and their respective impacts, offering a clearer picture of the current limitations and opportunities for further research. The most significant sub-domains for each domain can be mentioned as follows:

• HLS

  -

  Behavioral Synthesis: Converting behavioral descriptions directly into RTL [161,162].

  -

  Scheduling: Managing operation timing to meet performance and resource constraints [163,164].

  -

  Resource Allocation: Assigning operations to hardware resources such as registers or functional units [165,166].

  -

  Loop Unrolling and Pipelining: Enhance parallelism by optimizing loop handling [167,168].

  -

  Bit-width Optimization: Minimizing the width of variables without sacrificing accuracy [169].

  -

  Control Flow Management: Managing control flow statements (if-else, switch-case) for hardware synthesis [170].

  -

  Memory Access Optimization: Efficiently handling memory access patterns to reduce latency [171,172].

  -

  Interface Generation: Creating interfaces for communication between blocks during synthesis [173].

• HDL Generation

  -

  Synthesis-ready HDL Code Generation: Automatically generating Verilog or VHDL that is ready for synthesis [174].

  -

  Parameterized HDL Code: Creating reusable code with configurable parameters [175].

  -

  State Machine Code Generation: Creating FSMs based on behavioral specifications [176,177].

  -

  HDL Code Translation: Translating high-level or behavioral code into synthesizable HDL [178,179].

  -

  Testbench Generation: Automatically generating testbenches for HDL verification [180,181,182].

  -

  Hierarchical Module Design: Automatically generating modular and hierarchical HDL blocks [183].

  -

  Assertions and Constraints: Generating assertion-based HDL for formal verification [148,184].

  -

  Code Formatting and Cleanup: Ensuring HDL code is clean, well-formatted, and error-free 56.

• Component Integration

  -

  Interface Synthesis: Automatically generating interfaces (e.g. Advanced eXtensible Interface (AXI), Advanced Microcontroller Bus Architecture specification (AMBA)) between hardware modules [185].

  -

  Signal Mapping: Automating the signal connection and mapping between modules [186].

  -

  Inter-module Communication: Managing and optimizing data and control flow between different hardware blocks [187].

  -

  Bus Arbitration: Design of efficient bus systems for shared resources [188].

  -

  Protocol Handling: Automating protocol management for communication between modules 7.

  -

  System Verification: Automatically generating testbenches to verify integrated systems [176,189].

  -

  Hierarchical Integration: Building hierarchical hardware designs by integrating reusable IP blocks [190,191].

• Design Optimization

  -

  Timing Optimization: Optimizing designs to meet timing constraints [192,193].

  -

  Power Optimization: Minimizing power consumption through design improvements [194,195,196].

  -

  Area Optimization: Reducing the overall silicon area used by the hardware [197].

  -

  Pipelining and Parallelization: Introducing pipelining and parallelization to improve performance [198,199].

  -

  Data Path Optimization: Optimizing the data path for reduced latency and better resource utilization [200].

  -

  Power-Performance Trade-offs: Balancing between performance gains and power savings [201,202].

  -

  Critical Path Reduction: Shortening the critical path to improve clock frequency [203,204].

• FSM Design

  -

  State Minimization: Reducing the number of states to optimize hardware usage [205,206,207].

  -

  Hierarchical FSM Design: Creating complex FSMs using a hierarchical approach [208].

  -

  Error Detection FSMs: Design of FSMs with built-in error detection mechanisms [209,210].

  -

  Power-aware FSM Design: Creating FSMs optimized for low power consumption [211].

  -

  State Encoding Optimization: Optimizing state encodings (e.g., one-hot, binary) for efficiency [212].

  -

  Timing-aware FSM Design: Ensure that FSMs meet timing constraints [213].

• DSE

  -

  Pareto-optimal Design Space Exploration: Exploring the design space to identify Pareto-optimal trade-offs between power, area, and performance [214].

  -

  Multi-objective Optimization: Optimizing designs for multiple conflicting objectives (e.g., power vs. performance) [215].

  -

  Parametric Design Exploration: Exploring various parameter configurations to achieve optimal results [216].

  -

  Constraint-driven Design: Ensure that all design options meet predefined constraints [217].

  -

  ML-Assisted DSE: Using ML models to predict design performance and help in exploration [218,219,220].

  -

  Scenario-based DSE: Exploring designs based on different use-case scenarios (e.g., high-performance vs. low-power modes) [221].

• Power-Aware Design

  -

  Dynamic Power Management: Real-time power adjustments based on workload [201,202,222,223].

  -

  Clock Gating: Reducing power consumption by turning off clocks in idle modules [194,224].

  -

  Voltage Scaling: Dynamically adjust the voltage supply to minimize power usage [225].

  -

  Multi-VDD Design: Design of hardware that operates at multiple voltage levels for different modules [226].

  -

  Leakage Power Reduction: Techniques to reduce leakage power in idle states [227].

  -

  Low-power Memory Design: Optimizing memory access and design to minimize power [228,229].

  -

  Thermal-aware Power Optimization: Ensuring power optimizations also consider thermal constraints [230,231].

• Timing Analysis and Optimization

  -

  Static Timing Analysis (STA): Automatically analyzing and optimizing timing paths [232].

  -

  Critical Path Analysis: Identifying and optimizing the critical path to ensure timing closure [233].

  -

  Clock Skew Minimization: Optimizing the clock distribution to minimize the skew in the design [234].

  -

  Multi-clock Domain Design: Managing and optimizing designs with multiple clock domains [235,236].

  -

  Hold and Setup Time Optimization: Ensure that all paths meet the hold and setup time constraints [237].

  -

  Path Delay Optimization: Shortening the longest paths in the design to improve performance [238].

• Floorplanning and Physical Design

  -

  Component Placement Optimization: Place components to minimize delays and area usage [239].

  -

  Power Grid Design: Design of power distribution networks to ensure reliable power delivery [240].

  -

  Routing Congestion Management: Optimize placement to avoid routing congestion and improve performance [241].

  -

  Thermal-aware Floorplanning: Ensure that heat-generating components are placed efficiently for heat dissipation [242,243].

  -

  Timing-aware Floorplanning: Ensure that critical timing paths are optimized in the placement process [244].

• Low-Power Design Techniques

  -

  Multi-threshold Design: Using multiple threshold voltages to optimize power [245,246].

  -

  Adaptive Voltage Scaling (AVS): Dynamically adjust voltage levels for power savings [247,248].

  -

  Dynamic Power Gating: Turn off parts of the circuit during inactivity to save power [249,250].

  -

  Energy-efficient Logic Synthesis: Design circuits with low power consumption from the ground up [251].

  -

  Sleep Mode Design: Implement deep sleep modes for minimal power consumption during idle periods [252].

• Hardware Accelerators

  -

  AI/ML Accelerators: Design of hardware specialized for AI/ML tasks (for example, matrix multiplication, deep learning) [253,254].

  -

  DSP Accelerators: Optimizing hardware for DSP tasks such as filtering and transforms [255,256].

  -

  Parallel Computing Accelerators: Designing hardware for parallel task execution, such as multi-core GPUs or FPGAs [257,258].

  -

  Memory Bandwidth Optimization: Ensure that memory subsystems can handle the data bandwidth required for high-throughput hardware accelerators [259,260].

  -

  Reconfigurable Accelerators: Design flexible hardware accelerators, such as FPGAs, that can be adapted to different tasks [261,262].

  -

  Low-Power Accelerators: Optimizing accelerators to be power efficient, especially for mobile and embedded systems [263,264].

  -

  AI Hardware/Software Co-optimization: Co-design of AI software and hardware to maximize hardware utilization and performance [265,266].

• CTS

  -

  Clock Skew Minimization: Ensure that clock signals arrive at all components simultaneously to minimize skew [267].

  -

  Power-aware CTS: Design of clock trees to minimize power consumption [268].

  -

  Multi-domain Clock Tree Design: Managing multiple clock domains to ensure efficient clock distribution [269].

  -

  Clock Buffer Insertion: Strategic placement of buffers to reduce clock delay and skew [270].

  -

  Gated Clock Design: Reduction of power by turning off the clock in inactive areas of the design [271,272].

  -

  CTS for Low-power Designs: Techniques to reduce clock power consumption, like multi-threshold designs or clock gating [273].

• Chip Architecture Design

  -

  Multi-core Architecture Design: Defining architectures with multiple cores for parallelism and performance improvements [274,275].

  -

  Memory Hierarchy Design: Design of the memory subsystem, including caches, RAM, and register files, to optimize data access speeds [276,277].

  -

  Dataflow Architecture: Optimizing the flow of data between processing units to maximize throughput [278,279].

  -

  Instruction Set Architecture (ISA): Defining and optimizing the instruction set used by processors [280,281].

  -

  On-chip Communication Networks: Design of networks for communication between cores and other processing units [282,283,284,285,286,287].

  -

  Performance Bottleneck Identification: Analyze architectural designs to find and eliminate performance bottlenecks [288,289].

  -

  Scalability Optimization: Ensure that the chip architecture scales well with increasing system complexity (e.g., more cores or memory) [290,291].

• Physical Layout and Routing

  -

  Component Placement Optimization: Identifying the optimal placement of components to reduce routing length and delay [239,292].

  -

  Wire Routing: Definition of the path of signal wires to minimize delay, power consumption, and congestion [293,294].

  -

  Congestion Management: Preventing over-congested regions in the design by optimizing component placement and routing [295,296].

  -

  Thermal-aware Layout: Ensure that heat-generating components are placed strategically for heat dissipation [297,298].

  -

  Signal Integrity Optimization: Preventing issues such as crosstalk and electromagnetic interference in wire routing [299,300].

  -

  Area Optimization: Minimizing the total area occupied by the physical layout of the components [301].

• ASIC Design

  -

  Standard Cell Library Selection: Choosing the right standard cell libraries for performance, power, and area trade-offs [302].

  -

  Custom Cell Design: Design of custom logic cells optimized for specific performance and area requirements [303].

  -

  Power Grid Optimization: Design of efficient power distribution networks across ASICs [304].

  -

  Design-for-Test (DFT): Embedding testability features into the design to ensure that the ASIC is testable after fabrication [305,306].

  -

  ASIC Synthesis: Translating high-level designs into a netlist using synthesis tools [307,308].

  -

  Timing Closure for ASICs: Ensure that the final design meets the timing constraints before manufacturing [309,310].

  -

  Packaging and I/O Design: Optimizing external interfaces and packaging for the ASIC [311].

• Fault-Tolerant Design

  -

  Error Detection and Correction (EDAC): Design hardware with the ability to detect and correct errors (e.g., Error Correction Code (ECC)) [312,313].

  -

  Redundancy Techniques: Implementing redundancy at various levels (e.g., triple modular redundancy, hot spares) [163,314,315].

  -

  Built-in Self-test (BIST): Design self-test circuits that can detect faults during run-time [316,317].

  -

  Fault-tolerant FSMs: Design of finite-state machines that can recover from faulty states [318,319].

  -

  Radiation Hardening: Designing hardware that is resilient to radiation effects, particularly for aerospace applications [320,321].

  -

  Soft Error Mitigation: Techniques for preventing or recovering transient errors caused by cosmic rays or other environmental factors [322,323].

  -

  Failure Mode Analysis: Analyze hardware to predict and mitigate potential failure modes [324,325].

• Verification Plan Generation

  -

  Testbench Automation: Automatically generating testbenches for verifying different parts of the hardware design [182,189].

  -

  Random Test Generation: Creating random test sequences to stress the design and catch edge cases [326].

  -

  Constraint-based Verification: Defining constraints for test generation to ensure valid input/output scenarios [327].

  -

  Formal Verification: Using mathematical techniques to prove that a design meets its specifications [328,329].

  -

  Assertion-based Verification (ABV): Embedding assertions in the design to ensure it behaves as expected under all conditions [330,331].

  -

  Equivalence Checking: Ensure that two versions of a design (e.g., RTL vs. synthesized) are functionally equivalent [332,333].

  -

  Universal Verification Methodology (UVM) and SystemVerilog: Implement advanced verification techniques using UVM and SystemVerilog [334].

4.3. Methodological Improvements

With making significant improvements in hardware design and verification, several aspects of LLMs need to be reinforced. These enhancements would target specific challenges in hardware engineering and leverage LLMs to their full potential in assisting designers and verification teams. The key aspects that should be reinforced are as follows:

1) Domain-Specific Understanding and Contextual Knowledge: LLMs need a deeper and more precise understanding of hardware design languages, methodologies, and tools. While LLMs excel at NLP, they often lack in-depth knowledge of domain-specific languages like Verilog, VHDL, SystemVerilog, and UVM. To truly aid hardware designers, LLMs must be fine-tuned on vast datasets of HDLs, verification code, design documentation, and real-world projects. Additionally, understanding the context of a design, such as the specific requirements of a given project (e.g., low-power design, high-performance, etc.), will enable LLMs to make more relevant suggestions during both the design and verification processes.

2) Enhanced Formal Reasoning Capabilities: LLMs need to improve their formal reasoning abilities, especially for tasks such as formal verification, model checking, and constraint satisfaction, which are essential to hardware verification. Hardware design often involves proving that a design meets certain formal specifications, such as safety or liveness properties. Currently, LLMs struggle with formal logic and mathematical rigor. Enhancing their capability to handle formal methods—like understanding temporal logic, SVA, and finite state machines—would significantly improve their utility in verification tasks. This would allow LLMs to automatically generate and validate formal properties from natural language specifications, ensuring that hardware designs conform to their intended behavior.

3) Code Generation for Synthesis-Ready HDL: While LLMs can generate HDL code from behavioral descriptions, they must become more adept at creating synthesis-ready code. This requires not only understanding how to describe hardware behavior but also generating optimized code that meets the constraints of modern hardware synthesis tools. To achieve this, LLMs need reinforcement in optimizing generated code for real-world constraints such as timing, power, and area. Incorporating feedback from synthesis and place-and-route tools into the LLM’s training data can improve its ability to generate resource-efficient, high-performance HDL designs.

4) Design Space Exploration and Optimization: One of the critical tasks in hardware design is balancing multiple design constraints—performance, power, area, and cost—through DSE. LLMs should be reinforced with advanced optimization techniques and predictive modeling capabilities to help guide the exploration of various design parameters. Reinforcement learning approaches combined with LLMs can enable them to predict the impact of parameter choices on design metrics and suggest optimal configurations based on trade-offs. By enhancing LLMs’ ability to navigate complex design spaces, designers could receive better support in exploring Pareto-optimal designs that balance competing objectives.

5) Error Detection and Debugging: LLMs can play a crucial role in identifying bugs and design flaws, but their error detection capabilities must be reinforced to be more effective in the hardware domain. This includes being able to recognize subtle errors in HDL, such as incorrect state machine transitions, misaligned clock domain crossings, or resource contention. LLMs need to be trained on common hardware design errors and verification failures, improving their ability to offer precise feedback on potential issues. Additionally, LLMs should be reinforced with an understanding of simulation and synthesis reports, enabling them to trace errors back to their root causes and provide actionable debugging suggestions.

6) Verification Automation and Coverage Analysis: Verification is one of the most time-consuming aspects of hardware development. To improve LLMs’ contributions in this domain, they should be reinforced with better tools for generating comprehensive testbenches, performing functional coverage analysis, and creating directed random tests. Specifically, LLMs should be enhanced to recognize coverage gaps in verification plans and generate appropriate tests to fill those gaps, ensuring that designs are thoroughly tested. Furthermore, improving the LLM’s ability to integrate with simulation tools, extract meaningful insights from waveform data, and recommend additional verification steps will reduce the manual burden on verification teams.

7) Learning from Hardware Development Iterations: LLMs should be able to learn from previous iterations of a hardware design to assist in continuous improvement. By analyzing successive versions of a design, LLMs can identify what changes led to better performance, lower power consumption, or reduced area. Reinforcing this ability would enable LLMs to provide context-specific recommendations based on past design choices, helping hardware designers optimize their designs more effectively across multiple development cycles. This capability could also be extended to learning from community-wide datasets of hardware designs to suggest best practices and design patterns that are tailored to specific project goals.

8) Interaction with EDA Tools and Integration into Workflows: For LLMs to be more effective in hardware design, they need tighter integration with EDA tools and workflows. LLMs should be able to interface with common hardware design tools (such as simulation, synthesis, and formal verification tools), extract relevant data, and act on that information in real-time. By integrating LLMs with these tools, designers can receive real-time feedback on design choices, simulation results, or synthesis reports. LLMs should also be capable of automating repetitive tasks within the EDA workflow, such as setting up project configurations, running simulations, and analyzing results, reducing the overall design time.

9) Memory and State Tracking for Large-Scale Projects: Hardware design projects can span months or years and involve numerous changes over time. LLMs should be reinforced with better long-term memory and state-tracking capabilities so that they can keep track of ongoing changes across large projects. This would allow LLMs to assist designers by recalling relevant design decisions, tracking the evolution of specific modules or components, and ensuring consistency across the entire design. This state-tracking ability is crucial for handling complex projects with multiple designers, where coordination and memory of past decisions are key to success.

10) Security and Safety in Hardware Design: LLMs should be enhanced to understand and enforce security and safety requirements during the design process. With the growing need for hardware security, LLMs must be able to detect potential vulnerabilities, such as insecure communication protocols or improper handling of sensitive data. Similarly, in safety-critical designs, such as automotive or aerospace systems, LLMs need reinforcement to ensure compliance with safety standards and protocols. By improving LLMs’ capabilities in these areas, designers can be alerted to potential security risks and safety violations early in the design phase.

5. Conclusions

This survey has explored the emerging role of LLMs in hardware design and verification, presenting an overview of the current state of the literature, key challenges, and open issues. The integration of LLMs into this domain offers significant potential to revolutionize traditional workflows, enhancing productivity and enabling more automated and intelligent design processes. From facilitating high-level hardware description generation to improving verification through natural language interfaces, LLMs provide promising avenues to reduce complexity, improve accuracy, and accelerate time-to-market in hardware development cycles. However, despite the promising advancements, several challenges remain. The specialized nature of hardware design and verification demands precise and domain-specific capabilities from LLMs, which are not always easy to align with the general-purpose nature of current models. Additionally, issues like explainability, the handling of large-scale designs, the ability to generate verifiably correct hardware descriptions, and integration into existing workflows highlight the limitations of current LLM-based approaches. Moreover, concerns around model reliability, data privacy, and security in industrial settings require further attention before widespread adoption.

5.1. Summary of Findings

This paper discusses the transformative role of LLMs in hardware design and verification. It highlights several key areas where LLMs are improving efficiency and accuracy in hardware design by automating repetitive tasks, generating code, and facilitating better communication among engineers. The paper emphasizes the following contributions:

1) Core Applications:LLMs are applied to various tasks, including generating HDL code, optimizing design parameters, and automating verification processes like test case generation and bug detection. They can also enhance documentation and project management.

2) Challenges:Despite the advancements, challenges such as data scarcity, the need for specialized training, and integration with existing tools remain. The complexity of hardware design requires fine-tuning LLMs for specific tasks.

3) Future Directions:The paper suggests potential areas for future research, including improving LLM integration in hardware design workflows, refining LLM-generated outputs, and addressing open issues such as handling high-dimensional data and design complexity.

In summary, LLMs hold significant potential to transform hardware design and verification by automating complex tasks, enhancing productivity, and ensuring higher quality designs. However, there are still challenges related to training, data availability, and tool integration that need to be addressed to fully realize this potential.

5.2. Implications and Recommendations

There are several key areas where future research can focus to advance the application of LLMs in hardware design and verification:

1) Domain-Specific LLMs:Developing LLMs tailored to the specific needs of hardware design and verification could enhance their effectiveness. This includes models trained on HDL, circuit layouts, and specialized verification protocols.

2) Improving Verification Capabilities: Expanding the capacity of LLMs to automatically verify hardware designs through formal methods and simulation could reduce the burden of manual verification and lead to more robust, error-free hardware.

3)Hybrid Systems: Combining LLMs with other AI and traditional formal verification techniques could result in hybrid systems that leverage the strengths of both approaches, improving the accuracy and reliability of hardware designs.

4) Explainability and Interpretability: Ensuring that LLM-generated hardware descriptions are transparent and interpretable by engineers is critical. Future research could focus on developing methods to make the reasoning behind LLM outputs more understandable and trustworthy.

5) Real-World Applications: More real-world case studies are needed to evaluate the practical utility of LLMs in large-scale hardware projects. This will provide insights into the models’ performance in complex, industrial settings and help identify further areas of improvement.

6) Data Privacy and Security: Addressing concerns around the secure use of LLMs in proprietary hardware design environments, including techniques for ensuring that sensitive data remains protected during model training and deployment, will be crucial for industrial adoption.