Thermal Analysis and Evaluation of Memristor-based Compute-in-Memory Chips

1. Introduction

AI technologies, for example autonomous driving, large language models (LLMs), and cloud computing, have garnered significant attention and widespread application in recent years [1,2,3]. However, the rapidly increasing demand for computational hardware has become a critical challenge [4]. The memory wall has emerged as a bottleneck for improving the performance of integrated circuits [5]. Significant energy and latency are consumed during frequent data transfers between physically separated memory and computing units in the conventional Von Neumann architecture [6,7]. CIM technology based on emerging non-volatile memory （eNVM） device, such as RRAM, can perform both storage and computation functions within a single device [8,9,10]. This technology demonstrates significant potential for overcoming the bottleneck of the memory wall, and has become a promising choice for AI applications including image classification, human-machine interface, and image-reconstruction [11,12,13].

RRAM-based CIM chips have developed rapidly in recent years. However, power density of CIM chips has an increasing trend and addressing thermal issues has become increasingly urgent [14]. Recent studies indicate that RRAM-based CIM chips are capable of performing matrix-vector multiplication [15,16,17,18]. During the computing process, the parallel operation of RRAM devices results in high local power consumption and high local temperatures [19]. Furthermore, Both the characteristics of RRAM devices and overall chip performance are highly temperature-sensitive [20]. For example, the conductance of analog RRAM device drifts with temperature. In particular, when analog devices are used to represent multiple bits, the conductance drift leads to significant degradation in computing accuracy of the chip. Without effective thermal management methods, the performance of CIM chips will decline significantly [21]. Some existing works primarily focus on the RRAM device characteristic, for example compact model of the temperature coefficient , retention characteristics [20,22]. However, these models fail to account for the impact of temperature distribution and neural network from a system-level perspective. Chip-level thermal simulations often treat the entire chip as a uniform heat source, disregarding the detailed circuit structure [23]. Some works have explored multi-scale thermal simulations [19], but comprehensive analyses of temperature distribution and thermal effects in CIM chips remain incomplete, lacking detailed analysis of packaging process.

In this work, a multi-scale thermal model is developed to evaluate the temperature distribution in RRAM-based CIM chips. This model is further employed to analyze the influence of boundary thermal resistance, the layout of digital circuits and analog computing module tiles, the thickness of the EMC, the thickness of the substrate. Furthermore, the impact of temperature on RRAM device conductance and the inference accuracy of RRAM-based CIM chips is investigated. Simulation results reveal that the computing accuracy loss of CIM chips due to rising temperatures is unacceptable and the computation correction method can reduce the accuracy loss from 66.4% to 1.4% at 85 ℃.

2. Thermal modeling

The thermal modeling and analysis framework for RRAM-based CIM chips is shown in Figure 1. First, a multi-scale thermal model is constructed for CIM chips with parameters extracted from a real 28nm CIM chip, and device characteristics as well as packaging factors are also considered. Second, based on Fourier's law and the finite element method (FEM), the equations are formulated and solved under steady-state conditions in this work. Third, we evaluate the temperature distribution of the CIM chips and the impact of various factors on temperature, including process, circuit architecture, and cooling methods. Fourth, the thermal effects on RRAM devices and CIM chips will be evaluated. The conductance range of RRAM devices in this work is from 2 to 20 μS. Within this range, the conductance of the RRAM devices exhibits semiconductor behavior with a positive temperature coefficient. A compact model of RRAM conductance is used to evaluate the impact of temperature on the conductance of the RRAM devices. Next, a typical deep neural network, the 18-layer ResNet neural network (ResNet-18) [24] is used to classify images from the CIFAR-10 [25] dataset to evaluate the thermal effect on computing accuracy of RRAM-based CIM chips. Finally, based on the simulation results, several design guidelines for optimizing the temperature distribution and thermal-induced performance degradation are provided.

The architecture of the RRAM-based CIM chip from RRAM device to the full CIM chip used in the modeling is shown in Figure 2. RRAM device can change its state through ion movement and process analog signals. The RRAM used is the TiN/HfOx/TaOy/TiN memristor in one-transistor-one-resistor (1T1R) cell structure. Based on Ohm's law and Kirchhoff's law, the RRAM array performs matrix-vector multiplication. One RRAM array and the corresponding peripheral circuits, such as analog-digital converters (ADCs) and digital-analog converters (DACs), form one processing element (PE). Each tile consists of 4 PEs and the corresponding peripheral circuits, such as the tile controller, shift and adder, activation function, and pooling modules. Nine tiles and their corresponding peripheral circuits constitute the RRAM-based CIM chip. The convolutional parameters of ResNet-18 are mapped onto the RRAM arrays, while certain specialized computations, such as pooling, are handled by the digital circuit section.

The parameters used in this work are listed in Table 1. The conductance range of RRAM is between 2 and 20 μS, and each RRAM device has 16 independent conductance states.

The flip-chip packaging method is adopted for the CIM chip in this work. Joule heat is primarily released into the environment from the upper surface. The heat dissipation from the four sides and the bottom side is minimal, with their thermal resistances approximated as adiabatic. The ambient temperature is assumed to be 26.85 °C.

3. Results of thermal effect evaluation

Based on the multi-scale thermal model, the temperature distribution of the RRAM-based CIM chip is obtained. The temperature map of the CIM chip is shown in Figure 3a, when power of each RRAM array is 40mW with the upper thermal resistance of 7 cm²·K/W. As illustrated in the figure, there are hotspots in the PE areas with higher power density, while the temperature in the digital circuit areas remains relatively low. The temperature difference across the chip reaches approximately 10 ℃. The impact of boundary thermal resistance on the temperature distribution of CIM chips is evaluated. As Figure 3b shows, with the increase of boundary thermal resistance, both the average temperature and maximum temperature of the CIM chip increase and the temperature difference also increases significantly. Furthermore, Furthermore, the difference between the maximum and average temperatures grows as the boundary thermal resistance increases. Therefore, reducing boundary thermal resistance can effectively lower the chip's hotspot temperature.. If the temperature limit is set to 85 °C, the boundary thermal resistance should not exceed 6.8 cm²K/W to ensure the safe operation of the chip.

The thermal conductivity of the EMC material is low, so the thickness of the EMC layer significantly impacts the maximum temperature of the chip. As shown in Figure 4a, the maximum temperature differs by 6.8 °C between a 50 μm thick EMC cap and no EMC layer. The substrate material of the chip is monocrystalline silicon, which has a high thermal conductivity of 148 W/m/K. Figure 4b evaluates the impact of substrate thickness on both the maximum and average temperatures in the chip. As shown in the figure, when the substrate thickness is less than 300 μm, the hotspot temperature decreases rapidly as the substrate thickness increases. However, when the substrate thickness exceeds 300 μm, the temperature reduction effect becomes less significant as the thickness increases further. As the substrate thickness increases, the average temperature of the chip shows a slow increasing trend.

To reduce the hotspot temperature, chip design optimization can be applied. As shown in Figure 3a, the temperatures in the digital circuit area are lower than that in the tile area with high power. By adjusting the position of the tiles and placing the digital circuits around the tiles, the distribution of temperature can be improved. The optimized result, shown in Figure 5a, indicates a more uniform temperature distribution. The temperature difference between the maximum and minimum temperatures on the chip is only about 2°C. Compared with the original configuration shown in Figure 5b, the maximum temperature has decreased by 4.04 °C after optimization.

Based on the above evaluations, hotspot temperatures on the chip can be reduced by optimizing the layout of tiles and digital circuits, decreasing the EMC thickness, appropriately increasing the substrate thickness, and reducing boundary thermal resistance. Additionally, reducing the power consumption of various on-chip modules can further help lower the temperature.

The drift of RRAM conductance with temperature is a basic characteristic of semiconductor materials. The conductance of RRAM device drifts when temperatures increase. The 4-bit analog RRAM devices are used in the simulation, and the conductance difference between two adjacent states is only about 1 μS. As shown in Figure 6a, the conductance of RRAM device drift with temperature based on the compact model of RRAM conductance [19]. The conductance drift significantly exceeds the difference between two states, resulting in data drift of the CIM. As a result, the mapped weight of DNN on CIM chips will drift when temperature increases, which leads to accuracy loss of the CIM chip. The computing accuracy loss of the CIM chip is shown in Figure 6a, where the accuracy loss is found to be significant and unacceptable. And as the temperature increases, the accuracy decreases more. Optimizing the temperature coefficient of RRAM materials can reduce conductivity drift and computing accuracy loss of CIM chips. For the current RRAM devices, we explore optimization from the circuit design level. The temperatures within a single RRAM array are almost uniform, leading to a similar drift ration of RRAM conductance. The ADC quantization process must be scaled to correct the conductance change with temperature. According to the simulation results in Figure 6b, ADC correction can decrease the accuracy loss from 66.4% to 1.4% at 85 ℃.

4. Conclusions

A multi-scale thermal simulation model for RRAM-based CIM chips is developed in this paper. Then the temperature distribution characteristics of CIM chips and the impact of boundary thermal resistance are evaluated. 4.04 ℃ has been reduced by optimizing the layout of digital circuits and tiles. Finally, the effect of temperature on RRAM conductance and CIM computing accuracy is analyzed, and optimized solutions are proposed. Based on the evaluation results, the ADC correction method is shown to reduce the accuracy loss from 66.4% to 1.4% at 85 ℃.