Sort by
Identification of Important Nodes Based on Local Effective Distance Integration with Gravity Model
Sheng Zhang,
FuHao Liu,
YuYuan Huang,
ZiQiang Luo,
Ka Sun,
HongMei Mao
Posted: 12 March 2025
Breaking the Bottleneck Advances in Efficient Transformer Design
Yawen Bao
Transformers have become the backbone of numerous advancements in deep learning, excelling across domains such as natural language processing, computer vision, and scientific modeling. Despite their remarkable performance, the high computational and memory costs of the standard Transformer architecture pose significant challenges, particularly for long sequences and resource-constrained environments. In response, a wealth of research has been dedicated to improving the efficiency of Transformers, resulting in a diverse array of innovative techniques. This survey provides a comprehensive overview of these efficiency-driven advancements. We categorize existing approaches into four major areas: (1) approximating or sparsifying the self-attention mechanism, (2) reducing input or intermediate representation dimensions, (3) leveraging hierarchical and multiscale architectures, and (4) optimizing hardware utilization through parallelism and quantization. For each category, we discuss the underlying principles, representative methods, and the trade-offs involved. We also identify key challenges in the field, including balancing efficiency with performance, scaling to extremely long sequences, addressing hardware constraints, and mitigating the environmental impact of large-scale models. To guide future research, we highlight promising directions such as unified frameworks, dynamic and sparse architectures, energy-aware designs, and cross-domain adaptations. By synthesizing the latest advancements and providing insights into unresolved challenges, this survey aims to serve as a valuable resource for researchers and practitioners seeking to develop or apply efficient Transformer models. Ultimately, the pursuit of efficiency is crucial for ensuring that the transformative potential of Transformers can be realized in a sustainable, accessible, and impactful manner.
Transformers have become the backbone of numerous advancements in deep learning, excelling across domains such as natural language processing, computer vision, and scientific modeling. Despite their remarkable performance, the high computational and memory costs of the standard Transformer architecture pose significant challenges, particularly for long sequences and resource-constrained environments. In response, a wealth of research has been dedicated to improving the efficiency of Transformers, resulting in a diverse array of innovative techniques. This survey provides a comprehensive overview of these efficiency-driven advancements. We categorize existing approaches into four major areas: (1) approximating or sparsifying the self-attention mechanism, (2) reducing input or intermediate representation dimensions, (3) leveraging hierarchical and multiscale architectures, and (4) optimizing hardware utilization through parallelism and quantization. For each category, we discuss the underlying principles, representative methods, and the trade-offs involved. We also identify key challenges in the field, including balancing efficiency with performance, scaling to extremely long sequences, addressing hardware constraints, and mitigating the environmental impact of large-scale models. To guide future research, we highlight promising directions such as unified frameworks, dynamic and sparse architectures, energy-aware designs, and cross-domain adaptations. By synthesizing the latest advancements and providing insights into unresolved challenges, this survey aims to serve as a valuable resource for researchers and practitioners seeking to develop or apply efficient Transformer models. Ultimately, the pursuit of efficiency is crucial for ensuring that the transformative potential of Transformers can be realized in a sustainable, accessible, and impactful manner.
Posted: 28 February 2025
Assigning Candidate Tutors to Modules: A Preference Adjustment Matching Algorithm (PAMA)
Nikos Karousos,
Despoina Pantazi,
George Vorvilas,
Vassilios S. Verykios
Posted: 28 February 2025
SAT in Polynomial Time: A Proof of P = NP
Frank Vega
Posted: 20 February 2025
Forecasting Ethanol and Gasoline Consumption in Brazil: Advanced Temporal Models for Sustainable Energy Management
André Luiz Marques Serrano,
Patricia Helena Santos Martins,
Guilherme Fay Vergara,
Guilherme Dantas Bispo,
Gabriel Arquelau Pimenta Rodrigues,
Letícia Rezende Mosquéra,
Matheus Noschang de Oliveira,
Clovis Neumann,
Maria Gabriela Mendonca Peixoto,
Vinícius Pereira Gonçalves
Posted: 20 February 2025
Dynamic Evolution of EEG Complexity in Schizophrenia Across Cognitive Tasks
Rosa Molina,
Yasmina Crespo-Cobo,
Francisco J. Esteban,
Ana Victoria Arias,
Javier Rodríguez-Árbol,
Maria Felipa Soriano,
Antonio J. Ibañez-Molina,
Sergio Iglesias-Parro
Posted: 06 February 2025
Multi-Model Approach for Stock Price Prediction and Trading Recommendations
Zhenrui Chen,
Zhibo Dai,
Huiyan Xing,
Junyu Chen
Posted: 06 February 2025
Empirical Time Complexity for Assessing the Algorithm Computational Consumption on a Hardware
Yue Wu,
Carlo Vittorio Cannistraci
Posted: 29 January 2025
Computational Offloading and Time Allocation Policies in Mobile Edge Computing and Mobile Cloudlet
Isna Ahsan,
Mahmil Butt,
Zaiwar Ali,
Mohamad A. Alawad,
Abdulmajeed M. Alenezi,
Sheroz Khan,
And Muhammad Yahya
Posted: 13 January 2025
Optimization of Active Disturbance Rejection Control System for Vehicle Servo Platform Based on Artificial Intelligence Algorithm
Fei Yang,
Xiaopeng Su,
Xuemei Ren
The rapid growth of automotive intelligence and automation technology has made it difficult for traditional in vehicle servo systems to satisfy the demands of modern intelligent systems when facing complex problems such as external disturbances, nonlinearity, and parameter uncertainty. To improve the anti-interference ability and control accuracy of the system, this study proposes a joint control method of electronic mechanical braking control combined with anti-lock braking system. This method has developed a new type of actuator in the electronic mechanical brake control system, and introduced particle swarm optimization algorithm to optimize the parameters of the self disturbance rejection control system. At the same time, it combines adaptive inversion algorithm to optimize the anti-lock braking system. The results indicated that the speed variation of the developed actuator and the actual signal completely stopped at 1.9 seconds. During speed control and deceleration, the actuator could respond quickly and accurately to control commands as expected. On asphalt pavement, the maximum slip rate error of the optimized control method was 0.0428, while the original control method was 0.0492. The optimized method reduced the maximum error by about 12.9%. On icy and snowy roads, the maximum error of the optimization method was 0.0632, significantly lower than the original method's 0.1266. The optimization method could significantly reduce slip rate fluctuations under extreme road conditions. The proposed method can significantly improve the control performance of the vehicle mounted servo platform, reduce the sensitivity of the system to external disturbances, and has high practical value.
The rapid growth of automotive intelligence and automation technology has made it difficult for traditional in vehicle servo systems to satisfy the demands of modern intelligent systems when facing complex problems such as external disturbances, nonlinearity, and parameter uncertainty. To improve the anti-interference ability and control accuracy of the system, this study proposes a joint control method of electronic mechanical braking control combined with anti-lock braking system. This method has developed a new type of actuator in the electronic mechanical brake control system, and introduced particle swarm optimization algorithm to optimize the parameters of the self disturbance rejection control system. At the same time, it combines adaptive inversion algorithm to optimize the anti-lock braking system. The results indicated that the speed variation of the developed actuator and the actual signal completely stopped at 1.9 seconds. During speed control and deceleration, the actuator could respond quickly and accurately to control commands as expected. On asphalt pavement, the maximum slip rate error of the optimized control method was 0.0428, while the original control method was 0.0492. The optimized method reduced the maximum error by about 12.9%. On icy and snowy roads, the maximum error of the optimization method was 0.0632, significantly lower than the original method's 0.1266. The optimization method could significantly reduce slip rate fluctuations under extreme road conditions. The proposed method can significantly improve the control performance of the vehicle mounted servo platform, reduce the sensitivity of the system to external disturbances, and has high practical value.
Posted: 06 January 2025
A Comprehensive Survey of Cryptocurrency Forecasting: Methods, Trends, and Challenges
Mahmood Yousaf,
Muhammad Tariq,
Abdul Jabbar,
Syed Qaisar Jalil
Posted: 29 November 2024
Exploring Dominating Functions and Their Complexity in Subclasses of Weighted Chordal Graphs and Bipartite Graphs
Chuan-Min Lee
Posted: 27 November 2024
How to Parallelize “Non-Parallelizable” Minimization Functions
Dmitry Lukyanenko,
Sergei Torbin,
Valentin Shinkarev
Posted: 18 November 2024
Detecting Signatures of Criticality Using Divergence Rate
Tenzin Chan,
De Wen Soh,
Christopher Hillar
Posted: 11 November 2024
Overview of Clustering Techniques: From k-Means to Spectral Methods
Richard Murdoch Montgomery
Posted: 17 October 2024
Query Based Construction of Chronic Disease Datasets
Vuong M. Ngo,
Geetika Sood,
Patricia Kearney,
Fionnuala Donohue,
Dongyun Nie,
Mark Roantree
Posted: 16 October 2024
Travel Sales Problem: Evolutionary Algorithms and Complex Networks
Victor Andres Bucheli,
Mauricio Gaona,
Oswaldo Solarte Pabón
Posted: 11 October 2024
Enhancing Data Compression: Recent Innovations in LZ77 Algorithms
Aaron Hong,
Christina Boucher
Posted: 25 September 2024
Improvement of Electric Fish Optimization Algorithm for Standstill Label Combined with Levy Flight Strategy
Wangzhou Luo,
Hailong Wu,
Jiegang Peng
Posted: 16 September 2024
Solving NP-Complete Problems Efficiently
Frank Vega
Posted: 13 September 2024
of 8