Submitted:
27 January 2025
Posted:
28 January 2025
You are already at the latest version
Abstract
Keywords:
1. Introduction
- Development of an ultra low-power memristor-based unsupervised on-chip and online learning system for edge security applications.
- Resolution of circuit challenges for implementing online threshold computation to support real-time learning.
- Design and simulation of a Euclidean Distance computation circuit, integrated into the memristor-based neuromorphic system for unsupervised online training.
2. Network Dataset
3. Unsupervised Learning
4. Memristor Implementation
4.1. Memristor Neuron Circuit
4.2. Crossbar Training
5. Online Learning and Related Works
5.1. Online Learning Memristor Neuron Circuit
5.2. Related Works on Online Learning
5.3. Online Learning Systems
6. Memristor Based Online Learning Systems
7. Analog Threshold Computing
8. Simulation of Online Threshold Computing Circuit
9. Results and Discussion
9.1. Crossbar Traainnig Analysis
9.2. Pretrained AE
9.3. Online Training Analysis
9.4. System Energy, Power and Performance Analysis
10. Conclusions
Funding
References
- IBM. Edge computing acts on data at the source. Available online: https://www.ibm.com/cloud/what-is-edge-computing (accessed on 3 July 2023).
- Rai, A. Why real-time data monitoring is essential in preventing security threats. Available online: https://www.lepide.com/blog/why-real-time-data-monitoring-is-essential-in-preventing-security-threats/ (accessed on 29 September 2022).
- Bhavsar, M.; Roy, K.; Kelly, J.; Olusola, O. Anomaly based intrusion detection system for IoT application. Discover Internet of Things 2023, 3, 5. [Google Scholar] [CrossRef]
- Zarpelão, B. B.; Miani, R. S.; Kawakani, C.T.; de Alvarenga, S .C. A survey of intrusion detection in Internet of Things. Journal of Network and Computer Applications 2017, 84, 25–37. [Google Scholar] [CrossRef]
- Thudumu, S.; Branch, P.; Jin, J.; Singh, J. A comprehensive survey of anomaly detection techniques for high dimensional big data. Jornal of Big Data 2020, 7, 42. [Google Scholar] [CrossRef]
- Srinivas, T.; Aditya Sai, G.; Mahalaxmi, R. A comprehensive survey of techniques, applications, and challenges in deep learning: A revolution in machine learning. International Journal of Mechanical Engineering 2022, 7, 286–296. [Google Scholar]
- Naumov, M.; Kim, J.; Mudigere, D.; Sridharan, S.; Wang, X.; Zhao, W.; Yilmaz, S.; Kim, C.; Yuen, H.; Ozdal, M.; Nair, K. Deep learning training in facebook data centers: Design of scale-up and scale-out systems. arXiv 2020, arXiv:2003.09518. [Google Scholar]
- Lin, J.; Zhu, L.; Chen, W.M.; Wang, W.C.; Gan, C.; Han, S. On-device training under 256kb memory. Advances in Neural Information Processing Systems 2022, 35, 22941–22954. [Google Scholar]
- Luo, Y.; Peng, X.; Yu, S. MLP+ NeuroSimV3. 0: Improving on-chip learning performance with device to algorithm optimizations. In Proceedings of the International Conference on Neuromorphic Systems; 2019; pp. 1–7. [Google Scholar]
- Sebastian, A.; Le Gallo, M.; Khaddam-Aljameh, R., E. Eleftherious. Memory devices and applications for in-memory computing. Nat. Nanotechnol. 2020, 15, 529–544. [Google Scholar] [CrossRef]
- Rao, M.; Tang, H.; Wu, J.; Song, W.; Zhang, M.; Yin, W.; Zhuo, Y.; Kiani, F.; Chen, B.; Jiang, X.; et al. Memristor devices denoised to achieve thousands of conductance levels. NATURE 2023, 615, 823–829. [Google Scholar] [CrossRef] [PubMed]
- NSL-KDD dataset, https://www.unb.ca/cic/datasets/nsl.html.
- Takiddin, A.; Ismail, M.; Zafar, U.; Serpedin, E. Deep autoencoder-based anomaly detection of electricity theft cyberattacks in smart grids. IEEE Systems Journal 2022, 16, 4106–4117. [Google Scholar] [CrossRef]
- Sorrenson, P.; Draxler, F.; Rousselot, A.; Hummerich, S.; Zimmerman, L.; Köthe, U. Maximum Likelihood Training of Autoencoders. arXiv 2023, arXiv:2306.01843. [Google Scholar]
- Krestinskaya, O.; Salama, K.N.; James, A. P. Learning in memristive neural network architectures using analog backpropagation circuits. IEEE Transactions on Circuits and Systems I: Regular Papers 2018, 66, 719–732. [Google Scholar] [CrossRef]
- Tsai, S.; Ambrogio, S.; Narayanan, P.; Shelby, B.; Mackin, C.; Burr, G. Analog memory-based techniques for accelerating the training of fully-connected deep neural networks. In SPIE Advanced Lithography; 2019. [Google Scholar]
- Hayes, T.L.; Kanan, C. Lifelong machine learning with deep streaming linear discriminant analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops; 2020; pp. 220–221. [Google Scholar]
- Shao, C.; Feng, Y. Overcoming catastrophic forgetting beyond continual learning: Balanced training for neural machine translation. arXiv 2022, arXiv:2203.03910. [Google Scholar]
- Chaudhri, Z.; Liu, B. Lifelong Machine Learning; Springer International Publishing: Switzerland, 2022. [Google Scholar]
- Rebuffi, S.-A.; Kolesnikov, A.; Sperl, G.; Lampert, C.H. iCaRL: Incremental Classifier and Representation Learning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017; pp. 5533–5542. [Google Scholar]
- Ashfahani, A.; Pratama, M.; Lughofer, E.; Ong, Y. S. DEVDAN: Deep evolving denoising autoencoder. Neurocomputing 2020, 390, 297–314. [Google Scholar] [CrossRef]
- Sarwar, S.S.; Ankit, A.; Roy, K. Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing. IEEE Access 2020, 8, 4615–4628. [Google Scholar] [CrossRef]
- Malawade, A.V.; Costa, N.D.; Muthirayan, D.; Khargonekar, P.P.; Al Faruque, M.A. Neuroscience-Inspired Algorithms for the Predictive Maintenance of Manufacturing Systems. IEEE Transactions on Industrial Informatics 2021, 17, 7980–7990. [Google Scholar] [CrossRef]
- Monakhov, V.; Thambawita, V.; Halvorsen, P.; Riegler, M.A. GridHTM: Grid-Based Hierarchical Temporal Memory for Anomaly Detection in Videos. Sensors 2023, 23, 2087. [Google Scholar] [CrossRef]
- Faezi, S.; Yasaei, R.; Barua, A.; Al Faruque, M.A. Brain-inspired golden chip free hardware trojan detection. IEEE Transactions on Information Forensics and Security 2021, 16, 2697–2708. [Google Scholar] [CrossRef]
- Wang, J.; Lu, S.; Wang, S.H.; Zhang, Y.D. A review on extreme learning machine. Multimedia Tools and Applications 2022, 81, 41611–41660. [Google Scholar] [CrossRef]
- Zhang, J.; Li, Y.; Xiao, W. Adaptive online sequential extreme learning machine for dynamic modeling. Soft Computing 2021, 25, 2177–2189. [Google Scholar] [CrossRef]
- Wang, X.; Tu, S.; Zhao, W.; Shi, C. A novel energy-based online sequential extreme learning machine to detect anomalies over real-time data streams. Neural Computing and Applications 2022, 34, 823–831. [Google Scholar] [CrossRef]
- Albadr, M.A.A.; Tiun, S.; Ayob, M.; Al-Dhief, F.T.; Abdali, T.-A.N.; Abbas, A.F. Extreme Learning Machine for Automatic Language Identification Utilizing Emotion Speech Data. 2021 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), Kuala Lumpur, Malaysia, 2021; pp. 1–6. [Google Scholar]
- Alam, M.S.; Yakopcic, C.; Subramanyam, G.; Taha, T.M. Memristor based neuromorphic adaptive resonance theory for one-shot online learning and network intrusion detection. International Conference on Neuromorphic Systems, 2020 (ICONS 2020). Article No. 25. pp.1-8. July 2020.
- Jones, C.B.; Carter, C.; Thomas, Z. Intrusion Detection & Response using an Unsupervised Artificial Neural Network on a Single Board Computer for Building Control Resilience. 2018 Resilience Week (RWS), Denver, CO, USA, 2018; pp. 31–37. [Google Scholar]
- Kumar, A.; Sachdeva, N. Cyberbullying checker: Online bully content detection using Hybrid Supervised Learning. In International Conference on Intelligent Computing and Smart Communication: Proceedings of ICSC 2019; Springer Singapore, 2020; pp. 371–382. [Google Scholar]
- Dong, Z.; Lai, C.S.; Zhang, Z.; Qi, D.; Gao, M.; Duan, S. Neuromorphic extreme learning machines with bimodal memristive synapses. Neurocomputing 2021, 453, 38–49. [Google Scholar] [CrossRef]
- Liang, H.; Cheng, H.; Wei, J.; Zhang, L.; Yang, L.; Zhao, Y.; Guo, H. Memristive neural networks: A neuromorphic paradigm for extreme learning machine. IEEE Transactions on Emerging Topics in Computational Intelligence 2019, 3, 15–23. [Google Scholar] [CrossRef]
- Bureneva, O.I.; Prasad, M.S.; Verma, S. FPGA-based Hardware Implementation of the ART-1 classifier. In 2023 XXVI International Conference on Soft Computing and Measurements (SCM); Saint Petersburg, Russian Federation, 2023; pp. 171–174. [Google Scholar]
- Govorkova, E.; Puljak, E.; Aarrestad, T.; James, T.; Loncar, V.; Pierini, M.; Pol, A.A.; Ghielmetti, N.; Graczyk, M.; Summers, S.; Ngadiuba, J. Autoencoders on field-programmable gate arrays for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider. Nature Machine Intelligence 2022, 4, 154–161. [Google Scholar] [CrossRef]
- Alam, M.S.; Fernando, B.R.; Jaoudi, Y.; Yakopcic, C.; Hasan, R.; Taha, T.M.; Subramanyam, G. Memristor based autoencoder for unsupervised real-time network intrusion and anomaly detection. In Proceedings of the International Conference on Neuromorphic Systems, July 2019; pp. 1–8. [Google Scholar]
- Alam, M.S.; Yakopcic, C.; Taha, T.M. University of Dayton. Unsupervised learning of memristor crossbar neuromorphic processing systyems. U.S. Patent Application 17/384,306, 2022. [Google Scholar]
- Nag, A.; Paily, R.P. Low power squaring and square root circuits using subthreshold MOS transistors. 2009 International Conference on Emerging Trends in Electronic and Photonic Devices & Systems, Varanasi, India, 2009; pp. 96–99. [Google Scholar]
- Jiang, H.; Li, W.; Huang, S.; Cosemans, S.; Catthoor, F.; Yu, S. Analog-to-Digital Converter Design Exploration for Compute-in-Memory Accelerators. IEEE Design & Test 2022, 39, 48–55. [Google Scholar]
- ASUS. Tinkerboard, https://tinker-board.asus.com/product/tinker-board.html.
- Suleiman, M.F.; Issac, B. Performance Comparison of Intrusion Detection Machine Learning Classifiers on Benchmark and New Datasets. 2018 28th International Conference on Computer Theory and Applications (ICCTA), Alexandria, Egypt, 2018; pp. 19–23. [Google Scholar]
- Karthikeyan, D.; Mohanraj, V.; Suresh, Y.; Senthilkumar, J. Retracted: An efficient stacking model with SRPF classifier technique for intrusion detection system. International Journal of Communication Systems 2021, 34, e4737. [Google Scholar] [CrossRef]
- Hamolia, V.; Melnyk, V.; Zhezhnych, P.; Shilinh, A. Intrusion detection in computer networks using latent space representation and machine learning. International Journal of Computing 2020, 19, 442–448. [Google Scholar] [CrossRef]
- Yakopcic, C.; Taha, T.M.; Subramanyam, G.; Pino, R.E. Memristor SPICE model and crossbar simulation based on devices with nanosecond switching time. The 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, 2013; pp. 1–7. [Google Scholar]
- Hasan, R.; Taha, T.M.; Yakopcic, C. On-chip training of memristor based deep neural networks. 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 2017; pp. 3527–3534. [Google Scholar]
















| Parameter | Magnitude | Unit |
|---|---|---|
| R | 1000 | Ohm |
| R1 | 1.50×106 | Ohm |
| M1 | 1.99/0.65 | W/L |
| M2 | 1.99/0.65 | W/L |
| M3 | 0.2/0.16 | W/L |
| M4 | 0.24/0.18 | W/L |
| M5 | 1.28/0.65 | W/L |
| M6 | 1.85/0.65 | W/L |
| M7 | 1.85/0.65 | W/L |
| Vbias | 0.4 | volt |
| Ibias | 210 | nA |
| RH | 10000 | Ohm |
| Parameter | Magnitude | Unit |
|---|---|---|
| Cycle time | 2×10-9 | sec |
| Max training pulse duration | 5×10-9 | sec |
| Roff | 10 | MΩ |
| Ron | 50 | KΩ |
| Ravg | 5.025 | MΩ |
| Wire Resistance | 5 | Ω |
| Vmem | 1.3 | V |
| Pmem | 0.336 | µW |
| Op-Amp power | 3µ | W |
| Max read voltage | 1.3 | V |
| Feature size, F | 45 | nm |
| Transistor size | 50 | F2 |
| Memristor Area | 10000 | nm2 |
| Cycle time | 2 | ns |
| Crossbar processing time | 50 | ns |
| Parameters | TinkerBoard | Memristor System |
|---|---|---|
| Test Sample | 2000 | 2000 |
| Time (sec) | 1.807563 | 2.3×10-3 |
| Time/Sample | 9.04×10-4 | 1.17×10-6 |
| Speedup | 1 | 774 |
| Energy (joule) | 4.52×10-3 | 2.08×10-7 |
| Power (W) | 5 | 0.0205 |
| Performance (OPS) | 20×106 | 16.1×109 |
| Energy Efficiency (OPS/W) | 4.12×106 | 7.83×1011 |
| Area (mm2) | --- | 4.43×10-3 |
| Test Sample | 2000 | 2000 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).