We introduce a novel hybrid Transformer-CNN architecture for robotic grasp detection, designed to enhance the accuracy of grasping unknown objects. Our proposed architecture has two key designs. Firstly, we develop a hierarchical transformer as the encoder, incorporating the external attention to effectively capture the correlation features across the data. Secondly, the decoder is constructed with cross-layer connections to efficiently fuse multi-scale features. Channel attention is introduced in the decoder to model the correlation between channels and to adaptively recalibrate the channel correlation feature response, thereby increasing the weight of the effective channels. Our method is evaluated on the Cornell and Jacquard public datasets, achieving an image-wise detection accuracy of 98.3% and 95.8% on each dataset, respectively. Additionally, we achieve object-wise detection accuracy of 96.9% and 92.4% on the same datasets. A physical experiment is also performed using the Elite 6Dof robot, with a grasping accuracy rate of 93.3%, demonstrating the proposed method's ability to grasp unknown objects in real-world scenarios. The results of this study show that our proposed method outperforms other state-of-the-art methods.

Previous camera self-calibration methods have exhibited certain notable shortcomings. On one hand, they either exclusively emphasized scene cues or solely focused on vehicle-related cues, resulting in a lack of adaptability to diverse scenarios and a limited number of effective features. Furthermore, these methods either solely utilized geometric features within traffic scenes or exclusively extracted semantic information, failing to comprehensively consider both aspects. This limited the comprehensive feature extraction from scenes, ultimately leading to a decrease in calibration accuracy. Additionally, conventional vanishing point-based self-calibration methods often required the design of additional edge-background models and manual parameter tuning, thereby increasing operational complexity and the potential for errors. Given these observed limitations, and in order to address these challenges, we propose an innovative roadside camera self-calibration model based on the Transformer architecture. This model possesses a unique capability to simultaneously learn scene features and vehicle features within traffic scenarios while considering both geometric and semantic information. Through this approach, our model can overcome the constraints of prior methods, enhancing calibration accuracy and robustness while reducing operational complexity and the potential for errors. Our method outperforms existing approaches on both real-world dataset scenarios and publicly available datasets, demonstrating the effectiveness of our approach.

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time-consuming due to the numerous images generated per case, and lesion detection accuracy may rely on the operators' skills and experiences. Hence, many researchers have recently developed deep learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts' frame selection skills, we suggest using whole video frames as the input to the deep-learning system. Thus, we propose a new Transformer-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% sensitivity. These results may significantly advance automated lesion detection techniques for WCE images. Our code is available at https://github.com/syupoh/VWCE-Net.git.

Detection of partial discharge (PD) activities in high-voltage equipment can be done according to the several mechanisms of signal detection, which are electromagnetic wave signal detection, acoustic signal detection, chemical reactions, electrical signal detection, or optical emission detection. Recently, multiple methods of detection and localization of partial discharge activities occurred in power transformers and gas-insulated switchgear (GIS) have been proposed to monitor the health condition of high-voltage equipment, especially when the awareness regarding preventive maintenance has been emphasized at the industrial level and electrical providers. In aligning the needs of industrial and the improvement of PD detection methods, this manuscript focused on reviewing the current practice methods for the detection and localization of PD signals in high-voltage equipment and comparing their efficacy, in summarizing the future direction of research work-related methods of PD detection. The comparative reviews are discussed in terms of the mechanism of PD signal detection, indication parameters, calibration technique, advantages, and limitations of each method of PD measurement in detail.

In practical applications, the detection of objects with various sizes is a common requirement for most detectors. The feature pyramid network (FPN) is widely adopted as a framework to address this challenge. The field is witnessing an increasing number of transformer-based target detectors due to the widespread adoption of transformer technology. This paper initially examines the design flaws in FPN and transformer-based target detectors, followed by the introduction of a new transformer-based approach called Texturized Instance Guidance (TIG-DETR) to address these issues. Specifically, TIG-DETR comprises a backbone network, a new pyramidal structure known as Texture-Enhanced FPN (TE-FPN), and an enhanced DETR detector.The TE-FPN is composed of three components: a bottom-up pathway for enhancing texture information in the feature map, a lightweight attention module to address confounding effects resulting from cross-scale fusion, and a standard attention module to enhance the final output features.The improved DETR detector utilizes Shifted Window based Self-Attention to replace the multi-headed self-attention module in DETR, thereby accelerating model convergence. Moreover, it incorporates an Instance Based Advanced Guidance Module to enhance instance perception in the image by employing a pre-local self-attentive mechanism for recognizing larger instances. By employing TE-FPN instead of FPN in Faster RCNN with Resnet-50 as the backbone network, we achieve a 1.9% improvement in average accuracy. TIG-DETR achieves an average accuracy of 44.1 with Resnet-50 as the backbone network.

Existing methods for 3D human pose estimation mainly divide the task into two stages. The first stage identifies the 2D coordinates of the human joints in the input image, namely the 2D human joint coordinates. The second stage uses the results from the first stage as input to recover the depth information of human joints from the 2D human joint coordinates to achieve 3D human pose estimation. However, the recognition accuracy of the two-stage method relies heavily on the results of the first stage and includes too many redundant processing steps, which reduces the inference efficiency of the network. To address these issues, we propose the EDD, a fully End-to-end 3D human pose estimation method based on transformer architecture with Dual Decoders. By learning multiple human poses, the model can directly infer all 3D human poses in the image using a pose decoder, and then further optimize the recognition result using a joint decoder based on the kinematic relations between joints. With the attention mechanism, this method can adaptively focus on the most relevant features to the target joint, effectively overcoming the feature misalignment problem in the human pose estimation task and greatly improving the model performance. Any complex post-processing step, such as non-maximum suppression, is eliminated, further improving the efficiency of the model. The results show that the method achieves an accuracy of 87.4% on the MuPoTS-3D dataset, significantly improving the accuracy of end-to-end 3D human pose estimation methods.

In power systems there are complex transformer structures, whose accurate analysis is not possible using the techniques available today. This paper presents a systematic data driven analysis method for coupled inductors of arbitrary complexity. The method first establishes a winding matrix N mapping the windings to the limbs of the transformer. A permeance matrix P is created from the reluctance network of the magnetic core. A generalized inductance matrix L mapping currents in the transformer windings to the induced voltages is generated based on the winding (N) and permeance (P) matrices. The inductance matrix representation of a coupled inductor is then transformed to an admittance matrix, which can be integrated to the nodal analysis of the electrical circuit surrounding the coupled inductor. The method presented is validated by simulations with real transformer structures using electromagnetic transient program (EMTP/ATP).

Large power transformers are generally associated with a maximum capacity rating of 100 MVA or higher. These large liquid dielectric power transformers are a custom-built piece of equipment, thus very expensive, and a backbone element of the power grid. In extreme cases as, for example, severe geomagnetic disturbances, permanently monitoring their condition will enhance their electrical reliability and resilience to guarantee efficient management of its life cycle. However, some traditional monitoring/diagnosis techniques have singular features when applied to large power transformers and their interlinked subsystems. In this context, and since that information is hardly put in evidence and compiled in the literature, this paper reviews the particularities of monitoring and diagnosing those assets.

The Chat Generative Pre-training Transformer (GPT), also known as ChatGPT, is a powerful generative AI model that can simulate human-like dialogues across a variety of domains. However, this popularity has attracted the attention of malicious actors who exploit ChatGPT to launch cyberattacks. This paper examines the tactics that adversaries use to leverage ChatGPT in a variety of cyberattacks. Attackers pose as regular users and manipulate ChatGPT’s vulnerability to malicious interactions, particularly in the context of cyber assault. The paper presents illustrative examples of cyberattacks that are possible with ChatGPT and discusses the realm of ChatGPT-fueled cybersecurity threats. The paper also investigates the extent of user awareness of the relationship between ChatGPT and cyberattacks. A survey of 253 participants was conducted, and their responses were measured on a three-point Likert scale. The results provide a comprehensive understanding of how ChatGPT can be used to improve business processes and identify areas for improvement. Over 80% of the participants agreed that cyber criminals use ChatGPT for malicious purposes. This finding underscores the importance of improving the security of this novel model. Organizations must take steps to protect their computational infrastructure. This analysis also highlights opportunities for streamlining processes, improving service quality, and increasing efficiency. Finally, the paper provides recommendations for using ChatGPT in a secure manner, outlining ways to mitigate potential cyberattacks and strengthen defenses against adversaries.

Structural magnetic resonance imaging (sMRI) is widely used in the clinical diagnosis of diseases due to its advantages: high definition and noninvasive. Therefore, computer-aided diagnosis based on sMRI images is broadly applied in classifying Alzheimer’s disease (AD). Due to the excellent performance of Transformer in computer vision, Vision Transformer (ViT) has been employed for AD classification in recent years. ViT relies on access to large datasets, while the sample size of brain imaging datasets is relatively insufficient. Moreover, the pre-processing procedures of brain sMRI images are complex and labor-intensive. To overcome the limitations mentioned above, we propose Resizer Swin Transformer (RST), a deep learning model that can extract information from brain sMRI images that are only briefly processed to achieve multi-scale and cross-channel features. In addition, we pre-trained our RST on a natural image dataset and obtained better performance. The experimental results of ADNI and AIBL datasets prove that RST can achieve better classification performance in AD prediction compared with CNN-based and Transformer models.

Buildings are responsible for approximately 40% of the world’s energy consumption and 36% of the total carbon dioxide emissions. Building occupancy is essential, enabling Occupant-Centric Control for zero emissions and decarbonization. Although existing machine learning and deep learning methods for building occupancy prediction have achieved remarked progress, their analyses remain limited when applied to complex real-world scenarios. Besides, there is a high expectation for Transformer algorithms to predict building occupancy accurately. Therefore, this paper presents an Occupancy Prediction Transformer network (OPTnet). We fuse and feed multi-sensor data (building occupancy, indoor environmental conditions, HVAC operations) into a Transformer model to forecast the future occupancy presence in multiple zones. We perform experimental analysis and compare it to different occupancy prediction methods (e.g., Decision Tree, Long Short-Term Memory networks, Multi-layer perceptron) and diverse time horizons (1,2,3,5,10,20,30mins). The performance metrics (e.g., Accuracy and Mean Squared Error) are employed to evaluate the effectiveness of prediction algorithms. Our OPTnet method achieves superior performance on our experimental two-week data compared to existing methods. The improved performance signifies its potential to enhance HVAC control systems and energy optimization strategies. We will make the code publicly available at https://github.com/kailaisun/occupancy-prediction-binary to promote transparency and reproducibility.

Communication is a key method of expressing one's thoughts and opinions. Amongst many modalities, speech and writing are the most powerful and common forms of human communication. Analysing what and how people think has inherently been an interesting and progressive research domain. This includes bimodal sentiment analysis which is an emerging area in natural language processing (NLP) and has received a great deal of attention in recent years in a variety of areas including social opinion mining, health care, banking, and so on. At present, there are limited studies on bimodal conversational sentiment analysis as it proves to be a challenging area given the complex nature of the way humans express sentiment cues across various modalities. To address this gap, a comparison of the performance of multiple data modality models has been conducted on the MELD dataset, a widely-used dataset for benchmarking sentiment analysis within the research community. Our work then demonstrates the results of combining acoustic and linguistic representations. Lastly, our proposed neural network-based ensemble learning technique is employed over six transformer and deep learning-based models, achieving a State-Of-The-Art (SOTA) accuracy.

Deep learning technologies for skin cancer detection have been dramatically advanced based on high resolution dermoscopic images. The low-cost approach based on clinical skin images of low resolution is promising but remains technically challenging due to their undermined image quality. In this paper, we propose a coarse-to-fine efficient super resolution transformer (CF-ESRT) network to reconstruct the dermoscopy-level high resolution skin image from a low resolution clinical image. By connecting the refinement network to the original super resolution transformers and applying perceptual and gradient losses, our framework noticeably improves the finer texture details of skin lesions in the super resolution (SR) images, and is effective to elevate the perceptual quality of the SR images. Quantitative and qualitative evaluations show that our method outperforms ESRT the basis model as well as the other state-of-the-art SR models.

The prediction of drug-target interactions plays a fundamental role in facilitating drug discovery, where the goal is to find prospective drug candidates. With the increase in the number of drug-protein interactions, machine learning techniques, especially deep learning methods, have become applicable for drug-target interaction discovery because they significantly reduce the required experimental workload. In this paper, we present a spontaneous formulation of the drug-target interaction prediction problem as an instance of multi-instance learning. We address the problem in three stages, first organizing given drug and target sequences into instances via a private-public mechanism, then identifying the predicted scores of all instances in the same bag, and finally combining all the predicted scores as the output prediction. A comprehensive evaluation demonstrates that the proposed method outperforms other state-of-the-art methods on three benchmark datasets.

Heat transfer fluids are used in various industrial systems to maintain them in perfect operating conditions. Extensive research efforts have been dedicated to enhance the thermal properties of these heat transfer fluids to improve their efficiency. Developing nanofluids is a potential candidate for such enhancements. This study investigates the impact of incorporating TiO2 nanoparticles into two types of oils: transformer oil (NYTRO LIBRA) and virgin coconut oil (manufactured by Govi Aruna Pvt. Ltd.) at different temperatures and with varying volume fractions. The nanofluids were prepared using a two-step method by adding CTAB (Cetyltrimethylammonium bromide) surfactant. To minimize nanoparticle agglomeration, this study employed relatively low-volume fractions. Thermal properties by means of thermal conductivity, thermal diffusivity, and volumetric heat capacity were measured in accordance with ASTM (American Society for Testing and Materials) standard methods, using a multifunctional thermal conductivity meter (LAMBDA thermal conductivity meter). The measured thermal conductivity values were compared with theoretical models and previous research findings. It was confirmed that the modification of thermal properties was enhanced by doping TiO2 nanoparticles with different volume fraction.

The research results presented in the article were carried out during the realization of the project, the aim of which is to develop a method of drying cellulose insulation in power transformers with the use of synthetic ester. This method uses a very high water absorption of the ester. During the drying of transformers, the ester is systematically contaminated with mineral oil, which gradually loses its ability to absorb water. Information on the oil concentration in the mixture is needed in two cases: at the stage of making a decision on the treatment of the mixture and during its treatment. The article presents the results of investigations of two methods: 1) based on the measurement of the mixture density, and 2) based on the measurement of the capacitance of the capacitor immersed in the mixture. The conducted research shows that the method of measuring the density of the mixture gives an uncertainty of 2.6 p. %, while the method of measuring the capacitance of a capacitor gives an uncertainty of 2.2 p. %. A significant advantage of the method of measuring the capacitance is the possibility of using it online to control the ester treatment process.

Plasma generation by means of electrical discharge requires specialized power supply systems. The applicability of plasma for various plasma processes depends on its parameters, and these in turn depend on the parameters of the power supply systems. Arc plasma can be unstable, generating a lot of electromagnetic interference and overvoltage and overcurrent. The power system of a plasma reactor must guarantee good plasma control characteristics, be immune to disturbances and ensure good cooperation with the power grid. The article analyzes the cooperation of a three-phase plasma reactor, with gliding arc discharge, with a power supply system of a new type. This system integrates an AC/DC/AC converter with a five-column transformer of special design in a single device. Using the properties of magnetic circuits, it was possible to integrate the functions of ignition and sustaining the burning of the discharge in the reactor in a single transformer. Proper design of the transformer is crucial to achieve good cooperation of the AC/DC/AC converter with both the plasma reactor and the power supply network. The presented power supply design shows a number of positive features predisposing it to powering arc plasma reactors.

Identification of plants is a challenging task which aims to identify the family, genus, and species level according to morphological features. Automated deep learning-based computer vision algorithms are widely used for identifying plants and can help users to narrow down the possibilities. However, numerous morphological similarities between and within species make the classification difficult. In this paper, we tested a custom convolution neural network (CNN) and vision transformer (ViT) based models using the PyTorch framework to classify plants. We used a large dataset of 88K and 16K images for classifying plants at genus and species levels respectively. Our results show that for classifying plants at the genus level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420, and other state-of-the-art CNN-based models suggested in previous studies on a similar dataset. The ViT model achieved top accuracy of 83.3% for classifying plants at the genus level. ViT models also perform better for classifying plants at the species level compared to CNN-based models ResNet50 and ResNet-RS-420, with a top accuracy of 92.5%. We show that the correct set of augmentation techniques plays an important role in classification success.

Due to the influence of background conditions, lighting conditions, occlusion issues and the image resolution, how to extract robust person features is one of the difficulties in ReID research. Vision in Transformers (ViT) has achieved significant results in the field of computer vision. However, the existing problems still limit its application in ReID due to slow extraction of person features and difficulty in utilizing local features of people. To solve the mentioned problems, we utilize Pyramid Vision Transformer (PVT) as the backbone of feature extraction and propose a PVT-based ReID method in conjunction with other studies. Firstly, some improvements suitable for ReID are used on the PVT backbone, and we establish a basic model by using powerful methods verified on CNN-based ReID. Secondly, in an effort to further promote the robustness of the person features extracted by the PVT backbone, two new modules are designed. (1) The local feature clustering (LFC) is recommend to enhance the robustness of person features by calculating the distance between local features and global feature to select the most discrete local features and clustering them. (2) The side information embeddings (SIE) are used to encode non-visual information and send it into the network for training to reduce its impact on person features. Finally, the experiments show that PVTReID has achieved excellent results in ReID datasets and are 20% faster on average than CNN-based ReID methods.

Fine-grained image classification remains an ongoing challenge in the computer vision field, which is particularly intended to identify objects within sub-categories. It is a difficult task since there is a minimal and substantial intra-class variance. The current methods address the issue by first locating selective regions with Region Proposal Networks (RPN), object localization, or part localization, followed by implementing a CNN Network or SVM classifier to those selective regions. This approach, however, makes the process simple by implementing a single-stage end-to-end feature encoding with a localization method, which leads to improved feature representations of individual tokens/regions by integrating the transformer encoder blocks into the Yolov5 backbone structure. These Transformer Encoder Blocks, with their self-attention mechanism, effectively captured the global dependencies and enabled the model to learn relationships between distant regions. This improved the model ability to understand context and captured long-range spatial relationships in the image. We also replaced the Yolov5 detection heads with three transformer heads at the output for object recognition using the discriminative and informative features maps from transformer encoder blocks. We established the potential of the single stage detector for the fine-grained image recognition task, by achieving state of the art 93.4% accuracy, as well as outperforming the existing Yolov5 model. The effectiveness of our approach is assessed using the Stanford car dataset, which includes 16,185 images of 196 different classes of vehicles with significantly identical visual appearances.

Safe, high-quality and economical electric energy transportation is the basic requirements of modern power system operation. Transformer, as one of the core equipment of power system and national grid system, has the characteristics of diverse types, variable models and wide deployment. It is the basic equipment for power system to realize voltage change and electric energy distribution. Because the power system transformer needs to operate with load for a long time, the probability of failure is usually higher than that of other power equipment in general. This paper starts from the transformer fault and fault diagnosis. Firstly, the transformer fault types, traditional transformer fault diagnosis techniques and the advantages of machine learning in transformer fault diagnosis are reviewed. Secondly, the application of information system theory and information entropy in transformer fault diagnosis is introduced. Then it introduces machine learning technology, feature selection and extraction technology in transformer fault diagnosis, and machine learning algorithms such as support vector machine and extreme learning machine for transformer fault diagnosis. Finally, the potential development trend of information system theory, information entropy and machine learning in transformer fault diagnosis is forecasted. The combination of transformer fault prediction and machine learning algorithm is helpful for power system maintenance personnel to accurately predict the running state of power equipment, and also provides a new method and technology for the safe and reliable operation and regular maintenance of power system.

This paper describes the successful application of the Transformer model used in the natural language processing and vision tasks as a means of processing the time series of signals from gyroscope and accelerometer sensors for the classification of human activities. The Transformer model is based on deep neural networks with many layers which can generalize well on signals. All measured signals come from a smartphone placed in a waist bag. Activity prediction is sequence-to-sequence, each time step of the signal is assigned a designation of the performed activity. Emphasis is placed on attention mechanisms, which express individual dependencies between signal values within a time series. In comparison with another recent result, the recognition precision was improved from 89.67 percent to 99.2 percent. The transformer model should in the future be included among the top options in machine learning methods for human activity recognition.

Low-frequency audible noise generated by the magnetostriction effect inherent to the operation of power transformers has become a major drawback, especially in cases where the electrical substation is located in urban areas subject to strict environmental regulations that imposes sound pressure limits, differing for day and night periods. Such regulations apply a +5 dB penalty if a tonal component of noise is present, which is clearly the case of magnetostriction noise, typically concentrated at twice the industrial frequency (50 Hz or 60 Hz, depending on the country). The strategy used to eliminate the tonal characteristics, therefore contributing to establish compliance with the applicable regulation and to alleviate the discomfort it causes to the human ear, consisted in superimposing to the substation noise a masking sound synthesized from “sounds of nature” with suitable intensities, to flatten the noise spectrum while enhancing the soundscape. The masking system (heavy-duty speakers powered by a microprocessor platform) was validated at an already judicialized urban scenario. Measurement results confirmed that the masking solution was capable of flattening the tonal frequencies, whose beneficial effect yielded the cancellation of the public civil action filed by the neighbors. The proposed solution is ready to be replicated to other scenarios.

The measurement range of a conventional current sensor is narrow because it is used with signals relative to the rated values of the measurement range from a voltage-type device. Consequently, multiple current sensors must be used in accordance with the measurement range. To address this problem, this paper proposes a new current sensor with a clamp-shaped part for low current measurement and a simple straight structure for high current measurement. The output signals of the proposed current sensor are amplified with a Hall element using the magnetic force of a rectangular air gap inside the clamp. To verify the characteristics of the proposed current sensor, a current was applied to an external load, and the value determined by the current sensor noted. Then, electromagnetic field analysis was performed through current sensor modeling and the results obtained compared to the actual sensor results. The proposed sensor had a 1% linearity in the output signals and exhibited dynamic characteristics over a wide current range.

Artificial intelligence (AI)-based models hold the potential to transform higher education if adopted properly and ethically. A prime example is ChatGPT with earlier studies indicating its widespread adoption in academia and by university students. The current study aimed to identify the factors influencing the attitude towards ChatGPT and its usage among university students in Arab countries. A previous survey instrument termed Technology Acceptance Model Edited to Assess ChatGPT Adoption (TAME-ChatGPT) was administered online using a convenience-based approach among the contacts of the authors. Confirmatory factor analysis (CFA) for the survey constructs was done using root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI). The final study sample comprised a total of 2240 participant divided as follows: Iraq (n=736, 32.9%), Kuwait (n=582, 26.0%), Egypt (n=417, 18.6%), Lebanon (n=263, 11.7%), and Jordan (n=242, 10.8%). A total of 1048 respondents heard of ChatGPT before the study (46.8%), of which 551 used ChatGPT (52.6%). The mean scores of TAME-ChatGPT constructs showed that the ease of ChatGPT use, positive attitude towards technology and social influence, higher perceived usefulness, the influence of behavioral/cognitive factors, low perceived risks and low anxiety were the determinants of positive attitude to ChatGPT and its use. For both the attitude and usage scales of TAME-ChatGPT, CFA collectively yielded satisfactory fit indices as indicated by low RMSEA and SRMR together with high CFI and TLI. Multivariate analysis showed that attitude to ChatGPT use was significantly influenced by country of residence, age, university type, and the latest grade point average of the students. The current study confirmed the validity of TAME-ChatGPT as a survey instrument to assess the possible determinants of ChatGPT use among university students in Arab countries. The study findings highlighted that successful adoption of ChatGPT in higher education could be dependent on perceived ease of use, usefulness, positive attitudes to technology, social influence, behavioral/cognitive factors, lower anxiety, and minimal perceived risks. The utility of ChatGPT in higher education requires policies that should be tailored for various settings, considering the differences observed in attitude towards ChatGPT among participating students in this study.

Infrared and visible image fusion technologies are used to characterize the same scene by diverse modalities. However, most existing deep learning-based fusion methods are designed as symmetric networks, which ignore the differences between modal images and lead to the source image information loss during feature extraction. In this paper, we propose a new fusion framework for the different characteristics of infrared and visible images. Specifically, we design a dual-stream asymmetric network with two different feature extraction networks to extract infrared and visible feature maps respectively. The transformer architecture is introduced in the infrared feature extraction branch, which can force the network to focus on the local features of infrared images while still obtaining their contextual information. And the visible feature extraction branch uses residual dense blocks to fully extract the rich background and texture detail information of visible images. In this way, it can provide better infrared targets and visible details for the fused image. Experimental results on multiple datasets indicate that DSA-Net outperforms state-of-the-art methods in both qualitative and quantitative evaluations. In addition, we also apply the fusion results to the target detection task, which indirectly demonstrates the fusion performances of our method.

Many engineering systems are described by using differential models whose solutions, generally obtained after discretization, can exhibit a noticeable deviation with respect to the response of the physical systems that those models are expected to represent. In those circumstances one possibility consists of enriching the model in order to reproduce the physical system behavior. The present paper considers a dynamical system, and proposes enriching the model solution by learning the dynamical model of the gap between the system response and the model-based prediction, while ensuring that the time integration of the learnt model remains stable. The proposed methodology is applied on the simulation of the top-oil temperature evolution of a power transformer, whose data is provided by RTE, the French Electricity Transmission System Operator.

The battle of currents between AC and DC reignited as a result of the development in the field of power electronics. The efficiency of DC distribution systems is highly dependent on the efficiency of distribution converter, which calls for optimized schemes for efficiency enhancement of distribution converters. Modular solid-state transformers play a vital role in DC Distribution Networks and Renewable Energy systems (RES).This paper deals with efficiency-based load distribution for Solid State Transformers (SSTs) in DC distribution networks. Aim is to achieve a set of minimum inputs that are consistent with output while considering constraints and efficiency. As the main feature of modularity is associated with a three-stage structure of SSTs. This modular structure has been optimized using Ant Lion Optimizer (ALO) and validated by applying it EIA (Energy Information Agency) DC Distribution Network which contains SSTs. In the DC distribution grid, modular SSTs provide promising conversion of DC power from medium voltage to lower DC range (400V). The proposed algorithm is simulated in MATLAB and also compared with two other metaheuristic algorithms. The obtained results prove that the proposed method can significantly reduce input requirements for producing the same output while satisfying the specified constraints.

3D point cloud classification tasks have been a hot topic in recent years. Most existing point cloud processing frameworks lack context-aware features due to the deficiency of sufficient local feature extraction information. Therefore, we design an augmented sampling and grouping (ASG) module to efficiently obtain fine-grained features from the original point cloud. In particular, this method strengthens the domain near each centroid and makes reasonable use of the local mean and global standard deviation to mine point cloud’s local and global features. In addition to this, inspired by the transformer structure UFO-ViT in 2D vision tasks, we first try to use a linearly-normalized attention mechanism in point cloud processing tasks, investigating a novel transformer-based point cloud classification architecture UFO-Net. An effective local feature learning module is adopted as a bridging technique to connect different feature extraction modules. Importantly, UFO-Net employs multiple stacked blocks to better capture feature representation of the point cloud. Extensive ablation experiments on public datasets show that our method outperforms other state-of-the-art methods. For instance, our network performed with 93.7% overall accuracy on the ModelNet40 dataset, which was 0.5% higher than PCT. Our network also archived 83.8% overall accuracy on the ScanObjectNN dataset, which is 3.8% better than PCT.

Indoor localization is an active area of research dominated by traditional machine-learning techniques. Deep learning-based systems have shown unprecedented improvements and have accomplished exceptional results over the past decade, especially the Transformer network within natural language processing (NLP) and computer vision domains. We propose the hyper-class Transformer (HyTra), an encoder-only Transformer with multiple classification heads (one per class) and learnable embeddings, to investigate the effectiveness of Transformer-based models for received signal strength (RSS) based WiFi fingerprinting. HyTra leverages learnable embeddings and the self-attention mechanism to determine the relative position of the wireless access points (WAPs) within the high-dimensional embedding space, improving the prediction of user location. From an NLP perspective, we consider a fixed order sequence of all observed WAPs as a sentence and the captured RSS value(s) for every given WAP at a given reference point from a given user as words. We test our proposed network on public and private datasets of different sizes, proving that the quality of the learned embeddings and overall accuracy improves with increments in samples.

Predicting Length of Stay (LoS) and understanding its underlying factors is essential to minimize the risk of hospital-acquired conditions, improve financial, operational, and clinical outcomes, and to better manage future pandemics. The purpose of this study is to forecast patients’ LoS using a deep learning model and analyze cohorts of risk factors minimizing or maximizing LoS. We employed various pre-processing techniques, SMOTE-N to balance data, and Tab-Transformer model to forecast LoS. Finally, Apriori algorithm was applied to analyze cohorts of risk factors influencing LoS at hospital. The Tab-Transformer outperformed the base Machine Learning models with an F1-score (.92), precision (.83), recall (.93), and accuracy (.73) for discharge dataset, and F1-score (.84), precision (.75), recall (.98), and accuracy (.77) for deceased dataset. The association mining algorithm was able to identify significant risk factors/indicators belonging to lab, X-Ray, and clinical data such as elevated LDH, and D-Dimer, lymphocytes count, and comorbidities such as hypertension and diabetes responsible for extending patients LoS. It also reveals what treatments has reduced the symptoms of COVID-19 patients leading to reduction in LoS particularly when no vaccines or medication such as Paxlovid were available.

A low-cost medium power carbon dioxide (CO₂) laser system is designed, constructed, and characterized to produce coherent, monochromatic laser radiation in the Infrared region. The laser cavity is simulated and designed by using ZEMAX optic studio. A switch-mode high tension pump source is designed and constructed using a flyback transformer and simulated using NI Multisim to study the voltage behavior at different node points. A prototype cooling system/chiller is designed and built using the Thermo-Electric Coolers (TEC) to remove the excess heat produced during laser action. Various parameters, like pumping mechanism, chiller stability, efficiency, output power, and current at different applied voltages, are studied. The chiller efficiency at different output powers of the laser is analyzed, which clearly shows that the chiller's cooling rate is good enough to compensate for the heat generated by the laser system. The center wavelength of the carbon dioxide laser is 10.6μm with FWHM of 1.2nm simulated in the ZEMAX optic studio. The output beam penetration through salt rock (NaCl), wood, and acrylic sheet at various output powers is analyzed to measure the penetration depth rate of the CO₂ laser.

Neural decoding from spiking activity is an essential tool for understanding the information encoded in population neurons, especially in applications like brain-computer interface (BCI). Various quantitative methods have been proposed and have shown superiorities under different scenarios respectively. From the machine learning perspective, the decoding task is to map the high-dimensional spatial & temporal neuronal activity to the low-dimensional physical quantities (e.g., velocity, position). Because of the complex interactions and the abundant dynamics among neural circuits, good decoding algorithms usually have the capability of capturing flexible spatiotemporal structures embedded in the input feature space. Recently, the Transformer-based models are widely used in processing natural languages and images due to its superior performances in handling long-range and global dependencies. Hence, in this work we examine the potential applications of Transformers in neural decoding and introduce two Transformer-based models. Besides adapting the Transformer to neuronal data, we also propose a data augmentation method for overcoming the data shortage issue. We test our models on three experimental datasets and their performances are comparable to the previous state-of-the-art (SOTA) RNN-based methods. In addition, Transformer-based models show increased decoding performances when the input sequences are longer, while LSTM-based models deteriorate quickly. Our research suggests that Transformer-based models are important additions to the existing neural decoding solutions, especially for large datasets with long temporal dependencies.

Social media platforms are increasingly enabling the propagation of content from groups related to drug use, thus posing risks for the wider population and, in particular, individuals who are amenable to drug use and drug addiction. The detection of drug use content on social media platforms is a priority for governments, technology companies, and drug law enforcement organizations. To counter this issue, various techniques have been developed to identify and promptly remove drug use content, while also blocking its creators from network access. In this paper, we introduce a manually annotated Twitter dataset, comprising 156,521 tweets published between 2008 and 2022, specifically compiled for the purpose of drug use detection. The dataset underwent annotation by several group of expert annotators who classified the tweets as either drug use or non-drug use. Exploratory data analysis was conducted to comprehend the dataset's characteristics. Various classification algorithms, including SVM, XGBoost, RF, NB, LSTM, and BERT were employed using the dataset. Among the traditional machine learning models, SVM utilizing term frequency-inverse document frequency features achieved the highest F1-Score (0.9017). However, BERT with textual features concatenated with numerical and categorical features in ensemble method surpassed the performance of traditional models, attaining F1-Score of 0.9112. To facilitate future research and enhance English online drug use classification accuracy, the dataset will be made publicly available.

The continuous advancements in healthcare technology have empowered the discovery, diagnosis, and prediction of diseases, revolutionizing the field. Artificial intelligence (AI) is expected to play a pivotal role in achieving the goals of precision medicine, particularly in disease prevention, detection, and personalized treatment. This study aims to determine the optimal combination of the mother wavelet and AI model for the analysis of pediatric electroretinogram (ERG) signals. The dataset, consisting of signals and corresponding diagnoses, undergoes Continuous Wavelet Transform (CWT) using commonly used wavelets to obtain a time-frequency representation. Wavelet images were used for the training of five widely used deep learning models: VGG-11, ResNet-50, DensNet-121, ResNext-50, and Vision Transformer, to evaluate their accuracy in classifying healthy and unhealthy patients. The findings demonstrate that the combination of Ricker Wavelet and Vision Transformer consistently yields the highest median accuracy values for ERG analysis, as evidenced by the upper and lower quartile values. The median balanced accuracy of the obtained combination of the three considered types of ERG signals in the article are 0.83, 0.85, and 0.88. However, other wavelet types also achieved high accuracy levels, indicating the importance of carefully selecting the mother wavelet for accurate classification. The study provides valuable insights into the effectiveness of different combinations of wavelets and models in classifying ERG wavelet scalograms.

This primer provides an overview of the rapidly evolving field of generative artificial intelligence, specifically focusing on large language models like ChatGPT (OpenAI) and Bard (Google). Large language models have demonstrated unprecedented capabilities in responding to natural language prompts. The aim of this primer is to demystify the underlying theory and architecture of large language models, providing intuitive explanations for a broader audience. Learners seeking to gain insight into the technical underpinnings of large language models must sift through rapidly growing and fragmented literature on the topic. This primer brings all the main concepts into a single digestible document. Topics covered include text tokenization, vocabulary construction, token embedding, context embedding with attention mechanisms, artificial neural networks, and objective functions in model training. The primer also explores state-of-the-art methods in training large language models to generalize on specific applications and to align with human intentions. Finally, an introduction to the concept of prompt engineering highlights the importance of effective human-machine interaction through natural language in harnessing the full potential of artificial intelligence chatbots. This comprehensive yet accessible primer will benefit students and researchers seeking foundational knowledge and a deeper understanding of the inner workings of existing and emerging artificial intelligence models. The author hopes that the primer will encourage further responsible innovation and informed discussions about these increasingly powerful tools.

Large Language Models (LLMs) have revolutionized the field of Artificial Intelligence with their ability to understand and generate natural language discourse. This has led to the use of LLMs in various creative fields such as music, art, and storytelling, which has sparked a debate on the impact of LLMs on human creativity. This article explores the cyclic effect of LLMs on human creativity, where LLMs can generate an ever-evolving cycle of creative output from both humans and machines. The article also highlights the importance of responsible and ethical use of LLMs, including privacy and copyright issues with the training data used by LLMs. The article emphasizes the need for transparency, data security measures, and AI governance policies to protect user data and ensure the safe use of LLMs. It also calls for the development of educational materials and resources to increase public understanding of LLMs and their ethical implications. The article concludes by highlighting the potential of LLMs as powerful tools for communication and education, but also emphasizes the need for ethical considerations to ensure the responsible and beneficial use of LLMs in society.

Paraphrase Generation is one of the most important and challenging tasks in the field of Natural Language Generation. The paraphrasing techniques help to identify or to extract/generate phrases/sentences conveying the similar meaning. The paraphrasing task can be bifurcated into two sub-tasks namely, Paraphrase Identification (PI) and Paraphrase Generation (PG). Most of the existing proposed state-of-the-art systems have the potential to solve only one problem at a time. This paper proposes a light-weight unified model that can simultaneously classify whether given pair of sentences are paraphrases of each other and the model can also generate multiple paraphrases given an input sentence. Paraphrase Generation module aims to generate fluent and semantically similar paraphrases and the Paraphrase Identification systemaims to classify whether sentences pair are paraphrases of each other or not. The proposed approach uses an amalgamation of data sampling or data variety with a granular fine-tuned Text-To-Text Transfer Transformer (T5) model. This paper proposes a unified approach which aims to solve the problems of Paraphrase Identification and generation by using carefully selected data-points and a fine-tuned T5 model. The highlight of this study is that the same light-weight model trained by keeping the objective of Paraphrase Generation can also be used for solving the Paraphrase Identification task. Hence, the proposed system is light-weight in terms of the model’s size along with the data used to train the model which facilitates the quick learning of the model without having to compromise with the results. The proposed system is then evaluated against the popular evaluation metrics like BLEU (BiLingual Evaluation Understudy):, ROUGE (Recall-Oriented Understudy for Gisting Evaluation), METEOR, WER (Word Error Rate), and GLEU (Google-BLEU) for Paraphrase Generation and classification metrics like accuracy, precision, recall and F1-score for Paraphrase Identification system. The proposed model achieves state-of-the-art results on both the tasks of Paraphrase Identification and paraphrase Generation.

Human pose estimation is a complex detection task in which the network needs to capture the rich information contained in the images. In this paper, we propose MSTPose (Multi-Scale Transformer for human Pose estimation). Specifically, MSTPose leverages a high-resolution Convolution neural network (CNN) to extract texture information from images. For the feature maps from three different scales produced by the backbone network, each branch performs the coordinate attention operations. The feature maps are then spatially and channel-wise flattened, combined with keypoint tokens generated through random initialization, and fed into a parallel Transformer structure to learn spatial dependencies between features. As the Transformer outputs one-dimensional sequential features, the mainstream two-dimensional heatmap method is abandoned in favor of one-dimensional coordinate vector regression. The experiments show that MSTPose outperforms other CNN-based pose estimation models and demonstrates clear advantages over CNN + Transformer networks of similar types.

The computer network world is changing and the NetDevOps approach has brought the dynamics of applications and systems into the field of communication infrastructure. Businesses are changing and businesses are faced with difficulties related to the diversity of hardware and software that make up those infrastructures. The "Intent-Based Networking - Concepts and Definitions" document describes the different parts of the ecosystem that could be involved in NetDevOps. The recognize, generate intent, translate and refine features need a new way to implement algorithms. This is where artificial intelligence comes in.

"Finding fresh water in the ocean of data." is a challenge that all deep learning domains struggle with, especially in the area of hyperspectral image analysis. As hyperspectral remote sensing technology advances by leaps and bounds, there are increasing amounts of hyperspectral images(HSIs) can be available. Whereas in fact, these unlabeled HSIs are powerless to be used as material to driven a supervised learning task due to the extremely expensive labeling costs and some unknown regions. Although learning-based methods have achieved remarkable performance due to their superior ability to represent features, at the cost, these methods are complex, inflexible and tough to carry out transfer learning. In this paper, we propose the "Instructional Mask AutoEncoder"(IMAE), which is a simple and powerful self-supervised learner for HSI classification that uses a transformer-based mask autoencoder to extract the general features of HSIs through a self-reconstructing agent task. Moreover, we utilize the metric learning to perform an instructor which can direct the model focus on the human interested region of the input so that we can alleviate the defects of transformer-based model such as local attention distraction, lack of inductive bias and tremendous training data requirement. In downstream forward propagation, instead of global average pooling, we employ a learnable aggregation to put the tokens into fullplay. The obtained results illustrate that our method effectively accelerates the convergence rate and promotes the performance in downstream task.

Interactions between proteins and ions are essential for various biological functions like structural stability, metabolism, and signal transport. Given that more than half of all proteins bind to ions, it becomes crucial to identify ion-binding sites. Accurate identification of protein-ion binding sites helps us to understand proteins’ biological functions and plays a significant role in drug discovery. While several computational approaches have been proposed, this remains a challenging problem due to the small size and high versatility of metals and acid radicals. In this study, we propose IonPred, a sequence-based approach that employs ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately) to predict ion binding sites using only raw protein sequences. We successfully fine-tuned our pretrained model to predict the binding sites for nine metal ions (Zn2+, Cu2+, Fe2+, Fe3+, Ca2+, Mg2+, Mn2+, Na+, and K+) and four acid radical ion ligands (CO32−, SO42−, PO43−, NO2−). IonPred surpassed six current state-of-the-art tools by over 44.65% and 28.46% respectively in F1 score and MCC when compared on an independent test dataset. Our method is more computationally efficient than existing tools producing prediction results for a hundred sequences for a specific ion in under ten minutes.

Accurate prediction of crop production is essential in effectively managing agricultural countries' food security and economic resilience. This study evaluates the performance of statistical and machine learning-based methods for large-scale crop production forecasting. We predict the quarterly production of 325 crops (including fruits, vegetables, cereals, non-food, and industrial crops) across 83 provinces in the Philippines. Using a comprehensive dataset of 10,949 time series over 13 years, we demonstrate that a global forecasting approach using a state-of-the-art deep learning architecture, the transformer, significantly outperforms traditional local forecasting approaches built on statistical and baseline methods. By leveraging cross-series information, our proposed way is scalable and works well even with time series that are short, sparse, intermittent, or exhibit structural breaks/regime shifts. The results of this study further advance the field of applied forecasting in agricultural production and provide a practical and effective decision-support tool for policymakers that oversee the farm sector on a national scale.

Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operation, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying breast cancer from unregistered multi-view mammograms. This motivates us to leverage the architecture of Vision Transformers to capture long-range relationships of multiple mammograms from the same patient. For this purpose, we employ local Transformer blocks to separately learn patch relationships within a specific view (CC/MLO) of one side (right/left) of mammogram. The outputs from different views and sides are concatenated and fed into global Transformer blocks, to jointly learn patch relationships between two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of CC and MLO view mammograms, which include 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our two-view Transformer-based model achieves lesion classification accuracy of 77.0% and area under ROC curve (AUC = 0.814), outperforming state-of-the-art multi-view CNNs by 3.1% and 3%, respectively. Meanwhile, the new two-view model improves mammographic case classification accuracies of two single-view models by 7.4% (CC) and 4.5% (MLO), respectively. The promising results unveil the great potential of using Transformers to develop high-performing computer-aided diagnosis (CADx) schemes of mammography.

Requirements Engineering (RE) is an important step in the whole software development lifecycle. The problem in RE is to determine the class of the software requirements as functional and non-functional. Proper and early identification of these requirements is vital for the entire development cycle. On the other hand, manual government of requirement classes is a timewasting task, and it needs intensive effort. Automatic requirement classification techniques through advanced Machine Learning (ML) strategies are started to be developed to address this problem. Basically, software requirements are generated as natural language explanations and therefore requirement classification may be handled as a particular Natural Language Processing (NLP) problem. From ML point of view, classification strategies require dataset to be used in the training/learning phase. For this goal, we generated a unique Turkish dataset having collected the requiremenst from real-world software projects with 4600 samples. Of these requirements, 3000 and 1600 were labeled as functional and non-functional, respectively. The data set has been evaluated using (i) traditional machine learning algorithms, (ii) deep learning algorithms, and (iii) transformer models respectively. As a result BERTurk was found to be the most successful algorithm to discriminate functional and non-functional requirements in terms of 95% f-score metric.

The E-learning system has achieved great development after the pandemic. In this work, we proposed three artificial intelligence-based enhancements to our linguistic interactive E-learning system from different aspects. Compared with the original phonetic transcription exam system, our enhancements include an MFCC+CNN-based disordered speech classification module, a Transformer-based Grapheme-to-Phoneme converter, and a Tacotron2-based IPA-to-Speech speech synthesis system. This work not only provides a better experience for the users of this system but also explores the utilization of artificial intelligence technologies in the E-learning field and linguistic field.

In the leading/trailing edge’s adaptive machining of the near-net-shaped blade, a small portion of the theoretical part is retained for securing aerodynamic performance by manual work. However, this procedure is time-consuming and depends on the human experience. In this paper, we defined retained theoretical leading/trailing edge as the reconstruction area. To accelerate the reconstruction process, an anchor-free neural network model based on Transformer was proposed, named LETR (Leading/trailing Edge Transformer). LETR extracts image features from an aspect of mixed frequency and channel domain. We also integrated LETR with the newest meta-Acon activation function. We tested our model on the self-made dataset LDEG2021 on a single GPU and got an mAP of 91.9\%, which surpassed our baseline model, Deformable DETR by 1.1\%. Furthermore, we modified LETR’s convolution layer and named the new model after GLETR (Ghost Leading/trailing Edge Transformer) as a lightweight model for real-time detection. It is proved that GLETR has fewer weight parameters and converges faster than LETR with an acceptable decrease in mAP (0.1\%) by test results.

U-Net, based on a deep convolutional neural network (CNN), has been clinically used to au-to-segment normal organs and potentially target volumes. However, CNNs with local geometric dependencies may limit the accuracy of segmentation. Additionally, the performance of CNNs can vary depending on the selection of network hyper-parameters, which was mitigated by the proposition of nnU-Net. We chose a vision transformer architecture called VT U-Net, which features a self-attention excluding the convolution layer, to overcome the limitations of CNNs by utilizing global geometric information of images. The VT U-Net v.2 became more powerful thanks to the adaptive hyper-parameter optimizer embedded in nnU-Net. However, despite leveraging the benefits of nnU-Net, VT U-Net v.2 still had additional network hyper-parameters that needed to be optimally chosen. Accordingly, among various hyper-parameters, this study attempted to find the optimal combination of the patch size and the embedded dimension regarding the transformer. From the 4-fold cross-validation, the modified VT U-Net v.2 showed the highest average performance for planning target volume (PTV) segmentation among the investigated networks. Though nnU-Net was based on convolution layers, the adaptive hyper-parameter optimizers turned out to enhance the performance. It was also confirmed that network hyper-parameters affected the segmentation accuracy of vision transformers.

One of the critical problems in multiclass classification tasks is the imbalance of the dataset. This is especially true when using contemporary pre-trained neural networks, where, in fact, the last layers of the neural network are retrained. Therefore, the large datasets with highly unbalanced classes are not good for models’ training since the use of such a dataset leads to overfitting and, accordingly, poor metrics on test and validation datasets. In this paper the sensitivity to a dataset imbalance of Xception, ViT-384, ViT-224, VGG19, ResNet34, ResNet50, ResNet101, Inception_v3, DenseNet201, DenseNet161, DeIT was studied using a highly imbalanced dataset of 20,971 images sorted into 7 classes. It is shown that the best metrics were obtained when using a cropped dataset with augmentation of missing images in classes up to 15% of the initial number. So, the metrics can be increased by 2-6% compared to the metrics of the models on the initial unbalanced data set. Moreover, the metrics of the rare classes' classification also improved significantly – the TruePositive value can be increased by 0.3 and more. As result, the best approach to train considered networks on an initially unbalanced dataset was formulated.

OpenAI has managed to turn 100 million heads in two months towards their new Language model tool (LM) ChatGPT. The third generation Generative Pretrained Transformer (GPT-3) has the capacity to tackle simple to complex and sophisticated problems, while providing reasoning behind its generated answers. ChatGPT can be used to increase productivity and improve efficiency and face challenging problems. Programming mobile applications is a challenging task that requires professional software engineers, skills and abilities to be developed. The following paper takes a case study approach to assess how novice app developers can use ChatGPT to generate Java scripts that will be used in Android studio to create a functional application. The results after, many iterations and ongoing conversations with ChatGPT managed to create an application for the anticipated function. Important insights have been drawn from the case study that could set the ground for any novice user seeking to create applications using Java scripting and Android Studio.

Oil immersed paper insulation condition is a crucial aspect of power transformer’s life condition diagnostic. The measurement testing database collected over the years made it possible for researchers to implement classification analysis to in-service power transformer. This article presents classification analysis of transformer oil-immersed paper insulation condition. The measurements data (dielectric characteristics, dissolved gas analysis, and furanic compounds) of 149 transformers with primary voltage of 150 kV had been gathered and analyzed. The algorithm used for developing classification model is Support Vector Machine (SVM). The model has been trained and tested using different datasets. Different models have been created and the best chosen, resulting in 90.63% accuracy in predicting the oil-immersed paper insulation condition. Further implementation was executed to classify oil-paper condition of 19 Transformers which Furan data is not available. The classification results combined, reviewed, and compared to conventional assessment methods and standards, confirming that the model developed has the ability to do classification of current oil-paper condition based on Dissolved Gasses and Dielectric Characteristics.

Compared with conventional methods of fault diagnosis for power transformers, which have defects such as imperfect encoding and too absolute encoding boundaries, this paper systematically discusses various intelligent approaches applied in fault diagnosis and decision making for large oil-immersed power transformers based on dissolved gas analysis (DGA), including expert system (EPS), artificial neural network (ANN), fuzzy theory, rough sets theory (RST), grey system theory (GST), swarm intelligence (SI) algorithms, data mining technology, machine learning (ML), and other intelligent diagnosis tools, and summarizes existing problems and solutions. From this survey, it is found that a single intelligent approach for fault diagnosis can only reflect operation status of the transformer in one particular aspect, causing various degrees of shortcomings that cannot be resolved effectively. Combined with the current research status in this field, the problems that must be addressed in DGA-based transformer fault diagnosis are identified, and the prospects for future development trends and research directions are outlined. This contribution presents a detailed and systematic survey on various intelligent approaches to faults diagnosing and decisions making of the power transformer, in which their merits and demerits are thoroughly investigated, as well as their improvement schemes and future development trends are proposed. Moreover, this paper concludes that a variety of intelligent algorithms should be combined for mutual complementation to form a hybrid fault diagnosis network, such that avoiding these algorithms falling into a local optimum. Moreover, it is necessary to improve the detection instruments so as to acquire reasonable characteristic gas data samples. The research summary, empirical generalization and analysis of predicament in this paper provide some thoughts and suggestions for the research of complex power grid in the new environment, as well as references and guidance for researchers to choose optimal approach to achieve DGA-based fault diagnosis and decision of the large oil-immersed power transformers in preventive electrical tests.

This paper develops a novel dynamic correction method for the reliability assessment of large oil-immersed power transformers. First, with the transformer oil-paper insulation system (TOPIS) as the target of evaluation and the winding hot spot temperature (HST) as the core point, an HST-based static ageing failure model is built according to the Weibull distribution and Arrhenius reaction law, in order to describe the transformer ageing process and calculate the winding HST for obtaining the failure rate and life expectancy of TOPIS. A grey target theory based dynamic correction model is then developed, combined with the data of Dissolved Gas Analysis (DGA) in power transformer oil, in order to dynamically modify the life expectancy calculated by the built static model, such that the corresponding relationship between the state grade and life expectancy correction coefficient of TOPIS can be built. Furthermore, the life expectancy loss recovery factor is introduced to correct the life expectancy of TOPIS again. Lastly, a practical case study of an operating transformer has been undertaken, in which the failure rate curve after introducing dynamic corrections can be obtained for the reliability assessment of this transformer. The curve shows a better ability of tracking the actual reliability level of transformer, thus verifying the validity of the proposed method and providing a new way for transformer reliability assessment. This contribution presents a novel model for the reliability assessment of TOPIS, in which the DGA data, as a source of information for the dynamic correction, is processed based on the grey target theory, thus the internal faults of power transformer can be diagnosed accurately as well as its life expectancy updated in time, ensuring that the dynamic assessment values can commendably track and reflect the actual operation state of the power transformers.

The unstable international economic situation is reflected in the supply chain stress, lack or increased cost of some raw materials, fuel or semi-finished products is forcing organizations to perform new optimization initiatives in the utilization of their equipment and assets pointed to obtain the maximum value from them, while maintaining and even improving the quality of their products. The achievement of these objectives involves the reduction or minimization of equipment downtime to maintain the advantage over their competitors and ensure the organization's competitiveness. The intelligent maintenance system (IMS) provides adequate support for decision-making related to equipment maintenance, since poor maintenance results in unplanned stoppages, with the consequent additional cost and increased customer dissatisfaction, and an over-maintenance can result in an additional labor cost, time and the replacement of parts that are in good conditions. The utilization of new tools and technologies introduced by Industry 4.0 offers multiple opportunities for enhancement through communication and computerized data processing, aiming to improve the maintainability of a hydrogen compressor using neural networks based on attention mechanisms combined with linear regression.

In recent years, the analysis of macro- and micro-expression has drawn the attention of researchers since they provide visual cues of an individual's emotions for a broad range of potential applications such as lie detection and criminal detection. In this paper, we address the challenge of spotting facial macro- and micro-expression from videos and present compelling results by using a deep learning approach to analyze the optical flow features. Different from other deep learning approaches that are mainly based on Convolutional Neural Networks (CNNs), we propose a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval. In contrast to other Transformer-based models that achieve high performance by being pre-trained on large datasets, our deep learning model, called SL-Swin, which incorporates Shifted Patch Tokenization and Locality Self-Attention into the backbone network Swin Transformer, effectively spots macro- and micro-expressions by being trained from scratch on small-size expression datasets. Our evaluation outcomes surpass the MEGC 2022 spotting baseline result, obtaining an overall F1-score of 0.1366. Additionally, our approach also performs well in the MEGC 2021 spotting task with an overall F1-score of 0.1824 and 0.1357 on CAS(ME)^2 and SAMM Long Videos, respectively. The code is publicly available on GitHub (https://github.com/eddiehe99/pytorch-expression-spotting).

Background: In regenerative dentistry the graft material influences the success. It should act as an osteoconductive scaffold, providing a mineral substrate during resorption and inducing the activity of osteoinductive cells capable of producing new bone, platelet growth factors, and cell differentiation factors inducing the differentiation of undifferentiated mesenchymal cells into. Given that dentin shares many biochemical characteristics with bone tissue, it has recently attracted considerable interest as a biomaterial for bone repair. The aim of this study is to compare two grinder types in order to determine the optimal method for producing dentinal particles using a mechanical grinder. Materials and methods: A sample of 40 natural human teeth without restorations, prostheses or root canal treatments was used and divided into two groups subjected to two different grinder speeds (high-speed and low-speed). The high-speed showed a greater dispersion (53.5+-9.89% of the tooth) due to the pulverisation (highly thin granules) of part of the tooth. The low-speed grinder does not pulverize the dentin and the percentage of tooth loss is 9.16+/-2.34%. Conclusion: The low-speed gringer allows to save a major part of the tooth and to have a maximum quantity of graft material but requires more time. Further studies must be promoted to optimise the grinding procedures.

The increased complexity of modern systems is calling for an integrated and comprehensive approach to system design and development and in particular, a shift towards Model-Based Systems Engineering (MBSE) approaches for system design. The requirements that serve as the foundation for these intricate systems are still primarily expressed in Natural Language (NL), which can contain ambiguities and inconsistencies that hinder their direct translation into models. The colossal developments in the field of Natural Language Processing (NLP) in general and Large Language Models (LLMs) in particular can serve as an enabler for the conversion of NL requirements into semi-machine-readable requirements. This is expected to facilitate their standardization and use in a model-based environment. This paper discusses a two-fold strategy for converting NL requirements into semi-machine-readable requirements using language models. The first approach involves creating a requirements table by extracting information from free-form NL requirements. The second approach is an agile methodology that facilitates the identification of boilerplate templates for different types of requirements based on observed linguistic patterns. For this study, three different LLMs were utilized. Two of these models were fine-tuned versions of Bidirectional Encoder Representations from Transformers (BERT), specifically aeroBERT-NER and aeroBERT-Classifier, which were trained on annotated aerospace corpora. Another LLM, called flair/chunk-english, was utilized to identify sentence chunks present in NL requirements. All three language models were utilized together to achieve the standardization of requirements. To demonstrate the effectiveness of the methodologies, requirements from Parts 23 and 25 of Title 14 Code of Federal Regulations (CFRs) were employed, and a total of two, five, and three boilerplate templates were identified for design, functional, and performance requirements, respectively.

Because imbalanced power will cause the loss of the propulsion motor of the railway vehicle, and the increase in temperature will shorten the service life of the electric vehicle. Not only this, but also increase the cost of electricity and maintenance. In the past, the industry only focused on methods to improve power quality such as load capacity, relay setting, and harmonic resolution. Now, the consider of three-phase unbalance rate (TPUR) must be applied. I propose special transformers wiring (STW) to improve the three unbalance rates and provide different transformer wiring methods. According to the IEEE Committee, in the future, power companies will need to install balanced relay stations to improve three-phase unbalance rate. the internal regulations of Taipower must be less than 4.5% (voltage unbalance rate (NPSUR)of 2.5% and motor temperature rise of 12.5%). the derivation of the transformer "three-phase unbalance rate" model is the focus of the railway system. This research is based on the model derivation of different wiring methods to improve the hot problem caused by the three-phase imbalance and improve the service life of the train. And pointed out that Scott, Le-Blanc, Modified-Woodbridge three wiring methods can be applied to future railway system routes to improve the three-phase unbalance rate, in line with the IEEE standard of less than 2%.

This paper presents a symmetrical topology for the design of solid-state transformer, made up of power switching converters, to replace conventional bulky transformers. The proposed circuitry not only reduces the overall size but also provides power flow control with the ability to be interfaced with renewable energy resources (RESs) to fulfill the future grid requirements at consumer end. Solid state transformer provides bidirectional power flow with variable voltage and frequency operation and has the ability to maintain unity power factor, and current total harmonic distortion (THD) for any type of load within defined limits of IEEE standard. Solid State Transformer offers much smaller size as compared to that of the conventional iron core transformer. MATLAB/Simulink platform is adopted to test the validity of the proposed circuit for different scenarios by providing the simulation results evaluated at 25 kHz switching frequency.