Domain generalization has led to remarkable achievements in Hyperspectral Image (HSI) classification. Inspired by contrastive language-image pre-training (CLIP), the language-aware domain generalization method has been explored to learn cross-domain-invariant representation. However, existing methods face some challenges: 1) The weak capacity to extract long-range contextual information and inter-class correlation. 2) Due to the inadequacies of the large-scale pre-training for HSI data, the spatial-spectral features of HSI and linguistic features can not be straightforwardly alignment. To address the above problems, a novel network has been proposed with a CLIP framework, which consists of an image encoder, based on an encoder-only transformer to obtain the global contextual information and inter-class correlation, a frozen text encoder, and a cross-attention mechanism, named Linguistic-Interact-with-Visual Engager (LIVE), enhances the interaction between two modalities. Extensive experiments demonstrating superior performance over state-of-the-art methods in HSI Domain Generalization with a CLIP framework.

The increasing use of deep learning techniques to manipulate images and videos, commonly referred to as "deepfakes," is making more and more challenging to differentiate between real and fake content. While various deepfake detection systems have been developed, they often struggle to detect deepfakes in real-world situations. In particular, these methods are often unable to effectively distinguish images or videos when these are modified using novel techniques which have not been used in the training set. In this study, we carry out an analysis of different deep learning architectures in an attempt to understand which is more capable of better generalizing the concept of deepfake. According to our results, it appears that Convolutional Neural Networks (CNNs) seem to be more capable of storing specific anomalies and thus excel in cases of datasets with a limited number of elements and manipulation methodologies. The Vision Transformer, conversely, is more effective when trained with more varied datasets, achieving more outstanding generalization capabilities than the other methods analysed. Finally, the Swin Transformer appears to be a good alternative for using an attention-based method in a more limited data regime. All the analyzed architectures seem to have a different way to look at deepfakes but since in a real-world environment, the generalization capability is essential, based on the carried out experiments the Vision Transformer seems to provide superior performances.

(1) Purpose: This work aims at developing a generalizable MRI reconstruction model in the meta-learning framework. The standard benchmarks in meta-learning are challenged by learning on diverse task distributions. The proposed network learns the regularization function in a variational model and reconstructs MR images with various under-sampling ratios or patterns that may or may not be seen in the training data by leveraging a heterogeneous dataset. (2) Methods: We propose an unrolling network induced by learnable optimization algorithms (LOA) for solving our nonconvex nonsmooth variational model for MRI reconstruction. In this model, the learnable regularization function contains a task-invariant common feature encoder and task-specific learner represented by a shallow network. To train the network we split the training data into two parts: training and validation, and introduce a bilevel optimization algorithm. The lower-level optimization trains task-invariant parameters for the feature encoder with fixed parameters of the task-specific learner on the training dataset, and the upper-level optimizes the parameters of the task-specific learner on the validation dataset. (3) Results: The PSNR increases 1.5 dB on average compared to the network trained through conventional supervised learning on the seen CS ratios. We test the result of quick adaption on the unseen tasks after meta-training, the average PSNR arises 1.22 dB compared to the conventional learning procedure that is directly trained on the unseen CS ratios in the meanwhile saving half of the training time. The average PSNR arises 1.87 dB for unseen sampling patterns comparing to conventional learning; (4) Conclusion: We proposed a meta-learning framework consisting of the base network architecture, design of regularization, and bi-level optimization-based training. The network inherits the convergence property of the LOA and interpretation of the variational model. The generalization ability is improved by the designated regularization and bilevel optimization-based training algorithm.

Deep learning machines are computational models composed of multiple processing layers of adaptive weights to learn representations of data with multiple levels of abstraction. Their structure is mainly reflecting the intuitive plausibility of decomposing a problem into multiple levels of computation and representation since it is believed that higher layers of representation allow a system to learn complex functions. Surprisingly, after decades of research, from learning and design perspectives these models are still deployed in a heuristic manner. In this paper, deep learning feed-forward machines are modeled from a statistical mechanics point of view as disordered physical systems where its macroscopic behavior is determined in terms of the interactions defined between the basic constituent of these models, namely, the artificial neuron. They are viewed as the equilibrium states of a theoretical body that is subject to the law of increase of the entropy. The study of the changes in energy of the body when passing from one equilibrium state to another is used to understand the structure and role of the phase space of the system, the stability of the equilibrium states, and the resulting degree of disorder. It is shown that the topology of these models is strongly linked to their stability and resulting level of disorder. Furthermore, the proposed theoretical characterization permit to assess the thermodynamic efficiency with which information can be processed by these models, and to provide a practical methodology to quantitatively estimate and compare their expected learning and generalization capabilities. These theoretical results provides new insights to the theory of deep learning and their implications are shown to be consistent through a set of benchmarks designed to experimentally assess their validity.

Building outlines extracted from remote sensing images and raster maps often have irregular boundaries, redundant points, inaccurate positions, and unclear turns due to factors such as image quality, complexity of the surrounding environment, and extraction methods. This study proposes a regularization algorithm for right-angled polygon building outlines with jagged edges. First, the minimum bounding rectangle of the building outline is established and populated with a square grid based on the smallest visible length principle. An overlay analysis is then applied to the grid and original building to extract the turning points of the outline. Finally, the building orientation is used as a reference axis to sort the turning points and reconstruct the simplified building outline. Analysis of the experimental results shows that the proposed simplification method enhances the morphological characteristics of building outlines, such as parallelism and orthogonality, while considering simplification principles, such as the preservation of direction, position, area, and shape of the building. The proposed algorithm provides a new regularization method for building outlines with jagged edges.

Change Detection (CD) of aerial images refers to identifying and analyzing changes between two or more aerial images of the same location taken at different times. The CD is a highly challenging task due to the need to distinguish relevant changes, such as urban expansion, deforestation, or post-disaster damage assessment, from irrelevant ones, such as light conditions, shadows, and seasonal variations. Many CD papers have recently been published, where most of the papers that proposed a new model contained a comparison between their proposed and state-of-the-art (SOTA) models. While many recent studies propose new deep learning (DL) models for improving CD performance, their comparative analyses are often restricted, lacking comprehensive insights into the proposed models' real-world generalizability, robustness, and performance trade-offs across diverse change characteristics. This paper presents a novel generic framework to systematically benchmark and assess DL-based CD models through three parallel pipelines: 1) cross-testing models on diverse benchmark datasets to evaluate generalization, 2) robustness analysis against different image corruptions, and 3) multi-faceted contour-level analytics evaluating detection sensitivity to change size/complexity. The framework is applied to comparatively evaluate five state-of-the-art DL-based CD models - Changeformer, BIT, Tiny, SNUNet, and CSA-CDGAN. Extensive experiments unveil each model's strengths, limitations and biases, highlighting their relative proficiencies in generalizing across data distributions, resilience to noise corruption, and discriminative capabilities for changes of varying characteristics. The proposed benchmarking framework demonstrates significant potential for guiding the selection of suitable CD models tailored to specific application requirements by comprehensively evaluating their generalizability, robustness, and detection capabilities across diverse real-world scenarios. This systematic evaluation approach can drive future research into developing more robust and versatile CD solutions aligned with practical needs.

A common challenge in real-world classification scenarios with sequentially appending target domain data is insufficient training datasets during the training phase. Therefore, conventional deep learning and transfer learning classifiers are not applicable especially when individual classes are not represented or are severely underrepresented at the outset. Domain Generalization approaches reach their limits when domain shifts become too large, making them occasionally unsuitable as well. In many (technical) domains, however, it is only the defect/ worn/ reject classes that are insufficiently represented, while the non-defect class is often available from the beginning. The proposed classification approach addresses such conditions and is based on a CNN encoder. Following a contrastive learning approach, it is trained with a modified triplet loss function using two datasets: Besides the non-defective target domain class (= 1st dataset), a state-of-the-art labeled source domain dataset that contains highly related classes (e.g., a related manufacturing error or wear defect) but originates from a (highly) different domain (e.g., different product, material, or appearance) (= 2nd dataset) is utilized. The approach learns the classification features from the source domain dataset while at the same time learning the differences between the source and the target domain in a single training step, aiming to transfer the relevant features to the target domain. The classifier becomes sensitive to the classification features and – by architecture – robust against the highly domain-specific context. The approach is benchmarked in a technical and a non-technical domain and shows convincing classification results. In particular, it is shown that the domain generalization capabilities and classification results are improved by the proposed architecture, allowing for larger domain shifts between source and target domains.

The escalating reliance on revolutionary online web services has introduced heightened security risks, with persistent challenges posed by phishing despite extensive security measures. Traditional phishing systems, reliant on machine learning and manual features, struggle with evolving tactics. Recent advances in deep learning offer promising avenues for tackling novel phishing challenges and malicious URLs. This paper introduces a two-phase stack generalized model named AntiPhishStack, designed to detect phishing sites. The model leverages the learning of URLs and character-level TF-IDF features symmetrically, enhancing its ability to combat emerging phishing threats. In Phase I, features are trained on a base machine learning classifier, employing K-fold cross-validation for robust mean prediction. Phase II employs a two-layered stacked-based LSTM network with five adaptive optimizers for dynamic compilation, ensuring premier prediction on these features. Additionally, the symmetrical predictions from both phases are optimized and integrated to train a meta-XGBoost classifier, contributing to a final robust prediction. This work's significance lies in advancing phishing detection with AntiPhishStack, operating without prior phishing-specific feature knowledge. Experimental validation on two benchmark datasets, comprising benign and phishing or malicious URLs, demonstrates the model's exceptional performance, achieving a notable 96.04% accuracy. This research adds value to the ongoing discourse on symmetry and asymmetry in information security and provides a forward-thinking solution for enhancing network security in the face of evolving cyber threats.

Lower Back Pain (LBP) is a disease that needs immediate attention. Person with back pain shall go immediately to doctor for treatment. Injury, excessive works and some medical conditions are result of back pain. Back pain is common to any age of human for different reasons. Due to factors such as previous occupation and degenerative disk disease the chance of developing lower back pain increases for older people. It hampers the working condition of people common reason for seeking medical treatment. The result is absence from work and is unable to normal due to pain. It creates uncomfortable and debilitating situations. Hence, detecting this disease at an early stage will assist the medical field experts to suggest counter measures to the patients. Detection of lower back pain is implemented in this paper by applying ensemble machine learning technique. This paper proposes Stacking ensemble classifier as an automated tool that will predict lower back pain tendency of a patient. Experimental result implies that the proposed method reaches an accuracy of 76.34%, f1-score of 0.76 and MSE of 0.34.

In theory and applications, it is often inevitable to work with projectors onto convex sets, where a linear transform is involved. In this article, a novel projector is presented, which generalizes previous results in that it admits a broader family of linear transforms, but on the other hand it is limited to box-type convex sets in the transformed domain. The new projector has an explicit formula and it can be interpreted within the framework of proximal optimization. The benefit of the new projector is demonstrated on an example from signal processing, where it was possible to speed up the convergence of a signal declipping algorithm by a factor of more than two.

Benchmarks for general language understanding have been rapidly developing in recent years of NLP research, particularly because of their utility in choosing strong-performing models for practical downstream applications. While benchmarks have been proposed in the legal language domain, virtually no such benchmarks exist for privacy policies despite their increasing importance in modern digital life. This could be explained by privacy policies falling under the legal language domain, but we find evidence to the contrary that motivates a separate benchmark for privacy policies. Consequently, we propose PrivacyGLUE as the first comprehensive benchmark of relevant and high-quality privacy tasks for measuring general language understanding in the privacy language domain. Furthermore, we release performances from multiple transformer language models and perform model-pair agreement analysis to detect tasks where models benefited from domain specialization. Our findings show the importance of in-domain pretraining for privacy policies. We believe PrivacyGLUE can accelerate NLP research and improve general language understanding for humans and AI algorithms in the privacy language domain, thus supporting the adoption and acceptance rates of solutions based on it.

The objective of this study was to develop guidelines for rainwater harvesting system sizing of shopping centres in South Africa. Three generalized dimensionless relationships relating rainwater supply and demand to tank size, yield and reliability were developed based on 101 years long daily time step simulations of rainwater harvesting of 19 shopping centres located in four regions. Daily rainfalls were obtained from nearby rainfall stations and daily non-potable demands were based on the size of the retail area. The simulations revealed within-year storage behaviour with considerable variation of the yield specified as the number of days for which demand was met each year. The Weibull plotting position formula was applied on the time series of yields to obtain yield-reliability relationships. Simulation results of the hydrologically optimum systems were used to develop two of the generalized relationships and an additional one based on the dependence of the slope of the reliability – yield plots on the optimum yield was formulated to enable analysis of hydrologically non-optimal systems. Most of the relationships fitted best to the non-linear power law form with high correlation coefficients averaging 0.92 and ranging from 0.82 to 1.00. The application of the models for tank sizing and assessing system performance is demonstrated.

The R23Cu7Mg4 (R = Ca, Eu) intermetallics, studied by single crystal X-ray diffraction, were found to be isostructural with the Yb23Cu7Mg4 prototype (hP68, k4h2fca, space group P63/mmc), forming a small group inside the bigger 23:7:4 family, otherwise adopting the hP68-Pr23Ir7Mg4 crystal structure. The observed structural peculiarity is connected with the divalent character of the R component and with a noticeable volume contraction, resulting in a clear clustering of title compounds inside the whole 23:7:4 family. The occurrence of fragments typical of similar compounds, particularly Cu-centered trigonal prisms and Mg-centered core-shell polyicosahedral clusters with R at vertices, induced the search of significant structural relationships. In this work, a description of the hexagonal crystal structure of the studied compounds is proposed as a linear intergrowth along the c-direction of the two types of slabs R10CuMg3 (parent type: hP28-kh2ca, SG 194) and R13Cu6Mg (parent type: hR60-b6a2, SG 160). The ratio of these slabs in the studied structure is 2:2 per unit cell, corresponding to the simple equation 2XR10CuMg3 + 2XR13Cu6Mg = 2XR23Cu7Mg4. This description assimilates the studied compounds to the (Ca,Eu,Yb)4CuMg ones, where the same slabs (of p3m1 layer symmetry) are stacked in a different way/ratio, and constitutes a further step towards a structural generalization of R-rich ternary intermetallics.

Purpose of this article is to introduce a modification of Phillips operators on the interval $\left[ \frac{1}{2},\infty \right) $ via Dunkl generalization. This type of modification enables a better error estimation on the interval $\left[ \frac{1}{2},\infty \right) $ rather than the classical Dunkl Phillips operators on $\left[ 0,\infty \right) $. We discuss the convergence results and obtain the degrees of approximations. Furthermore, we calculate the rate of convergence by means of modulus of continuity, Lipschitz type maximal functions, Peetre's $K$-functional and second order modulus of continuity.

Purpose of this article is to introduce a modification of Phillips operators on the interval $\left[ \frac{1}{2}% ,\infty \right) $ via Dunkl generalization. This type of modification enables a better error estimation on the interval $\left[ \frac{1}{2},\infty \right) $ rather than the classical Dunkl Phillips operators on $\left[ 0,\infty \right) $. We discuss the convergence results and obtain the degrees of approximations. Furthermore, we calculate the rate of convergence by means of modulus of continuity, Lipschitz type maximal functions, Peetre's $K$-functional and second order modulus of continuity.

Deep Neutral Networks (DNNs) were initially proposed towards the midst of the 20th century, motivated by the neural structure and mechanism of the human brain. Thanks to major ad-vancements in computational resources, during the past decade DNN-based systems have demonstrated unprecedented performance in terms of accuracy and speed. However, recent work has shown that such models may not be sufficiently robust during the inference process. Furthermore, due to the data-driven learning nature of DNNs, designing interpretable and gen-eralizable networks is a major challenge, especially in critical applications, such as medical Computer Aided Diagnostics (CAD) and other medical imaging tasks, including classification, regression and reconstruction. Within this context, a line of physics-driven approaches for deep learning has recently emerged, aimed at improving the stability and generalization capacity of DNNs for medical imaging applications. In this paper, we review recent work focused on phys-ics-driven or prior-information learning for a variety of imaging modalities and medical applica-tions. We discuss how the inclusion of domain-related knowledge into the learning process and networks’ design supports their stability and generalization capability. In addition, we highlight current and future challenges within this scope.

Bayesian formulations of learning imply that whenever the evidence for a correlation between events in an animal’s habitat is sufficient, the correlation is learned. This implies that regularities can be learnt rapidly, from small numbers of learning examples. This speed of learning gives maximum possible fitness, and no faster learning is possible. There is evidence in many domains that animals and people can learn at nearly Bayesian optimal speeds. These domains include associative conditioning, and the more complex domains of navigation and language. There are computational models of learning which learn at near-Bayesian speeds in complex domains, and which can scale well – to learn thousands of pieces of knowledge (i.e., relations and associations). These are not neural net models. They can be defined in computational terms, as algorithms and data structures at David Marr’s [1] Level Two. Their key data structures are composite feature structures, which are graphs of multiple linked nodes. This leads to the hypothesis that animal learning results not from deep neural nets (which typically require thousands of training exam-ples), but from neural implementations of the Level Two models of fast learning; and that neu-rons provide the facilities needed to implement those models at Marr’s Level Three. The required facilities include feature structures, dynamic binding, one-shot memory for many feature struc-tures, pattern-based associative retrieval, unification and generalization of feature structures. These may be supported by multiplexing of data and metadata in the same neural fibres.

Deep learning heavily relies on statistical correlations to drive artificial intelligence (AI) innovations, particularly in computer vision applications like autonomous driving and robotics. However, despite providing a solid foundation for deep learning, these statistical correlations can be vulnerable to unforeseen and uncontrolled factors. The lack of prior knowledge guidance can result in spurious correlations, introducing confounding factors and affecting the model's robustness. To address this challenge, recent research efforts have focused on integrating causal theory into deep learning methodologies. By modelling the inherent and unbiased causal structure, causal theory can potentially mitigate the impact of spurious correlations effectively. Hence, this paper explores the basics of causal methodologies in image classification.

One of the most rapidly advancing areas of deep learning research aims at creating models that learn to disentangle the latent factors of variation from a data distribution. However, modeling joint probability mass functions is usually prohibitive, which motivates the use of conditional models assuming that some information is given as input. In the domain of numerical cognition, deep learning architectures have successfully demonstrated that approximate numerosity representations can emerge in multi-layer networks that build latent representations of a set of images with a varying number of items. However, existing models have focused on tasks requiring to conditionally estimate numerosity information from a given image. Here we focus on a set of much more challenging tasks, which require to conditionally generate synthetic images containing a given number of items. We show that attention-based architectures operating at the pixel level can learn to produce well-formed images approximately containing a specific number of items, even when the target numerosity was not present in the training distribution.

In the paper, the authors present unified generalizations for the Bell numbers and polynomials, establish explicit formulas and inversion formulas for these generalizations in terms of the Stirling numbers of the first and second kinds with the help of the Faà di Bruno formula, properties of the Bell polynomials of the second kind, and the inversion theorem connected with the Stirling numbers of the first and second kinds, construct determinantal and product inequalities for these generalizations with aid of properties of the completely monotonic functions, and derive the logarithmic convexity for the sequence of these generalizations.