In this work, we investigate the ability of a data assimilation technique and space-borne observations to quantify and monitor changes in nitrogen oxides (NOx) emissions over North-Western Greece for the summers of 2018 and 2019. In this region, four lignite-burning power plants are located. The data assimilation technique, based on the Ensemble Kalman Filter method, is employed to combine space-borne atmospheric observations from the high spatial resolution Sentinel-5 Precursor (S5P) Tropospheric Monitoring Instrument (TROPOMI) and simulations using the LOTOS-EUROS Chemical Transport model. The Copernicus Atmosphere Monitoring Service-Regional European emissions (CAMS-REG, version 4.2) inventory based on year 2015 is used as the a priori in the simulations. Surface measurements of nitrogen dioxide (NO2) from air quality stations operating in the region are compared with the model surface NO2 output using either the a priori (base run) or the a posteriori (assimilated run) NOx emissions. The high biases found between the in situ NO2 measurements and the base run surface NO2 decrease in the assimilated run in most cases. The bias in the station near the largest power plant decreases to 2.0 μg/m3 (2.83 μg/m3) from 10.5 μg/m3 (8.46 μg/m3) in 2019 (2018 respectively). Concerning the estimated annual a posteriori NOx emissions it was found that, for the pixels hosting the two largest power plants, the assimilated run results in emissions decreased by ~40-50% for 2018 compared to 2015, whereas a larger decrease, of ~70% for both power plants, was found for 2019, after assimilating the space-born observations. For the same power plants, the European Pollutant Release and Transfer Register (E-PRTR) reports decreased emissions in 2018 and 2019 compared to 2015 (-35% and -38% in 2018, -62% and -72% in 2019), in good agreement with the estimated emissions. We further compare the a posteriori emissions to the reported energy production of the power plants during the summer of 2018 and 2019. Mean decreases of about -35% and-63% in NOx emissions are estimated for the two larger power plants in summer of 2018 and 2019, respectively, which are supported by similar decreases in the reported energy production of the power plants (~-30% and -70%, respectively).

Accurate and timely rainfall prediction enhances productivity and can aid proper planning in sectors such as agriculture, health, transport and water resources. This study is aimed at improving rainfall prediction using ensemble methods. It first assesses the performance of six convective schemes (Kain–Fritsch (KF); Betts–Miller–Janji´c (BMJ); Grell–Fretas (GF); Grell 3D ensemble (G3); New–Tiedke (NT) and Grell–Devenyi (GD)) using the root mean square error (RMSE) and mean error (ME) focusing on the March–May 2013 rainfall period over Uganda. 18 ensemble members are generated from the three best performing convective schemes (i.e. KF, GF & G3). The performance of three ensemble methods (i.e. ensemble mean (EM); ensemble mean analogue (EMA) and multi–member analogue ensemble (MAEM)) is also analyzed using the RMSE and ME. The EM presented a smaller RMSE compared to individual schemes (EM:10.02; KF:23.96; BMJ:26.04; GF:25.85; G3:24.07; NT:29.13 & GD:26.27) and a better bias (EM:-1.28; KF:-1.62; BMJ:-4.04; GF:-3.90; G3:-3.62; NT:-5.41 & GD:-4.07). The EMA and MAEM presented 13 out of 21 stations & 17 out of 21 stations respectively with smaller RMSE compared to EM thus demonstrating additional improvement in predictive performance. The MAEM is a new approach proposed and described in the study.

In this paper, we introduce deboost, a Python library devoted to weighted distance ensembling of predictions for regression and classification tasks. Its backbone resides on the scikit-learn library for default models and data preprocessing functions. It offers flexible choices of models for the ensemble as long as they contain the predict method, like the models available from scikit-learn. deboost is released under the MIT open-source license and can be downloaded from the Python Package Index (PyPI) at https://pypi.org/project/deboost. The source scripts are also available on a GitHub repository at https://github.com/weihao94/DEBoost.

We propose a new ensemble classification algorithm, named Super Random Subspace Ensemble (Super RaSE), to tackle the sparse classification problem. The proposed algorithm is motivated by the Random Subspace Ensemble algorithm (RaSE). The RaSE method was shown to be a flexible framework that can be coupled with any existing base classification. However, the success of RaSE largely depends on the proper choice of the base classifier, which is unfortunately unknown to us. In this work, we show that Super RaSE avoids the need to choose a base classifier by randomly sampling a collection of classifiers together with the subspace. As a result, Super RaSE is more flexible and robust than RaSE. In addition to the vanilla Super RaSE, we also develop the iterative Super RaSE, which adaptively changes the base classifier distribution as well as the subspace distribution. We show the Super RaSE algorithm and its iterative version perform competitively for a wide range of simulated datasets and two real data examples. The new Super RaSE algorithm and its iterative version are implemented in a new version of the R package RaSEn.

Bootstrap resampling techinques, introduced by Efron and Rubin, can be presented in a general Bayesian framework, approximating the statistical distribution of a statistical functional φ(F), where F is a random distribution function. Efron’s and Rubin’s bootstrap procedures can be extended introducing an informative prior through the Proper Bayesian bootstrap. In this paper different bootstrap techniques are used and compared in predictive classification and regression models based on ensemble approaches, i.e. bagging models involving decision trees. Proper Bayesian bootstrap, proposed by Muliere and Secchi, is used to sample the posterior distribution over trees, introducing prior distributions on the covariates and the target variable. The results obtained are compared with respect to other competitive procedures employing different bootstrap techniques. The empirical analysis reports the results obtained on simulated and real data.

The conventional machine learning (ML) algorithms are continuously advancing and evolving at a fast-paced by introducing the novel learning algorithms. ML models are continually improving using hybridization and ensemble techniques to empower computation, functionality, robustness, and accuracy aspects of modeling. Currently, numerous hybrid and ensemble ML models have been introduced. However, they have not been surveyed in a comprehensive manner. This paper presents the state of the art of novel ML models and their performance and application domains through a novel taxonomy.

More than 110 countries including 500 cities worldwide have set the goal of reaching carbon neutrality. Heating contributes to most of the residential energy consumption and carbon emissions. The green energy transition of fossil-based heating systems is needed to reach the emission goals. However, the heating systems vary in energy source, heating technology, equipment location, and these complexities make it challenging for households to compare heating systems and make decisions. Hence, a decision support tool that provides a generalized ranking of individual heating alternatives is proposed for households as decision-makers to identify the optimal choice. This paper presents an analysis of 13 heating alternatives and 19 quantitative criteria in technological, environmental, and financial aspects, combines ideal solution based Multi-Criteria Decision Making with 6 weighting methods and 4 normalization methods, and introduces ensemble learning with a fuzzy membership function derived from Cauchy distribution to finalize the final ranking. The robustness of the proposed method is verified by 3 sensitive analyses from different aspects. Air to water heat pump, solar heating and direct district heating are the top three rankings in the final result under Danish national average data. A framework is designed to guide the decision-makers apply this ranking guideline with their practical feasible situations.

While the use of crime data has been widely advocated in the literature, its availability is often limited to large urban cities and isolated databases tend not to allow for spatial comparisons. This paper presents an efficient machine learning framework capable of predicting spatial crime occurrences, without using past crime as a predictor, and at a relatively high resolution: the U.S. Census Block Group level. The proposed framework is based on an in-depth multidisciplinary literature review allowing the selection of 188 best-fit crime predictors from socio-economic, demographic, spatial, and environmental data. Such data are published periodically for the entire United States. The selection of the appropriate predictive model was made through a comparative study of different machine learning families of algorithms, including generalized linear models, deep learning, and ensemble learning. The gradient boosting model was found to yield the most accurate predictions for violent crimes, property crimes, motor vehicle thefts, vandalism, and the total count of crimes. Extensive experiments on real-world datasets of crimes reported in 11 U.S. cities demonstrated that the proposed framework achieves an accuracy of 73 and 77% when predicting property crimes and violent crimes, respectively.

Day-ahead forecasting of electricity prices is important in deregulated electricity markets for all the stakeholders: energy wholesalers, traders, retailers, and consumers. Electricity price forecasting is an inherently difficult problem due to its special characteristic of dynamicity and non-stationarity. In this paper, we present a robust price forecasting mechanism that shows resilience towards aggregate demand response effect and provides highly accurate forecasted electricity prices to the stakeholders in a dynamic environment. We employ an ensemble prediction model in which a group of different algorithms participate in predicting the price for each hour of a day. We propose two different strategies, namely, Fixed Weight Method (FWM) and Varying Weight Method (VWM), for selecting each hour's expert algorithm from the set of participating algorithms. In addition, we utilize a carefully engineered set of features selected from a pool of features derived from information such as past electricity price data, weather data, and calendar data. The proposed ensemble model offers better results than both the Pattern Sequence-based Forecasting (PSF) method and our own previous work using Artificial Neural Networks (ANN) alone do on the datasets for New York, Australian, and Spanish electricity markets.

Covid-19 has caused infections and deaths worldwide. While research in the field of Data Science has contributed good predictions of positive Covid-19 case numbers, this study's review of literature shows there is little research in the use of variants of the virus in predictions. We set out to define and evaluate novel variant features. We find that features relating to variant trends, thresholds and amino acid substitutions are especially powerful in two tasks. In the first task, predicting Covid-19 case numbers, accuracy improved from 71.53% without variant features to 82.12% with variant features. In the second task, predicting transmission severity of variants between two classes, we created a method to build some variable ensembles through selecting appropriate models that are generated with variant features. The test results showed that our ensembles are more accurate and reliable. One particular ensemble of 14 models correctly classified 90.91% of variants, outperforming other models including the popular Random Forest ensemble. In addition, as the variant features have represented more underlying information about Covid-19 pathophysiology, our ensemble methods use only a few data samples to achieve an accurate prediction. The ensemble of 14 models uses only 50 cases of each variant, an ability that could be exploited for early detection of highly infectious variants. These research findings may benefit public health professionals, policy makers, and the research community in the collective efforts to overcome this disease.

Convolutional Neural Networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem by generating new data points providing additional information. In this paper, we investigate the performance of more than ten different sets of data augmentation methods, with two novel approaches proposed here: one based on the Discrete Wavelet Transform and the other on the Constant-Q Gabor transform. Pretrained ResNet50 networks are finetuned on each augmentation method. Combinations of these networks are evaluated and compared across three benchmark data sets of images representing diverse problems and collected by instruments that capture information at different scales: a virus data set, a bark data set, and a LIGO glitches data set. Experiments demonstrate the superiority of this approach. The best ensemble proposed in this work achieves state-of-the-art performance across all three data sets. This result shows that varying data augmentation is a feasible way for building an ensemble of classifiers for image classification (code available at https://github.com/LorisNanni).

The novel coronavirus disease (COVID-19) has created immense threats to public health on various levels around the globe. The unpredictable outbreak of this disease and the pandemic situation are causing severe depression, anxiety and other mental as physical health related problems among the human beings. To combat against this disease, vaccination is essential as it will boost the immune system of human beings while being in the contact with the infected people. The vaccination process is thus necessary to confront the outbreak of COVID-19. This deadly disease has put social, economic condition of the entire world into an enormous challenge. The worldwide vaccination progress should be tracked to identify how fast the entire economic as well as social life will be stabilized. The monitor ofthe vaccination progress, a machine learning based Regressor model is approached in this study. This tracking process has been applied on the data starting from 14^th December, 2020 to 24^th April, 2021. A couple of ensemble based machine learning Regressor models such as Random Forest, Extra Trees, Gradient Boosting, AdaBoost and Extreme Gradient Boosting are implemented and their predictive performance are compared. The comparative study reveals that the AdaBoostRegressor outperforms with minimized mean absolute error (MAE) of 9.968 and root mean squared error (RMSE) of 11.133.

Recent studies on seafloor mapping have presented different modelling methods for the automatic classification of seafloor sediments. However, most of these studies have applied these models to seafloor data with appropriate number of ground-truth samples, which raises the question whether these methods are applicable to studies with smaller numbers of ground-truth data. In this study, we aim to address this issue by conducting sediment class-specific predictions using ensemble modelling to map areas with limited or without ground-truth data and combined with hydro-acoustic datasets. The resulting class-specific maps were then assembled into one map, where the most probable class was assigned to the appropriate location. Our approach was able to predict sediment classes without bias to the class with more ground-truth data and produced reliable seafloor sediment distributions maps that can be used for seafloor monitoring. Sediment shifts of a heterogenous seafloor in the Sylt Outer Reef, German North Sea were also assessed to understand the sediment dynamics in the area. The analyses of sediment shifts showed that the western area of the Sylt Outer Reef is highly active, and the results of the analyses assisted in providing recommendations on future seafloor monitoring activities.

Climate change is expected to increase the occurrence of droughts with the hydrology in alpine systems being largely determined by snow dynamics. In this paper we propose a methodology to assess the impact of climate change on both meteorological and hydrological droughts taking into account the dynamics of the snow cover area (SCA). We will also analyse the correlation between these types of droughts. We have generated ensembles of local climate scenarios based on regional climate models (RCMs) representative of potential future conditions. We have considered several sources of uncertainty: different historical climate databases, simulations obtained with several RCMs, and some statistical downscaling techniques. We then used a stochastic weather generator (SWG) to generate multiple climatic series preserving the characteristics of the ensemble scenario. These were simulated within a cellular automata (CA) model to generate multiple SCA future series. They were used to calculate multiple series of meteorological drought indices, the Standardized Precipitation Index (SPI) and Standardized Precipitation Evapotranspiration Index (SPEI)) and a novel hydrological drought index (Standardized Snow Cover Index (SSCI)). A linear correlation analysis was applied to both types of drought to analyse how they propagate and the time delay between them. We applied the proposed methodology to the Sierra Nevada (southern Spain) where we estimated a general increase in meteorological and hydrological drought magnitude and duration for the horizon 2071-2100 under the RCP 8.5 emission scenario. The SCA droughts also revealed a significant increase in drought intensity. The meteorological drought propagation to SCA droughts is reflected in an immediate or short time (1 month), obtaining significant correlations in lower accumulation periods of drought indices (3 and 6 months). This will allow us to obtain information about meteorological drought from SCA deficits and vice versa.

Capability assessment plays a crucial role in the demonstration and construction of equipment. To improve the accuracy and stability of capability assessment, we study the neural network learning algorithms in the field of capability assessment and index sensitivity. Aiming at the problem of over-fitting and parameter optimization in neural network learning, the paper proposes an improved machine learning algorithm—the Ensemble Learning Based on Policy Optimization Neural Networks (ELPONN) algorithm with the policy optimization and ensemble learning. This algorithm presents optimized neural network learning algorithm through different strategies evolution, and builds an ensemble learning model of multi-intelligent algorithms to assessment the capability and analyze the sensitivity of the indexes. Through the assessment of capabilities, the algorithm effectively avoids parameter optimization from entering the minimum point in performance to improve the accuracy of equipment capability assessment, which is significantly better than previous neural network assessment methods. The experimental results show that the mean relative error is 4.10%, which is better than BP, GABP, and early stopping. The ELPONN algorithm has better accuracy and stability performance, and meets the requirements of capability assessment.

In this paper, we propose methods for brain tumor detection in MRI images based on ensemble learning. We build upon prior research on ensemble methods by testing the concatenation of pre-trained models: features extracted via transfer learning are merged and segmented by classification algorithms or a stacked ensemble of those algorithms. The proposed approach achieved accuracy scores of 0.98 , outperforming a benchmark VGG-16 model. Considerations to granular computing are given in the paper as well.

Melanoma is considered the most serious and aggressive type of skin cancer, and metastasis appears to be the most important factor in prognosticating this type of cancer. With the emergence of new therapeutic strategies for metastatic melanoma that have shown improvement in patient survival, we developed a transfer learning-based biomarker discovery model that could help in the diagnosis and prognosis of this disease. After applying it to the ensemble machine learning model, results reveal that the genes we found show consistency with other methodologies previously applied to the same TCGA (The Cancer Genome Atlas) data set, and our methods found novel biomarker genes as well. Our ensemble model achieved Area Under the Receiver Operating Characteristic (AUC) of 0.9861, an accuracy of 91.05, and an F1 score of 90.60 using an independent validation data set. This study was able to identify potential genes for diagnostic classification (C7 and GRIK5) and diagnostic and prognostic biomarkers (S100A7, S100A7, KRT14, KRT17, KRT6B, KRTDAP, SERPINB4, TSHR, PVRL4, WFDC5, IL20RB). We also assessed the potential sources of bias for our model and confirmed some of them by the model's performance.

Scattering hyperspectral technology is a nondestructive testing method with many advantages. Here, we propose a method to improve the accuracy of egg freshness, research the influence of incident angles of light source on the accuracy and explain its mechanism. A variety of weak classifiers classify eggs based on the spectra after preprocessing and feature wavelength extraction to obtain three classifiers with the highest accuracy. The three classifiers are used as metamodels of stacking ensemble learning to improve the highest accuracy from 96.25% to 100%. Moreover, the highest accuracy of scattering, reflection, transmission and mixed hyperspectral of eggs are 100.00%, 88.75%, 95.00% and 96.25%, respectively, indicating that the scattering hyperspectral for egg freshness detection is better than that of the others. In addition, the accuracy is inversely proportional to the angle of incidence due that the smaller the incident angle, the camera collects a larger proportion of scattering light, which contains more biochemical parameters of an egg than that of reflection and transmission. These results are very important for improving the accuracy of non-destructive testing and selecting the incident angle of the light source, and have potential applications in online non-destructive testing.

To improve the monitoring of the electrical power grid it is necessary to evaluate the influence of contamination in relation to leakage current and its progression to a disruptive discharge. In this paper, insulators were tested in a saline chamber to simulate the increase of salt contamination on their surface, and to evaluate the supportability of these components. From the time series forecasting of the leakage current, it is possible to evaluate the development of the fault before a flashover occurs. Choosing which method to use is always a difficult task since some models may have higher computational effort. In this paper, for a complete evaluation, the long short-term memory (LSTM), group method of data handling (GMDH), adaptive neuro-fuzzy inference system (ANFIS), bootstrap aggregation (bagging), sequential learning (boosting), random subspace, and stacked generalization (stacking) ensemble learning models are analyzed. A review and comparison of these well-established methods for time series forecasting is performed. From the results of the best structure of the model, the hyperparameters are evaluated and the Wavelet transform is used to obtain an enhanced model.

In this paper, we examine two strategies for boosting the performance of ensembles of Siamese networks (SNNs) for image classification using two loss functions (Triplet and Binary Cross Entropy) and two methods for building the dissimilarity spaces (FULLY and DEEPER). With FULLY, the distance between a pattern and a prototype is calculated by comparing two images using the fully connected layer of the Siamese network. With DEEPER, each pattern is described using a deeper layer combined with dimensionality reduction. The basic design of the SNNs takes advantage of supervised k-means clustering for building the dissimilarity spaces that train a set of support vector machines, which are then combined by sum rule for a final decision. The robustness and versatility of this approach are demonstrated on several cross-domain image data sets, including a portrait data set, two bioimage and two animal vocalization data sets. Results show that the strategies employed in this work to increase the performance of dissimilarity image classification using SNN is closing the gap with standalone CNNs. Moreover, when our best system is combined with an ensemble of CNNs, the resulting performance is superior to an ensemble of CNNs, demonstrating that our new strategy is extracting additional information.

This paper presents a comparison of the predictive capability of three hydrological models in a heavily influenced catchment in Peninsula India. In catchments where there is a high dependence on both streamflow and groundwater to meet demands, it is of importance to capture the catchment processes correctly. This study highlights the performance evaluation of a multi-model ensemble consisting of GWAVA (Global Water AVailability Assessment) model, SWAT (Soil Water Assessment Tool) and VIC (Variable Infiltration Capacity) model for comparative purposes and the key catchment hydrological processes. The three models were compared in several sub-catchments in the upstream reaches of the Cauvery river catchment. Model performances for monthly streamflow simulations from 1983 – 2005 were analysed for five catchments in the Upper Cauvery. The analysis was undertaken using Nash- Sutcliffe Efficiency, Kling- Gupta Efficiency and percent bias. Additionally, a mean ensemble is presented. The application of a multi-model ensemble approach can be useful in overcoming the uncertainties associated with individual models. The ensemble mean has a better predictive ability in catchments with reservoirs than the individual models. Utilising multiple models could be a suitable methodology to offset uncertainty in input data and poor reservoir operation functionality within individual models. This study has highlighted the importance of an accurate spatial representation of precipitation for input into hydrological models and comprehensive reservoir functionality is paramount to obtaining good results in this region.

There are uncertainties associated with every phase of the stock assessment process, ranging from the collection of data, assessment model choice, model assumptions and interpretation of risk to the implementation of management advice. The dynamics of fish populations are complex, and our incomplete understanding of those dynamics (and limited observations of important mechanisms) necessitate that models are simpler than nature. The aim is for the model to capture enough of the dynamics to accurately estimate trends and abundance and to provide advice to managers about sustainable harvests. The \textit{status quo} approach to assessment modelling has been to identify the `best' model, based on diagnostics and model selection criteria, and to generate advice from that model, mostly ignoring advice from other model configurations regardless of how closely they performed relative to the chosen model. We review the suitability of the ensemble modelling paradigm to more fully capture uncertainty in stock assessment model building and the provision of advice. We recommend further research to evaluate potential gains in modelling performance and advice from the use of ensemble modelling, while also suggesting revisions to the formal process for reviewing models and providing advice to management bodies.

Lower Back Pain (LBP) is a disease that needs immediate attention. Person with back pain shall go immediately to doctor for treatment. Injury, excessive works and some medical conditions are result of back pain. Back pain is common to any age of human for different reasons. Due to factors such as previous occupation and degenerative disk disease the chance of developing lower back pain increases for older people. It hampers the working condition of people common reason for seeking medical treatment. The result is absence from work and is unable to normal due to pain. It creates uncomfortable and debilitating situations. Hence, detecting this disease at an early stage will assist the medical field experts to suggest counter measures to the patients. Detection of lower back pain is implemented in this paper by applying ensemble machine learning technique. This paper proposes Stacking ensemble classifier as an automated tool that will predict lower back pain tendency of a patient. Experimental result implies that the proposed method reaches an accuracy of 76.34%, f1-score of 0.76 and MSE of 0.34.

In this study an adjusting post-processing approach is implemented for improving intra-hourly forecasts of solar power and ramp events of PV solar power systems at different locations in the United States. This study also serves as an out-of-sample test to evaluate the performance of the adjusting approach with different locations and timescales. Thus, various individual intra-hourly forecasts of solar power are combined and adjusted by applying the adjusting approach. Both point and probabilistic forecasts of solar power are included. After that, solar power ramp event forecasting by the adjusting approach is carried out.

This study proposes a novel hybrid machine learning approach for modeling of rainfall-induced shallow landslides. The proposed approach is a combination of an instance-based learning algorithm (k-NN) and Rotation Forest (RF), state of the art machine techniques that have seldom explored for landslide modeling. The Lang Son city area (Vietnam) is selected as a case study. For this purpose, a spatial database for the study area was constructed, and then, was used to build and evaluate the hybrid model. Performance of the model was assessed using Receiver Operating Characteristic (ROC), area under the ROC curve (AUC), success rate and prediction rate, and several statistical evaluation metrics. The results showed that the model has high performance with both the training data (AUC = 0.948) and the validation data (AUC = 0.848). The results were compared with those obtained from soft computing techniques i.e. Random Forest, J48 Decision Trees, and Multilayer Perceptron Neural Networks. Overall, the performance of the proposed model is better than those obtained from the above methods. Therefore, the proposed model is a promising tool for landslide modeling. The research result can be highly useful for land use planning and management in landslide prone areas.

InSAR and associated analytic methods enable relative surface deformation measurements from low Earth orbit with a potential accuracy of centimeters to millimeters. However, assessing the actual accuracy can be quite difficult. The analytic methods are complicated enough that naive analytic error propagation is infeasible, and, in many settings, InSAR practitioners lack sufficient ground truth to assess results. Phase noise due to partial decorrelation from changes in the scattering properties of the ground is a prominent source of accuracy loss. In this paper we present a method to assess the loss of precision due to this component of phase noise. The proposed method consists of generating synthetic data stacks whose statistical properties match those of the actual input SAR data stacks, and then using the synthetic data for an ensemble calculation. The spread of the results of the ensemble calculation indicates the loss of precision. We show examples of the ensemble analysis at a mining operation in South Africa, and demonstrate the ability to estimate the precision of two InSAR deformation retrieval methods on a point-by-point and epoch-by-epoch basis.

Ultrasound (US) is commonly used for the diagnosis of liver masses. Ensemble learning has been widely used for image classification, but its methods have not been fully optimized. This study was performed to investigate the usefulness of ensemble learning and compare a number of ensemble learning techniques using multiple convolutional neural network (CNN)-trained models for image classification of liver masses in US images. The US imaging data set was classified into four categories: benign liver tumor (BLT, 6320 images), liver cyst (LCY, 2320 images), metastatic liver cancer (MLC, 9720 images), and primary liver cancer (PLC, 7840 images). In this study, 250 test images were randomly selected for each class, for a total of 1000 images, and the remaining images were used for training. Sixteen different CNNs were used for to train and test the US images. All four types of ensemble learning—soft voting (SV), stacking (ST), weighted average voting (WAV), and weighted hard voting (WHV)—showed greater accuracy than the single CNN. All four types also showed significantly better deep learning performance than ResNeXt101 alone. For image classification of liver masses using US images, ensemble learning improved the performance of deep learning over a single CNN.

Ransomware is one of the most dangerous types of malware, which is frequently intended to spread through a network to damage the designated client by encrypting the client’s vulnerable data. Conventional signature-based ransomware detection technique falls behind because it can only detect known anomalies. When it comes to new and non-familiar ransomware traditional system unveils huge shortcomings. For detecting unknown patterns and sorts of new ransomware families,behavior-based anomaly detection approaches are likely to be the most efficient approach. In the wake of this alarming condition, this paper presents an ensemble classification model consisting of three widely used machine learning techniques that include Decision Tree (DT), RandomForest (RF), and K-nearest neighbor (KNN). To achieve the best outcome ensemble soft voting and hard voting techniques are used while classifying ransomware families based on attack attributes. Performance analysis is done by comparing our proposed ensemble models with standalone models on behavioral attributes based ransomware dataset..

Tephra plumes can cause a significant hazard for surrounding towns, infrastructure, and air traffic. The current work presents the use of a small and compact X-band Multi-Parameter (X-MP) radar for the remote tephra detection and tracking of two eruptive events at Merapi Volcano, Indonesia, in May and June 2018. Tephra detection was done by analysing the multiple parameters of radar: copolar correlation and reflectivity intensity. These parameters were used to cancel unwanted clutter and retrieve tephra properties, which are grain size and concentration. Real-time spatial and temporal forecasting of tephra dispersal was performed by applying an advection scheme (nowcasting) in the manner of Ensemble Prediction System (EPS). Cross-validation was done using field-survey data, radar observations, and Himawari-8 imagery. The nowcasting model computed both the displacement and growth and decaying rate of the plume based on the temporal changes in two-dimensional movement and tephra concentration, respectively. Our results with ground-based data, where the radar-based estimated grain size distribution fell within the range of in-situ data. The uncertainty of real-time forecasted tephra plume depends on the initial condition, which affects the growth-and decaying rate estimation. The EPS improves the predictability rate by reducing the number of missed and false forecasted events. Our findings and the method presented here are suitable for early warning of tephra fall hazard at the local scale.

This paper provides the state of the art of data science in economics. Through a novel taxonomy of applications and methods advances in data science are investigated. The data science advances are investigated in three individual classes of deep learning models, ensemble models, and hybrid models. Application domains include stock market, marketing, E-commerce, corporate banking, and cryptocurrency. Prisma method, a systematic literature review methodology is used to ensure the quality of the survey. The findings revealed that the trends are on advancement of hybrid models as more than 51% of the reviewed articles applied hybrid model. On the other hand, it is found that based on the RMSE accuracy metric, hybrid models had higher prediction accuracy than other algorithms. While it is expected the trends go toward the advancements of deep learning models.

Brain Tumor is one of the severe diseases and occurrence of this disease threats human life. Detection of brain tumor in advance can secure patient’s life from unwanted loss. Well-timed and swift disease detection and treatment strategy can lead to improved quality of life in these patients. This paper attempts to use Machine Learning based ensemble approaches for recognising patients with brain tumor. Ensemble technique based AdaBoost classifier and 10-fold stratified cross-validation method are assembled in single platform is proposed in this paper for prediction of brain tumor. This prediction is compared against three baseline classifiers such as Gradient Boost, Random Forest and Extra Trees classifier. Experimental result implies the superiority of this model with an accuracy of 98.97%, f1-score of 0.99, kappa statistics score of 0.95 and MSE of 0.0103.

Nowadays, we live in a world in which people are facing with a lot of data that should be stored or displayed. One of the key methods to control and manage this data refers to grouping and classifying them in clusters. Today, clustering has a critical role in information retrieval methods for organizing large collections inside a few significant clusters. One of the main motivations for the use of clustering is to determine and reveal the hidden and inherent structure of a set of data. Ensemble clustering algorithms combine multiple clustering algorithms to finally reach an overall clustering system. Ensemble clustering methods by lack of information fusing utilize several primary partitions of data to find better ways. Since various clustering algorithms look at the different data points, they can produce various partitions from such data. It is possible to create a partition with high performance by combining the partitions obtained from different algorithms, even if the clusters to be very dense from each other. Most studies in this area have examined all the initial clusters. In this study, a new method is used in which the most sustainable clusters are utilized instead of all primary produced clusters. Consensus function based on co-association matrixes used to select more stable clusters. The most stable clusters selection method is done by cluster stability criterion based on F-measure. Optimization functions are used to optimize the obtained final clusters. The genetic algorithm is the optimizer used in this article to find the ultimate clusters participated in a consensus. Experimental results on several datasets show that the output of proposed method is various clusters with high stability.

We investigate the stationary and dynamic properties of the celebrated Nosé-Hoover dynamics of many-body interacting Hamiltonian systems, with an emphasis on the effect of inter-particle interactions. To this end, we consider a model system with both short- and long-range interactions. The Nosé-Hoover dynamics aims to generate the canonical equilibrium distribution of a system at a desired temperature by employing a set of time-reversible, deterministic equations of motion. A signature of canonical equilibrium is a single-particle momentum distribution that is Gaussian. We find that the equilibrium properties of the system within the Nosé-Hoover dynamics coincides with that within the canonical ensemble. Moreover, starting from out-of-equilibrium initial conditions, the average kinetic energy of the system relaxes to its target value over a size-independent timescale. However, quite surprisingly, our results indicate that under the same conditions and with only long-range interactions present in the system, the momentum distribution relaxes to its Gaussian form in equilibrium over a scale that diverges with the system size. On adding short-range interactions, the relaxation is found to occur over a timescale that has a much weaker dependence on system size. This system-size dependence of the timescale vanishes when only short-range interactions are present in the system. An implication of such an ultra-slow relaxation when only long-range interactions are present in the system is that macroscopic observables other than the average kinetic energy when estimated in the Nosé-Hoover dynamics may take an unusually long time to relax to its canonical equilibrium value. Our work underlines the crucial role that interactions play in deciding the equivalence between Nosé-Hoover and canonical equilibrium.

For the soil in different regions, the nutrient fertility contained in it is different, and the detection and zoning management of soil nutrients before tillage every year can improve grain yield. In this paper, an integrated learning strategy model based on black soil hyperspectral data is designed for rapid classification of organic matter content classification of black soil. Soil hyperspectral image dataset of Xiangyang Experimental Base was collected; by changing the internal structure of the stacking model, an LSVM-stacking model with (MLP, SVC, DTree, XGBl, kNN) five classifiers as the L1 layer was built, and the simulated annealing algorithm was used for hyperparameter optimization. Compared to other stacking models, the LSVM-stacking metrics are significantly improved. The accuracy rate of hyperparameter optimization is improved by 38.6515%, the accuracy rate of the independent test data set is 0.9488, and the comparison of individual learners can improve the recognition classification accuracy of label"1" to 1.0.

An electrocardiograph (ECG) reflects the health of the human heart and is used to help diagnose arrhythmia and myocardial infarction(MI) in clinical practice. Early diagnosis of arrhythmia helps implement preventive measures and plays a crucial role in saving a patient's life. With the increasing demand of clinicians for ECG analysis technology, intelligent detection and diagnosis of ECG signals has become a more efficient means to assist physicians in diagnosing cardiovascular diseases. This paper introduces an ECG diagnosis approach based on an ensemble deep learning combination of CNN(convolutional neural network) and SLAP(stacked-long short term memory architecture for prediction) architecture. ECG data is denoised and further divided into single heartbeats to achieve data standardization and sample diversity. Adam optimizer and BCEwithlogitsloss multi-classification loss function were used to enhance the model effect, and the system achieved the classification effect of 99.3% average accuracy, 99.0% F1-value, and 99.2% sensitivity in MIT-BIH standard database classification. It also shows good generalization ability on the Tianchi data set.

In recent years, the field of deep learning achieved considerable success in pattern recognition, image segmentation and may other classification fields. There are a lot of studies and practical applications of deep learning on images, video or text classification. In this study, we suggest a method for changing the architecture of the most performing CNN models with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLu layer) by a different activation function stochastically drawn from a set of activation functions: in this way the resulting CNN has a different set of activation function layers.

This study shows the application of self-organizing maps (SOMs) to probabilistic forecasts of wind power generation and ramps in Japan. SOMs are applied to atmospheric variables obtained from atmospheric reanalysis over the region, thus deriving classified weather patterns (WPs). Probabilistic relationships are established between the synoptic-scale atmospheric variables over East Japan and the generation of regionally integrated wind power in East Japan. Medium-range probabilistic wind power predictions are derived by SOM, as analog ensembles based on the WPs of the multi-center ensemble forecasts. As this analog approach handles stochastic uncertainties effectively, probabilistic wind power forecasts are rapidly generated from a very large number of forecast ensembles. The use of a multi-model ensemble provides better results than a one-forecast model. The hybrid ensemble forecasts further improve the probabilistic predictability skill of wind power generation, as compared with non-hybrid methods. It is expected that long-term wind forecasts will provide better guidance to transmission grid operators. The advantage of this method is that it can include an interpretative analysis of meteorological factors for variations in renewable energy.

Real-world optimization problems are often governed by one or more constraints. Over the last few decades, extensive research has been performed in Constrained Optimization Problems (COPs) fueled by advances in computational intelligence. In particular, Evolutionary Algorithms (EAs) are a preferred tool for practitioners for solving these COPs within practicable time limits. We propose an ensemble of multi- method hybrid EA framework with four mutation operators, two crossover operators, multi-search [Differential Evolution (DE) & Gaining Sharing Knowledge (GSK)] optimization algorithm, and ensemble of constraint handling techniques to solve global real- world constrained optimization problem. The proposed frame- work FEPEA has an ascendancy of multiple adaptation strategies concerning the control parameters, search mechanisms, two sub-populations as well as uses knowledge sharing mechanism between junior and senior phases. The algorithm also combines the power of four popular constraint handling techniques (CHT) and uses a voting mechanism to select any particular CHT. On top of that, this algorithm also uses both linear and non- linear population size reduction in every step of the evolutionary process. We test our method on 57 real-world problems provided as part of the CEC 2020 special session & competition on real- world constrained optimization benchmark suite. Experimental results indicate that FEPEA is able to achieve state-of-the- art performance on real-world constrained global optimization when compared against other well-known real-world constrained optimizers.

The classifier system proposed in this work combines the dissimilarity spaces produced by a set of Siamese neural networks (SNNs) designed using 4 different backbones, with different clustering techniques for training SVMs for automated animal audio classification. The system is evaluated on two animal audio datasets: one for cat and another for bird vocalizations. Different clustering methods reduce the spectrograms in the dataset to a set of centroids that generate (in both a supervised and unsupervised fashion) the dissimilarity space through the Siamese networks. In addition to feeding the SNNs with spectrograms, additional experiments process the spectrograms using the Heterogeneous Auto-Similarities of Characteristics. Once the similarity spaces are computed, a vector space representation of each pattern is generated that is then trained on a Support Vector Machine (SVM) to classify a spectrogram by its dissimilarity vector. Results demonstrate that the proposed approach performs competitively (without ad-hoc optimization of the clustering methods) on both animal vocalization datasets. To further demonstrate the power of the proposed system, the best stand-alone approach is also evaluated on the challenging Dataset for Environmental Sound Classification (ESC50) dataset. The MATLAB code used in this study is available at https://github.com/LorisNanni.

Protein phosphorylation and ADP-ribosylation (ADPr), as two types of post-translational modifications (PTM), are the process of adding phosphate group and ADP-ribose moieties to proteins, respectively. Although both PTM types can occur on many amino acid types, serine is the most common. Serine phosphorylation (pS), serine ADPr (SADPr), and their in situ crosstalks (pSADPr) play essential roles in biological processes. Although in silico classifiers have been developed for predicting pS and SADPr sites, the classifier for predicting pSADPr sites is unavailable. In this study, we developed classifiers to predict pSADPr sites. Specifically, we collected 3250 human pSADPr, 7520 SADPr, 151,227 pS and 80,096 unmodified serine sites. Based on them, we investigated the characteristics of pSADPr sites and constructed three classifiers to predict pSADPr sites from the pS dataset, the SADPr dataset and the protein sequences separately. We built and evaluated five deep-learning classifiers in ten-fold cross-validation and independent test datasets. Three of them (e.g. Convolutional Neural Network with the One-Hot encoding, dubbed CNNOH) performed better than the rest two. For instance, CNNOH had the AUC values of 0.700, 0.914 and 0.954 for recognizing pSADPr sites from the SADPr, pS and unmodified serine sites.Therefore, it is challenging to distinguish pSADPr sites from SADPr sites compared to the other two. It is consistent with our observation that pSADPr's characteristics are more similar to those of SADPr than the rest. Furthermore, we used the classifiers as base classifiers to develop a few stacking-based ensemble classifiers to improve performance. However, none of the ensemble classifiers showed better performances, suggesting that the base classifiers have good enough performances. Finally, we developed an online tool for extensively predicting human pSADPr sites based on the CNNOH classifier, dubbed EdeepSADPr. It is freely available through http://edeepsadpr.bioinfogo.org/.

Although automated Acute Lymphoblastic Leukemia (ALL) detection is essential, it is challenging due to the morphological correlation between malignant and normal cells. The traditional ALL classification strategy is arduous, time-consuming, often suffers inter-observer variations, and necessitates experienced pathologists. This article has automated the ALL detection task, employing deep Convolutional Neural Networks (CNNs). We explore the weighted ensemble of deep CNNs to recommend a better ALL cell classifier. The weights are estimated from ensemble candidates' corresponding metrics, such as accuracy, F1-score, AUC, and kappa values. Various data augmentations and pre-processing are incorporated for achieving a better generalization of the network. We train and evaluate the proposed model utilizing the publicly available C-NMC-2019 ALL dataset. Our proposed weighted ensemble model has outputted a weighted F1-score of 88.6%, a balanced accuracy of 86.2%, and an AUC of 0.941 in the preliminary test set. The qualitative results displaying the gradient class activation maps confirm that the introduced model has a concentrated learned region. In contrast, the ensemble candidate models, such as Xception, VGG-16, DenseNet-121, MobileNet, and InceptionResNet-V2, separately produce coarse and scatter learned areas for most example cases. Since the proposed ensemble yields a better result for the aimed task, it can experiment in other domains of medical diagnostic applications.

One of the most basic pieces of information gained from dynamic electromyography is accurately defining muscle action and phase timing within the gait cycle. The human gait relies on selective timing and the intensity of appropriate muscle activations for stability, loading, and progression over the supporting foot during stance, and further to advance the limb in the swing phase. A common clinical practice is utilizing a low-pass filter to denoise integrated electromyogram (EMG) signals and to determine onset and cessation events using a predefined threshold. However, the accuracy of the defining period of significant muscle activations via EMG varies with the temporal shift involved in filtering the signals; thus, the low-pass filtering method with a fixed order and cut-off frequency will introduce a time delay depending on the frequency of the signal. In order to precisely identify muscle activation and to determine the onset and cessation times of the muscles, we have explored here onset and cessation epochs with denoised EMG signals using different filter banks: the wavelet method, empirical mode decomposition (EMD) method, and ensemble empirical mode decomposition (EEMD) method. In this study, gastrocnemius muscle onset and cessation were determined in sixteen participants within two different age groups and under two different walking conditions. Low-pass filtering of integrated EMG (iEMG) signals resulted in premature onset (28% stance duration) in younger and delayed onset (38% stance duration) in older participants, showing the time-delay problem involved in this filtering method. Comparatively, the wavelet denoising approach detected onset for normal walking events most precisely, whereas the EEMD method showed the smallest onset deviation. In addition, EEMD denoised signals could further detect pre-activation onsets during a fast walking condition. A comprehensive comparison is discussed on denoising EMG signals using EMD, EEMD, and wavelet denoising in order to accurately define an onset of muscle under different walking conditions.

Users of meteorological forecasts are often faced with the question of whether to make a decision now based on the current forecast or whether to wait for the next and hopefully more accurate forecast before making the decision. One would imagine that the answer to this question should depend on the extent to which there is a benefit in making the decision now rather than later, combined with an understanding of how the skill of the forecast improves, and information about the possible size and nature of forecast changes. We extend the well-known cost-loss model for forecast-based decision making to capture an idealized version of this situation. We find that within this extended cost-loss model, the question of whether to decide now or wait depends on two specific aspects of the forecast, both of which involve probabilities of probabilities. For the special case of weather and climate forecasts in the form of normal distributions we derive a simulation algorithm, and equivalent analytical expressions, for calculating these two probabilities. We apply the algorithm to forecasts of temperature and find that the algorithm leads to better decisions relative to three simpler alternative decision-making schemes. Similar problems have been studied in many other fields, and we explore some of the connections.

The current study aims to apply and compare the performance of six machine learning algorithms, including three basic classifiers: random forest (RF), gradient boosting decision tree (GBDT), and extreme gradient boosting (XGB), as well as their hybrid classifiers, using the logistic regression (LR) method (RF+LR, GBDT+LR, and XGB+LR), in order to map the landslide susceptibility of Zhangjiajie City, Hunan Province, China. First, a landslide inventory map was created with 206 historical landslide points and 412 non-landslide points, which was randomly divided into two datasets for model training (80%) and model testing (20%). Second, 15 landslide conditioning factors (i.e., altitude, slope, aspect, plane curvature, profile curvature, relief, roughness, rainfall, topographic wetness index (TWI), normalized difference vegetative index (NDVI), distance to roads, distance to rivers, land use/land cover (LULC), soil texture, and lithology) were initially selected to establish a landslide factor database. Thereafter, the multicollinearity test and information gain ratio (IGR) technique were applied to rank the importance of the factors. Subsequently, we used a series of metrics (e.g., accuracy, precision, recall, f-measure, area under the ROC (receiver operating characteristic) curve (AUC), kappa index, mean absolute error (MAE), and root mean square error (RMSE)) to evaluate the accuracy and performance of the six models. Based on the AUC values derived from the models, the GBDT+LR model with the highest AUC value (0.8168) was identified as the most efficient model for mapping landslide susceptibility, followed by the XGB+LR, XGB, RF+LR, GBDT, and RF models, which achieved AUC values of 0.8124, 0.8118, 0.8060, 0.7927, and 0.7883, respectively. The results from this study suggest that the stacking ensemble machine learning method is promising for use in landslide susceptibility mapping in the Zhangjiajie area and is capable of targeting the areas prone to landslides.

Photovoltaic (PV) generation is potentially uncertain. Probabilistic PV generation forecasting methods have been proposed with prediction intervals (PIs). However, several studies have dealt with geographically distributed PVs in a certain area. In this study, a two-step probabilistic forecast scheme is proposed for geographically distributed PV generation forecasting. Each step of the proposed scheme adopts ensemble forecasting based on three different machine-learning methods. In this case study, the proposed scheme was compared with conventional non-multistep forecasting. The proposed scheme improved the reliability of the PIs and deterministic PV forecasting results through 30 days of continuous operation with real data in Japan.

Freezing of Gait (FOG) is an impairment that affects the majority of patients in the advanced stages of Parkinson’s Disease (PD). FOG can lead to sudden falls and injuries, negatively impacting the quality of life for the patients and their families. Rhythmic Auditory Stimulation (RAS) can be used to help patients recover from FOG and resume normal gait. RAS might be ineffective due to the latency between the start of a FOG event, it’s detection and initialization of RAS. We propose a system capable of both FOG prediction and detection using signals from tri-axial accelerometer sensors that will be useful in initializing RAS with minimal latency. We compared the performance of several time frequency analysis techniques, including moving windows extracted from the signals, handcrafted features, Recurrence Plots (RP), Short Time Fourier Transform (STFT), Discreet Wavelet Transform (DWT) and Pseudo Wigner Ville Distribution (PWVD) with Deep Learning (DL) based Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN). We also propose three Ensemble Network Architectures that combine all the time frequency representations and DL architectures. Experimental results show that our ensemble architectures significantly improve the performance compared with existing techniques. We also present the results of applying our method trained on publicly available dataset to data collected from patients using wearable sensors in collaboration with A.T. Still University.

Computers have evolved over the years and as the evolution continues, we have been ushered into an era where high-speed internet has made it possible for devices in our homes, hospital, energy and industry to communicate with each other. This era is what is known as the Internet of Things (IoT). IoT has several benefits in the health, energy, transportation and agriculture sectors of a country’s economy. These enormous benefits coupled with the computational constraint of IoT devices which makes it difficult to deploy enhanced security protocols on them make IoT devices a target of cyber-attacks. One approach that has been used in traditional computing over the years to fight cyber-attacks is Intrusion Detection System (IDS). However, it is practically impossible to deploy IDS meant for traditional computers in IoT environments because of the computational constraint of these devices. In this regard, this study proposes a lightweight IDS for IoT devices using an incremental ensemble learning technique. We used Gaussian Naive Bayes and Hoeffding tree to build our incremental ensemble model. The model was then evaluated on the TON IoT dataset. Our proposed model was compared with other state-of-the-art methods proposed and evaluated using the same dataset. The experimental results show that the proposed model achieved an average accuracy of 99.98\%. We also evaluated the memory consumption of our model which showed that our model achieved a lightweight model status of 650.11KB as the highest memory consumption and 122.38KB as the lowest memory consumption.

Ensembles of Species Distribution Models (SDMs) represent the geographic ranges of pathogen vectors by combining alternative analytical approaches and merging information on vector occurrences with more extensive environmental data. Biased collection data impact SDMs, regardless of the target species but no studies have compared the differences in the distributions predicted by the ensemble models when different sampling frameworks are used for the same species. We compared Ensemble SDMs for two important Ixodid tick vectors, Amblyomma americanum and Ixodes scapularis in mainland Florida, USA, when inputs were either convenience samples of ticks, or collections obtained using the standard protocols promulgated by the U.S. Centers for Disease Control and Prevention. The Ensemble SDMs for the convenience samples and standard surveys showed only a slight agreement (Kappa = 0.060, A. americanum; 0.053, I. scapularis). Convenience sample SDMs indicated A. americanum and I. scapularis should be absent from 34.5% and 30.9% of the state where standard surveys predicted the highest likelihood of occurrence of the respective vectors. Ensemble models from standard surveys predicted 81.4% and 72.5% (A. americanum and I. scapularis) of convenience sample sites. Omission errors by standard survey SDMs, of the convenience collections, frequently were associated with adjacency to at least one SDM or errors in geocoding algorithms that failed to correctly locate convenience samples. These geocoding errors emphasize commonly overlooked needs to explicitly evaluate and improve data quality for vector survey data used in spatial models.

Local binary pattern is one of the visual descriptors and can be used as a powerful feature extractor for texture classification. In this paper, a novel representation for face recognition is proposed, called it Bilateral Line Local Binary Patterns (BL-LBP). This scheme is an extension of Line Local Binary Patterns descriptors in the statistical learning subspace. The present bilateral descriptors are fused with an ensemble learning of calibrated SVM models. The performance of this scheme is evaluated using 5 standard face databases. It is found that it is robust against illumination variation, diverse facial expressions and head pose variations and its recognition accuracy reaches 98 percent, running on a mobile device with a processing speed of 63 ms per face. Results suggest that our proposed method can be very useful for the vision systems that have limited resources where the computational cost is critical.

Molecular dynamics simulation is currently the theoretical technique eligible to simulate a wide range of systems from soft condensed matter to biological systems. However, of the excellent results that the technique has arrogated, this approach remains computationally expensive, but with the emergence of the new supercomputing technologies bases on graphics processing units graphical processing units-based systems GPUs, the perspective has changed. The GPUs allow performing large and complex simulations at a significantly reduced time. In this work, we present recent innovations in the acceleration of molecular dynamics in GPUs to simulate non-Hamiltonian systems. In particular, we show the performance of measure-preserving geometric integrator in the canonical ensemble, that is, at constant temperature. We provide a validation and performance evaluation of the code by calculating the thermodynamic properties of a Lennard-Jones fluid. Our results are in excellent agreement with reported data reported from literature, which were calculated with CPUs. The scope and limitations for performing simulations of high-throughput MD under rigorous statistical thermodynamics in the canonical ensemble are discussed and analyzed.

Software fault prediction is the very consequent research topic for software quality assurance. Data driven approaches provide robust mechanisms to deal with software fault prediction. However, the prediction performance of the model highly depends on the quality of dataset. Many software datasets suffers from the problem of class imbalance. In this regard, under-sampling is a popular data pre-processing method in dealing with class imbalance problem, Easy Ensemble (EE) present a robust approach to achieve a high classification rate and address the biasness towards majority class samples. However, imbalance class is not the only issue that harms performance of classifiers. Some noisy examples and irrelevant features may additionally reduce the rate of predictive accuracy of the classifier. In this paper, we proposed two-stage data pre-processing which incorporates feature selection and a new Rough set Easy Ensemble scheme. In feature selection stage, we eliminate the irrelevant features by feature ranking algorithm. In the second stage of a new Rough set Easy Ensemble by incorporating Rough K nearest neighbor rule filter (RK) afore executing Easy Ensemble (EE), named RKEE for short. RK can remove noisy examples from both minority and majority class. Experimental evaluation on real-world software projects, such as NASA and Eclipse dataset, is performed in order to demonstrate the effectiveness of our proposed approach. Furthermore, this paper comprehensively investigates the influencing factor in our approach. Such as, the impact of Rough set theory on noise-filter, the relationship between model performance and imbalance ratio etc. comprehensive experiments indicate that the proposed approach shows outstanding performance with significance in terms of area-under-the-curve (AUC).

This paper presents ensemble learning of multi-source satellite sensors dataset to obtain better predictive performance of the forest biomass. Spectral, spectral-indices, and spectral-textural features were generated from two optical satellite sensors, Landsat 8 Operational Land Imager (OLI) and Sentinel-2 Multispectral Instrument (MSI). In addition, two radar satellite sensors, Sentinel-1 C-band Synthetic Aperture Radar (CSAR), and Advanced Land Observing Satellite (ALOS-2) Phased Array type L-band Synthetic Aperture Radar (PALSAR-2) were utilized to generate backscattering and backscattering-textural features. The plot-wise above ground biomass data available from five forests in New England region were utilized. Ensemble learning of multi-source satellite sensors dataset was carried out by employing four machine learning regressors namely, Support Vector Machines (SVM), Random Forests (RF), Gradient Boosting (GB), and Multilayer Perceptron (MLP). A five-fold cross-validation method was used to evaluate predictive performance of the multi-source satellite sensors. The integration of multi-source satellite features, comprising of spectral, spectral-indices, backscattering, spectral-textural, and backscattering-textural information, through ensemble learning and cross-validation approach implemented in the research showed promising results (R² = 0.81, RMSE = 46.2 Mg/ha) for the estimation of plots-level forest biomass in New England region.

The progressive impairment analysis in gait from neurological diseases patients such as Hereditary Ataxias (HA) has been carried out using gait data collected with movement sensors. This research is focused on finding the minimum amount required of gait features to recognize efficiently and less intrusive way, HA patients based on data collected with iPhone movement sensors placed on the ankles from 14 HA patients and 14 healthy people. A twofold proposal is made , first a local minimum prominent peak criterion to find out the starting point of each stride, to get 10-stride window about which 56 spatial-temporal features are derived; second a search strategy based on Hill Climbing algorithm to reduce the number of gait features and sensors. The main results were the findings that with two gait patterns a 96% of classification accuracy was achieved by using K-Nearest Neighbors (KNN) and Multi-Layer Perceptron (MLP) algorithms, but in addition, MLP only right ankle sensor patterns were required which also allows to reduce the intrusion.

Automated medical diagnosis is one of the important machine learning applications in the domain of healthcare. In this regard, most of the approaches primarily focus on optimizing the accuracy of classification models. In this research, we argue that unlike general-purpose classification problems, medical applications, such as chronic kidney disease (CKD) diagnosis, require special treatment. In the case of CKD, apart from model performance, other factors such as the cost of data acquisition may also be taken into account to enhance the applicability of the automated diagnosis system. In this research, we have proposed two techniques for cost-sensitive feature ranking. An ensemble of decision tree models is employed in both the techniques for computing the worth of a feature in the CKD dataset. An automatic threshold selection heuristic is also introduced which is based on the intersection of features’ worth and their accumulated cost. A set of experiments are conducted to evaluate the efficacy of the proposed techniques on both tree-based and non-tree based classification models. The proposed approaches are also evaluated against several comparative techniques. Furthermore, it is demonstrated that the proposed techniques select around 1/4^th of the original CKD features while reducing the cost by a factor of 7.42 of the original feature set. Based on the extensive experimentation it is concluded that the proposed techniques employing feature-cost interaction heuristic tend to select feature subsets that are both useful and cost-effective.

To understand functions of biomolecules such as proteins, not only structures but their conformational change and kinetics are important to be characterized but its atomistic details are hard to obtain both experimentally and computationally. We review our recent computational studies using novel enhanced sampling techniques for conformational sampling of biomolecules and calculations of their kinetics. For efficiently characterizing the free energy landscape of a biomolecule, we introduce the multiscale enhanced sampling method, which uses a combined system of atomistic and coarse-grained models. Based on the idea of Hamiltonian replica exchange, we can recover the statistical properties of the atomistic model without any biases. We next introduce the string method as a path search method to calculate the minimum free energy pathways along a multidimensional curve in high dimensional space. Finally we introduce novel methods to calculate kinetics of biomolecules based on the ideas of path sampling: One is the Onsager-Machlup action method, and the other is the weighted ensemble method. Some applications of above methods to biomolecular systems are also discussed and illustrated.

Multilabel learning goes beyond standard supervised learning models by associating a sample with more than one class label. Among the many techniques developed in the last decade to handle multilabel learning best approaches are those harnessing the power of ensembles and deep learners. This work proposes merging both methods by combining a set of gated recurrent units, temporal convolutional neural networks, and long short-term memory networks trained with variants of the Adam optimization approach. We examine many Adam variants, each fundamentally based on the difference between present and past gradients, with step size adjusted for each parameter. We also combine Incorporating Multiple Clustering Centers and a bootstrap-aggregated decision trees ensemble, which is shown to further boost classification performance. In addition, we provide an ablation study for assessing the performance improvement that each module of our ensemble produces. Multiple experiments on a large set of datasets representing a wide variety of multilabel tasks demonstrate the robustness of our best ensemble, which is shown to outperform the state-of-the-art. The MATLAB code for generating the best ensembles in the experimental section will be made available at https://github.com/LorisNanni.

Soil temperature is a fundamental parameter in water resources and engineering. A cost-effective model which can forecast soil temperature accurately is extensively needed. Recently, many studies have applied artificial intelligence (AI) at both surface and underground levels for soil temperature prediction. However, there is no comprehensive and detailed assessment of the performance of different AI approaches in soil temperature estimation, and primarily limited atmospheric variables are used as input data for AI models. In the present study, great varieties of various land and atmospheric variables are applied to evaluate the performance of a wide range of AI methods on soil temperature prediction. Herein, thirteen approaches, from classic regressions to well-established methods of random forest and gradient boosting to advanced AI techniques like multi-layer perceptron and deep learning are taken into account. The results show that AI is a promising approach in climate parameter forecast and deep learning demonstrates the best performance among other models. It has the highest R-squared ranging from 0.957 to 0.980, the lowest NRMSE ranging from 2.237% to 3.287% and the lowest MAE, ranging from 0.510 to 0.743 in predicting soil temperature. The prediction is repeated for different sizes of data, and prediction outcomes confirm the conclusion mentioned above.

Automated lying-posture tracking is important in preventing bed-related disorders such as pressure injuries, sleep apnea, and lower-back pain. Prior research studied in-bed lying posture tracking using sensors of different modalities (e.g., accelerometer and pressure sensors). However, there remain significant gaps in research about how to design efficient in-bed lying posture tracking systems. These gaps can be articulated through several research questions as follows. First, can we design a single-sensor, pervasive, and inexpensive system that can accurately detect lying postures? Second, what computational models are most effective in the accurate detection of lying postures? Finally, what physical configuration of the sensor system is most effective for lying posture tracking? To answer these important research questions, in this article, we propose a comprehensive approach to design a sensor system that uses a single accelerometer along with machine learning algorithms for in-bed lying posture classification. We design two categories of machine learning algorithms based on deep learning and traditional classification with handcrafted features to detect lying postures. We also investigate what wearing sites are most effective in accurate detection of lying postures. We extensively evaluate the performance of the proposed algorithms on nine different body locations and four human lying postures using two datasets. Our results show that a system with a single accelerometer can be used with either deep learning or traditional classifiers to accurately detect lying postures. The best models in our approach achieve an F-Score that ranges from 95.2% to 97.8% with a coefficient of variation from 0.03 to 0.05. The results also identify the thighs and chest as the most salient body sites for lying posture tracking. Our findings in this article suggest that because accelerometers are ubiquitous and inexpensive sensors, they can be a viable source of information for pervasive monitoring of in-bed postures.

Labelling techniques such as electron paramagnetic resonance spectroscopy and single-molecule fluorescence resonance energy transfer, allow access to distances in the range of tens of angstroms, corresponding to the size of proteins and small to medium-sized protein complexes. Such measurements do not require long-range ordering and are therefore applicable to systems with partial disorder. Data from spin-label-based measurements can be processed into distance distributions that provide information about the extent of such disorder. Using such information in modelling presents several challenges, including a small number of restraints, the influence of the label itself on the measured distance and distribution width, and balancing the fitting quality of the long-range restraints with the fitting quality of other restraint subsets. Starting with general considerations about integrative and hybrid structural modelling, this review provides an overview of recent approaches to these problems and identifies where further progress is needed.

The rate of technical innovation, system interconnection, and advanced communications undoubtedly boost distributed energy networks' efficiency. However, when an additional attack surface is made available, the possibility of an increase in attacks is an unavoidable result. The energy ecosystem's significant variety draws attackers with various goals, making any critical infrastructure a threat, regardless of scale. Outdated technology and other antiquated countermeasures that worked years ago cannot address the complexity of current threats. As a result, robust artificial intelligence cyber-defense solutions are more important than ever. Based on the above challenge, this paper proposes an ensemble transfer learning spiking immune system for adaptive smart grid protection. It is an innovative Artificial Immune System (AIS) that uses a swarm of Evolving Izhikevich Neural Networks (EINN) in an Ensemble architecture, which optimally integrates Transfer Learning methodologies. The effectiveness of the proposed innovative system is demonstrated experimentally in multiple complex scenarios that optimally simulate the modern energy environment. In this way, the proposed system fully automates the strategic security planning of energy networks with computational intelligence methods. It allows the complete control of the digital strategies of the potential infrastructure that frames it, thus contributing to the timely and valid decision-making during cyber-attacks.

Three significant changes have occurred in the winter climate in Europe recently: increased UK flooding; Iberian drought; and warmer temperatures north of the Alps. The literature links all three to a persistent, significant increase in sea level pressure over Southern Europe, the Mediterranean, Iberia and the Eastern Atlantic (SEMIEA) which changes the atmospheric circulation system: forcing cold fronts to the north away from Iberia; and creating a south westerly flow around the northern perimeter of the high-pressure region bringing warmer, moist air from the subtropical Atlantic to the UK and Europe which increases precipitation in the UK and raises the temperature in Europe. I use the Last Millennium Ensemble, MERRA-2 and Terra-NCEP data to demonstrate that the extreme, anthropogenic, West African aerosol Plume (WAP) which only exists from December to April perturbs the northern, regional Hadley Circulation creating the high pressure in the SEMIEA. I also show that the anthropogenic WAP has only existed in its extreme form in recent decades as the two major sources of the WAP aerosols: biomass burning; and gas flaring have both increased significantly since 1950 due to: a four-fold increase in population; and gas flaring rising from zero to 7.4 billion m3/annum and note that this time span coincides with the changes in the three elements of the winter climate of Europe. I also suggest that it may be possible to eliminate the WAP and return the winter climate of Europe to its natural state after the crucial first step of recognising the cause of the changes is taken.

Climate analogues provide forestry practice empirical evidence of how forests are managed in “twin” regions, i.e. regions where the current climate is comparable to the expected future climate at a site of interest. But the uncertain future climate creates uncertainty in how to adapt the forests. We therefore investigate how the uncertainty in future climate affects tree species suitability and whether there is a common underlying pattern. Like most studies we employ different ensemble variants of RCP 4.5 and 8.5. But instead of focusing on a single point in future time, we resolve each variant in a climate trajectory from 2000 to 2100. We calculate climatic distances between the climate trajectories of our site of interest and the current climate in Europe, generating maps with twin regions from 2000 to 2100. Forest inventories from the twin regions allow us to trace the changes in the prevalence of 23 major tree species. We find that it is not the direction but rather the velocity of the change that differs between the scenarios. We use this pattern to propose a tree species suitability concept that integrates the uncertainty in future climate. Twin regions provide further information on silvicultural practices, pest management, product chains etc.

This paper briefly reviews the state of the art in artificial intelligence including inductive and deductive methods. Deep learning and ensemble machine learning lie in inductive methods while automated reasoning implemented in deductive computer languages (Prolog, Otter, and Z3) is based on deductive methods. In the inductive methods, intelligence is inferred by pseudorandom number for creating the sophisticated decision trees in Go (game), Shogi (game), and quiz bowl questions. This paper demonstrates how to wisely use the pseudorandom number for solving coin-weighing puzzles with the deductive method. Monte Carlo approach is a general purpose problem-solving method using random number. The proposed method using pseudorandom number lies in one of Monte Carlo methods. In the proposed method, pseudorandom number plays a key role in generating constrained solution candidates for coin-weighing puzzles. This may be the first attempt that every solution candidate is solely generated by pseudorandom number while deductive rules are used for verifying solution candidates. In this paper, the performance of the proposed method was measured by comparing with the existing open source codes by solving 12-coin and 24-coin puzzles respectively.

The radar sensor described realizes healthcare monitoring capable of detecting subject chest-wall movement caused by cardiopulmonary activities, and wirelessly estimating the respiration and heartbeat rates of the subject without attaching any devices to the body. Conventional single-tone Doppler radar can only capture Doppler signatures because of a lack of bandwidth information with noncontact sensors. In contrast, we take full advantage of impulse radio ultra-wideband (IR-UWB) radar to achieve low power consumption and convenient portability, with a flexible detection range and desirable accuracy. A noise reduction method based on improved ensemble empirical mode decomposition (EEMD) and a vital sign separation method based on the continuous-wavelet transform (CWT) are proposed jointly to improve the signal-to-noise ratio (SNR) in order to acquire accurate respiration and heartbeat rates. Experimental results illustrate that respiration and heartbeat signals can be extracted accurately under different conditions. This noncontact healthcare sensor system proves the commercial feasibility and considerable accessibility of using compact IR-UWB radar for emerging biomedical applications.

Nowadays, according to spectacular improvement in health care and biomedical level, a tremendous amount of data is recorded by hospitals. In addition, the most effective approach to reduce disease mortality is to diagnose it as soon as possible. As a result, data mining by applying machine learning in the field of diseases provides good opportunities to examine the hidden patterns of this collection. An exact forecast of the mortality after heart surgery will cause Successful medical treatment and fewer costs. This research wants to recommend a new stacking predictive model after utilizing the random forest feature importance method to foresee the mortality after heart surgery on a highly unbalanced dataset by using the most practical features. To solve the unbalanced data problem, a combination of the SVM-SMOTE over-sampling algorithm and the Edited-Nearest-Neighbor under-sampling algorithm is used. This research compares the introduced model with some other machine learning classifiers to ensure efficiency through shuffle hold-out and 10-fold cross-validation strategies. In order to validate the performance of the implemented machine learning methods in this research, both shuffle hold-out, and 10-fold cross-validation results indicated that our model had the highest efficiency compared to the other models. Furthermore, the Friedman statistical test is applied to survey the differences between models. The result demonstrates that the introduced stacking model reached the most accurate predicting performance after Logistic Regression.

The construction of different roads, such as freeways, highways, major roads or minor roads must be accompanied by constant monitoring and evaluation of service delivery. Pavements are generally assessed by engineers in terms of the smoothness, surface condition, structural condition and surface safety. Pavement assessment is often conducted using the qualitative indices such as international roughness index (IRI), pavement condition index (PCI), structural condition index (SCI) and skid resistance value (SRV), which are used for smoothness assessment, surface condition assessment, structural condition assessment, and surface safety assessment, respectively. In this paper, Tehran-Qom Freeway in Iran has been selected as the case study and its smoothness and pavement surface conditions are assessed. At 2-km intervals, a 100-meter sample unit is selected in the slow-speed lane (totally, 118 sample units). In these sample units, the PCI is calculated after a visual inspection of the pavement and the recording of distresses. Then, in each sample unit, the average IRI is computed. The purpose of this study is to provide a method for estimating PCI based on IRI. The proposed theory was developed by Random Forest (RF), and Random Forest optimized by Genetic Algorithm (RF-GA) methods and these methods were validated using correlation coefficient (CC), scattered index (SI), and Willmott’s index of agreement (WI) criteria. The proposed method reduces costs, saves time and eliminates the safety risks.

The classical Rotational Isomeric State (RIS) model, originally proposed by Flory, has been used to rationalize a wide range of physicochemical properties of neutral polymers. However, many weak polyelectrolytes of interest are able to regulate their charge depending on the conformational state of the bonds. Recently, it has been shown that the RIS model can be coupled with the Site Binding (SB) model, for which the ionizable sites can adopt two states: protonated or deprotonated. The resulting combined scheme, the SBRIS model, allows to analyse ionization and conformational equilibria on the same foot. In the present work this approach is extended to include pH-dependent electrostatic Long Range (LR) interactions, ubiquitous in weak polyelectrolytes at moderate and low ionic strengths. With this aim the original LR interactions are taken into account by defining effective Short Range (SR) and pH-dependent parameters, such as effective microscopic protonation constants and rotational bond energies. The new parameters are systematically calculated using variational methods. The machinery of statistical mechanics for SR interactions, including the powerful and fast transfer matrix methods, can then be applied. The resulting technique, to which we will refer as Local Effective Interaction Parameters (LEIP) method, is illustrated with a minimal model of a flexible linear polyelectrolyte containing only one type of rotating bonds. LEIP reproduces very well the pH dependence of the degree of protonation and bond probabilities obtained by semi-grand canonical Monte Carlo simulations, where LR interactions are taken explicitly into account. The reduction in the computational time in several orders of magnitude suggests that the LEIP technique could be useful in a range of areas involving linear weak polyelectrolytes, allowing direct fitting of the relevant physical parameters to the experimental quantities.

This work addresses the role of charge regulation (CR) and the associated fluctuations in the conformational and mechanical properties of weak polyelectrolytes. Due to CR, changes in the pH-value modifies the average macromolecular charge and conformational equilibria. A second effect is that, for a given average charge per site, fluctuations can alter the intensity of the interactions by means of correlation between binding sites. We investigate both effects by means of Monte Carlo simulations at constant pH-value, so that the charge is a fluctuating quantity. Once the average charge per site is available, we turn off the fluctuations by assigning the same average charge to every site. A constant charge MC simulation is then performed. We make use of a model which accounts for the main fundamental aspects of a linear flexible polyelectrolyte i.e. proton binding, angle internal rotation, bond stretching and bending. Steric excluded volume and differentiated treatment for short-range and long-range interactions are also included in the model. It can be regarded as a kind of "minimal'' model in the sense that contains a minimum number of parameters but still preserving the atomistic detail. It is shown that, if fluctuations are activated, gauche state bond probabilities increase, and the persistence length decreases, so that the polymer becomes more folded. Macromolecular stretching is also analyzed in presence of CR (the charge depends on the applied force) and without CR (the charge is fixed to the value at zero force). The analysis of the low force scaling behavior concludes that Pincus exponent becomes pH-dependent. Both with and without CR, a transition from 1/2 at high pH-values (phantom chain) to 3/5 to low pH-values (Pincus regime), is observed. Finally, the intermediate force stretching regime is investigated. It is found that CR induces a moderate influence in the force-extension curves and persistence length (which in this force regime becomes force-dependent). It is thus concluded that the effect of CR on the stretching curves is mainly due to changes in the average charge at zero force. It is also found that, for the cases studied, the effect of steric excluded volume is almost irrelevant compared to electrostatic interactions.

The need to increase the energy efficiency of buildings as well as the use of local renewable heat sources has caused that heat meters are used not only to calculate the consumed energy but also for the active management of central heating systems. Increasing the reading frequency and the use of measurement data to control the heating system expands the requirements for the reliability of heat meters. The aim of the research is to analyse a large set of meters in the real network and predict their faults to avoid inaccurate readings, incorrect billing, heating system disruption and unnecessary maintenance. The reliability analysis of heat metres, based on historical data collected over several years, shows some regularities which cannot be easily described by physics-based models. The failure rate is almost constant and does depend on the past but is a non-linear combination of state variables. To predict meters' failures in the next settlement period, three independent machine learning models are implemented and compared with selected metrics because even the high performance of a single model (87\% True Positive for Neural Network) may be insufficient to make a maintenance decision. Additionally, performing hyperparameters optimisation boosts models' performance by a few percent. Finally, three improved models are used to build an ensemble classifier which outperforms the individual models. The proposed procedure ensures the high efficiency of fault detection (>95\%), while maintaining overfitting at the minimum level. The methodology is universal and can be utilised to study the reliability and predict faults of other types of meters and different objects with the constant failure rate.

Pathogenicity is unknown for the majority of human gene variants. For prioritization of sequenced somatic and germline mutation variants, in silico approaches can be utilized. In this study, 84 million non-synonymous Single Nucleotide Variants (SNVs) in the human coding genome were annotated using consensus Variant Effect Prediction (cVEP) method. An algorithm, implemented as a stacked ensemble of supervised learners, performed combination of the 39 functional, conservation mutation impact scores from dbNSFP4.0. Adding gene indispensability score, accounting for differences in the pathogenicities of the variants in the essential and the mutation-tolerant genes, improved the predictions. For each SNV the consensus combination gives either a continuous-value pathogenicity score, or a categorical score in five classes: pathogenic, likely pathogenic, uncertain significance, likely benign, benign. The provided class database is aimed for direct use in clinical practice. The trained prediction models were 5-fold cross-validated on the evidence-based categorical annotations from the ClinVar database. The rankings of the scores based on their ability to predict pathogenicity were obtained. A two-step strategy using the rankings, scores and class annotations is suggested for filtering and prioritization of the human exome mutations in clinical and biological applications of NGS technology.

Electricity demand prediction is vital for energy production management and proper exploitation of the present resources. Recently, several novel machine learning (ML) models have been employed for electricity demand prediction to estimate the future prospects of the energy requirements. The main objective of this study is to review the various ML models applied for electricity demand prediction. Through a novel search and taxonomy, the most relevant original research articles in the field are identified and further classified according to the ML modeling technique, perdition type, and the application area. A comprehensive review of the literature identifies the major ML models, their applications and a discussion on the evaluation of their performance. This paper further makes a discussion on the trend and the performance of the ML models. As the result, this research reports an outstanding rise in the accuracy, robustness, precision and the generalization ability of the prediction models using the hybrid and ensemble ML algorithms.