Many clinical research datasets have a large percentage of missing values that directly impacts their usefulness in yielding high accuracy classifiers when used for training in supervised machine learning. While missing value imputation methods have been shown to work well with smaller percentages of missing values, their ability to impute sparse clinical research data can be problem specific. We previously attempted to learn quantitative guidelines for ordering cardiac magnetic resonance imaging during the evaluation for pediatric cardiomyopathy, but missing data significantly reduced our usable sample size. In this work, we sought to determine if increasing the usable sample size through imputation would allow us to learn better guidelines. We first review several machine learning methods for estimating missing data. Then, we apply four popular methods (mean imputation, decision tree, k-nearest neighbors, and self-organizing maps) to a clinical research dataset of pediatric patients undergoing evaluation for cardiomyopathy. Using Bayesian Rule Learning (BRL) to learn ruleset models, we compared the performance of imputation-augmented models versus unaugmented models. We found that all four imputation-augmented models performed similarly to unaugmented models. While imputation did not improve performance, it did provide evidence for the robustness of our learned models.

Censoring occurs when complete follow-up time information is unavailable for patients enrolled in a clinical study. The process is considered to be informative (nonignorable) if the likelihood function for the censoring model cannot be partitioned into a set of response parameters that are independent of the censoring parameters. In such cases, estimated survival time probabilities may be biased, prompting the need for special statistical methods to remedy the situation. The problem is especially salient when censoring is skewed toward the early phase of a study. In this paper, we describe a method to impute censored follow-up times using a counting process method.

Aiming at the difficult problem of broadband oscillation localization in power systems, the intel-ligent localization method of oscillation source based on the digital twin is proposed and the os-cillation source localization system based on the digital twin is constructed. Firstly, a digital twin-based oscillation source localization method and system architecture are proposed. Fur-thermore, an intelligent positioning method of oscillation source based on data-driven and mechanism fusion is proposed. It includes three steps: oscillation signal preprocessing, oscillation modal analysis and oscillation source localization. For the oscillation signal preprocessing, the generative adversarial imputation network is used to repair the missing samples, and the su-per-resolution technique is used to realize the super-resolution measurement of broadband os-cillation. In the oscillation modal analysis, the spectrum of the oscillation signal is extracted using the fast Fourier transform. To accurately locate the oscillation source, the branch potential energy is used as input to the data-driven model, such as LSTM and CNN. Finally, an oscillation source localization system is developed based on the digital twin workshop CloudPSS-XStudio, which can locate the oscillation source quickly and accurately.

The monitoring of surface-water quality followed by water-quality modeling and analysis is essential for generating effective strategies in water-resource management. However, worldwide, particularly in developing countries, water-quality studies are limited due to the lack of a complete and reliable dataset of surface-water-quality variables. In this context, several statistical and machine-learning models were assessed for imputing water-quality data at six monitoring stations located in the Santa Lucía Chico river (Uruguay), a mixed lotic and lentic river system. The challenge of this study is represented by the high percentage of missing data (between 50% and 70%) and the high temporal and spatial variability that characterizes the water-quality variables. The competing algorithms implemented belonged to both univariate and multivariate imputation methods (inverse distance weighting (IDW), Random Forest Regressor (RFR), Ridge (R), Bayesian Ridge (BR), AdaBoost (AB), Hubber Regressor (HR), Support Vector Regressor (SVR), and K-nearest neighbors Regressor (KNNR)). According to the results, more than 76% of the imputation outcomes are considered satisfactory (NSE > 0.45). The imputation performance shows better results at the monitoring stations located inside the reservoir than the ones positioned along the mainstream. IDW was the most chosen model for data imputation.

Gas chromatography-coupled mass spectrometry (GC-MS) has been used in biomedical research to analyze volatile, non-polar, and polar metabolites in a wide array of sample types. Despite advances in technology, missing values are still common in metabolomics datasets and must be properly handled. We evaluated the performance of ten commonly used missing value imputation methods with metabolites analyzed on an HR GC-MS instrument. By introducing missing values into the complete (i.e., data without any missing values) NIST plasma dataset we demonstrate that Random Forest (RF), Glmnet Ridge Regression (GRR), and Bayesian Principal Component Analysis (BPCA) shared the lowest Root Mean Squared Error (RMSE) in technical replicate data. Further examination of these three methods in data from baboon plasma and liver samples demonstrated they all maintained high accuracy. Overall, our analysis suggests that any of the three imputation methods can be applied effectively to untargeted metabolomics datasets with high accuracy. However, it is important to note that imputation will alter the correlation structure of the dataset, and bias downstream regression coefficients and p-values.

Multiple imputation with chained equations (MICE) is a widespread approach to account for missing data in empirical research. Combining MICE with bootstrapping, that is, repeatedly resampling with replacement from the data to estimate variances and confidence intervals of statistics of interest, is not straightforward. The current document provides an overview of how to use bootstrapping with imputed data in Stata. Two main approaches (impute first and then bootstrap or vice-versa) are discussed and shortly compared. Code is provided for Stata.

Genotype imputation is widely used to enrich genetic datasets. The operation relies of panels of known reference haplotypes with typically whole-genome sequencing data. How to choose a reference panel has been widely studied and it is essential to have a panel that is well matched to the individuals who require imputation of missing genotypes. However, it is broadly accepted that such an imputation panel will have an enhanced performance with the inclusion of diversity; haplotypes from many different populations. We investigate this observation in this work by examining in fine detail exactly which reference haplotypes are contributing at different regions of the genome. This is achieved using a novel method of inserting synthetic genetic variation into the reference panel in order to track the performance of leading imputation algorithms. We show that while diversity may globally improve imputation accuracy, there can be occasions where incorrect genotypes are imputed following the inclusion of more diverse haplotypes in the reference panel. We however demonstrate a technique for retaining and benefitting from the diversity in the reference panel whilst avoiding the occasional adverse effects on imputation accuracy. What is more, our results elucidate more clearly the role of the diversity in a reference panel than has been shown in previous studies.

Missing observations constitute one of the most important issues in data analysis in applied research studies. The magnitude and their structure impact parameters estimation in the modeling with important consequences for decision-making. This study aims to evaluate the efficiency of imputation methods combined with the backpropagation algorithm in a nonlinear regression context. The evaluation is conducted through a simulation study including sample sizes (50, 100, 200, 300 and 400) with different missing data rates (10, 20, 30 40 and 50%) and three missingness mechanisms (MCAR, MAR and MNAR). Four imputation methods (Last Observation Carried Forward, Random Forest, Amelia and MICE) were used to impute datasets before making prediction with backpropagation. 3-MLP model was used by varying the activation functions (Logistic-Linear, Logistic-Exponential, TanH-Linear and TanH-Exponentiel), the number of nodes in the hidden layer (3 - 15) and the learning rate (20 - 70%). Analysis of the performance criteria (R2, r and RMSE) of the network revealed good performances when it is trained with TanH-Linear functions, 11 nodes in the hidden layer and a learning rate of 50%. MICE and Random Forest were the most appropriate for data imputation. These methods can support up to 50% of missing rate with an optimal sample size of 200.

Background: All Payer Claims Databases (APCD) are a rich source of health information, however, race and ethnicity (R&E) data are largely missing. Bayesian Improved Surname Geocoding (BISG) is a common R&E imputation method, yet, validation of BISG in APCDs is lacking. We used the BISG to impute missing R&E in the Oregon APCD. Methods: BISG imputed R&E for Asian Pacific Islanders (API), Blacks, Hispanics and Whites were contrasted to the gold standard (vital statistics) and sensitivity and specificity improvements were assessed. Logistic regression examined whether missing R&E was random across patient characteristics. Results: Among 85,857 individuals in the study, 32.1% (n=27,594) had missing R&E. Missing R&E was not randomly distributed. There were higher odds of missingness among males, Whites, those age 65 and older, and commercially insured individuals. Differences in the percent missing were also found by co-morbid conditions and mortality causes. Imputing the missing R&E with BISG method improved the sensitivity to identify White, Black, API, and Hispanics. Conclusions: APCDs can benefit from enhancing missing R&E with BISG imputation to perform more robust population-health level analyses and identify inequities according to R&E without losing power or dropping non-random records with missing R&E data.

The BRFSS is an annual survey conducted by each state and designed to identify trends in a representative sample of the resident population. In its 2019 field survey, the U.S. state of Georgia tested a new 3 – item module to measure the numbers of bereaved, resident adults. Bereavement means that participants answered, ‘Yes’ to the item ‘Have you experi-enced the death of a family member or close friend in the years 2018 or 2019?’. This analysis addresses two questions. Can estimates for bereavement prevalence be derived without large sampling errors, low precision, and small subsamples? Can multiple imputation techniques be applied to overcome non-response and missing data to support multivari-ate modeling? Analyses in this study were conducted under two scenarios. Scenario 1 ap-plies the complex sample weights created by the Centers for Disease Control and imputes values for missing responses. Scenario 2 treats the data as a panel – no weighting com-bined with removal of persons with missing data. Scenario 1 reflects the use of BRFSS data for public health and policy, while Scenario 2 reflects data as it is commonly used in so-cial science research. The bereavement item has a response rate (RR) of 70.8% (5206 of 7534 persons). Subgroups have RR of 55% or more. Under Scenario 1, the prevalence of bereavement is 45.38%, meaning that 3,739,120 adults reported bereaved in 2018 or 2019. The prevalence is 46.02% with Scenario 2 which removes persons with any missing data (4,289 persons). Scenario 2 overestimates the bereavement prevalence by 1.39%. An il-lustrative logistic model is included to show the performance of exposure to bereavement under each scenario. Recent bereavement can be ascertained in a surveillance survey without biases in response. This survey is limited to one US state in a single year and ex-cludes persons aged 17 years and younger.

The extensive deployment of wireless infrastructure provides a low-cost way to track mobile users in indoor environment. This paper demonstrates a prototype model of an accurate and reliable room location awareness system in a real public environment, where three typical problems arise. First, a massive number of access points (APs) can be sensed leading to a high-dimensional classification problem. Second, heterogeneous devices record different received signal strength (RSS) levels due to the variations in chip-set and antenna attenuation. Third, APs are not necessarily visible in every scanning cycle leading to missing data. This paper presents a probabilistic Wi-Fi fingerprinting method in a hidden Markov model (HMM) framework for mobile user tracking. Considering the spatial correlation of the signal strengths from multiple APs, a Multivariate Gaussian Mixture Model (MVGMM) is fitted to model the probability distribution of RSS measurements in each cell. Furthermore, the unseen property of invisible AP has been investigated in this research, and demonstrated the efficiency of differentiation between cells. The proposed system is able to achieve comparable localization performance. The filed test results present a reliable 97% localization room level accuracy of multiple mobile users in a real university campus WiFi network without any prior knowledge of the environment.

Missing item responses are prevalent in educational large-scale assessment studies like the programme for international student assessment (PISA). The current operational practice scores missing item responses as wrong, but several psychometricians advocated a model-based treatment based on latent ignorability assumption. In this approach, item responses and response indicators are jointly modeled conditional on a latent ability and a latent response propensity variable. Alternatively, imputation-based approaches can be used. The latent ignorability assumption is weakened in the Mislevy-Wu model that characterizes a nonignorable missingness mechanism and allows the missingness of an item to depend on the item itself. The scoring of missing item responses as wrong and the latent ignorable model are submodels of the Mislevy-Wu model. This article uses the PISA 2018 mathematics dataset to investigate the consequences of different missing data treatments on country means. Obtained country means can substantially differ for the different scaling models. In contrast to previous statements in the literature, the scoring of missing item responses as incorrect provided a better model fit than a latent ignorable model for most countries. Furthermore, the dependence of the missingness of an item from the item itself after conditioning on the latent response propensity was much more pronounced for constructed-response items than for multiple-choice items. As a consequence, scaling models that presuppose latent ignorability should be refused from two perspectives. First, the Mislevy-Wu model is preferred over the latent ignorable model for reasons of model fit. Second, we argue that model fit should only play a minor role in choosing psychometric models in large-scale assessment studies because validity aspects are most relevant. Missing data treatments that countries can simply manipulate (and, hence, their students) result in unfair country comparisons.