Outliers can influence regression model parameters and change the direction of the estimated effect, over-estimating or under-estimating the strength of the association between a response variable and an exposure of interest. Identifying visit-level outliers from longitudinal data with continuous time-dependent covariates is important especially when the distribution of such variable is highly skewed at follow-up visits. The primary objective was to identify potential outliers at follow-up visits using interquartile range (IQR) statistic, motivated by a large TEDDY dietary longitudinal and time-to-event data with a continuous time varying vitamin B12 intake as the exposure of interest and time to developing Islet Autoimmunity (IA) as the response variable. The IQR method was also applied to simulated data. To assess the impact of IQR-method detected outliers, data was analyzed using Cox-proportional hazard model with robust sandwich estimator. Partial residual diagnostic plots were used to detect highly influential outliers. Results showed how some of the detected outliers had large influence on the Cox regression model and changed both the direction of hazard ratios and the strength of association with the risk of developing IA. In conclusion, the IQR method is useful in identifying potential outliers at visit-level which can be further investigated.

This paper focuses on a joint model to analyse longitudinal proportion and survival data. After transforming the longitudinal proportional data by a logit function, we adopt a partially linear mixed-effect model for it, in which nonlinear function of time is fitted via using B-splines technique and a centered Dirichlet Process Mixture Model (CDPMM) is specified for a general distribution of random effects. The survival data is assumed to a Cox proportional hazard model, the sharing random effects joint model is developed for the two types of data. Combining the Gibbs sampler and the Metropolis-Hastings algorithm, we propose a Bayesian Lasso (BLasso) method to simultaneously estimate unknown parameters and select important covariates. Simulation studies are conducted to investigate the finite sample performance of the proposed methods. An example from the MA.5 research experiment is used to illustrate the proposed methodologies.

The Smart Energy Research Lab (SERL) Observatory dataset described here comprises half-hourly and daily electricity and gas data, SERL survey data, Energy Performance Certificate (EPC) input data and 24 local hourly climate reanalysis variables from the European Centre for Medium-Range Weather Forecasts (ECMWF) for over 13,000 households in Great Britain (GB). Participants were recruited in September 2019, September 2020 and January 2021 and their smart meter data are collected from up to one year prior to sign up. Data collection will continue until at least August 2022, and longer if funding allows. Survey data relating to the dwelling, appliances, household demographics and attitudes was collected at sign up. Data are linked at the household level and UK-based academic researchers can apply for access within a secure virtual environment for research projects in the public interest. This is a data descriptor paper describing how the data was collected, the variables available and the representativeness of the sample compared to national estimates. It is intended as a guide for researchers working with or considering using the SERL Observatory dataset, or simply looking to learn more about it.

Background: Observational health data has the potential to be a rich resource to inform clinical practice and regulatory decision making. However, the lack of standard data quality processes makes it difficult to know if these data are research ready. The EHDEN COVID-19 Rapid Col-laboration Call presented the opportunity to assess how the newly developed open-source tool Data Quality Dashboard (DQD) informs the quality of data in a federated network. Methods: 15 Data Partners (DPs) from 10 different countries worked with the EHDEN taskforce to map their data to the OMOP CDM. Throughout the process at least two DQD results were collected and compared for each DP. Results: All DPs showed an improvement in their data quality between the first and last run of the DQD. The DQD excelled at helping DPs identify and fix conformance is-sues but showed less of an impact on completeness and plausibility checks. Conclusions: This is the first study to apply the DQD on multiple, disparate databases across a network. While study-specific checks should still be run, we recommend that all data holders converting their data to the OMOP CDM use the DQD as it ensures conformance to the model specifications and that a database meets a baseline level of completeness and plausibility for use in research.

Objective: Determine the feasibility of utilizing longitudinal electronic dental record (EDR) data to track change over time in patient periodontal disease (PD) and to generate three patient cohorts: 1) patients whose disease did not change over time, 2) patients whose PD progressed, and 3) patients whose disease improved over time using informatics approaches. Methods: We conducted a retrospective study of 28,908 patients who received a comprehensive oral evaluation between January 1, 2009, and December 31, 2014, at the Indiana University School of Dentistry (IUSD) clinics. We developed and tested three automated computer applications to: 1) diagnose periodontitis cases from periodontal charting, 2) retrieve clinician-documented diagnoses from clinical notes, and 3) track disease change over time. We also evaluated the density of longi-tudinal EDR data for the following follow-up times: 1) none, 2) up to 5 years, 3) >5 and <=10 years, and 4) >10 and <=15 years Results: Thirty-four percent (n=9,954) of the study cohort had up to five years of follow-up visits with an average of 2.78 visits with periodontal charting information. An average of three patient visits per year that contained periodontal charts (63,552) were utilized to obtain a diagnosis, which is considered excellent. For clinician-documented diagnoses from clinical notes, 42% of patients (n=5,562) had at least two PD diagnoses to determine their disease change. In this cohort with cli-nician-documented diagnoses, 72% percent of patients (n=3,919) did not have a disease status change between their first and last visits, 669 (13%) patients' disease status progressed, and 589 (11%) patients’ disease improved. Conclusions: This study demonstrated the feasibility of utilizing longitudinal EDR data to track disease changes over 15 years during the observation study period. We found excellent longitudinal data when diagnoses generated from periodontal charting were considered (three visits per pa-tient). This information can be now utilized for studying the clinical course of periodontitis.

Advances in DNA sequencing technology are facilitating genomic analyses of unprecedented scope and scale, widening the gap between our abilities to generate and fully exploit biological sequence data. Comparable analytical challenges are encountered in other data-intensive fields involving sequential data, such as signal processing, in which dimensionality reduction (i.e., compression) methods are routinely used to lessen the computational burden of analyses. In this work we explore the application of dimensionality reduction methods to numerically represent high-throughput sequence data for three important biological applications of virus sequence data: reference-based mapping, short sequence classification and de novo assembly. Despite using highly compressed sequence transformations to accelerate the processes, our sequence processing approach yielded comparable accuracy to existing approaches, and are ideally suited for sequences originating from highly diverse virus populations. We demonstrate the application of our methodology to both synthetic and real viral pathogen sequence data. Our results show that the use of highly compressed sequence approximations can provide accurate results and that useful analytical performance can be retained and even enhanced through appropriate dimensionality reduction of sequence data.

This article proposes a measurement solution designed to monitor instantaneous frequency in power systems. It uses a data acquisition module and a GPS receiver for time stamping. A program in Python takes care of receiving the data, calculating the frequency, and finally transferring the measurement results to a database. The frequency is calculated with two different methods, which are compared in the article. The stored data is visualized using the Grafana platform, thus demonstrating its potential for comparing scientific data. The system as a whole constitutes an efficient low cost solution as a data acquisition system.

This research is a preliminary phase of a general effort to develop a generic-data transferring capability via human haptic thermal sensation (generic-data refers to a coded language like Morse or Braille). For the capability to be effective, it must include a large variety of short recognizable cues. Hence, we propose the concept of cues based on sequences of thermal pulses, i.e., combinations of warm and cool pulses with several levels of intensity. The objective of this study was to determine the feasibility of basing a generic-data-transfer capability on thermal cues composed of sequences of short pulses. The research included defining the basic characteristics of the stimuli parameters and developing practical methods for generating and measuring them. Several sequences were designed in light of the relevant data known to date, and tests were conducted. The thermal cues presented to the participants were sensed and recognized by touch. The results indicate high feasibility for a capability that is applicable in various scenarios. In addition, the low impact on human skin temperature due to short stimuli duration represents an inherent advantage for later implementation. This report presents promising findings and offers insights for further investigation.

Background Increasingly, healthcare organizations are using technology for the efficient management of data. The aim of this study was to compare the data quality of digital records with the quality of the corresponding paper-based records by using data quality assessment framework. Methodology We conducted a desk review of paper-based and digital records over the study duration from April 2016 to July 2016 at six enrolled TB clinics. We input all data fields of the patient treatment (TB01) card into a spreadsheet-based template to undertake a field-to-field comparison of the shared fields between TB01 and digital data. Findings A total of 117 TB01 cards were prepared at six enrolled sites, whereas just 50% of the records (n=59; 59 out of 117 TB01 cards) were digitized. There were 1,239 comparable data fields, out of which 65% (n=803) were correctly matched between paper based and digital records. However, 35% of the data fields (n=436) had anomalies, either in paper-based records or in digital records. 1.9 data quality issues were calculated per digital patient record, whereas it was 2.1 issues per record for paper-based record. Based on the analysis of valid data quality issues, it was found that there were more data quality issues in paper-based records (n=123) than in digital records (n=110). Conclusion There were fewer data quality issues in digital records as compared to the corresponding paper-based records. Greater use of mobile data capture and continued use of the data quality assessment framework can deliver more meaningful information for decision making.

The increasing availability of longitudinal data (repeated numerical observations of same units at different times) requires the development of flexible techniques to automatically detect common errors in such data. Besides obvious and easily identifiable cases, such as missing or out-of-range data, large longitudinal dataset often present problems not easily traceable by the techniques used for generic datasets. In particular, elusive and baffling problems are i) inversion of one or more values from one unit to another; ii) anomalous jumps in the series of values, iii) errors in the timing of the values due to a recalculation operated by the data providers to compensate previous errors. This work proposes a statistical-mathematical approach based on a system of indicators that is able to capture the complexity of the described problems by working at the formal level, regardless of the specific meaning of the data. The proposed approach identifies suspect erroneous data and is applicable in a variety of contexts. We implement this approach in a relevant database of European Higher Education institutions (ETER) by analyzing Total academic staff, that is one of the most important variables, used in empirical analysis as proxy of size and also considered by policy makers at European level.

Prediction of protein-ligand binding affinities is crucial for computational drug discovery. A number of deep learning approaches have been developed in recent years to improve the accuracy of such affinity prediction. While the predicting power of these systems have advanced to some degrees depending on the dataset used for model training and testing, the effects of the quality and quantity of the underlying data have not been thoroughly examined. In this study, we employed erroneous datasets and data subsets of different sizes, created from one of the largest databases of experimental binding affinities, to train and evaluate a deep learning system based on convolutional neural networks. Our results show that data quality and quantity do have significant impacts on the prediction performance of trained models. Depending on the variations in data quality and quantity, the performance discrepancies could be comparable to or even larger than those observed among different deep learning approaches. In particular, the presence of proteins during model training leads to a dramatic increase in prediction accuracy. This implies that continued accumulation of high-quality affinity data, especially for new protein targets, is indispensable for improving deep learning models to better predict protein-ligand binding affinities.

The monitoring of surface-water quality followed by water-quality modeling and analysis is essential for generating effective strategies in water-resource management. However, worldwide, particularly in developing countries, water-quality studies are limited due to the lack of a complete and reliable dataset of surface-water-quality variables. In this context, several statistical and machine-learning models were assessed for imputing water-quality data at six monitoring stations located in the Santa Lucía Chico river (Uruguay), a mixed lotic and lentic river system. The challenge of this study is represented by the high percentage of missing data (between 50% and 70%) and the high temporal and spatial variability that characterizes the water-quality variables. The competing algorithms implemented belonged to both univariate and multivariate imputation methods (inverse distance weighting (IDW), Random Forest Regressor (RFR), Ridge (R), Bayesian Ridge (BR), AdaBoost (AB), Hubber Regressor (HR), Support Vector Regressor (SVR), and K-nearest neighbors Regressor (KNNR)). According to the results, more than 76% of the imputation outcomes are considered satisfactory (NSE > 0.45). The imputation performance shows better results at the monitoring stations located inside the reservoir than the ones positioned along the mainstream. IDW was the most chosen model for data imputation.

In the rapidly evolving field of data management, numerous terminologies, such as data warehouse, data lake, data lakehouse, and data mesh , have emerged, each representing a unique analytical data architecture. However, the distinctions and similarities among these paradigms often remain unclear. The present paper aimed to navigate the data architecture landscape by conducting a comparative analysis of these paradigms. The analysis a identified and elucidated the differences and similari- ties in features, capabilities, and limitations of these architectural constructs. The study outcome serves as a comprehensive guide, assisting practitioners in selecting the most suitable analytical data architecture for their specific applications.

Conducting quantitative research involving adolescents demands a thoughtful approach to the question of supervision, given that each option comes with its distinct set of implications. This study reviews these implications and empirically tests whether differences in data quality can be found among three modes of standardized survey research with medium sized groups of adolescents (12-17 years). Data basis is a quasi-experimental survey study testing different forms of digital, hybrid or in-person supervision, that took place in 2021 in secondary schools in Germany (N=923). Aim of this study is testing how aspects of data quality – item-nonresponse, interview duration, drop-out-rate and response patterns – differ between these forms of supervision. Results could help researchers surveying young people to decide (1) whether they allow confidants or other adults to be present during interviews, (2) if they can rely on teachers alone when surveying classrooms and (3) if it is cost-efficient to send out external supervisors for classroom sessions. While dropout-rates do not differ, item non-response, interview duration, and response patterns differ significantly, students supervised at home by external interviewers answered more questions, took more time to answer and are less likely to give potentially meaningless answers in grid questions. The implications drawn from the findings question the common approach to solely rely on teachers for survey administration without the support of external supervisors or adequate training. Recruiting respondents via schools and surveying them online in their homes during school hours shows to be a robust method with regard to the analyzed indicators.

Timely and accurate information on tree species is crucial for the sustainable management of natural resources, forest inventory, biodiversity detection, and carbon stock calculation. The advancement of remote sensing technology and artificial intelligence has facilitated the acquisition and analysis of remote sensing data, resulting in more precise and effective classification of tree species. Multimodal remote sensing data and deep learning seem to be the current tree species classification research mainstream, whether or not. The current review on the remote sensing data and deep learning tree species classification methods perspectives to analyze the unimodal and multimodal remote sensing data and classification methods in this realm is missing. To bridge the gap, we search for major trends in the remote sensing data and tree species classification methods, provide a detailed overview of classic deep learning-based methods for tree species classification, and discuss the limitations.

As public health laboratories expand their genomic sequencing and bioinformatics capacity for the surveillance of different pathogens, labs must carry out robust validation, training, and optimization of wet- and dry-lab procedures. Achieving these goals for algorithms, pipelines and instruments often requires that lower-quality datasets be made available for analysis and comparison alongside those of higher-quality. This range of data quality in reference sets can complicate the sharing of sub-optimal datasets that are vital for the community and for the reproducibility of assays. Sharing of useful, but sub-optimal datasets requires careful annotation and documentation of known issues to enable appropriate interpretation, avoid being mistaken for better quality information, and for these data (and their derivatives) to be easily identifiable in repositories. Unfortunately, there are currently no standardized attributes or mechanisms for tagging poor-quality datasets, or datasets generated for a specific purpose, to maximize their utility, searchability, accessibility and reuse. The Public Health Alliance for Genomic Epidemiology (PHA4GE) is an international community of scientists from public health, industry and academia focused on improving the reproducibility, interoperability, portability and openness of public health bioinformatic software, skills, tools and data. To address the challenges of sharing lower quality datasets, PHA4GE has developed a set of standardized contextual data tags, namely fields and terms, that can be included in public repository submissions as a means of flagging pathogen sequence data with known quality issues, increasing their discoverability. The contextual data tags were developed through consultations with the community including input from the International Nucleotide Sequence Data Collaboration (INSDC), and have been standardized using ontologies, community-based resources for defining the tag properties and the relationships between them. The standardized tags are agnostic to the organism and the sequencing technique used and thus can be applied to data generated from any pathogen using an array of sequencing techniques. The list of standardized tags is maintained by PHA4GE and can be found at https://github.com/pha4ge/contextual_data_QC_tags. Definitions, ontology IDs, examples of use, as well as a JSON representation, are provided. The PHA4GE QC tags were tested, and are now implemented, by the FDA’s GenomeTrakr laboratory network as part of its routine submission process for SARS-CoV-2 wastewater surveillance. We hope that these simple, standardized tags will help improve communication regarding quality control in public repositories, in addition to making datasets of variable quality more easily identifiable. Suggestions for additional tags can be submitted to PHA4GE via the New Term Request Form in the GitHub repository. By providing a mechanism for feedback and suggestions, we also expect that the tags will evolve with the needs of the community.

Crowdsourced Data (CSD) generated by citizens is becoming more popular as its potential utilisation in many applications is increasing due to its currency and availability. However, the quality of CSD, including its relevance, is often questioned as the data is not generated by professionals nor follows standard data collection procedures. The quality of CSD can be assessed according to a range of attributes including its relevance. Information relevance has been explored through using in Geographic Information Retrieval (GIR) techniques to identify relevant information. This research tested a relevance assessment approach for CSD by adapting relevance assessment techniques available in the GIR domain. The thematic and geographic relevance were assessed using the Term Frequency-Inverse Document Frequency (TF-IDF), Vector Space Model (VSM) and Natural Language Processing (NLP) techniques. The thematic and geographic specificities of the queries were calculated as 0.44 and 0.67 respectively, which indicates the queries used were more geographically specific than thematically specific. The Spearman's rho value of 0.62 indicated that the final ranked relevance lists showed reasonable agreement with a manually classified list and confirmed the potential of the approach for CSD relevance assessment for other possible crowdsourced data analysis.

Nowadays, one of the greatest challenges in Data Meshes revolves around detecting and creating Data Domains and Data Products for providing the ability to adapt easily and quickly to changing business needs. This requires a disciplined approach to identify, differentiate and prioritize distinct data sources according to their content and diversity. The current paper tackles this highly com-plicated issue and suggests a standardized approach that integrates the concept of Data Blueprints with Data Meshes. In essence, a novel standardization framework is proposed that creates Data Products using a metadata semantic enrichment mechanism, the latter also offering Data Domain readiness and alignment. The approach is demonstrated using real-world data produced by mul-tiple sources in a poultry meat production factory. A set of functional attributes is used to compare qualitatively the proposed approach against existing data structures utilized in storage architectures with quite promising results. Finally, experimentation with different scenarios varying in data product complexity and granularity suggests successful performance.

1. Background Septic shock is a rare and serious infectious disease. Patients surviving the early phase are at high risk of multiple amputations of limbs. They are potential sources of serious disability. 2. Methods The objective was to describe the functional prognosis and quality of life of the four-limb amputee patients following septic shock. This is a descriptive, retrospective and mono-centric study. The patients included between 2010 and 2019 had septic shock after 18 years, responsible for amputation of the four limbs. We studied the mean age of septic shock, the causative organisms, the infectious door enter, the different levels of amputation, the fitting process, the resumption of walking, the Functional Independence Measure (FIM) and the SF-12 Quality of Life Questionnaire. 3. Results We suceed to include 15 patients.. 53.3% women and 46.7% men with an average age of 46.1 years. 80% presented bilateral trans-tibial amputations, all were able to return walking even over short distances, within an average of 8.4 months. 40% did not require a technical walking aid, 33.3% had no walking perimeter limitation. 47% had a higher mental and social quality of life score than the general population. The younger the patients, the better their physical quality of life. After two years of follow-up, the younger the patients, the better their functional autonomy. Patients with better functional autonomy had a better physical quality of life. 4. Conclusion The number of amputations due to spetic shock is likely to increase in the years to come throughout the world. Our study prove that quadri-amputee patient can regain a fully acceptable functional and quality of life being monitored in a specialized health center with multidisciplinary health professional to help him.

The rising importance of Big Data in modern information analysis is supported by vast quantities of user data, but it is only possible to collect sufficient data for all tasks within certain data-gathering contexts. There are many cases where a domain is too novel, too niche, or too sparsely collected to adequately support Big Data tasks. To remedy this, we have created ADG Engine that allows for the generation of additional data that follows the trends and patterns of the data that’s already been collected. Using a database structure that tracks users across different activity types, ADG Engine can use all available information to maximize the authenticity of the generated data. Our efforts are particularly geared towards data analytics by identifying abnormalities in the data and allowing the user to generate normal and abnormal data at custom ratios. In situations where it would be impractical or impossible to expand the available dataset by collecting more data, it can still be possible to move forward with algorithmically expanded data datasets.

Accurate power curve modelling is essential to continuously evaluate the performance of a wind turbine (WT). In this work, we characterize the wind power curves using SCADA data acquired at a frequency of 5 minutes in a wind farm (WF) consisting of 5 WTs. Regarding the non-parametric methods, we select artificial neural networks (ANNs) to make curve estimations. Given that, we have the curves provided by the manufacturer of the WTs given by some very precisely measured pair of wind speed and power points. We can evaluate the difference between the manufacturer characterization and the ones estimated with the data provided by the SCADA system. Before the estimation, we propose a method of filtering the anomalies based on the characteristics provided by the manufacturer. We use three-quarters of the available data for curve estimation and one-quarter for the test. One WT suffered a break in the test part, so we can check how the test estimates reflect this problem in its wind-power curve compared to the estimations obtained in the WTs that worked adequately.

Improvements in web platforms for data exchange and trading are creating more opportunities for users to obtain data from data providers of different domains. However, the current data exchange platforms are limited to unilateral information provision from data providers to users. In contrast, there are insufficient means for data providers to learn what kinds of data users desire and for what purposes. In this paper, we propose and discuss the description items for sharing users’ call for data as data requests in the data marketplace. We also discuss structural differences in data requests and providable data using variables, as well as possibilities of data matching. In the study, we developed an interactive platform, treasuring every encounter of data affairs (TEEDA), to facilitate matching and interactions between data providers and users. The basic features of TEEDA are described in this paper. From experiments, we found the same distributions of the frequency of variables but different distributions of the number of variables in each piece of data, which are important factors to consider in the discussion of data matching in the data marketplace.

In this paper, we present a method to map information regarding service activity provision residing in governmental portals across European Commission. In order to perform this, we used as a basis the enriched Greek e-GIF ontology, modeling concepts, and relations in one of the two data portals (i.e., Points of Single Contacts) examined, since relevant information on the second was not provided. Mapping consisted in transforming information appearing in governmental portals in RDF format (i.e., as Linked data), in order to be easily exchangeable. Mapping proved a tedious task, since description on how information is modeled in the second Point of Single Contact is not provided and must be extracted in a manual manner.

One of the challenges of deploying Wireless Sensor Networks (WSN) at large scale is to operate for prolonged periods of time relying on the limited amounts of energy stored in its nodes. Generally, data communication is the main factor of draining energy. Consequently, reducing the amount of transmitted data by the nodes is of great interest. At the same time, reducing data to be transmitted should not decrease the quality of the deduced information. Solving this problematic is what we call in this article "Ensuring the Quality of Information Service (QoIS)". This document provides a state-of-the-art for the QoIS regarding WSNs. At first, we give a quick overview of the Internet of Things (IoT) paradigm and the WSN technologies in order to understand how data is gathered and used in these networks. Also, we introduce a novel concept: QoIS, in contrast with classic QoS. Then, it is expected a summary of criteria that asses the QoIS in WSNs plus some techniques to improve it. In this article, we will focus on data management techniques. These techniques can improve raw data, information and/or knowledge quality in energy-constrained WSNs. Finally, we give some conclusions and a glimpse about our future work.

This article presents an analysis of monthly precipitation totals based on the Global Precipitation Climatology Centre database and monthly mean temperatures based on National Oceanic and Atmospheric Administration data for 377 catchments located around the world. Data sequences covering 110 years from 1901 to 2010 were analysed. The long-term sequences of precipitation and temperature were used to assess the variability of climate extremes, referred to here as polarisation. Measures of polarisation used in the natural sciences are discussed. A simple measure of polarisation was presented and applied to long-term sequences of monthly precipitation totals and monthly mean temperatures. Due to the nature of the proposed polarisation measure, other characteristics of precipitation and temperature sequences are also presented as a background for the discussion of the polarisation index. The study showed the existence of trends related to the polarisation of temperature and precipitation phenomena. As a result of the analysis, trends of polarisation factors in the area of precipitation and temperature were identified in 11 catchments out of the 377 analysed catchments. The trend analysis used Mann-Kendall tests at a level of significance of 5%. The Pettitt test was used to determine the point of trend change for precipitation and temperature data. The whole investigation is supported by rich graphical analyses, and the results are presented in tabular form.

The effective integration of aerial remote sensing data and ground multi-source data has always been one of the difficulties of quantitative remote sensing. A new monitoring mode is designed which installs the hyperspectral imager on the UAV and places a buoy spectrometer on the river. Water samples are collected simultaneously to obtain in situ assay data of total phosphorus, total nitrogen, COD, turbidity and chlorophyll during data collection. The cross correlogram spectral matching (CCSM) algorithm is used to match the data of the buoy spectrometer with the UAV spectral data to reduce the UAV data noise significantly. An absorption characteristics recognition algorithm (ACR) is designed to realize a new method for comparing UAV data with laboratory data. This method takes into account the spectral characteristics and the correlation characteristics of test data synchronously. It is concluded that the most accurate water quality parameters can be calculated by using the regression method under five scales after the regression tests of multiple linear regression method (MLR), support vector machine method (SVM) and neural network (NN) method. This new working mode of integrating spectral imager data with point spectrometer data will become a trend in water quality monitoring.

Network effects, economies of scale, and lock-in-effects increasingly lead to a concentration of digital resources and capabilities, hindering the free and equitable development of digital entrepreneurship (SDG9), new skills, and jobs (SDG8), especially in small communities (SDG11) and their small and medium-sized enterprises (“SMEs”). To ensure the affordability and accessibility of technologies, promote digital entrepreneurship and community well-being (SDG3), and protect digital rights, we propose data cooperatives [1,2] as a vehicle for secure, trusted, and sovereign data exchange [3,4]. In post-pandemic times, community/SME-led cooperatives can play a vital role by ensuring that supply chains to support digital commons are uninterrupted, resilient, and decentralized [5]. Digital commons and data sovereignty provide communities with affordable and easy access to information and the ability to collectively negotiate data-related decisions. Moreover, cooperative commons (a) provide access to the infrastructure that underpins the modern economy, (b) preserve property rights, and (c) ensure that privatization and monopolization do not further erode self-determination, especially in a world increasingly mediated by AI. Thus, governance plays a significant role in accelerating communities’/SMEs’ digital transformation and addressing their challenges. Cooperatives thrive on digital governance and standards such as open trusted Application Programming Interfaces (APIs) that increase the efficiency, technological capabilities, and capacities of participants and, most importantly, integrate, enable, and accelerate the digital transformation of SMEs in the overall process. This policy paper presents and discusses several transformative use cases for cooperative data governance. The use cases demonstrate how platform/data-cooperatives, and their novel value creation can be leveraged to take digital commons and value chains to a new level of collaboration while addressing the most pressing community issues. The proposed framework for a digital federated and sovereign reference architecture will create a blueprint for sustainable development both in the Global South and North.

The article presents an analysis of monthly precipitation totals based on the GPCC database, and monthly mean temperatures (NOAA data) for 377 catchments distributed across the globe. The analysis included 110-year data sequences from 1901 to 2010 calculated from grid data with a spatial resolution of 0.5°x 0.5° longitude and latitude. Long-term sequences of precipitation and temperature were used to assess the polarization of climatic phenomena. The noticeable impact of polarization in the area of extreme changes in temperature and precipitation is related to anthropogenic factors such as greenhouse gas emissions, deforestation and pollution, which affect ecosystem functioning, biodiversity, water resources and economies. The paper demonstrates the existence of trends related to the polarization of temperature and precipitation phenomena. The measures of polarization used in science are discussed. A simple measure of polarization was proposed and applied to both long-term sequences of monthly precipitation totals and monthly mean temperature. Due to the nature of the proposed polarization measure, other characteristics of the precipitation and temperature sequences are also presented as background for the discussion of the polarization index. The paper presents, for a selection of several hundred catchments from around the world, analyses of the assessment of precipitation and temperature trends using non-parametric tests. The trend analyses use Mann -Kendall tests at the 5% significance level. A Pettitt test was used to determine the trend change point for precipitation and temperature data. The whole is supported by rich graphical analyses and the results are presented tabularly.

Plant Factory Transplanter (PFT) is a key component of the plant factory system. Its operation status directly affects the quality and survival rate of planted seedlings, which in turn affects the overall yield and economic efficiency. To monitor the operation status and transplanting quality of a transplanting machine in a timely manner, the primary task is to use a computerized and easy-to-use method to monitor the transplanting units. Inspired by the latest developments in augmented reality and robotics, a DT model-based and data-driven online monitoring method for plant factory transplanting equipment is proposed. First, a data-driven and virtual model approach is combined to construct a multi-domain digital twin of the transplanting equipment. Then, taking the vibration frequency domain signal above the transplanting manipulator and the image features of the transplanting seedling tray as input variables, the evaluation method and configuration method of the PFT digital twin system are proposed. Finally, the effect of the transplanter is evaluated, and the cycle can be repeated to optimize the transplanter to achieve optimal operation parameters. The results show that the digital twin model can effectively use the sensor data to identify the mechanical vibration characteristics and avoid affecting transplanting quality due to mechanical resonance. At a transplanting rate of 3000 plants/h, the transplanting efficiency can be maintained at a high level and the vibration signal of X, Y, Z-axis above the transplanting manipulator is relatively calm. In this case, Combined the optimal threshold method with the traditional Wiener algorithm, the identification rate of healthy potted seedlings can reach 94.3%. Through comprehensively using the optimal threshold method and 3D block matching filtering algorithm for image threshold segmentation and denoising, the recognition rate of healthy seedlings has reached over 96.10%. In addition, the developed digital twin can predict the operational efficiency and optimal timing of the detected transplanter, even if the environmental and sensor data are not included in the training. The proposed digital twin model can be used for damage detection and operational effectiveness assessment of other plant factory equipment structures.

Cloud computing refers to applications delivered as services over the Internet. Cloud systems employ policies that are inherently dynamic in nature and that depend on temporal conditions defined in terms of external events, such as the measurement of bandwidth, use of hosts, intrusion detection or specific time events. In this paper, we investigate an optimized resource management scheme named v-Mapper. The basic premise of v-Mapper is to exploit application-awareness concepts using software-defined networking (SDN) features. This paper makes three key contributions to the field: (1) We propose a virtual machine (VM) placement scheme that can effectively mitigate the VM placement issues for data-intensive applications; (2) We propose a validation scheme that will ensure that a service is entertained only if there are sufficient resources available for its execution and (3) We present a scheduling policy that aims to eliminate network load constraints. An evaluation was carried out with various benchmarks and demonstrated that v-Mapper shows improved performance over other state-of-the-art approaches in terms of average task completion time, service delay time and bandwidth utilization. Given the growing importance of supporting large-scale data processing and analysis in datacentres, the v-Mapper system has the potential to make a positive impact in improving datacentre performance in the future.

In recent years, more and more health data are being generated. These data come not only from professional health systems, but also from wearable devices. All these data combined form ‘big data’ that can be utilized to optimize treatments for each unique patient (‘precision medicine’). To achieve this precision medicine, it is necessary that hospitals, academia and industry work together to bridge the ‘valley of death’ of translational medicine. However, hospitals and academia often have problems with sharing their data, even though the patient is actually the owner of his/her own health data, and the sharing of data is associated with increased citation rate. Academic hospitals usually invest a lot of time in setting up clinical trials and collecting data, and want to be the first ones to publish papers on this data. The idea that society benefits the most if the patient’s data are shared as soon as possible so that other researchers can work with it, has not taken root yet. There are some publicly available datasets, but these are usually only shared after studies are finished and/or publications have been written based on the data, which means a severe delay of months or even years before others can use the data for analysis. One solution is to incentivize the hospitals to share their data with (other) academic institutes and the industry. Here we discuss several aspects of data sharing in the medical domain: publisher requirements, data ownership, support for data sharing, data sharing initiatives and how the use of federated data might be a solution. We also discuss some potential future developments around data sharing.

In the information age today, data are getting more and more important. While other industries achieve tangible improvement by applying cutting edge information technology, the construction industry is still far from being enough. Cost, schedule, and performance control are three major functions in the project execution phase. Along with their individual importance, cost-schedule integration has been a significant challenge over the past five decades in the construction industry. Although a lot of efforts have been put into this development, there is no method used in construction practice. The purpose of this study is to propose a new method to integrate cost and schedule data using big data technology. The proposed algorithm is designed to provide data integrity and flexibility in the integration process, considerable time reduction on building and changing database, and practical use in a construction site. It is expected that the proposed method can transform the current way that field engineers regard information management as one of the troublesome tasks in a data-friendly way.

Estimating ground surface PM2.5 with fine spatiotemporal resolution is a critical technique for exposure assessments in epidemiological studies of its health risks. Previous studies have utilized monitoring, satellite remote sensing or air quality modeling data to evaluate the spatiotemporal variations of PM2.5 concentrations, but such studies rarely combined these data simultaneously. We develop a three-stage model to fuse PM2.5 monitoring data, satellite-derived aerosol optical depth (AOD) and community multi-scale air quality (CMAQ) simulations together and apply it to estimate daily PM2.5 at a spatial resolution of 0.1˚ over China. Performance of the three-stage model is evaluated using a cross-validation (CV) method step by step. CV results show that the finally fused estimator of PM2.5 is in good agreement with the observational data (RMSE = 23.00 μg/m^3 and R2 = 0.72) and outperforms either AOD-retrieved PM2.5 (R2 = 0.62) or CMAQ simulations (R2 = 0.51). According to step-specific CVs, in data fusion, AOD-retrieved PM2.5 plays a key role to reduce mean bias, whereas CMAQ provides all-spacetime-covered predictions, which avoids sampling bias caused by non-random incompleteness in satellite-derived AOD. Our fused products are more capable than either CMAQ simulations or AOD-based estimates in characterizing the polluting procedure during haze episodes and thus can support both chronic and acute exposure assessments of ambient PM2.5. Based on the products, averaged concentration of annual exposure to PM2.5 was 55.75 μg/m3, while averaged count of polluted days (PM2.5 > 75 μg/m3) was 81, across China during 2014. Fused estimates will be publicly available for future health-related studies.

In recent years, governments applied intelligent transportation system (ITS) technique to provide several convenience services (e.g., garbage truck app) for residents. This study proposes a garbage truck fleet management system (GTFMS) and data feature selection and data clustering methods for travel time prediction. A GTFMS includes mobile devices (MD), on-board units, fleet management server, and data analysis server (DAS). When user uses MD to request the arrival time of garbage truck, DAS can perform the procedure of data feature selection and data clustering methods to analyses travel time of garbage truck. The proposed methods can cluster the records of travel time and reduce variation for the improvement of travel time prediction. After predicting travel time and arrival time, the predicted information can be sent to user’s MD. In experimental environment, the results showed that the accuracies of previous method and proposed method are 16.73% and 85.97%, respectively. Therefore, the proposed data feature selection and data clustering methods can be used to predict stop-to-stop travel time of garbage truck.

Soil quality is related to food security and human survival and development. In recent years, due to the acceleration of urbanization and the increase of abandoned land, land degradation occurs, poor topsoil quality. In this study, the minimum data set (MDS) was constructed through principal component analysis (PCA) to determine the indicator data set for evaluating topsoil quality in Tieling County, China. In addition, the soil quality index (SQI) was calculated to analyze the spatial distribution characteristics and influencing reasons of topsoil quality in Tieling County. The re-sults showed that MDS included total potassium (TK), Clay, zinc (Zn), soil organic matter (SOM), soil water content (SWC), cation exchange capacity (CEC), pH, and copper (Cu). The MDS indicators can well replace all indicators to evaluate the topsoil quality in the study area. The overall soil quality of Tieling County showed a trend of low in the east and high in the west, and gradually increased from the hilly area to the plain area. The evaluation results are consistent with field research, which can provide reference for other topsoil quality evaluation, and it also provide a basis for the formulation of soil quality improvement measures.

Data Meshes is an approach to data architecture and organization that treats data as a product and focuses on decentralizing data ownership and access. It has recently emerged as a field that presents quite a few challenges related to data ownership, governance, security, monitoring, and observability. To address these challenges, this paper introduces an innovative algorithmic framework leveraging data blueprints to enable the dynamic creation of Data Meshes and Data Products in response to user requests, ensuring that stakeholders will have access to specific portions of the Data Mesh as needed. Ownership and governance concerns are addressed through a unique mechanism involving Blockchain and Non-Fungible Tokens (NFTs). This facilitates secure and transparent transfer of data ownership, with the ability to mint time-based NFTs. By combining these advancements with the fundamental tenets of Data Meshes, this research offers a comprehensive solution to the challenges surrounding data ownership and governance. It empowers stakeholders to navigate the complexities of data management within a decentralized architecture, ensuring a secure, efficient, and user-centric approach to data utilization. The proposed framework is demonstrated using real-world data from a poultry meat production factory.

This paper presents the results of a recent study that was conducted with a number of German municipalities/cities. Based on the obtained and briefly presented recommendations emerging from the study, the authors propose the concept of an Urban Data Space (UDS), which facilitates an eco-system for data exchange and added value creation thereby utilizing the various types of data within a smart city/municipality. Looking at an Urban Data Space from within a German context and considering the current situation and developments in German municipalities, this paper proposes a reasonable classification of urban data that allows to relate the various data types to legal aspects and to conduct solid considerations regarding technical implementation designs and decisions. Furthermore, the Urban Data Space is described/analyzed in detail, and relevant stakeholders are identified, as well as corresponding technical artifacts are introduced. The authors propose to setup Urban Data Spaces based on emerging standards from the area of ICT reference architectures for Smart Cities, such as DIN SPEC 91357 “Open Urban Platform” and EIP SCC. Thereby, the paper walks the reader through the construction of an UDS based on the above mentioned architectures and outlines all the goals, recommendations and potentials, which an Urban Data Space can reveal to a municipality/city.

The use of hyperthermia is one of the effective methods of treating cancer. In this case, the problem arises of constructing and predicting the distribution of the thermal field inside tissues depending on their type. This paper discusses the use of linear and polynomial regression methods, as well as regression algorithms based on decision tree, random forest and K nearest neighbors methods to recover missing temperature values. The influence of the size of the training sample when performing regression on the quality of the reconstructed values is investigated. Experimental recommendations on sample size are given for the selected decision tree and random forest methods. The possibility of a significant (10 times or more) reduction in the initial sample size, without leading to a decrease in the R² metric by more than 0.05 for various tissues, has been shown.

Modern experimental astroparticle physics features large-scale setups measuring different messengers, namely high-energy particles generated by cosmic accelerators (e.g. supernova remnants, active galactic nuclei, etc): cosmic and gamma rays, neutrinos and recently discovered gravitational waves. Ongoing and future experiments are distributed over the Earth including ground, underground/underwater setups as well as balloon payloads and spacecrafts. The data acquired by these experiments have different formats, storage concepts and publication policies. Such differences are a crucial issue in the era of big data and of multi-messenger analysis strategies in astroparticle physics. We propose a service ASTROPARTICLE.ONLINE in the frame of which we develop an open science system which enables to publish, store, search, select and analyse astroparticle physics data. The cosmic-ray experiments KASCADE-Grande and TAIGA were chosen as pilot experiments to be included in this framework. In the first step of our initiative we will develop and test the following components of the full data life cycle concept: (i) describing, storing and reusing of astroparticle data; (ii) software for performing multi-experiment and multi-messenger analyses like deep-learning methods; (iii) outreach including example applications and tutorial for students and scientists outside the specific research field. In the present paper we describe the concepts of our initiative, and in particular the plans toward a common, federated astroparticle data storage.

Although other methods exist to store and manage data in modern information technology, the standard solution are file systems. Therefore keeping well-organized file structures and file system layouts can be key to a sustainable research data management infrastructure. However, file structures alone are lacking several important capabilities for FAIR data management: The two most striking are insufficient visualization of data and inadequate possibilities for searching and getting an overview. Research data management systems (RDMS) can fill this gap, but many do not support the simultaneous use of the file system and the RDMS. This simultaneous use can have many benefits, but keeping data in the RDMS in synchrony with the file structure is challenging. Here, we present concepts that allow to keep file structures and semantic data models (in RDMS) synchronous. Furthermore, we propose a specification in yaml-format that allows for a structured and extensible declaration and implementation of a mapping between the file system and data models used in semantic research data management. Implementing these concepts will facilitate the re-use of specifications for multiple use cases. Furthermore, the specification can serve as a machine-readable and, at the same time, human-readable documentation of specific file system structures. We demonstrate our work using the Open Source RDMS CaosDB.

The implementation of the European Cohesion Policy aiming at fostering regions competitiveness, economic growth and creation of new jobs is documented over the period 2014–2020 in the publicly available Open Data Portal for the European Structural and Investment funds. On the base of this source, this paper aims at describing the process of data mining and visualization for information production on regional programmes performace in achieving effective expenditure of resouces.

The present high-tech landscape has allowed institutes to undergo digital transformation in addition to the storing of exceptional bulks of information from several resources, such as mobile phones, debit cards, GPS, transactions, online logs, and e-records. With the growth of technology, big data has grown to be a huge resource for several corporations that helped in encouraging enhanced strategies and innovative enterprise prospects. This advancement has also offered the expansion of linkable data resources. One of the famous data sources is social media platforms. Ideas and different types of content are being posted by thousands of people via social networking sites. These sites have provided a modern method for operating companies efficiently. However, some studies showed that social media platforms can be a source for misinformation at which some users tend to misuse social media data. In this work, the ethical concerns and conduct in online communities has been reviewed in order to see how social media data from different platforms has been misused, and to highlight some of the ways to avoid the misuse of social media data.

With an increasing amount of biological data available publicly, there is a need for a guide on how to successfully download and use this data. The Ten simple rules for using public biological data are: 1) use public data purposefully in your research, 2) evaluate data for your use case, 3) check data reuse requirements and embargoes, 4) be aware of ethics for data reuse, 5) plan for data storage and compute requirements, 6) know what you are downloading, 7) download programmatically and verify integrity, 8) properly cite data, 9) make reprocessed data and models Findable, Accessible, Interoperable, and Reusable (FAIR) and share, and 10) make pipelines and code FAIR and share. These rules are intended as a guide for researchers wanting to make use of available data and to increase data reuse and reproducibility.

The aim of this article is to summarize recent bioinformatic and statistical developments applicable to NMR-based metabolomics. Extracting relevant information from large multivariate datasets by statistical data analysis strategies may be of considerable complexity. Typical tasks comprise for example classification of specimens, identification of differentially produced metabolites, and estimation of fold changes. In this context it is of prime importance to minimize contributions from unwanted biases and experimental variance prior to these analyses. This is the goal of data normalization. Therefore, special emphasize is given to different data normalization strategies. In the first part, we will discuss the requirements and the pros and cons for a variety of commonly applied strategies. In the second part, we will concentrate on possible solutions in case that the requirements for the standard strategies are not fulfilled. In the last part, very recent developments will be discussed that allow reliable estimation of metabolic signatures for sample classification without prior data normalization. In this contribution special emphasis will be given to techniques that have worked well in our hands.

Industrial Control Systems (ICS) are integral to managing and optimizing processes in various industries, including manufacturing, power generation, and more. However, the scarcity of widely adopted ICS datasets hampers research efforts in areas like optimization and security. This scarcity arises due to the substantial cost and technical expertise required to create physical ICS environments. In response to these challenges, this paper presents a groundbreaking approach to generating synthetic ICS data through a data-driven ICS network simulation. We circumvent the need for expensive hardware by recreating the entire ICS environment in software. Moreover, rather than manually replicating the control logic of ICS components, we leverage existing data to autonomously generate control logic. The core of our method involves the stochastic setting of setpoints, which introduces randomness into the generated data. Setpoints serve as target values for controlling the operation of the ICS process. This approach enables us to augment existing ICS datasets and cater to the data requirements of machine learning-based ICS intrusion detection systems and other data-driven applications. Our simulated ICS environment employs virtualized containers to mimic the behavior of real-world PLCs and SCADA systems, while control logic is deduced from publicly available ICS datasets. Setpoints are generated probabilistically to ensure data diversity. Experimental results validate the fidelity of our synthetic data, emphasizing its ability to closely replicate temporal and statistical characteristics of real-world ICS networks. In conclusion, this innovative data-driven ICS network simulation offers a cost-effective and scalable solution for generating synthetic ICS data. It empowers researchers in the field of ICS optimization and security with diverse, realistic datasets, furthering advancements in this critical domain. Future work may involve refining the simulation model and exploring additional applications for synthetic ICS data.

The study investigated the usage of open data in public organizations, focusing on the motives, preferences, and governance ideals that determine their quality. The study shed light on how open data platforms are created and improved, as well as their influence on government operations. The primary goal was to understand the motivations behind the usage of open data, evaluate gov-ernance ideals, and examine institutions' efforts to standardize data formats. Hierarchical regres-sion analysis divides variables into core reasons and different preferences, demonstrating a need for "diversification" in order to use open data effectively. Efficiency, accountability, community impact, knowledge, and high-quality service delivery are all emphasized in governance value analysis. The institution's dedication to standardizing open data formats is evidenced by legislative measures and the integration of many technologies. The findings illustrate the institution's dual strategy of em-ploying traditional techniques and new technologies for effective data management. The report suggests increasing data openness and standardization, investing in ongoing training, upgrading alerting and awareness systems, and adopting specific regulations for open data use. Monitoring and feedback techniques are critical for adjusting and refining open data services to satisfy de-velopment needs. This study adds to the scientific knowledge of open data use by highlighting important motives, preferences, and governance ideals. The offered guidelines provide practical insights for public organizations looking to improve their open data platforms by promoting openness, efficiency, and community participation.

The study was conducted on applying machine learning and data mining methods to personalizing the treatment. This allows investigating individual patient characteristics. Personalization is built on the clustering method and associative rules. It was suggested to determine the average distance between instances for optimal performance metrics finding. The formalization of the medical data pre-processing stage for finding personalized solutions based on current standards and pharmaceutical protocols is proposed. The model of patient data is built. The paper presents the novel approach to clustering built on ensemble of cluster algorithm with better than k-means algorithm Hopkins metrics. The personalized treatment usually is based on decision tree. Such approach requires a lot of computation time and cannot be paralyzed. Therefore, it is proposed to classify persons by conditions, to determine deviations of parameters from the normative parameters of the group, as well as the average parameters. This made it possible to create a personalized approach to treatment for each patient based on long-term monitoring. According to the results of the analysis, it becomes possible to predict the optimal conditions for a particular patient and to find the medicaments treatment according to personal characteristics.

In the face of escalating cybercriminal sophistication, an innovative approach to network anomaly detection has been pursued in this longitudinal study, integrating computational data analytics in a geographic, organizational, and behavioral context. A data-driven scoring mechanism was employed to systematically analyze and correlate source countries of IP addresses and organization-associated Autonomous System (AS) Numbers (ASN) with network anomalies. Significant correlations between certain countries, specific organizations, and high behavior scores were identified through the data analytics. An increase in connection requests was also found to be linked with elevated behavior scores. Validated by cross-validation techniques, these findings emphasize the necessity for continuous model recalibration. The transformative role of integrative data analytics in cybersecurity is underscored, paving the way for the development of more sophisticated, context-aware anomaly detection systems. Specifically, the analysis underscores the need for organizations to adopt a proactive and adaptive approach to cybersecurity that can keep pace with the evolving threat landscape.

With the spread of compulsory education emerged school management problems continued, and the quality of school management in compulsory education has attracted a great deal of attention in China. However, the application of information technology in the field is not yet detailed and wide, resulting in problems of heavy workload and high difficulty in the whole evaluation process. Accordingly, we use big data technologies such as Apache Spark, Apache Hive, and SPSS to carry out data cleaning, correlation analysis, dynamic factor analysis, principal component analysis, and visual display on 1760 sample data from 40 primary and secondary schools in Q Province in China, and constructs a model school management of quality evaluation in the compulsory education stage, which reduces the 22 management tasks required for previous evaluation to 5, greatly reducing the workload and difficulty of evaluation. It has improved the efficiency and accuracy of evaluation, and further promoted the simultaneous development of education of five domains and education equity in the compulsory education stage.

In Data Science we are concerned with the integration of relevant sciences in observed and empirical contexts. This results in the unification of analytical methodologies, and of observed and empirical data contexts. Given the dynamic nature of convergence, described are the origins and many evolutions of the Data Science theme. The following are covered in this article: the rapidly growing post-graduate university course provisioning for Data Science; a preliminary study of employability requirements, and how past eminent work in the social sciences and other areas, certainly mathematics, can be of immediate and direct relevance and benefit for innovative methodology, and for facing and addressing the ethical aspect of Big Data analytics, relating to data aggregation and scale effects. Associated also with Data Science is how direct and indirect outcomes and consequences of Data Science include decision support and policy making, and both qualitative as well as quantitative outcomes. For such reasons, the importance is noted of how Data Science builds collaboratively on other domains, potentially with innovative methodologies and practice. Further sections point towards some of the most major current research issues.

With the fast development of blockchain technology in latest years, its application in scenarios that require privacy, such as health area, became encouraged and widely discussed. This paper presents an architecture to ensure the privacy of health-related data, which are stored and shared within a blockchain network in a decentralized manner, through the use of encryption with the RSA, ECC and AES algorithms. Evaluation tests were performed to verify the impact of cryptography on the proposed architecture in terms of computational effort, memory usage and execution time. The results demonstrate an impact mainly on the execution time and on the increase in the computational effort for sending data to the blockchain, however, justifiable considering the privacy and security provided with the architecture and encryption.

The scientific community has long benefited from the opportunities provided by data reuse. Recognizing the need to identify the challenges and bottlenecks to reuse in the agricultural research community and propose solutions for them, the data reuse working group was started within the AgBioData consortium (https://www.agbiodata.org/) framework. Here, we identify the limitations of data standards, metadata deficiencies, data interoperability, data ownership, data availability, user skill level, resource availability, and equity issues, with a specific focus on agricultural genomics research. We propose possible solutions stakeholders could implement to mitigate and overcome these challenges and provide an optimistic perspective on the future of genomics and transcriptomics data reuse.

To gain insight into the real-life care of patients in the healthcare system, data from hospital information systems and insurance systems are required. Consequently, linking clinical data with claims data is necessary. To ensure their syntactic and semantic interoperability, the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) was chosen. However, there is no detailed guide that would allow researchers to follow a consistent process for data harmonization. Thus, the aim of this paper is to conceptualize a generic data harmonization process for OMOP CDM. For this purpose, we conducted a literature review focusing on publications that address the harmonization of clinical or claims data in OMOP CDM. Subsequently, the process steps used and their chronological order were extracted for each included publication. The results were then compared to derive a generic sequence of the process steps. From 23 publications included, a generic data harmonization process for OMOP CDM was conceptualized, consisting of nine process steps: dataset specification, data profiling, vocabulary identification, coverage analysis of vocabularies, semantic mapping, structural mapping, extract-transform-load-process, qualitative and quantitative data quality analysis. This process can be used as a step-by-step guide to assist other researchers in harmonizing source data in OMOP CDM.

Educational data-mining is an evolving discipline that focuses on the improvement of self-learning and adaptive methods. It is used for finding hidden patterns or intrinsic structures of educational data. In the arena of education, the heterogeneous data is involved and continuously growing in the paradigm of big-data. To extract meaningful information adaptively from big educational data, some specific data mining techniques are needed. This paper presents a clustering approach to partition students into different groups or clusters based on their learning behavior. Furthermore, personalized e-learning system architecture is also presented which detects and responds teaching contents according to the students’ learning capabilities. The primary objective includes the discovery of optimal settings, in which learners can improve their learning capabilities. Moreover, the administration can find essential hidden patterns to bring the effective reforms in the existing system. The clustering methods K-Means, K-Medoids, Density-based Spatial Clustering of Applications with Noise, Agglomerative Hierarchical Cluster Tree and Clustering by Fast Search and Finding of Density Peaks via Heat Diffusion (CFSFDP-HD) are analyzed using educational data mining. It is observed that more robust results can be achieved by the replacement of existing methods with CFSFDP-HD. The data mining techniques are equally effective to analyze the big data to make education systems vigorous.

In the age of the digital economy, the security and privacy for multimedia systems issues are increasingly manifested in the form of data. The level of data protection governance capability determines whether the security and privacy of multimedia systems can be protected. In recent years, China has introduced laws and regulations on the protection of personal information, personal privacy, data security, and cyber-security. Although data protection is still to be improved, China has been refining the provisions on data security management through practice and the adoption of supporting regulations.

Despite the extensive literature on missing data theory and cautionary articles emphasizing the importance of realistic analysis for healthcare data, a critical gap persists in incorporating domain knowledge into missing data problem formulation, assumption specification, and method development. In this paper, we highlight the gap particularly for observational data from healthcare facilities. We address this gap by identifying ten fundamental missingness scenarios arising during data measurement, recording, and pre-processing in observational health data, influenced by physicians, patients, healthcare facilities, and data scientists. We analyze the effect of scenarios on estimand formulation, missing data identification, estimation, and sensitivity analysis. To emphasize how domain-informed analysis can improve method reliability, we conduct simulation studies under the influence of various missingness scenarios. We compare the results of three common methods in medical data analysis (complete-case analysis, Missforest imputation, and inverse probability weighting estimation) for two estimands (variable mean estimation and classification accuracy). We advocate for our analysis approach as a reference for the analysis of observational health data. Furthermore, we posit that the proposed analysis framework is applicable to other medical domains, including medical wearable data analysis.

It is strongly believed that technology can reap the best only when it can be tamed by all stakeholders. Big data technology has no exception for this and even after a decade of emergence, the technology is still a herculean task and is in nascent stage with respect to applicability for many people. Having understood the gaps in the technology adoption for big data in the contemporary world, the present exploratory research work intended to highlight the possible prospects of big data technologies. It is also advocated as to how the challenges of various fields can be converted as opportunities with the shift in the perspective towards this evolving concept. Examples of apex organizations like (IMF and ITU) and their initiatives of big data technologies with respect to the Sustainable Development Goals (SDGs) are also cited for a broader outlook. The intervention of the responsible organizations along with the respective governments is also much sought for encouraging the technology adoption across all the sections of the market players.

This work develops a new method for vector data augmentation. The proposed method applies the Principal Component Analysis (PCA) and determines the Eigenvectors of a set of training vectors for a Machine Learning (ML) method and uses them to generate the distilled vectors. The training and the PCA distilled vectors have the same dimension. The user chooses the number of vectors to be distilled and augmented to the set of training vectors. A statistical approach determines the lowest number of vectors to be distilled such that when augmented to the original vectors the extended set trains a ML classifier to achieve a required accuracy. Hence, the novelty of this study is, the distillation of vectors with the PCA method and their use to augment the original set of vectors. The advantage that comes from the novelty is that it increases the statistics of ML classifiers. To validate the advantage, we conducted experiments with four public databases and applied four classifiers: Neural Network, Logistic Regression, Support Vector Machine with linear and polynomial kernels. For the purpose of augmentation, we conduct several distillations including nested (double distillation). The latter notion means that new vectors were distilled from already distilled vectors. We train the classifiers with three sets of vectors: original vectors; original augmented with vectors distilled by PCA; original augmented with distilled and double distilled, by PCA, vectors. The experimental results are presented in the paper, and they confirm the advantage that the PCA distilled vectors increase the classification statistics of ML methods, if the distilled vectors augment the original training vectors.

The concentration of data from multiple links to a single output is an essential task performed by High-Energy Physics (HEP) Data Acquisition Systems (DAQs). At high and varying data rates combined with the large width of the concentrator’s output interface, this task is non-trivial. This paper presents a concentrator based on the Beneš network, which provides efficient concentration without using a high-frequency clock internally. It warrants that empty data are eliminated and does not disturb the data time-ordering if the data rates significantly differ between inputs. Additionally, it is well suited to FPGA implementation. It is based on simple data-routing primitives and may be fully pipelined.

The purpose of this paper is to develop WebSecuDMiner algorithm to discover unusual web access patterns based on analysing the potential rules hidden in web server log and user navigation history. Design/methodology/approach: WebSecuDMiner uses equivalence class transformation (ECLAT) algorithm to extract user access patterns from the web log data, which will be used to identify the user access behaviours pattern and detect unusual one. Data extracted from the web serve log and user browsing behaviour is exploited to retrieve the web access pattern that is produced by the same user. Findings: WebSecuDMiner is used to detect whether any unauthorized access have been posed and take appropriate decisions regarding the review of the original rights of suspicious user. Research limitations/implications: The present work uses the database which is extracted from web serve log file and user browsing behaviour. Although the page is viewed by the user, the visit is not recorded in the server log file, since it can be access from the browser's cache.

In this paper, the Shannon entropy measure was used to assess changes in precipitation and temperature conditions. Due to the short, low-volume sequences of precipitation and temperature data analysed, a bootstrap method was used in the procedure for calculating Shannon entropy. The analysis used minimum and maximum values of monthly precipitation totals and monthly mean temperatures for 377 catchments distributed across the globe. A 110-year data series from 1901 to 2010 was analysed. Entropy values for the estimated parameters of the generalised extreme value distribution (GEV) were calculated for the adopted data. Entropy value calculations were performed for the left-hand constraint, based on minimum values, and for the right-hand constraint, based on maximum values. The applicability of Shannon's entropy measure in the analysis of climate change was demonstrated by allowing the degree of disorder and complexity of the distributions describing climate variables in the form of precipitation and temperature to be measured. This made it possible to obtain information on the directions of changes occurring with regard to minimum and maximum values in the field of monthly precipitation and mean temperatures in the analysed catchments. The study demonstrated the existence of Shannon entropy trends. The evaluation of entropy trends for precipitation and temperature sequences was performed using non-parametric tests. Mann -Kendall tests at the 5% significance level were used for trend analyses. The Pettitt test was performed to determine the point of change in trend for the rainfall and temperature data. The performed analysis was supported by graphical presentations.

A medical entity (hospital, nursing home, rest home, revalidation center, etc.) usually includes a multitude of information systems that allow for quick decision-making close to the medical sensors. The Internet of Medical Things (IoMT) is an area of IoT that generates a lot of data of different natures (radio, CT scan, medical reports, medical sensor data). However, these systems need to share and exchange medical information in a seamless, timely, and efficient manner with systems that are either located within the same entity or located in other healthcare entities. The lack of inter and intra entity interoperability causes major problems in the analysis of patient records and leads to additional financial costs (e.g., redone examinations). In order to develop a medical data interoperability architecture model that will allow providers and different actors in the medical community to exchange patient summary information with other caregivers and partners in order to improve the quality of care, the level of data security, and the efficiency of care, we take stock of the state of knowledge.

Health-related data originating from diverse sources are commonly stored in manifold databases and formats, making it difficult to find, access and gather data for research purposes. In addition, so-called secondary use scenarios for health data are usually hindered by local data codes, missing dictionaries and the lack of metadata and context descriptions. Following the FAIR principles (Findable, Accessible, Interoperable and Reusable), we developed a decentralized infrastructure to overcome these hurdles and enable collaborative research by making the meaning of health-related data understandable to both, humans and machines. This infrastructure is currently being implemented in the realm of the Swiss Personalized Health Network (SPHN), a research infrastructure initiative for enabling the use and exchange of health-related data for research in Switzerland. The SPHN ecosystem for FAIR data consists of the SPHN Dataset (semantic definitions), the SPHN RDF Schema (linkage and transport of the semantics in a machine-readable format), a project RDF template, extensive guidelines and conventions on how to generate SPHN RDF schema, a Terminology Service (converter of clinical terminologies in RDF), and a Quality Assurance Framework (automated data validation with SHACLs and SPARQLs). The SPHN ecosystem has been built in a way that it can easily be adapted and extended by any SPHN project to fit individual needs. By providing such a national ecosystem, SPHN supports researchers in generating, processing and sharing FAIR data.

Over the years, there has been a significant rise in the world's scientific knowledge. However, most of it lacks structure and is often termed as Dark Data. Both humans and expert systems have continually faced difficulty in analyzing and comprehending such overwhelming amounts of information which is crucial in solving several real-world problems. Information and data visualization techniques proffer a promising solution to explore such data by allowing quick comprehension of information, the discovery of emerging trends, identification of relationships and patterns, etc. In this tutorial, we utilize the rich corpus of PubMed comprising of more than 30 million citations from biomedical literature to visually explore and understand the underlying key-insights using various information visualization techniques. With this study, we aim to diminish the limitation of human cognition and perception in handling and examining such large volumes of data by speeding up the process of decision making and pattern recognition and enabling decision-makers to fully understand data insights and make informed decisions.

The mobile devices caused a constant struggle for the pursuit of data privacy. Nowadays, it appears that the number of mobile devices in the world is increasing. With this increase and technological evolution, thousands of data associated with everyone are generated and stored remotely. Thus, the topic of data privacy is highlighted in several areas. There is a need for control and management of data in circulation inherent to this theme. This article presents an approach of the interaction between the individual and the public environment, where this interaction will determine the access to information. This analysis was based on a data privacy management model in public environments created after reading and analyzing the current technologies. A mobile application based on location via Global Positioning System (GPS) was created to substantiate this model, which it considers the General Data Protection Regulation (GDPR) to control and manage access to the data of each individual.

One of the most remarkable features in the 20th century was the digitalization of technical progress, which changed the output of companies worldwide and became a defining feature of the century. The growth of information technology systems and the implementation of new technical advances, which enhance the integrity, agility and long-term organizational performance of the supply chain, can distinguish a digital supply chain from other supply chains. For example, the Internet of Things (IoT)-enabled information exchange and Big Data analysis might be used to regulate the mismatch between supply and demand. In order to assess contemporary ideas and concepts in the field of data analysis in the context of supply chain management, this literary investigation has been decided. The research was conducted in the form of a comprehensive literature review. In the SLR investigation, a total of 71 papers from leading journals were used. SLR has found that data analytics integrate into supply chain management can have long-term benefits on supply chain management from the input side, i.e., improved strategic development, management and other areas.

The road traffic injuries risk factors such as driving offenses and average speed are concerns for health organizations to reduce the number of injuries. Without any comprehensive view of each road, one cannot decide about the effective policy. In this manner, the data-driven policy will help to improve and assess the decisions. The count data near the road of two airports is surveyed for investigating the time-varying speed zones. The descriptive statistics, ANOVA, and functional data analysis were used. The hourly data of traffic counts for four different locations at the entrance of the two airports, international and domestics, were collected for one the year 2018 to 2019.The hourly pattern of driving offenses for each road was assessed and the to and from airport roads had different peaks (<0.05). The hour, weekdays, type of airport, direction and their interactions were statistically significant (<0.05) for the chance of driving offenses. The speed average during the day was statistically different (<0.5) by the number of different types of vehicles. The traffic count data is a great resource for decision making in safe driving subjects such as driving offenses. With functional data analysis, we can analyze them to get the most of the characteristics of this data. The airports are public places with high traffic demand in all countries that yields the different pattern of traffic transportation, therefore we extract the factors that affect the driving offenses. Finally, we conclude that conducting a time-varying speed zone near the airports seems vital.

Vienna Atomic Line Database (VALD) contains data on atomic and molecular energy levels and parameters of spectral lines required for stellar spectra analysis. Hundreds of millions lines for fine spectral synthesis and for opacity calculations are collected in present version of VALD (VALD3). Critical evaluation of the data and the diversity of extraction tools support high popularity of VALD among users. The data model of VALD3 incorporates obligatory links to the bibliography making our database more attractive as publishing platform for data producers. The VALD data quality and completeness are constantly improving allowing better reproduction of stellar spectra. To illustrate continuous evolution of the data content we present a comparative analysis of the recent experimental and theoretical atomic data for Fe-group elements, which will be included in the next VALD release. This release will also include a possibility for extracting the line data with full isotopic and hyperfine structures.

In personalized healthcare, an ecosystem for the manipulation of reliable and safe private data should be orchestrated. This paper describes a first approach for the generation of fake electrocardiograms (ECGs) based on Generative Adversarial Networks (GANs) with the objective of anonymizing users’ information for privacy issues. This is intended to create valuable data that can be used both, in educational and research areas, while avoiding the risk of a sensitive data leakage. As GANs are mainly exploited on images and video frames, we are proposing general raw data processing after transformation into an image, so it can be managed through a GAN, then decoded back to the original data domain. The feasibility of our transformation and processing hypothesis is primarily demonstrated. Next, from the proposed procedure, main drawbacks for each step in the procedure are addressed for the particular case of ECGs. Hence, a novel research pathway on health data anonymization using GANs is opened and further straightforward developments are expected.

The increasing rate of adoption of innovative technological achievements along with the penetration of the Next Generation Internet (NGI) technologies and Artificial Intelligence (AI) in the water sector, are leading to a shift to a Water-Smart Society. New challenges have emerged in terms of data interoperability, sharing, and trustworthiness due to the rapidly increasing volume of heterogeneous data generated by multiple technologies. Hence, there is a need for efficient harmonisation and smart modeling of the data to foster advanced AI analytical processes which will lead to efficient water data management. The main objective of this work is to propose two Smart Data Models focusing on the modeling of the Satellite Imaginary data and the Flood Risk Assessment processes. The utilisation of those models reinforces the fusion and homogenisation of diverse information and data facilitating the adoption of AI technologies for flood mapping and monitoring. Furthermore, a holistic framework has been developed and evaluated via qualitative and quantitative performance indicators revealing the efficacy of the proposed models concerning the usage of the models in real cases. The framework is based on the well-known and compatible technologies on NGSI-LD standards which are customised and applicable easily to support the water data management processes effectively.

Background: We aimed to assess the risk of chemotherapy- and radiotherapy-related cognitive impairment in colorectal cancer patients. Methods: We randomly selected 40% of colorectal cancer patients from Korean National Health Insurance Database (NHID), 2004-2018 (N=148,848). Patients with one or more ICD-10 diagnostic codes for dementia or mild cognitive impairment was defined as cognitive impairment cases. Patients who were aged 18 or younger, diagnosed with cognitive impairment before colorectal cancer (N=8,225) and did not receive primary resection (N=45,320) were excluded. The effects of each chemotherapy agent on cognitive impairment were estimated. We additionally estimated the effect of radiotherapy in rectal cancer patients. Time-dependent competing risk Cox regression was conducted to estimate overall and age-specific hazard ratios (HR) separately for colon and rectal cancer. Results: In colon cancer, capecitabine and irinotecan was associated with higher cognitive im-pairment, while 5-fluorouracil was not. In rectal cancer, no chemotherapy agents increased the risk of cognitive impairment, nor did radiotherapy. Hazardous association of irinotecan was estimated larger in elderly patients compared with younger counterparts. Conclusion: Heterogeneous associations between various chemotherapy agents and cognitive impairment were observed. Elderly patients were more vulnerable to possible adverse cognitive effects. Radiotherapy did not increase the risk of cognitive impairment.

This contribution shows the possibilities of a low-cost multipurpose data logger that was built around an Arduino Mega 2560 single board computer. To transform the Arduino Mega into such a data logger, a sensor shield was designed that contains an SD-card, real-time clock, and different kinds of connectors so that sensors can easily be attached to the device. The software considers a wide range of predefined sensors from which a user can choose. To assess the performance of such a device, short-term monitoring campaigns in relation to outdoor air quality, human activity in an office, motion of a journey on a bike, and exhaust gas monitoring of a diesel generator were realized. Besides the possibilities to apply the data logger in different kinds of applications, a method is proposed to assess the credibility of such a system. The assessment based on the various short-term campaigns showed that the system scores positively on most of the performance indicators, but that unexpected (black swan) events affect the assessment over the longer term. This makes the development of low-cost scientific instruments harder than expected. The stability and long-term performance of this type of design requires continuous evaluation and engineering correction.

Social media data is widely used to gain insights about social incidents, whether on a local or global scale. Within the process of analyzing and evaluating the data, it is common practice to download and store it locally. Considerations about privacy protection of social media users are often neglected thereby. However, protecting privacy when dealing with personal data is demanded by laws and ethics. In this paper we introduce a method to store social media data using the cardinality estimator HyperLogLog. Based on an exemplary disaster management scenario, we show that social media data can be analyzed by counting occurrences of posts, without becoming in possession of the actual raw data. For social media data analyses like these, that are based on counting occurrences, cardinality estimation suffices the task. Thus, the risk of abuse, loss or public exposure of the data can be mitigated and privacy of social media users can be preserved. The ability to do unions and intersections on multiple data sets further encourages the use of this technology. We provide a proof-of-concept implementation for our introduced method, using data provided by the Twitter API.

In computer networks, loss of data packets is inevitable, in particular, because of the buffer memory overflow of at least one of the nodes located on the path from the source to the receiver, including the latter. Such losses associated with overflows are hereinafter referred to as congestion of network nodes. There are many ways to prevent and eliminate overloads; these methods, in the majority, are based on the management of data flows. A special place is taken by the maintenance of packages, taking into account their priorities. The article considers a number of original technical solutions to improve the quality of control and reduce the required amount of buffer memory of network nodes. The ideas of these solutions are quite simple for their implementation in the development of appropriate software and hardware for telecommunication devices.

As of 2020, public data for game ratings provided by Game Ratings And Administration Committee(GRAC) are more limited than public data for movie and video ratings provided by Korea Media Ratings Board and do not provide data which allow us to see information on ratings clearly and in detail. To get information on game ratings, we need to find information by searching for specific target on homepage which is inconvenient for us. In order to improve such inconvenience and extend scope of provision in public data, the author of this paper intends to study public data API which has been extended based on information on video ratings. To draw items to be extended, this study analyzes data for ratings on homepage of GRAC and designs collection system to build database. This study intends to implement system that provides data collected based on extended public data items in a form which users want. This study is expected to provide information on ratings to GRAC which will strengthen fairness and satisfy game users and people’s rights to know and contribute to promotion and development of game industry.

The rich emission and absorption line spectra of Fe I may be used to extract crucial information on astrophysical plasmas, such as stellar metallicities. There is currently a lack, in quality and quantity, of accurate level-resolved effective electron-impact collision strengths and oscillator strengths for radiative transitions. Here, we discuss the challenges in obtaining a sufficiently good structure for neutral iron and compare our theoretical fine-structure energy levels with observation for several increasingly large models. Radiative data is presented for several transitions for which the atomic data is accurately known.

Big data analytics tools are the use of advanced analytic techniques targeting large and diverse volumes of data that include structured, semi-structured, and unstructured data from different sources and in different sizes from terabytes to zetabytes. The health sector is faced with the need to generate and manage large data sets from various health systems, such as electronic health records and clinical decision support systems. This data can be used by providers, clinicians, and policymakers to plan and implement interventions, detect disease more quickly, predict outcomes, and personalize care delivery. However, little attention is paid to the connection between big data analytics tools and the health sector. Thus, a systematic review of the bibliometric literature (LRSB) was developed to study how the adoption of big data analytics tools and infrastructures will revolutionize the healthcare industry. The review integrated 77 scientific and/or academic documents indexed in SCOPUS presenting up‐to‐date knowledge on current insights on how big data analytics technologies influence the healthcare sector and the different big data analytical tools used. The LRSB provides findings related to the impact of Big Data analytics on the health sector by introducing opportunities and technologies that provide practical solutions to various challenges.

Big Data in medicine includes possibly fast processing of large data sets, both current and historical in purpose supporting the diagnosis and therapy of patients' diseases. Support systems for these activities may include pre-programmed rules based on data obtained from the interview medical and automatic analysis of test results diagnostic results will lead to classification of observations to a specific disease entity. The current revolution using Big Data significantly expands the role of computer science in achieving these goals, which is why we propose a Big Data computer data processing system using artificial intelligence to analyze and process medical images.

Data from Earth Observation (EO) satellites are increasingly used to monitor the environment, understand variability and change, inform evaluations of climate model forecasts and manage natural resources. Policy makers are progressively relying on the information derived from these datasets to make decisions on mitigating and adapting to climate change. These decisions should be evidence based, which requires confidence in derived products as well as the reference measurements used to calibrate, validate or inform product development. In support of the European Union’s Earth Observation Programmes Copernicus Climate Change Service, the Quality Assurance for Essential Climate Variables (QA4ECV) project fulfilled a gap in the delivery of climate quality satellite derived datasets by prototyping a robust, generic system for the implementation and evaluation of Quality Assurance (QA) measures for satellite-derived ECV climate data record products. The project demonstrated the QA system on six new long-term, climate quality ECV data records for surface Albedo, Leaf Area Index, FAPAR, NO₂, HCHO and CO. Provision of standardized QA information provides data users with evidence-based confidence in the products and enables judgement on the fitness-for-purpose of various ECV data products their specific applications.

The flourishing landscape of agritourism in Albania has witnessed a steady increase in destinations over the years. Recognizing their pivotal role in driving economic development in rural areas, considerable attention is directed towards the promotion and expansion of these agricultural tourism spots. To enhance interactivity with visitors, these agritourism destinations actively leverage social networks and various digital platforms. However, the wealth of information pertaining to these destinations is currently scattered across different platforms, posing a challenge to accessibility. To address this issue, these points are strategically placed on Google Maps, along with essential details collected from users, such as websites, contact numbers, visitor ratings, and comments. Nevertheless, other valuable data remains dispersed on various social networks. In response to this fragmentation, this study proposes the development of algorithms and scripts to systematically collect and analyze this dispersed information. By doing so, the goal is to centralize and streamline the data, enabling a comprehensive understanding of the agritourism landscape in Albania. This approach not only facilitates efficient access to information for potential visitors but also provides valuable insights for stakeholders involved in the development and management of these agritourism destinations. The study aims to contribute to the sustainable growth of agritourism in Albania by harnessing the power of digital platforms to create a unified and organized database. Through this integration, the research seeks to unveil patterns, trends, and opportunities that can further guide policy-making, marketing strategies, and overall decision-making processes in the realm of agritourism development.

Abstract ObjectivesThe aim of this perspective is to report the use of synthetic data as a viable method in women’s health given the current challenges linked to obtaining life-course data within a short period of time and accessing electronic healthcare data. Methods We used a 3-point perspective method to report an overview of data science, common applications, and ethical implications. Results There are several ethical challenges linked to using real-world data, consequently, generating synthetic data provides an alternative method to conduct comprehensive research when used effectively. The use of clinical characteristics to develop synthetic data is a useful method to consider. Aligning this data as closely as possible to the clinical phenotype would enable researchers to provide data that is very similar to that of the real-world. Discussion Population diversity and disease characterisation is important to optimally use data science. There are several artificial intelligence techniques that can be used to develop synthetic data. ConclusionSynthetic data demonstrates promise and versatility when used efficiently aligned to clinical problems. Therefore, exploring this option as a viable method in women’s health, in particular for epidemiology may be useful.

Big Data warehouses are a new class of databases that largely use unstructured and volatile data for analytical purpose. Examples of this kind of data sources are those coming from the Web, such as social networks and blogs, or from sensor networks, where huge amounts of data may be available only for short intervals of time. In order to manage massive data sources, a strategy must be adopted to define multidimensional schemas in presence of fast-changing situations or even undefined business requirements. In the paper, we propose a design methodology that adopts agile and automatic approaches, in order to reduce the time necessary to integrate new data sources and to include new business requirements on the fly. The data are immediately available for analyses, since the underlying architecture is based on a virtual data warehouse that does not require the importing phase. Examples of application of the methodology are presented along the paper in order to show the validity of this approach compared to a traditional one.

Dyadic data contains co-occurrences of objects, which is often modeled by finite mixture model which in turn is learned by expectation maximization (EM) algorithm. Objects in traditional dyadic data are identified by names, causing the drawback which is that it is impossible to extract implicit valuable knowledge under objects. In this research, I propose the so-called attributed dyadic data (ADD) in which each object has an informative attribute and each co-occurrence of two objects is associated with a value. ADD is flexible and covers most of structures / forms of dyadic data. Conditional mixture model (CMM), which is a variant of finite mixture model, is applied into learning ADD. Moreover, a significant feature of CMM is that any co-occurrence of two objects is based on some conditional variable. As a result, CMM can predict or estimate co-occurrent values based on regression model, which extends applications of ADD and CMM.

Cities are undergoing huge shifts in technology and operations in recent days, and `data science' is driving the change in the current age of the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting insights or actionable knowledge from city data and building a corresponding data-driven model is the key to making a city system automated and intelligent. Data science is typically the study and analysis of actual happenings with historical data using a variety of scientific methodology, machine learning techniques, processes, and systems. In this paper, we concentrate on and explore ``Smart City Data Science", where city data collected from various sources like sensors and Internet-connected devices, is being mined for insights and hidden correlations to enhance decision-making processes and deliver better and more intelligent services to citizens. To achieve this goal, various machine learning analytical modeling can be employed to provide deeper knowledge about city data, which makes the computing process more actionable and intelligent in various real-world services of today's cities. Finally, we identify and highlight ten open research issues for future development and research in the context of data-driven smart cities. Overall, we aim to provide an insight into smart city data science conceptualization on a broad scale, which can be used as a reference guide for the researchers, professionals, as well as policy-makers of a country, particularly, from the technological point of view.

The Release of government dataset for public use can potentially strengthen the relationship between the government and its constituents. However, research shows that there are several challenges for open data effectiveness. This paper reviews current determinants and issues associated with the open government data (OGD) procedures. The review concentrates on two ends of the spectrum: First, from the perspective of the preparation by the government, focusing on the organization of traditional governmental datasets and how the recording of the data is administered. Second, from the perspective of the users, focusing on the way in which the data is released to the general public and on human-computer interaction (HCI) issues between end-user and data-consumption interfaces. Following a thorough analysis of these two opposing challenges, the paper proposes approaches to mitigate them. This review and subsequent recommendations contribute and expand current understanding of open government data effectiveness and can lead to public policy changes, development of new procedures and strategies, and ultimately improvements at both ends of the federal open data endeavor.

The capability of extracting information and analyze it into a common format is essential for performing predictions, comparing projects through cost benchmarking, and for having a deeper understanding of the project costs. However, the lack of standardization and the manual inclusion of the data makes this process very time-consuming, unreliable, and inefficient. To tackle this problem, a novel approach with a big impact is presented combining the benefits of data mining, statistics, and machine learning to extract and analyze the information related to railway costs infrastructure data. To validate the suggested approach, data from 23 real historical projects from the client network rail was extracted, allowing their costs to be comparable. Finally, some machine learning and data analytics methods were implemented to identify the most relevant factors allowing for costs benchmarking. The presented method proves the benefits of data extraction being able to gather, analyze and benchmark each project in an efficient manner, and deeply understand the relationships and the relevant factors that matter in infrastructure costs.

Nowadays, information security is a challenge especially when transmitted or shared in public clouds. Many of researchers have been proposed technique which fails to provide data integrity, security, authentication and another issue related to sensitivity data. The most common techniques were used to protect data during transmission on public cloud are cryptography, steganography, and compression. The proposed scheme suggests an entirely new approach for data security on public cloud. Authors have suggested an entirely new approach that completely makes secret data invisible behind carrier object and it is not been detected with the image performance parameters like PSNR, MSE, entropy and others. The details of results are explain in result section of paper. Proposed technique have better outcome than any other existing technique as a security mechanism on a public cloud. Primary focus of suggested approach is to minimize integrity loss of public storage data due to unrestricted access rights by uses. To improve reusability of carrier even after data concealed is really a challenging task and achieved through suggested approach.

The present grievous situation of the tuberculosis disease can be improved by efficient case management and timely follow-up evaluations. With the advent of digital technology this can be achieved by quick summarization of the patient-centric data. The aim of our study was to assess the effectiveness of the ODK Scan paper-to-digital system during testing period of three months. A sequential, explanatory mixed-method research approach was employed to elucidate technology use. Training, smartphones, application and 3G enabled SIMs were provided to the four field workers. At the beginning, baseline measures of the data management aspects were recorded and compared with endline measures to see the impact of ODK Scan. Additionally, at the end, users’ feedback was collected regarding app usability, user interface design and workflow changes. 122 patients’ records were retrieved from the server and analysed for quality. It was found that ODK Scan recognized 99.2% of multiple choice bubble responses and 79.4% of numerical digit responses correctly. However, the overall quality of the digital data was decreased in comparison to manually entered data. Using ODK Scan, a significant time reduction is observed in data aggregation and data transfer activities, however, data verification and form filling activities took more time. Interviews revealed that field workers saw value in using ODK Scan, however, they were more concerned about the time consuming aspects of the use of ODK Scan. Therefore, it is concluded that minimal disturbance in the existing workflow, continuous feedback and value additions are the important considerations for the implementing organization to ensure technology adoption and workflow improvements.

The monitoring of maternal and child health, using wearable devices made with wireless sensor technologies, is expected to reduce maternal and child death rates. Wireless sensor technologies have been used in wireless sensor networks to enable the acquisition of data for monitoring machines, smart cities, transportation, asset tracking, and tracking of human activity. Applications based on wireless body area network (WBAN) have been used in healthcare for measuring and monitoring of patient health and activity through integration with wearable devices. Wireless sensors used in WBAN can be cost-effective, enable remote availability, and can be integrated with electronic health record (EHR) management systems. Interoperability of WBAN sensor data with other linked data has the potential to improve health for all, including maternal and child health through the improvement of data access, data quality and healthcare access. This paper presents a survey of the state-of-the-art techniques for managing WBAN sensor data interoperability. The findings in this study will provide reliable support to enable policymakers and health care providers to take action to enhance the use of e-health to improve maternal-child health and reduce the mortality rates of women and children.

Use of a multi-sensor approach can provide citizens a holistic insight in the air quality in their immediate surroundings and assessment of personal exposure to urban stressors. Our work, as part of the ICARUS H2020 project, which included over 600 participants from 7 European cities, discusses data fusion and harmonization on a diverse set of multi-sensor data streams to provide a comprehensive and understandable report for participants, and offers possible solutions and improvements. Harmonizing the data streams identified issues with the used devices and protocols, such as non-uniform timestamps, data gaps, difficult data retrieval from commercial devices, and coarse activity data logging. Our process of data fusion and harmonization allowed us to automate the process of generating visualizations and reports and consequently provide each participant with a detailed individualized report. Results showed that a key solution was to streamline the code and speed up the process, which necessitated certain compromises in visualizing the data. A thought-out process of data fusion and harmonization on a diverse set of multi-sensor data streams considerably improved the quality and quantity of data that a research participant receives. Though automatization accelerated the production of the reports considerably, manual structured double checks are strongly recommended.

A crucial role is played by personal biomedical data when it comes to maintaining proficient access to health records by patients as well as health professionals. However, it is difficult to get a unified view pertaining to health data that have been scattered across various health center/hospital sections. To be specific, health records are distributed across many places and cannot be found integrated easily. In recent years, blockchain is regarded as a promising explanation that helps to achieve individual biomedical information sharing in a secured way along with privacy preservation, because of its benefit of immutability. This research work put forwards a blockchain-based managing scheme that helps to establish interpretation improvements pertaining to electronic biomedical systems. In this scheme, two blockchain were employed to construct the base of it, where the second blockchain algorithm is used to generate a secure sequence for the hash key that generated in first blockchain algorithm. The adaptively feature enable the algorithm to use multiple data types and combine between various biomedical images and text records as well. All the data, including keywords, digital records as well as the identity of patients are private key encrypted along with keyword searching capability so as to maintain data privacy preservation, access control and protected search. The obtained results which show the low latency (less than 750 ms) at 400 requests / second indicate the ability to use it within several health care units such as hospitals and clinics.

This manuscript describes the construction and validation of high resolution daily gridded (0.05° × 0.05°) rainfall and maximum and minimum temperature data for Bangladesh : the Enhancing National Climate Services for Bangladesh Meteorological Department (ENACTS-BMD) dataset. The dataset was generated by merging data from weather stations, satellite products (for rainfall) and reanalysis (for temperature). ENACTS-BMD is the first high-resolution gridded surface meteorological dataset developed specifically for studies of surface climate processes in Bangladesh. Its record begins in January 1981 and is updated in real-time monthly and outputs have daily, decadal and monthly time resolution. The Climate Data Tools (CDT), developed by the International Research Institute for Climate and Society (IRI), Columbia University, is used to generate the dataset. This data processing includes the collection of weather and gridded data, quality control of stations data, downscaling of the reanalysis for temperature, bias correction of both satellite rainfall and downscaled reanalysis of temperature, and the combination of station and bias-corrected gridded data. The ENACTS-BMD dataset is available as an open-access product at BMD’s official website, allowing the enhancement of the provision of services, overcoming the challenges of data quality, availability, and access, promoting at the same time the engagement and use by stakeholders.

Big data has revolutionised science and technology leading to the transformation of our societies. High Performance Computing (HPC) provides the necessary computational power for big data analysis using artificial intelligence and methods. Traditionally HPC and big data had focused on different problem domains and had grown into two different ecosystems. Efforts have been underway for the last few years on bringing the best of both paradigms into HPC and big converged architectures. Designing HPC and big data converged systems is a hard task requiring careful placement of data, analytics, and other computational tasks such that the desired performance is achieved with the least amount of resources. Energy efficiency has become the biggest hurdle in the realisation of HPC, big data, and converged systems capable of delivering exascale and beyond performance. Data locality is a key parameter of HPDA system design as moving even a byte costs heavily both in time and energy with an increase in the size of the system. Performance in terms of time and energy are the most important factors for users, particularly energy, due to it being the major hurdle in high performance system design and the increasing focus on green energy systems due to environmental sustainability. Data locality is a broad term that encapsulates different aspects including bringing computations to data, minimizing data movement by efficient exploitation of cache hierarchies, reducing intra- and inter-node communications, locality-aware process and thread mapping, and in-situ and in-transit data analysis. This paper provides an extensive review of the cutting-edge on data locality in HPC, big data, and converged systems. We review the literature on data locality in HPC, big data, and converged environments and discuss challenges, opportunities, and future directions. Subsequently, using the knowledge gained from this extensive review, we propose a system architecture for future HPC and big data converged systems. To the best of our knowledge, there is no such review on data locality in converged HPC and big data systems.

Digital federated platforms and data cooperatives for secure, trusted and sovereign data exchange will play a central role in the construction industry of the future. With the help of platforms, cooperatives and their novel value creation, the digital transformation and the degree of organization of the construction value chain can be taken to a new level of collaboration. The goal of this research project was to develop an experimental prototype for a federated innovation data platform along with a suitable exemplary use case. The prototype is to serve the construction industry as a demonstrator for further developments and form the basis for an innovation platform. It exemplifies how an overall concept is concretely implemented along one or more use cases that address high-priority industry pain points. This concept will create a blueprint and a framework for further developments, which will then be further established in the market. The research project illuminates the perspective of various governance innovations to increase industry collaboration, productivity and capital project performance and transparency as well as the overall potential of possible platform business models. However, a comprehensive expert survey revealed that there are considerable obstacles to trust-based data exchange between the key stakeholders in the industry value network. The obstacles to cooperation are predominantly not of a technical nature, but of a competitive, predominantly trust-related nature. To overcome these obstacles and create a pre-competitive space of trust, the authors therefore propose the governance structure of a data cooperative model, which is discussed in detail in this paper.

In this paper, the set-membership affine projection (SM-AP) algorithm is utilized to censor non-informative data in big data applications. To this end, the probability distribution of the additive noise signal and the excess of mean-squared error (EMSE) in steady-state are employed in order to estimate the threshold parameter of the single threshold SM-AP (ST-SM-AP) algorithm aiming at attaining the desired update rate. Furthermore, by defining an acceptable range for the error signal, the double threshold SM-AP (DT-SM-AP) algorithm is proposed to detect very large errors due to the irrelevant data such as outliers. The DT-SM-AP algorithm can censor non-informative and irrelevant data in big data applications, and it can improve misalignment and convergence rate of the learning process with high computational efficiency. The simulation and numerical results corroborate the superiority of the proposed algorithms over traditional algorithms.

The aim of this paper is to study a semi-functional partial linear regression model (SFPLR) for spatial data with responses missing at random. The estimators are constructed by the kernel method, and some asymptotic properties such as probability convergence rates of the nonparametric component and asymptotic distribution of the parametric and nonparametric components are established under certain conditions. Next, the performances and the superiority of these estimators are presented and examined using a study on simulated data and on real data by carrying out a comparison between our semi-functional partially linear model with MAR estimator (SFPLRM), the semi-functional partially linear model with the full-case estimator (SFPLRC) and the nonparametric functional model estimator with MAR (FNPM). The results show that the proposed estimators outperform existing estimators as the number of random missing data increases.

Disease surveillance is critical to monitor ongoing control activities, detect early outbreaks and to inform intervention priorities and policies. Unfortunately, most data from disease surveillance remain under-utilised to support decision-making in real-time. Using the Brazilian Amazon malaria surveillance data as a case study, we explore unsupervised anomaly detection machine learning techniques to analyse and discover potential anomalies. We found that our models are able to detect early outbreaks, peak of outbreaks as well as change points in the proportion of positive malaria cases. Specifically, the sustained rise in malaria in the Brazilian Amazon in 2016 was flagged by several models. We also found that no single model detects all the anomalies across all health regions. The approaches using Clustering-based local outlier algorithm ranked first before Principal component analysis and Stochastic outlier selection in maximising the number of anomalies detected in local health regions. Because of this, we also provide the minimum number of machine learning models (top-k models) to maximise the number of anomalies detected across different health regions. We discovered that the top-3 models that maximise the coverage of the number and types of anomalies detected across the 13 health regions are: Principal component analysis, Stochastic outlier selection and Multi-covariance determinant. Anomaly detection approaches provide interesting solutions to discover patterns of epidemiological importance when confronted with a large volume of data across space and time. Our exploratory approach can be replicated for other diseases and locations to inform timely interventions and actions toward endemic disease control.

Nowadays, cloud computing is one of the important and rapidly growing paradigms that extend its capabilities and applications in various areas of life. The cloud computing system challenges many security issues, such as scalability, integrity, confidentiality, and unauthorized access, etc. An illegitimate intruder may gain access to the sensitive cloud computing system and use the data for inappropriate purposes that may lead to losses in business or system damage. This paper proposes a hybrid unauthorized data handling (HUDH) scheme for Big data in cloud computing. The HUDU aims to restrict illegitimate users from accessing the cloud and data security provision. The proposed HUDH consists of three steps: data encryption, data access, and intrusion detection. HUDH involves three algorithms; Advanced Encryption Standards (AES) for encryption, Attribute-Based Access Control (ABAC) for data access control, and Hybrid Intrusion Detection (HID) for unauthorized access detection. The proposed scheme is implemented using Python and Java language. Testing results demonstrate that the HUDH can delegate computation overhead to powerful cloud servers. User confidentiality, access privilege, and user secret key accountability can be attained with more than 97% high accuracy.

Quality of Service (QoS) has a significant role in the provision of resources within service oriented distributed systems. In Quality of Service, cloud computing creates new challenges for improvements using the concept of virtualization. Currently, Cloud Computing is very emerging technology in every field of data storage and resource distribution over the network. Considering this new emerging technology, for the ease of data accessibility, price, resource use, restoration, response time and number of constraints the quality performance measures need to be upgraded. The paper highlights the research gap in providing a solution to achieve a Quality of Services in Cloud Computing. We also review the issues and challenges arising in cloud computing to guarantee quality.

Data science is facing the following major challenges: (1) developing scalable cross-disciplinary capabilities, (2) dealing with the increasing data volumes and their inherent complexity, (3) building tools that help to build trust, (4) creating mechanisms to efficiently operate in the domain of scientific assertions, (5) turning data into actionable knowledge units and (6) promoting data interoperability. As a way to overcome these challenges, we further develop the proposals by early Internet pioneers for Digital Objects as encapsulations of data and metadata made accessible by persistent identifiers. In the past decade, this concept was revisited by various groups within the Research Data Alliance and put in the context of the FAIR Guiding Principles for findable, accessible, interoperable and reusable data. The basic components of a FAIR Digital Object (FDO) as a self-contained, typed, machine-actionable data package are explained. A survey of use cases has indicated the growing interest of research communities in FDO solutions. We conclude that the FDO concept has the potential to act as the interoperable federative core of a hyperinfrastructure initiative such as the European Open Science Cloud (EOSC).