ARTICLE | doi:10.20944/preprints202007.0330.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Data Privacy; Mobile devices; Environment Privacy; General Data Protection Regulation (GDPR).
Online: 15 July 2020 (09:30:42 CEST)
The mobile devices caused a constant struggle for the pursuit of data privacy. Nowadays, it appears that the number of mobile devices in the world is increasing. With this increase and technological evolution, thousands of data associated with everyone are generated and stored remotely. Thus, the topic of data privacy is highlighted in several areas. There is a need for control and management of data in circulation inherent to this theme. This article presents an approach of the interaction between the individual and the public environment, where this interaction will determine the access to information. This analysis was based on a data privacy management model in public environments created after reading and analyzing the current technologies. A mobile application based on location via Global Positioning System (GPS) was created to substantiate this model, which it considers the General Data Protection Regulation (GDPR) to control and manage access to the data of each individual.
ARTICLE | doi:10.20944/preprints202209.0094.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Blockchain; Cryptography; DApp; Health Data; Privacy.
Online: 7 September 2022 (03:06:09 CEST)
With the fast development of blockchain technology in latest years, its application in scenarios that require privacy, such as health area, became encouraged and widely discussed. This paper presents an architecture to ensure the privacy of health-related data, which are stored and shared within a blockchain network in a decentralized manner, through the use of encryption with the RSA, ECC and AES algorithms. Evaluation tests were performed to verify the impact of cryptography on the proposed architecture in terms of computational effort, memory usage and execution time. The results demonstrate an impact mainly on the execution time and on the increase in the computational effort for sending data to the blockchain, however, justifiable considering the privacy and security provided with the architecture and encryption.
ARTICLE | doi:10.20944/preprints202007.0369.v1
Subject: Mathematics & Computer Science, Other Keywords: Data privacy; Ambient intelligence; COVID-19
Online: 17 July 2020 (08:17:07 CEST)
The COVID-19 pandemic plagues the whole world, bringing numerous challenges which need to be addressed. One of them is the privacy of patient data. There are several problems related to data privacy in IoT environments, the use of applications, devices, and functionalities in hospital processes. Therefore, we have compared works from the literature and developed a taxonomy consisting of the requirements necessary to control patient privacy data in a hospital setting in the current pandemic. Based on the studies, an application was modeled and implemented. According to the tests and comparisons drawn between the variables, the application yielded satisfactory results.
ARTICLE | doi:10.20944/preprints202009.0161.v1
Subject: Engineering, General Engineering Keywords: Smartphone; cloud; privacy; framework; mobile privacy; blockchain; permission system; data security; Android OS; Zygote; Dalvik VM
Online: 7 September 2020 (08:53:02 CEST)
The Smartphone industry has expanded significantly over the last few years. According to the available data, each year, a marked increase in the number of devices in use is observed. Most consumers opt for Smartphones due to the extensive number of software applications that can be downloaded on their devices, thus increasing their functionality. However, this growing trend of application installation brings an issue of user protection, as most applications seek permission to access data on a user’s device. The risks this poses to sensitive data is real to both corporate and individual users. While Android has grown in popularity, this trend has not been followed by the efforts to increase the security of its users. This is a well-known set of problems, and prior solutions have approached it from the ground up; that is, they have focused on implementing reasonable security policies within the Android’s open-source kernel. While these solutions have achieved the goals of improving Android with such security policies, they are severely hampered by the way in which they have implemented them. In this work, a framework referred to as CenterYou is proposed to overcome these issues. It applies a pseudo data technique and a cloud-based decision-making system to scan and protect Smartphone devices from unnecessarily requested permissions by installed applications and identifies potential privacy leakages. The current paper demonstrated all aspects of the CenterYou application technical design. The work presented here provides a significant contribution to the field, as the technique based on pseudo data is used in the actual permissions administration of Android applications. Moreover, this system is user and cloud-driven, rather than being governed by over-privileged applications.
ARTICLE | doi:10.20944/preprints202007.0502.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: COVID-19; contact tracing; privacy concern; secure communication; healthcare data; blockchain
Online: 22 July 2020 (06:19:47 CEST)
Contact tracing has become an indispensable tool of various extensive measures to control the spread of COVID-19 pandemic due to novel coronavirus. This essential tool helps to identify, isolate and quarantine the contacted persons of a COVID-19 patient. However, the existing contact tracing applications developed by various countries, health organizations to trace down the contacts after identifying a COVID-19 patient suffers from several security and privacy concerns. In this work, we have identified those security and privacy issues of several leading contact tracing applications and proposed a blockchain-based framework to overcome the major security and privacy challenges imposed by the applications. We have discussed the security and privacy measures that are achieved by the proposed framework to show the effectiveness against the security and privacy issues raised by the existing mobile contact tracing applications.
ARTICLE | doi:10.20944/preprints202209.0413.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Consortium Blockchain; Ring signature; Blockchain privacy; Blockchain security; Access Control; Blockchain big data
Online: 27 September 2022 (07:35:53 CEST)
Banking sectors commit modern working frameworks and models smooth development based on decentralization with keeping money confront in unused ranges and differing activities. Consortium Blockchain Privacy becomes a major concern and the challenge of Most of banking sectors.Development without being hampered being a major concern it can store confirmed, Data privacy includes assuring protection for both insider ad outsider threats therefore access control of Ring signature could help to secure Privacy of inside and outside threats by secure process by RSBAC using CIA triad privacy Confidentiality, Availability, Integrity.This paper proposes a ring signature-based on access control mechanism for determining who a user is and then regulating that person's access to and use of a system's resources. In a nutshell, access control restricts who has access to a system. It also restricts access to system resources to users who have been identified as having the necessary privileges and permissions. The proposed paradigm satisfies the needs of both workflow and non-workflow systems in an enterprise setting. The traits of the conditional purposes, roles, responsibilities, and policies provide the foundation for it. It ensures that internal risks such as database administrators are protected.Finally, it provides the necessary protection in the event that the data is published.
ARTICLE | doi:10.20944/preprints201906.0144.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: data mining; network security; association rules; DDoS
Online: 16 June 2019 (02:42:59 CEST)
Typical modern information systems are required to process copious data. Conventional manual approaches can no longer effectively analyze such massive amounts of data, and thus humans resort to smart techniques and tools to complement human effort. Currently, network security events occur frequently, and generate abundant log and alert files. Processing such vast quantities of data particularly requires smart techniques. This study reviewed several crucial developments of existent data mining algorithms, including those that compile alerts generated by heterogeneous IDSs into scenarios and employ various HMMs to detect complex network attacks. Moreover, sequential pattern mining algorithms were examined to develop multi-step intrusion detection. These studies can focus on applying these algorithms in practical settings to effectively reduce the occurrence of false alerts. This article researched the application of data mining algorithms in network security. The academic community has recently generated numerous studies on this topic.
ARTICLE | doi:10.20944/preprints201810.0618.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: classification; machine learning; chaos-based cryptography; Hadoop; data clustering; biometrics
Online: 26 October 2018 (05:50:53 CEST)
Authentication systems based on biometrics characteristics and data represents one of the most important trend in the evolution of our world. In the near future, biometrics systems will be everywhere in the society, such as government, education, smart cities, banks etc. Due to its uniqueness characteristic, biometrics systems will become also vulnerable, privacy being one of the most important challenge. The classic cryptographic primitives are not sufficient to assure a strong level of secureness for privacy. The following work paper represents an effort to present the main cryptographic techniques and algorithms that can give us the possibility to raise a certain level of secureness for privacy. We will show their own challenges (strengths and weaknesses). We will demonstrate how we can use the most common and well-known techniques and algorithms in order to get a maximum efficiency and a high level in assuring the integrity of the biometrics data.
Subject: Earth Sciences, Atmospheric Science Keywords: disaster management; virtual operation support teams; privacy; data retention; hyperloglog; focus group discussion
Online: 1 October 2020 (13:58:16 CEST)
Social media data is heavily used to analyze and evaluate situations in times of disasters, and derive decisions for action from it. A cruicial part of the analysis is to avoid unnecessary data retention during that process, in order to prevent subsequent abuse, theft or public exposure of collected datasets and thus, protect the privacy of social media users. There are a number of technical approaches out to face the problem. One of them is using a cardinality estimation algorithm called HyperLogLog to store data in a privacy-aware structure, that can not be used for purposes other than the originally intended. In this case study, we developed and conducted a focus group discussion with teams of social media analysts, in which we identified challenges and opportunities of working with such a privacy-enhanced social media data structure in place of conventional techniques. Our findings show that, with the exception of training scenarios, deploying HyperLogLog in the data acquisiton process will not distract the data analysis process. Instead, it will improve working with huge datasets due to the improved characteristics of the resulting data structure.
ARTICLE | doi:10.20944/preprints202209.0009.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Digital Twin; Internet-of-Medical-Things (IoMT); Security; Privacy; Blockchain; Non-fungible Token (NFT); Virtual Healthcare Services; Access Control; Data Sharing
Online: 1 September 2022 (07:21:25 CEST)
Seniors safety is a compelling need, which necessitates 24/7 real-time monitoring and timely dangerous action recognition. Being able to mirror characteristics of physical objects (PO) to corresponding logical objects (LO) and seamlessly monitor their footprints thus detect anomaly parameters, Digital Twins (DT) has been considered a practical way to provide virtual health services for seniors safety. Meanwhile, widely adopted Internet of Medical Things (IoMT) consisting of wearable sensors and non-contact optical cameras for self and remote health data monitoring also raises concerns on information security and privacy violation. Therefore, security of POs, LOs and reliable data sharing among healthcare professionals are challenging as constructing trust and privacy-preserving virtual health services. Thanks to characteristics of decentralization, traceability and unalterability, Blockchain is promising to enhance security and privacy properties in many areas like data analysis, finance and healthcare. This paper envisions a lightweight authentication framework (LAF) to enable secure and privacy-preserving virtual healthcare services. Leveraging Non-Fungible Token (NFT) technology to tokenize LOs and data streams on blockchain, anyone can certify the authenticity of a digital LO along with its synchronized data between PO without relying on a third-party agency. In addition, the NFT-based tokenization not only allows owners fully control their IoMT devices and data, but it also enables verifiable ownership and traceable transferability during data sharing process. Moreover, NFT only contains references to encrypted raw data that are saved on off-chain storage like local files or distributed databases, such a hybrid storage strategy ensures privacy-preservation for sensitive information. A proof-of-concept prototype is implemented and tests are conducted on a case study of seniors safety. The experimental evaluation shows the feasibility and effectiveness of the proposed LAF solution.
ARTICLE | doi:10.20944/preprints202301.0465.v1
Subject: Social Sciences, Law Keywords: right to privacy; personal data; indemnity; sustainability of public finances; Covid-19
Online: 26 January 2023 (03:09:31 CET)
The basic constitutional freedoms and rights of a person and citizen are in principle unlimited: the full scope of their exercise is the rule, and the restriction determined by law can only be an exception based on explicit constitutional authority and the legitimate aim of the restriction determined by the Constitution. That being so, the restrictions - in addition to being based on constitutional authority and pursuing constitutional objectives - should be commensurate with the needs to achieve these objectives. This means that restrictive legal rules must be suitable for achieving the legitimate aim pursued, must not be stricter than necessary and must be balanced between the constitutionally guaranteed subjective right of the individual and the interests of society. In this scholar paper, the authors point out the economic and legal consequences of the violation of individual privacy and data protection rights caused by the public disclosure of personal data of people who, at a certain time, were obliged to self-isolate due to suspicion of Covid-19 virus infection.
ARTICLE | doi:10.20944/preprints201909.0040.v1
Subject: Social Sciences, Business And Administrative Sciences Keywords: data mining; security; association rule; ECLAT
Online: 4 September 2019 (03:48:58 CEST)
The purpose of this paper is to develop WebSecuDMiner algorithm to discover unusual web access patterns based on analysing the potential rules hidden in web server log and user navigation history. Design/methodology/approach: WebSecuDMiner uses equivalence class transformation (ECLAT) algorithm to extract user access patterns from the web log data, which will be used to identify the user access behaviours pattern and detect unusual one. Data extracted from the web serve log and user browsing behaviour is exploited to retrieve the web access pattern that is produced by the same user. Findings: WebSecuDMiner is used to detect whether any unauthorized access have been posed and take appropriate decisions regarding the review of the original rights of suspicious user. Research limitations/implications: The present work uses the database which is extracted from web serve log file and user browsing behaviour. Although the page is viewed by the user, the visit is not recorded in the server log file, since it can be access from the browser's cache.
CONCEPT PAPER | doi:10.20944/preprints201810.0724.v2
Subject: Social Sciences, Political Science Keywords: Social-Ecological System; Water security; Governance; Institution; Learning; Data-Cube
Online: 22 November 2018 (14:47:31 CET)
The Social-Ecological Systems (SES) framework serves as a valuable framework to explore and understand social and ecological interactions, and pathways in water governance. Yet, it lacks a robust understanding of change. We argue an analytical and methodological approach to engaging global changes in SES is critical to strengthening the scope and relevance of the SES framework. Relying on SES and resilience thinking, we propose an institutional and cognitive model of change that institutions and natural resources systems co-evolve to provide a dynamic understanding of SES that stands on three causal mechanisms: institutional complexity trap, rigidity trap, and learning processes. We illustrate how Data Cube technology could overcome current limitations and offer reliable avenues to test hypothesis about the dynamics of social-ecological systems and water security by offering to combine spatial and time data with no major technical requirements for users.
REVIEW | doi:10.20944/preprints201607.0077.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Data aggregation, Security and Wireless Sensor Network
Online: 25 July 2016 (14:38:07 CEST)
Multiple sensor nodes known as detection stations make the sensor networks; each node is light and portable. Every sensor node contains power source, microcomputer, transducer and transceiver. Power source provides power to each node. Micro-computer is used for storing and processing the output coming from the sensors. The transducer is used to generate the signals and the transceiver is used to receive and transmit data to the central computer. Eavesdropping gets facilitated with wireless communication, and it has many useful applications in military, homeland, hostile and uncontrolled environments. So it is prone to the high level of security. The process in which information is gathered to form a summarized type for analysis is known as data aggregation, as it is used to reduce the energy consumption in wireless sensor networks. The security issues have become crucial in data aggregation, especially when gets deployed in hostile and remote environment. In wireless sensor networks many secure aggregations have been proposed. It still faces some resource constrained that’s why new techniques are needed. In our survey we will discuss those approaches and their pros and cons.
REVIEW | doi:10.20944/preprints202101.0457.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Cybersecurity; artificial intelligence; machine learning; cyber data analytics; cyber-attacks; anomaly; intrusion detection; security intelligence
Online: 25 January 2021 (09:19:10 CET)
Artificial Intelligence (AI) is one of the key technologies of the Fourth Industrial Revolution (Industry 4.0), which can be used for the protection of Internet-connected systems from cyber-threats, attacks, damage, or unauthorized access. To intelligently solve today's various cybersecurity issues, popular AI techniques involving Machine Learning (ML) and Deep Learning (DL) methods, the concept of Natural Language Processing (NLP), Knowledge Representation and Reasoning (KRR), as well as the concept of knowledge or rule-based Expert Systems (ES) modeling can be used. Based on these AI methods, in this paper, we present a comprehensive view on "AI-driven Cybersecurity" that can play an important role for intelligent cybersecurity services and management. The security intelligence modeling based on such AI methods can make the cybersecurity computing process automated and intelligent than the conventional security systems. We also highlight several research directions within the scope of our study, which can help researchers do future research in the area. Overall, this paper's ultimate objective is to serve as a reference point and guidelines for cybersecurity researchers as well as industry professionals in the area, especially from an AI-based technical point of view.
ARTICLE | doi:10.20944/preprints202108.0032.v1
Subject: Social Sciences, Accounting Keywords: Artificial Intelligence; Securty; Security Of Data; Security Systems
Online: 2 August 2021 (12:36:22 CEST)
Diverse forms of artificial intelligence (AI in further text) are at the forefront of triggering digital security innovations, based on the threats that are arising in this post COVID world. On the one hand, companies are experiencing difficulty in dealing with security challenges with regard to a variety of issues ranging from system openness, decision making, quality control and web domain, just to mention a few. On the other hand, in the last decade, research has focused on security capabilities based on tools such as platform complacency, intelligent trees, modeling methods and outage management systems, in an effort to understanding the interplay between AI and those issues. The dependence on the emergence of AI in running industries and shaping the education, transports and health sectors is now well known in literature. AI is increasingly employed in managing data security across economic sectors. Thus, a literature review of AI and system secu-rity within the current digital society is opportune. This paper aims at identifying research trends in the field through a Systematic Bibliometric Literature Review (LRSB) of research on AI and system security. The review entails 77 articles published in Scopus® database, presenting up-to-date knowledge on the topic. The LRSB results were synthesized across current research subthemes. Findings are presented. The originality of the paper relies on its LRSB method, together with extant review of articles that have not been categorized so far. Implications for future re-search are suggested.
ARTICLE | doi:10.20944/preprints201611.0010.v1
Subject: Earth Sciences, Atmospheric Science Keywords: millimeter-wavelength cloud radar; attenuation correction; dual-radar; data fusion
Online: 1 November 2016 (10:05:18 CET)
In order to correct attenuated millimeter-wavelength (Ka-band) radar data and address the problem of instability, an attenuation correction methodology (attenuation correction with variation trend constraint; VTC) was developed. Using synchronous observation conditions and multi-band radars, the VTC method adopts the variation trends of reflectivity in X-band radar data captured with wavelet transform as a constraint to adjust reflectivity factors of millimeter-wavelength radar. The correction was evaluated by comparing reflectivities obtained by millimeter-wavelength cloud radar and X-band weather radar. Experiments showed that attenuation was a major contributory factor in the different reflectivities of the two radars when relatively intense echoes exist, and the attenuation correction developed in this study significantly improved data quality for millimeter-wavelength radar. Reflectivity differences between the two radars were reduced and reflectivity correlations were enhanced. Errors caused by attenuation were eliminated, while variation details in the reflectivity factors were retained. The VTC method is superior to the bin-by-bin method in terms of correction amplitude and can be used for attenuation correction of shorter wavelength radar assisted by longer wavelength radar data.
ARTICLE | doi:10.20944/preprints201608.0232.v2
Subject: Medicine & Pharmacology, Nursing & Health Studies Keywords: mHealth; ODK scan; mobile health application; digitizing data collection; data management processes; paper-to-digital system; technology-assisted data management; treatment adherence
Online: 2 September 2016 (03:17:38 CEST)
The present grievous situation of the tuberculosis disease can be improved by efficient case management and timely follow-up evaluations. With the advent of digital technology this can be achieved by quick summarization of the patient-centric data. The aim of our study was to assess the effectiveness of the ODK Scan paper-to-digital system during testing period of three months. A sequential, explanatory mixed-method research approach was employed to elucidate technology use. Training, smartphones, application and 3G enabled SIMs were provided to the four field workers. At the beginning, baseline measures of the data management aspects were recorded and compared with endline measures to see the impact of ODK Scan. Additionally, at the end, users’ feedback was collected regarding app usability, user interface design and workflow changes. 122 patients’ records were retrieved from the server and analysed for quality. It was found that ODK Scan recognized 99.2% of multiple choice bubble responses and 79.4% of numerical digit responses correctly. However, the overall quality of the digital data was decreased in comparison to manually entered data. Using ODK Scan, a significant time reduction is observed in data aggregation and data transfer activities, however, data verification and form filling activities took more time. Interviews revealed that field workers saw value in using ODK Scan, however, they were more concerned about the time consuming aspects of the use of ODK Scan. Therefore, it is concluded that minimal disturbance in the existing workflow, continuous feedback and value additions are the important considerations for the implementing organization to ensure technology adoption and workflow improvements.
REVIEW | doi:10.20944/preprints201607.0075.v1
Online: 25 July 2016 (06:34:26 CEST)
Over the past couple of decades, Global positioning system (GPS) technology has been utilized to collect large-scale data from travel surveys. As the precise spatiotemporal characteristics of travel could be provided by GPS devices, the issues of traditional travel survey, such as misreporting and non-response, could be addressed. Considering the defects of dedicated GPS devices (e.g., need much money to buy devices, forget to take devices to collect data, limit the simple size because of the number of devices, etc.), and the phenomenon that the smartphone is becoming one of necessities of life, there is a great chance for the smartphone to replace dedicated GPS devices. Although, several general reviews have been done about smartphone-based GPS travel survey in the literature review section in some articles, a systematic review from smartphone-based GPS data collection to travel mode detection has none. The included studies were searched from six databases. The purpose of this review is to critically assess the current literature on the existing methodologies of travel mode detection based on GPS raw data collected by smartphones. Meanwhile, according to the systematic comparison among different methods from data-preprocessing to travel mode detection, this paper could carefully provide the Strengths and Weaknesses of existing methods. Furthermore, it is the crucial step to develop the methodologies and applications of GPS raw data collected by smartphones.
ARTICLE | doi:10.20944/preprints201610.0012.v1
Online: 5 October 2016 (15:08:32 CEST)
Bio-molecular reagents like antibodies required in experimental biology are expensive and their effectiveness, among other things, is critical to the success of the experiment. Although such resources are sometimes donated by one investigator to another through personal communication between the two, there is no previous study to our knowledge on the extent of such donations, nor a central platform that directs resource seekers to donors. In this paper, we describe, to our knowledge, a first attempt at building a web-portal titled Bio-Resource Exchange that attempts to bridge this gap between resource seekers and donors in the domain of experimental biology. Users on this portal can request for or donate antibodies, cell-lines and DNA Constructs. This resource could also serve as a crowd-sourced database of resources for experimental biology. Further, in order to index donations outside of our portal, we mined scientific articles to find instances of donations of antibodies and attempted to extract information about these donations at the finest granularity. Specifically, we extracted the name of the donor, his/her affiliation and the name of the antibody for every donation by parsing the acknowledgements sections of articles. To extract annotations at this level, we propose two approaches – a rule based algorithm and a bootstrapped relation learning algorithm. The algorithms extracted donor names, affiliations and antibody names with average accuracies of 57% and 62% respectively. We also created a dataset of 50 expert-annotated acknowledgements sections that will serve as a gold standard dataset to evaluate extraction algorithms in the future. Contact: firstname.lastname@example.org, email@example.com Database URL: http://tonks.dbmi.pitt.edu/brx Supplementary information: Supplementary data are available at Database online.
ARTICLE | doi:10.20944/preprints201607.0047.v1
Subject: Social Sciences, Econometrics & Statistics Keywords: SAINT model; SiZer; local linear fitting; mortality data
Online: 18 July 2016 (10:35:40 CEST)
The main contribution of this paper is to develop a graphical tool to evaluate the suitability of a candidate parametric model to fit a data set. The practical motivation is to check the adequacy of the so called SAINT model proposed in Jarner and Kryger (2011). This model has been implemented in practice by an important European pension fund concerned with setting annuity reserves for all current or former employees of Denmark. So, one particular relevant question is whether this mortality model is actually fitting old-age. Our graphical test can be described as follows. First we transform the data by means of the parametric model which is being evaluated. Should the model be correct, the transformed data will be in accordance with an Exponential distribution with rate 1. Then we construct a family of local linear hazard estimates based on the data on the transformed scale. Finally we use the statistical tool SiZer to assess the goodness-of-fit of the Exponential distribution to the data on the transformed scale. If the model is correct the SiZer map should not reveal any significant feature in the family of kernel estimates. Our method allow us to establish a diagnostic regarding the validity of the SAINT model when describing mortality patterns in four different countries.
ARTICLE | doi:10.20944/preprints201612.0091.v2
Subject: Earth Sciences, Geology Keywords: reanalysis climate data; hydrologic modeling; comparative analysis
Online: 3 February 2017 (03:50:07 CET)
Large-scale hydrological modeling in China is challenging given the sparse meteorological stations and large uncertainties associated with atmospheric forcing data.Here we introduce the development and use of the China Meteorological Assimilation Driving Datasets for the SWAT model (CMADS) in the Heihe River Basin(HRB) for improving hydrologic modeling, by leveraging the datasets from the China Meteorological Administration Land Data Assimilation System (CLDAS)(including climate data from nearly 40000 area encryption stations, 2700 national automatic weather stations, FengYun (FY) 2 satellite and radar stations). CMADS uses the Space Time Multiscale Analysis System (STMAS) to fuse data based on ECWMF ambient field and ensure data accuracy. In addition, compared with CLDAS, CMADS includes relative humidity and climate data of varied resolutions to drive hydrological models such as the Soil and Water Assessment Tool (SWAT) model. Here, we compared climate data from CMADS, Climate Forecast System Reanalysis (CFSR) and traditional weather station (TWS) climate forcing data and evaluatedtheir applicability for driving large scale hydrologic modeling with SWAT. In general, CMADS has higher accuracy than CFRS when evaluated against observations at TWS; CMADS also provides spatially continuous climate field to drive distributed hydrologic models, which is an important advantage over TWS climate data, particular in regions with sparse weather stations. Therefore, SWAT model simulations driven with CMADS and TWS achieved similar performances in terms of monthly and daily stream flow simulations, and both of them outperformed CFRS. For example, for the three hydrological stations (Ying Luoxia, Qilian Mountain, and ZhaMasheke) in the HRB at the monthly and daily Nash-Sutcliffe efficiency ranges of 0.75-0.95 and 0.58-0.78, respectively, which are much higher than corresponding efficiency statistics achieved with CFSR (monthly: 0.32-0.49 and daily: 0.26 – 0.45). The CMADS dataset is available free of charge and is expected to a valuable addition to the existing climate reanalysis datasets for deriving distributed hydrologic modeling in China and other countries in East Asia.
REVIEW | doi:10.20944/preprints202203.0087.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Internet of Things; cyber-attacks; anomalies; machine learning; deep learning; IoT data analytics; intelligent decision-making; security intelligence
Online: 7 March 2022 (02:39:58 CET)
The Internet of Things (IoT) is one of the most widely used technologies today, and it has a significant effect on our lives in a variety of ways, including social, commercial, and economic aspects. In terms of automation, productivity, and comfort for consumers across a wide range of application areas, from education to smart cities, the present and future IoT technologies hold great promise for improving the overall quality of human life. However, cyber-attacks and threats greatly affect smart applications in the environment of IoT. The traditional IoT security techniques are insufficient with the recent security challenges considering the advanced booming of different kinds of attacks and threats. Utilizing artificial intelligence (AI) expertise, especially machine and deep learning solutions, is the key to delivering a dynamically enhanced and up-to-date security system for the next-generation IoT system. Throughout this article, we present a comprehensive picture on IoT security intelligence, which is built on machine and deep learning technologies that extract insights from raw data to intelligently protect IoT devices against a variety of cyber-attacks. Finally, based on our study, we highlight the associated research issues and future directions within the scope of our study. Overall, this article aspires to serve as a reference point and guide, particularly from a technical standpoint, for cybersecurity experts and researchers working in the context of IoT.
ARTICLE | doi:10.20944/preprints201609.0088.v1
Subject: Engineering, Civil Engineering Keywords: classification; railway; power line; mobile laser scanning data; conditional random field; layout compatibility
Online: 26 September 2016 (09:33:05 CEST)
Railway has been used as one of the most crucial means of transportation in public mobility and economic development. For efficiently operating railways, the electrification system in railway infrastructure, which supplies electric power to trains, is essential facilities for stable train operation. Due to its important role, the electrification system needs to be rigorously and regularly inspected and managed. This paper presents a supervised learning method to classify Mobile Laser Scanning (MLS) data into ten target classes representing overhead wires, movable brackets and poles, which are recognized key objects in the electrification system. In general, the layout of railway electrification system shows a strong regularity of spatial relations among object classes. The proposed classifier is developed based on Conditional Random Field (CRF), which characterizes not only labeling homogeneity at short range, but also the layout compatibility between different object classes at long range in the probabilistic graphical model. This multi-range CRF model consists of a unary term and three pairwise contextual terms. In order to gain computational efficiency, MLS point clouds is converted into a set of line segments where the labeling process is applied. Support Vector Machine (SVM) is used as a local classifier considering only node features for producing the unary potentials of CRF model. As the short-range pairwise contextual term, Potts model is applied to enforce a local smoothness in short-range graph. While, long-range pairwise potentials are designed to enhance spatial regularities of both horizontal and vertical layouts among railway objects. We formulate two long-range pairwise potentials as the log posterior probability obtained by Naïve Bayes classifier. The directional layout compatibilities are characterized in probability look-up tables which represent co-occurrence rate of spatial relations in horizontal and vertical directions. The likelihood function is formulated by multivariate Gaussian distributions. In the proposed multi-range CRF model, the weight parameters to balance four sub-terms are estimated by applying the Stochastic Gradient Descent (SGD). The results show that the proposed multi-range CRF can effectively classify detailed railway elements, representing the average recall of 97.66% and the average precision of 97.07% for all classes.
ARTICLE | doi:10.20944/preprints202010.0577.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Cloud Computing; Health Systems; Security; Privacy; Data Protection; GDPR
Online: 28 October 2020 (10:00:55 CET)
Currently, there are several challenges that Cloud-based health-care Systems, around the world, are facing. The most important issue is to ensure security and privacy or in other words to ensure the confidentiality, integrity and availability of the data. Although the main provisions for data security and privacy were present in the former legal framework for the protection of personal data, the General Data Protection Regulation (GDPR) introduces new concepts and new requirements. In this paper, we present the main changes and the key challenges of the General Data Protection Regulation, and also at the same time we present how the Cloud-based Security Policy methodology proposed in  could be modified in order to be compliant with the GDPR and how Cloud environments can assist developers to build secure and GDPR compliant Cloud-based health Systems. The major concept of this paper is, primarily, to facilitate Cloud Providers in comprehending the framework of the new General Data Protection Regulation and secondly, to identify security measures and security policy rules for the protection of sensitive data in a Cloud-based Health System, following our risk-based Security Policy Methodology that assesses the associated security risks and takes into account different requirements from patients, hospitals, and various other professional and organizational actors.
ARTICLE | doi:10.20944/preprints201610.0067.v1
Subject: Mathematics & Computer Science, Applied Mathematics Keywords: point information gain; Rényi entropy; data processing
Online: 17 October 2016 (11:35:13 CEST)
We generalize the point information gain (PIG) and derived quantities, i.e., point information gain entropy (PIE) and point information gain entropy density (PIED), for the case of the Rényi entropy and simulate the behavior of PIG for typical distributions. We also use these methods for the analysis of multidimensional datasets. We demonstrate the main properties of PIE/PIED spectra for the real data on the example of several images, and discuss further possible utilizations in other fields of data processing.
ARTICLE | doi:10.20944/preprints201612.0079.v1
Subject: Earth Sciences, Environmental Sciences Keywords: fire detection; upwelling radiation; diurnal variation; training data; geostationary sensors
Online: 15 December 2016 (09:22:10 CET)
Fire detection from satellite sensors relies on an accurate estimation of the unperturbed state of a target pixel, from which an anomaly can be isolated. Methods for estimating the radiation budget of a pixel without fire depend upon training data derived from the location's recent history of brightness temperature variation over the diurnal cycle, which can be vulnerable to cloud contamination and the effects of weather. This study proposes a new method that utilises the common solar budget found at a given latitude in conjunction with an area's local solar time to aggregate a broad-area training dataset, which can be used to model the expected diurnal temperature cycle of a location. This training data is then used in a temperature fitting process with the measured brightness temperatures in a pixel, and compared to pixel-derived training data and contextual methods of background temperature determination. Results of this study show similar accuracy between clear-sky medium wave infrared upwelling radiation and the diurnal temperature cycle estimation compared to previous methods, with demonstrable improvements in processing time and training data availability. This method can be used in conjunction with brightness temperature thresholds to provide a baseline for upwelling radiation, from which positive thermal anomalies such as fire can be isolated.
ARTICLE | doi:10.20944/preprints201608.0123.v1
Subject: Engineering, Civil Engineering Keywords: limited sensor data; structural health monitoring; strain/stress response reconstruction; empirical mode decomposition
Online: 11 August 2016 (11:06:16 CEST)
Structural health monitoring has been studied by a number of researchers as well as various industries to keep up with the increasing demand for preventive maintenance routines. This work presents a novel method for reconstruct prompt, informed strain/stress responses at the hot spots of the structures based on strain measurements at remote locations. The structural responses measured from usage monitoring system at available locations are decomposed into modal responses using empirical mode decomposition. Transformation equations based on finite element modeling are derived to extrapolate the modal responses from the measured locations to critical locations where direct sensor measurements are not available. Then, two numerical examples (a two-span beam and a 19956-degree of freedom simplified airfoil) are used to demonstrate the overall reconstruction method. Finally, the present work investigates the effectiveness and accuracy of the method through a set of experiments conducted on an aluminium alloy cantilever beam commonly used in air vehicle and spacecraft. The experiments collect the vibration strain signals of the beam via optical fiber sensors. Reconstruction results are compared with theoretical solutions and a detailed error analysis is also provided.
ARTICLE | doi:10.20944/preprints201702.0074.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: network; systems; cloud computing; data centre; performance; software-defined; virtual machine; scheduling; admission control; application-aware;
Online: 20 February 2017 (04:56:24 CET)
Cloud computing refers to applications delivered as services over the Internet. Cloud systems employ policies that are inherently dynamic in nature and that depend on temporal conditions defined in terms of external events, such as the measurement of bandwidth, use of hosts, intrusion detection or specific time events. In this paper, we investigate an optimized resource management scheme named v-Mapper. The basic premise of v-Mapper is to exploit application-awareness concepts using software-defined networking (SDN) features. This paper makes three key contributions to the field: (1) We propose a virtual machine (VM) placement scheme that can effectively mitigate the VM placement issues for data-intensive applications; (2) We propose a validation scheme that will ensure that a service is entertained only if there are sufficient resources available for its execution and (3) We present a scheduling policy that aims to eliminate network load constraints. An evaluation was carried out with various benchmarks and demonstrated that v-Mapper shows improved performance over other state-of-the-art approaches in terms of average task completion time, service delay time and bandwidth utilization. Given the growing importance of supporting large-scale data processing and analysis in datacentres, the v-Mapper system has the potential to make a positive impact in improving datacentre performance in the future.
ARTICLE | doi:10.20944/preprints201611.0110.v1
Subject: Social Sciences, Finance Keywords: capital structure; firm’s performance; panel data; unit root analysis; Bangladesh
Online: 22 November 2016 (09:36:36 CET)
Capital structure decision plays an imperative role in firm’s performance. Recognizing the importance, there has been many studies inspected the rapport of capital structure with performance of firms and findings of those studies are inconclusive. In addition, there is relative deficiency of empirical studies examining the link of capital structure with performance of banks in Bangladesh. This paper attempted to fill this gap. Using panel data of 22 banks for the period of 2005-2014, this study empirically examined the impacts of capital structure on the performance of Bangladeshi banks assessed by return on equity, return on assets and earnings per share. Results from pooled ordinary least square analysis show that there are inverse impacts of capital structure on bank’s performance. Empirical findings of this study is of greater significance for the developing countries like Bangladesh because it will call upon concentration of the bank management and policy makers to pursue such policies to reduce reliance on debt and to accomplish optimal level capital structure. This research also contributes to empirical literatures by reconfirming (or otherwise) findings of previous studies.
ARTICLE | doi:10.20944/preprints201703.0058.v1
Subject: Mathematics & Computer Science, Other Keywords: Smartphone sensing; mobile-social integration; automatic recognition; social data; long-term health monitoring
Online: 10 March 2017 (17:32:31 CET)
Over the past decades, overweight and obesity has become a global epidemic and the leading threat for death. To prevent the serious risk, an overweight or obese individual must apply a long-term weight-management strategy to control food intake and physical activities, which is however, not easy. Recently, with the advances of information technology, more and more people can use wearable devices and smartphones to obtain physical activity information, while they can also access various health-related information from Internet online social networks (OSNs). Nevertheless, there is a lack of an integrated approach that can combine these two methods in an efficient way. In this paper, we address this issue and propose a novel mobile-social framework for health recognition and recommendation, namely, H-Rec2. The main ideas of H-Rec2 include (1) to recognize the individual's health status using smartphone as a general platform, and (2) to recommend physical activity and food intake based on personal health information, life science principles, and health-related information obtained from OSNs. To demonstrate the potentials of the H-Rec2 framework, we develop a prototype that consists of four important components: (1) an activity recognition module that senses physical activity using accelerometer, (2) a health status modeling module that applies a novel algorithm to generate personalized health status index, (3) a restaurant information collection module that collects relevant information from OSN, and (4) a restaurant recommendation module that provides personalized and context-aware recommendation. To evaluate the prototype, we conduct both objective and subjective experiments, which confirm the performance and effectiveness of the proposed system.
ARTICLE | doi:10.20944/preprints201608.0204.v1
Subject: Social Sciences, Economics Keywords: logistics industry; sustainability; data envelopment analysis (DEA); grey forecasting
Online: 25 August 2016 (10:12:27 CEST)
Logistics plays an important role in globalized companies and contributes to the development of foreign trade. A large number of external conditions, such as recession and inflation, affect logistics. Therefore, managers should find ways to improve operational performance, enabling them to increase efficiency while considering environmental sustainability due to the industry’s large scale of energy consumption. Based on data collected from the financial reports of top global logistics companies, this study uses a DEA model to calculate corporate efficiency by implementing a Grey forecasting approach to forecast future sustainability values. Consequently, the study addresses the problem of how to enhance operational performance while accounting for the impact of external conditions. This research can help logistics companies develop operation strategies in the future that will enhance their competitiveness vis-à-vis rivals in a time of global economic volatility.
CONFERENCE PAPER | doi:10.20944/preprints201612.0011.v1
Subject: Earth Sciences, Environmental Sciences Keywords: satellite data; fine particulate matter; air pollution; geographic information system; health risks; spatial analysis; Saudi Arabia
Online: 1 December 2016 (15:25:56 CET)
The study of the concentrations and effects of fine particulate matter in urban areas have been of great interest to researchers in recent times. This is due to the acknowledgment of the far-reaching impacts of fine particulate matter on public health. Remote sensing data have been used to monitor the trend of concentrations of particulate matter by deriving aerosol optical depth (AOD) from satellite images. The Center for International Earth Science Information Network (CIESIN) has released the second version of its global PM2.5 data with improvement in spatial resolution. This paper revisits the study of spatial and temporal variations in particulate matter in Saudi Arabia by exploring the cluster analysis of the new data. Cluster analysis of the PM2.5 values of Saudi cities is performed by using Anselin local Moran’s I statistic. Also, the analysis is carried out at the regional level by using self-organizing map (SOM). The results show an increasing trend in the concentrations of particulate matter in Saudi Arabia, especially in some selected urban areas. The eastern and south-western parts of the Kingdom have significantly clustering high values. Some of the PM2.5 values have passed the threshold indicated by the World Health Organization (WHO) standard and targets posing health risks to Saudi urban population.
ARTICLE | doi:10.20944/preprints201612.0002.v1
Subject: Mathematics & Computer Science, Applied Mathematics Keywords: change point; estimation; consistency; panel data; short panels; boundary issue; structural change; bootstrap; non-life insurance; change in claim amounts
Online: 1 December 2016 (10:02:03 CET)
Panel data of our interest consist of a moderate number of panels, while the panels contain a small number of observations. An estimator of common breaks in panel means without a boundary issue for this kind of scenario is proposed. In particular, the novel estimator is able to detect a common break point even when the change happens immediately after the first time point or just before the last observation period. Another advantage of the elaborated change point estimator is that it results in the last observation in situations with no structural breaks. The consistency of the change point estimator in panel data is established. The results are illustrated through a simulation study. As a by-product of the developed estimation technique, a theoretical utilization for correlation structure estimation, hypothesis testing, and bootstrapping in panel data is demonstrated. A practical application to non-life insurance is presented as well.
ARTICLE | doi:10.20944/preprints201703.0028.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: GPS trajectory; GPS sensor; trajectory similarity measure; spatial-temporal data
Online: 6 March 2017 (06:51:37 CET)
With the rapid spread of built-in GPS handheld smart devices, the trajectory data from GPS sensors has grown explosively. Trajectory data has spatio-temporal characteristics and rich information. Using trajectory data processing techniques can mine the patterns of human activities and the moving patterns of vehicles in the intelligent transportation systems. A trajectory similarity measure is one of the most important issues in trajectory data mining (clustering, classification, frequent pattern mining, etc.). Unfortunately, the main similarity measure algorithms with the trajectory data have been found to be inaccurate, highly sensitive of sampling methods, and have low robustness for the noise data. To solve the above problems, three distances and their corresponding computation methods are proposed in this paper. The point-segment distance can decrease the sensitivity of the point sampling methods. The prediction distance optimizes the temporal distance with the features of trajectory data. The segment-segment distance introduces the trajectory shape factor into the similarity measurement to improve the accuracy. The three kinds of distance are integrated with the traditional dynamic time warping algorithm (DTW) algorithm to propose a new segment–based dynamic time warping algorithm (SDTW). The experimental results show that the SDTW algorithm can exhibit about 57%, 86%, and 31% better accuracy than the longest common subsequence algorithm (LCSS), and edit distance on real sequence algorithm (EDR) , and DTW, respectively, and that the sensitivity to the noise data is lower than that those algorithms.
ARTICLE | doi:10.20944/preprints201608.0040.v1
Subject: Life Sciences, Microbiology Keywords: seeds; ELISA; Fusarium; morphological data analysis; mycotoxins; phylogenetic analysis S
Online: 4 August 2016 (10:12:54 CEST)
Adlay seed samples were collected from 3 adlay growing regions (Yeoncheon, Jeonnam and Eumseong regions) in Korea during 2012. Among all the samples collected, 400 seeds were tested for fungal occurrence by standard blotter and test tube agar methods and different taxonomic groups of fungal genera were detected. The most predominant fungal genera encountered were Fusarium, Phoma, Alternaria, Cladosporium, Curvularia, Cochliobolus and Leptosphaerulina. The occurrence of Fusarium species were 45.6% and based on the combined sequences of two protein coding genes, EF-1a, Beta-tubulin and phylogenetic analysis, 10 species were characterized as F. incarnatum (11.67%), F. kyushense (10.33%), F. fujikuroi (8.67%), F. concentricum (6.00%), F. asiaticum (5.67%), F. graminearum (1.67%), F. miscanthi (0.67%), F. polyphialidiom (0.33%), F. armeniacum (0.33%) and F. thapsinum (0.33%). The ability of these isolates to produce mycotoxins fumonisin (FUM) and zeralenone (ZEN) were tested by ELISA quantitative analysis method. The result revealed that fumonisin (FUM) was produced only by F. fujikuroi and zeralenone (ZEN) by F. asiaticum & F. graminearum. Mycotoxigenic species were then examined for their morphological characteristics to confirm their identity. Morphological observations of the species correlated well with their molecular identification and confirmed as F. asiaticum, F. fujikuroi and F. graminearum.
ARTICLE | doi:10.20944/preprints201609.0027.v1
Subject: Social Sciences, Organizational Economics & Management Keywords: customer complaint process improvement; customer complaint service; big data analysis
Online: 7 September 2016 (11:38:33 CEST)
With the advances in industry and commerce, passengers have become more accepting of environmental sustainability issues; thus, more people now choose to travel by bus. Government administration constitutes an important part of bus transportation services as the government gives the right-of-way to transportation companies allowing them to provide services. When these services are of poor quality, passengers may lodge complaints. The increase in consumer awareness and developments in wireless communication technologies have made it possible for passengers to easily and immediately submit complaints about transportation companies to government institutions, which has brought drastic changes to the supply-demand chain comprised of the public sector, transportation companies, and passengers. This study proposed the use of big data analysis technology including systematized case assignment and data visualization to improve management processes in the public sector and optimize customer complaint services. Taichung City, Taiwan was selected as the research area. There, the customer complaint management process in public sector was improved, effectively solving such issues as station-skipping, allowing the public sector to fully grasp the service level of transportation companies, improving the sustainability of bus operations, and supporting the sustainable development of the public sector-transportation company-passenger supply chain.
ARTICLE | doi:10.20944/preprints202209.0271.v1
Subject: Mathematics & Computer Science, Analysis Keywords: COVID-19; human mobility; spatial autocorrelation; temporal autocorrelation; Facebook mobility data
Online: 19 September 2022 (09:33:10 CEST)
COVID-19 is the most severe health crisis of the 21st century. COVID-19 presents a threat to almost all countries world-wide. The restriction of human mobility is one of the strategies used to control the transmission of COVID-19. However, it has yet to be determined how effective this restriction is in controlling the rise in COVID-19 cases, particularly in major capital cities such as Jakarta, Indonesia. Using Facebook's mobility data, our study explores the impact of restricting human mobility on COVID-19 case control in Jakarta. Our main contribution is showing how the restriction of human mobility data can give important information about how COVID-19 spreads in different places. We proposed modifying a global regression model into a local regression model by accounting for the spatial and temporal interdependence of COVID-19 transmission across space and time. We applied Bayesian hierarchical Poisson spatiotemporal models with spatially varying regression coefficients. We estimated the regression parameters using an Integrated Nested Laplace Approximation. We found that the local regression model with spatially varying regression coefficients outperforms the global regression model based on DIC, WAIC, MPL, and R2 criteria for model selection. In Jakarta's 44 districts, the impact of human mobility varies significantly. The impacts of human mobility on the log relative risk of COVID-19 range from –4.445 to 2.353. The prevention strategy involving the restriction of human mobility may be beneficial in some districts but ineffective in others. Therefore, a cost-effective strategy had to be adopted.
Subject: Engineering, Electrical & Electronic Engineering Keywords: optical fibre data; transmission; microcomb
Online: 15 March 2020 (15:20:23 CET)
Micro-combs [1-4] - optical frequency combs generated by integrated micro-cavity resonators – offer the full potential of their bulk counterparts [5,6], but in an integrated footprint. The discovery of temporal soliton states (DKS – dissipative Kerr solitons) [4,7-11] as a means of mode-locking micro-combs has enabled breakthroughs in many fields including spectroscopy [12,13], microwave photonics , frequency synthesis , optical ranging [16,17], quantum sources [18,19], metrology [20,21] and more. One of their most promising applications has been optical fibre communications where they have enabled massively parallel ultrahigh capacity multiplexed data transmission [22,23]. Here, by using a new and powerful class of micro-comb called “soliton crystals” , we achieve unprecedented data transmission over standard optical fibre using a single integrated chip source. We demonstrate a line rate of 44.2 Terabits per second (Tb/s) using the telecommunications C-band at 1550nm with a spectral efficiency – a critically important performance metric - of 10.4 bits/s/Hz. Soliton crystals exhibit robust and stable generation and operation as well as a high intrinsic efficiency that, together with a low soliton micro-comb spacing of 48.9 GHz enable the use of a very high coherent data modulation format of 64 QAM (quadrature amplitude modulated). We demonstrate error free transmission over 75 km of standard optical fibre in the laboratory as well as in a field trial over an installed metropolitan optical fibre network. These experiments were greatly aided by the ability of the soliton crystals to operate without stabilization or feedback control. This work demonstrates the capability of optical soliton crystal micro-combs to perform in demanding and practical optical communications networks.
ARTICLE | doi:10.20944/preprints201608.0202.v2
Subject: Earth Sciences, Environmental Sciences Keywords: HR satellite remote sensing; urban fabric vulnerability; UHI & heat waves; landsat & MODIS sensors; LST & urban heating; segmentation & objects classification; data mining; feature extraction & selection; stepwise regression & model calibration
Online: 26 October 2021 (13:11:23 CEST)
Densely urbanized areas, with a low percentage of green vegetation, are highly exposed to Heat Waves (HW) which nowadays are increasing in terms of frequency and intensity also in the middle-latitude regions, due to ongoing Climate Change (CC). Their negative effects may combine with those of the UHI (Urban Heat Island), a local phenomenon where air temperatures in the compact built up cores of towns increase more than those in the surrounding rural areas, with significant impact on the quality of urban environment, on citizens health and energy consumption and transport, as it has occurred in the summer of 2003 on France and Italian central-northern areas. In this context this work aims at designing and developing a methodology based on aero-spatial remote sensing (EO) at medium-high resolution and most recent GIS techniques, for the extensive characterization of the urban fabric response to these climatic impacts related to the temperature within the general framework of supporting local and national strategies and policies of adaptation to CC. Due to its extension and variety of built-up typologies, the municipality of Rome was selected as test area for the methodology development and validation. First of all, we started by operating through photointerpretation of cartography at detailed scale (CTR 1: 5000) on a reference area consisting of a transect of about 5x20 km, extending from the downtown to the suburbs and including all the built-up classes of interest. The reference built-up vulnerability classes found inside the transect were then exploited as training areas to classify the entire territory of Rome municipality. To this end, the satellite EO HR (High Resolution) multispectral data, provided by the Landsat sensors were used within a on purpose developed "supervised" classification procedure, based on data mining and “object-classification” techniques. The classification results were then exploited for implementing a calibration method, based on a typical UHI temperature distribution, derived from MODIS satellite sensor LST (Land Surface Temperature) data of the summer 2003, to obtain an analytical expression of the vulnerability model, previously introduced on a semi-empirical basis.
ARTICLE | doi:10.20944/preprints202003.0268.v1
Subject: Social Sciences, Library & Information Science Keywords: matching; data marketplace; data platform; data visualization; call for data
Online: 17 March 2020 (04:10:28 CET)
Improvements in web platforms for data exchange and trading are creating more opportunities for users to obtain data from data providers of different domains. However, the current data exchange platforms are limited to unilateral information provision from data providers to users. In contrast, there are insufficient means for data providers to learn what kinds of data users desire and for what purposes. In this paper, we propose and discuss the description items for sharing users’ call for data as data requests in the data marketplace. We also discuss structural differences in data requests and providable data using variables, as well as possibilities of data matching. In the study, we developed an interactive platform, treasuring every encounter of data affairs (TEEDA), to facilitate matching and interactions between data providers and users. The basic features of TEEDA are described in this paper. From experiments, we found the same distributions of the frequency of variables but different distributions of the number of variables in each piece of data, which are important factors to consider in the discussion of data matching in the data marketplace.
ARTICLE | doi:10.20944/preprints202103.0593.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Business Inteligence; Data Mining; Data Warehouse.
Online: 24 March 2021 (13:47:31 CET)
In the coming years, digital applications and services that continue to use the country's native cloud systems will be huge. By 2023, that will exceed 500 million, according to IDC. This corresponds to the sum of all applications developed in the last 40 years. If you are the one you answered, yes! This article is for you!
Subject: Engineering, Automotive Engineering Keywords: Business Intelligence; Data warehouse; Data Marts; Architecture; Data; Information; cloud; Data Mining; evolution; technologic companies; tools; software
Online: 24 March 2021 (13:06:53 CET)
Information has been and will be a vital element for a person or department groups in an organization. That is why there are technologies that help us to give them the proper management of data; Business Intelligence is responsible for bringing technological solutions that correctly and effectively manage the entire volume of necessary and important information for companies. Among the solutions offered by Business Intelligence are Data Warehouses, Data Mining, among other business technologies that working together achieve the objectives proposed by an organization. It is important to highlight that these business technologies have been present since the 50's and have been evolving through time, improving processes, infrastructure, methodologies and implementing new technologies, which have helped to correct past mistakes based on information management for companies. There are questions about Business Intelligence. Could it be that in the not-too-distant future it will be used as an essential standard or norm in any organization for data management, since it provides many benefits and avoids failures at the time of classifying information. On the other hand, Cloud storage has been the best alternative to safeguard information and not depend on physical storage media, which are not 100% secure and are exposed to partial or total loss of information, by presenting hardware failures or security failures due to mishandling that can be given to such information.
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Academic Analytics; data storage; education and big data; analysis of data; learning analytics
Online: 19 July 2020 (20:37:39 CEST)
Business Intelligence, defined by  as "the ability to understand the interrelations of the facts that are presented in such a way that it can guide the action towards achieving a desired goal", has been used since 1958 for the transformation of data into information, and of information into knowledge, to be used when making decisions in a business environment. But, what would happen if we took the same principles of business intelligence and applied them to the academic environment? The answer would be the creation of Academic Analytics, a term defined by  as the process of evaluating and analyzing organizational information from university systems for reporting and making decisions, whose characteristics allow it to be used more and more in institutions, since the information they accumulate about their students and teachers gathers data such as academic performance, student success, persistence, and retention . Academic Analytics enables an analysis of data that is very important for making decisions in the educational institutional environment, aggregating valuable information in the academic research activity and providing easy to use business intelligence tools. This article shows a proposal for creating an information system based on Academic Analytics, using ASP.Net technology and trusting storage in the database engine Microsoft SQL Server, designing a model that is supported by Academic Analytics for the collection and analysis of data from the information systems of educational institutions. The idea that was conceived proposes a system that is capable of displaying statistics on the historical data of students and teachers taken over academic periods, without having direct access to institutional databases, with the purpose of gathering the information that the director, the teacher, and finally the student need for making decisions. The model was validated with information taken from students and teachers during the last five years, and the export format of the data was pdf, csv, and xls files. The findings allow us to state that it is extremely important to analyze the data that is in the information systems of the educational institutions for making decisions. After the validation of the model, it was established that it is a must for students to know the reports of their academic performance in order to carry out a process of self-evaluation, as well as for teachers to be able to see the results of the data obtained in order to carry out processes of self-evaluation, and adaptation of content and dynamics in the classrooms, and finally for the head of the program to make decisions.
ARTICLE | doi:10.20944/preprints202012.0468.v1
Online: 18 December 2020 (13:29:38 CET)
This manuscript describes the construction and validation of high resolution daily gridded (0.05° × 0.05°) rainfall and maximum and minimum temperature data for Bangladesh : the Enhancing National Climate Services for Bangladesh Meteorological Department (ENACTS-BMD) dataset. The dataset was generated by merging data from weather stations, satellite products (for rainfall) and reanalysis (for temperature). ENACTS-BMD is the first high-resolution gridded surface meteorological dataset developed specifically for studies of surface climate processes in Bangladesh. Its record begins in January 1981 and is updated in real-time monthly and outputs have daily, decadal and monthly time resolution. The Climate Data Tools (CDT), developed by the International Research Institute for Climate and Society (IRI), Columbia University, is used to generate the dataset. This data processing includes the collection of weather and gridded data, quality control of stations data, downscaling of the reanalysis for temperature, bias correction of both satellite rainfall and downscaled reanalysis of temperature, and the combination of station and bias-corrected gridded data. The ENACTS-BMD dataset is available as an open-access product at BMD’s official website, allowing the enhancement of the provision of services, overcoming the challenges of data quality, availability, and access, promoting at the same time the engagement and use by stakeholders.
ARTICLE | doi:10.20944/preprints201701.0090.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: transportation data; data interlinking; automatic schema matching
Online: 20 January 2017 (03:38:06 CET)
Multimodality requires integration of heterogeneous transportation data to construct a broad view of the transportation network. Many new transportation services are emerging with being isolated from previously existing networks. This lead them to publish their data sources to the web -- according to Linked Data Principles -- in order to gain visibility. Our interest is to use these data to construct an extended transportation network that links these new services to existing ones. The main problems we tackle in this article fall in the categories of automatic schema matching and data interlinking. We propose an approach that uses web services as mediators to help in automatically detect geospatial properties and map them between two different schemas. On the other hand, we propose a new interlinking approach that enables user to define rich semantic links between datasets in a flexible and customizable way.
ARTICLE | doi:10.20944/preprints202111.0410.v1
Subject: Engineering, Other Keywords: Data compression; data hiding; psnr; mse; virtual data; public cloud; quantization error
Online: 22 November 2021 (15:17:12 CET)
Nowadays, information security is a challenge especially when transmitted or shared in public clouds. Many of researchers have been proposed technique which fails to provide data integrity, security, authentication and another issue related to sensitivity data. The most common techniques were used to protect data during transmission on public cloud are cryptography, steganography, and compression. The proposed scheme suggests an entirely new approach for data security on public cloud. Authors have suggested an entirely new approach that completely makes secret data invisible behind carrier object and it is not been detected with the image performance parameters like PSNR, MSE, entropy and others. The details of results are explain in result section of paper. Proposed technique have better outcome than any other existing technique as a security mechanism on a public cloud. Primary focus of suggested approach is to minimize integrity loss of public storage data due to unrestricted access rights by uses. To improve reusability of carrier even after data concealed is really a challenging task and achieved through suggested approach.
DATA DESCRIPTOR | doi:10.20944/preprints202109.0370.v1
Subject: Engineering, Energy & Fuel Technology Keywords: smart meter data; household survey; EPC; energy data; energy demand; energy consumption; longitudinal; energy modelling; electricity data; gas data
Online: 22 September 2021 (10:16:05 CEST)
The Smart Energy Research Lab (SERL) Observatory dataset described here comprises half-hourly and daily electricity and gas data, SERL survey data, Energy Performance Certificate (EPC) input data and 24 local hourly climate reanalysis variables from the European Centre for Medium-Range Weather Forecasts (ECMWF) for over 13,000 households in Great Britain (GB). Participants were recruited in September 2019, September 2020 and January 2021 and their smart meter data are collected from up to one year prior to sign up. Data collection will continue until at least August 2022, and longer if funding allows. Survey data relating to the dwelling, appliances, household demographics and attitudes was collected at sign up. Data are linked at the household level and UK-based academic researchers can apply for access within a secure virtual environment for research projects in the public interest. This is a data descriptor paper describing how the data was collected, the variables available and the representativeness of the sample compared to national estimates. It is intended as a guide for researchers working with or considering using the SERL Observatory dataset, or simply looking to learn more about it.
Subject: Social Sciences, Econometrics & Statistics Keywords: poverty; composite indicators; interval data; symbolic data
Online: 24 August 2021 (15:46:09 CEST)
The analysis and measurement of poverty is a crucial issue in the field of social science. Poverty is a multidimensional notion that can be measured using composite indicators relevant to synthesizing statistical indicators. Subjective choices could, however, affect these indicators. We propose interval-based composite indicators to avoid the problem, enabling us in this context to obtain robust and reliable measures. Based on a relevant conceptual model of poverty we have identified, we will consider all the various factors identified. Then, considering a different random configuration of the various factors, we will compute a different composite indicator. We can obtain a different interval for each region based on the distinct factor choices on the different assumptions for constructing the composite indicator. So we will create an interval-based composite indicator based on the results obtained by the Monte-Carlo simulation of all the different assumptions. The different intervals can be compared, and various rankings for poverty can be obtained. For their parameters, such as center, minimum, maximum, and range, the poverty interval composite indicator can be considered and compared. The results demonstrate a relevant and consistent measurement of the indicator and the shadow sector's relevant impact on the final measures.
Online: 30 October 2020 (15:35:00 CET)
In the information age today, data are getting more and more important. While other industries achieve tangible improvement by applying cutting edge information technology, the construction industry is still far from being enough. Cost, schedule, and performance control are three major functions in the project execution phase. Along with their individual importance, cost-schedule integration has been a significant challenge over the past five decades in the construction industry. Although a lot of efforts have been put into this development, there is no method used in construction practice. The purpose of this study is to propose a new method to integrate cost and schedule data using big data technology. The proposed algorithm is designed to provide data integrity and flexibility in the integration process, considerable time reduction on building and changing database, and practical use in a construction site. It is expected that the proposed method can transform the current way that field engineers regard information management as one of the troublesome tasks in a data-friendly way.
REVIEW | doi:10.20944/preprints202003.0141.v1
Subject: Medicine & Pharmacology, General Medical Research Keywords: data sharing; data management; data science; big data; healthcare
Online: 8 March 2020 (16:46:20 CET)
In recent years, more and more health data are being generated. These data come not only from professional health systems, but also from wearable devices. All these data combined form ‘big data’ that can be utilized to optimize treatments for each unique patient (‘precision medicine’). To achieve this precision medicine, it is necessary that hospitals, academia and industry work together to bridge the ‘valley of death’ of translational medicine. However, hospitals and academia often have problems with sharing their data, even though the patient is actually the owner of his/her own health data, and the sharing of data is associated with increased citation rate. Academic hospitals usually invest a lot of time in setting up clinical trials and collecting data, and want to be the first ones to publish papers on this data. The idea that society benefits the most if the patient’s data are shared as soon as possible so that other researchers can work with it, has not taken root yet. There are some publicly available datasets, but these are usually only shared after studies are finished and/or publications have been written based on the data, which means a severe delay of months or even years before others can use the data for analysis. One solution is to incentivize the hospitals to share their data with (other) academic institutes and the industry. Here we discuss several aspects of data sharing in the medical domain: publisher requirements, data ownership, support for data sharing, data sharing initiatives and how the use of federated data might be a solution. We also discuss some potential future developments around data sharing.
ARTICLE | doi:10.20944/preprints201812.0071.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: data governance; data sovereignty; urban data spaces; ICT reference architecture; open urban platform
Online: 6 December 2018 (05:09:54 CET)
This paper presents the results of a recent study that was conducted with a number of German municipalities/cities. Based on the obtained and briefly presented recommendations emerging from the study, the authors propose the concept of an Urban Data Space (UDS), which facilitates an eco-system for data exchange and added value creation thereby utilizing the various types of data within a smart city/municipality. Looking at an Urban Data Space from within a German context and considering the current situation and developments in German municipalities, this paper proposes a reasonable classification of urban data that allows to relate the various data types to legal aspects and to conduct solid considerations regarding technical implementation designs and decisions. Furthermore, the Urban Data Space is described/analyzed in detail, and relevant stakeholders are identified, as well as corresponding technical artifacts are introduced. The authors propose to setup Urban Data Spaces based on emerging standards from the area of ICT reference architectures for Smart Cities, such as DIN SPEC 91357 “Open Urban Platform” and EIP SCC. Thereby, the paper walks the reader through the construction of an UDS based on the above mentioned architectures and outlines all the goals, recommendations and potentials, which an Urban Data Space can reveal to a municipality/city.
SHORT NOTE | doi:10.20944/preprints202001.0196.v1
Subject: Biology, Entomology Keywords: reproducibility; open access; data curation; data mangement; pre-print servers
Online: 18 January 2020 (09:05:49 CET)
The ability to replicate scientific experiments is a cornerstone of the scientific method. Sharing ideas, workflows, data, and protocols facilitates testing the generalizability of results, increases the speed that science progresses, and enhances quality control of published work. Fields of science such as medicine, the social sciences, and the physical sciences have embraced practices designed to increase replicability. Granting agencies, for example, may require data management plans and journals may require data and code availability statements along with the deposition of data and code in publicly available repositories. While many tools commonly used in replicable workflows such as distributed version control systems (e.g. “git”) or scripted programming languages for data cleaning and analysis may have a steep learning curve, their adoption can increase individual efficiency and facilitate collaborations both within entomology and across disciplines. The open science movement is developing within the discipline of entomology, but practitioners of these concepts or those desiring to work more collaboratively across disciplines may be unsure where or how to embrace these initiatives. This article is meant to introduce some of the tools entomologists can incorporate into their workflows to increase the replicability and openness of their work. We describe these tools and others, recommend additional resources for learning more about these tools, and discuss the benefits to both individuals and the scientific community and potential drawbacks associated with implementing a replicable workflow.
ARTICLE | doi:10.20944/preprints202206.0320.v4
Subject: Life Sciences, Other Keywords: data; reproducibility; FAIR; data reuse; public data; big data; analysis
Online: 2 November 2022 (02:55:49 CET)
With an increasing amount of biological data available publicly, there is a need for a guide on how to successfully download and use this data. The Ten simple rules for using public biological data are: 1) use public data purposefully in your research, 2) evaluate data for your use case, 3) check data reuse requirements and embargoes, 4) be aware of ethics for data reuse, 5) plan for data storage and compute requirements, 6) know what you are downloading, 7) download programmatically and verify integrity, 8) properly cite data, 9) make reprocessed data and models Findable, Accessible, Interoperable, and Reusable (FAIR) and share, and 10) make pipelines and code FAIR and share. These rules are intended as a guide for researchers wanting to make use of available data and to increase data reuse and reproducibility.
ARTICLE | doi:10.20944/preprints202109.0518.v1
Subject: Earth Sciences, Environmental Sciences Keywords: data fusion; multi-sensor; data visualization; data treatment; participant reports; air quality; exposure assessment
Online: 30 September 2021 (14:13:52 CEST)
Use of a multi-sensor approach can provide citizens a holistic insight in the air quality in their immediate surroundings and assessment of personal exposure to urban stressors. Our work, as part of the ICARUS H2020 project, which included over 600 participants from 7 European cities, discusses data fusion and harmonization on a diverse set of multi-sensor data streams to provide a comprehensive and understandable report for participants, and offers possible solutions and improvements. Harmonizing the data streams identified issues with the used devices and protocols, such as non-uniform timestamps, data gaps, difficult data retrieval from commercial devices, and coarse activity data logging. Our process of data fusion and harmonization allowed us to automate the process of generating visualizations and reports and consequently provide each participant with a detailed individualized report. Results showed that a key solution was to streamline the code and speed up the process, which necessitated certain compromises in visualizing the data. A thought-out process of data fusion and harmonization on a diverse set of multi-sensor data streams considerably improved the quality and quantity of data that a research participant receives. Though automatization accelerated the production of the reports considerably, manual structured double checks are strongly recommended.
COMMUNICATION | doi:10.20944/preprints201803.0054.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: data feature selection; data clustering; travel time prediction
Online: 7 March 2018 (13:30:06 CET)
In recent years, governments applied intelligent transportation system (ITS) technique to provide several convenience services (e.g., garbage truck app) for residents. This study proposes a garbage truck fleet management system (GTFMS) and data feature selection and data clustering methods for travel time prediction. A GTFMS includes mobile devices (MD), on-board units, fleet management server, and data analysis server (DAS). When user uses MD to request the arrival time of garbage truck, DAS can perform the procedure of data feature selection and data clustering methods to analyses travel time of garbage truck. The proposed methods can cluster the records of travel time and reduce variation for the improvement of travel time prediction. After predicting travel time and arrival time, the predicted information can be sent to user’s MD. In experimental environment, the results showed that the accuracies of previous method and proposed method are 16.73% and 85.97%, respectively. Therefore, the proposed data feature selection and data clustering methods can be used to predict stop-to-stop travel time of garbage truck.
ARTICLE | doi:10.20944/preprints202110.0260.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: big data; data acquisition; data visualization; data exchange; dashboard; frequency stability; Grafana lab; Power Quality; GPS reference; frequency measurement.
Online: 18 October 2021 (18:07:43 CEST)
This article proposes a measurement solution designed to monitor instantaneous frequency in power systems. It uses a data acquisition module and a GPS receiver for time stamping. A program in Python takes care of receiving the data, calculating the frequency, and finally transferring the measurement results to a database. The frequency is calculated with two different methods, which are compared in the article. The stored data is visualized using the Grafana platform, thus demonstrating its potential for comparing scientific data. The system as a whole constitutes an efficient low cost solution as a data acquisition system.
ARTICLE | doi:10.20944/preprints201806.0185.v1
Subject: Medicine & Pharmacology, Nursing & Health Studies Keywords: mHealth; mobile data collection; data quality; data quality assessment framework; Tuberculosis control; developing countries
Online: 12 June 2018 (10:34:33 CEST)
Background Increasingly, healthcare organizations are using technology for the efficient management of data. The aim of this study was to compare the data quality of digital records with the quality of the corresponding paper-based records by using data quality assessment framework. Methodology We conducted a desk review of paper-based and digital records over the study duration from April 2016 to July 2016 at six enrolled TB clinics. We input all data fields of the patient treatment (TB01) card into a spreadsheet-based template to undertake a field-to-field comparison of the shared fields between TB01 and digital data. Findings A total of 117 TB01 cards were prepared at six enrolled sites, whereas just 50% of the records (n=59; 59 out of 117 TB01 cards) were digitized. There were 1,239 comparable data fields, out of which 65% (n=803) were correctly matched between paper based and digital records. However, 35% of the data fields (n=436) had anomalies, either in paper-based records or in digital records. 1.9 data quality issues were calculated per digital patient record, whereas it was 2.1 issues per record for paper-based record. Based on the analysis of valid data quality issues, it was found that there were more data quality issues in paper-based records (n=123) than in digital records (n=110). Conclusion There were fewer data quality issues in digital records as compared to the corresponding paper-based records. Greater use of mobile data capture and continued use of the data quality assessment framework can deliver more meaningful information for decision making.
ARTICLE | doi:10.20944/preprints202206.0335.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: metadata; contextual data; harmonization; genomic surveillance; data management
Online: 24 June 2022 (08:46:04 CEST)
REVIEW | doi:10.20944/preprints202208.0420.v1
Subject: Social Sciences, Law Keywords: conversational commerce; data protection; law of obligations of data
Online: 24 August 2022 (10:55:06 CEST)
The possibilities and reach of social networks are increasing, the designs are becoming more diverse, and the ideas more visionary. Most recently, the former company “Facebook” announced the creation of a metaverse. With these technical possibilities, however, the danger of fraudsters is also growing. Using social bots, consumers are increasingly influenced on such platforms and business transactions are brought about through communication, i.e. conversational commerce. Minors or the elderly are particularly susceptible. This technical development is accompanied by a legal one: it is permitted by the Digital Services Directive and the Sale of Goods Directive to demand the provision of data as consideration for the sale of digital products. This raises legal problems at the level of the law of obligations and data protection law, whose regulations are intended to protect the aforementioned groups of individuals. This protection becomes even more important the more manipulative consumers are influenced by communicative bots. We show that there is a lack of knowledge about what objective data value can have in business transactions. Sufficient transparency of an objective data value can maintain legal protection, especially of vulnerable groups, and ensure the purpose of the laws.
Subject: Earth Sciences, Environmental Sciences Keywords: forest inventory; data harvesting; forest modeling; forest growth; macroecology; public data
Online: 26 November 2020 (10:38:58 CET)
Net CO2 emissions and sequestration from European forests are the result of removal and growth of flora. To arrive at aggregated measurements of these processes at a country's level, local observations of increments and harvest rates are up-scaled to national forest areas. Each country releases these statistics through their individual National Forest Inventory using their particular definitions and methodologies. In addition, five international processes deal with the harmonization and comparability of such forest datasets in Europe, namely the IPCC, SOEF, FAOSTAT, HPFFRE, FRA (definitions follow in the article). In this study, we retrieved living biomass dynamics from each of these sources for 27 European Union member states. To demonstrate the reproducibility of our method, we release an open source python package that allows for automated data retrieval and analysis, as new data becomes available. The comparison of the published values shows discrepancies in the magnitude of forest biomass changes for several countries. In some cases, the direction of these changes also differ between sources. The scarcity of the data provided, along with the low spatial resolution, forbids the creation or calibration of a pan-European forest dynamics model, which could ultimately be used to simulate future scenarios and support policy decisions. To attain these goals, an improvement in forest data availability and harmonization is needed.
ARTICLE | doi:10.20944/preprints201804.0054.v1
Subject: Earth Sciences, Other Keywords: metadata; documentation; data life-cycle; metadata life-cycle; hierarchical data
Online: 4 April 2018 (08:16:15 CEST)
The historic view of metadata as “data about data” is expanding to include data about other items that must be created, used and understood throughout the data and project life cycles. In this context, metadata might better be defined as the structured and standard part of documentation and the metadata life cycle can be described as the metadata content that is required for documentation in each phase of the project and data life cycles. This incremental approach to metadata creation is similar to the spiral model used in software development. Each phase also has distinct users and specific questions they need answers to. In many cases, the metadata life cycle involves hierarchies where latter phases have increased numbers of items. The relationships between metadata in different phases can be captured through structure in the metadata standard or through conventions for identifiers. Metadata creation and management can be streamlined and simplified by re-using metadata across many records. Many of these ideas are being used in metadata for documenting the life cycle of research projects in the Arctic.
ARTICLE | doi:10.20944/preprints202106.0738.v1
Subject: Earth Sciences, Atmospheric Science Keywords: time series; homogenization; ACMANT; observed data; data accuracy
Online: 30 June 2021 (13:08:39 CEST)
The removal of non-climatic biases, so-called inhomogeneities, from long climatic records needs sophistically developed statistical methods. One principle is that usually the differences between a candidate series and its neighbour series are analysed instead of directly the candidate series, in order to neutralize the possible impacts of regionally common natural climate variation on the detection of inhomogeneities. In most homogenization methods, two main kinds of time series comparisons are applied, i.e. composite reference series or pairwise comparisons. In composite reference series the inhomogeneities of neighbour series are attenuated by averaging the individual series, and the accuracy of homogenization can be improved by the iterative improvement of composite reference series. By contrast, pairwise comparisons have the advantage that coincidental inhomogeneities affecting several station series in a similar way can be identified with higher certainty than with composite reference series. In addition, homogenization with pairwise comparisons tends to facilitate the most accurate regional trend estimations. A new time series comparison method is presented here, which combines the use of pairwise comparisons and composite reference series in a way that their advantages are unified. This time series comparison method is embedded into the ACMANT homogenization method, and tested in large, commonly available monthly temperature test datasets.
ARTICLE | doi:10.20944/preprints202006.0258.v2
Subject: Engineering, Civil Engineering Keywords: Conservation laws; Data inference; Data discovery; Dimensionless form; PINN
Online: 30 September 2020 (03:51:25 CEST)
Deep learning has achieved remarkable success in diverse computer science applications, however, its use in other traditional engineering fields has emerged only recently. In this project, we solved several mechanics problems governed by differential equations, using physics informed neural networks (PINN). The PINN embeds the differential equations into the loss of the neural network using automatic differentiation. We present our developments in the context of solving two main classes of problems: data-driven solutions and data-driven discoveries, and we compare the results with either analytical solutions or numerical solutions using the finite element method. The remarkable achievements of the PINN model shown in this report suggest the bright prospect of the physics-informed surrogate models that are fully differentiable with respect to all input coordinates and free parameters. More broadly, this study shows that PINN provides an attractive alternative to solve traditional engineering problems.
ARTICLE | doi:10.20944/preprints202208.0224.v1
Subject: Engineering, Automotive Engineering Keywords: VR-XGBoost; K-VDTE; ETC data; ESAs; data mining
Online: 12 August 2022 (03:53:23 CEST)
To scientifically and effectively evaluate the service capacity of expressway service areas (ESAs) and improve the management level of ESAs, we propose a method for the recognition of vehicles entering ESAs (VeESAs) and estimation of vehicle dwell times using ETC data. First, the ETC data and their advantages are described in detail, and then the cleaning rules are designed according to the characteristics of the ETC data. Second, we established feature engineering according to the characteristics of VeESA, and proposed the XGBoost-based VeESA recognition (VR-XGBoost) model. Studied the driving rules in depth, we constructed a kinematics-based vehicle dwell time estimation (K-VDTE) model. The field validation in Part A/B of Yangli ESA using real ETC transaction data demonstrates that the effectiveness of our proposal outperforms the current state of the art. Specifically, in Part A and Part B, the recognition accuracies of VR-XGBoost are 95.9% and 97.4%, respectively, the mean absolute errors (MAEs) of dwell time are 52 s and 14 s, respectively, and the root mean square errors (RMSEs) are 69 s and 22 s, respectively. In addition, the confidence level of controlling the MAE of dwell time within 2 minutes is more than 97%. This work can effectively identify the VeESA, and accurately estimate the dwell time, which can provide a reference idea and theoretical basis for the service capacity evaluation and layout optimization of the ESA.
ARTICLE | doi:10.20944/preprints202008.0487.v1
Subject: Social Sciences, Geography Keywords: Twitter; data reliability; risk communication; data mining; Google Cloud Vision API
Online: 22 August 2020 (02:32:40 CEST)
While Twitter has been touted to provide up-to-date information about hazard events, the reliability of tweets is still a concern. Our previous publication extracted relevant tweets containing information about the 2013 Colorado flood event and its impacts. Using the relevant tweets, this research further examined the reliability (accuracy and trueness) of the tweets by examining the text and image content and comparing them to other publicly available data sources. Both manual identification of text information and automated (Google Cloud Vision API) extraction of images were implemented to balance accurate information verification and efficient processing time. The results showed that both the text and images contained useful information about damaged/flooded roads/street networks. This information will help emergency response coordination efforts and informed allocation of resources when enough tweets contain geocoordinates or locations/venue names. This research will help identify reliable crowdsourced risk information to enable near-real time emergency response through better use of crowdsourced risk communication platforms.
REVIEW | doi:10.20944/preprints202211.0161.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: High Performance Computing (HPC); big data; High Performance Data Analytics (HPDS); con-vergence; data locality; spark; Hadoop; design patterns; process mapping; in-situ data analysis
Online: 9 November 2022 (01:38:34 CET)
Big data has revolutionised science and technology leading to the transformation of our societies. High Performance Computing (HPC) provides the necessary computational power for big data analysis using artificial intelligence and methods. Traditionally HPC and big data had focused on different problem domains and had grown into two different ecosystems. Efforts have been underway for the last few years on bringing the best of both paradigms into HPC and big converged architectures. Designing HPC and big data converged systems is a hard task requiring careful placement of data, analytics, and other computational tasks such that the desired performance is achieved with the least amount of resources. Energy efficiency has become the biggest hurdle in the realisation of HPC, big data, and converged systems capable of delivering exascale and beyond performance. Data locality is a key parameter of HPDA system design as moving even a byte costs heavily both in time and energy with an increase in the size of the system. Performance in terms of time and energy are the most important factors for users, particularly energy, due to it being the major hurdle in high performance system design and the increasing focus on green energy systems due to environmental sustainability. Data locality is a broad term that encapsulates different aspects including bringing computations to data, minimizing data movement by efficient exploitation of cache hierarchies, reducing intra- and inter-node communications, locality-aware process and thread mapping, and in-situ and in-transit data analysis. This paper provides an extensive review of the cutting-edge on data locality in HPC, big data, and converged systems. We review the literature on data locality in HPC, big data, and converged environments and discuss challenges, opportunities, and future directions. Subsequently, using the knowledge gained from this extensive review, we propose a system architecture for future HPC and big data converged systems. To the best of our knowledge, there is no such review on data locality in converged HPC and big data systems.
ARTICLE | doi:10.20944/preprints202108.0303.v2
Online: 19 November 2021 (08:38:42 CET)
Science continues to become more interdisciplinary and to involve increasingly complex data sets. Many projects in the biomedical and health-related sciences follow or aim to follow the principles of FAIR data sharing, which has been demonstrated to foster collaboration, to lead to better research outcomes, and to help ensure reproducibility of results. Data generated in the course of biomedical and health research present specific challenges for FAIR sharing in the sense that they are heterogeneous and highly sensitive to context and the needs of protection and privacy. Data sharing must respect these features without impeding timely dissemination of results, so that they can contribute to time-critical advances in medical therapy and treatment. Modeling and simulation of biomedical processes have become established tools, and a global community has been developing algorithms, methodologies, and standards for applying biomedical simulation models in clinical research. However, it can be difficult for clinician scientists to follow the specific rules and recommendations for FAIR data sharing within this domain. We seek to clarify the standard workflow for sharing experimental and clinical data with the simulation modeling community. By following these recommendations, data sharing will be improved, collaborations will become more effective, and the FAIR publication and subsequent reuse of data will become possible at the level of quality necessary to support biomedical and health-related sciences.
REVIEW | doi:10.20944/preprints202007.0153.v1
Online: 8 July 2020 (11:53:33 CEST)
Large datasets that enable researchers to perform investigations with unprecedented rigor are growing increasingly common in neuroimaging. Due to the simultaneous increasing popularity of open science, these state-of-the-art datasets are more accessible than ever to researchers around the world. While analysis of these samples has pushed the field forward, they pose a new set of challenges that might cause difficulties for novice users. Here, we offer practical tips for working with large datasets from the end-user’s perspective. We cover all aspects of the data life cycle: from what to consider when downloading and storing the data, to tips on how to become acquainted with a dataset one did not collect, to what to share when communicating results. This manuscript serves as a practical guide one can use when working with large neuroimaging datasets, thus dissolving barriers to scientific discovery.
ARTICLE | doi:10.20944/preprints202105.0377.v1
Subject: Keywords: Sensor data, wireless body area network, wearable devices, sensor data interoperability
Online: 17 May 2021 (09:47:26 CEST)
The monitoring of maternal and child health, using wearable devices made with wireless sensor technologies, is expected to reduce maternal and child death rates. Wireless sensor technologies have been used in wireless sensor networks to enable the acquisition of data for monitoring machines, smart cities, transportation, asset tracking, and tracking of human activity. Applications based on wireless body area network (WBAN) have been used in healthcare for measuring and monitoring of patient health and activity through integration with wearable devices. Wireless sensors used in WBAN can be cost-effective, enable remote availability, and can be integrated with electronic health record (EHR) management systems. Interoperability of WBAN sensor data with other linked data has the potential to improve health for all, including maternal and child health through the improvement of data access, data quality and healthcare access. This paper presents a survey of the state-of-the-art techniques for managing WBAN sensor data interoperability. The findings in this study will provide reliable support to enable policymakers and health care providers to take action to enhance the use of e-health to improve maternal-child health and reduce the mortality rates of women and children.
ARTICLE | doi:10.20944/preprints201810.0273.v1
Subject: Physical Sciences, Astronomy & Astrophysics Keywords: astroparticle physics, cosmic rays, data life cycle management, data curation, meta data, big data, deep learning, open data
Online: 12 October 2018 (14:48:32 CEST)
Modern experimental astroparticle physics features large-scale setups measuring different messengers, namely high-energy particles generated by cosmic accelerators (e.g. supernova remnants, active galactic nuclei, etc): cosmic and gamma rays, neutrinos and recently discovered gravitational waves. Ongoing and future experiments are distributed over the Earth including ground, underground/underwater setups as well as balloon payloads and spacecrafts. The data acquired by these experiments have different formats, storage concepts and publication policies. Such differences are a crucial issue in the era of big data and of multi-messenger analysis strategies in astroparticle physics. We propose a service ASTROPARTICLE.ONLINE in the frame of which we develop an open science system which enables to publish, store, search, select and analyse astroparticle physics data. The cosmic-ray experiments KASCADE-Grande and TAIGA were chosen as pilot experiments to be included in this framework. In the first step of our initiative we will develop and test the following components of the full data life cycle concept: (i) describing, storing and reusing of astroparticle data; (ii) software for performing multi-experiment and multi-messenger analyses like deep-learning methods; (iii) outreach including example applications and tutorial for students and scientists outside the specific research field. In the present paper we describe the concepts of our initiative, and in particular the plans toward a common, federated astroparticle data storage.
ARTICLE | doi:10.20944/preprints201710.0076.v2
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: big data; machine learning; regularization; data quality; robust learning framework
Online: 17 October 2017 (03:47:41 CEST)
The concept of ‘big data’ has been widely discussed, and its value has been illuminated throughout a variety of domains. To quickly mine potential values and alleviate the ever-increasing volume of information, machine learning is playing an increasingly important role and faces more challenges than ever. Because few studies exist regarding how to modify machine learning techniques to accommodate big data environments, we provide a comprehensive overview of the history of the evolution of big data, the foundations of machine learning, and the bottlenecks and trends of machine learning in the big data era. More specifically, based on learning principals, we discuss regularization to enhance generalization. The challenges of quality in big data are reduced to the curse of dimensionality, class imbalances, concept drift and label noise, and the underlying reasons and mainstream methodologies to address these challenges are introduced. Learning model development has been driven by domain specifics, dataset complexities, and the presence or absence of human involvement. In this paper, we propose a robust learning paradigm by aggregating the aforementioned factors. Over the next few decades, we believe that these perspectives will lead to novel ideas and encourage more studies aimed at incorporating knowledge and establishing data-driven learning systems that involve both data quality considerations and human interactions.
ARTICLE | doi:10.20944/preprints202111.0019.v1
Subject: Engineering, Industrial & Manufacturing Engineering Keywords: Industry 4.0; Database; Data models; Big Data & Analytics; Asset Administration Shell
Online: 1 November 2021 (13:01:51 CET)
The data-oriented paradigm has proven to be fundamental for the technological transformation process that characterizes Industry 4.0 (I4.0) so that Big Data & Analytics is considered a technological pillar of this process. The literature reports a series of system architecture proposals that seek to implement the so-called Smart Factory, which is primarily data-driven. Many of these proposals treat data storage solutions as mere entities that support the architecture's functionalities. However, choosing which logical data model to use can significantly affect the performance of the architecture. This work identifies the advantages and disadvantages of relational (SQL) and non-relational (NoSQL) data models for I4.0, taking into account the nature of the data in this process. The characterization of data in the context of I4.0 is based on the five dimensions of Big Data and a standardized format for representing information of assets in the virtual world, the Asset Administration Shell. This work allows identifying appropriate transactional properties and logical data models according to the volume, variety, velocity, veracity, and value of the data. In this way, it is possible to describe the suitability of SQL and NoSQL databases for different scenarios within I4.0.
ARTICLE | doi:10.20944/preprints202105.0589.v1
Subject: Engineering, Automotive Engineering Keywords: Game Ratings; Public Data; Game Data; Data analysis; GRAC(Korea)
Online: 25 May 2021 (08:32:32 CEST)
As of 2020, public data for game ratings provided by Game Ratings And Administration Committee(GRAC) are more limited than public data for movie and video ratings provided by Korea Media Ratings Board and do not provide data which allow us to see information on ratings clearly and in detail. To get information on game ratings, we need to find information by searching for specific target on homepage which is inconvenient for us. In order to improve such inconvenience and extend scope of provision in public data, the author of this paper intends to study public data API which has been extended based on information on video ratings. To draw items to be extended, this study analyzes data for ratings on homepage of GRAC and designs collection system to build database. This study intends to implement system that provides data collected based on extended public data items in a form which users want. This study is expected to provide information on ratings to GRAC which will strengthen fairness and satisfy game users and people’s rights to know and contribute to promotion and development of game industry.
ARTICLE | doi:10.20944/preprints202103.0623.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: SARS-CoV-2; Big Data; Data Analytics; Predictive Models; Schools
Online: 25 March 2021 (14:35:53 CET)
Background: CoronaVirus Disease 2019 (COVID-19) is the main discussed topic world-wide in 2020 and at the beginning of the Italian epidemic, scientists tried to understand the virus diffusion and the epidemic curve of positive cases with controversial findings and numbers. Objectives: In this paper, a data analytics study on the diffusion of COVID-19 in Lombardy Region and Campania Region is developed in order to identify the driver that sparked the second wave in Italy Methods: Starting from all the available official data collected about the diffusion of COVID-19, we analyzed google mobility data, school data and infection data for two big regions in Italy: Lombardy Region and Campania Region, which adopted two different approaches in opening and closing schools. To reinforce our findings, we also extended the analysis to the Emilia Romagna Region. Results: The paper aims at showing how different policies adopted in school opening / closing may have on the impact on the COVID-19 spread. Conclusions: The paper shows that a clear correlation exists between the school contagion and the subsequent temporal overall contagion in a geographical area.
TECHNICAL NOTE | doi:10.20944/preprints202011.0038.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: dyadic data; co-occurrence data; expectation maximization (EM) algorithm; mixture model
Online: 2 November 2020 (12:06:26 CET)
Dyadic data which is also called co-occurrence data (COD) contains co-occurrences of objects. Searching for statistical models to represent dyadic data is necessary. Fortunately, finite mixture model is a solid statistical model to learn and make inference on dyadic data because mixture model is built smoothly and reliably by expectation maximization (EM) algorithm which is suitable to inherent spareness of dyadic data. This research summarizes mixture models for dyadic data. When each co-occurrence in dyadic data is associated with a value, there are many unaccomplished values because a lot of co-occurrences are inexistent. In this research, these unaccomplished values are estimated as mean (expectation) of random variable given partial probabilistic distributions inside dyadic mixture model.
ARTICLE | doi:10.20944/preprints201701.0080.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: wind turbine; failure detection; SCADA data; feature extraction; mutual information; copula
Online: 17 January 2017 (11:21:58 CET)
More and more works are using machine learning techniques while adopting supervisory control and data acquisition (SCADA) system for wind turbine anomaly or failure detection. While parameter selection is important for modelling a wind turbine’s health condition, only a few papers have been published focusing on this issue and in those papers interconnections among sub-components in a wind turbine are used to address this problem. However, merely the interconnections for decision making sometimes is too general to provide a parameter list considering the differences of each SCADA dataset. In this paper, a method is proposed to provide more detailed suggestions on parameter selection based on mutual information. Moreover, after proving that Copula, a multivariate probability distribution for which the marginal probability distribution of each variable is uniform is capable of simplifying the estimation of mutual information, an empirical copula based mutual information estimation method (ECMI) is introduced for an application. After that, a real SCADA dataset is adopted to test the method, and the results show the effectiveness of the ECMI in providing parameter selection suggestions when physical knowledge is not accurate enough.
ARTICLE | doi:10.20944/preprints201701.0079.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: accessibility; offshore; operation and maintenance; weather condition; Markov chain; data visualization
Online: 17 January 2017 (11:17:32 CET)
For offshore wind power generation, accessibility is one of the main factors that has great impact on operation and maintenance due to constraints on weather conditions for marine transportation. This paper presents a framework to explore the accessibility of an offshore site. At first, several maintenance types are defined and taken into account. Next, a data visualization procedure is introduced to provide an insight into the distribution of access periods over time. Then, a rigorous mathematical method based on finite state Markov chain is proposed to assess the accessibility of an offshore site from the maintenance perspective. A five-year weather data of a marine site is used to demonstrate the applicability and the outcomes of the proposed method. The main findings show that the proposed framework is effective in investigating the accessibility for different time scales and is able to catch the patterns of the distribution of the access periods. Moreover, based on the developed Markov chain, the average waiting time for a certain access period can be estimated. With more information on the maintenance of an offshore wind farm, the expected production loss due to time delay can be calculated.
REVIEW | doi:10.20944/preprints202103.0214.v2
Subject: Engineering, Automotive Engineering Keywords: data center; green data center; sustainability; energy efficiency; energy saving; ICT.
Online: 14 April 2021 (12:59:53 CEST)
Information and communication technologies (ICT) are increasingly permeating our daily life and we ever more commit our data to the cloud. Events like the COVID-19 pandemic put an exceptional burden upon ICT infrastructures. This involves increasing implementation and use of data centers, which increased energy use and environmental impact. The scope of this work is to take stock on data center impact, opportunities, and assessment. First, we estimate impact entity. Then, we review strategies for efficiency and energy conservation in data centers. Energy use pertain to power distribution, IT-equipment, and non-IT equipment (e.g. cooling): Existing and prospected strategies and initiatives in these sectors are identified. Among key elements are innovative cooling techniques, natural resources, automation, low-power electronics, and equipment with extended thermal limits. Research perspectives are identified and estimates of improvement opportunities are presented. Finally, we present an overview on existing metrics, regulatory framework, and bodies concerned.
ARTICLE | doi:10.20944/preprints202007.0078.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: personalization; decision making; medical data; artificial intelligence; Data-driving; Big Data; Data Mining; Machine Learning
Online: 5 July 2020 (15:04:17 CEST)
The study was conducted on applying machine learning and data mining methods to personalizing the treatment. This allows investigating individual patient characteristics. Personalization is built on the clustering method and associative rules. It was suggested to determine the average distance between instances for optimal performance metrics finding. The formalization of the medical data pre-processing stage for finding personalized solutions based on current standards and pharmaceutical protocols is proposed. The model of patient data is built. The paper presents the novel approach to clustering built on ensemble of cluster algorithm with better than k-means algorithm Hopkins metrics. The personalized treatment usually is based on decision tree. Such approach requires a lot of computation time and cannot be paralyzed. Therefore, it is proposed to classify persons by conditions, to determine deviations of parameters from the normative parameters of the group, as well as the average parameters. This made it possible to create a personalized approach to treatment for each patient based on long-term monitoring. According to the results of the analysis, it becomes possible to predict the optimal conditions for a particular patient and to find the medicaments treatment according to personal characteristics.
COMMUNICATION | doi:10.20944/preprints202206.0383.v2
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Exoskeleton; Twitter; Tweets; Big Data; social media; Data Mining; dataset; Data Science; Natural Language Processing; Information Retrieval
Online: 21 July 2022 (04:06:53 CEST)
The exoskeleton technology has been rapidly advancing in the recent past due to its multitude of applications and diverse use-cases in assisted living, military, healthcare, firefighting, and industry 4.0. The exoskeleton market is projected to increase by multiple times of its current value within the next two years. Therefore, it is crucial to study the degree and trends of user interest, views, opinions, perspectives, attitudes, acceptance, feedback, engagement, buying behavior, and satisfaction, towards exoskeletons, for which the availability of Big Data of conversations about exoskeletons is necessary. The Internet of Everything style of today's living, characterized by people spending more time on the internet than ever before, with a specific focus on social media platforms, holds the potential for the development of such a dataset, by the mining of relevant social media conversations. Twitter, one such social media platform, is highly popular amongst all age groups, where the topics found in the conversation paradigms include emerging technologies such as exoskeletons. To address this research challenge, this work makes two scientific contributions to this field. First, it presents an open-access dataset of about 140,000 tweets about exoskeletons that were posted in a 5-year period from May 21, 2017, to May 21, 2022. Second, based on a comprehensive review of the recent works in the fields of Big Data, Natural Language Processing, Information Retrieval, Data Mining, Pattern Recognition, and Artificial Intelligence that may be applied to relevant Twitter data for advancing research, innovation, and discovery in the field of exoskeleton research, a total of 100 Research Questions are presented for researchers to study, analyze, evaluate, ideate, and investigate based on this dataset.
ARTICLE | doi:10.20944/preprints202212.0390.v1
Subject: Earth Sciences, Environmental Sciences Keywords: hydraulic geometry; rating curves; flood mapping; accuracy; data acquisition; data needs
Online: 21 December 2022 (06:59:11 CET)
Hydraulic relationships are important for water resource management, hazard prediction, and modelling. Since Leopold first identified power law expressions that could relate streamflow to top-width, depth, and velocity, hydrologists have been estimating ‘At-a-station Hydraulic Geometries’ (AHG) to describe average flow hydraulics. As the amount of data, data sources, and application needs increase, the ability to apply, integrate and compare disparate and often noisy data is critical for applications ranging from reach to continental scales. However, even with quality data, the standard practice of solving each AHG relationship independently can lead to solutions that fail to conserve mass. The challenge addressed here is how to extend the physical properties of the AHG relations, while improving the way they are hydrologically addressed and fit. We present a framework for minimizing error while ensuring mass conservation at reach - or hydrologic Feature - scale geometries’(FHG) that complies with current state-of-the-practice conceptual and logical models. Through this framework, FHG relations are fit for the United States Geological Survey’s (USGS) Rating Curve database, the USGS HYDRoacoustic dataset in support of the Surface Water Oceanographic Topography satellite mission (HYDRoSWOT), and the hydraulic property tables produced as part of the NOAA/Oakridge Continental Flood Inundation Mapping framework. The paper describes and demonstrates the accuracy, interoperability, and application of these relationships to flood modelling and presents this framework in an R package.
ARTICLE | doi:10.20944/preprints202201.0365.v3
Subject: Life Sciences, Biochemistry Keywords: binding affinity prediction; machine learning; data quality; data quantity; deep learning
Online: 23 May 2022 (11:16:49 CEST)
Prediction of protein-ligand binding affinities is crucial for computational drug discovery. A number of deep learning approaches have been developed in recent years to improve the accuracy of such affinity prediction. While the predicting power of these systems have advanced to some degrees depending on the dataset used for model training and testing, the effects of the quality and quantity of the underlying data have not been thoroughly examined. In this study, we employed erroneous datasets and data subsets of different sizes, created from one of the largest databases of experimental binding affinities, to train and evaluate a deep learning system based on convolutional neural networks. Our results show that data quality and quantity do have significant impacts on the prediction performance of trained models. Depending on the variations in data quality and quantity, the performance discrepancies could be comparable to or even larger than those observed among different deep learning approaches. In particular, the presence of proteins during model training leads to a dramatic increase in prediction accuracy. This implies that continued accumulation of high-quality affinity data, especially for new protein targets, is indispensable for improving deep learning models to better predict protein-ligand binding affinities.
TECHNICAL NOTE | doi:10.20944/preprints202009.0357.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: data; data paper; omics; metadata; workflow; standards; FAIR principles, MIxS, MINSEQE
Online: 16 September 2020 (11:04:34 CEST)
Data papers have emerged as a powerful instrument for open data publishing, obtaining credit, and establishing priority for datasets generated in scientific experiments. Academic publishing improves data and metadata quality through peer-review and increases the impact of datasets by enhancing their visibility, accessibility, and re-usability. We aimed to establish a new type of article structure and template for omics studies: the omics data paper. To improve data interoperability and further incentivise researchers to publish high-quality data sets, we created a workflow for streamlined import of omics metadata directly into a data paper manuscript. An omics data paper template was designed by defining key article sections which encourage the description of omics datasets and methodologies. The workflow was based on REpresentational State Transfer services and Xpath to extract information from the European Nucleotide Archive, ArrayExpress and BioSamples databases, which follow community-agreed standards. The workflow for automatic import of standard-compliant metadata into an omics data paper manuscript facilitates the authoring process. It demonstrates the importance and potential of creating machine-readable and standard-compliant metadata. The omics data paper structure and workflow to import omics metadata improves the data publishing landscape by providing a novel mechanism for creating high-quality, enhanced metadata records, peer reviewing and publishing of these. It constitutes a powerful addition for distribution, visibility, reproducibility and re-usability of scientific data. We hope that streamlined metadata re-use for scholarly publishing encourages authors to improve the quality of their metadata to achieve a truly FAIR data world.
ARTICLE | doi:10.20944/preprints201811.0337.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Web API; SPARQL; micro-service; Data Integration; Linked Data; REST; Biodiversity
Online: 14 November 2018 (10:59:31 CET)
In recent years, Web APIs have become a de facto standard for exchanging machine-readable data on the Web. Despite this success though, they often fail in making resource descriptions interoperable due to the fact that they rely on proprietary vocabularies that lack formal semantics. The Linked Data principles similarly seek the massive publication of data on the Web, yet with the specific goal of ensuring semantic interoperability. Given their complementary goals, it is commonly admitted that cross-fertilization could stem from the automatic combination of Linked Data and Web APIs. Towards this goal, in this paper we leverage the micro-service architectural principles to define a SPARQL Micro-Service architecture, aimed at querying Web APIs using SPARQL. A SPARQL micro-service is a lightweight SPARQL endpoint that provides access to a small, resource-centric, virtual graph. In this context, we argue that full SPARQL Query expressiveness can be supported efficiently without jeopardizing servers availability. Furthermore, we demonstrate how this architecture can be used to dynamically assign dereferenceable URIs to Web API resources that do not have URIs beforehand, thus literally ``bringing'' Web APIs into the Web of Data. We believe that the emergence of an ecosystem of SPARQL micro-services published by independent providers would enable Linked Data-based applications to easily glean pieces of data from a wealth of distributed, scalable and reliable services. We describe a working prototype implementation and we finally illustrate the use of SPARQL micro-services in the context of two real-life use cases related to the biodiversity domain, developed in collaboration with the French National Museum of Natural History.
CASE REPORT | doi:10.20944/preprints201801.0066.v1
Online: 8 January 2018 (11:11:47 CET)
The implementation of the European Cohesion Policy aiming at fostering regions competitiveness, economic growth and creation of new jobs is documented over the period 2014–2020 in the publicly available Open Data Portal for the European Structural and Investment funds. On the base of this source, this paper aims at describing the process of data mining and visualization for information production on regional programmes performace in achieving effective expenditure of resouces.
ARTICLE | doi:10.20944/preprints202205.0344.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Linked (open) Data; Semantic Interoperability; Data Mapping; Governmental Data; SPARQL; Ontologies
Online: 25 May 2022 (08:18:46 CEST)
In this paper, we present a method to map information regarding service activity provision residing in governmental portals across European Commission. In order to perform this, we used as a basis the enriched Greek e-GIF ontology, modeling concepts, and relations in one of the two data portals (i.e., Points of Single Contacts) examined, since relevant information on the second was not provided. Mapping consisted in transforming information appearing in governmental portals in RDF format (i.e., as Linked data), in order to be easily exchangeable. Mapping proved a tedious task, since description on how information is modeled in the second Point of Single Contact is not provided and must be extracted in a manual manner.
ARTICLE | doi:10.20944/preprints202111.0073.v1
Subject: Medicine & Pharmacology, Other Keywords: data quality; OMOP CDM; EHDEN; healthcare data; real world data; RWD
Online: 3 November 2021 (09:12:54 CET)
Background: Observational health data has the potential to be a rich resource to inform clinical practice and regulatory decision making. However, the lack of standard data quality processes makes it difficult to know if these data are research ready. The EHDEN COVID-19 Rapid Col-laboration Call presented the opportunity to assess how the newly developed open-source tool Data Quality Dashboard (DQD) informs the quality of data in a federated network. Methods: 15 Data Partners (DPs) from 10 different countries worked with the EHDEN taskforce to map their data to the OMOP CDM. Throughout the process at least two DQD results were collected and compared for each DP. Results: All DPs showed an improvement in their data quality between the first and last run of the DQD. The DQD excelled at helping DPs identify and fix conformance is-sues but showed less of an impact on completeness and plausibility checks. Conclusions: This is the first study to apply the DQD on multiple, disparate databases across a network. While study-specific checks should still be run, we recommend that all data holders converting their data to the OMOP CDM use the DQD as it ensures conformance to the model specifications and that a database meets a baseline level of completeness and plausibility for use in research.
ARTICLE | doi:10.20944/preprints202110.0103.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Data Analytics; Analytics; Supply Chain Input; Supply Chain; Data Science; Data
Online: 6 October 2021 (10:38:42 CEST)
One of the most remarkable features in the 20th century was the digitalization of technical progress, which changed the output of companies worldwide and became a defining feature of the century. The growth of information technology systems and the implementation of new technical advances, which enhance the integrity, agility and long-term organizational performance of the supply chain, can distinguish a digital supply chain from other supply chains. For example, the Internet of Things (IoT)-enabled information exchange and Big Data analysis might be used to regulate the mismatch between supply and demand. In order to assess contemporary ideas and concepts in the field of data analysis in the context of supply chain management, this literary investigation has been decided. The research was conducted in the form of a comprehensive literature review. In the SLR investigation, a total of 71 papers from leading journals were used. SLR has found that data analytics integrate into supply chain management can have long-term benefits on supply chain management from the input side, i.e., improved strategic development, management and other areas.
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: geographic information fusion; data quality; data consistency checking; historic GIS; railway network; patrimonial data; crowdsourcing open data; volunteer geographic information VGI; wikipedia geo-spatial information extraction.
Online: 17 August 2020 (14:51:04 CEST)
Transportation of goods is as old as human civilizations : past networks and their evolution shed light on long term trends. Transportation impact on climate change is measured as major, as well as the impact on spreading a pandemic. These two reasons motivate the importance of providing relevant and reliable historical geographic datasets of these networks. This paper focuses on reconstructing the railway network in France at its maximal extent, a century ago. The active stations and lines are well documented by the French SNCF, in open public data. However, that information ignores past stations (ante 1980), which represent probably more than what is recorded in public data. Additional open data, individual or collaborative (eg. Wikipedia) are particularly valuable, but they are not always geo-coded, and two more sources are necessary to completing that geo-coding: ancient maps and aerial photography. Therefore, remote sensing and volunteer geographic information are the two pillars of past railway reconstruction. The methods developed are adapted to the extraction of information from these sources: automated parsing of Wikipedia Infoboxes, data extraction from simple tables, even from simple text. That series of sparse procedures can be merged into a comprehensive computer-assisted process. Beyond this, a huge effort in quality control is necessary when merging these data: automated wherever possible, or finally visually controlled by observation of remote sensing information. The main output is a reliable dataset, under ODbl, of more than 9100 stations, which can be combined with the information about the 35000 communes of France, for a large variety of studies. This work demonstrates two thesis: (a) it is possible to reconstruct transport network data from the past, and generic computer assisted methods can be developed; (b) the value of remote sensing and volunteered geo info is considerable (what archeologists already know).
ARTICLE | doi:10.20944/preprints202204.0068.v1
Subject: Mathematics & Computer Science, Computational Mathematics Keywords: Functional Data Analysis; Image Processing; Brain Imaging; Neuroimaging; Computational Neuroscience; Data Science
Online: 8 April 2022 (03:21:06 CEST)
Functional Data Analysis (FDA) is a relatively new field of statistics dealing with data expressed in the form of functions. FDA methodologies can be easily extended to the study of imaging data, an application proposed in Wang et al. (2020), where the authors settle the mathematical groundwork and properties of the proposed estimators. This methodology allows for the estimation of mean functions and simultaneous confidence corridors (SCC), also known as simultaneous confidence bands, for imaging data and for the difference between two groups of images. This is especially relevant for the field of medical imaging, as one of the most extended research setups consists on the comparison between two groups of images, a pathological set against a control set. FDA applied to medical imaging presents at least two advantages compared to previous methodologies: it avoids loss of information in complex data structures and avoids the multiple comparison problem arising from traditional pixel-to-pixel comparisons. Nonetheless, computing times for this technique have only been explored in reduced and simulated setups (Arias-López et al., 2021). In the present article, we apply this procedure to a practical case with data extracted from open neuroimaging databases and then measure computing times for the construction of Delaunay triangulations, and for the computation of mean function and SCC for one-group and two-group approaches. The results suggest that previous researcher has been too conservative in its parameter selection and that computing times for this methodology are reasonable, confirming that this method should be further studied and applied to the field of medical imaging.
Subject: Life Sciences, Other Keywords: data science; reuse; sequencing data; genomics; bioinformatics; databases; computational biology; open science
Online: 16 July 2020 (12:39:43 CEST)
The 'big data revolution' has enabled novel types of analyses in the life sciences, facilitated by public sharing and reuse of datasets. Here, we review the prodigious potential of reusing publicly available datasets and the challenges, limitations and risks associated with it. Possible solutions to issues and research integrity considerations are also discussed. Due to the prominence, abundance and wide distribution of sequencing data, we focus on the reuse of publicly available sequence datasets. We define ‘successful reuse’ as the use of previously published data to enable novel scientific findings and use selected examples of such reuse from different disciplines to illustrate the enormous potential of the practice, while acknowledging their respective limitations and risks. A checklist to determine the reuse value and potential of a particular dataset is also provided. The open discussion of data reuse and the establishment of the practice as a norm has the potential to benefit all stakeholders in the life sciences.
ARTICLE | doi:10.20944/preprints201702.0059.v1
Subject: Earth Sciences, Environmental Sciences Keywords: fine particulate matter (PM2.5); aerosol optical depth; community multi-scale air quality (CMAQ) model; data fusion; exposure assessment
Online: 16 February 2017 (08:58:09 CET)
Estimating ground surface PM2.5 with fine spatiotemporal resolution is a critical technique for exposure assessments in epidemiological studies of its health risks. Previous studies have utilized monitoring, satellite remote sensing or air quality modeling data to evaluate the spatiotemporal variations of PM2.5 concentrations, but such studies rarely combined these data simultaneously. We develop a three-stage model to fuse PM2.5 monitoring data, satellite-derived aerosol optical depth (AOD) and community multi-scale air quality (CMAQ) simulations together and apply it to estimate daily PM2.5 at a spatial resolution of 0.1˚ over China. Performance of the three-stage model is evaluated using a cross-validation (CV) method step by step. CV results show that the finally fused estimator of PM2.5 is in good agreement with the observational data (RMSE = 23.00 μg/m^3 and R2 = 0.72) and outperforms either AOD-retrieved PM2.5 (R2 = 0.62) or CMAQ simulations (R2 = 0.51). According to step-specific CVs, in data fusion, AOD-retrieved PM2.5 plays a key role to reduce mean bias, whereas CMAQ provides all-spacetime-covered predictions, which avoids sampling bias caused by non-random incompleteness in satellite-derived AOD. Our fused products are more capable than either CMAQ simulations or AOD-based estimates in characterizing the polluting procedure during haze episodes and thus can support both chronic and acute exposure assessments of ambient PM2.5. Based on the products, averaged concentration of annual exposure to PM2.5 was 55.75 μg/m3, while averaged count of polluted days (PM2.5 > 75 μg/m3) was 81, across China during 2014. Fused estimates will be publicly available for future health-related studies.
Subject: Materials Science, Biomaterials Keywords: Microscopy Image Segmentation; Deep Learning; Data Augmentation; Synthetic Training Data; Parametric Models
Online: 1 March 2021 (13:07:00 CET)
The analysis of microscopy images has always been an important yet time consuming process in in materials science. Convolutional Neural Networks (CNNs) have been very successfully used for a number of tasks, such as image segmentation. However, training a CNN requires a large amount of hand annotated data, which can be a problem for material science data. We present a procedure to generate synthetic data based on ad-hoc parametric data modelling for enhancing generalization of trained neural network models. Especially for situations where it is not possible to gather a lot of data, such an approach is beneficial and may enable to train a neural network reasonably. Furthermore, we show that targeted data generation by adaptively sampling the parameter space of the generative models gives superior results compared to generating random data points.
DATA DESCRIPTOR | doi:10.20944/preprints202208.0112.v1
Subject: Earth Sciences, Geoinformatics Keywords: ground truth data; drone; mobile application; windshield survey; sample design; crop mapping; agriculture statistics; data dissemination; earth observation data; spatial database.
Online: 4 August 2022 (16:18:26 CEST)
Over the last few years, Earth Observation (EO) data has shifted towards increased use to produce official statistics, particularly in the agriculture sector. National statistics offices worldwide, including in Asia and the Pacific, are expanding their use of EO data to produce agricultural statistics such as crop classification, yield estimation, irrigation mapping, and crop loss estimation. The advances in image classification, such as pixel-based and phenology-based classifications, and machine learning create new opportunities for researchers to analyze EO data applied to agriculture statistics. However, it requires the ground truth (GT) data because classification result mainly depends on the quality of GT. Therefore, in this study, we introduced a random sampling approach to design and collect GT data using EO imagery and ancillary data. As a result of data collection, GT data improve the algorithms and validates classification results. Nevertheless, despite the importance of GT data, they are rarely disseminated as a data product in themselves. Thus, this results in an untapped opportunity to share GT data as a global public good, and improved use of survey and census data as a source of GT data.
ARTICLE | doi:10.20944/preprints201808.0350.v2
Subject: Mathematics & Computer Science, Other Keywords: big data; clustering; data mining; educational data mining; e-learning; profile learning
Online: 19 October 2018 (05:58:05 CEST)
Educational data-mining is an evolving discipline that focuses on the improvement of self-learning and adaptive methods. It is used for finding hidden patterns or intrinsic structures of educational data. In the arena of education, the heterogeneous data is involved and continuously growing in the paradigm of big-data. To extract meaningful information adaptively from big educational data, some specific data mining techniques are needed. This paper presents a clustering approach to partition students into different groups or clusters based on their learning behavior. Furthermore, personalized e-learning system architecture is also presented which detects and responds teaching contents according to the students’ learning capabilities. The primary objective includes the discovery of optimal settings, in which learners can improve their learning capabilities. Moreover, the administration can find essential hidden patterns to bring the effective reforms in the existing system. The clustering methods K-Means, K-Medoids, Density-based Spatial Clustering of Applications with Noise, Agglomerative Hierarchical Cluster Tree and Clustering by Fast Search and Finding of Density Peaks via Heat Diffusion (CFSFDP-HD) are analyzed using educational data mining. It is observed that more robust results can be achieved by the replacement of existing methods with CFSFDP-HD. The data mining techniques are equally effective to analyze the big data to make education systems vigorous.
REVIEW | doi:10.20944/preprints201807.0059.v1
Subject: Life Sciences, Biophysics Keywords: data normalization; data scaling; zero-sum; metabolic fingerprinting; NMR; statistical data analysis
Online: 3 July 2018 (16:22:31 CEST)
The aim of this article is to summarize recent bioinformatic and statistical developments applicable to NMR-based metabolomics. Extracting relevant information from large multivariate datasets by statistical data analysis strategies may be of considerable complexity. Typical tasks comprise for example classification of specimens, identification of differentially produced metabolites, and estimation of fold changes. In this context it is of prime importance to minimize contributions from unwanted biases and experimental variance prior to these analyses. This is the goal of data normalization. Therefore, special emphasize is given to different data normalization strategies. In the first part, we will discuss the requirements and the pros and cons for a variety of commonly applied strategies. In the second part, we will concentrate on possible solutions in case that the requirements for the standard strategies are not fulfilled. In the last part, very recent developments will be discussed that allow reliable estimation of metabolic signatures for sample classification without prior data normalization. In this contribution special emphasis will be given to techniques that have worked well in our hands.
ARTICLE | doi:10.20944/preprints202106.0187.v3
Subject: Life Sciences, Biochemistry Keywords: SARS-CoV2; Biomathematics; Benford law; trials; Epidemiology; Fibonacci; data analysis; big data
Online: 11 June 2021 (15:47:44 CEST)
The Benford method can be used to detect manipulation of epidemiological or trial data during the validation of new drugs. We extend here the Benford method after having detected particular properties for the Fibonacci values 1, 2, 3, 5 and 8 of the first decimal of 10 runs of official epidemiological data published in France and Italy (positive cases, intensive care, and deaths) for the periods of March 1 to May 30, 2020 and 2021, each with 91 raw data. This new method – called “BFP” for Benford-Fibonacci-Perez - is positive in all 10 cases (i.e. 910 values) with an average of favorable cases close to 80%, which, in our opinion, would validate the reliability of these basic data.
ARTICLE | doi:10.20944/preprints202205.0238.v2
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: COVID-19; SARS-CoV-2; Omicron; Twitter; tweets; sentiment analysis; big data; Natural Language Processing; Data Science; Data Analysis
Online: 7 July 2022 (08:36:40 CEST)
This paper presents the findings of an exploratory study on the continuously generating Big Data on Twitter related to the sharing of information, news, views, opinions, ideas, knowledge, feedback, and experiences about the COVID-19 pandemic, with a specific focus on the Omicron variant, which is the globally dominant variant of SARS-CoV-2 at this time. A total of 12028 tweets about the Omicron variant were studied, and the specific characteristics of tweets that were analyzed include - sentiment, language, source, type, and embedded URLs. The findings of this study are manifold. First, from sentiment analysis, it was observed that 50.5% of tweets had the ‘neutral’ emotion. The other emotions - ‘bad’, ‘good’, ‘terrible’, and ‘great’ were found in 15.6%, 14.0%, 12.5%, and 7.5% of the tweets, respectively. Second, the findings of language interpretation showed that 65.9% of the tweets were posted in English. It was followed by Spanish or Castillian, French, Italian, Japanese, and other languages, which were found in 10.5%, 5.1%, 3.3%, 2.5%, and <2% of the tweets, respectively. Third, the findings from source tracking showed that “Twitter for Android” was associated with 35.2% of tweets. It was followed by “Twitter Web App”, “Twitter for iPhone”, “Twitter for iPad”, “TweetDeck”, and all other sources that accounted for 29.2%, 25.8%, 3.8%, 1.6%, and <1% of the tweets, respectively. Fourth, studying the type of tweets revealed that retweets accounted for 60.8% of the tweets, it was followed by original tweets and replies that accounted for 19.8% and 19.4% of the tweets, respectively. Fifth, in terms of embedded URL analysis, the most common domains embedded in the tweets were found to be twitter.com, which was followed by biorxiv.org, nature.com, wapo.st, nzherald.co.nz, recvprofits.com, science.org, and other URLs. Finally, to support similar research and development in this field centered around the analysis of tweets, we have developed an open-access Twitter dataset that comprises tweets about the SARS-CoV-2 omicron variant since the first detected case of this variant on November 24, 2021.
ARTICLE | doi:10.20944/preprints202011.0707.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: cluster flows; mesoscopic observations; data compression
Online: 30 November 2020 (08:09:24 CET)
Multiagent technologies give a new way to study and control complex systems. Local interactions between agents often lead to group synchronization also known as clusterization, which usually is a more rapid process in comparison with relatively slow changes in external environment. Usually, the goal of system control is defined by the behaviour of a system on long time intervals. When these time intervals are much longer than the time of cluster formation, clusters may be considered as new variables in a ``slow'' time model. We call such variables ``mesoscopic'' to emphasize their scale laying between the level of the whole system (macroscopic scale) and the level of individual agents (microscopic scale). Thus, it allows us to reduce significantly the dimensionality of a system by omitting considerations of each separated agent, so that we may hope to reduce the required amount of control inputs. Thus, we are often able to consider a system as a collection of ``flowing'' (morphing) clusters emerged form behaviour of a huge amount of individual agents. In this paper, we contrast such approach to the one where a system is considered as a network of elementary agents. We develop a mathematical framework for analysis of cluster flows in multiagent networks and use it to analyze the Kuramoto model as an attracting example of a complex networked system. In this model, a clusterization leads to sparse representation of dynamic trajectories in the whole quantized state space. With that in mind, compressive sensing allows to restore the trajectories in a high-dimensional discrete state space based on significantly lower amount of randomized integral mesoscopic observations. We propose a corresponding algorithm of quantized dynamic trajectory compression. It could allow us to efficiently transmit the state space data to a data center for further control synthesis. The theoretical results are illustrated for a simulated multiagent network with multiple clusters.