Version 1
: Received: 15 April 2024 / Approved: 15 April 2024 / Online: 16 April 2024 (16:26:06 CEST)
How to cite:
Pingos, M.; Andreou, A.S. Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints. Preprints2024, 2024041018. https://doi.org/10.20944/preprints202404.1018.v1
Pingos, M.; Andreou, A.S. Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints. Preprints 2024, 2024041018. https://doi.org/10.20944/preprints202404.1018.v1
Pingos, M.; Andreou, A.S. Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints. Preprints2024, 2024041018. https://doi.org/10.20944/preprints202404.1018.v1
APA Style
Pingos, M., & Andreou, A.S. (2024). Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints. Preprints. https://doi.org/10.20944/preprints202404.1018.v1
Chicago/Turabian Style
Pingos, M. and Andreas S. Andreou. 2024 "Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints" Preprints. https://doi.org/10.20944/preprints202404.1018.v1
Abstract
Nowadays, one of the greatest challenges in Data Meshes revolves around detecting and creating Data Domains and Data Products for providing the ability to adapt easily and quickly to changing business needs. This requires a disciplined approach to identify, differentiate and prioritize distinct data sources according to their content and diversity. The current paper tackles this highly com-plicated issue and suggests a standardized approach that integrates the concept of Data Blueprints with Data Meshes. In essence, a novel standardization framework is proposed that creates Data Products using a metadata semantic enrichment mechanism, the latter also offering Data Domain readiness and alignment. The approach is demonstrated using real-world data produced by mul-tiple sources in a poultry meat production factory. A set of functional attributes is used to compare qualitatively the proposed approach against existing data structures utilized in storage architectures with quite promising results. Finally, experimentation with different scenarios varying in data product complexity and granularity suggests successful performance.
Keywords
Big Data; Data Lakes; Data Meshes; Data Products; Data Blueprints; Metadata Semantic Enrichment
Subject
Computer Science and Mathematics, Computer Science
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
