Version 1
: Received: 31 October 2020 / Approved: 2 November 2020 / Online: 2 November 2020 (10:00:29 CET)
How to cite:
Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints2020, 2020110010. https://doi.org/10.20944/preprints202011.0010.v1.
Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints 2020, 2020110010. https://doi.org/10.20944/preprints202011.0010.v1.
Cite as:
Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints2020, 2020110010. https://doi.org/10.20944/preprints202011.0010.v1.
Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints 2020, 2020110010. https://doi.org/10.20944/preprints202011.0010.v1.
Abstract
In the field of machine learning, cluster analysis has always been a very important technology for determining useful or implicit characteristics in the data. However, the current mainstream cluster analysis algorithms require comprehensive analysis of the overall data to obtain the best parameters in the algorithm. As a result, handling large-scale datasets would be difficult. This research proposes a distributed related clustering mechanism for Unsupervised Learning, which assumes that if adjacent data are similar, a group can be formed by relating to more data points. Therefore, when processing data, large-scale datasets can be distributed to multiple computers, and the correlation of any two datasets in each computer can be calculated simultaneously. Later, results are processed through aggregation and filtering before assembled into groups. This method would greatly reduce the pre-processing and execution time of the dataset; in practical application, it only needs to focus on how the relevance of the data is designed. In addition, the experimental results show the accuracy, applicability, and ease of use of this method.
Keywords
Big Data; Clustering; Distributed system; Machine learning
Subject
MATHEMATICS & COMPUTER SCIENCE, Other
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
