Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets

Version 1 : Received: 31 October 2020 / Approved: 2 November 2020 / Online: 2 November 2020 (10:00:29 CET)

How to cite: Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints 2020, 2020110010 (doi: 10.20944/preprints202011.0010.v1). Lee, K.; Tsai, M.; Huang, C. Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets. Preprints 2020, 2020110010 (doi: 10.20944/preprints202011.0010.v1).

Abstract

In the field of machine learning, cluster analysis has always been a very important technology for determining useful or implicit characteristics in the data. However, the current mainstream cluster analysis algorithms require comprehensive analysis of the overall data to obtain the best parameters in the algorithm. As a result, handling large-scale datasets would be difficult. This research proposes a distributed related clustering mechanism for Unsupervised Learning, which assumes that if adjacent data are similar, a group can be formed by relating to more data points. Therefore, when processing data, large-scale datasets can be distributed to multiple computers, and the correlation of any two datasets in each computer can be calculated simultaneously. Later, results are processed through aggregation and filtering before assembled into groups. This method would greatly reduce the pre-processing and execution time of the dataset; in practical application, it only needs to focus on how the relevance of the data is designed. In addition, the experimental results show the accuracy, applicability, and ease of use of this method.

Subject Areas

Big Data; Clustering; Distributed system; Machine learning

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.