Version 1
: Received: 17 September 2022 / Approved: 19 September 2022 / Online: 19 September 2022 (10:31:53 CEST)
How to cite:
Muscoloni, A.; Cannistraci, C.V. Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC. Preprints2022, 2022090277. https://doi.org/10.20944/preprints202209.0277.v1
Muscoloni, A.; Cannistraci, C.V. Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC. Preprints 2022, 2022090277. https://doi.org/10.20944/preprints202209.0277.v1
Muscoloni, A.; Cannistraci, C.V. Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC. Preprints2022, 2022090277. https://doi.org/10.20944/preprints202209.0277.v1
APA Style
Muscoloni, A., & Cannistraci, C.V. (2022). Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC. Preprints. https://doi.org/10.20944/preprints202209.0277.v1
Chicago/Turabian Style
Muscoloni, A. and Carlo Vittorio Cannistraci. 2022 "Early Retrieval Problem and Link Prediction Evaluation via the Area Under the Magnified ROC" Preprints. https://doi.org/10.20944/preprints202209.0277.v1
Abstract
Link prediction is an unbalanced early retrieval problem, whose goal is to prioritize a small cohort of positive links on top of a list largely populated by unlabelled links. Differently from binary classification, here the evaluation focuses on how the predictor prioritizes the positive class because, in practice, a negative class does not exist. Previous studies explained that AUC-ROC is not apt for unbalanced class problems and is misleading for early retrieval problems, therefore standard AUC-ROC is not appropriate for evaluation of link prediction. However, some scholars argue that an AUC-ROC like evaluation accounting for the relative positioning of the few positive links among the vastness of unlabelled links remains a valid concept to pursue. Here we propose the area under the magnified ROC (AUC-mROC), a new measure that adjusts the standard AUC-ROC to work also for unbalanced early retrieval problems such as link prediction.
Keywords
link prediction; AUC-ROC; Early retrieval evaluation
Subject
Computer Science and Mathematics, Probability and Statistics
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
