OREVID: Open, Evidence-Based Drug Repurposing with a Benchmark Against TxGNN

Background. Knowledge-graph models can rank existing drugs for a disease, but their benchmarks mostly reward rediscovering known treatments, and their scores rarely come with evidence a researcher can check. We built OREVID, an open tool that shows a ranked candidate only when an evidence line supports it and reports how often each line was right on held-out data, and compared it with TxGNN on identical data. Methods. The graph combines PrimeKG, Open Targets, STRING and PubTator3 (157,822 nodes, 9.46 million edges). For each disease, 22 degree-weighted metapath features and two gene-level columns (whether the drug moves its targets in the direction human genetics asks, from protein and expression QTLs that colocalise with the disease's genome-wide association studies, and whether it acts on a gene Orphanet lists for the disease) feed a LightGBM LambdaRank model trained with masked neighbourhoods. A candidate is shown only if it is indicated for an ontology-neighbour disease or for a disease sharing at least 3 associated genes (hypergeometric p < 10⁻⁵ over the genes with at least one associated disease). The acceptance criteria for these rules were set in advance; the rules were revised on the hidden-treatment benchmarks and then measured on an external set of approvals not used to choose them. TxGNN was retrained with its official code on the same PrimeKG splits, and all tasks and predictions are released as OREVID-bench. Results. Zero-shot Recall@100 (mean ± SD over five splits) was 0.812 ± 0.038 for OREVID, 0.412 ± 0.042 for TxGNN and 0.358 ± 0.040 for a popularity baseline, with OREVID ahead on every split (paired difference with TxGNN 0.400, 95% CI 0.354–0.444, resampling diseases). On nine disease areas held out it was 0.627 against 0.237, ahead in all nine, although TxGNN had the higher AUPRC in four. Training OREVID without validation labels and excluding diseases where TxGNN outputs constant scores gave 0.815 against 0.421. On 3,718 approvals absent from PrimeKG, Recall@100 was 0.355 against 0.121 (0.366 ± 0.011 against 0.126 ± 0.004 over five training splits), and 0.175 against 0.085 for the 2,719 approvals not obvious from the disease ontology; a logistic regression without the drug-popularity feature and a model without any Open Targets feature (0.348) performed similarly. Of the candidates the website's rules select in this external exam, 12.6% (95% CI 9.8–15.5%, resampling diseases) of direct extensions and 4.4% (3.2–5.8%) of shared-genes candidates were among the unseen approvals, against about 0.07% by chance; random gene sets of the same size selected candidates that were right as often, so the shared genes are not evidence of a shared mechanism, although they contained a protein of the curated mechanism (DrugMechDB) more often than random genes of the same disease (25% against 17%). A drug acting on a disease gene, with no other evidence, was right 1.8–3.4% of the time, no better than candidates with no evidence line at all. With the graph and the model rebuilt as known in 2015 and every drug dated by its first approval, the uses that later reached phase 3 or approval had a Recall@100 of 0.283 against 0.181 for popularity (paired difference 0.102, 95% CI 0.078–0.126) and 0.117 for TxGNN trained on the same graph (difference 0.167, 0.142–0.192), in pre-registered tests. On labels absent from the graph (CIViC oncology evidence, indications only DrugCentral or MED-RT record, and European orphan designations) OREVID was ahead of popularity by 0.32, 0.27 and 0.16, but the candidates the website shows did not meet trial end points more often than ranked drugs it hides. Conclusions. Reporting the hit rate of each evidence line makes repurposing predictions checkable and shows how low their precision is on approved uses missing from the training graph. Most of the signal comes from the ranking and from what already works in related diseases, not from the specific genes they share. OREVID is a hypothesis generator, not a clinical tool; code, data and benchmark are public (https://orevid.org, doi:10.5281/zenodo.23323226).