ARTICLE | doi:10.20944/preprints202004.0066.v1
Subject: Biology And Life Sciences, Biology And Biotechnology Keywords: mitigation strategy; COVID-19; epidemics; health policy; public and global health; demographics; social distancing
Online: 6 April 2020 (15:20:55 CEST)
COVID-19 severity is heterogeneously distributed over age strata, but current mitigation strategies are homogeneously applied to all population. Social-distancing and stay-home are effective conservative approaches but lack economic sustainability on long term. Conversely, herd-immunity is a nonrestrictive strategy which can cost remarkable number of human lives and can melt the healthcare system down. Here I propose an Age Adaptive Social Distancing (AASD) engineering strategy to mitigate COVID-19 outbreak. AASD is based on the scientific evidence that the fatality rate grows nonlinearly with age, hence also the containing strategy should adapt nonlinearly. Essentially, AASD suggests that ‘silent spreaders’ (age 0-39) should avoid/minimize direct and indirect contacts with individuals in ‘dangerous zone’ (age 40+). The rationale is: 0-19 should follow parents strategy, healthy 20-39 (low fatality rate) might conduct screened life under active surveillance, to sustain economy and acquire rational immunity; 40-59 should respect social distancing (waiting a therapy); 60+ should stay at home (waiting a vaccine). This might save human lives, reduce healthcare demand and improve economic sustainability. The final take-home message is that future studies should design precision and personalized strategies for specific contagious diseases that integrate different social constrains, active surveillance and contact tracing.
ARTICLE | doi:10.20944/preprints202209.0277.v1
Subject: Computer Science And Mathematics, Probability And Statistics Keywords: link prediction; AUC-ROC; Early retrieval evaluation
Online: 19 September 2022 (10:31:53 CEST)
Link prediction is an unbalanced early retrieval problem, whose goal is to prioritize a small cohort of positive links on top of a list largely populated by unlabelled links. Differently from binary classification, here the evaluation focuses on how the predictor prioritizes the positive class because, in practice, a negative class does not exist. Previous studies explained that AUC-ROC is not apt for unbalanced class problems and is misleading for early retrieval problems, therefore standard AUC-ROC is not appropriate for evaluation of link prediction. However, some scholars argue that an AUC-ROC like evaluation accounting for the relative positioning of the few positive links among the vastness of unlabelled links remains a valid concept to pursue. Here we propose the area under the magnified ROC (AUC-mROC), a new measure that adjusts the standard AUC-ROC to work also for unbalanced early retrieval problems such as link prediction.
SHORT NOTE | doi:10.20944/preprints202105.0689.v1
Subject: Physical Sciences, Mathematical Physics Keywords: complex networks; network models; link prediction; automata theory; network automata; Cannistraci-Hebb; stacking modelling
Online: 28 May 2021 (10:13:43 CEST)
Link prediction is an iconic problem in complex networks because deals with the ability to predict nonobserved existing or future parts of the network structure. The impact of this prediction on real applications can be disruptive: from prediction of covert links between terrorists in their social networks to repositioning of drugs in molecular diseasome networks. Here we compare: (1) an ensemble meta-learning method (Ghasemian et al.), which uses an artificial intelligence (AI) stacking strategy to create a single meta-model from hundreds of other models; (2) a structural predictability method (SPM, Lü et al.), which relies on a theory derived from quantum mechanics and does not assume any model; (3) a modelling rule named Cannistraci-Hebb (CH, Muscoloni et al.), which relies on one brain-bioinspired model adapting to the intrinsic network structure.We conclude that brute-force stacking of algorithms by AI does not perform better than (and is often significantly outperformed by) SPM and one simple brain-bioinspired rule such as CH. This agrees with the Gödel incompleteness: stacking is optimal but incomplete, you cannot squeeze out more than what is already in your features. Hence, we should also pursue AI that resembles human-like physical ‘understanding’ of simple generalized rules associated to complexity. The future might be populated by AI that ‘steals for us the fire from Gods’, towards machine intelligence that creates new rules rather than stacking the ones already known.
HYPOTHESIS | doi:10.20944/preprints202005.0005.v1
Subject: Biology And Life Sciences, Virology Keywords: social intervention; COVID-19; health policy; public health; age; gender
Online: 2 May 2020 (12:04:58 CEST)
Many governments particularly in Europe are designing social interventions for the first post COVID-19 emergency phase. Definition of a ‘best practice’ for restriction release is urgent. Although data uncertainty generate difficulties, we believe near term analysis must shift from attempting to understand the numerous ‘unknowns’ to the clarification and interpretation of the few ‘knowns’, to create stepping stones towards rapid evidence-based decision making.Here, open access data on COVID-19 severity in three European countries were analyzed. Spain’s data were more comprehensive than those from Italy and Germany. Overall, COVID-19 severity shows a remarkable nonlinear growth with age that is significantly higher in adult males. Hence, age-adaptive and gender-balanced social interventions might represent efficient repopulation options for public health policymakers. Furthermore, we urge wider governmental effort for open access to relevant data. Their analysis will allow consolidation of existing trends, validation of key observations and thus facilitation of timely decisions.
ARTICLE | doi:10.20944/preprints202207.0139.v3
Subject: Computer Science And Mathematics, Artificial Intelligence And Machine Learning Keywords: sparse training; neural networks; link prediction; network automata; Cannistraci-Hebb; epitopological learning
Online: 30 October 2023 (05:54:50 CET)
Sparse training (ST) aims to ameliorate deep learning by replacing fully connected artificial neural networks (ANNs) with sparse or ultra-sparse ones, such as brain networks are, therefore it might benefit to borrow brain-inspired learning paradigms from complex network intelligence theory. Here, we launch the ultrasparse advantage challenge, whose goal is to offer evidence on the extent to which ultra-sparse (around 1% connection retained) topologies can achieve any leaning advantage on fully connected. Epitopological learning is a field of network science and complex network intelligence that studies how to implement learning on complex networks by changing the shape of their connectivity structure (epitopological plasticity). One way to implement Epitopological (epi- means new) Learning is via link prediction: predicting the likelihood of nonobserved links to appear in the network. Cannistraci-Hebb learning theory inspired the CH3-L3 network automata rule for link prediction which is effective for general-purpose link prediction. Here, starting from CH3-L3 we propose Epitopological Sparse Meta-deep Learning (ESML) to apply Epitopological Learning to sparse training. In empirical experiments, we find that ESML learns ANNs with ultra-sparse hyperbolic (epi-)topology in which emerges a community layer organization that is meta-deep (meaning that each layer also has an internal depth due to powerlaw node hierarchy). Furthermore, we discover that ESML can in many cases automatically sparse the neurons during training (arriving even to 30% neurons left in hidden layers), this process of node dynamic removal is called percolation. Starting from this network science evidence, we design Cannistraci-Hebb training (CHT), a 4-step training methodology that put ESML at its heart. We conduct experiments on 6 datasets and 3 network structures (MLPs, VGG16, ResNet50) comparing CHT to dynamic sparse training SOTA algorithms and fully connected network. The results indicate that, with a mere 1% of links retained during training, CHT surpasses fully connected networks on VGG16 and ResNet50. This key finding is an evidence for ultra-sparse advantage and signs a milestone in deep learning. CHT acts akin to a gradient-free oracle which adopts CH3-L3 based epitopological learning to guide the placement of new links in the ultra-sparse network topology to facilitate sparse-weight gradient learning, and this in turn reduces the convergence time of ultra-sparse training. Finally, CHT offers first examples of parsimony dynamic sparse training because, in many datasets, it canretain network performance by percolating and significantly reducing the node network size.
ARTICLE | doi:10.20944/preprints202012.0808.v3
Subject: Physical Sciences, Theoretical Physics Keywords: complex networks; network models; link prediction; automata theory; network automata; Cannistraci-Hebb theory
Online: 11 May 2022 (14:51:01 CEST)
Many complex networks have a connectivity that might be only partially detected or that tends to grow over time, hence the prediction of non-observed links is a fundamental problem in network science. The aim of topological link prediction is to forecast these non-observed links by only exploiting features intrinsic to the network topology. It has a wide range of real applications, like suggesting friendships in social networks or predicting interactions in biological networks.The Cannistraci-Hebb theory is a recent achievement in network science that includes a theoretical framework to understand local-based link prediction on paths of length n. In this study we introduce two innovations: theory of modelling (science) and theory of realization (engineering). For the theory of modelling we first recall a definition of network automata as a general framework for modelling the growth of connectivity in complex networks. We then show that several deterministic models previously developed fall within this framework and we introduce novel network automata following the Cannistraci-Hebb rule. For the theory of realization, we present how to build adaptive network automata for link prediction, which incorporate multiple deterministic models of self-organization and automatically choose the rule that better explains the patterns of connectivity in the network under investigation. We compare Cannistraci-Hebb adaptive (CHA) network automaton against state-of-the-art link prediction methods such as structural perturbation method (SPM), stochastic block models (SBM) and artificial intelligence algorithms for graph embedding. CHA displays an overall higher link prediction performance across different evaluation frameworks on 1386 networks. Finally, we highlight that CHA offers the key advantage to explicitly explain the mechanistic rule of self-organization which leads to the link prediction performance, whereas SPM and graph embedding not. In comparison to CHA, SBM unfortunately shows irrelevant and unsatisfactory performance demonstrating that SBM modelling is not adequate for link prediction in real networks.
ARTICLE | doi:10.20944/preprints202103.0196.v1
Subject: Biology And Life Sciences, Anatomy And Physiology Keywords: Single cell RNA-seq; spatial reconstruction; development; coalescent embedding
Online: 5 March 2021 (21:21:59 CET)
Single cell RNA-seq (scRNA-seq) profiles conceal temporal and spatial tissue developmental information. De novo reconstruction of single cell temporal trajectory has been fairly addressed, but reverse engineering single cell 3D spatial tissue localization is hitherto landmark based, and de novo spatial reconstruction is a compelling computational open problem. Here we show that a new algorithm - named D-CE - for coalescent embedding of single cell transcriptomic networks can address this open problem. We rely merely on the spatial information encoded in the expression patterns of developmental signal transcription factor (DST) genes, and we find that D-CE of cell-cell association DST-transcriptomic networks reliably reconstructs the Geo-seq or single cell samples’ 3D spatial tissue distribution. Comparison to the novoSpaRC and CSOmap (recent and only available de novo 3D spatial reconstruction methods) on 16 datasets and 681 reconstructions, reveals a significantly distinctive superior performance of D-CE.
ARTICLE | doi:10.20944/preprints202306.0438.v1
Subject: Computer Science And Mathematics, Artificial Intelligence And Machine Learning Keywords: neuromorphic engineering; neuromorphic computing; dendritic computation; silent synapse; motion perception
Online: 6 June 2023 (10:04:05 CEST)
Most neuromorphic technologies use a point-neuron model, missing the spatiotemporal nature of neuronal computation performed in dendrites. Dendritic morphology and synaptic organization are structurally tailored for spatiotemporal information processing, enabling various computations like visual perception. Here, we report on a neuromorphic computational model termed ‘dendristor’, which integrates functional synaptic organization with dendritic tree-like morphology computation. The dendristor presents bioplausible nonlinear integration of excitatory and inhibitory synaptic inputs with silent synapses and diverse spatial distribution dependency. We show that the dendristor can emulate direction selectivity, which is the feature to react robustly to a preferred signal direction on the dendrite. We discover that silent synapses can remarkably enhance direction selectivity, turning out to be a crucial player in dendritic computation processing. Finally, we develop neuromorphic dendritic neural circuits that can emulate a cognitive function such as motion perception in the retina. Using dendritic morphology, we achieve visual perception of motion in 3D space by various mapping of spatial information on different dendritic branches. This neuromorphic dendritic computation innovates beyond current neuromorphic computation and provides solutions to explore new skylines in artificial intelligence, neurocomputation, and brain-inspired computing.
ARTICLE | doi:10.20944/preprints202310.1487.v1
Subject: Computer Science And Mathematics, Artificial Intelligence And Machine Learning Keywords: post-training pruning; combinatorial optimization; large language models; inference acceleration
Online: 24 October 2023 (07:51:13 CEST)
With the rapid growth of large language models (LLMs), there is increasing demand for memory and computation for LLMs. Recent efforts on post-training pruning of LLMs aim to reduce the model size and computation, yet the performance is still sub-optimal. In this paper, we present a plug-and-play solution for post-training pruning of LLMs. The proposed solution has two innovative components: 1) **Relative Importance and Activations** (RIA), a new pruning metric that jointly considers the weight and activations efficiently on LLMs; and 2) **Channel Permutation**, a new approach to maximally preserve important weights under N:M sparsity. The proposed two components can be readily combined to further enhance the N:M structuredly pruned LLMs. Our empirical experiments show that RIA alone can already surpass all existing post-training pruning methods on prevalent LLMs, e.g., LLaMA ranging from 7B to 65B. Furthermore, N:M structured pruning with channel permutation can even outperform the original LLaMA2 70B on zero-shot tasks, together with practical speed-up on specific hardware.
ARTICLE | doi:10.20944/preprints202012.0500.v1
Subject: Physical Sciences, Acoustics Keywords: complex multilayer networks; hypergraphs; complex systems
Online: 21 December 2020 (10:44:19 CET)
Many real complex systems present multilayer structure where high-order metadata on one layer refers to dyadic data on a lower layer. Significant progresses to analyse high-order metadata under the assumption of community organization have been done. However, there are no planted communities in real-world networks, and the necessity of new frameworks to analyze high-order metadata regardless of community organization has been raised.Here, we propose to adopt hyperedge organization. Predicting ‘entanglements’ between a hyperedge and nodes scattered in the rest of the network might suggest structural or functional liaisons, without assumption of any community organization. We introduce a novel concept: hyperedge entanglement (HE), which associates to each hyperedge an entangled hyperedge, by means of a network operator that predicts significant ‘interactions at distance’ between network nodes and existing hyperedges. We also introduce a new challenge termed hyperedge entanglement prediction (HEP), and an algorithm to perform this task. We evaluated HEP performance on social, biological and synthetic data where, given only topology and hyperedges (such as communities or functional modules), the goal is to predict whether nodes not connected to a certain hyperedge might be candidates for a significant entanglement. Finally, as real application in diseasome systems biomedicine, we perform HEP on the human protein interactome to predict unknown gene entanglements with the COPD disease gene hyperedge. HEP predictions are validated by biological experiments, enlarging our understanding of molecular mechanisms behind COPD/aneurysm comorbidity.