Preprint Article Version 2 Preserved in Portico This version is not peer-reviewed

Neural Networks and Adapted Optimal Transport

Version 1 : Received: 2 May 2023 / Approved: 4 May 2023 / Online: 4 May 2023 (10:04:19 CEST)
Version 2 : Received: 6 May 2023 / Approved: 9 May 2023 / Online: 9 May 2023 (04:33:48 CEST)
Version 3 : Received: 20 June 2023 / Approved: 20 June 2023 / Online: 20 June 2023 (11:12:07 CEST)

A peer-reviewed article of this Preprint also exists.

Di Persio, L.; Garbelli, M. From Optimal Control to Mean Field Optimal Transport via Stochastic Neural Networks. Symmetry 2023, 15, 1724. Di Persio, L.; Garbelli, M. From Optimal Control to Mean Field Optimal Transport via Stochastic Neural Networks. Symmetry 2023, 15, 1724.

Abstract

Within Data Science scenario both Machine Learning (ML) and Neural Networks (NNs) are widely used for a plethora of applications, spanning from engineering to biology, from finance to medicine. Meanwhile, the pure analytical analysis of related models used often lacks of detailed description. Trying to close this gap, during recent years the theory of Optimal Transport start becoming more a more popular within the Data Science arena, since it allows for efficient and scalable solutions about what can be broadly referred as Artificial Intelligence tasks, particularly by using approximate solvers through linear programming. More precisely, let us recall that both in ML and statistics, minimizing an objective function over the space of probability measures is fundamental problem. Along this line, the training of a Neural Network through Stochastic Gradient Descent has been shown to be equivalent to a gradient flow in Wasserstein space. Moreover, in many applications, it is crucial to consider the causal structure of the data generating process. In view of providing a possibly unifying as well as efficient approach to latter questions, we consider the Adapted Optimal Transport (AOT) method which allows to incorporate causality constraints into the optimization of the Wasserstein distance. AOT is particularly useful when dealing with laws of stochastic processes, being exploitable to encode an adapted (non-anticipative) constraint into the allocation of mass of the classical OT problem. Accordingly, we provide an in depth exploration of the connections between supervised learning and AOT, providing theoretical insights into the benefits of including causality constraints for developing more robust, scalable and accurate algorithms, while maintainig their computational efforts at minimum possible levels.

Keywords

Neural Network; Machine Learning; Adapted Optimal Transport; Mean Field function

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Comments (1)

Comment 1
Received: 9 May 2023
Commenter: Matteo Garbelli
Commenter's Conflict of Interests: Author
Comment: Fixed typos.
+ Respond to this comment

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 1
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.