Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

A New Computational Algorithm for Assessing Overdispersion in Machine Learning Count Models with Python

Version 1 : Received: 7 March 2024 / Approved: 7 March 2024 / Online: 8 March 2024 (09:30:50 CET)

A peer-reviewed article of this Preprint also exists.

Fávero, L.P.L.; Duarte, A.; Santos, H.P. A New Computational Algorithm for Assessing Overdispersion and Zero-Inflation in Machine Learning Count Models with Python. Computers 2024, 13, 88. Fávero, L.P.L.; Duarte, A.; Santos, H.P. A New Computational Algorithm for Assessing Overdispersion and Zero-Inflation in Machine Learning Count Models with Python. Computers 2024, 13, 88.

Abstract

Count data analysis presents unique challenges due to its discrete nature, often exhibiting excessive zeros and overdispersion. To address these complexities, count models, such as Poisson regression and Negative Binomial regression, have been developed, enabling modeling and prediction of count-based phenomena. Additionally, it’s important to notice that zero inflation, a phenomenon commonly observed in count data, requires specialized techniques for robust analysis. This article provides an overview of count data and count models, explores zero inflation, introduces Likelihood Ratio Tests, and explains how the Vuong Test can be used as a model selection criteria. Furthermore, we created a Vuong Test implementation from scratch using the Python programming language. This implementation enhances the accessibility and applicability of the Vuong Test in real-world scenarios, providing a valuable contribution to the academic community, since Python didn’t have an implementation of this statistical test.

Keywords

Count data; Machine learning; Negative binomial regression; Overdispersion; Poisson regression; Python; Vuong Test; Zero inflation

Subject

Computer Science and Mathematics, Data Structures, Algorithms and Complexity

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.