4Cast Lab: Open-Source App for Machine Learning & Statistical Analysis

1. Introduction

Machine learning and statistical modelling are now standard tools in teaching, research and practice across many fields, yet using them well meets two obstacles. The first is technical: users without programming experience (especially students) spend much of their time installing software and debugging code rather than on modelling choices. The second is methodological: general-purpose tools rarely warn against the errors that matter most in real data, such as look-ahead bias, model selection on the test sample, and claims of forecasting success that actually do not beat a random walk.

4Cast Lab addresses both. It opens as a single file and guides the user from data import to a documented, tested model. Its defaults follow the forecasting literature rather than generic machine learning practice: time-ordered splits, selection by cross-validation, benchmarks against naive forecasts, and significance tests of predictive accuracy. The aim is that a user who follows the defaults produces results that would survive a referee's first questions.

The paper makes three contributions. It describes the design and interface of the software (Section 2 and Section 3). It reports a systematic validation of the numerical results against established reference software (Section 4) which is a precondition for using any program in theses and published research. Finally, it illustrates the workflow with three examples that show what the software reports and how to read it (Section 5).

2. Design and Architecture

4Cast Lab is a single HTML file written in JavaScript. All learning algorithms and estimators are implemented from scratch, so no server, Python environment or package manager is needed. Chart.js draws the figures and SheetJS reads and writes Excel files. The same file is compiled into a portable Windows executable that runs without installation or administrator rights. Four design choices shape the app:

• Privacy by construction. All computation is local. Nothing is uploaded, which allows the use of confidential, proprietary or licensed data. The only network connections are an update check and, if the user chooses to use it, an optional AI interpretation of the results through a provider of the user's choice.
• Two views. A Core view shows the essential settings, each with a short explanation; an Expert view exposes the full set of options and abilities of the app, and the Estimation mode. Switching views only shows or hides panels: the results are identical in both.
• Guidance in the interface. Every panel carries a "How to read this" note, every setting a help icon, and every set of results an "In plain words" summary that states the conclusion in one paragraph, including when a model fails to beat its benchmark. A Glossary tab defines every metric and test.
• Reproducibility. Every run records its settings and random seed, can be saved as a session file (.4cl) and reopened, and can be exported as a pandas and scikit-learn script that replays the split, preprocessing, model and metrics. A Methods Report writes the prose for the Methods section of a paper or thesis, and a Full Report documents the analysis from start to end.

Figure 1. The Core view (Forecasting mode). Settings are on the left; results appear in the tabs on the right, each with a "How to read this" note.

Figure 1. The Core view (Forecasting mode). Settings are on the left; results appear in the tabs on the right, each with a "How to read this" note.

3. Methods and Workflow

3.1. Forecasting (Machine Learning) Mode

A run in Forecasting mode follows six steps, each on its own tab:

• Data. Import (or drag and drop) CSV or Excel files, or use the built-in examples. An import report and a data preview list the column types, missing values, date coverage, gaps and duplicates. The app flags possible target leakage, checks multicollinearity (VIF) and supports panel data through a unit column.
• Target and horizon. For forecasting, the target is shifted to y(t+h), with h = 1 by default. Lags, differences and transformations are computed within each unit.
• Validation design. A time-series split with an optional embargo, and expanding-window cross-validation that purges rows around each fold (López de Prado, 2018), to avoid data leakage: no future information enters the training set. Random, stratified splits are available for cross-sections, with a warning when they are used on dated data.
• Training and tuning. Eight algorithms are available, individually or all at once (Train All Models & Compare); Table 1 lists them. Hyperparameters are tuned by cross-validation on the training set only, with a quick grid, full research grids, exhaustive grid search or random search (Bergstra and Bengio, 2012), and optional nested cross-validation. Variable selection offers LASSO, elastic net, recursive feature elimination, stability selection and filter methods.
• Evaluation. Regression reports RMSE, MAE, MAPE, sMAPE, R² and MASE (Hyndman and Koehler, 2006), with conformal prediction intervals (Vovk, Gammerman and Shafer, 2005). Classification reports ROC-AUC, PR-AUC, accuracy, sensitivity, specificity, F1, MCC, the Brier score and log loss, and adds class-imbalance handling, probability calibration (Platt scaling or isotonic regression) and cost-based thresholds. Every metric is shown for a) the cross-validation folds, b) the whole training rows and c) the Test Set, and are kept apart.
• Interpretation and reporting. Permutation importance (Fisher, Rudin and Dominici, 2019), Shapley values (Štrumbelj and Kononenko, 2010), impurity importance, mutual information and correlation rank the predictors; their agreement is measured by Kendall's W (Kendall and Babington Smith, 1939). Partial-dependence and ICE curves show the shape of each effect. Forecasts for new data and multi-step forecasts beyond the sample (direct or iterated) come with conformal or simulated bands.

Kernels. The support vector machine (Cortes and Vapnik, 1995) algorithm offers linear, RBF (Gaussian) and polynomial kernels, with the kernel treated as a hyperparameter in Train All. Classification is solved exactly by sequential minimal optimisation (Platt, 1998), as in LIBSVM (Chang and Lin, 2011) and scikit-learn, with a faster approximate solver as an alternative; the kernel width defaults to the scikit-learn "scale" rule, γ = 1/(p·Var X). For regression the linear kernel uses support vector regression, and the RBF and polynomial kernels use kernel ridge regression in closed form. Because non-linear kernels need an n × n Gram matrix, the app warns when the training sample is large enough to make them slow.

3.2. Benchmarks and the Evaluation of the Selected Model

The central device is the benchmark. On time-ordered data every model is compared with the stronger of the random walk and the training mean, chosen on training rows only. The Diebold and Mariano (1995) test with the Harvey, Leybourne and Newbold (1997) correction judges whether the difference in accuracy is significant; the Clark and West (2007) test allows for the benchmark being nested in the model; and the Pesaran and Timmermann (1992) test checks directional accuracy. For classification the benchmark is the base-rate forecast, and the AUC is tested against 0.5 (DeLong, DeLong and Clarke-Pearson, 1988).

Model selection is kept separate from evaluation. Train All gives each algorithm its own cross-validated grid search and selects the model with the best mean cross-validation score; the Test Set is used only once, to evaluate the selected model on data that no model has seen. A second table then asks whether the selected model is really better: it compares the selected model with each other model and with the benchmark on the same Test Set rows, using the Diebold-Mariano, DeLong or McNemar (1947) test according to the selection metric, a paired bootstrap interval of the difference, and Holm's (1979) correction for the number of comparisons. These tests check the choice made by cross-validation; they never change it, so the Test Set does not leak into the selection (Figure 2).

3.3. Estimation (Econometrics) Mode

Estimation mode, in the Expert view, fits models to the whole sample for explanation and inference, as in Stata or EViews (Figure 3). It offers OLS with classical, White (1980) HC0–HC3, Newey and West (1987) HAC and clustered standard errors; logit and probit with average marginal effects; IV/2SLS with first-stage diagnostics; ARIMA and SARIMA by exact maximum likelihood; ARDL with the bounds test of Pesaran, Shin and Smith (2001); error-correction models; VAR with impulse responses and variance decompositions; Johansen (1991) cointegration and VECM; and GARCH (Bollerslev, 1986). A specification box adds dummies, interactions, polynomials, lags, trends and seasonal dummies. The Tests tab reports residual diagnostics, specification tests, Wald tests of any linear restriction, multicollinearity and structural breaks (Chow, Bai-Perron), each with a plain reading at the 5% level. Estimated models can be collected in a journal-style Results Table exported to LaTeX, Word or Excel (Figure 4), and an ARIMA, ARDL or VAR can be sent to Forecasting mode as a benchmark for the machine learning models on the same Test Set rows.

4. Validation

Software used for theses and published research must reproduce established results. Version 4.6.2 was therefore tested against reference implementations on standard public datasets and certified benchmarks. Each test calls the app's own numerical routines, or drives its interface in a headless browser, and compares the output with the reference on identical inputs: statsmodels 0.15 (Seabold and Perktold, 2010), linearmodels 7.0, arch 8.0, scikit-learn 1.9 (Pedregosa et al., 2011), the NIST StRD certified values for the Longley (1967) data, and R where statsmodels follows a different convention. Table 2 summarises the 1,234 checks.

The application passes all checks. The deterministic estimators (least squares, maximum likelihood with closed-form or well-conditioned optima, tests and their p-values) agree with the references to within 10⁻⁶ in relative terms, most to 10⁻⁹ or better, including the ill-conditioned Longley regression. Random forest, gradient boosting and the neural network cannot match scikit-learn exactly, because their random number streams and solvers differ; they were checked for equal accuracy on the same rows instead and passed.

Where the app and a reference disagree because they follow different conventions, the app follows the convention used in the econometric literature and in Stata, and the Methods Report states it. Examples are the regression form of the Wu-Hausman test, t(G−1) inference with clustered standard errors, the HC1 small-sample factor, and the textbook form of the Johansen test with a trend. In four cases the reference itself was at fault: an off-by-one count of regressors in the critical values of the statsmodels bounds test, an approximation in the statsmodels Johansen test with a trend, a projection variant of the Wu-Hausman test in linearmodels, and printed values for the Box-Jenkins airline model that come from an older estimator.

The 1,234 checks also revealed six defects in a pre-release build, all corrected in the released version 4.6.2: a crash in the GARCH starting values for 22 of 36 model orders; standard errors that became undefined when a GARCH parameter lay on its bound; a local optimum for GJR-GARCH(1,2,1); a small ridge penalty in the machine learning linear regression, now replaced by an exact QR solution; and two wording errors in the Methods Report about tuning and the SVM kernel. The complete validation report, with every check and its tolerance, and the test scripts are available from the author and with the release in Github.

5. Illustrative Examples

The three examples below use the built-in teaching datasets, so any reader can reproduce them; all three are simulated with a known data-generating process. Each takes a few minutes and follows the Quick Start videos of the software.

5.1. Classification: Credit Default

The Credit Default example has 250 loan applicants, a binary target (default) and three predictors: income, debt and credit score. A stratified random split with a fixed seed keeps 51 applicants (20%) as the Test Set. A random forest with quick tuning reaches a cross-validated AUC of 0.978 and a Test Set AUC of 0.996, against 0.5 for the base-rate forecast (DeLong p < 0.001), and a Brier score of 0.047 against 0.208. Train All then tunes all ten configurations on the same rows (Table 3). Cross-validation selects the SVM with a polynomial kernel (C = 10; mean CV AUC 0.997, SE 0.002), whose Test Set AUC is 1.000. The evaluation of the selected model shows that it is significantly better than the benchmark but statistically tied with all nine alternatives after the Holm correction: on 51 rows, the data cannot separate the best models, and the app says so rather than simply declaring a winner. In the Interpretability tab, debt and credit score rank above income on most measures, with moderate agreement between the five criteria (Kendall's W = 0.52, p = 0.074).

5.2. Forecasting: Benchmark First

The Macro example has 288 simulated months (2000–2023) of industrial production, prices, money, the policy rate, stock returns and unemployment, which follows Okun's law. The target is next month's unemployment rate (h = 1) and the predictors are the other five series. The time-series split keeps the last 58 months as the Test Set, and each algorithm is tuned by expanding-window cross-validation on the first 229. Cross-validation selects gradient boosting (CV RMSE 0.358). Its Test Set R² is 0.87, which would look like a success in isolation. Table 4 shows why it is not: the random walk, chosen as the benchmark on the training rows, forecasts the Test Set better, with an RMSE of 0.183 against 0.224. On the training rows the ratio is 0.43, so the model's apparent advantage there is an artefact of fitting. The Diebold-Mariano test does not reject equal accuracy at 5% (p = 0.054), and the pairwise table finds gradient boosting tied with the random walk after the Holm correction (p = 0.378). The Clark-West test (p = 0.032) indicates that the predictors carry some information beyond the random walk, but not enough to produce more accurate forecasts on this sample, and directional accuracy is not significant (Pesaran-Timmermann p = 0.380). The "In plain words" summary states this conclusion directly.

5.3. Econometrics: Wage Equation and ARIMA

The Wages example is a simulated cross-section of 1,000 workers. Table 5 reports the OLS regression of the hourly wage on education, experience, sex and union membership with White HC1 standard errors, as it appears in the Results Table. One more year of education raises the wage by 1.33 (p < 0.001), holding the other regressors constant. The Tests tab supports the choice of robust errors: the Koenker form of the Breusch-Pagan test rejects homoskedasticity (LM = 62.5, p < 0.001), while Breusch-Godfrey finds no serial correlation (p = 0.43). The RESET test (F = 10.7, p < 0.001) suggests that the level specification is too restrictive; in this simulated design wages are log-linear in education, which the user can verify by building log(wage) on the Data tab. The same dataset supports IV estimation with distance to college as the instrument.

For time series, an ARIMA(1,1,1) of the monthly unemployment rate gives φ = 0.732 (SE 0.096) and θ = −0.429 (SE 0.127), with an insignificant drift, by exact maximum likelihood (log-likelihood 114.46, AIC −220.9). The Dynamics & Forecasts tab reports forecasts for twelve months with 95% intervals that widen with the horizon, as they should for an integrated model (Figure 5). The same model can be sent to Forecasting mode and compared with the machine learning models of Section 5.2 on the same Test Set.

Figure 4. The Results Table: estimated models side by side in journal layout, exported to LaTeX, Word or Excel.

Figure 4. The Results Table: estimated models side by side in journal layout, exported to LaTeX, Word or Excel.

Figure 5. Dynamics & Forecasts: the ARIMA(1,1,1) forecast of unemployment with its 95% interval.

Figure 5. Dynamics & Forecasts: the ARIMA(1,1,1) forecast of unemployment with its 95% interval.

6. Uses in Research and Teaching

For research and professional work, the Expert view, nested cross-validation, the pairwise tests, the reports and the Python export allow a model built in the app to be described and reproduced in a paper or a technical report. In teaching, the Core view, the plain-language summaries and the built-in examples support courses on machine learning, econometrics and data analytics, and three exercises illustrate the methodology: benchmark first, as in Section 5.2; leakage by example, where the same model is run with a random split and with an embargoed time split to show how look-ahead bias inflates accuracy; and costs and thresholds, where students set misclassification costs on the credit data and see the optimal threshold, the confusion matrix and the expected cost on the ROC panel. Two Quick Start videos and a written guide introduce the Forecasting and Estimation modes in a few minutes each. Doctoral students of the E|M|E|F Lab tested successive versions on their own datasets; their reports led to corrections in panel handling, benchmark choice and date parsing.

7. Limitations and Future Work

Because it runs in a browser engine, 4Cast Lab suits datasets of thousands to tens of thousands of rows rather than big data, and kernel methods are slower still because of the Gram matrix. Full tuning of all learners on 1,000 rows takes about two minutes. Random forest, boosting and the neural network agree with scikit-learn only approximately, since solvers and random number streams differ. Classification is currently binary, and the predictive-accuracy tests are applied to the Test Set only.

Planned work includes multi-class classification, rolling-origin evaluation of multi-step forecasts, and data-snooping-robust comparison of many models through White's (2000) Reality Check, Hansen's (2005) SPA test and the Model Confidence Set of Hansen, Lunde and Nason (2011).

8. Availability and Citation

4Cast Lab is free software under the MIT license. The source code, releases, a Quick Start guide and the Quick Start videos are linked from https://github.com/PerryGogas/4Cast-Lab, and each release is archived on Zenodo. Users are asked to cite the software through its concept DOI, which always resolves to the latest version: 10.5281/zenodo.22818865.

Suggested citation: Gogas, P. (2026). 4Cast Lab: Open-source app for machine learning and statistical analyses (Version 4.6.2) [Computer software]. E|M|E|F Lab, Democritus University of Thrace. https://doi.org/10.5281/zenodo.22818865

Declaration of generative AI use. During the preparation of this work the author used Claude (Anthropic) to help draft and edit the text, to add screenshots of the app, to run the validation tests against reference software, and to identify and correct software defects. The author reviewed and edited all content, verified the results, and takes full responsibility for the publication.