Extending the Fixed-Effects Model on Nonlinear Least Squares

1. Introduction

The panel data over both cross-section and time series gives users the possibility of accounting for multiple sources of unobserved heterogeneity in economics especially econometrics and statistics. Model formulations in which individual and/or time heterogeneity factors are considered fixed parameters than random variables, are called fixed-effects model [1]. The commonality (i.e., coeffect) of the distribution characteristics of time series datas can reflect the overall trend of the panel data, which is affected by independent variables including time respectively. In general, if the influence of time on the distribution of panel data satisfies certain rules, it is considered that there is time-effect in the distribution of the panel data. After removing the coeffect from the panel data, if the distribution of a time series data satisfies a certain rule, it is considered that the time series data in the panel data satisfies individual fixed-effect. Time-effects and individual fixed-effects are collectively called two-way fixed-effects or fixed-effects. Sometimes, fixed-effects in the narrow sense refer to individual fixed-effects.

1.1. Basic Definitions and Methods

There are some assumptions on the classical definitions of fixed-effects[2]. First, all time series datas distribute in linear with same slope(or slopes) between the independent variable(or variables) except the time and the dependent variable(or variables). Second, if there is a time fixed-effect in the panel data, the time fixed-effect is relatively independent and has nothing to do with other independent variables. Third, if there is an individual fixed-effect in the panel data, it is either relatively independent and has nothing to do with any independent variable beside the time (i.e., random-effects) or with some independent variables(i.e., narrow fixed-effects). Moreover, if all the time series have the same slope and intercept, some scholars call them mixed-effects. Besides, random-effects and mixed-effects are thought to be generalized fixed-effects [3]. In this paper, fixed-effects refer to fixed-effects in a broad sense. Therefore, functional expressions for fixed-effects differ in the number of dependent variables, and then they exist in two basic forms.The one form is single independent (i.e., input) variable beside the time with single dependent(i.e., output) variable. The other form is multiple independent variables beside the time with single dependent variable.

In the subsection, the random-effects are taken as an example to give the classical definitions and estimation method.

Definition 1. 

[4] Given the independent variables $\{x_1,x_2,\dots ,x_k\}$ where $k\in \mathbf{N}$ representing the count of independent variables. Let $x_{i,j,t}$ represent the value of $x_j$ for the $i-th$ individual series in time t. Let $y_{i,t}$ be the value of dependent variable y for the $i-th$ individual series in time t while ${\epsilon }_{i,t}$ is the random residual error. Thus, the corresponding linear regression equation is shown below.

$$y_{i,t}=\sum _{j=1}^k({\beta }_j\times x_{i,j,t}+{\alpha }_j)+u_i+v_t+{\epsilon }_{i,t}$$

(1)

In the equation 1, all time series datas share the same slope ${\beta }_j$ with the same intercept ${\alpha }_j$ for the independent variable $x_j$. Meanwhile, the individual fixed-effect $u_i$ changes with individual i, but remains internally consistent for the same time series data. Similarly, the time fixed-effect $v_t$ changes with the time t, but remains internally consistent for the same cross-section data.

Since $y_{i,t}$ and $x_{i,j,t}$ have been given by a panel data for each i, j and t, the parameters like ${\alpha }_j$, ${\beta }_j$, $u_i$ and $v_t$ should be estimated with a residual ${\epsilon }_{i,t}$ as small as possible to check whether there are individual fixed-effects (i.e., $u_i$) and time fixed-effect (i.e., $v_t$). Let the time cycle be from 1 to $\tau$ (i.e., $t\in \{1,2,\cdots ,\tau \}$). Therefore, there are three main methods to estimate parameters such as fixed-effects transformation, dummy variable, first difference, Monte Carlo simulation and so on [5]. For example, the fixed-effects transformation is as following.

$$\begin{array}{cc}\hfill & y_{i,t}-\overline{y_i}\hfill \\ \hfill =& (\sum _{j=1}^k({\beta }_j\times x_{i,j,t}+{\alpha }_j)+u_i+v_t+{\epsilon }_{i,t})-(\sum _{t=1}^{\tau }\sum _{j=1}^k({\beta }_j\times x_{i,j,t}+{\alpha }_j)+u_i+v_t+{\epsilon }_{i,t}))/\tau \hfill \\ \hfill =& \sum _{j=1}^k({\beta }_j\times (x_{i,j,t}-\overline{x_{i,j}})+({\alpha }_j-\overline{{\alpha }_j}))+(u_i-\overline{u_i})+(v_t-\sum _{t^{\prime }=1}^{\tau }{v}_{t^{\prime }})+({\epsilon }_{i,t}-\sum _{t^{\prime }=1}^{\tau }{\epsilon }_{i,t^{\prime }}/\tau )\hfill \\ \hfill =& \sum _{j=1}^k({\beta }_j\times (x_{i,j,t}-\overline{x_{i,j}}))+(v_t-\sum _{t^{\prime }=1}^{\tau }{v}_{t^{\prime }})+({\epsilon }_{i,t}-\overline{{\epsilon }_i})\hfill \end{array}$$

(2)

In equation 2, the individual fixed-effects variable $u_i$ can be eliminated as the fixed-effect of each individual doesn’t change with the time (i.e., $u_i=\overline{u_i}$). Let $y_{i,t}-\overline{y_i}={\ddot{y}}_{i,t}$, $x_{i,j,t}-\overline{x_{i,j}}={\ddot{x}}_{i,j,t}$, $v_t-{\sum }_{t^{\prime }=1}^{\tau }{v}_{t^{\prime }}={\ddot{v}}_{i,t}$, ${\epsilon }_{i,t}-\overline{{\epsilon }_i}={\ddot{\epsilon }}_{i,t}$. Thus, the equation 2 can be changed as

$${\ddot{y}}_{i,t}=\sum _{j=1}^k({\beta }_j\times {\left(\ddot{y}\right)}_{i,t})+{\ddot{v}}_{i,t}+{\ddot{\epsilon }}_{i,t}$$

(3)

Furthermore, the parameters in the equation 3 can be estimated by the mixed Ordinary Least-Squares regression (i.e., OLS) [6].

1.2. Related Works

In order to meet the application needs of economics, management and other disciplines, the fixed-effect model has been continuously expanded and deepened. Since the count of independent variables get to be larger and larger, Correia et al. [7] propose a new command for estimation of (pseudo-)Poisson regression models with multiple high-dimensional fixed-effects. Similarly, sufficient dimension reduction is used for fixed-effects anylysis [8]. Meanwhile, meta analysis also works on fixed-effects [9]. On the contrary, some datas(i.e., long panels) have long time periods, Fernández-Val Iván and Weidner Martin focus on semiparametric models with unobserved individual and time-effects [10] while short time is also mentioned [11]. As fixed-effects anylysis must work on data, repeated measures data, unmeasured confounding data, and others are noticed and resolved [12,13]. Moreover, lagged periods are sued in dynamic fixed-effects modeling for that the effects of variables may be cross-period [14,15]. With the advent of the era of artificial intelligence, algorithms such as LASSO, tree-structured clustering, DNN, LSTM, multimodal, Bayesian and so on have been introduced to improve fixed-effects analysis [16,17,18,19,20,21]. However, major of fixed-effects analysis methods assumes that the panel data distributes in linear while nonlinear relationships are a common phenomenon in the real word [22]. In fact, fixed-effects in nonlinear is also be mentioned. William use Monte Carlo methods to examine the small sample bias in the binary probit and logit models, the ordered probit model, the tobit model, the Poisson regression model for count data and the exponential regression model for a nonnegative random variable [23]. Karyne investigates whether existing methods can be modified to eliminate multiple fixed-effects for some specific models in which the incidental parameter problem has already been solved in the presence of a single fixed-effect [24]. Irene and Chris discuss identification of counterfactual parameters in a class of nonlinear semiparametric panel models with fixed-effects and arbitrary time-effects [25], but it needs to derive bounds of time-stationarity. In order to better explain causality, the explainability of fixed-effects is hotly debated [26,27]. Although Monte Carlo methods sometimes work well in the nonlinear panel data, the interpretability of the relationship between process and outcome is open to question. Additionally, existing nonlinear fixed-effect models work on logistic regression but not others like polynomial regression.

1.3. Main Contributions

To sum up, existing fixed-effects methods have three distinct restrictions at least. Most importantly, nonlinear relationships beside logistic regression are common in the real world, but not covered by existing methods. Additionally, expert knowledge is not fully utilized in the fixed-effects analysis and only exists in the form of preliminary judgments. In other words, experts only preliminarily judge whether there may be fixed-effects and rely on ready-made models for verification, but do not use their professional knowledge to intervene in the verification of fixed-effects models. Furthermore, time-effects do not eliminate the coeffect of time on the panel data as a whole, that is, the time-effects fail to reflect heterogeneity.

In this paper, an extension fixed-effects model is proposed on the curve_fit nonlinear least squares based on the expert knowledge.

• First, the regressions between independent variables including time and dependent variable is estimated on each time series data, and the parameters are estimated by curve_fit. Experts should guess the distribution functions between independent variables and the dependent variable based on professional theory and experience. The functions can be linear, which can be thought of as specially nonlinear, or they can be nonlinear. Users follow curve_fit to verify the matching degree of the functions until satisfied, otherwise adjust the functions. Similarly, the coeffect of the whole panel data is also initially estimated.
• Second, the distance between regressions is defined, and then the coeffect function between the series regressions is obtained on the initial co-effect with curve_fit. Series regressions may be affected by coeffect in panel data, which reflects the commonality of these functions in distributions. To find coeffect functions that minimizes the sum of the squares of the average distance from each series sub-function as the common function. Thus, curve_fitting method is improved for the co-effect finding.
• Third, the time-effects and individual fixed-effects for each series data in the panel data are estimated on the differences of series regressions and coeffect function. However, coeffect may be affected by time or other independent variables, which may cause bias in the analysis. Thus, We divide each regression into two sub-functions according to time and other independent variables. By eliminating the coeffect in series regressions, new individual fixed-effect and time-effect are obtained to better reflect the heterogeneity.
• Additionally, the effectiveness of the proposed method is varified on the synthetic data. The method of verifying the validity of the classical model is usually based on expert knowledge and lacks objective metrics. This paper generates synthetic data based on the theory of fixed-effects, and determines the validity of model estimation by comparing the similarity between the expected fixed-effects and the estimated fixed-effects.

Therefore, there are three main contributions in our works.

(1)

First of all, the nonlinear relationships which generally exist in the real world are reflected by nonlinear functions, so that the fixed-effects model can be effectively extended at the application level.

(2)

Secondly, the prior knowledge of experts is fully utilized in the proposed fixed-effects analysis through the estimation function to prevent the separation of theory and empirical research, so that the model has better interpretability and is easier to be understood.

(3)

Thirdly, the proposed time-effects eliminate the coeffect of time in panel data, which can more effectively reflect the heterogeneity of time series datas in time distribution. Finally, individual fixed-effects can be either correlated or uncorrelated with independent variables, avoiding the distinction between fixed-effects and random effects.

(4)

In addition, based on synthetic data, a method to verify the validity of the fixed-effects model is proposed.

2. Proposed Method

2.1. Redefinition

In order to highlight the research topic, this paper does not discuss the endogeneity of variables and latent variables, and assumes that the selection of independent and dependent variables is reasonable. Given a pannel data $Pd$ over cross-section and time series datas with k independent variable vector X (i.e., $\{x_1,x_2,\cdots ,x_k\}$) following by cross-section variable i ($i\in \{1,2,\cdots ,l\}$) in time periods t ($t\in \{1,2,\cdots ,\tau \}$, and the dependent variable y. Thus, $x_{i,j,t}$ represents the value of $x_j$ on the $i-th$ series in time t. Meanwhile, $y_{i,t}$ means the value of y on the $i-th$ series in time t.

It is assumed that in panel data $Pd$, the distributions of $Pd$ follows certain rules (or characteristics) on time series with heterogeneity. First, regardless of the series datas, all data distribution in the panel data follows a coeffect $y=f_c(X,t)$. Second, in the $i-th$ time series data, the data distributions follow the function $y_i=f_i(X,t)$.

Definition 2. 

The differece $f_{i-c}(X,t)$ between the $i-th$ time series data distribution and the coeffect calls the fixed-effect of the $i-th$ time series data in the panel data $Pd$.

$$f_{i-c}(X,t)=f_i(X,t)-f_c(X,t)$$

(4)

If X and t are independent with each other in both $f_i(X,t)$ and $f_c(X,t)$, $f_i(X,t)$ and $f_c(X,t)$ can be decomposed into $f_i^{\prime }\left(X\right)+g_i\left(t\right)$ and $f_c^{\prime }\left(X\right)+g_c\left(t\right)$,respectively. Namely,

$$f_i(X,t)=f_i^{\prime }\left(X\right)+g_i\left(t\right)$$

(5)

$$f_c(X,t)=f_c^{\prime }\left(X\right)+g_c\left(t\right)$$

(6)

Therefore, the equation 4 can be rewritten as

$$\begin{array}{cc}\hfill f_{i-c}(X,t)& =(f_i^{\prime }\left(X\right)+g_i\left(t\right))-(f_c^{\prime }\left(X\right)+g_c\left(t\right))\hfill \\ & =(f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right))+(g_i\left(t\right)-g_c\left(t\right))\hfill \end{array}$$

(7)

If there is no appropriate function to fit the data distribution, the corresponding function can be 0 (i.e., false).

Since $f_c^{\prime }\left(X\right)$ represents the effect of X on y on panel data $Pd$, which is generic. And $f_i^{\prime }\left(X\right)$ represents the effect of X on y on the $i-th$ time series data. Thus, $f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right)$ represents the personality effect of X on y on the $i-th$ time series data. We call that

Definition 3. 

$f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right)$ is the individual fixed-effect of the $i-th$ series data on the panel data $Pd$.

If $f_c^{\prime }\left(X\right)=0$ but $f_i^{\prime }\left(X\right)\ne 0$, it means there is an absolute heterogeneiry (i.e., absolute individual fiexed-effect) affected by X in the $i-th$ series data. Additionally, if $f_i^{\prime }\left(X\right)=0$ with most likely $f_c^{\prime }\left(X\right)=0$ because series stochasticity affected by X most likely leads to panel stochasticity affected by X, it means the series data is randomly distributed and the significance of heterogeneity affected by X (i.e., individual fixed-effect) is not explored. If $f_i^{\prime }\left(X\right)=f_c^{\prime }\left(X\right)\ne 0$, that is to say, the $i-th$ series data distributes the same affected by X with the whole panel data. In other words, there is no heterogeneity affected by X (i.e., no individual fixed-effect) between the $i-th$ series data and the panel data when $f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right)=0$.

$$i{e}_i=\left\{\begin{array}{cc}False\hfill & f_i^{\prime }\left(X\right)=0or{f}_i^{\prime }\left(X\right)=f_c^{\prime }\left(X\right)\hfill \\ f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right)\hfill & others\hfill \end{array}\right.$$

(8)

where $i{e}_i$ represents the individual fixed-effect of the $i-th$ series data in the the panel data.

Similarlly, because $g_i\left(t\right)$ represents the effect of t on y on panel data $Pd$, which is generic. And $g_c\left(t\right)$ represents the effect of t on y on the $i-th$ time series data. Thus, $g_i\left(t\right)-g_c\left(t\right)$ represents the personality effect of t on y on the $i-th$ time series data. We call that

Definition 4. 

$g_i\left(t\right)-g_c\left(t\right)$ is the time-effect of the $i-th$ series data on the panel data $Pd$.

If $g_c\left(t\right)=0$ but $g_i\left(t\right)\ne 0$, it means there is an absolute heterogeneiry affected by time t (i.e., absolute time-effect) in the $i-th$ series data. Additionally, if $g_i\left(t\right)=0$ with most likely $g_c\left(t\right)=0$ because series stochasticity affected by t most likely leads to panel stochasticity affected by t, it means the series data is randomly distributed and the significance of heterogeneity affected by t (i.e., time-effect) is not explored. If $g_i\left(t\right)=g_c\left(t\right)\ne 0$, that is to say, the $i-th$ series data distributes the same affected by t with the whole panel data. In other words, there is no heterogeneity affected by t (i.e., no time-effect) between the $i-th$ series data and the panel data when $g_i\left(t\right)-g_c\left(t\right)=0$.

$$t{e}_i=\left\{\begin{array}{cc}False\hfill & g_i\left(t\right)=0or{g}_i\left(t\right)=g_c\left(t\right)\hfill \\ g_i\left(t\right)-g_c\left(t\right)\hfill & others\hfill \end{array}\right.$$

(9)

where $t{e}_i$ represents the time-effect of the $i-th$ series data in the the panel data.

Compared with the equation 1, it can be seen in the equation 7 that $f_i^{\prime }\left(X\right)$, $f_c^{\prime }\left(X\right)$, $g_i\left(t\right)$, and $g_c\left(t\right)$ are undetermined. All of them can be either in linear or nonlinear, let alone with the same slope in time series data distributions. Furthermore, the time-effects are not only a function of the time period, but also the results of eliminating the ceffect of time on the panel data in time series datas. Obviously, in comparison with the classical time-effects, the conditions of time-effects in the denifition 4 are relaxed, and the commonality is no longer emphasized, but the individuality is highlighted, which is more in line with the purpose of fixed-effect analysis. In a word, the classical fixed-effects are extended.

2.2. Solutions

It is well known that when analyzing whether there are fixed-effects in a panel data, only the values of $x_{i,j,t}$ for each independent variable j ($i\in \{1,2,\cdots ,k\}$) in each $i-th$ ($i\in \{1,2,\cdots ,l\}$) series data in the time period t ($i\in \{1,2,\cdots ,\tau \}$) are usually given without knowing the distribution functions.

Therefore, there are two main works should do. First, we need to know what type of distributions $f_i^{\prime }\left(X\right)$, $f_c^{\prime }\left(X\right)$, $g_i\left(t\right)$, and $g_c\left(t\right)$ satisfy (e.g., polynomial distribution, linear distribution, or constants). Second, we need to estimate their parameters according to their distribution types, so as to realize the fixed-effects prediction. Of course, if there are no fixed-effects in the original panel data, such speculation and estimation should be proved not to be accepted. Otherwise, the model is invalid.

Since these functions (i.e., $f_i^{\prime }\left(X\right)$, $f_c^{\prime }\left(X\right)$, $g_i\left(t\right)$, and $g_c\left(t\right)$) are curves in the pannel data over time series datas, curves fitting on the nonlinear least square is considered the main solution.

2.2.1. Curve Fitting

The idea of curve fitting is to find a mathematical model that fits a series of data points, possibly subject to constraints. It is assumed that users have theoretical reasons for picking a function of a certain form in general. The curve fitting finds the specific coefficients (parameters) which make that function match the data as closely as possible. In detail,raw data and a function with unknown coefficients are given in curve fitting. The main target is to find values for the coefficients such that the function matches the raw data as well as possible. The best values of the coefficients are the ones that minimize the value of chi-square. Chi-square is defined as

$$\sum _i{\left(\frac{f(x_i,\theta )-y_i}{{\sigma }_i}\right)}^2$$

(10)

where $f(x_i,\theta )$ is a fitted value (model value) for a given point $(x_i,y_i)$ on the function f with the estimated parameter vector $\theta$. $y_i$ is the measured data value for $x_i$ and ${\sigma }_i$ is an estimate of the standard deviation for $y_i$. In other words, for raw data points consisting of $(x_i,y_i)$, the minimum chi-square with the estimated parameter set $\theta$ for the given function f is expected. To get best paramters $\theta$, nonlinear least square is used.

For the nonlinear built-in data fitting functions and for user-defined functions, the operation must be iterative. Igor tries various values for the unknown coefficients. For each try, it computes chisquare searching for the coefficient values that yield the minimum value of chi-square.

Given some initial guess of the parameter values $\theta ={\theta }_0$, hold $x_i$ constant and form the first order Taylor approximation

$$\begin{array}{cc}\hfill & f(x_i,\theta )-y_i\hfill \\ \hfill \approx & \left[(f(x_i,{\theta }_0)+\frac{d}{d\theta }f(x_i,\theta ){|}_{\theta ={\theta }_0}(\theta -{\theta }_0))\right]-y_i\hfill \\ \hfill \approx & [f(x_i,{\theta }_0)+f_{\theta }(x_i,{\theta }_0)(\theta -{\theta }_0)]-y_i\hfill \\ \hfill \approx & [f(x_i,{\theta }_0)-f_{\theta }(x_i,{\theta }_0){\theta }_0-y_i]+f_{\theta }(x_i,{\theta }_0)\theta \hfill \\ \hfill \approx & y_i^{\prime }+f_{\theta }(x_i,{\theta }_0)\theta \hfill \end{array}$$

(11)

and the nonlinear least square problem has become a linear least square promblem [28].

2.2.2. Detailed Steps

(1)Curves fitting in series datas

A panel data includes a group of time series datas. For each time series data, the industry expert can give a function to describe the data distribution between independent variables including time period and dependent variable according to their prior knowledge. First, the distribution type of the function marked as ${\psi }_i\left({\theta }_i\right)$ can be guessed based on the theoretical knowledge held by the expert where ${\psi }_i\left({\theta }_i\right)$ represents the funtion ${\psi }_i$ whose paramter vector is ${\theta }_i$. For example, the threshold effect may be related to the sigmoid function, and that U-shaped curves usually correspond to quadratic functions. Second, the initial parameters ${\psi }_i$ can be guessed as ${\psi }_{i0}$ by the expert. Third, given the function ${\psi }_i\left({\theta }_i\right)$ and initial parameters ${\psi }_{i0}$, truth parameters can be estimated by the curve-fitting with the nonlinear least square following the equations 10 and 11. According to the equation 4, the ideal result is

$${\psi }_i\left({\theta }_i\right)=f_i(X,t)$$

(12)

However, $f_i(X,t)$ is usually unknown. Thus, the smaller the residual(i.e., the equation 10), the better. If the residual is too big(e.g., greater than a give threshold), ${\psi }_i\left({\theta }_i\right)$ should be adjusted to meet the requirements, or the fitting should be rejected if they are not satisfied after many adjustments.

Particularly, ${\psi }_i\left({\theta }_i\right)$ can be either linear or nonlinear. That is to say, the functions satisfy the nonlinear leads an extension of curve fitting in classical fixed-effects analysis.

Similarly, if the time series data are mixed and the overall panel data distribution characteristics(marked ${\psi }_c\left({\theta }_c\right)$) are estimated. The ideal result is

$${\psi }_c\left({\theta }_c\right)=f_c(X,t)$$

(13)

as shown in the equation 4. The smaller the residual(i.e., the equation 10), the better. If the residual is too big(e.g., greater than a give threshold), ${\psi }_i\left({\theta }_c\right)$ should be adjusted to meet the requirements, or the fitting should be rejected if they are not satisfied after many adjustments.

Therefore, if both ${\psi }_i\left({\theta }_i\right)$ and ${\psi }_c\left({\theta }_c\right)$ are acceptable, the heterogeneity(i.e., fixed-effect) of the $i-th$ time series data in the panel data $Pd$ is

$${\psi }_i\left({\theta }_i\right)-{\psi }_c\left({\theta }_c\right)$$

(14)

,or else the fixed-effect isn’t acceptable.

(2)Time-effects and individual fixed-effects estimate

If the independent variables and the time periods are independent of each other when there are fixed-effects in the panel data $Pd$, separate ${\psi }_i\left({\theta }_i\right)$ and ${\psi }_c\left({\theta }_c\right)$ with time and other independent variables into two parts, namely,

$${\psi }_i\left({\theta }_i\right)={\varphi }_i\left({\theta }_{i\setminus t}\right)+{\omega }_i\left({\theta }_t\right)$$

(15)

where ${\varphi }_i\left({\theta }_{i\setminus t}\right)$ fits the effects of independent variables except the time and ${\omega }_i\left({\theta }_t\right)$ fits the effects of time on the dependent variable in each series data.

Similarly,

$${\psi }_c\left({\theta }_c\right)={\varphi }_c\left({\theta }_{c\setminus t}\right)+{\omega }_c\left({\theta }_t\right)$$

(16)

where ${\varphi }_c\left({\theta }_{c\setminus t}\right)$ fits the effects of independent variables except the time and ${\omega }_c\left({\theta }_t\right)$ fits the effects of time on the dependent variable in the global panel data.

Combining the equation 15 and 16 and comparing with the equation 7, we can see that the closer ${\varphi }_i\left({\theta }_{i\setminus t}\right)$, ${\omega }_i\left({\theta }_t\right)$, ${\varphi }_c\left({\theta }_{c\setminus t}\right)$, ${\omega }_c\left({\theta }_t\right)$ are correspondingly to $f_i^{\prime }\left(X\right)$, $g_i\left(t\right))$, $f_c^{\prime }\left(X\right)$, $g_c\left(t\right)$, the better. Thus, following the equation 7, the time-effect of the $i-th$ series data in the panel data can be initialized as

$${\omega }_i\left({\theta }_t\right)-{\omega }_c\left({\theta }_t\right)$$

(17)

, and the corresponding individual fixed-effect can be initialized as

$${\varphi }_i\left({\theta }_{i\setminus t}\right)-{\varphi }_c\left({\theta }_{c\setminus t}\right)$$

(18)

However, either ${\omega }_c\left({\theta }_t\right)$ or ${\varphi }_c\left({\theta }_{c\setminus t}\right)$ may be offset by estimating together with other independent variables. In order to correct this bias, we use the principle of least squares to correct ${\omega }_c\left({\theta }_t\right)$ and ${\varphi }_c\left({\theta }_{c\setminus t}\right)$ based on the loss function over the the function similarity. The loss function is defined as

$$\frac{{\sum }_{i=1}^k{\int }_{t=1}^{\tau }{\left(\frac{{\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)-{\omega }_i\left({\theta }_t\right)}{min({\omega }_c^{\prime }\left({\theta }_t^{\prime }\right),{\omega }_i\left({\theta }_t\right))}\right)}^2}{k}$$

(19)

instead of the equation 10. And then, the function with the best parameter vector for ${\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)$ can be catched.

Similarly, the loss function is defined as

$$\frac{{\sum }_{i=1}^k{\int }_{t=1}^{\tau }{\left(\frac{{\varphi }_c^{\prime }\left({\theta }_{i\setminus t}^{\prime }\right)-{\varphi }_i\left({\theta }_{i\setminus t}\right)}{min({\varphi }_c^{\prime }\left({\theta }_{i\setminus t}^{\prime }\right),{\varphi }_i\left({\theta }_{i\setminus t}\right)}\right)}^2}{k}$$

(20)

instead of the equation 10. And then, the function with the best parameter vector for ${\varphi }_c^{\prime }\left({\theta }_{i\setminus t}^{\prime }\right)$ can be catched.

Along this line, the time-effects can be corrected as

$${\omega }_i\left({\theta }_t\right)-{\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)$$

(21)

, and the corresponding individual fixed-effect can be corrected as

$${\varphi }_i\left({\theta }_{i\setminus t}\right)-{\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)$$

(22)

If ${\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)=0$ but ${\omega }_i\left({\theta }_t\right)\ne 0$, it means there is an absolute heterogeneiry affected by time t (i.e., absolute time-effect) in the $i-th$ series data. Additionally, if ${\omega }_i\left({\theta }_t\right)=0$ with most likely ${\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)=0$ because series stochasticity affected by t most likely leads to panel stochasticity affected by t, it means the series data is randomly distributed and the significance of heterogeneity affected by t (i.e., time-effect) is not explored. If ${\omega }_i\left({\theta }_t\right)={\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)\ne 0$, that is to say, the $i-th$ series data distributes the same affected by t with the whole panel data. In other words, there is no heterogeneity affected by t (i.e., no time-effect) between the $i-th$ series data and the panel data when ${\omega }_i\left({\theta }_t\right)-{\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)=0$.

$$t{e}_i=\left\{\begin{array}{cc}False\hfill & {\omega }_i\left({\theta }_t\right)=0or{\omega }_i\left({\theta }_t\right)={\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)\hfill \\ {\omega }_i\left({\theta }_t\right)-{\omega }_c^{\prime }\left({\theta }_t^{\prime }\right)\hfill & others\hfill \end{array}\right.$$

(23)

where $t{e}_i$ represents the time-effect of the $i-th$ series data in the the panel data.

If ${\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)=0$ but ${\varphi }_i\left({\theta }_{i\setminus t}\right)\ne 0$, it means there is an absolute heterogeneiry (i.e., absolute individual fiexed-effect) affected by X in the $i-th$ series data. Additionally, if ${\varphi }_i\left({\theta }_{i\setminus t}\right)=0$ with most likely ${\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)=0$ because series stochasticity affected by X most likely leads to panel stochasticity affected by X, it means the series data is randomly distributed and the significance of heterogeneity affected by X (i.e., individual fixed-effect) is not explored. If ${\varphi }_i\left({\theta }_{i\setminus t}\right)={\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)\ne 0$, that is to say, the $i-th$ series data distributes the same affected by X with the whole panel data. In other words, there is no heterogeneity affected by X (i.e., no individual fixed-effect) between the $i-th$ series data and the panel data when ${\varphi }_i\left({\theta }_{i\setminus t}\right)-{\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)=0$.

$$i{e}_i=\left\{\begin{array}{cc}False\hfill & {\varphi }_i\left({\theta }_{i\setminus t}\right)=0or{\varphi }_i\left({\theta }_{i\setminus t}\right)={\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)\hfill \\ {\varphi }_i\left({\theta }_{i\setminus t}\right)-{\varphi }_c^{\prime }\left({\theta }_{c\setminus t}^{\prime }\right)\hfill & others\hfill \end{array}\right.$$

(24)

where $i{e}_i$ represents the individual fixed-effect of the $i-th$ series data in the the panel data.

2.3. Algorithms

Based on the previous equations, this subsection gives the estimation algorithms (i.e., algorithm 1 and 2) of nonlinear fixed-effects. If there are time-effects or individual fixed-effects, their corresponding expressions are given, or not, false is given correspondingly.

Since each algorithm has been clearly commented, the detailed steps of the algorithm will not be explained separately in this section. If readers have any questions, please contact the corresponding author and request the corresponding codes.

Algorithm 1 fits each series data based on Algorithm 2 and returns the fitting results. If a fitting accepts, it returns the function; otherwise, it returns 0. According to time and other independent variables, the fitting functions are split to distinguish time-effects from individual fixed-effects. And then, the initial coeffect is fitted on the overall panel data. The fitting function of each series data takes the initial coeffect as the starting point and corrects the deviation of their coeffect functions. Finally, the individual fixed-effects and the time-effects are estimated as the differences between the series functions and the coeffect functions.

3. Experimental Results

In this section, to verify the validity of the proposed model, we generate synthetic data on the given functions and use the proposed model to verify the similarity between the estimated functions and the given functions.

3.1. Data Synthesis

Following the equation 7, let $f_c^{\prime }\left(X\right)=0.5x^3-x^2+2x-2$ where the independent variable X is x($x\in [-\gamma ,\gamma ]$ where $\gamma$ is a random integer between 0 and 10 for each series data). Meanwhile, let $g_c\left(t\right)=3t^2-2t+7$ where the time t is t($t\in \{1,2,\cdots ,500\}$). Moreover, for each $i-th$($i\in \{1,2,3,4,5\}$, without loss of generality and easy to visualize, we use Series1, Series2, Series3, and Panel to correspondingly represent the fist, second, third and the whole panel datas.) series data, let $f_i^{\prime }\left(X\right)-f_c^{\prime }\left(X\right)=a{x}^2+bx+c$(i.e., the truth individual fixed-effect of the $i-th$ series data on the panel data) and $g_i\left(t\right)-g_c\left(t\right)=d{t}^2+et+f$(i.e., the truth time-effect of the $i-th$ series data on the panel data) where $a,b,c,d,e,f$ are random numbers in [-3,3],[-7,7],[-5,5],[-4,2],[11,19],[-3,6], correspondingly. Therefore, corresponding to the equation 4, for each $i-th$ series data, $f_i(X,t)=0.5x^3+(a-1)x^2+(b+2)x+(3+d)t^2+(e-2)t+c+f+5$ and $f_c(X,t)=0.5x^3-x^2+2x+3t^2-2t+5$. In order to make the data closer to the real state of reality, random numbers between 0.00003 and 0.01 are added for perturbation to form the residuals of the fitted data. Thus, $f_i(X,t)=0.5x^3+(a-1)x^2+(b+2)x+(3+d)t^2+(e-2)t+c+f+5+{\epsilon }_{i,t}$ where ${\epsilon }_{i,t}$ is the random residual.

If fixed-effects are estimated using the classical model, the results are shown in Figure 1. It can be seen from Figure 1 that the classical model considers that there are both time-effect (Time) and individual fixed-effect (Entity) on a given data set. However, by observing its parameters, it can be seen that it is 832X+13210, which is far from the real value. Therefore, it can be argued that the existence of individual fixed-effects on a given data set should not be accepted.

It can be seen from Figure 2 that the representative series data show obvious characteristics of third-order polynomial distribution affected by x, as well as the overall data distribution. However, on the overall data, it is difficult to find a common third-order polynomial distribution function to fit them. Therefore, it is reasonable to assume that there are individual fixed-effects in each series data.

Similarly, it can be seen from Figure 3 that, affected by time t, the representative series data show obvious characteristics of second-order polynomial distribution, as well as the overall data distribution. However, on the overall data, it is difficult to find a common second-order polynomial distribution function to fit them. Therefore, it is reasonable to consider the existence of time-effect in each series data.

3.2. Data Distribution Observation

Figure 4 shows that it is difficult to observe how x alone affects the distribution of y when the effects of x and t on the data are mixed. In fact, it is equally difficult to observe the distributional impact of t on y. Similarly, in the subsequent results, it can also be found that it is very difficult to observe the influence of x and t on y at the same time.

3.3. Estimate Results

Fortunately, with the help of expert knowledge, it is assumed that x and y may satisfy a third-order polynomial distribution, and t and y may satisfy a second-order polynomial distribution, and they are independent of each other. Based on Algorithm 1, time-effects and individual fixed-effects are effectively estimated, and the results are shown in Table 1. At the same time, Table 1 also presents the estimation results based on the classical model. Obviously, the method in this paper is significantly better than the classical method.

Perhaps the results given in Table 1 are not intuitive enough. Figure 5 shows the superiority of the method in this paper. It can be seen that the function fitting effects are particularly good on the three series datas, and the corresponding data points can be almost covered seamlessly. At the same time, the panel subfigure of Figure 5 shows that the estimated function can cover most of the data points, but not completely, which is the distribution adaptation with the synthetic data, precisely reflecting the heterogeneity among the serial datas.

Figure 6 shows the fitting effects of the classical method on the three series data, showing that the fitting is not good and it is difficult to meet users’ expectations. Similarly, the fitting effect on the panel subgraph is not good. On the surface, heterogeneity among series data can be obtained. However, since the data fittings on the first three subgraphs are not effective, it is possible that this heterogeneity is caused by random distribution and should not be accepted.

4. Discussion and Conclusions

4.1. Discussion

Experiments show that, relying on expert knowledge and curve fitting, compared with the classical methods, the proposed method has better results in data fitting, and can better obtain the time-effects and individual fixed-effects on the synthetic data set, which is close to the real values. It can be seen that the result of fixed-effect verification depends on the accuracy of expert knowledge, and the higher the accuracy, the better the fitting effect. At the same time, the fitting algorithm also affects the results of verification, especially the new function fitting method can extend the data-based curve fitting to function fitting, and significantly improve the applicability of the algorithm. In addition, we also verify the results when the definition of fixed-effect is not met, and find that the classical method may incorrectly accept the existence of fixed-effect, while our method can better avoid this misjudgment.

4.2. Conclusions

In view of the fact that the classical models of fixed-effect analysis are mainly based on linear and logistic regression, without paying attention to the ubiquitous linear relationship, this paper proposes a nonlinear fixed-effect analysis model based on the least square method. Firstly, the model determines the type of data distribution (polynomial distribution or power distribution, etc.) based on expert knowledge, and gives the preliminary distribution function of each series data. Next, the parameters are effectively estimated by fitting the data based on the initial distribution function through the least squares method. Then, based on the loss function of least square method, the objective function of function fitting of multiple fitting functions is designed, and the fitted function is obtained. The differences between each series function and the fitted function are expressed as functions of the fixed-effects. If time and other independent variables are independent in the fitting function, then the fitting function is split according to time and other variables, and then the time-effect and individual fixed-effect are obtained by using similar methods. On the synthetic data set, the effect of our method is compared with that of the classical method. It is found that our method has significant advantages and can achieve the expected results. In addition, we found in the experiment that the fixed-effect can be rejected for the data that do not meet the fixed-effect.

4.3. Future Works

First of all, in the analysis of time-effect and individual fixed-effect, this paper assumes that time and other independent variables are independent of each other, while in reality they may be interdependent. Thus, relaxing this assumption is the focus of the future research. Secondly, there may be some problems in the judgment of complex data by expert knowledge. It is valueable to analyze how to minimize the unsatisfactory effect of data fitting in the case of lack of expert knowledge. Lastly, variable selection in nonlinear fixed-effect analysis and cross-period dynamic analysis are also directions for future consideration.