Bootstrap-Based Initialization for Maximum Likelihood Estimation in The Infinite Mixture Distributions and Its Application to Automobile Insurance Claims

1. Introduction

An infinite mixture distribution is an extension of the finite mixture model where the number of components is not fixed in advance and can, in principle, be countably infinite. Instead of a finite sum of distributions, the mixture becomes an infinite sum or an integral over a space of possible parameters (Klugman et al., 2019). In actuarial science, the infinite mixture distributions are used in credibility theory and bonus malus systems to model claim frequency and claim severity (Herzog, 2010; Lemaire, 1995). The infinite mixture distributions have advantages over standard distributions, namely being able to capture complex data structures and data with high heterogeneity. Such data are typically multimodal, skewed, or heavy-tailed in the case of continuous data (Ghosal & van der Vaart, 2001), and the variance of the data is larger than expected by standard distribution models, especially the Poisson model (Canale & Prünster, 2017; Wade & Ghahramani, 2018).

Several studies have discussed distributions that are included in the infinite mixture distributions for discrete and continuous cases. Some examples of distributions that are included in discrete distributions are the Poisson-Lindley distribution (Sankaran, 1970); the Poisson-lognormal distribution (Bulmer, 1974); the Poisson-gamma distribution, the Poisson-inverse Gaussian distribution, the Poisson-generalized inverse Gaussian distribution, the Poisson-beta distribution, the Poisson-uniform distribution (G. Willmot, 1986; G. E. Willmot, 1987); the negative binomial-Lindley distribution (Zamani, 2010); the Poisson-weighted Lindley distribution (Abd El-Monsef & Sohsah, 2014); the Poisson-Sujatha distribution (Shanker, 2016b); the Poisson-Amarendra distribution (Shanker, 2016a); the Poisson-Aradhana distribution (Shanker, 2017); the New Poisson mixed weighted Lindley distribution (Atikankul et al., 2020). Some examples of distributions that are included in continuous distributions are the inverse exponential-gamma distribution (Anantasopon et al., 2015), the lognormal-gamma distribution (Moumeesri et al., 2020), the exponential-gamma distribution (McNulty, 2021), and the Rayleigh-Rayleigh distribution (Jaroengeratikun et al., 2022).

The Maximum likelihood estimation method of the parameters of the infinite mixture distributions often requires using numerical methods such as the Newton-Raphson method because no analytical solution is obtained. The problem in the Newton-Raphson method is determining the initial values. In several studies (Anantasopon et al., 2015; McNulty, 2021; Moumeesri et al., 2020; G. Willmot, 1986), determining the initial parameter values for the maximum likelihood estimator (MLE) is not mentioned or recommended even though it is a very important step. Some studies use other parameter estimation methods as initial values, for example the method of moments (Abd El-Monsef & Sohsah, 2014; Jaroengeratikun et al., 2022; Sankaran, 1970; Shanker, 2016b, 2016a, 2017). However, sometimes the use of the moment method does not provide an analytical solution (Atikankul et al., 2020; Zamani, 2010). Therefore, a method is needed to determine the initial parameter values for the MLE of the infinite mixture distributions so the numerical method can be run.

This paper discusses determining the initial parameter values based on the bootstrap approach for the MLE of the infinite mixture distributions. The bootstrap approach is used to generate the mixing distribution. The parameter estimates from the mixing distribution are used as the initial parameter values for the MLE of the infinite mixture distributions. Both nonparametric and parametric bootstrap approaches will be used in this paper. Monte Carlo simulations are used to assess the performance of both bootstrap approaches. The Indonesian automobile insurance claims data are used to illustrate the proposed method. The rest of this paper is arranged as follows. In Section 2, the theory of the infinite mixture distributions is discussed, along with several examples of distributions. In Section 3, nonparametric and parametric bootstrap approaches are proposed as ways to obtain the initial parameter values for the MLE of the infinite mixture distributions. The simulation study is conducted in Section 4. Section 5 discusses the application of the proposed method. The discussion and conclusion are presented in Section 6.

2. The Infinite Mixture Distributions

In statistics, creating new distributions can be done in various ways, and one important method is through the concept of mixing. Mixing is the process of forming a new distribution by combining two or more distributions, usually by applying weights. The result of this process is called a mixture distribution. An infinite mixture distribution is a type of probability distribution created by mixing (i.e., combining) an infinite number of component distributions. An infinite mixture distribution can be written as:

$$f_X\left(x\right)=\int f_{X|\mathsf{\Lambda }}\left(x|\lambda \right)f_{\mathsf{\Lambda }}\left(\lambda \right)d\lambda$$

(1)

where the integral is taken over all values of λ with positive probability, $f_{X|\mathsf{\Lambda }}\left(x|\lambda \right)$ is the probability density function (pdf) or probability mass function (pmf) of the random variable $X$ conditional on $\mathsf{\Lambda }$, and $f_{\mathsf{\Lambda }}\left(\lambda \right)$ is the pdf of the random variable Λ known as the mixing distribution (Klugman et al., 2019). The distribution function can be determined from

$$F_X\left(x\right)=\int F_{X|\mathsf{\Lambda }}\left(x|\lambda \right)f_{\mathsf{\Lambda }}\left(\lambda \right)d\lambda$$

(2)

Moments of the infinite mixture distribution can be found from

$$\mathrm{E}\left(X^k\right)=\mathrm{E}\left[\mathrm{E}\left(X^k|\mathsf{\Lambda }\right)\right]$$

(3)

and, in particular,

$$\mathrm{V}\mathrm{a}\mathrm{r}\left(X\right)=\mathrm{E}\left[\mathrm{V}\mathrm{a}\mathrm{r}\left(X|\mathsf{\Lambda }\right)\right]+\mathrm{V}\mathrm{a}\mathrm{r}\left[\mathrm{E}\left(X|\mathsf{\Lambda }\right)\right]$$

(4)

In this section, we discuss 3 distributions that include in the infinite mixture distributions, namely the Poisson-Lindley distribution (Sankaran, 1970), the Inverse exponential-gamma distribution (Anantasopon et al., 2015), the lognormal-gamma distribution (Moumeesri et al., 2020).

2.1. Poisson-Lindley Distribution

The Poisson–Lindley distribution is a discrete probability distribution that arises as a mixed Poisson distribution, where the Poisson parameter $\lambda$ follows a Lindley distribution. Let $X|\lambda$ be a random variable with a Poisson distribution with parameter $\lambda$, and the Poisson parameter $\lambda$ follows a Lindley distribution with parameter $\theta$. Then, the marginal distribution of $X$ is called the Poisson–Lindley distribution, with the pmf (Sankaran, 1970):

$$p_X\left(x\right)={\displaystyle \frac{{\theta }^2\left(\theta +2+x\right)}{{\left(\theta +1\right)}^{x+3}}};x=0,1,2,\dots ;\theta >0$$

(5)

The Poisson–Lindley distribution is used to model automobile insurance claim frequency data to calculate the Bayesian premium (Moumeesri et al., 2020). Suppose that $X_1$, $X_2$, $\dots$, $X_n$ is a random sample from the Poisson–Lindley distribution. Let $x_1$, $x_2$, $\dots$, $x_n$ is a realization from the random sample. The parameter estimate of the Poisson-Lindley distribution, θ, using the method of moments is (Sankaran, 1970)

$$\tilde{\theta }={\displaystyle \frac{-\left(\overline{x}-1\right)+\sqrt{{\left(\overline{x}-1\right)}^2+8\overline{x}}}{2\overline{x}}}$$

(6)

where $\overline{x}=\sum _{i=1}^n{x}_i/n$. The parameter estimate of the Poisson-Lindley distribution, $\theta$, using the maximum likelihood estimation method is the solution of the following equation (Sankaran, 1970)

$${\displaystyle \frac{2n}{\theta }}-{\displaystyle \frac{n\left(\overline{x}+3\right)}{\theta +1}}+\sum _{i=1}^n{\displaystyle \frac{1}{x_i+\theta +2}}=0$$

(7)

Since the estimation of the parameter cannot be found in closed form, numerical method such as the Newton-Raphson method can be used to solve the above equation. The estimated parameter value through the method of moments can be used as initial value for the Newton-Raphson method.

2.2. Inverse Exponential-Gamma Distribution

Let $X|\lambda$ be a random variable with an inverse exponential distribution with parameter $\lambda$, and the inverse exponential parameter $\lambda$ follows a gamma distribution with parameters $\alpha$, and $\beta$. Then, the marginal distribution of $X$ is called the inverse exponential-gamma distribution, with the pdf (Anantasopon et al., 2015)

$$f_X\left(x\right)={\displaystyle \frac{\alpha {\beta }^{\alpha }{x}^{\alpha -1}}{{\left(1+\beta x\right)}^{\alpha +1}}};x>0;\alpha >0;\beta >0$$

(8)

the inverse exponential-gamma distribution is used to model motor insurance claim (Anantasopon et al., 2015). There are no $k$-th raw moments ($k$ is a positive integer) for this distribution (Klugman et al., 2019). The estimation of the parameters of the inverse exponential-gamma distribution, $\alpha$, and $\beta$, using the maximum likelihood estimation method is the solution of the following two equations (Anantasopon et al., 2015)

$${\displaystyle \frac{n}{\alpha }}+n\mathrm{l}\mathrm{n}\left(\beta \right)+\sum _{i=1}^n\mathrm{l}\mathrm{n}\left(x_i\right)-\sum _{i=1}^n\ln \left(1+\beta x_i\right)=0$$

(9)

$${\displaystyle \frac{n\alpha }{\beta }}-\left(\alpha +1\right)\sum _{i=1}^n{\displaystyle \frac{x_i}{1+\beta x_i}}=0$$

(10)

Because closed-form solutions for the parameter estimates are unavailable, the two equations above can be solved using numerical methods, for example, the Newton–Raphson method. In Anantasopon et al. (2015), there is no proposal to use initial values for the estimated parameters of the inverse exponential-gamma distribution via the Newton-Raphson method.

2.3. Lognormal-Gamma Distribution

Let $X|\lambda$ be a random variable with a lognormal distribution with parameters $\mu$, and $\lambda =1/{\sigma }^2$, and the lognormal parameter $\lambda$ follows a gamma distribution with parameters $\tau$, and $\eta$. Then, the marginal distribution of $X$ is called the lognormal-gamma distribution, with the pdf (Moumeesri et al., 2020)

$$f_X\left(x\right)={\displaystyle \frac{{\eta }^{\tau }}{x\sqrt{2\pi }\mathsf{\Gamma }\left(\tau \right)}}{\displaystyle \frac{\mathsf{\Gamma }\left(\tau +\frac{1}{2}\right)}{{\left[\eta +\frac{1}{2}{\left(\ln x-\mu \right)}^2\right]}^{\tau +\frac{1}{2}}}};x>0;\tau >0;\eta >0;-\infty <\mu <\infty$$

(11)

In Moumeesri et al. (2020), the lognormal-gamma distribution is used to model automobile insurance claim severity data to calculate the Bayesian premium. The log-Cauchy distribution is a special case of the lognormal-gamma distribution for $\tau =1/2$with the pdf

$$f_X\left(x\right)={\displaystyle \frac{1}{x\pi }}{\displaystyle \frac{{\left(2\eta \right)}^{1/2}}{\left[{\left(\ln x-\mu \right)}^2+2\eta \right]}};x>0;\eta >0;-\infty <\mu <\infty$$

(12)

The log-Cauchy distribution does not have defined moments. Therefore, the method of moments cannot be used to estimate the parameters of this distribution. The estimation of the parameters of the log-Cauchy distribution, $\eta$ and $\mu$, using the maximum likelihood estimation method is the solution of the following two equations

$${\displaystyle \frac{n}{2\eta }}-2\sum _{i=1}^n{\displaystyle \frac{1}{\left[{\left(\ln x_i-\mu \right)}^2+2\eta \right]}}=0$$

(13)

$$2\sum _{i=1}^n{\displaystyle \frac{\left(\ln x_i-\mu \right)}{\left[{\left(\ln x_i-\mu \right)}^2+2\eta \right]}}=0$$

(14)

Since closed-form expressions for the parameters do not exist, numerical approaches like the Newton–Raphson method may be applied to solve the two equations above.

3. Bootstrap-Based Initialization

In this section, a method is proposed to determine the initial parameter values for the MLE of the infinite mixture distributions using the bootstrap approach. Two bootstrap approaches, namely nonparametric and parametric bootstrap are used in this method.

Suppose that a dataset (original data) of size n, denoted as $\mathit{x}=\left\{x_1,x_2,\cdots ,x_n\right\}$. The steps in determining the initial parameter values for the MLE of the infinite mixture distribution using the nonparametric bootstrap approach are as follows

• Randomly draw a new sample of size n with replacement from the original data to create a bootstrap sample, ${\mathit{x}}_1^{*}$;
• Calculate the estimated value of the parameter, $\lambda$, which in the theory of the infinite mixture distribution (Section 2) is distributed with the pdf or pmf $f_{X|\mathsf{\Lambda }}\left(x|\lambda \right)$ for the bootstrap sample, denoted as ${\lambda }_1^{*}=s\left({\mathit{x}}_1^{*}\right)$;
• Repeat steps 1-2 $B$ times to get ${\lambda }_1^{*}$, ${\lambda }_2^{*}$, $\cdots$, ${\lambda }_B^{*}$.
• Calculate the estimated values of the parameters of the mixing distribution based on the data ${\lambda }_1^{*}$, ${\lambda }_2^{*}$, $\cdots$, ${\lambda }_B^{*}$. The estimated values of these parameters become the initial parameter values for the MLE of the infinite mixture distribution.

The steps in determining the initial parameter values for the MLE of the infinite mixture distribution using the parametric bootstrap approach are as follows

• Calculate the estimated value of the parameter, $\lambda$, using maximum likelihood estimation method which is distributed with the pdf or pmf $f_{X|\mathsf{\Lambda }}\left(x|\lambda \right)$ for the original data, denoted as $\widehat{\lambda }=g\left(\mathit{x}\right)$;
• Randomly draw a new sample of size n from the distribution $f_{X|\mathsf{\Lambda }}\left(x|\widehat{\lambda }\right)$ to create a bootstrap sample, ${\mathit{x}}_1^{*}$;
• Calculate the estimated value of the parameter, $\lambda$, which in the theory of the infinite mixture distribution (Section 2) is distributed with the pdf or pmf $f_{X|\mathsf{\Lambda }}\left(x|\lambda \right)$ for the bootstrap sample, denoted as ${\lambda }_1^{*}=s\left({\mathit{x}}_1^{*}\right)$;
• Repeat steps 2-3 $B$ times to get ${\lambda }_1^{*}$, ${\lambda }_2^{*}$, $\cdots$, ${\lambda }_B^{*}$.
• Calculate the estimated values of the parameters of the mixing distribution based on the data ${\lambda }_1^{*}$, ${\lambda }_2^{*}$, $\cdots$, ${\lambda }_B^{*}$. The estimated values of these parameters become the initial parameter values for the MLE of the infinite mixture distribution.

4. Simulation Study

In this section, a simulation study is conducted to evaluate the performance of bootstrap-based initialization in determining the initial parameter values for the MLE of the infinite mixture distribution. The two bootstrap approaches proposed in Section 3 will be evaluated for their performance along with the method of moments. The three distributions discussed in Section 2 will be used to evaluate the performance of the proposed method. Several cases from these distributions will be applied in the simulation study. These cases are presented in Table 1. There are 9 cases of the Poisson-Lindley distribution, 18 cases of the inverse exponential gamma distribution, and 18 cases of the lognormal-gamma distribution.

The results of the simulation study of all cases in Table 1 are shown in Table 2, Table 3 and Table 4 with a simulation number of 1,000 times and B = 1,000 bootstrap replications. There are two methods for determining the initial parameter values for the MLE of the infinite mixture distributions whose performance is assessed in this simulation study, namely the method of moments (MM) and bootstrap-based methods, namely nonparametric bootstrap (NP) and parametric bootstrap (P). The performance of the two methods is assessed based on the average number of iterations and the root mean square error (RMSE) of the MLE. It is known that RMSE includes the variance and bias of the parameter estimator.

In this section, a simulation study is conducted to evaluate the performance of bootstrap-based initialization in determining the initial parameter values for the MLE of the infinite mixture distribution. The two bootstrap approaches proposed in section 3 will be evaluated for their performance along with the method of moments. The three distributions discussed in section 2 will be used to evaluate the performance of the proposed method. Several cases from these distributions will be applied in the simulation study. These cases are presented in Table 1. There are 9 cases of the Poisson-Lindley distribution, 18 cases of the inverse exponential gamma distribution, and 18 cases of the lognormal-gamma distribution.

Table 3 presents the simulation study results of the cases of the inverse exponential-gamma distribution. Only bootstrap-based methods were evaluated for their performance in the cases of the inverse exponential-gamma distribution. It can be seen that the average number of iterations is all below 30 iterations for both bootstrap-based approaches (NP, and P) and for all distribution cases. Another result is that the average number of iterations of the nonparametric bootstrap (NP) approach is less than that of the parametric bootstrap (P) approach. The difference in the results of the two approaches ranges from 2 to 12 iterations. This indicates that the nonparametric bootstrap approach has better performance than the parametric bootstrap approach as an approach in determining the initial parameter values for the MLE of the infinite mixture distributions. Another result is that all RMSE values are the same for both bootstrap-based approaches (NP, and P) and for all distribution cases. This can be concluded that bootstrap-based approaches produce the same MLE for the cases of the inverse exponential-gamma distribution.

Table 4 presents the simulation study results of the cases of the lognormal-gamma distribution. Only bootstrap-based methods were evaluated for their performance in the cases of the lognormal-gamma distribution. It can be seen that the average number of iterations is all below 25 iterations and does not differ much for both bootstrap-based approaches (NP, and P) and for all distribution cases. This shows that both bootstrap-based approaches have equally good performance as methods for determining the initial parameter values for the MLE of the infinite mixture distributions for the cases of the lognormal-gamma distribution. Another result is that all RMSE values are the same for both bootstrap-based approaches (NP, and P) and for all distribution cases. This can be concluded that bootstrap-based approaches produce the same MLE for the cases of the lognormal-gamma distribution.

5. Applications

We consider the application of the proposed method to two automobile insurance claim datasets in Indonesia for policyholders who took comprehensive coverage with expanded third-party liability in underwriting year 2019. The first dataset contains the claim frequency of the third-party liability, as presented in Table 5. The sample mean and variance for the claim frequency data are 0.00400, and 0.00402, respectively. The second dataset contains the claim severity of the third-party liability, which has 226 observations. The sample mean and standard deviation for the claim severity data are IDR 5,760,172, and IDR 7,162,913, respectively. Figure 1 shows a histogram for the claim severity data.

The Poisson-Lindley distribution will be applied to model claim frequency data of the third-party liability. Meanwhile, the inverse exponential-gamma distribution, and the lognormal-gamma distribution will be applied to model claim severity data of the third-party liability. The parameters of the three distributions will be estimated using the maximum likelihood estimation method through the Newton-Raphson numerical method with the initial values of the estimated distribution parameters determined using the bootstrap-based method proposed in this paper.

The bootstrap approach steps in Section 3 are performed on the claim frequency data of the third-party liability to obtain initial parameter estimates using both nonparametric and parametric bootstrap approaches. The results were 250.7964 and 250.5977, respectively. These initial values were used to obtain the MLE of the Poisson-Lindley distribution using the Newton-Raphson method. The results were the same for both approaches, namely 250.9390 at iteration 3. Table 5 presents claim frequency data (observed frequency) of the third-party liability, estimated probability for Poisson-Lindley distribution and expected frequency. The values in Table 5 are needed to calculate the chi-square test statistic to test the suitability of the Poisson-Lindley distribution. The result is a chi-square test statistic of 0.0120 with a p-value of 0.9128. Thus, it can be concluded that the Poisson-Lindley distribution is suitable for modeling claim frequency data of third-party liability.

Table 6 presents the results of applying the method proposed in Section 3 to claim severity data of the third-party liability as well as the results of the Kolmogorov-Smirnov test for the inverse exponential-gamma (IEG) distribution and the lognormal-gamma (LG) distribution. It can be seen that the initial parameter values of the MLE for the inverse exponential-gamma distribution are ${\widehat{\alpha }}_0=169.7453$, and ${\widehat{\beta }}_0=8.8832\times {10}^{-5}$ for the nonparametric bootstrap approach, and ${\widehat{\alpha }}_0=216.8241$, and ${\widehat{\beta }}_0=1.1342\times {10}^{-4}$ for the parametric bootstrap approach. The values of the MLE for the inverse exponential-gamma distribution are $\widehat{\alpha }=5.9356$, and $\widehat{\beta }=2.6170\times {10}^{-6}$ for both approaches. For the nonparametric bootstrap approach, it was obtained at iteration 30, while for the parametric bootstrap approach, it was obtained at iteration 54. The results for the lognormal-gamma distribution can be seen in the last column of Table 6.

The K-S test statistic values for the inverse exponential-gamma distribution and the lognormal-gamma distribution are 0.0951 and 0.0973, respectively. The critical value of the K-S test at the 1% significance level is 0.1011 (Sheskin, 2000). Thus, the inverse exponential-gamma distribution and the lognormal-gamma distribution are suitable for modeling claim severity data of the third-party liability. Figure 2 depicts the empirical cumulative distribution function (EDF) curve, the cumulative distribution function curve for the inverse exponential-gamma distribution (IEG), and the lognormal-gamma distribution (LG). It can be seen that the cumulative distribution function curve for the inverse exponential-gamma distribution is closer to the EDF. Thus, the inverse exponential-gamma distribution is more suitable for modeling claim severity data of the third-party liability than the lognormal-gamma distribution. This is supported by the smallest K-S test statistic value and the largest log-likelihood function value for the inverse exponential-gamma distribution.

6. Discussion and Conclusions

In this paper, a bootstrap-based method for determining initial values in numerical methods to obtain maximum likelihood estimators in the infinite mixture distributions is proposed. Two bootstrap approaches were used, namely nonparametric bootstrap and parametric bootstrap. Detailed steps for the two bootstrap-based approaches are outlined. This proposed method is very useful especially when faced with the problem of estimating parameters through numerical methods where there are no recommended initial values. This may occur when the moments of the distribution are not defined or there is no analytical solution for estimation using the method of moments.

In simulation study, the problem discussed is the performance assessment of the initial value determination method, not the performance of the parameter estimation method. Therefore, for further research, it is necessary to assess the performance of parameter estimation methods for distributions included in the infinite mixture distribution. The method proposed in this paper can be an alternative method for estimating distribution parameters besides existing methods. In simulation studies, we sometimes encounter situations where the iteration process of the numerical method fails to converge. This is because the method proposed in this paper is based on random data generation.

The results of the simulation study show that the method of moments and two bootstrap-based approaches produce the same maximum likelihood estimator. Another result is that when faced with the problem of no raw moments or undefined moments, two bootstrap-based approaches, especially nonparametric bootstrap, become a reliable solution as a method for determining initial values. The application of the proposed method to the third-party liability claims data for automobile insurance in Indonesia for underwriting year 2019 showed good performance.