Some Matrix-variate Models Applicable in Different Areas

1. Introduction

The following notations will be used in this paper: Real scalar variables, whether mathematical or random, will be denoted by lower-case letters such as $x,y$. Real vector/matrix variables, whether mathematical or random, whether square matrices or rectangular matrices, will be denoted by capital letters such as $X,Y$. Scalar constants will be denoted by lower-case letters such as $a,b$ and vector/matrix constants will be denoted by $A,B$ etc. A tilde will be used to designate variables in the complex domain such as $\tilde{x},\tilde{y},\tilde{X},\tilde{Y}$. No tilde will be used on constants. When Greek letters and other symbols appear, the notations will be explained then and there. Let $X=\left(x_{ij}\right)$ be a $p\times q$ matrix where the elements are functionally independent or distinct real scalar variables. Then, the wedge product of differentials will be defined as $\mathrm{d}X={\wedge }_{i=1}^p{\wedge }_{j=1}^q\mathrm{d}{x}_{ij}$. When x and y are real scalars, then the wedge product of their differentials is defined as $\mathrm{d}x\wedge \mathrm{d}y=-\mathrm{d}y\wedge \mathrm{d}x$ so that $\mathrm{d}x\wedge \mathrm{d}x=0,\mathrm{d}y\wedge \mathrm{d}y=0$. For a square matrix A the determinant will be denoted as $\left|A\right|$ or as $\det \left(A\right)$. When A is in the complex domain, then the absolute value of the determinant or modulus of the determinant will be denoted as $\left|\det \right(A\left)\right|$. If $\left|A\right|=a+ib,i=\sqrt{(-1)},a,b$ real scalars, then $\left|\det \left(A\right)\right|=\sqrt{(a^2+b^2)}$. If $\tilde{X}$ is in the complex domain, then one can write $\tilde{X}=X_1+i{X}_2,i=\sqrt{(-1)},X_1,X_2$ real, then the wedge product of differentials in $\tilde{X}$ will be defined as $\mathrm{d}\tilde{X}=\mathrm{d}{X}_1\wedge \mathrm{d}{X}_2$. We will consider only real-valued scalar functions in this paper. ${\int }_{X}f\left(X\right)\mathrm{d}X$ will denote integral over X of the real-valued scalar function $f\left(X\right)$ of X. When $f\left(X\right)$ is a real-valued scalar function of X, whether X is scalar or vector or matrix in the real or complex domain, and if $f\left(X\right)\ge 0$ for all X and ${\int }_{X}f\left(X\right)\mathrm{d}X=1$, then $f\left(X\right)$ will be defined as a density or statistical density. When a square matrix X is positive definite then it will be denoted as $X>O$ where $X=X^{\prime }$, a prime denoting the transpose. Conjugate transpose of any matrix $\tilde{Y}$ in the complex domain will be written as ${\tilde{Y}}^{*}$. When a square matrix $\tilde{X}$ is in the complex domain and if $\tilde{X}={\tilde{X}}^{*}$ then $\tilde{X}$ is Hermitian. If $\tilde{X}={\tilde{X}}^{*}>O$ then $\tilde{X}$ is called Hermitian positive definite. When $Y>O$, then ${\int }_A^{B}f\left(X\right)\mathrm{d}X$ means the integral of the real-valued scalar function $f\left(X\right)$ over the real positive definite matrix X such that $X>O,A>O,B>O,X-A>O,B-X>O$ (all positive definite), where $A>O$ and $B>O$ are constant matrices, and similar notation and interpretation in the complex domain also. In order to avoid multiplicity of numbers, the following procedure will be used. For a function number or equation number in the complex domain, corresponding to the same in the real domain, a letter c will be affixed to the function number and section number of the equation number. For example $f_{1c}\left(\tilde{X}\right)$ will correspond to $f_1\left(X\right)$ in the real domain and equation number $(2c.5)$ will correspond to $\left(2.5\right)$ in the real domain. This notation will enable a reader to recognize a function or equation in the complex domain instantly by recognizing the subscript c. Other notations will be explained whenever they occur for the first time.

Matrix-variate statistical distributions are widely used in all types of disciplines such as Statistics, Physics, Communication Theory, Engineering problems. A matrix-variate density where a trace with an exponent enters into the density as a product and when the exponential trace has an arbitrary power, known as Kotz’ model in the literature, is widely used in the analysis of data coming from various areas such as multi-look return signals in radar and sonar, see for example, [2] regarding the analysis of PolSAR (Polarimetric Synthetic Aperture Radar) data. Kotz’ model is a generalization of the basic matrix-variate Gaussian model or it can also be considered as a generalization of matrix-variate gamma model or Wishart model. When analysing radar data, it is found that Gaussian-based models fit well when the surface is disturbance-free. It is found that Gaussian-based models are not appropriate in certain regions such as urban area, sea surface, forests etc, see for example, [3–6]. Hence, we will consider some non-Gaussian or non-Wishart models also in this paper, along with Gaussian-based models. In most of the applications in engineering areas, each scalar variable has two components, such as time and phase, so that a complex variable is very appropriate to represent such a scalar variable. Hence, it is found that distributions in the complex domain are more important in applications in physical sciences and engineering areas. When a statistical density is used in any applied problem, computation of the normalizing constant there is the most important step because when studying all sorts of properties of such a model, the computations naturally follow the format of the evaluation of the normalizing constant in that model. Explicit evaluation of the normalizing constant in a general model, often referred to also as a Kotz’ model in the real domain, does not seem to be available in the literature. Normalizing constant in the general model in the real domain appearing in [7], which the authors claim to have been available elsewhere in earlier literature, seems to be the one widely used in all the applications where Kotz’ model in the real domain is used. But, unfortunately the normalizing constant quoted in [7] does not seem to be correct. Kotz’ type model in the complex domain does not seem to be available in the literature and the normalizing constant therein does not seem to be available also. Hence, one of the aims of this paper is to give the derivation of the normalizing constant in the general model in detail, in the real and complex domains, and also to extend the ideas to Mathai’s pathway family [1], namely matrix-variate gamma, type-1 beta and type-2 beta families of densities. Since the derivation of the normalizing constant is the most important step in the construction of any statistical model, various matrix-variate models are listed in this paper by showing the computations of the normalizing constants in each case. Some applications Kotz’ model in the real domain may be seen from [8–11].

This paper is organized as follows: Section 1 contains the introductory material. Section 2 gives explicit evaluation of the normalizing constant in an extended matrix-variate gamma type or Gaussian type or Wishart type or Kotz type model, both in the real and complex domains, and then deals with multivariate and matrix-variate extended Gaussian and gamma type distributions. Section 3 examines extended matrix-variate type-2 beta type models in the real and complex domains. Section 4 contains extended matrix-variate models of the type-1 beta type in the real and complex domains. Throughout the paper, the results in the complex domain are listed side by side with the corresponding results in the real domain. Detailed derivations are done for the real domain cases only, since most of the steps in the complex domain are parallel to those in the real domain.

2. Evaluation of Some Matrix-variate Integrals and the Resulting Models

Let us start with an example of the evaluation of an integral in the real domain which will show the different types of hurdles to overcome to achieve the final result. Let $X=\left(x_{ij}\right)$ be a $p\times q,p\le q$ matrix of rank p where the $pq$ elements $x_{ij}$’s are functionally independent (distinct) real scalar variables. Suppose that we wish to evaluate the following integral, where $f\left(X\right)$ is a real-valued scalar function of the $p\times q$ matrix X, integral over X and the wedge product of differentials $\mathrm{d}X$ are already explained in Section 1:

$$\begin{array}{cc}\hfill & {\int }_{X}f\left(X\right)\mathrm{d}X\hfill \\ \hfill & =c{\int }_X{\left|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right|}^{\gamma }{\left[\mathrm{tr}\left(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right)\right]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right)\right]}^{\delta }}\mathrm{d}X\hfill \end{array}$$

(2.1)

where $\delta >0,M=E\left[X\right],\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},A>O$ is $p\times p$ and $B>O$ is $q\times q$ positive definite constant matrices, $A^{\frac{1}{2}}$ is the positive definite square root of the positive definite matrix $A>O$, $E[·]$ denotes the expected value of $[·]$ and $\Re (·)$ means the real part of $(·)$. The first step here is to simplify the matrix $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ into a convenient form by making a transformation $Y=A^{\frac{1}{2}}(X-M)B^{\frac{1}{2}}\Rightarrow \mathrm{d}Y={\left|A\right|}^{\frac{q}{2}}{\left|B\right|}^{\frac{p}{2}}\mathrm{d}X$ from the Lemma 2.1 given below by observing that $\mathrm{d}(X-M)=\mathrm{d}X$ since M is a constant matrix. The corresponding integral in the complex domain is the following:

$$\begin{array}{c}\hfill {\int }_{\tilde{X}}{f}_c\left(\tilde{X}\right)\mathrm{d}\tilde{X}\\ \hfill & =\tilde{c}{\int }_X{\left|\det \left(A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}}\right)\right]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}}\right)\right]}^{\delta }}\mathrm{d}\tilde{X}\hfill \end{array}$$

(2c.1)

where $A=A^{*}>O,B=B^{*}>O$ (both Hermitian positive definite), A is $p\times p$, B is $q\times q$ and $\tilde{M}=\tilde{X}$. The transformation in the complex case is $\tilde{Y}=A^{\frac{1}{2}}(\tilde{X}-\tilde{M})B^{\frac{1}{2}}\Rightarrow \mathrm{d}\tilde{Y}=|\det \left(A\right){|}^q{\left|\det \left(B\right)\right|}^p\mathrm{d}\tilde{X}$.

Lemma 2.1. 

Let the $m\times n$ matrix $X=\left(x_{ij}\right)$ be in the real domain where the $mn$ elements $x_{ij}$’s are functionally independent (distinct) real scalar variables and let A be a $p\times p$ and B be a $q\times q$ nonsingular constant matrices. Then,

$$Y=AXB,\left|A\right|\ne 0,\left|B\right|\ne 0\Rightarrow \mathrm{d}Y={\left|A\right|}^n{\left|B\right|}^m\mathrm{d}X.$$

Let the $m\times n$ matrix $\tilde{X}$ be in the complex domain and where A and B be $p\times p$ and $q\times q$ nonsingular constant matrices respectively in the real or complex domain, then

$$\begin{array}{cc}\hfill \tilde{Y}& =A\tilde{X}B,\left|A\right|\ne 0,\left|B\right|\ne 0\Rightarrow \hfill \\ \hfill \mathrm{d}\tilde{Y}& ={\left|\det \left(A\right)\right|}^{2q}{\left|\det \left(B\right)\right|}^{2p}\mathrm{d}\tilde{X}=|\det \left(A{A}^{*}\right){|}^q{\left|\det \left(B^{*}B\right)\right|}^p\mathrm{d}\tilde{X}\hfill \end{array}$$

where $\left|\det \right(·\left)\right|$ denotes the absolute value of the determinant of $(·)$.

The proof of Lemma 2.1 and other lemmas to follow, may be seen from [12]. When a $p\times p$ matrix X is symmetric, $X=X^{\prime }$ then we have a companion result to Lemma 2.1 which will be stated next.

Lemma 2.2. 

Let $X=X^{\prime }$ be a symmetric $m\times m$ matrix and let A be a $m\times m$ constant nonsingular matrix. Then,

$$Y=AX{A}^{\prime },\left|A\right|\ne 0,\Rightarrow \mathrm{d}Y={\left|A\right|}^{p+1}\mathrm{d}X$$

and when a $m\times m$ matrix $\tilde{X}={\tilde{X}}^{*}$ in the complex domain is Hermitian and when A is a $m\times m$ nonsingular constant matrix in the real or complex domain, then

$$\tilde{Y}=A\tilde{X}{A}^{*},\left|A\right|\ne 0,\Rightarrow \mathrm{d}\tilde{Y}={\left|\det \left(A\right)\right|}^{2m}\mathrm{d}\tilde{X}={\left|\det \left(A{A}^{*}\right)\right|}^m\mathrm{d}\tilde{X}.$$

Now, under Lemma 2.1, $\left(2.1\right)$ reduces to an integral over Y. Let us denote it as $f_1\left(Y\right)$. Then,

$${\int }_Y{f}_1\left(Y\right)\mathrm{d}Y={c\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}{\int }_Y{\left|Y{Y}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\delta }}\mathrm{d}Y.$$

(2.2)

The corresponding integral in the complex case is the following:

$${\int }_{\tilde{Y}}{f}_{1c}\left(\tilde{Y}\right)\mathrm{d}\tilde{Y}=\tilde{c}|\det \left(A\right){|}^{-q}|\det \left(B\right){|}^{-p}{\int }_{\tilde{Y}}{\left|\det \left(\tilde{Y}{\tilde{Y}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\delta }}\mathrm{d}\tilde{Y}.$$

(2c.2) 

The function $f_1\left(Y\right)$ in the real domain when $\gamma =0,\eta \ne 0,\delta \ne 1$ is often referred to as Kotz’ model by most of the authors who use such a model. When the exponent of the determinant $\gamma \ne 0$ then the evaluation of the integral over $f_1\left(Y\right)$ is very difficult, which will be seen from the computations to follow. When $\gamma \ne 0$, and in the real domain, [7] calls the model also as Kotz’ model but the normalizing constant given by them and claimed to be available in earlier literature, does not seem to be correct. The correct normalizing constant and its evaluation in the real and complex domains will be given in detail below. Since $f_2\left(Y\right)$ involves a determinant and a trace where the determinant is a product of eigenvalues and trace is a sum, two elementary symmetric functions, if there a transformation involving elementary symmetric functions then one can handle the determinant and trace together. This author does not know any such transformation. Going through eigenvalues does not seem to be a good option because the Jacobian element will involve a Vandermonde determinant and not very convenient to handle. The next possibility is triangularization and in this case also one can make the determinant as a product of scalar variables and trace as a sum. Then, one can use a general polar coordinate transformation so that the trace becomes a single variable, namely the radial variable r and in the product r and sine and cosine product will be separated also. Hence, this approach will be a convenient one. Continuing with the evaluation of $\left(2.2\right)$ in the real case, we have the following situations: if $\delta =1$ and $\eta =0$, or $\gamma =0$, then one would immediately convert $\mathrm{d}X$ into $\mathrm{d}S,S=X{X}^{\prime }$ and integrate out by using a real matrix-variate gamma integral in the case of $\eta =0$ and $\delta =1$ and integrate out by using the scalar variable gamma integral in the case $\gamma =0$. This conversion can be done with the help of Lemma 2.3 given below.

Lemma 2.3. 

Let the $m\times n,m\le n$ matrix X of rank m be in the real domain with $mn$ distinct elements $x_{ij}$’s. Let the $m\times m$ matrix be denoted by $S=X{X}^{\prime }$ which is positive definite. Then, going through a transformation involving a lower triangular matrix with positive diagonal elements and a semi-orthonormal matrix and after integrating out the differential element corresponding to the semi-orthonormal matrix, we will have the following connection between $\mathrm{d}X$ and $\mathrm{d}S$, see the details from [12]:

$$\mathrm{d}X=\frac{{\pi }^{\frac{mn}{2}}}{{\Gamma }_m\left(\frac{n}{2}\right)}{\left|S\right|}^{\frac{n}{2}-\frac{m+1}{2}}\mathrm{d}S$$

where, for example, ${\Gamma }_m\left(\alpha \right)$ is the real matrix-variate gamma function given by

$$\begin{array}{cc}\hfill {\Gamma }_m\left(\alpha \right)& ={\pi }^{\frac{m(m-1)}{4}}\Gamma \left(\alpha \right)\Gamma (\alpha -\frac{1}{2})...\Gamma (\alpha -\frac{m-1}{2}),\Re \left(\alpha \right)>\frac{m-1}{2}\hfill \\ \hfill & ={\int }_{Z>O}{\left|Z\right|}^{\alpha -\frac{m+1}{2}}{\mathrm{e}}^{-\mathrm{tr}\left(Z\right)}\mathrm{d}Z,\Re \left(\alpha \right)>\frac{m-1}{2}\hfill \end{array}$$

where $\mathrm{tr}(·)$ means the trace of the square matrix $(·)$. Since ${\Gamma }_m\left(\alpha \right)$ is associated with the above real matrix-variate gamma integral, we call ${\Gamma }_m\left(\alpha \right)$ a real matrix-variate gamma function. This ${\Gamma }_m\left(\alpha \right)$ is also known by different names in the literature. When the $m\times n,m\le n$ matrix $\tilde{X}$ of rank m, with distinct elements, is in the complex domain and letting $\tilde{S}=\tilde{X}{\tilde{X}}^{*}$, which is $m\times m$ and Hermitian positive definite, then, going through a transformation involving a lower triangular matrix with real and positive diagonal elements and a semi-unitary matrix, we can establish the following connection between $\mathrm{d}\tilde{X}$ and $\mathrm{d}\tilde{S}$, [12]:

$$\mathrm{d}\tilde{X}=\frac{{\pi }^{mn}}{{\tilde{\Gamma }}_m\left(n\right)}{\left|\det \left(\tilde{S}\right)\right|}^{n-m}\mathrm{d}\tilde{S}$$

where, for example, ${\tilde{\Gamma }}_m\left(\alpha \right)$ is the complex matrix-variate gamma function given by

$$\begin{array}{cc}\hfill {\tilde{\Gamma }}_m\left(\alpha \right)& ={\pi }^{\frac{m(m-1)}{2}}\Gamma \left(\alpha \right)\Gamma (\alpha -1)...\Gamma (\alpha -m+1),\Re \left(\alpha \right)>m-1\hfill \\ \hfill & ={\int }_{\tilde{Z}>O}{\left|\det \left(\tilde{Z}\right)\right|}^{\alpha -m}{\mathrm{e}}^{-\mathrm{tr}\left(\tilde{Z}\right)}\mathrm{d}\tilde{Z},\Re \left(\alpha \right)>m-1.\hfill \end{array}$$

We call ${\tilde{\Gamma }}_m\left(\alpha \right)$ the complex matrix-variate gamma because it is associated with a matrix-variate gamma integral in the complex domain.

But in our $\left(2.2\right)$, both the determinant and trace enter as multiplicative factors and there is an exponent $\delta >0$ for the exponential trace. In order to tackle this situation, we will convert $\mathrm{d}X$ to $\mathrm{d}T$, where T is a lower triangular matrix, by using Theorem 2.14 of [12] which is restated here as a lemma. The idea is that in this case $|X{X}^{\prime }|=|T{T}^{\prime }|$ becomes product of the squares of the diagonal elements in T only and $\mathrm{tr}\left(T{T}^{\prime }\right)$ is a sum of squares also. This conversion can also be achieved by converting $\mathrm{d}S$ of Lemma 2.3 to a $\mathrm{d}T$ by using another result, where T is lower triangular.

Lemma 2.4. 

Let X be $m\times n,m\le n$ matrix of rank m with functionally independent $mn$ real scalar variables as elements. Let T be a lower triangular matrix and let $U_1$ be a semi-orthonormal matrix, $U_1{U}_1^{\prime }=I_p$. Consider the transformation $X=T{U}_1$ where both T and $U_1$ are uniquely selected, for example, with the diagonal elements positive in T and with the first column elements positive in $U_1$. Then, after integrating out the differential element corresponding the semi-orthonormal matrix $U_1$, one has the following connection between $\mathrm{d}X$ and $\mathrm{d}T$, [12]:

$$X=T{U}_1\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{mn}{2}}}{{\Gamma }_m\left(\frac{n}{2}\right)}\{\prod _{j=1}^m|t_{jj}{|}^{n-j}\}\mathrm{d}T$$

and in the complex case, let $\tilde{X}$ be $m\times n,m\le n$ matrix of rank m with $mn$ distinct elements in the complex domain. Let $\tilde{T}$ be a lower triangular matrix in the complex domain with the diagonal elements real and positive and ${\tilde{U}}_1$ be a semi-unitary matrix, ${\tilde{U}}_1{\tilde{U}}_1^{*}=I_p$, where $\tilde{T}$ and ${\tilde{U}}_1$ are uniquely chosen. Then, after integrating out the differential element corresponding to ${\tilde{U}}_1$, one has the following connection between $\mathrm{d}\tilde{X}$ and $\mathrm{d}\tilde{T}$:

$$\tilde{X}=\tilde{T}{\tilde{U}}_1\Rightarrow \mathrm{d}\tilde{X}=\frac{{\pi }^{mn}}{{\tilde{\Gamma }}_m\left(n\right)}\left\{\prod _{j=1}^m{\left(t_{jj}\right)}^{2(n-j)+1}\right\}\mathrm{d}\tilde{T}.$$

Let us consider the evaluation of $\left(2.2\right)$ in the real case first. Converting $\mathrm{d}Y$ in $\left(2.2\right)$ to $\mathrm{d}T$ by using Lemma 2.4, the integral part of $\left(2.2\right)$ over Y, is the following, denoted by $f_2\left(T\right)$:

$$\begin{array}{cc}\hfill {\int }_T{f}_2\left(T\right)\mathrm{d}T& ={c\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}{\int }_T{\left|T{T}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(T{T}^{\prime }\right)\right]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(T{T}^{\prime }\right)\right]}^{\delta }}\{\prod _{j=1}^p|t_{jj}{|}^{q-j}\}\mathrm{d}T.\hfill \end{array}$$

(2.3)

The corresponding equation in the complex domain is the following:

$$\begin{array}{cc}\hfill {\int }_{\tilde{T}}{f}_{2c}\left(\tilde{T}\right)\mathrm{d}\tilde{T}& =\tilde{c}{\left|\det \left(A\right)\right|}^{-q}{\left|\det \left(B\right)\right|}^{-p}\frac{{\pi }^{pq}}{{\Gamma }_p\left(q\right)}{\int }_{\tilde{T}}{\left|\det \left(\tilde{T}{\tilde{T}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{T}{\tilde{T}}^{*}\right)\right]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{T}{\tilde{T}}^{*}\right)\right]}^{\delta }}\left\{\prod _{j=1}^p{t}_{jj}^{2(q-j)+1}\right\}\mathrm{d}\tilde{T}.\hfill \end{array}$$

(2c.3)

Note that, in the real case

$$|T{T}^{\prime }|=\prod _{j=1}^p{t}_{jj}^2,\mathrm{tr}\left(T{T}^{\prime }\right)=\sum _{j=1}^p{t}_{jj}^2+\sum _{i>j}{t}_{ij}^2$$

where in ${\sum }_{j=1}^p{t}_{jj}^2$ there are p terms and second sum has $p(p-1)/2$ terms, thus a total of $k=p(p+1)/2$ terms. The corresponding quantity in the complex domain is the following:

$$|\det \left(\tilde{T}{\tilde{T}}^{*}\right)|=\prod _{j=1}^p{t}_{jj}^2,\mathrm{tr}\left(\tilde{T}{\tilde{T}}^{*}\right)=\sum _{j=1}^p{t}_{jj}^2+\sum _{j>k}|{\tilde{t}}_{jk}{|}^2,{\left|{\tilde{t}}_{jk}\right|}^2=t_{jk1}^2+t_{jk2}^2$$

where ${\tilde{t}}_{jk}=t_{jk1}+i{t}_{jk2},i=\sqrt{(-1)},t_{jk1},t_{jk2}$ real, and in the first sum there are p square terms but in the sum ${\sum }_{j>k}{\left|{\tilde{t}}_{jk}\right|}^2$ there are a total of $2\left[\frac{p(p-1)}{2}\right]=p(p-1)$ square terms, thus a total of $p^2$ square terms in the complex case.

Let us consider a polar coordinate transformation in the real case on all the $k=p(p+1)/2$ terms by using the transformation on page 44 of [12] which is restated here for convenience, that is $\{t_{11},t_{22},...,t_{pp},t_{21},...,t_{pp-1}\}\to \{r,{\theta }_1,...,{\theta }_{k-1}\}$, $k=p(p+1)/2$.

$$\begin{array}{cc}\hfill t_{11}& =rsin{\theta }_1\hfill \\ \hfill t_{22}& =rcos{\theta }_{1}sin{\theta }_2\hfill \\ \hfill ⋮\\ \hfill t_{pp}& =rcos{\theta }_1...cos{\theta }_{p-1}sin{\theta }_p\hfill \\ \hfill t_{21}& =rcos{\theta }_1...cos{\theta }_{p}sin{\theta }_{p+1}\hfill \\ \hfill ⋮\\ \hfill t_{pp-1}& =rcos{\theta }_1...cos{\theta }_{k-1}\hfill \end{array}$$

(2.4,2c.4)

for $-\frac{\pi }{2}<{\theta }_j\le \frac{\pi }{2},j=1,...,k-2;-\pi <{\theta }_{k-1}\le \pi$ for $k=p(p+1)/2$ in the real case and $k=p^2$ in the complex case. The structure of the polar coordinate transformation in the complex case remains as in the real case, we will denote it as (2c.4), the only change is that in the real case $k=p(p+1)/2$ and in the complex case $k=p^2$. The Jacobian of the transformation in the real case is

$$\mathrm{d}{t}_{11}\wedge ...\wedge \mathrm{d}{t}_{pp-1}=r^{k-1}\{\prod _{j=1}^{k-1}|cos{\theta }_j{|}^{k-1-j}\}\mathrm{d}r\wedge \mathrm{d}{\theta }_1\wedge ...\wedge \mathrm{d}{\theta }_{k-1},k=p(p+1)/2$$

and in the complex case the Jacobian is given by

$$\mathrm{d}{t}_{11}\wedge ...\wedge \mathrm{d}{t}_{pp}\wedge ...\wedge \mathrm{d}{t}_{pp-12}=r^{p^2-1}\{\prod _{j=1}^{p^2-1}|cos{\theta }_j{|}^{p^2-j-1}\}\mathrm{d}r\wedge \mathrm{d}{\theta }_1\wedge ...\wedge \mathrm{d}{\theta }_{p^2-1}$$

for the same ranges for ${\theta }_j$’s as in the real case, but in the complex case $k=p^2$.

The normalizing constant c in the real case coming from (2.3) is quoted in [7] by citing earlier works. But none of them seems to have given the evaluation of the integral in (2.3) explicitly. The normalizing constant c given in [7] does not seem to be correct. Since the integral in (2.3) appears in very many places as Kotz’ integral, and used in many disciplines, a detailed evaluation of the integral in (2.3) is warranted. Also, none seems to have given $\tilde{c}$ in the complex case. Hence, the evaluations of c and $\tilde{c}$ in the real and complex cases will be given here in detail.

2.1,2c.1 Evaluation of the integral in (2.3) in the real case and (2c.3) in the complex case

Note that ${\sum }_{ij}{t}_{ij}^2=r^2$. From the Jacobian part, the factor containing r is $r^{k-1}={\left(r^2\right)}^{\frac{k}{2}-\frac{1}{2}}$. In the product ${\prod }_{j=1}^p{t}_{jj}^2$ each $t_{jj}^2$ contains a $r^2$. Also, the Jacobian part ${\prod }_{j=1}^p|t_{jj}{|}^{q-j}={\prod }_{j=1}^p{\left(t_{jj}^2\right)}^{\frac{q}{2}-\frac{j}{2}}={\left|T{T}^{\prime }\right|}^{\frac{q}{2}-\frac{j}{2}}$. Collecting all r, the exponent of $r^2$ in the real case is the following:

$$(\gamma +\frac{q}{2}-\frac{1}{2})+(\gamma +\frac{q}{2}-\frac{2}{2})+...+(\gamma +\frac{q}{2}-\frac{p}{2})+\eta +\frac{p(p+1)}{4}-\frac{1}{2}=p(\gamma +\frac{q}{2})+\eta -\frac{1}{2}.$$

Then, integration over r gives the following:

$${\int }_0^{\infty }{\left(r^2\right)}^{p((\gamma +\frac{q}{2})+\eta )-\frac{1}{2}}{\mathrm{e}}^{-\alpha {\left(r^2\right)}^{\delta }}\mathrm{d}r=\frac{1}{2\delta }\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]{\alpha }^{-\left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]}$$

(2.5)

for $\Re \left(\gamma \right)>-\frac{q}{2},\Re \left(\eta \right)>0,\alpha >0,\delta >0$. The corresponding integral over r in the complex domain is the following:

$${\int }_0^{\infty }{\left(r^2\right)}^{p(\gamma +q)+\eta -\frac{1}{2}}{\mathrm{e}}^{-\alpha {\left(r^2\right)}^{\delta }}\mathrm{d}r=\frac{1}{2\delta }\Gamma \left[\frac{1}{\delta }(p(\gamma +q)+\eta )\right]{\alpha }^{-\frac{1}{\delta }(p(\gamma +q)+\eta )}$$

(2c.5) 

for $\Re \left(\gamma \right)>-q,\delta >0,\alpha >0,\Re \left(\eta \right)>0$.

2.2 Evaluation of the sine and cosine product in the real case

Consider the integration of factors containing the ${\theta }_j$’s in the real case. These ${\theta }_j$’s come from ${\prod }_{j=1}^p{t}_{jj}^2$ and from the Jacobian part. Consider ${\theta }_1$. The exponent of ${sin}^2{\theta }_1$ is $\gamma +\frac{q}{2}-\frac{1}{2}$. The exponent of ${cos}^2{\theta }_1$ is $(\gamma +\frac{q}{2}-\frac{2}{2})+(\gamma +\frac{q}{2}-\frac{3}{2})+...+(\gamma +\frac{q}{2}-\frac{p}{2})=(p-1)(\gamma +\frac{q}{2})-\frac{p(p+1)}{4}+\frac{1}{2}$ and the part coming from the Jacobian $|cos{\theta }_1{|}^{k-1-1}$. Note that $|cos{\theta }_1{|}^{k-1-1}={\left({cos}^2{\theta }_1\right)}^{\frac{p(p+1)}{4}-\frac{3}{2}}|cos{\theta }_1|$. Then, the integral over over ${\theta }_1$, denoting the integral over ${\theta }_1$ as $I_{{\theta }_1}$, gives the following, where in all the integrations over the ${\theta }_j$’s to follow, we will use the transformations $x=sin{\theta }_j,u=x^2$:

$$\begin{array}{cc}\hfill I_{{\theta }_1}& ={\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}{\left({sin}^2{\theta }_1\right)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{\left({cos}^2{\theta }_1\right)}^{(p-1)(\gamma +\frac{q}{2})-1}|cos{\theta }_1|\mathrm{d}{\theta }_1\hfill \\ \hfill & =2{\int }_0^{\frac{\pi }{2}}{\left({sin}^2{\theta }_1\right)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{\left({cos}^2{\theta }_1\right)}^{(p-1)(\gamma +\frac{q}{2})-1}|cos{\theta }_1|\mathrm{d}{\theta }_1\hfill \\ \hfill & =2{\int }_0^1{\left(x^2\right)}^{\gamma +\frac{q}{2}-\frac{1}{2}}{(1-x^2)}^{(p-1)(\gamma +\frac{q}{2})-1}\mathrm{d}x={\int }_0^1{u}^{\gamma +\frac{q}{2}-1}{(1-u)}^{(p-1)(\gamma +\frac{q}{2})-1}\mathrm{d}u\hfill \\ \hfill & =\frac{\Gamma (\gamma +\frac{q}{2})\Gamma \left[(p-1)(\gamma +\frac{q}{2})\right]}{\Gamma \left[p(\gamma +\frac{q}{2})\right]},\Re \left(\gamma \right)>-\frac{q}{2}.\hfill \end{array}$$

(i)

Now, collecting all factors containing ${\theta }_2$ and proceeding as in the case of ${\theta }_1$ we have the following result for the integral over ${\theta }_2$:

$$I_{{\theta }_2}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{1}{2})\Gamma [(p-2)(\gamma +\frac{q}{2})+\frac{1}{2}]}{\Gamma \left[(p-1)(\gamma +\frac{q}{2})\right]},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{1}{2}.$$

(ii) 

Note that the denominator gamma in $\left(ii\right)$ cancels with one numerator gamma in $\left(i\right)$. The pattern of cancellation of the denominator gamma in the next step with one numerator gamma in its previous step will continue leaving only one factor in the numerator and no factor in the denominator, except the very first step involving $\left(i\right)$ and $\left(ii\right)$ where the first denominator gamma, namely $\Gamma \left(p(\gamma +\frac{q}{2})\right)$ is left out. When, integrating ${\theta }_{p-1}$, we have the following:

$$I_{{\theta }_{p-1}}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{p-2}{2})\Gamma ((\gamma +\frac{q}{2})-\frac{p}{2}-\frac{p-1}{2}+\frac{p(p+1)}{4})}{\Gamma (2(\gamma +\frac{q}{2})-\frac{p-2}{2}-\frac{p-1}{2}-\frac{p}{2}+\frac{p(p+1)}{4})},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-2}{2}.$$

(iii) 

Note that when considering ${\theta }_p$ there is no cosine factor coming from $t_{pp}$ and the cosine factor comes only from the Jacobian part. We can see that

$$I_{{\theta }_p}=\frac{\Gamma (\gamma +\frac{q}{2}-\frac{p-1}{2})\Gamma (\frac{p(p+1)}{4}-\frac{p}{2})}{\Gamma ((\gamma +\frac{q}{2})-\frac{p}{2}-\frac{p-1}{2}+\frac{p(p+1)}{4})},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}.$$

(iv) 

Again, the denominator gamma in $\left(iv\right)$ cancels with one numerator gamma in $\left(iii\right)$. This pattern will continue for $k=p+1,p+2,...$. For the integrals over ${\theta }_j,j=p+1,p+2,...$ the only contribution is from the Jacobian part, no sine factor will be there. Consider ${\theta }_{p+1}$. We see that

$$I_{{\theta }_{p+1}}=\frac{\Gamma \left(\frac{1}{2}\right)\Gamma (\frac{p(p+1)}{4}-\frac{p+1}{2})}{\Gamma (\frac{p(p+1)}{4}-\frac{p}{2})},p>2.$$

(v) 

Again, cancellation will hold. Now, consider a few last cases of ${\theta }_j$. For $j=\frac{p(p+1)}{2}-3=k-3$ we have

$$I_{{\theta }_{k-3}}=\frac{\Gamma \left(\frac{1}{2}\right)\Gamma \left(\frac{3}{2}\right)}{\Gamma \left(2\right)}$$

and for $j=k-2$

$$I_{{\theta }_{k-2}}=\frac{\Gamma \left(\frac{1}{2}\right)\Gamma \left(1\right)}{\Gamma \left(\frac{3}{2}\right)}$$

and the last ${\theta }_j$ goes from $-\pi$ to $\pi$ and no contribution from the Jacobian part and hence

$$I_{{\theta }_{k-1}}=2\pi .$$

Note that starting from $j=p+1$ to $j=k-2$ the gamma factor left in the numerator is $\Gamma \left(\frac{1}{2}\right)=\sqrt{\pi }$. There are $\frac{p(p-1}{2}-2$ such factors and the last one is $2\pi$, thus the product is $2{\pi }^{\frac{p(p-1)}{4}}$. For $j=1,...,p$ the factors left out in the numerator are $\Gamma (\gamma +\frac{q}{2})\Gamma (\gamma +\frac{q}{2}-\frac{1}{2})...\Gamma (\gamma +\frac{q}{2}-\frac{p-1}{2})$ and for $j=p+1,...,k-1$ we have ${\pi }^{\frac{p(p-1)}{4}}$ giving ${\Gamma }_p(\gamma +\frac{q}{2}),\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$. For $j=1$ there is one gamma left in the denominator, namely, $\Gamma \left(p(\gamma +\frac{q}{2})\right)$. Taking the product of integral over all ${\theta }_j$’s in the real case is

$$2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma \left(p(\gamma +\frac{q}{2})\right)$$

(vi) 

where ${\Gamma }_p(·)$ is the real matrix-variate gamma defined in Lemma 2.3.

2c.2 Evaluation of the integral over the ${\theta }_j$’s in the complex case

The sine and cosine functions come from the transformations corresponding to $t_{11},...,t_{pp}$, from the Jacobian when going from $\tilde{X}$ to $\tilde{T}$ and from the Jacobian in the polar coordinate transformation. The Jacobian part in the polar coordinate transformation is

$$\begin{array}{cc}\hfill \prod _{j=1}^{p^2-1}{|cos{\theta }_j|}^{p^2-j-1}& =\left\{\prod _{j=1}^{p^2-1}{\left({cos}^2{\theta }_j\right)}^{\frac{p^2}{2}-\frac{j}{2}-1}\right|cos{\theta }_j\left|\right\}\hfill \\ \hfill |\tilde{T}{\tilde{T}}^{*}{|}^{\gamma }\left[\prod _{j=1}^p{t}_{jj}^{2(q-j)+1}\right]& =\prod _{j=1}^p{\left(t_{jj}^2\right)}^{\gamma +q-j+\frac{1}{2}}.\hfill \end{array}$$

(i)

Collecting factors containing ${\theta }_1$, observing that $sin{\theta }_1$ comes from $t_{11}$ and $cos{\theta }_1$ comes from $t_{22},...,t_{pp}$ and the Jacobian part. The exponent of ${sin}^2{\theta }_1$ is $\gamma +q-\frac{1}{2}$ and the exponent of ${cos}^2{\theta }_1$ is $(p-1)(\gamma +q+\frac{1}{2})-\frac{p(p+1)}{2}+1+\frac{p^2}{2}-\frac{1}{2}-1=(p-1)(\gamma +q)-1$. In all the integrals to follow, ${\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}(·)\mathrm{d}\theta =2{\int }_0^{\frac{\pi }{2}}(·)\mathrm{d}\theta$ due to evenness of the integrand. Then, we will use the transformations $x=sin\theta ,u=x^2$, the steps parallel to those in the real case. Therefore

$${\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}{\left({sin}^2{\theta }_1\right)}^{\gamma +q-\frac{1}{2}}{\left({cos}^2{\theta }_1\right)}^{(p-1)(\gamma +q)-1}|cos{\theta }_1|\mathrm{d}{\theta }_1=\frac{\Gamma (\gamma +q)\Gamma \left(\right(p-1\left)\right(\gamma +q\left)\right)}{\Gamma \left(p\right(\gamma +q\left)\right)},$$

(ii) 

for $\Re \left(\gamma \right)>-q$. Collecting the factors containing ${\theta }_2$, we note that the exponent of ${sin}^2{\theta }_2$ is $\gamma +q-2+\frac{1}{2}$ and the exponent of ${cos}^2{\theta }_2$ is $(p-2)(\gamma +q+\frac{1}{2})-\frac{p(p+1)}{2}+1+2+\frac{p^2}{2}-2=(p-2)(\gamma +q)$. Hence,

$$\begin{array}{cc}\hfill & 2{\int }_0^{\frac{pi}{2}}{\left({sin}^2{\theta }_2\right)}^{(\gamma +q)-\frac{3}{2}}{\left({cos}^2{\theta }_2\right)}^{(p-2)(\gamma +q)+1-1}|cos{\theta }_2|\mathrm{d}{\theta }_2\hfill \\ \hfill & =\frac{\Gamma (\gamma +q-1)\Gamma \left(\right(p-2\left)\right(\gamma +q)+1)}{\Gamma \left(\right(p-1\left)\right(\gamma +q\left)\right)},\Re \left(\gamma \right)>-q+1.\hfill \end{array}$$

(iii) 

Note that $\Gamma \left(\right(p-1\left)\right(\gamma +q\left)\right)$ from the denominator of $\left(iii\right)$ cancels with the same in the numerator of $\left(ii\right)$ leaving one gamma, namely $\Gamma (\gamma +q)$ in the numerator of $\left(ii\right)$ and one gamma, namely $\Gamma \left(p\right(\gamma +q\left)\right)$ in the denominator of $\left(ii\right)$. The pattern of cancellation of the gamma in the denominator of a step canceling with a gamma in the numerator of the previous step will continue as seen in the real case. Let us check for $j=p,j=p+1$ to see whether the pattern is continuing, where in $j=p+1$ there is no contribution of sine function, the only cosine function coming is from the Jacobian part. For $j=p$ we have

$${\int }_{{\theta }_p}{\left({sin}^2{\theta }_p\right)}^{\gamma +q+\frac{1}{2}-p}{\left({cos}^2{\theta }_p\right)}^{\frac{p(p-1)}{2}-1}|cos{\theta }_p|\mathrm{d}{\theta }_p=\frac{\Gamma (\gamma +q-p+1)\Gamma \left(\frac{p(p-1)}{2}\right)}{\Gamma (\gamma +1-p+1+\frac{p(p-1)}{1})}.$$

(iv) 

For $j=p+1$

$${\int }_{{\theta }_{p+1}}{\left({cos}^2{\theta }_{p+1}\right)}^{\frac{p^2-p-3}{2}}|cos{\theta }_{p+1}|\mathrm{d}{\theta }_{p+1}=\frac{\Gamma \left(\frac{1}{2}\right)\Gamma (\frac{p(p-1)}{2}-\frac{1}{2})}{\Gamma \left(\frac{p(p-1)}{2}\right)}.$$

(v) 

The pattern of cancelation is continuing. But, starting from $j=p+1,...,p^2-2$ the factor left out in the numerator is $\Gamma \left(\frac{1}{2}\right)=\sqrt{\pi }$ and the last factor gives $2\pi$ because the range here is $-\pi <{\theta }_{p^2-1}\le \pi$ and hence the factors left out in the numerator are ${\left(\sqrt{\pi }\right)}^{p(p-1)}\Gamma (\gamma +q)\Gamma (\gamma +q-1)...\Gamma (\gamma +q-p+1)={\tilde{\Gamma }}_p(\gamma +q),\Re \left(\gamma \right)>-q+p-1$ and one gamma, namely $\Gamma \left(p\right(\gamma +q\left)\right)$ is left out in the denominator of the integration over ${\theta }_1$. Hence, the integration over all sine and cosine functions in the complex case is

$$\frac{2{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma \left(p\right(\gamma +q\left)\right)}$$

(vi) 

where ${\tilde{\Gamma }}_p(·)$ is the complex matrix-variate gamma function defined in Lemma 2.3. Then the final result of integration over r and integration over all ${\theta }_j$’s in the real case is the following:

$$\begin{array}{cc}\hfill {\int }_{X}f\left(X\right)\mathrm{d}X& ={\int }_{X}c{\left|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right|}^{\gamma }{\left[\mathrm{tr}\left(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right)\right]}^{\eta }\hfill \\ \hfill & \times {\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right)\right]}^{\delta }}\mathrm{d}X\hfill \\ \hfill & =c\frac{1}{2\delta }\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]{\alpha }^{-\frac{1}{\delta }[p(\gamma +\frac{q}{2})+\eta ]}\hfill \\ \hfill & \times 2\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma \left(p(\gamma +\frac{q}{2})\right)}{\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}\hfill \\ \hfill & =c\frac{{\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}{\pi }^{\frac{pq}{2}}}{\delta {\Gamma }_p\left(\frac{q}{2}\right){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}\frac{\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma \left(p(\gamma +\frac{q}{2})\right)}\hfill \end{array}$$

(2.6)

for $\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\Re \left(\eta \right)>0,\delta >0,\alpha >0,p\le q,M=E\left[X\right]$,and $A>O,B>O$ are constant matrices where A is $p\times p$ and B is $q\times q$ and X is $p\times q,p\le q$ real matrix of rank p, and the corresponding integral in the complex case is given by

$${\int }_{\tilde{X}}{f}_c\left(\tilde{X}\right)\mathrm{d}\tilde{X}=\tilde{c}\frac{{\pi }^{pq}}{\delta {\tilde{\Gamma }}_p{\left(q\right)\left|\det \left(A\right)\right|}^q{\left|\det \left(B\right)\right|}^p{\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}\frac{\Gamma \left[\frac{1}{\delta }(p(\gamma +q)+\eta )\right]{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma \left(p\right(\gamma +q\left)\right)}$$

(2c.6) 

for $\Re \left(\gamma \right)>-q+p-1,\Re \left(\eta \right)>0,\delta >0,\alpha >0,\tilde{M}=E\left[\tilde{X}\right],A=A^{*}>O,B=B^{*}>O,p\le q$. Therefore the normalizing constants c and $\tilde{c}$ are the following:

$$c=\frac{{\left|A\right|}^{\frac{q}{2}}{\left|B\right|}^{\frac{p}{2}}\delta \Gamma \left(p(\gamma +\frac{q}{2})\right){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}{\Gamma }_p\left(\frac{q}{2}\right)}{{\Gamma }_p(\gamma +\frac{q}{2})\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]{\pi }^{\frac{pq}{2}}},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},$$

(2.7)

for $\delta >0,\alpha >0,\Re \left(\eta \right)>0,p\le q$ and

$$\tilde{c}=\frac{{\left|\det \left(A\right)\right|}^q{\left|\det \left(B\right)\right|}^p\delta {\tilde{\Gamma }}_p\left(q\right){\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}\Gamma \left(p(\gamma +q)\right)}{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma \left[\frac{1}{\delta }(p(\gamma +q)+\eta )\right]},\Re \left(\gamma \right)>-q+p-1$$

(2c.7) 

for $\delta >0,\alpha >0,\Re \left(\eta \right)>0,p\le q$.

From the general results in (2.6, 2c.6) we can have the following interesting special cases: $A=I,\eta =0$; $\eta =0,B=I$; $A=I,B=I,\eta =0$; $\gamma =0$; $\gamma =0,\eta =0$; $\eta =0,\delta =1$. Since the integral over the sine and cosine product is very important in many types of applications, we will give these as theorems here.

Theorem 2.1. 

Let ${\Theta }_k=\{{\theta }_1,...,{\theta }_{k-1}\},k=\frac{p(p+1)}{2}$ in the real case, and integral over ${\Theta }_k$ denoted by $I_{{\Theta }_k}$, is the following:

$$\begin{array}{cc}\hfill I_{{\Theta }_k}& ={\int }_{\Theta }\{\prod _{j=1}^p{(co{s}^2{\theta }_1{cos}^2{\theta }_2...{cos}^2{\theta }_{j-1}{sin}^2{\theta }_j)}^{\gamma +\frac{q}{2}-\frac{j}{2}}|cos{\theta }_j{|}^{k-1-j}\left\}\right\{\prod _{j=p+1}^{k-1}|cos{\theta }_j{|}^{k-1-j}\}\mathrm{d}\Theta \hfill \\ \hfill & =\frac{2{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma \left(p(\gamma +\frac{q}{2})\right)},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},p\le q,p\ge 2,k=\frac{p(p+1)}{2}.\hfill \end{array}$$

The corresponding result in the complex case is the following, where ${\Theta }_k$ here has the same format as in the real case but here $k=p^2$.

Theorem 2c.1. 

Let ${\Theta }_k=\{{\theta }_1,...,{\theta }_{k-1}\},k=p^2$. The integral over ${\Theta }_k$ in the complex case is the following:

$$\begin{array}{cc}\hfill I_{{\Theta }_k}& ={\int }_{{\Theta }_k}\{\prod _{j=1}^p{(co{s}^2{\theta }_1{cos}^2{\theta }_1...{cos}^2{\theta }_{j-1}{sin}^2{\theta }_j)}^{\gamma +q-j+\frac{1}{2}}{\left({cos}^2{\theta }_j\right)}^{\frac{p^2-j-2}{2}}|cos{\theta }_j\left|\right\}\hfill \\ \hfill & \times \{\prod _{j=p+1}^{p^2-1}{\left({cos}^2{\theta }_j\right)}^{\frac{p^2-j-2}{2}}|cos{\theta }_j\left|\right\}\mathrm{d}{\Theta }_k\hfill \\ \hfill & =\frac{2{\tilde{\Gamma }}_p(\gamma +q)}{\Gamma \left(p\right(\gamma +q\left)\right)},\Re \left(\gamma \right)>-q+p-1,p\le q,p\ge 2,k=p^2.\hfill \end{array}$$

From (2.6) in the real case we have the following theorems:

Theorem 2.2. 

Let Y be a $p\times q,p\le q$ matrix of rank p with the $pq$ elements functionally independent real scalar variables. For $\delta >0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$,

$$\begin{array}{cc}\hfill & {\int }_Y{\left|Y{Y}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\delta }}\mathrm{d}Y\hfill \\ \hfill & =\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}\frac{\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]{\Gamma }_p(\gamma +\frac{q}{2})}{\delta \Gamma \left(p(\gamma +\frac{q}{2})\right){\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}.\hfill \end{array}$$

Note 2.1. 

In the widely used normalizing constant in [7], which was quoted from earlier references, and corresponding to the normalizing constant c in (2.3) above, a gamma factor, namely, $\Gamma \left(p(\gamma +\frac{q}{2})\right)$ in the denominator is missing. Either it is a computational slip or due to the use of some wrong results in the derivation of the normalizing constant in [7] and in earlier works of others.

Theorem 2c.2. 

Let $\tilde{Y}$ be a $p\times q,p\le q$ matrix in the complex domain with rank p where the $pq$ elements are functionally independent complex scalar variables. For $\delta >0,\alpha >0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-q+p-1$

$$\begin{array}{cc}\hfill & {\int }_{\tilde{Y}}{\left|\det \left(\tilde{Y}{\tilde{Y}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\delta }}\mathrm{d}\tilde{Y}\hfill \\ \hfill & =\frac{{\pi }^{pq}}{{\tilde{\Gamma }}_p\left(q\right)}\frac{\Gamma \left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right){\tilde{\Gamma }}_p(\gamma +q)}{\delta \Gamma \left(p(\gamma +q)\right){\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}.\hfill \end{array}$$

Corollary 2.1. 

For $Y,\delta ,\gamma ,\alpha$ as defined in Theorem 2.2,

$${\int }_Y{\left|Y{Y}^{\prime }\right|}^{\gamma }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\delta }}\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})\Gamma \left(\frac{p}{\delta }(\gamma +\frac{q}{2})\right)}{\delta {\Gamma }_p\left(\frac{q}{2}\right)\Gamma \left(p(\gamma +\frac{q}{2})\right){\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}$$

for $\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\delta >0,p\le q$.

The results quoted from some earlier works of others and reported in [7], corresponding to our Theorems 2.2 and Corollary 2.1, do not agree with our results.

The result corresponding Corollary 2.1 in the complex case is the following:

Corollary 2c.1. 

For $\tilde{Y},\delta ,\gamma ,\alpha$ as defined in Theorem 2c.2,

$${\int }_{\tilde{Y}}{\left|\det \left(\tilde{Y}{\tilde{Y}}^{*}\right)\right|}^{\gamma }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\delta }}\mathrm{d}\tilde{Y}=\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma \left(\frac{p}{\delta }(\gamma +q)\right)}{\delta {\tilde{\Gamma }}_p\left(q\right)\Gamma \left(p(\gamma +q)\right){\alpha }^{\frac{p}{\delta }(\gamma +q)}}$$

for $\Re \left(\gamma \right)>-q+p-1,\delta >0,\alpha >0$.

Corollary 2.2. 

For $Y,\gamma ,\alpha$ as defined in Theorem 2.2,

$${\int }_Y{\left|Y{Y}^{\prime }\right|}^{\gamma }{\mathrm{e}}^{-\alpha \left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{{\alpha }^{p(\gamma +\frac{q}{2})}},\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\alpha >0,p\le q.$$

The corresponding result in the complex domain is the following:

Corollary 2c.2. 

For $\tilde{Y},\gamma ,\alpha$ as defined in Theorem 2c.2,

$${\int }_{\tilde{Y}}{\left|\det \left(\tilde{Y}{\tilde{Y}}^{*}\right)\right|}^{\gamma }{\mathrm{e}}^{-\alpha \left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}\mathrm{d}\tilde{Y}=\frac{{\pi }^{pq}}{{\tilde{\Gamma }}_p\left(q\right)}\frac{{\tilde{\Gamma }}_p(\gamma +q)}{{\alpha }^{p(\gamma +q)}}$$

for $\Re \left(\gamma \right)>-q+p-1,\alpha >0$.

Theorem 2.3. 

Let $U=\left(u_{ij}\right)>O$ be a $p\times p$ real positive definite matrix with $p(p+1)/2$ functionally independent real scalar variables $u_{ij}$’s. Then, the following integral over $U>O$ is equivalent to the integral over Y where Y is $p\times q,p\le q$ and of rank p with $pq$ distinct real scalar elements. Then,

$$\begin{array}{cc}\hfill & {\int }_{U>O}{\left|U\right|}^{\gamma +\frac{q}{2}-\frac{p+1}{2}}{\left[\mathrm{tr}\left(U\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(U\right)\right]}^{\delta }}\mathrm{d}U\hfill \end{array}$$

(i)

$$\begin{array}{cc}\hfill & =\frac{{\Gamma }_p\left(\frac{q}{2}\right)}{{\pi }^{\frac{pq}{2}}}{\int }_Y{\left|Y{Y}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(Y{Y}^{\prime }\right)\right]}^{\delta }}\mathrm{d}Y\hfill \end{array}$$

(ii)

for $\alpha >0,\delta >0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$.

This result enables us to go back and forth from a real full-rank rectangular matrix to a real positive definite matrix. The proof is straightforward. Let $Y{Y}^{\prime }=G$. Then $G=G^{\prime }>O$. Then, from Lemma 2.3, $\mathrm{d}Y=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}{\left|G\right|}^{\frac{q}{2}-\frac{p+1}{2}}$ which establishes the result. The corresponding result in the complex domain is the following:

Theorem 2c.3. 

Let the $p\times p$ matrix in the complex domain $\tilde{U}=\left({\tilde{u}}_{ij}\right)={\tilde{U}}^{*}>O$ Hermitian positive definite, where the $p(p+1)/2$ distinct scalar complex variables be the elements ${\tilde{u}}_{ij}$’s. Then, the following integral over $\tilde{U}$ is equivalent to the integral over $\tilde{Y}$ where $\tilde{Y}$ is a $p\times q,p\le q$ matrix in the complex domain of rank p with distinct $pq$ complex variables as elements. Then,

$$\begin{array}{cc}\hfill & {\int }_{\tilde{U}>O}{\left|\det \left(\tilde{U}\right)\right|}^{\gamma +q-p}{\left[\mathrm{tr}\left(\tilde{U}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{U}\right)\right]}^{\delta }}\mathrm{d}U\hfill \\ \hfill & =\frac{{\tilde{\Gamma }}_p\left(q\right)}{{\pi }^{pq}}{\int }_{\tilde{Y}}{\left|\det \left(\tilde{Y}{\tilde{Y}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{Y}{\tilde{Y}}^{*}\right)\right]}^{\delta }}\mathrm{d}\tilde{Y}\hfill \end{array}$$

for $\alpha >0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-q+p-1$.

The proof is parallel to that in the real case. Here we use Lemma 2.3 in the complex case, that is all the difference.

3. Some Integrals involving Type-2 Beta Form

Let X be a $p\times 1$ vector in the real domain with distinct scalar variables as elements. Then, we have the following multivariate type-2 beta density:

Theorem 3.1. 

$${\int }_X\frac{{\left(X^{\prime }X\right)}^{\delta }}{{[1+\alpha {\left(X^{\prime }X\right)}^{\eta }]}^{\gamma }}\mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}\frac{\Gamma \left[\frac{1}{\eta }(\delta +\frac{p}{2})\right]\Gamma (\gamma -\frac{1}{\eta }(\delta +\frac{p}{2}))}{\eta \Gamma \left(\gamma \right){\alpha }^{\frac{1}{\eta }(\delta +\frac{p}{2})}}$$

(i) 

for $\Re \left(\gamma \right)>\frac{1}{\eta }(\Re \left(\delta \right)+\frac{p}{2}),\eta >0,\Re \left(\delta \right)>-\frac{p}{2},\alpha >0$.

This result is easily seen from Lemma 2.3. Note that $\mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}{u}^{\frac{p}{2}-1}\mathrm{d}u$. Let $v=u^{\eta }$. Now, integrate out by using a scalar variable type-2 beta integral to establish the result. Integrand of the left side divided by the right side gives a statistical density. One can generalize the result in Theorem 3.1 by replacing $X^{\prime }X$ by ${(X-\mu )}^{\prime }A(X-\mu )$ where $\mu =E\left[X\right],A>O$ is a constant positive definite matrix. Then, the only change is that the right side of $\left(i\right)$ is multiplied by ${\left|A\right|}^{-\frac{1}{2}}$ the positive definite square root of the positive definite matrix $A>O$. The result corresponding to Theorem 3.1 in the complex domain will be stated next without any proof because the derivation is parallel to that in the real case.

Theorem 3c.1. 

Let $\tilde{X}$ be a $p\times 1$ vector of p distinct scalar complex variables as elements. Then, we have the following multivariate type-2 beta density:

$${\int }_{\tilde{X}}\frac{{\left[{\tilde{X}}^{*}\tilde{X}\right]}^{\delta }}{{(1+\alpha {\left[{\tilde{X}}^{*}\tilde{X}\right]}^{\eta })}^{\gamma }}\mathrm{d}\tilde{X}=\frac{{\pi }^p}{\Gamma \left(p\right)}\frac{\Gamma \left(\frac{1}{\eta }(\delta +p)\right)\Gamma (\gamma -\frac{1}{\eta }(\delta +p))}{\eta \Gamma \left(\gamma \right){\alpha }^{\frac{1}{\eta }(\delta +p)}}$$

for $\Re \left(\gamma \right)>\frac{1}{\eta }(\Re \left(\delta \right)+p),\eta >0,\Re \left(\delta \right)>-p,\alpha >0$.

Now, we consider the evaluation of a rectangular matrix-variate type-2 beta integral in the real case.

Theorem 3.2. 

Let $X=\left(x_{ij}\right)$ be $p\times q,p\le q$ matrix in the real domain of rank p with $pq$ distinct real scalar variables as elements $x_{ij}$’s. Then,

$${\int }_X\frac{{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}{{[1+\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }]}^{\gamma }}\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{\Gamma \left(\frac{pq}{2}\right)}\frac{\Gamma \left[\frac{1}{\eta }(\delta +\frac{pq}{2})\right]\Gamma (\gamma -\frac{1}{\eta }(\delta +\frac{pq}{2}))}{\eta \Gamma \left(\gamma \right){\alpha }^{\frac{1}{\eta }(\delta +\frac{pq}{2})}}$$

(ii) 

for $\Re \left(\delta \right)>-\frac{pq}{2},\Re \left(\gamma \right)>\frac{1}{\eta }(\Re \left(\delta \right)+\frac{pq}{2}),\eta >0,\alpha >0$.

Note that $\mathrm{tr}\left(X{X}^{\prime }\right)$ is the sum of squares of $pq$ elements. Then, from Lemma 2.3 the result follows, or from Theorem 3.1 the result follows by replacing $\frac{p}{2}$ by $\frac{pq}{2}$. A more general situation is to replace $X{X}^{\prime }$ by $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ where $M=E\left[X\right],A>O,B>O$ where A is $p\times p$ and B is $q\times q$ constant positive definite matrices. In this case the only change will be to multiply the right side of $\left(ii\right)$ by ${\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}$. The result corresponding to Theorem 3.2 in the complex domain will be stated next without the details because the details as well as the generalizations are parallel to those in the real case.

Theorem 3c.2. 

Let $\tilde{X}=\left({\tilde{x}}_{ij}\right)$ be $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements. Then,

$${\int }_{\tilde{X}}\frac{{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }}{{(1+\alpha {\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta })}^{\gamma }}\mathrm{d}\tilde{X}=\frac{{\pi }^{pq}}{\Gamma \left(pq\right)}\frac{\Gamma \left[\frac{1}{\eta }(\delta +pq)\right]\Gamma (\gamma -\frac{1}{\eta }(\delta +pq))}{\eta \Gamma \left(\gamma \right){\alpha }^{\frac{1}{\eta }(\delta +pq)}}$$

for $\Re \left(\delta \right)>-pq,\Re \left(\gamma \right)>\frac{1}{\eta }(\Re \left(\delta \right)+pq),\eta >0,\alpha >0$.

The next result involves a determinant

Theorem 3.3. 

Let $X=\left(x_{ij}\right)$ be a $p\times q,p\le q$ real matrix of rank p where the elements $x_{ij}$’s are distinct real scalar variables. Then

$${\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }}{{[1+\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{\rho }}\mathrm{d}X=\frac{\Gamma \left(\frac{p}{\delta }(\gamma +\frac{q}{2})\right)\Gamma (\rho -\frac{p}{\delta }(\gamma +\frac{q}{2}))}{\delta {\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}\frac{{\Gamma }_p(\gamma +\frac{q}{2})}{\Gamma \left(p(\gamma +\frac{q}{2})\right)}$$

for $\alpha >0,\delta >0,\Re \left(\rho \right)>\frac{p}{\delta }(\gamma +\frac{q}{2}),\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$.

Proof: 

Let $X=T{U}_1$ where T is a lower triangular matrix and $U_1$ is a semi-orthonormal matrix, $U_1{U}_1^{\prime }=I_p$ and let T and $U_1$ be uniquely chosen. Then, from Lemma 2.4

$$\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}\{\prod _{j=1}^p|t_{jj}{|}^{q-j}\}\mathrm{d}T,X{X}^{\prime }=T{T}^{\prime }.$$

Note that $|X{X}^{\prime }|=|T{T}^{\prime }|={\prod }_{j=1}^p{t}_{jj}^2$ and $\mathrm{tr}\left(T{T}^{\prime }\right)={\sum }_{j=1}^p{t}_{jj}^2+{\sum }_{i>j}{t}_{ij}^2=$ sum of squares of $p(p+1)/2$ real scalar variables. Now, apply a polar coordinate transformation on these $p(p+1)/2$ variables $t_{ij}$’s, taking

$$[t_{11},t_{22},...,t_{pp},t_{21},...t_{pp-1}]\to [r,{\theta }_1,...,{\theta }_{k-1}],k=p(p+1)/2.$$

Collecting all factors containing r, we have ${\left(r^2\right)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}$. Now, integrating over r we have

$${\int }_0^{\infty }{\left(r^2\right)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{[1+\alpha {\left(r^2\right)}^{\delta }]}^{-\rho }\mathrm{d}r=\frac{\Gamma \left[\frac{p}{\delta }(\gamma +\frac{q}{2})\right]\Gamma (\rho -\frac{p}{\delta }(\gamma +\frac{q}{2}))}{2\delta \Gamma \left(\rho \right){\alpha }^{\frac{p}{\delta }(\gamma +\frac{q}{2})}}$$

for $\delta >0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\Re \left(\rho \right)>\frac{p}{\delta }(\Re \left(\gamma \right)+\frac{q}{2}),\alpha >0$. From Theorem 2.1, the integral over the ${\theta }_j$’s gives $2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma \left(p(\gamma +\frac{q}{2})\right)$ and from the transformation of $X{X}^{\prime }$ to $T{T}^{\prime }$ we have ${\pi }^{\frac{pq}{2}}/{\Gamma }_p\left(\frac{q}{2}\right)$. Hence the product of these three quantities establish the theorem. Result corresponding to Theorem 3.3 in the complex domain will be given next without the proof. The proof goes parallel to that in the real case. In this connection, observe the derivation of the sine and cosine factors in the complex case given earlier, the number of terms in $\mathrm{tr}\left(\tilde{T}{\tilde{T}}^{*}\right)$ will be $p^2$ in the complex case and it is $p(p+1)/2$ in the real case.

Theorem 3c.3. 

Let $\tilde{X}=\left({\tilde{x}}_{ij}\right)$ be a $p\times q,p\le q$ matrix in the complex domain of rank p and with $pq$ distinct scalar complex variables as elements. Then,

$${\int }_{\tilde{X}}\frac{|\det \left(\tilde{X}{\tilde{X}}^{*}\right){|}^{\gamma }}{{[1+\alpha {\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }]}^{\rho }}\mathrm{d}\tilde{X}=\frac{\Gamma \left(\frac{p}{\delta }(\gamma +q)\right)\Gamma (\rho -\frac{p}{\delta }(\gamma +q))}{\delta {\alpha }^{\frac{p}{\delta }(\gamma +q)}}\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)}{{\tilde{\Gamma }}_p\left(q\right)\Gamma \left(p(\gamma +q)\right)}$$

for $\alpha >0,\delta >0,\Re \left(\rho \right)>\frac{p}{\delta }(\gamma +q),\Re \left(\gamma \right)>-q+p-1$, where ${\tilde{\Gamma }}_p(·)$ is the complex matrix-variate gamma.

The next result will involve a determinant and trace raised to an arbitrary power in the numerator.

Theorem 3.4. 

Let $X,p,q,\delta$ and ρ be as defined in Theorem 3.3. Let $\Re \left(\eta \right)>0$ Then

$$\begin{array}{cc}\hfill {\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }}{{[1+\alpha \left[\mathrm{tr}{\left(X{X}^{\prime }\right)}^{\delta }\right]]}^{\rho }}\mathrm{d}X& =\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})}{{\Gamma }_p\left(\frac{q}{2}\right)\Gamma \left(p(\gamma +\frac{q}{2})\right)}\hfill \\ \hfill & \times \frac{\Gamma \left(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right)\Gamma (\rho -\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ))}{\delta {\alpha }^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}}\hfill \end{array}$$

for $\Re \left(\rho \right)>\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ),\delta >0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\Re \left(\eta \right)>0,\alpha >0$.

The corresponding result in the complex domain is the following:

Theorem 3c.4. 

Let $\tilde{X},p,q,\delta$ and ρ be as defined in Theorem 3c.3. Let $\Re \left(\eta \right)>0$. Then,

$$\begin{array}{cc}\hfill {\int }_{\tilde{X}}\frac{|\det \left(\tilde{X}{\tilde{X}}^{*}\right){|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta }}{{[1+\alpha {\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }]}^{\rho }}& =\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)}{{\tilde{\Gamma }}_p\left(q\right)\Gamma \left(p(\gamma +q)\right)}\hfill \\ \hfill & \times \frac{\Gamma \left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right)\Gamma (\rho -\frac{1}{\delta }(p(\gamma +q)+\eta ))}{\delta {\alpha }^{\frac{1}{\delta }(p(\gamma +q)+\eta )}}\hfill \end{array}$$

for $\Re \left(\rho \right)>\frac{1}{\delta }(p(\Re \left(\gamma \right)+q)+\Re \left(\eta \right)),\delta >0,\Re \left(\eta \right)>0,\alpha >0,\Re \left(\gamma \right)>-q+p-1$.

Note 3.1.. 

Theorems 3.2,3.3,3.4 can be generalized by replacing $X{X}^{\prime }$ by $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ where $M=E\left[X\right],A>O,B>O$ are respectively $p\times p$ and $q\times q$ constant positive definite matrices and $A^{\frac{1}{2}}$ is the positive definite square root of $A>O$. The only change is that the right side of the equations are to be multiplied by ${\left|A\right|}^{-\frac{q}{2}}{\left|B\right|}^{-\frac{p}{2}}$. If $\alpha \to 0$ and if $\rho =\frac{\xi }{\alpha },\xi >0$ then

$${[1+\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{-\frac{\xi }{\alpha }}\to {\mathrm{e}}^{-\xi {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}$$

then the integrands in Theorems 3.2,3.3,3.4 are Mathai’s pathway model [1]. Then, the models in Section 3 go to the models in Section 2. In the complex case one can have the corresponding generalizations. $\tilde{X}{\tilde{X}}^{*}$ may be replaced by $A^{\frac{1}{2}}(\tilde{X}-\tilde{M}B{(\tilde{X}-\tilde{M})}^{*}{A}^{\frac{1}{2}}$ where $\tilde{M}=E\left[\tilde{X}\right],A=A^{*}>O,B=B^{*}>O$ (both Hermitian positive definite) and $A^{\frac{1}{2}}$ denotes the Hermitian positive definite square root of the Hermitian positive definite matrix A. Now, we consider a generalized logistic format or an exponentiated beta form of a matrix-variate integral.

Theorem 3.5. 

Let X be $p\times q,p\le q$ matrix of rank p where the $pq$ elements are distinct real scalar variables. Then,

$$\begin{array}{cc}\hfill {\int }_X\frac{|X{X}^{\prime }{|}^{\gamma }{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}}{{(1+a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }})}^{\alpha +\beta }}\mathrm{d}X& =\frac{{\pi }^{\frac{pq}{2}}{\Gamma }_p(\gamma +\frac{q}{2})}{{\Gamma }_p\left(\frac{q}{2}\right)}\frac{\Gamma \left[\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right]}{\delta \Gamma \left(p(\gamma +\frac{q}{2})\right)}\hfill \\ \hfill & \times \zeta [\left\{(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ),\alpha )\right\}:\alpha +\beta ;;-a]\hfill \end{array}$$

for $\delta >0,0<a<1,\Re \left(\alpha \right)>0,\Re \left(\beta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},\Re \left(\eta \right)>0$ and $\zeta [·]$ is Mathai’s extended zeta function defined in [13], which is also given in Note 3.2 below.

Proof: 

Since $0<a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}<1$, one can use a binomial expansion

$${[1+a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}]}^{-(\alpha +\beta )}=\sum _{k=0}^{\infty }{(\alpha +\beta )}_k\frac{{(-a)}^k}{k!}{\mathrm{e}}^{-k{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}$$

and ${\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}{\mathrm{e}}^{-k{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}={\mathrm{e}}^{-(\alpha +k){\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}$. Now, apply Theorem 2.2 to see the result.

Note 3.2. 

For the real scalar variable x the logistic density is

$$\frac{{\mathrm{e}}^{-x}}{{(1+{\mathrm{e}}^{-x})}^2}=\frac{{\mathrm{e}}^x}{{(1+{\mathrm{e}}^x)}^2},-\infty <x<\infty .$$

(i) 

This density behaves like a standard Gaussian density but the logistic density has a thicker tail compared to that of the standard Gaussian. Hence, in many industrial applications, a logistic model is preferred to a standard Gaussian model. A generalized logistic density was introduced by [14] which is the following:

$$f\left(x\right)=\frac{\Gamma (\alpha +\beta )}{\Gamma \left(\alpha \right)\Gamma \left(\beta \right)}\frac{{\mathrm{e}}^{-\alpha x}}{{(1+{\mathrm{e}}^{-x})}^{\alpha +\beta }},-\infty <x<\infty .$$

(ii) 

This model in $\left(ii\right)$ is more viable and asymmetric situations can also be covered under this generalized model $\left(ii\right)$ compared to $\left(i\right)$. Note that for $\alpha =1=\beta$ in $\left(ii\right)$ we have $\left(i\right)$. Hence, the matrix-variate analogues of logistic-based models are connected to the generalized logistic density in $\left(ii\right)$ above. The model in $\left(ii\right)$ is the exponentiated type-2 beta density. Make the transformation $x={\mathrm{e}}^{-y}$ in a type-2 beta density to go to model $\left(ii\right)$. Matrix-variate versions of logistic-based densities usually end up in an extended form of a generalized zeta function. The zeta function $\zeta \left(\rho \right)$ and the generalized zeta function $\zeta (\rho ,\alpha )$, available in the literature, are the following:

$$z\left(\rho \right)=\sum _{j=1}^{\infty }\frac{1}{k^{\rho }},\Re \left(\rho \right)>1;\zeta (\rho ,\alpha )=\sum _{j=0}^{\infty }\frac{1}{{(\alpha +k)}^{\rho }},\Re \left(\rho \right)>1,\alpha \ne 0,-1,....$$

(iii) 

The extended zeta function defined by [13] is the following:

$$\begin{array}{cc}\hfill {\zeta }_{p,q}^r\left(x\right)& =\zeta [\{(m_1,{\alpha }_1),...,(m_r,{\alpha }_r)\}:a_1,...,a_p;b_1,...,b_q;x]\hfill \\ \hfill & =\sum _{k=0}^{\infty }\frac{1}{{({\alpha }_1+k)}^{m_1}...{({\alpha }_r+k)}^{m_r}}\frac{{\left(a_1\right)}_k...{\left(a_p\right)}_k}{{\left(b_1\right)}_k...{\left(b_q\right)}_k}\frac{x^k}{k!}\hfill \end{array}$$

for ${\sum }_{j=1}^r{m}_r>1,{\alpha }_j\ne 0,-1,...,j=1,...,r;b_j\ne 0,-1,...,j=1,...,q;q\ge p$ or $p=q+1$ and $\left|x\right|<1$, where for example, ${\left(a\right)}_k$ is the Pochhamer symbol defined as ${\left(a\right)}_k=a(a+1)...(a+k-1),a\ne 0,{\left(a\right)}_0=1$. The result corresponding to Theorem 3.5 in the complex case is the following: Details of the derivations are parallel to those in the real case and hence not given here.

Theorem 3c.5. 

Let $\tilde{X}$ be a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements. Then,

$$\begin{array}{cc}\hfill & {\int }_{\tilde{X}}\frac{|\det \left(\tilde{X}{\tilde{X}}^{*}\right){|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }}}{{(1+a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }})}^{\alpha +\beta }}\mathrm{d}\tilde{X}\hfill \\ \hfill & =\frac{{\pi }^{pq}{\tilde{\Gamma }}_p(\gamma +q)\Gamma \left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right)}{\delta {\tilde{\Gamma }}_p\left(q\right)\Gamma \left(p(\gamma +q)\right)}\zeta [\left\{(\left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right),\alpha )\right\}:\alpha +\beta ;;-a]\hfill \end{array}$$

for $\delta >0,0<a<1,\Re \left(\alpha \right)>0,\Re \left(\beta \right)>0,\Re \left(\gamma \right)>-q+p-1,\Re \left(\eta \right)>0$ and $\zeta [·]$ is defined in Theorem 3.5 above.

4. Matrix-variate Type-1 Beta Forms

Let X be a $p\times 1$ vector of distinct real scalar variables. Consider the following multivariate function

$$f_1\left(X\right)=c_1{\left[X^{\prime }X\right]}^{\gamma }{[1-a{\left(X^{\prime }X\right)}^{\delta }]}^{\beta -1},\Re \left(\beta \right)>0,a>0,a{\left(X^{\prime }X\right)}^{\delta }<1$$

that is, X is confined to the interior of the p-dimensional sphere of radius ${\left(\frac{1}{a}\right)}^{\frac{1}{\delta }}$ and $f_1\left(X\right)$ is assumed to be zero outside this sphere. If $c_1$ is the normalizing constant there so that $f_1\left(X\right)$ is a density, let us compute $c_1$. Let $u=X^{\prime }X\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}{u}^{\frac{p}{2}-1}\mathrm{d}u$ by Lemma 2.3. Let $v=u^{\delta }\Rightarrow \mathrm{d}u=\frac{1}{\delta }{v}^{\frac{1}{\delta }-1}\mathrm{d}v$. Then,

$$\begin{array}{cc}\hfill 1& ={\int }_X{f}_1\left(X\right)\mathrm{d}X=c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}{\int }_u{u}^{\gamma +\frac{p}{2}-1}{(1-a{u}^{\delta })}^{\beta -1}\mathrm{d}u\hfill \\ \hfill & =c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}\frac{1}{\delta }{\int }_v{v}^{\frac{1}{\delta }(\gamma +\frac{p}{2})-1}{(1-av)}^{\beta -1}\mathrm{d}v\hfill \\ \hfill & =c_1\frac{{\pi }^{\frac{p}{2}}}{\Gamma \left(\frac{p}{2}\right)}\frac{1}{\delta }\frac{\Gamma \left(\frac{1}{\delta }(\gamma +\frac{p}{2})\right)\Gamma \left(\beta \right)a^{-\frac{1}{\delta }(\gamma +\frac{p}{2})}}{\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))},\Re \left(\gamma \right)>-\frac{p}{2}\Rightarrow \hfill \\ \hfill c_1& =\frac{\delta \Gamma \left(\frac{p}{2}\right)\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))a^{\frac{1}{\delta }(\gamma +\frac{p}{2})}}{{\pi }^{\frac{p}{2}}\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(\gamma +\frac{p}{2})\right)},\Re \left(\gamma \right)>-\frac{p}{2}\hfill \end{array}$$

(4.1)

for $0<a<1,\delta >0,\Re \left(\beta \right)>0$. Note that $f_1\left(X\right)$ is also the density connected with type-1 beta distributed isotropic random points in geometrical probability problems, see [15]. Corresponding format in Section 3 is associated with type-2 beta distributed random points. A more general model is available by replacing $X^{\prime }X$ by ${(X-\mu )}^{\prime }A(X-\mu ),\mu =E\left[X\right],A>O$ is a $p\times p$ real constant positive definite matrix. The only change will be that the normalizing constant $c_1$ will be multiplied by ${\left|A\right|}^{\frac{1}{2}}$ and the structure of the function remains the same.

Theorem 4.1. 

Let X be a $p\times 1$ real vector of distinct real scalar variables as elements. Consider the quadratic form ${(X-\mu )}^{\prime }A(X-\mu ),E\left[X\right]=\mu ,A>O$ where A is a $p\times p$ constant positive definite matrix. Let $0<a<1,\delta >0,0<a{\left[{(X-\mu )}^{\prime }A(X-\mu )\right]}^{\delta }<1,\Re \left(\beta \right)>0,\Re \left(\eta \right)>0$. Then, $c_1$ in

$$f_1\left(X\right)=c_1{\left[\mathrm{tr}\left({(X-\mu )}^{\prime }A(X-\mu )\right)\right]}^{\eta }{[1-a{\left[\mathrm{tr}\left({(X-\mu )}^{\prime }A(X-\mu )\right)\right]}^{\delta }]}^{\beta -1},$$

for $a{\left[\mathrm{tr}\left({(X-\mu )}^{\prime }A(X-\mu )\right)\right]}^{\delta }<1$, and $f_1\left(X\right)=0$ elsewhere, is given by

$$c_1=\frac{{\delta \left|A\right|}^{\frac{1}{2}}\Gamma \left(\frac{p}{2}\right)a^{\frac{1}{\delta }(\gamma +\frac{p}{2})}\Gamma (\beta +\frac{1}{\delta }(\gamma +\frac{p}{2}))}{{\pi }^{\frac{p}{2}}\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(\gamma +\frac{p}{2})\right)},\Re \left(\gamma \right)>-\frac{p}{2}.$$

The density and the normalizing constant in the complex case, corresponding to Theorem 4.1, are the following. The evaluation of the normalizing constant is parallel to that in the real case and hence only the results are given here.

Theorem 4c.1. 

Let $\tilde{X}$ be a $p\times 1$ vector in the complex domain with distinct scalar complex variables as elements. Let $\tilde{u}={(\tilde{X}-\tilde{\mu })}^{*}A(\tilde{X}-\tilde{\mu })$, a Hermitian form where $\tilde{\mu }=E\left[X\right],A=A^{*}>O$ is a constant Hermitian positive definite matrix. Note that the Hermitian form $\tilde{u}$ is real and hence the following function $f_{1c}\left(\tilde{X}\right)$ is real-valued and hence a density when ${\tilde{c}}_1$ is the normalizing constant there. Let $0<a<1,\delta >0,0<\tilde{u}<1,\Re \left(\beta \right)>0,\Re \left(\eta \right)>0$. Then, the density $f_{1c}\left(\tilde{X}\right)$ and the normalizing constant ${\tilde{c}}_1$ are the following:

$$f_{1c}\left(\tilde{X}\right)={\tilde{c}}_1{\left[u\right]}^{\eta }{[1-a{u}^{\delta }]}^{\beta -1},u={(\tilde{X}-\tilde{\mu })}^{*}A(\tilde{X}-\tilde{\mu }),a<u^{\delta }<1$$

(4c.1) 

and $f_{1c}\left(\tilde{X}\right)=0$ elsewhere, and

$${\tilde{c}}_1=\frac{\delta \left|\det \left(A\right)\right|\Gamma \left(p\right)a^{\frac{1}{\delta }(\gamma +p)}\Gamma (\beta +\frac{1}{\delta }(\gamma +p))}{{\pi }^p\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(\gamma +p)\right)},\Re \left(\gamma \right)>-p.$$

Now, consider $X=\left(x_{ij}\right)$ a $p\times q,p\le q$ matrix of rank p where the $pq$ elements $x_{ij}$’s are distinct real scalar variables. Consider the model

$$f_2\left(X\right)=c_2{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }{[1-a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{\beta -1},$$

(4.2)

for $a>0,\delta >0,\Re \left(\beta \right)>0,\Re \left(\eta \right)>-\frac{pq}{2},a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1$ and $f_2\left(X\right)=0$ outside this sphere. Note that $\mathrm{tr}\left(X{X}^{\prime }\right)=$ sum of squares of $pq$ real scalar variables here and $u=\mathrm{tr}\left(X{X}^{\prime }\right)\Rightarrow \mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{\Gamma \left(\frac{pq}{2}\right)}{u}^{\frac{pq}{2}-1}\mathrm{d}u.$ Then, proceeding as in the derivation of $c_1$ in $f_1\left(X\right)$, or in Theorem 4.1, we have the following:

Theorem 4.2. 

Let $f_2\left(X\right)$ be as defined in (4.2) for X a real $p\times q,p\le q$ matrix of rank p, then the normalizing constant $c_2$ in $f_2\left(X\right)$ is given by

$$c_2=\frac{\Gamma \left(\frac{pq}{2}\right)}{{\pi }^{\frac{pq}{2}}}\frac{\delta \Gamma (\beta +\frac{1}{\delta }(\eta +\frac{pq}{2}))a^{\frac{1}{\delta }(\eta +\frac{pq}{2})}}{\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(\gamma +\frac{pq}{2})\right)}$$

(4.3)

for $\Re \left(\beta \right)>0,a>0,\delta >0,\Re \left(\eta \right)>-\frac{pq}{2}$.

In the corresponding complex case the result is the following:

Theorem 4c.2. 

Let $\tilde{X}$ be $p\times q,p\le q$ matrix in the complex domain of rank p where the $pq$ elements are distinct scalar complex variables. Then, the following function $f_{2c}\left(\tilde{X}\right)$ is a density:

$$f_{2c}\left(\tilde{X}\right)={\tilde{c}}_2{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta }{[1-a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }]}^{\beta -1},0<a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }<1$$

(4c.2) 

for $a>0,\delta >0,\Re \left(\eta \right)>-pq,\Re \left(\beta \right)>0$ and $f_{2c}\left(\tilde{X}\right)=0$ elsewhere, where

$${\tilde{c}}_2=\frac{\Gamma \left(pq\right)\delta \Gamma (\beta +\frac{1}{\delta }(\gamma +pq))a^{\frac{1}{\delta }(\gamma +pq)}}{{\pi }^{pq}\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(\gamma +pq)\right)}$$

(4c.3) 

for $\Re \left(\beta \right)>0,0<a<1,\delta >0,\Re \left(\eta \right)>-pq$.

A more general model is available in the real case by replacing $X{X}^{\prime }$ by $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ where $A>O,B>O$ are constant $p\times p$ and $q\times q$ matrices respectively and $M=E\left[X\right]$. Consider the transformation $Y=A^{\frac{1}{2}}(X-M)B^{\frac{1}{2}}\Rightarrow \mathrm{d}Y={\left|A\right|}^{\frac{q}{2}}|B^{\frac{p}{2}}\mathrm{d}X$ by Lemma 2.1. Then, the density of Y is the same as $f_2\left(X\right)$ of (4.3). Hence, the only change will be that the normalizing constant in (4.3) is multiplied by ${\left|A\right|}^{\frac{q}{2}}{\left|B\right|}^{\frac{p}{2}}$. Therefore this case is not listed here separately. A similar comment holds in the complex case also. In the complex case the multiplicative factor is ${\left|\det \left(A\right)\right|}^q{\left|\det \left(B\right)\right|}^p$.

Note 4.1. 

From the model in Theorem 4.2, one can easily evaluate the density of $\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]$ or in the general case the density of $\left[\mathrm{tr}\left(A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}\right)\right]$ from the normalizing constant $c_2$. Treating $c_2=c_2\left(\eta \right)$, we have $E\left[{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^h\right]=\frac{c_2\left(\eta \right)}{c_2(\eta +h)}$ for an arbitrary h. Hence, from the inverse Mellin transform one has the density of $\mathrm{tr}\left(X{X}^{\prime }\right)$ or that in the general case. The same comment holds for Theorem 4.1 also. Similar comments hold in the complex case also.

If the multiplicative factor ${\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }$ in the real case is replaced by a determinant $|X{X}^{\prime }{|}^{\gamma }$ let us see what happens to such a model. Again, let X be a $p\times q,p\le q$ matrix of rank p with distinct $pq$ real scalar variables as elements. Consider the model

$$f_3\left(X\right)=c_3{\left|X{X}^{\prime }\right|}^{\gamma }{[1-a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{\beta -1},$$

(4.4)

for $\Re \left(\beta \right)>0,0<a<1,a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1,\delta >0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$ and $f_3\left(X\right)=0$ elsewhere. Then, we have the following result:

Theorem 4.3. 

Let X and the parameters be as defined in (4.4). Then,

$$c_3=\frac{{\Gamma }_p\left(\frac{q}{2}\right)\delta a^{\frac{p}{\delta }(\gamma +\frac{q}{2})}\Gamma (\beta +\frac{p}{\delta }(\gamma +\frac{q}{2}))\Gamma \left(p(\gamma +\frac{q}{2})\right)}{{\pi }^{\frac{pq}{2}}\Gamma \left(\beta \right)\Gamma \left(\frac{p}{\delta }(\gamma +\frac{q}{2})\right){\Gamma }_p(\gamma +\frac{q}{2})}$$

for $\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},0<a<1,\delta >0,\Re \left(\beta \right)>0,p\le q.$

Proof: 

Let $X=T{U}_1$ where $T=\left(t_{ij}\right)$ is a lower triangular matrix and $U_1$ is a semi-orthonormal matrix, $U_1{U}_1^{\prime }=I_p$, where both T and $U_1$ uniquely chosen. Then, from Lemma 2.4, after integrating out the differential element corresponding to the semi-orthonormal matrix $U_1$, one has the relationship

$$\mathrm{d}X=\frac{{\pi }^{\frac{pq}{2}}}{{\Gamma }_p\left(\frac{q}{2}\right)}[\prod _{j=1}^p|t_{jj}{|}^{q-j}]\mathrm{d}T,X{X}^{\prime }=T{T}^{\prime }.$$

(i) 

Note that $|X{X}^{\prime }|=|T{T}^{\prime }|={\prod }_{j=1}^p{t}_{jj}^2$ and

$$|T{T}^{\prime }{|}^{\gamma }\{\prod _{j=1}^p|t_{jj}{|}^{q-j}\}=\prod _{j=1}^p{\left(t_{jj}^2\right)}^{\gamma +\frac{q}{2}-\frac{j}{2}},\mathrm{tr}\left(T{T}^{\prime }\right)=\sum _{j=1}^p{t}_{jj}^2+\sum _{i>j}{t}_{ij}^2$$

(ii) 

where in ${\sum }_{i>j}{t}_{ij}^2$ there are $p(p-1)/2$ terms. Consider a polar coordinate transformation on all these $p(p+1)/2$ terms $t_{ij}$’s. $\{t_{11},t_{22},...,t_{pp},t_{21},...,t_{pp-1}\}\to \{r,{\theta }_1,...,{\theta }_{k-1}\},k=p(p+1)/2$. Then, the Jacobian element is already discussed in the proof of Theorem 2.1. r has exponent $k-1$ in the Jacobian element. Then, collecting all factors containing r in the transformed $f_3\left(X\right)$ we have

$${\left(r^2\right)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{[1-a{\left(r^2\right)}^{\delta }]}^{\beta -1}$$

and the integral over r gives the following:

$${\int }_0^{\infty }{\left(r^2\right)}^{p(\gamma +\frac{q}{2})-\frac{1}{2}}{(1-a{\left(r^2\right)}^{\delta })}^{\beta -1}\mathrm{d}r=\frac{1}{2\delta }{a}^{-\frac{p}{\delta }(\gamma +\frac{q}{2})}\frac{\Gamma \left(\frac{p}{\delta }(\gamma +\frac{q}{2})\right)\Gamma \left(\beta \right)}{\Gamma (\beta +\frac{p}{\delta }(\gamma +\frac{q}{2}))}$$

(iii) 

for $\Re \left(\beta \right)>0,a>0,\delta >0,\Re \left(\gamma \right)>-\frac{q}{2}$. The integral over all the sine and cosine product is available from the proof of Theorem 2.1, which is $2{\Gamma }_p(\gamma +\frac{q}{2})/\Gamma \left(p(\gamma +\frac{q}{2})\right)$ for $\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$. Taking the product with that in $\left(iii\right)$ establishes the theorem. In the complex case the density and the normalizing constant are the following:

$$f_{3c}(\tilde{X}={\tilde{c}}_3{\left|\det \left(\tilde{X}{\tilde{X}}^{*}\right)\right|}^{\gamma }{[1-a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }]}^{\beta -1}$$

(4c.4) 

for $\tilde{X}$ a $p\times q,p\le q$ matrix of rank p in the complex domain with distinct $pq$ complex scalar variables as elements such that $a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }<1,a>0,\delta >0,\Re \left(\gamma \right)>-q+p-1$ and $f_{3c}\left(\tilde{X}\right)=0$ elsewhere. Then, the normalizing constant ${\tilde{c}}_3$ is available from the following theorem.

Theorem 4c.3. 

For $\tilde{X},p,q,\delta ,a,\gamma$ as defined in (4c.4) and following through the derivation parallel to that in the real case, the normalizing constant ${\tilde{c}}_3$ is the following:

$${\tilde{c}}_3=\frac{{\tilde{\Gamma }}_p\left(q\right)\delta a^{\frac{p}{\delta }(\gamma +q)}\Gamma (\beta +\frac{p}{\delta }(\gamma +q))\Gamma \left(p(\gamma +q)\right)}{{\pi }^{pq}\Gamma \left(\beta \right)\Gamma \left(\frac{p}{\delta }(\gamma +q)\right){\tilde{\Gamma }}_p(\gamma +q)}$$

for $\Re \left(\gamma \right)>-q+p-1,a>0,\delta >0,\Re \left(\beta \right)>0,p\le q$.

As explained in Note 4.1, arbitrary moments and exact density of $|X{X}^{\prime }|$ or its general form $|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|$ in the real case are available from the normalizing constant $c_3$. Corresponding comment holds in the complex case also.

Note that here also a more general model in the real case is available by replacing $X{X}^{\prime }$ by $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}$ as mentioned before. The only change will be that the normalizing constant will be multiplied by ${\left|A\right|}^{\frac{q}{2}}{\left|B\right|}^{\frac{p}{2}}$. Hence, this general case is not listed here separately. In the complex case, the multiplicative factor is ${\left|\det \left(A\right)\right|}^q{\left|\det \left(B\right)\right|}^p$. A more general case is available by introducing another factor containing a trace also into $f_3\left(X\right)$. Consider the following model in the real case:

$$f_4\left(X\right)=c_4{\left|X{X}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }{[1-a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{\beta -1},a>0,\delta >0,a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1$$

(4.5)

for $\Re \left(\beta \right)>0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2},X$ is $p\times q,p\le q$ and of rank p, $a>0,a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1,\delta >0$ and $f_4\left(X\right)=0$ outside this sphere. Proceeding exactly as in the proof of Theorem 4.3, we have the following result:

Theorem 4.4. 

Let $X,p,q,\delta ,\eta ,a,\gamma$ be as defined in (4.5) and $\Re \left(\eta \right)>0$. Then, the normalizing constant $c_4$ is given by

$$c_4=\frac{{\Gamma }_p\left(\frac{q}{2}\right)}{{\pi }^{\frac{pq}{2}}}\frac{\delta a^{\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )}\Gamma (\beta +\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta ))\Gamma \left(p(\gamma +\frac{q}{2})\right)}{\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right){\Gamma }_p(\gamma +\frac{q}{2})}.$$

Model corresponding to the one in (4.5) in the complex case, is the following, where $\tilde{X}$ is a $p\times q,p\le q$ matrix in the complex domain of rank p with $pq$ distinct scalar complex variables as elements such that $a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }<1,a>0,\delta >0$, the parameters are such that $\Re \left(\gamma \right)>-q+p-1,\Re \left(\eta \right)>0,\Re \left(\beta \right)>0$:

$$f_{4c}\left(\tilde{X}\right)={\tilde{c}}_4{\left|\det \left(\tilde{X}{\tilde{X}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta }{[1-a{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }]}^{\beta -1}$$

(4c.5) 

and $f_{4c}\left(\tilde{X}\right)=0$ elsewhere. By using steps parallel to those in the derivation of the normalizing constant $c_4$ in the real case, one can see that the normalizing constant in the complex case is given in the following theorem:

Theorem 4c.4. 

For $\tilde{X},a,\delta ,\eta ,\delta ,\gamma$ as defined in (4c.5), the normalizing constant ${\tilde{c}}_4$ is the following:

$${\tilde{c}}_4=\frac{{\tilde{\Gamma }}_p\left(q\right)\delta a^{\frac{1}{\delta }(p(\gamma +q)+\eta )}\Gamma (\beta +\frac{1}{\delta }(p(\gamma +q)+\eta ))\Gamma \left(p(\gamma +q)\right)}{{\pi }^{pq}\Gamma \left(\beta \right)\Gamma \left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right){\tilde{\Gamma }}_p(\gamma +q)}.$$

Again, a more general model in the real case is available by replacing $X{X}^{\prime }$ by $A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}},A>O,B>O,M=E\left[X\right]$. Also, from the structure of $f_4\left(X\right)$ it is clear that the exact density and arbitrary moments of the determinant $\left|XX\right|$ or $|A^{\frac{1}{2}}(X-M)B{(X-M)}^{\prime }{A}^{\frac{1}{2}}|$ is available by replacing the parameter $\gamma$ by $\gamma +h$ and then taking the ratio of the normalizing constant $c_4$ as explained before. Similarly, the exact density or arbitrary moments of $\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]$ or its general form is available by replacing $\eta$ by $\eta +h$ and taking the ratio of the normalizing constant $c_4$. Similar comments hold in the complex domain also, the multiplicative factor will be ${\left|\det \left(A\right)\right|}^q{\left|\det \left(B\right)\right|}^p$ in the complex case.

Now, we will consider an exponentiated type-1 beta type model. Again, let X be $p\times q,p\le q$ matrix of rank p with $pq$ distinct real scalar variables as elements. Consider the model

$$f_5\left(X\right)=c_5{\left|X{X}^{\prime }\right|}^{\gamma }{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}{[1-a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}]}^{\beta -1}$$

(4.6)

for $a>0,a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1,\delta >0,\Re \left(\beta \right)>0,\Re \left(\alpha \right)>0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$ and $f_5\left(X\right)=0$ outside the sphere. Then, we have the following result:

Theorem 4.5. 

Let X and the parameters be as defined in $f_5\left(X\right)$. Then, the normalizing constant $c_5$ is given by the following:

$$\begin{array}{cc}\hfill c_5& =\frac{{\Gamma }_p\left(\frac{q}{2}\right)\delta \Gamma \left(p(\gamma +\frac{q}{2})\right)}{{\pi }^{\frac{pq}{2}}\Gamma \left(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right){\Gamma }_p(\gamma +\frac{q}{2})}\hfill \\ \hfill & \times {\left[\zeta [\left\{(\left(\frac{1}{\delta }(p(\gamma +\frac{q}{2})+\eta )\right),\alpha )\right\}:1-\beta ;;a]\right]}^{-1}\hfill \end{array}$$

for $\delta >0,a>0,\Re \left(\alpha \right)>0,\Re \left(\beta \right)>0,\Re \left(\eta \right)>0,\Re \left(\gamma \right)>-\frac{q}{2}+\frac{p-1}{2}$, where $\zeta (·)$ is the extended zeta function defined in Theorem 3.5.

The proof is straightforward. Since $0<a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }<1$ we can use a binomial expansion and write

$${[1-a{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }]}^{\beta -1}=\sum _{k=0}^{\infty }{(1-\beta )}_k\frac{a^k}{k!}{\mathrm{e}}^{-k{\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}$$

(i) 

Now, the exponential trace part joins with the exponential trace part remaining in $f_5\left(X\right)$ becoming ${\mathrm{e}}^{-(\alpha +k){\left[\mathrm{tr}\left(X{X}^{\prime }\right)\right]}^{\delta }}$. Then, one can integrate out by using Theorem 4.4 by replacing $\alpha$ there by $\alpha +k$ and interpreting the result in terms of an extended zeta function, Theorem 4.5 is established. The corresponding model in the complex domain is the following:

$$f_{5c}(\tilde{X}={\tilde{c}}_5{\left|\det \left(\tilde{X}{\tilde{X}}^{*}\right)\right|}^{\gamma }{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\eta }{\mathrm{e}}^{-\alpha {\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }}{[1-a{\mathrm{e}}^{-{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }}]}^{\beta -1}$$

(4c.6) 

for $\tilde{X}$ a $p\times q,p\le q$ and of rank p matrix in the complex domain with $pq$ distinct scalar complex variables as elements, $a>0,\delta >0,a[{\mathrm{e}}^{-{\left[\mathrm{tr}\left(\tilde{X}{\tilde{X}}^{*}\right)\right]}^{\delta }}<1,\Re \left(\eta \right)>0,\Re \left(\beta \right)>0,\Re \left(\gamma \right)>-q+p-1$ and $f_{5c}\left(\tilde{X}\right)=0$ elsewhere. Then, following through the derivation parallel to that in the real case, the normalizing constant ${\tilde{c}}_5$ is the following:

Theorem 4c.5. 

Under the conditions stated in (4c.6),

$$\begin{array}{cc}\hfill {\tilde{c}}_5& =\frac{{\tilde{\Gamma }}_p\left(q\right)\delta \Gamma \left(p(\gamma +q)\right)}{{\pi }^{pq}\Gamma \left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right){\tilde{\Gamma }}_p(\gamma +q)}\hfill \\ \hfill & \times {\left[\zeta [\left\{(\left(\frac{1}{\delta }(p(\gamma +q)+\eta )\right),\alpha )\right\}:1-\beta ;;a]\right]}^{-1}.\hfill \end{array}$$