The Completed Density Functional Theory (cDFT) for Accurate Description and Prediction of Properties of Materials

1. Introduction

The first foundational theorem of density functional theory, as stated by Rothenberg and Kohn [1], follows. “Let an arbitrary number of electrons in a box be subject to an external potential and the electron-electron Coulomb repulsion, then, this potential is a unique functional of the electron density, except for an additive constant.” The first corollary of this theorem is that the energy functional, E_v[Ψ’] = (Ψ’, HΨ’), is a unique functional of the electron density. This energy is the sum of the occupied energies in the spectrum of the Hamiltonian. In 2014, Bagayoko [2] introduced the second corollary of the above theorem, namely, that the spectrum of the Hamiltonian is a unique functional of the electron charge density.

The second DFT theorem, also known as the DFT variational principle, states that the energy functional reaches its minimum when the electron density is that of the ground state. Specifically, according to Hohenberg and Kohn, “It is well known that for a system of N particles, the energy functional of Ψ′.

Ev[Ψ′] = (Ψ′, VΨ′) + (Ψ′, (T+U) Ψ′)

(1)

has a minimum at the correct ground state Ψ, relative to arbitrary variations of Ψ′ in which the total number of particles is kept constant.” We added emphasis with bold letters in the quote. In the formula, the theorem is:

$${\displaystyle \int v(\mathbf{r})n\text{'}(\mathbf{r})d\mathbf{r}+F[n\text{'}]}>{\displaystyle \int v(\mathbf{r})n(\mathbf{r})d\mathbf{r}+F[n]}$$

(2)

where n(r) is the ground state density and n’(r) is any other density different from n(r). Let us note that the same potential v(r) is on both sides of the inequality. Any reasonable basis set Ψ’, upon reaching self-consistency, leads to a corresponding electron density n’(r). “Reasonable” in this context means that the referenced basis set accounts for all the electrons in the system. For any given system, there exists an infinite number of reasonable basis sets. Each one of them produces a corresponding charge density upon the attainment of self-consistency:

$$\mathit{n}\left(\mathit{r}\right)={\sum }_{\mathit{i}=1}^{\mathit{N}\mathit{o}\mathit{c}\mathit{c}}{\left|{\mathit{\psi }}_{\mathit{i}}\left(\mathit{r}\right)\right|}^2$$

(3)

where Nocc is the number of occupied states.

2. The Incompleteness of The Original Dft

As per the second theorem, any calculation that purports to utilize density functional theory for predicting or describing electronic and related properties of materials has to utilize the ground state charge density in computations. Unfortunately, the 3-dimensional ground state charge densities of atoms, molecules, semiconductors, insulators, and metals are à priori unknown.

Throughout the world, researchers have been following a suggestion made by Kohn and Sham [3]. It consists of judiciously selecting a basis set to perform self-consistent iteration in order to produce results that are taken to represent those of the ground state. Unfortunately, a state of a material obtained with self-consistency iterations with a single basis set is a stationary one among an infinite number of such states.

It should be noted that before the above suggestion, Kohn and Sham [3] emphatically stated that the four equations defining the local density approximation (LDA) are the ones to be solved simultaneously. Once the expression of the exchange-correlation energy is known, its functional derivative provides the potential. Then, the four equations are reduced to the Kohn-Sham equation and the one giving the charge density in terms of the wave functions of the occupied states, as in Equation 3. With the knowledge of the Rayleigh theorem for eigenvalues, solving these two equations simultaneously requires the variation of the size of the basis set [4-5], along with its radial features and angular richness. The “arbitrary variations” in the second theorem are constrained only by the “reasonableness” defined above.

Almost universally, results of calculations using a single basis set have disagreed with experiments for atoms, molecules, semiconductors, and insulators –for the energy or band gaps. Specifically, they generally underestimate by 30-50% or more the measured energy gaps (for finite systems) and band gaps for crystalline semiconductors and insulators.

3. The Completed DfT (cDFT)

In 1998, Bagayoko and collaborators introduced a computational method radically different from the mainstream utilization of a single basis set for DFT calculations. In 1999, Zhao, Bagayoko, and Williams [5] applied the method to study successfully GaN, carbon, silicon, and RuO2. The resulting Bagayoko, Zhao, and Williams (BZW) method was inspired by Bagayoko’s utilization of the Rayleigh theorem for eigenvalues in performing calculations with contracted basis sets [6]. This theorem states that when the same eigenvalue equation is solved with N and (N+1) basis functions, where the N functions are included in the (N+1), then the resulting, ordered eigenvalues satisfy the inequality:

Ei^(N+1) ≤ Ei^N for i=1 to N.

(4)

Clearly, this theorem states that successive augmentation of basis sets leads to the steady lowering of the eigenvalues down to their ground state values! The question then becomes one of knowing when the ground state is reached. We simply perform successive calculations with augmented basis sets up to the point where three (3) consecutive ones produce the same occupied energies. Those stable occupied energies represent the true ground state of the system under study.

At first, the Bagayoko, Zhao, and Williams (BZW) method augmented a basis set by adding orbitals in the increasing order of the excited energies they represent. Subsequent works by Ekuma [7] and Franklin [8] added orbitals differently: For a given quantum number n, the orbitals are added in the p, d, f, and s order, if applicable, instead of that of increasing energy. “If applicable” means that said orbital was occupied in one of the neutral atoms in the system under study. This enhancement by Ekuma and Franklin led to the BZW-EF method. This counter-intuitive approach led to descriptions and predictions in excellent agreement with experimental findings. This “excellent agreement” stems in part from the fact that for valence electrons, each one of which is under the influence two or more atomic or ionic sites, polarization has primacy over spherical symmetry. Further examination of the original DFT paper [1] revealed that, indeed, one could have “arbitrary variations” of the reasonable basis set provided the total number of particles is kept constant - according to the second DFT theorem.

The above process of the BZW and BZW-EF method completes DFT by providing a rigorous algorithm for finding the true ground state of the system under study. The first basis set to produce the ground state, as confirmed by two subsequent calculations, is the one providing the completed DFT (cDFT) description of the material under study. The basis set for this calculation is known as the optimal basis set. Upon the attainment of self-consistency, it produces the true ground state charge density. The calculations following it also yield the ground state. However, any eigenvalue produced by these subsequent calculations and that is not obtained with the optimal basis set, does not belong to the spectrum of the Hamiltonian, a unique functional of the ground state charge density - as per the second corollary of the first DFT theorem.

4. Recommendations for Using cDFT

There are distinct issues that mainstream condensed matter theory community should address to realize the presently missed benefits of cDFT, including the attainment of the objectives of the Materials Genome Initiative (MGI) [9].

4.1. cDFT Functional

The first of these issues is the recognition that while the exact functional of DFT (or cDFT) is not known, good to excellent approximations exist that are not presently utilized properly. Indeed, even the exact functional will lead to energy gaps (for finite systems) or band gaps (for crystalline systems) that disagree with corresponding experimental findings - as long as the calculations do not verifiably reach the ground states of materials under study. Specifically, the Ceperley and Alder functional [10], with the parameterization of Vosko, Wilk, and Nusair (VWN) [11], has led to accurate descriptions of electronic and related properties of more than 30 semiconductors. With the VWN potential, cDFT has correctly described these materials and predicted band gaps and other properties for several semiconductors [2]. In the spectacular case of rutile TiO2[12], our prediction of an indirect band gap, subsequently confirmed, was contrary not only to findings from mainstream DFT calculations but also to experimental results.

4.2. Ad hoc Potentials

The condensed matter theory community, for the second issue, could dramatically reduce the use and the development of ad hoc potentials. While these potentials have served us well in describing some materials, it should be recognized that they have no predictive capacity. Indeed, as underscored by Bagayoko [2], any potential not straightforwardly obtained as the functional derivative of an exchange-correlation energy is not entirely a DFT potential. We posit that the need for ad hoc potential stemmed from the failure of calculations using the incomplete DFT, i.e., ones where iterations with a single basis set are erroneously presumed to lead to the ground states of materials under study. In light of the highly accurate results from cDFT, as illustrated elsewhere [2, 13] and below, the current investment of talents, efforts, and funds in the development of ad hoc potential should be drastically reduced. Medvedev et al. [14] noted that “density functional theory is straying from the path toward an exact functional.” Indeed, after examining some 128 functionals, they found that up to the year 2000, densities resulting from developed functionals were closer to corresponding, exact ones. After 2000, however, “this trend was reversed by unconstrained functionals sacrificing physical rigor for the flexibility of empirical fitting [14].”

4.3. The Need for Exponential and Gaussian Functions and the Attainment of the Ground State

The angular richness rendered by spherical or cubic harmonics cannot be obtained with plane waves. Additionally, the adequacy of the radial parts of exponential or Gaussian orbitals in describing materials cannot be equaled by plane waves. Consequently, the full implementation of cDFT requires software packages utilizing (a) full potentials (b) exponential or Gaussian orbitals, and (c) the flexibility of augmenting the basis set, one orbital at a time, to reach the true ground state of a material under study. While pseudopotential and plane wave program packages have served the community well, obtaining calculated results that possess the full physical content of cDFT and agree with corresponding, experimental ones requires (a), (b), and (c) as spelled out above.

Several of our cDFT calculations provide a table [7-8] showing the successive augmentation of the basis set, one orbital at a time, for the consecutive, self-consistent calculations. The highest number of such calculations for all the materials we have studied to date is eight (8). For several semiconductors, this number is between five (5) and seven (7). While mainstream calculations utilize iterations with a single basis set to reach stationary states, the noted augmentation of the basis set lowers the occupied energies down to the ground state of a material. With machine learning (ML), the referenced augmentation of the basis set can be significantly facilitated, even for large molecules with different atomic species or supercell calculations for materials with no translation symmetry.

4.4. The Role of Funding Sources and of Journals

Most reputable funding sources and journals have knowledgeable program officers and manuscript referees, respectively. If public and private funding sources do not encourage the development of software packages that permit cDFT calculations, the attainment of the objectives of the Materials Genome Initiative (MGI) may remain in doubt. Similarly, the funding of calculations that adhere to (a) through (c) above will be necessary for significant progress in the realization of the potentials of cDFT in accurately describing and predicting properties of materials, including organic ones.

In 2016, Bagayoko published the correct understanding and the completion [15] of the relativistic generalization of density functional theory by Rajagopal and Callaway [16]. As was the case for non-relativistic DFT, the correct application of the relativistic one requires the verifiable attainment of the ground state – with successive augmentation of the four (4) component spinors basis set. Upon reaching the true ground state, self-consistent iterations lead to the ground state current density whose fourth component gives the ground state charge density. Presently, we do not know of a single semi-relativistic program package, let alone a fully relativistic one, that permits the successive augmentation of the spinor basis set to search for the true ground state. The development of a fully relativistic program package for electronic structure calculations, either for atoms, molecules, or for solids, is expected to require a significant funding. Much remains to be theoretically understood about the electronic and related properties of heavy elements and of materials containing them, including several high-temperature superconductors.

Journal editors and manuscript referees may have to abandon some beliefs that arose as the result of the incorrect application of the incomplete, original density functional theory. Indeed, the derivative discontinuity of the exchange-correlation energy was partly introduced to remedy the failure of mainstream calculations to produce band gaps that agree with corresponding, experimental ones. Many of these mainstream band gaps are 30 to 50% or more smaller than the corresponding measured ones. As noted above, cDFT calculations, by strictly adhering to the fundamental theorems of DFT, produce the actual ground states of the systems under study. In doing so, they obtain results that possess the full physical content of DFT and agree with corresponding experimental ones.

5. Illustrative Results from cDFT

With the BZW and BZW-EF methods, we have correctly predicted electronic and related properties of cubic silicon nitride rutile TiO2[12], (c-Si3N4) [17], cubic indium nitride (c-InN) [18], wurtzite InN [19], and cubic magnesium silicide (Mg2Si) [20]. Of course, the band gap is a central electronic property these calculations accurately predicted. The table below provides illustrative examples of results for small to large band gaps semiconductors and BeO, an insulator.

Table 1. Band gaps from ab initio, self-consistent DFT calculations in the literature for the identified materials. The number of calculations before our cDFT ones for each material is in Colum 3. Column 4 shows the range of the band gap values obtained by these previous DFT calculations. These previous values are all underestimates of the corresponding experimental values in Column 5, which agree with our cDFT results in Column 6,

Material	DFT Potential	Number of Calculated Gaps	Ranges of Calculated Gaps	Experimental Gaps	DFT BZW orBZW-EF Gaps
Cubic InN	LDA & GGA	10	-0.55 - + 0.08	0.61 eV	0.65 eV18
c-Mg2Si	LDA &GGA	10	0.12 – 0.42	0.65 – 0.80 eV	0.89 eV20
w-AlN	LDA & GGA	11	3.9 – 4.78 eV	6.2 – 6.2 ±0.2 eV	6.28 eV21
zb-ZnS	LDA & GGA	5	1.65 – 2.37 eV	3.723 eV	3.725 eV22
w-GaN	LDA & GGA	17	1.68 – 2.52 eV	3.3-3.5 eV	3.20 & 3.29 eV23
w-BeO	LDA	9	7.0 – 7.8 eV	10.24 – 10.63$\pm$0.10 eV	10.3 eV24
Rutile TiO2	LDA & GGA	18	1.67 – 2.12 eV	3.00 – 3.10 eV	2.95 & 3.05 eV12
w-ZnO	LDA	12	0.23 – 2.26 eV	3.30 – 3.40 eV	3.39 eV8
zb-BP	LDA & GGA	15	1.11 – 1.38 eV	2.02±0.05 eV	2.02 eV25
c-BN	LDA & GGA	9	4.20 – 4.47 eV	6.20 – 6.4±0.5 eV	6.48 eV26

Table 1. Band gaps from ab initio, self-consistent DFT calculations in the literature for the identified materials. The number of calculations before our cDFT ones for each material is in Colum 3. Column 4 shows the range of the band gap values obtained by these previous DFT calculations. These previous values are all underestimates of the corresponding experimental values in Column 5, which agree with our cDFT results in Column 6,

Material	DFT Potential	Number of Calculated Gaps	Ranges of Calculated Gaps	Experimental Gaps	DFT BZW orBZW-EF Gaps
Cubic InN	LDA & GGA	10	-0.55 - + 0.08	0.61 eV	0.65 eV18
c-Mg2Si	LDA &GGA	10	0.12 – 0.42	0.65 – 0.80 eV	0.89 eV20
w-AlN	LDA & GGA	11	3.9 – 4.78 eV	6.2 – 6.2 ±0.2 eV	6.28 eV21
zb-ZnS	LDA & GGA	5	1.65 – 2.37 eV	3.723 eV	3.725 eV22
w-GaN	LDA & GGA	17	1.68 – 2.52 eV	3.3-3.5 eV	3.20 & 3.29 eV23
w-BeO	LDA	9	7.0 – 7.8 eV	10.24 – 10.63$\pm$0.10 eV	10.3 eV24
Rutile TiO2	LDA & GGA	18	1.67 – 2.12 eV	3.00 – 3.10 eV	2.95 & 3.05 eV12
w-ZnO	LDA	12	0.23 – 2.26 eV	3.30 – 3.40 eV	3.39 eV8
zb-BP	LDA & GGA	15	1.11 – 1.38 eV	2.02±0.05 eV	2.02 eV25
c-BN	LDA & GGA	9	4.20 – 4.47 eV	6.20 – 6.4±0.5 eV	6.48 eV26

Once a calculation leads to an erroneous band gap, it can no longer continue to obtain accurate dielectric or optical properties. The effective mass for the electron is highly sensitive to the curvature around the minimum of the unoccupied bands. With the correct electronic structure and related wave functions, most other properties of materials can be studied theoretically. Several publications from our group, as per their titles, provide predictions of electronic and related properties of materials. We urge readers to consult AIP Advances 2014 [2], MRS Advances 2023 [13], and the 2024 Proceedings of MSAS [27] for numerous examples of accurate predictions or descriptions of electronic and related properties of materials, with emphasis on semiconductors.

6. Conclusions

We note that the above results, and the many others in the indicated publications, were obtained using cDFT. With successively augmented basis sets, we searched for and verifiably reached the ground states of the materials under study. In doing so, we did not need to invoke self-interaction correction or the derivative discontinuity of the exchange-correlation energy. As for the latter, Bagayoko proved [2] that it cannot be relevant to a solution of the Kohn-Sham equation. Indeed, the derivative discontinuity was derived for semiconductors and for insulators using systems whose total numbers of particles were not constant; the Kohn-Sham equation is obtained by setting the variation of the total number of particles equal to zero! Bagayoko added that while the referenced derivative discontinuities may be useful in some ways, they have no bearing on the calculated band gaps of materials. The quasi-universal underestimation of energy and band gaps by mainstream calculations stems squarely from their utilization of a single basis set to perform self-consistent calculations whose stationary states are mistakenly taken to be the ground state of the materials.