Constraint Unlearnability in Overparameterized Neural Networks

1. Introduction

Neural networks have achieved tremendous success across many domains. This success has been accompanied by the widespread use of architectural inductive biases that enforce pointwise constraints: softmax, for instance, ensures that outputs sum to one at every input. These biases are ubiquitous in practice—but given the expressive power of overparameterized networks, are they mathematically necessary, or merely an engineering choice?

Consider probability normalization. A network that outputs a distribution must satisfy ${\sum }_i{f}_{\theta }{\left(x\right)}_i=1$ for every input x. Softmax guarantees this by construction. Yet universal approximation tells us that a sufficiently wide network can approximate any continuous function, including those that satisfy this constraint. Whether the network could learn to satisfy it on its own, without the bias, is the question we address.

We prove that it cannot. For overparameterized feedforward neural networks with real-analytic activation functions (including GELU and Swish, which dominate modern architectures), in the absence of the corresponding inductive bias (i.e., no architectural guarantee that the constraint holds for all parameters), the set of parameters for which the network satisfies a real-analytic pointwise constraint has measure zero on the manifold of global minima (assuming this manifold is connected; see Section 3.2). Since gradient descent converges to this manifold, it almost surely finds solutions that violate the constraint. We call this phenomenon constraint unlearnability: the network cannot learn to satisfy the constraint exactly on the entire input domain, even though it does so on the training set.

The proof applies the identity theorem for real-analytic functions twice: first, to show that a network either satisfies the constraint everywhere or violates it almost everywhere; second, to show that the everywhere-satisfied case is non-generic on the manifold of global minima.

Cooper [1] characterized the geometry of the loss landscape for overparameterized networks; we use this geometry but ask a different question—not what the loss landscape looks like, but whether constraint satisfaction is possible within it. Methods that enforce constraints by construction [2] or by penalizing violations [3] implicitly acknowledge that unconstrained training fails; our result provides the mathematical reason.

Our contributions are: (1) a theorem establishing constraint unlearnability for overparameterized networks with real-analytic activations; and (2) the implication that architectural inductive biases for pointwise constraints are mathematically necessary for exact constraint satisfaction on the entire input domain, not merely an engineering choice. Section 2 discusses related work. Section 3 introduces the necessary background. Section 4 states and proves the main result. Section 5 discusses implications and limitations.

2. Related Work

Cooper [1] proved that the set of global minima of an overparameterized neural network is a smooth submanifold of parameter space, whose dimension equals the number of excess parameters. This structural result is the geometric foundation of our work: it gives the manifold of global minima on which we perform measure-theoretic analysis. Nguyen & Hein [4] showed that for sufficiently wide networks with analytic activations, almost every local minimum is global; Simsek et al. [5] further characterized how permutation symmetries shape the loss landscape. These works establish that overparameterization produces a well-behaved manifold of interpolating parameters, but do not address whether these solutions satisfy constraints beyond the training data.

A separate line of work quantifies how many parameters suffice for interpolation. For real-analytic activations—the setting of our main result—Madden & Thrampoulidis [6] proved that two-layer networks can memorize up to $\lfloor md/2\rfloor$ data points (m hidden neurons, input dimension d), and Madden [7] extended this to deeper architectures. Most relevant to our proof, Constantinescu & Popescu [8] proved that feedforward networks of arbitrary depth with non-affine activations in $L_{\mathrm{loc}}^1$ can interpolate arbitrary datasets—a result that subsumes the real-analytic case and is the interpolation theorem we invoke. For threshold and ReLU activations, Vershynin [9] established interpolation with $\Theta \left(md\right)$ parameters, and Yun et al. [10] gave tight bounds for shallow ReLU networks; these results show that the interpolation phenomenon is broad, though ReLU networks fall outside the scope of our theorem due to the non-analyticity of the ReLU.

The practical inadequacy of soft constraints is well documented. Beucler et al. [3] systematically compared architectural constraints (ACnet, which enforces conservation laws by construction) with loss-function penalties (LCnet), finding that only the former achieved exact satisfaction. Several methods enforce hard constraints by construction: Lu et al. [11] proposed hPINNs, combining penalty and augmented Lagrangian methods; Zhong et al. [12] developed constrained neural fields for differential constraints of arbitrary order; Balestriero & LeCun [13] provided provably optimal affine constraint enforcement; and Donti et al. [14] introduced a differentiable procedure for constraint satisfaction in optimization. Most recently, Min & Azizan [2] proposed a framework that enforces input-dependent linear equality and inequality constraints while retaining universal approximation guarantees. These methods share a common motivation: unconstrained training fails to satisfy constraints. Our result shows that this failure is not incidental—without architectural inductive bias, the set of parameters satisfying a constraint has measure zero on the manifold of global minima.

The identity theorem for real-analytic functions has appeared in neural network theory in several contexts. Sontag [15] used the fact that the zero set of a nontrivial real-analytic function has measure zero (via analytically thin sets) to prove that, for generic regression data, the set of critical points of the least-squares loss is countable—and finite under stronger assumptions. Madden & Thrampoulidis [6] used the identity theorem in two steps: first, to lift results from polynomial to real-analytic activations (by truncating Taylor expansions and invoking the identity theorem); second, to convert “rank condition function not identically zero” into “generically full rank.” Crăciun Ghoshdastidar [16] proved that the gradient descent map for networks with piecewise analytic activations is non-singular—it preserves sets of measure zero under preimages—establishing this through a layered analysis of almost-everywhere analytic functions, combined with Fubini’s theorem and the identity theorem. In contrast, our proof applies the identity theorem twice, in different variables: first to x, showing that constraint violation is all-or-nothing for each parameter, then to $\theta$, showing that the all-satisfied case has measure zero on the manifold of global minima.

A body of work studies what gradient descent implicitly prefers. Jacot et al. [17] showed that in the infinite-width limit, training dynamics are governed by the neural tangent kernel, which characterizes which functions are learned efficiently. Rahaman et al. [18] demonstrated spectral bias: deep networks learn low-frequency components first. Razin & Cohen [19] showed that implicit regularization in deep learning may not be explainable by norm minimization. These works characterize what gradient descent finds. Our result characterizes what it does not: the set of parameters satisfying a given constraint, which has measure zero on the manifold of global minima.

3. Preliminaries

3.1. Real-Analytic Functions and the Identity Theorem

Definition 1. 

A function $h:\Omega \to \mathbb{R}$ on an open set $\Omega \subseteq {\mathbb{R}}^n$ is real-analytic if for every $a\in \Omega$, h admits a convergent power series expansion in a neighborhood of a.

Real-analytic functions are closed under composition and arithmetic operations. Consequently, a feedforward network $f_{\theta }\left(x\right)$ built from real-analytic activations (sigmoid, tanh, GELU, Swish, Mish) and affine maps is real-analytic in x for each fixed $\theta$, and real-analytic in $\theta$ for each fixed x.

Theorem 1 

(Identity Theorem). Let $\Omega \subseteq {\mathbb{R}}^n$ be open and connected, and let $h:\Omega \to \mathbb{R}$ be real-analytic. If h vanishes on a set of positive Lebesgue measure, then $h\equiv 0$ on Ω.

Proof sketch 

For $n=1$: if $h\equiv 0$, then at every zero a there exists a smallest k such that the k-th Taylor coefficient at a is nonzero, so $h\left(x\right)={(x-a)}^{k}q\left(x\right)$ with $q\left(a\right)\ne 0$; by continuity, a is an isolated zero. (If all Taylor coefficients vanished at some point a, the set of such points would be both open and closed in $\Omega$, forcing $h\equiv 0$ by connectedness.) Isolated zeros form a countable, hence measure-zero, set.

For $n>1$: by Fubini’s theorem, positive measure of $h^{-1}\left(0\right)$ implies that for a positive-measure set of $x_1$-values, the section $x^{\prime }\mapsto h(x_1,x^{\prime })$ vanishes on a set of positive measure. By induction, these sections are identically zero. Fixing $x^{\prime }$, the function $x_1\mapsto h(x_1,x^{\prime })$ then vanishes on a positive-measure set; by the one-dimensional case, it is identically zero. Hence $h\equiv 0$. For a textbook treatment, see Krantz & Parks [20]. □

The contrapositive: a real-analytic function that is not identically zero vanishes only on a set of measure zero. This all-or-nothing property is the engine of our main result.

3.2. The Interpolating Manifold

Consider a feedforward neural network $f_{\theta }:\Omega \to \mathbb{R}$ parameterized by $\theta \in \Theta \subseteq {\mathbb{R}}^p$, with real-analytic activation functions. Given training data ${\left\{(x_i,y_i)\right\}}_{i=1}^m\subset \Omega \times \mathbb{R}$, define the evaluation map

$$F:\Theta \to {\mathbb{R}}^m,F\left(\theta \right)=(f_{\theta }\left(x_1\right),\dots ,f_{\theta }\left(x_m\right)),$$

(1)

which stacks the m network outputs into a single vector, and is real-analytic in $\theta$. The set of interpolating parameters is the fiber

$$M=F^{-1}\left(y\right)=\{\theta \in \Theta :f_{\theta }\left(x_i\right)=y_i,i=1,\dots ,m\},$$

(2)

where $y=(y_1,\dots ,y_m)\in {\mathbb{R}}^m$. This is the set of global minima of the empirical loss, viewed through the constraint $F\left(\theta \right)=y$ rather than the scalar loss.

Theorem 2 

(Cooper, 2021). If $p>m$ and the training data are in general position (a generic condition; see Cooper [1], Section 2), then y is a regular value of F. Consequently, $M=F^{-1}\left(y\right)$ is a smooth submanifold of ${\mathbb{R}}^p$ of dimension $p-m$ (possibly disconnected).

The regular value condition means that the Jacobian $DF\left(\theta \right)\in {\mathbb{R}}^{m\times p}$ has full row rank m at every $\theta \in M$: the m interpolation conditions are independent. Intuitively, overparameterization ($p>m$) provides enough degrees of freedom that these constraints do not degenerate. "General position" formalizes the intuition that the data points do not lie on any degenerate locus where the Jacobian loses rank; by Sard’s theorem, this holds for almost every dataset.

For our purposes, a stronger property of M is needed. Since F is real-analytic and y is a regular value, the real-analytic implicit function theorem applies: locally, M can be parametrized by real-analytic functions. This makes M a real-analytic submanifold of ${\mathbb{R}}^p$, not merely a smooth one. The consequence is that the restriction of any real-analytic function on ${\mathbb{R}}^p$ to M remains real-analytic, so the identity theorem applies to functions on M.

Since $p>m$, the manifold M has positive dimension $p-m$. We will additionally assume M is connected (Assumption 1 below), a non-trivial condition discussed in Section 4.

Remark 1. 

For networks with vector output $f_{\theta }:\Omega \to {\mathbb{R}}^k$, the evaluation map becomes $F:\Theta \to {\mathbb{R}}^{mk}$ and the overparameterization condition becomes $p>mk$. The constraint $g:{\mathbb{R}}^k\to \mathbb{R}$ is applied componentwise, and the identity theorem applies to $h_{\theta }\left(x\right)=g\left(f_{\theta }\left(x\right)\right)$ as before, since real-analyticity is preserved under composition. All results extend to this case by replacing m with $mk$ throughout. We restrict to scalar output for notational clarity.

4. Constraint Unlearnability

Let $\Omega \subseteq {\mathbb{R}}^n$ be open and connected. Let $f_{\theta }:\Omega \to \mathbb{R}$ and $M\subseteq {\mathbb{R}}^p$ be the network and interpolating manifold of Section 3. We assume $p>m+1$, slightly stronger than the overparameterization condition $p>m$ of Section 3. This condition ensures that the network can interpolate $m+1$ arbitrary points. Let $g:\mathbb{R}\to \mathbb{R}$ be a real-analytic function, defining the pointwise constraint

$$g\left(f_{\theta }\left(x\right)\right)=0,\forall x\in \Omega .$$

(3)

Assumption 1 

(Connectivity of M). The interpolating manifold $M=F^{-1}\left(y\right)$ is connected.

This assumption is non-trivial. Cooper’s theorem gives M as a smooth submanifold, but not its connectivity. Kuditipudi et al. [21] constructed datasets for which the locus of global minima is disconnected, so the assumption does not hold in full generality. However, empirical evidence suggests that connectivity holds for generic datasets and sufficiently overparameterized networks, and partial theoretical support is provided by Simsek et al. [5], who proved that the expansion manifold ${\Theta }_{r^{*}\to m}\left({\theta }^{*}\right)$ is connected for any $C^1$ activation, and that $M={\Theta }_{r^{*}\to m}\left({\theta }^{*}\right)$ under additional technical conditions on the activation. A complete characterization of when M is connected for real-analytic activations remains an open problem.

Definition 2 

(Inductive Bias). The architecture $f_{\theta }$ has aninductive biasfor the constraint g if $g\left(f_{\theta }\left(x\right)\right)=0$ for all $\theta \in \Theta$ and all $x\in \Omega$—that is, the constraint is satisfied regardless of parameter choice. If there exist $\theta \in \Theta$ and $x\in \Omega$ with $g\left(f_{\theta }\left(x\right)\right)\ne 0$, the architecture is without inductive bias for g.

Define

$$M_{\mathrm{good}}=\{\theta \in M:g\left(f_{\theta }\left(x\right)\right)=0,\forall x\in \Omega \},$$

(4)

the subset of M on which the network satisfies the constraint at every input. We prove that without inductive bias, $M_{\mathrm{good}}$ has measure zero in M.

4.1. All-or-Nothing for Pointwise Constraints

Proposition 1. 

For any fixed $\theta \in \Theta$, define $h_{\theta }:\Omega \to \mathbb{R}$ by $h_{\theta }\left(x\right)=g\left(f_{\theta }\left(x\right)\right)$. Then either $h_{\theta }\equiv 0$ on Ω, or $h_{\theta }^{-1}\left(0\right)$ has Lebesgue measure zero in Ω.

Proof. 

The network $f_{\theta }\left(x\right)$ is a composition of affine maps and real-analytic activation functions, hence real-analytic in x; g is real-analytic by assumption. Thus $h_{\theta }=g\circ f_{\theta }$ is real-analytic on $\Omega$. Since $\Omega$ is open and connected, the identity theorem gives either $h_{\theta }\equiv 0$ on $\Omega$, or $h_{\theta }^{-1}\left(0\right)$ has measure zero. □

Remark 2. 

This proposition holds for every $\theta \in \Theta$, independently of the training data. It rules out partial satisfaction: no parameter can make the network satisfy the constraint on a set of positive measure without satisfying it everywhere.

4.2. Measure Zero of the Satisfying Parameters

The argument applies the identity theorem a second time, now to a function of $\theta$ on the manifold M.

Choose a test point $x_0\in \Omega \setminus \{x_1,\dots ,x_m\}$. Define the evaluation map

$$\varphi :M\to \mathbb{R},\varphi \left(\theta \right)=f_{\theta }\left(x_0\right).$$

(5)

Lemma 1. 

ϕ is real-analytic on M.

Proof. 

The network output $f_{\theta }\left(x_0\right)$ is real-analytic in $\theta$ on $\Theta$, as a composition of affine maps in $\theta$ with real-analytic activation functions. Since M is a real-analytic submanifold of ${\mathbb{R}}^p$ (Section 3), the restriction $\varphi =f_{·}\left(x_0\right){|}_M$ preserves real-analyticity. □

Theorem 3 

(Main Theorem). If the architecture is without inductive bias for g, $p>m+1$, and M is connected (Assumption 1), then $M_{\mathrm{good}}$ has measure zero in M.

Proof. 

The strategy is to show that $g\circ \varphi$ is real-analytic and not identically zero on M, then apply the identity theorem. By Lemma 1, $\varphi$ is real-analytic on M; since g is real-analytic, the composition $g\circ \varphi$ is real-analytic on M.

To see that $g\circ \varphi \not\equiv 0$, note that since the architecture is without inductive bias (Definition 2), there exist ${\theta }_0\in \Theta$ and $x_0^{\prime }\in \Omega$ with $g\left(f_{{\theta }_0}\left(x_0^{\prime }\right)\right)\ne 0$. Set $y_0=f_{{\theta }_0}\left(x_0^{\prime }\right)$. Consider the augmented dataset ${\left\{(x_i,y_i)\right\}}_{i=1}^m\cup \left\{(x_0,y_0)\right\}$ of $m+1$ points. Since $p>m+1$, by the interpolation result of Constantinescu & Popescu [8], there exists ${\theta }^{*}\in \Theta$ such that

$$f_{{\theta }^{*}}\left(x_i\right)=y_i(i=1,\dots ,m),f_{{\theta }^{*}}\left(x_0\right)=y_0.$$

(6)

The first m conditions give ${\theta }^{*}\in M$; the last gives $\varphi \left({\theta }^{*}\right)=y_0$. Hence $g\left(\varphi \left({\theta }^{*}\right)\right)=g\left(y_0\right)\ne 0$.

Since M is connected (Assumption 1) and $g\circ \varphi$ is real-analytic and not identically zero, the identity theorem gives that ${(g\circ \varphi )}^{-1}\left(0\right)$ has measure zero in M. Since $M_{\mathrm{good}}\subseteq {(g\circ \varphi )}^{-1}\left(0\right)$—if $\theta \in M_{\mathrm{good}}$, then $g\left(f_{\theta }\left(x\right)\right)=0$ for all $x\in \Omega$, in particular $g\left(\varphi \right(\theta \left)\right)=0$—we conclude that $M_{\mathrm{good}}$ has measure zero in M. □

4.3. Implications for Gradient Descent

Corollary 1. 

Suppose the architecture is without inductive bias for g, $p>m+1$, and M is connected (Assumption 1). If gradient descent is initialized from a distribution absolutely continuous with respect to Lebesgue measure and converges to M, then it almost surely finds a parameter θ for which the constraint is violated for almost every $x\in \Omega$.

Proof. 

By the Main Theorem, $M_{\mathrm{good}}$ has measure zero in M, hence in ${\mathbb{R}}^p$ (since $dimM=p-m<p$). By the Proposition in Section 4.1, any $\theta \in M\setminus M_{\mathrm{good}}$ violates the constraint almost everywhere. So it suffices to show that gradient descent almost surely avoids $M_{\mathrm{good}}$.

Crăciun Ghoshdastidar [16] proved that for networks with piecewise analytic activations—a class that includes real-analytic activations—the gradient descent map $G_{\eta }$ is non-singular for almost all step-sizes $\eta$: the preimage of any measure-zero set under $G_{\eta }$ has measure zero. Since the composition of non-singular maps is non-singular, the T-step map $G_{\eta }^T=G_{\eta }\circ \dots \circ G_{\eta }$ is also non-singular. Thus for every finite T, under absolutely continuous initialization, $Pr[{\theta }_T\in M_{\mathrm{good}}]=0$.

Since gradient descent converges to M by assumption, the limit ${\theta }_{\infty }={lim}_{T\to \infty }{\theta }_T$ lies in M. Extending non-singularity from finite iterates to the limit is a standard step in the saddle-point avoidance framework [22]; under this framework, $Pr[{\theta }_{\infty }\in M_{\mathrm{good}}]=0$. Hence ${\theta }_{\infty }\in M\setminus M_{\mathrm{good}}$ almost surely, and the constraint is violated for almost every $x\in \Omega$. □

5. Discussion

5.1. Inductive Bias in Practice

Definition 2 formalizes a distinction that the practice of deep learning has long relied on: softmax, hard-constraint layers, and architectural enforcement [2,3,11,12] all ensure constraints by construction rather than by training. Our theorem provides the theoretical explanation for this practice—without inductive bias, exact constraint satisfaction is a measure-zero event, not merely a training failure. The result clarifies that expressivity (the constraint-satisfying function exists in the hypothesis class) does not imply learnability (gradient descent finds it): the constraint-satisfying parameters occupy measure zero on M, so a convergent trajectory almost surely misses them.

5.2. Implicit Regularization and Constraints

A natural question is whether implicit regularization—the tendency of gradient descent to prefer certain solutions—could substitute for inductive bias. Spectral bias [18] and the neural tangent kernel [17] show that gradient descent preferentially learns low-frequency, simple functions. This explains why it generalizes: the target function lies in a positive-measure subset of M that gradient descent can reach.

Constraint satisfaction, however, is structurally different. The set $M_{\mathrm{good}}$ is not a region that gradient descent avoids by preference; it is a measure-zero subset that gradient descent almost surely never reaches. Implicit regularization operates within the positive-measure set $M\setminus M_{\mathrm{good}}$, selecting solutions with certain regularity properties, but it does not redirect the trajectory toward the zero-measure $M_{\mathrm{good}}$.

Whether implicit regularization can achieve approximate constraint satisfaction—$\parallel g\circ f_{\theta }{\parallel }_{\infty }<ϵ$ for small $ϵ$—is a separate question that our result does not address. The $ϵ$-neighborhood of $M_{\mathrm{good}}$ may have positive measure, and characterizing whether gradient descent enters it requires tools beyond the identity theorem.

5.3. Limitations

Activation Functions

The proof relies on the identity theorem for real-analytic functions, which requires real-analytic activations. This class includes sigmoid, tanh, GELU, Swish, and Mish, but excludes ReLU, which is piecewise linear and not real-analytic. The zero set of a piecewise linear function can have positive measure without the function being identically zero, so the identity theorem does not apply. Whether an analogue of our result holds for ReLU networks remains open. The framework of Crăciun Ghoshdastidar [16] for piecewise analytic activations may offer a path, but the core tool—the identity theorem—has no direct replacement. We note that approximate implementations of GELU (e.g., using tanh) are also real-analytic, so the theorem applies to them as well.

Pointwise vs. Functional Constraints

Our framework covers pointwise constraints of the form $g\left(f_{\theta }\left(x\right)\right)=0$ for each x independently. Many important constraints are functional: equivariance ($f_{\theta }\left(\rho x\right)=\rho f_{\theta }\left(x\right)$), monotonicity ($x_1<x_2\Rightarrow f_{\theta }\left(x_1\right)<f_{\theta }\left(x_2\right)$), and differential constraints ($\nabla ·f_{\theta }=0$) relate values at different inputs. These require a different mathematical treatment, as they constrain the function as a whole rather than each output independently.

Exact vs. Approximate Satisfaction

The theorem concerns exact constraint satisfaction on the entire input domain. In practice, approximate satisfaction within a tolerance $ϵ$ is often sufficient. Our measure-zero result does not directly imply that approximate satisfaction fails: the $ϵ$-approximate satisfying set may have positive measure. Extending the analysis to approximate constraints is an open problem.

Parameterization Condition

The main theorem requires $p>m+1$, slightly stronger than the standard overparameterization condition $p>m$. This ensures interpolation of $m+1$ points in the proof of the Main Theorem. For sufficiently wide networks, the distinction is immaterial, but the tight bound remains to be determined.

Connectivity of M

The main theorem assumes M is connected (Assumption 1); see the discussion following the assumption for its non-triviality and partial theoretical support.

6. Conclusions

We have shown that overparameterized neural networks with real-analytic activations cannot learn to satisfy pointwise constraints exactly without architectural inductive bias. The proof hinges on a simple geometric fact: the identity theorem for real-analytic functions forces constraint satisfaction to be all-or-nothing, and overparameterization ensures that the “all” case occupies measure zero on the interpolating manifold. Combined with the non-singularity of the gradient descent map [16], this yields constraint unlearnability: gradient descent almost surely converges to parameters that violate the constraint almost everywhere.

The practical implication is that architectural inductive biases for pointwise constraints—softmax for normalization, hard-constraint layers for physical conservation laws, architectural enforcement of non-negativity—are not engineering conveniences but mathematical necessities for exact satisfaction. Their ubiquity in practice reflects this necessity, even when it has not been formally articulated.

The result is exact within its setting: real-analytic activations, pointwise constraints, exact satisfaction, and a connected interpolating manifold (see Section 5.3 for limitations). Extending constraint unlearnability beyond these boundaries defines a program for future work. The most pressing direction is whether an analogue holds for ReLU networks, the most widely used in practice; the identity theorem has no direct replacement there, but the framework of Crăciun Ghoshdastidar [16] for piecewise analytic activations may offer a path. A second direction is approximate satisfaction: whether implicit regularization can drive the network into an $ϵ$-neighborhood of $M_{\mathrm{good}}$, even though exact membership is a measure-zero event.