Hill-Enigma-SPN: A Novel Substitution–Permutation Network Cipher with GF(2) Matrix-Rotor Scheduling and Argon2id Key Derivation

1. Introduction

Substitution–permutation network (SPN) architectures implement iterated substitution and permutation layers within a product cipher framework, as originally motivated by Shannon’s principles of confusion and diffusion [1] and further articulated in early block cipher design studies [2,3].

The Hill cipher, introduced by Lester Hill in 1929 [4,5], applied linear transformations in the form of invertible matrix multiplication to plaintext blocks, producing a linear mapping in which each ciphertext symbol depends on all plaintext symbols within the block. This mechanism illustrates diffusion through linear algebraic mixing, and increasing the matrix dimension increases both the effective key space and intra-block diffusion [6,7,8]. However, as a single-stage transformation, it does not incorporate iterative nonlinear structure. As a consequence, the classical construction is more appropriately interpreted as a foundational diffusion mechanism within a broader design framework rather than as a standalone encryption architecture.

A substantial body of research has explored extensions of the Hill cipher that introduce additional structural elements, including substitution components, key variation, and multi-stage processing [6,7,9,10,11,12,13,14,15,16,17]. These efforts demonstrate that matrix-based diffusion remains a viable design component when combined with nonlinear and iterative elements. In particular, substitution–permutation networks, following Shannon’s principles of confusion and diffusion [1], provide a natural framework for integrating linear mixing with nonlinear substitution in a provably structured manner [3,18].

Recent work has demonstrated how Hill-type matrix diffusion can be embedded within SPN-style constructions that incorporate nonlinear substitution and iterative round structure [19]. The present work builds on this direction but adopts a different design emphasis. Rather than employing global diffusion layers defined over GF(2⁸), the proposed construction utilizes per-byte transformations over GF(2) that vary across rounds according to a rotor scheduling mechanism inspired by the rotor stepping of the Enigma Encoding Machine [20] and by prior work on key matrix element rotation [6,7]. This shift alters the diffusion profile of the cipher, prioritizing structural diversity and localized mixing over maximal branch number at a single layer.

To address these considerations, this paper introduces Hill-Enigma-SPN (HESPN), a 128-bit substitution–permutation network combining three structural elements: (i) a rotor-scheduled family of admissible GF(2) matrices providing per-byte diffusion; (ii) AES S-box substitution supplying nonlinear confusion with established cryptographic properties [21,22]; and (iii) a round-dependent routing permutation for inter-byte mixing; together with (iv) a specified Argon2id memory-hard key-derivation profile enhancing resistance to offline brute-force attacks in password-based operation [23]. The cipher's security analysis assumes a uniform 256-bit master key; the derivation of that key from a password is a property of the surrounding password-based encryption layer rather than of the block cipher itself (Section 3.7 and Section 4.8).

The proposed construction distinguishes itself from prior Hill-based SPN variants through its treatment of diffusion and key-dependent variability across rounds. In particular, it replaces global MDS-based mixing with multiple independent, key-dependent linear transformations applied at the byte level, each satisfying a provable branch number lower bound. This design provides a different balance between diffusion guarantees and structural variability across the cipher state.

The contributions of this work are therefore centered on the formulation and evaluation of this alternative diffusion model. These include: (i) the definition of a rotor scheduling mechanism, based on the Enigma Encoding Machine, over admissible GF(2) matrices with formally verified branch number B ≥ 4; (ii) the integration of this mechanism within a complete SPN round function; (iii) the specification of an Argon2id-based password-to-key derivation profile; and (iv) an experimental evaluation of statistical properties including avalanche behavior, differential propagation, linear correlation, and algebraic degree growth.

The remainder of this paper is organized as follows. Section 2 surveys related work across the Hill-cipher and SPN literatures. Section 3 develops the mathematical foundations, Section 4 specifies the round function and the full cipher, and Section 5 presents the empirical evaluation. Section 6 concludes and identifies open problems.

2. Related Work

2.1. The Hill Cipher and Its Cryptographic Limitations

The Hill cipher [4,5] introduced matrix-based block encryption in 1929, with each plaintext block encrypted by multiplication of a row vector by an invertible square key matrix over a modular ring. While offering good diffusion within a block, the cipher is vulnerable to known-plaintext attack: given n plaintext-ciphertext pairs for an n×n matrix, the key can be recovered by solving a system of linear equations [4,5]. The analysis of keyspace size for Hill-type ciphers is treated rigorously by Overbey, Traves & Wojdylo [8], who establish that |GL(8, GF(2))| ≈ 5.35 × 10¹⁸, representing approximately 29% of all 8×8 binary matrices over GF(2).

2.2. Strengthening the Hill Cipher: Prior Variants

Within the Hill-cipher lineage, Saeednia [24] addressed the cipher’s defining weakness of linearity that exposes it to known-plaintext recovery of the key by generating a fresh per-message key from a single shared matrix. For each message, a random vector selects a permutation applied simultaneously to the rows and columns of the secret matrix H, producing a conjugated session matrix H_t = P_t⁻¹HP_t; because conjugation commutes with inversion, decryption requires only a permutation of a precomputed H⁻¹ rather than a fresh matrix inversion. The construction is, however, entirely linear over the underlying modular ring: it supplies a keyed diffusion layer in the spirit of an SPN mixing step but contains no substitution stage and no round iteration, and its security rests on the secrecy of the permutation together with a counting argument over candidate matrices rather than on nonlinear confusion. Saeednia thus represents a diffusion-side ancestor of the present design and identifies the gap it is intended to fill.

Toorani & Falahati [17] introduced the Affine Hill cipher (TFHC), a two-protocol variant based on affine transformation which is still vulnerable to zero-plaintext attack: when the plaintext is the zero vector, the ciphertext directly reveals the auxiliary vector V, yielding a system of linear equations in the key matrix entries.

Valizadeh [25] addressed this zero-plaintext vulnerability in TFHC by introducing Arnold Cat Map (ACM) scrambling applied to both the key matrix K and the auxiliary matrix V before encryption. While effective against the specific zero-plaintext attack, this scheme retains the affine (degree-1) structure of TFHC and introduces no genuine nonlinear substitution layer; the ACM is a linear permutation, not a cryptographic S-box. Key derivation relies on a SHA/HMAC hash chain without memory hardness, leaving it vulnerable to GPU-accelerated brute force. No branch number, avalanche, differential, or linear bias metrics are reported. As developed in Section 2.5, the present design addresses these limitations: the AES S-box provides genuine nonlinearity (degree 7, NL = 112), Argon2id provides memory-hard key protection, and the zero-plaintext attack is structurally precluded by round-key injection prior to the matrix layer.

Paragas, Sison & Medina [14] proposed a Hill cipher variant using SHA-256 for key derivation and a modified S-box. However, the scheme operates over mod-64 arithmetic (not a proper field), uses a static key schedule, and operates without iterative composition across multiple transformation stages. No differential or linear bias metrics are reported beyond a basic avalanche measurement.

Several further variants combine permutation and iteration with the Hill core. Ismail, Amin & Diab [26] proposed repairing the Hill cipher by modifying the key matrix per block with an initial vector; Sastry, Shankar & Bhavani [27] introduced interweaving and iteration stages between successive matrix multiplications; Farmanbar & Chefranov [28] investigated Hill cipher modifications based on permutation and iteration; and Gutik, Popadiuk & Sulym [29] presented a recent algebraic generalization of the Hill construction. These designs reinforce a consistent pattern in the lineage: permutation and iteration strengthen the classical cipher, but none of these schemes combines a cryptographically vetted S-box with provable diffusion bounds and memory-hard key derivation.

2.3. Substitution–Permutation Networks and the Wide-Trail Strategy

The substitution–permutation paradigm that frames the present work originates with Shannon’s principles of confusion and diffusion [1] and their embodiment in practical cipher design. Feistel [2] translated these principles into a concrete architecture, alternating key-controlled substitution with permutation so that local nonlinearity is propagated into global statistical complexity across successive stages. It is this substitution–permutation philosophy, rather than any single primitive, that is the structural ancestor of the design developed here.

Kam and Davida [3] placed this architecture on a more formal footing, giving a structured method for constructing substitution–permutation networks that guarantee completeness, the property that every output bit depends on every input bit, and thereby linking S-box placement and interconnection to a verifiable diffusion criterion. Their treatment anticipates the wide-trail concerns [18] that later governed AES and motivate the branch-number analysis of Section 3.

Turning to SPN theory (a full review of the AES candidates is beyond our scope), the wide-trail design strategy of Daemen & Rijmen [18] provides the foundational framework for bounding differential and linear attack complexity in terms of active S-box counts. The AES (Rijndael) S-box, based on inversion in GF(2⁸) followed by an affine transformation, achieves optimal differential uniformity (DU) DU = 4 and nonlinearity (NL) NL = 112 [21,22]. Hill-Enigma-SPN (HESPN) adopts the AES S-box directly, inheriting these properties for its substitution layer.

The SAFER family [30,31,32] and SHARK [22] are classical SPN designs that form part of the lineage leading to AES. Serpent [33], an AES finalist, is a 128-bit SPN with 32 rounds and conservative security margins, providing a useful comparison point for the present work. PRESENT [34] is a lightweight bit-oriented SPN with 4×4 S-boxes. More recent work generalizes the substitution–permutation paradigm itself, as in the HADES design strategy [35]. The remaining AES finalists MARS, RC6, and Twofish all employ Feistel-based rather than SPN structures [12] and are therefore outside the direct scope of this comparison.

2.4. Memory-Hard Key Derivation

Memory-hard key derivation is addressed by Argon2id [23], the winner of the 2015 Password Hashing Competition. Argon2id combines data-dependent and data-independent memory access patterns, making it resistant to both GPU acceleration and side-channel attacks on memory access patterns.

2.5. Closest Prior Work and the Gap Addressed

The closest prior work is the multidimensional Hill substitution–permutation network of Coggins [19], which integrates Hill-style matrix diffusion with matrices of several dimensions within an iterated SPN framework and demonstrates how a multidimensional matrix-based diffusion can be carried inside a full substitution–permutation structure. Hill-Enigma-SPN departs from that design in its diffusion strategy: in place of global MDS-based mixing over GF(2⁸), it embeds a key-dependent, rotor-scheduled GF(2) matrix layer with a proven branch-number floor (Section 3), combined with AES S-box confusion, a routing permutation for inter-byte diffusion, and Argon2id memory-hard key derivation. This shift changes both the diffusion profile and the structural diversity of the cipher, offering an alternative route for integrating Hill-type mechanisms into SPN design.

Following a thorough review of the literature, prior Hill-based constructions address these properties in isolation. The present design reorganizes these elements into a unified architecture while emphasizing different diffusion tradeoffs than earlier SPN-style variants: (i) GF(2) admissible matrix diffusion with formally verified branch number; (ii) rotor scheduling inspired by the Enigma Encoding Machine; (iii) AES S-box substitution; and (iv) Argon2id memory-hard key derivation, within a principled SPN framework with empirical statistical evaluation.

Table 1. Structural parameters of representative SPN block ciphers. HESPN row highlighted in green.

Cipher	Block (bits)	Key (bits)	Rounds
HESPN(this work)	128	256	16 1
MD-Hill-SPN [19]	128	256	12
AES(Rijndael) [21,36]	128	128–256	10–14
Serpent [33]	128	128–256	32
PRESENT [34]	64	80/128	31
SAFER+ [30,31,37]	128	128–256	8–16
SHARK [22]	64	128	6

¹ The original 12-round parameterization was abandoned after counter-mode structured-input testing; see Section 5.6.

Table 1. Structural parameters of representative SPN block ciphers. HESPN row highlighted in green.

Cipher	Block (bits)	Key (bits)	Rounds
HESPN(this work)	128	256	16 1
MD-Hill-SPN [19]	128	256	12
AES(Rijndael) [21,36]	128	128–256	10–14
Serpent [33]	128	128–256	32
PRESENT [34]	64	80/128	31
SAFER+ [30,31,37]	128	128–256	8–16
SHARK [22]	64	128	6

¹ The original 12-round parameterization was abandoned after counter-mode structured-input testing; see Section 5.6.

Table 2. Substitution, diffusion, and throughput parameters of representative SPN block ciphers. HESPN row highlighted in green. SW = software implementation. DU = differential uniformity; NL = nonlinearity. Branch number field denotes the algebraic structure over which it is defined. Throughput figures are estimates from published sources or implementation reports; direct comparison is indicative only due to platform variation.

Cipher	S-box type	DU/NL	Branch number	SW throughput (est.)
HESPN(this work)	AES S-box 8-bit, GF(28)	4/112	4 (GF(2), proved)1	~60 MB/s (C, projected; not benchmarked)2
MD-Hill-SPN [19]	AES S-box 8-bit, GF(28), ×2/rnd	4/112	5/9/17 (GF(28), MDS)5	Not reported 6
AES(Rijndael) [21,36]	GF(28) inv. + affine	4/112	5 (GF(28), MDS)3	460–1,900 MB/s2
Serpent [28]	Eight 4×4 S-boxes/round	4/4	none (bit permut. layer)	~40–100 MB/s2
PRESENT [29]	4×4 nibble S-box ×16	4/4	none (bit permut. layer)	~20–50 MB/s2
SAFER+ [25,26,31]	Exp/log over ℤ256	N/A4	none (PHT/Had. diffusion)	~100–200 MB/s2
SHARK [21]	GF(28) inv. + affine	4/112	9 (GF(28), MDS)3	~150–300 MB/s2

Table 2. Substitution, diffusion, and throughput parameters of representative SPN block ciphers. HESPN row highlighted in green. SW = software implementation. DU = differential uniformity; NL = nonlinearity. Branch number field denotes the algebraic structure over which it is defined. Throughput figures are estimates from published sources or implementation reports; direct comparison is indicative only due to platform variation.

Cipher	S-box type	DU/NL	Branch number	SW throughput (est.)
HESPN(this work)	AES S-box 8-bit, GF(28)	4/112	4 (GF(2), proved)1	~60 MB/s (C, projected; not benchmarked)2
MD-Hill-SPN [19]	AES S-box 8-bit, GF(28), ×2/rnd	4/112	5/9/17 (GF(28), MDS)5	Not reported 6
AES(Rijndael) [21,36]	GF(28) inv. + affine	4/112	5 (GF(28), MDS)3	460–1,900 MB/s2
Serpent [28]	Eight 4×4 S-boxes/round	4/4	none (bit permut. layer)	~40–100 MB/s2
PRESENT [29]	4×4 nibble S-box ×16	4/4	none (bit permut. layer)	~20–50 MB/s2
SAFER+ [25,26,31]	Exp/log over ℤ256	N/A4	none (PHT/Had. diffusion)	~100–200 MB/s2
SHARK [21]	GF(28) inv. + affine	4/112	9 (GF(28), MDS)3	~150–300 MB/s2

Remarks

¹ Branch number (HESPN): B ≥ 4 is formally proved for all 64 (seed, orientation) pairs over GF(2) via the admissibility filter and R(M) = MᵀJ (Section 3.4/Definition 3.3). Branch number is defined per-byte over GF(2), not across bytes as in the GF(2⁸) MDS sense. AES MixColumns achieves B = 5 as an MDS code over GF(2⁸) across four bytes simultaneously, which is a stronger inter-byte claim.

² Throughput: HESPN: ~60 MB/s is a projected C figure for the 16-round cipher, extrapolated from the NumPy implementation's operation profile (12-round measurements scaled by 12/16); no optimized C implementation has yet been benchmarked. The pure Python prototype achieves ~27 bytes/s at 16 rounds (36 bytes/s measured at 12 rounds); NumPy vectorized ~6 MB/s (scaled from ~8 MB/s measured at 12 rounds; this work). AES: 460–1,900 MB/s includes AES-NI hardware acceleration; pure software AES ~200–500 MB/s. Serpent 40–100 MB/s from [12,33]. PRESENT: ~20–50 MB/s software estimate. SAFER+: ~100–200 MB/s estimate. SHARK: ~150–300 MB/s estimate; no modern benchmarks available.

³ Branch number (AES/SHARK): AES MixColumns is an MDS code over GF(2⁸) with branch number B = 5 across four bytes; SHARK’s linear layer is an MDS (Reed–Solomon) code over GF(2⁸) with branch number B = 9 across all eight bytes of its 64-bit block [21,22]. Both provide stronger inter-byte diffusion guarantees than HESPN’s per-byte GF(2) branch number, but operate over an extension field rather than GF(2) directly.

⁴ SAFER+ S-box DU/NL: SAFER+ uses exponential and logarithmic S-boxes based on the map x ↦ 45ˣ mod 257 (with the convention that the value 256 is represented as 0), together with its logarithmic inverse, rather than S-boxes over a binary field GF(2ⁿ). The standard differential uniformity and nonlinearity metrics as defined for GF(2ⁿ) S-boxes do not directly apply. SAFER’s diffusion layer uses the Pseudo-Hadamard Transform (PHT) rather than a matrix-based branch number construction.

⁵ Branch number (MD-Hill-SPN [19]): MD-Hill-SPN uses a three-tier MDS diffusion hierarchy over GF(2⁸): 4×4 (B = 5), 8×8 (B = 9), and 16×16 (B = 17) Cauchy matrices, each meeting the Singleton bound B = n + 1. Unlike HESPN’s per-byte GF(2) branch number, this is cross-byte GF(2⁸) MDS diffusion, with the 16×16 tier providing full-state mixing in a single round.

⁶ Throughput (MD-Hill-SPN [19]): No software throughput figure is reported; the multi-tier mix layer, particularly the full-state 16×16 GF(2⁸) matrix is deliberately heavier than a single MDS layer and is not benchmarked.

HESPN occupies a distinct position in this comparison: among the designs surveyed, it appears to be the only one combining a GF(2) rotor-scheduled matrix diffusion layer with formally verified branch number, AES S-box nonlinearity (DU = 4, NL = 112), and Argon2id memory-hard key derivation. The author is not aware of a prior Hill cipher variant achieving this combination. The closest prior design is the multidimensional Hill SPN of Coggins [19], which likewise pairs the AES S-box with Argon2id key derivation; it differs in its diffusion layer, using a three-tier GF(2⁸) MDS hierarchy (branch numbers 5, 9, and 17 for its 4×4, 8×8, and 16×16 matrices) rather than the key-dependent, rotor-scheduled GF(2) matrices used here, and reports no throughput benchmark. The general scheme diagram is represented in Figure 1 below.

The branch number comparison requires care: AES achieves B = 5 across four bytes and SHARK B = 9 across all eight bytes, both as MDS codes over GF(2⁸), which is a stronger inter-byte guarantee than HESPN’s B ≥ 4 over GF(2) per byte. However, HESPN compensates with 16 independent key-dependent matrices per round (versus AES’s single shared MixColumns), providing structural richness at the cost of approximately 3.5× more bit-level operations per block.

The throughput gap relative to AES is primarily due to the absence of dedicated hardware acceleration (AES-NI). An optimized C implementation of HESPN is projected at ~60 MB/s (Section 3.9), comparable to Serpent in software and substantially faster than PRESENT. Settings without AES-NI are, however, already well served by standardized software-oriented ciphers such as ChaCha20 [38]; the practical niche claimed here is accordingly modest, and the contribution of the present work is primarily architectural (Section 2.5).

3. Mathematical Foundations

This section presents the algebraic foundations of Hill-Enigma-SPN, including byte-wise binary matrices over GF(2), the coding-theoretic branch number as a diffusion metric, and the key-dependent rotor mechanism that instantiates the diffusion layer, building on prior work with rotating the elements of Hill cipher matrices [6,7]. This paper provides: (i) formal definitions and properties, (ii) implementation-aligned statements with clear correctness guarantees, and (iii) empirical diffusion observations that support, though do not replace, formal arguments. There is no claim of a completed wide-trail proof here; rather, this paper provides the foundation for the empirical analyses reported in Section 5.

3.1. Bit-Ordering Conventions (MSB-First)

We adopt a uniform MSB-first convention at every representation level. This ensures the implementation, proofs, and diagnostics use the same bit/byte orientation. Readers interested only in the high-level construction may proceed to Section 4; this subsection fixes conventions for reproducibility.

Byte level. A byte x ∈ {0,...,255} is represented as the column vector v(x) ∈ GF(2)⁸ with v(x)[i] = (x ≫ (7−i)) & 1, i = 0,...,7, so v(x) [0] is the MSB and v(x) [7] the LSB. The inverse is v⁻¹(b₀,...,b₇) = ∑_i b_i · 2^{7−i}.

Block level. A 128-bit block is the sequence (B₀,...,B₁₅) interpreted as the big-endian integer int(block) = ∑_n B_n · 2^{8(15−n)}. Bit index 0 is the MSB of B₀; bit index 127 is the LSB of B₁₅. The operator rotl128(block, k) rotates this integer left by k positions, where k ∈ {1,...,127} and no k in the schedule is a multiple of 8, ensuring cross-byte-boundary bit motion every round.

Matrix entry level. Row i of an 8×8 matrix M is stored as byte m_i. The (i, j) entry is M_ij = (m_i ≫ (7−j)) & 1 = v(m_i)[j], so column j = 0 is the MSB of m_i and column j = 7 is the LSB. The dot product of row i with input x is v(y)[i] = M_i0·x₀ ⊕ M_i1·x₁ ⊕ ··· ⊕ M_i7·x₇, which reduces to v(y)[i] = popcount(m_i AND x) mod 2, since multiplication in GF(2) is AND and addition in GF(2) is XOR.

Round-key level. Each round key is rkᵣ = SHA256(K ∥ “ROUNDKEY” ∥ r)[:16] (see the reference implementation, HESPNv4Rerun.py), with bit index 0 the MSB of its first byte, consistent with the block convention.

3.2. Linear Maps and 8×8 Binary Matrices

An 8×8 matrix M ∈ GF(2)⁸ˣ⁸ defines a linear map M: GF(2)⁸ → GF(2)⁸. Invertibility is equivalent to full rank over GF(2) and is verified by Gaussian elimination with MSB-first pivot selection.

By Overbey, Traves & Wojdylo [8], the number of invertible 8×8 matrices over GF(2) is |GL(8, GF(2))| = ∏_i₌₀⁷ (2⁸ − 2ⁱ) = 5,348,063,769,211,699,200 ≈ 5.35 × 10¹⁸, representing approximately 29% of all 8×8 binary matrices. The admissible seed set is a branch-number-filtered subset of this group. The 16 independent seed matrices yield up to (|GL(8, GF(2))|)¹⁶ ≈ (5.35 × 10¹⁸)¹⁶ possible seed matrix combinations; however, this combinatorial count does not add entropy beyond the master key, because the accepted matrices are deterministic functions of K. For a uniformly random 256-bit master key K, exhaustive search over K requires 2²⁵⁶ trials; in password-based operation, the effective search space is bounded by the entropy of the password distribution, with Argon2id increasing the per-guess cost rather than adding entropy.

Remark (contrast with AES MixColumns). AES achieves MDS diffusion over GF(2⁸) across four bytes simultaneously; Hill-Enigma-SPN instead applies 16 independent invertible rotor-scheduled 8×8 binary matrices over GF(2) to each byte individually, guaranteeing intra-byte diffusion with an explicit branch-number floor (Section 3.3). Inter-byte diffusion is driven by rotl128 and the routing permutation (Section 4).

3.3. Branch Number as a Diffusion Metric

Definition 3.1 (branch number).For a linear map M: GF(2)ⁿ → GF(2)ⁿ, B(M) = min_{x≠0} ( wt(x) + wt(Mx) ) [21]. Since this depends only on Hamming weights it is independent of MSB/LSB convention. A lower bound on B(M) forces nontrivial spreading of any input difference.

3.4. Clockwise 90° Rotation: Algebra and Consequences

For M ∈ GF(2)^{8×8}, define R(M) by R(M)_ij = M_{(7−j), i}. This permutes the 64 entries via the index map π: (i,j) ↦ (j, 7−i) and satisfies R⁴(M) = M.

Proposition 3.2 (Invertibility preservation).With J the reversal (antidiagonal identity) permutation matrix, R(M) = MᵀJ. Hence R preserves invertibility over GL(8, 2): transpose preserves invertibility and right-multiplication by a permutation matrix preserves invertibility. Therefore M∈ GL(8, 2) implies R(M)∈ GL(8, 2) automatically. □

Ring non-automorphism. Although R preserves invertibility, it is not a ring automorphism of GF(2)^{8×8}: in general R(MN) ≠ R(M)R(N). Branch number depends on the image set {Mx : x ≠ 0} under GF(2) matrix multiplication, a property of the ring structure that R does not preserve in general. We therefore verify B(·) ≥ 4 independently for each of the four orientations. Intuitively, this matters because the diffusion guarantee of a seed matrix is not automatically inherited by its rotations: each of the four orientations must meet the branch-number floor on its own. Checking all four is therefore what ensures the per-round diffusion bound of Section 3.6 holds no matter which rotation is active in a given round.

Definition 3.3 (admissible seed).A seed S is admissible if for all k∈ {0,1,2,3}, Rᵏ(S) is invertible over GF(2) and B(Rᵏ(S)) ≥ 4. In the implemented model, invertibility is algebraically guaranteed by R(M) = MᵀJ; nevertheless, the reference implementation verifies all four orientations explicitly for defensive correctness. The key setup procedure enforces this requirement: every seed matrix and all four of its rotor orientations are required to satisfy B(M) ≥ 4 before acceptance.

Corollary (branch-number guarantee). If key setup terminates with admissible seeds per Definition 3.3, then every matrix used by the cipher satisfies B(M) ≥ 4.

3.5. Rotor Scheduling: Key-Dependent Diffusion

Let S₀,...,S₁₅ be the 16 admissible seed matrices derived from master key K. In round r ∈ {0,...,15} and byte position j ∈ {0,...,15}, the active matrix is Mᵣ,_j = R^{(r+j) mod 4}(S_j). Across 16 rounds this yields 256 matrix applications; the 64 distinct (seed, orientation) pairs each appear exactly four times. The schedule satisfies Mᵣ₊₄,_j = Mᵣ,_j (period 4). Remark: round-key injection (distinct for all 16 rounds via SHA-256) breaks the periodicity at the key level even though the matrix schedule repeats. Slide attacks require self-similar rounds, including identical round keys; the 16 round keys are pairwise distinct outputs of a domain-separated hash, which breaks this self-similarity. Likewise, inducing a chosen relation among round keys from a chosen relation among master keys would require constructing related SHA-256 outputs, which is infeasible under standard assumptions on SHA-256; a dedicated related-key analysis is nonetheless identified as future work (Section 6).

3.6. Diffusion Properties of the Rotor Layer

Proposition 3.4.If M is invertible with B(M) ≥ 4, then for any nonzero x, Mx ≠ 0 and wt(Mx) ≥ 4 − wt(x). In particular, a single-bit input produces wt(Mx) ≥ 3.

Proof. Invertibility implies trivial kernel, so Mx ≠ 0 for all nonzero x. The bound wt(Mx) ≥ 4 − wt(x) follows directly from B(M) = min_{x≠0}(wt(x) + wt(Mx)) ≥ 4. Setting wt(x) = 1 gives wt(Mx) ≥ 3. □

Intuitively, the proposition says that no nonzero input can collapse to a near-empty output: flipping a single input bit always activates at least three output bits within the affected byte, so a difference cannot vanish as it passes through the matrix layer.

The constraint B(Mᵣ,_j) ≥ 4 ensures any single active bit entering byte j in round r produces at least 3 active output bits locally. Combined with cross-byte rotation (no schedule value a multiple of 8) and the routing permutation, this yields rapid multi-round diffusion. Empirically, dependency analysis and sampled differentials suggest full-byte diffusion by approximately round 5; with 50,000 pairs under a single-bit input difference, observed output differences are consistent with uniformity by rounds 8 and 12. A tight analytical bound on the number of active S-boxes across rounds does not follow automatically from the per-byte bound B(Mᵣ,_j) ≥ 4; deriving such a bound via the wide-trail strategy [18] (for example, through MILP modeling of the S-box and linear layers [39]) is left for future theoretical analysis. Structurally, the combination of the intra-byte floor B ≥ 4, rotl128 shifts that never align with byte boundaries, and the round-dependent routing permutation is intended to prevent low-weight differences from remaining confined to few bytes across rounds; this is a design argument, not a proved bound. Cross-session experimental support for this result is reported in Section 5.3.

Table 3 summarizes the proof status of the principal security properties of Hill-Enigma-SPN, distinguishing formally proved results from empirically supported claims. This distinction is maintained throughout Section 3 and Section 5.

3.7. Key Schedule: Argon2id-Based Derivation

Given password P and random salt σ ∈ {0,1}¹²⁸, the 256-bit master key is K = Argon2id(P, σ; t = 3, m = 65,536 KiB, p = 2, ℓ = 32) [23]. From K, the 16 admissible seed matrices and the 16 round keys are derived via SHA-256. The 64 MiB memory cost is chosen to raise the cost of GPU/ASIC offline password search. This derivation profile is a property of the password-based operating mode rather than of the block cipher itself: the cipher's security analysis assumes a uniform 256-bit master key K, and any key of that form, however derived, may be used.

Remark (comparison with prior work). Hill cipher variants in the prior literature have predominantly employed SHA-256 directly for key derivation without memory-hard hashing [14]. Argon2id raises the effective cost of brute-force key recovery by several orders of magnitude, providing a significantly stronger security foundation for the key derivation layer.

Keyspace. As computed in Section 3.2, invertible matrices constitute approximately 29% of all 8×8 binary matrices, so the admissibility filter (invertibility and B ≥ 4 across all four orientations) imposes only a modest constraint on the available matrix pool. For a uniformly random 256-bit master key K, exhaustive search over K requires 2²⁵⁶ trials. In password-based operation, effective security is additionally bounded by password entropy; Argon2id strengthens this mode by raising the memory and time cost of each password guess.

3.8. Decryption Well-Definedness

Proposition 3.5 (Bijectivity of the round function).The round function F : {0,1}¹²⁸ → {0,1}¹²⁸ is a bijection.

Proof. Invertibility follows component by component. The two position maps (rotl128 and the routing permutation) are reversible by inspection; the routing modes are self-inverse, since each transposes a single pair of index bits. Round-key addition is an XOR involution, and the AES S-box has a tabulated inverse. Each rotor matrix Mᵣ,_j lies in GL(8,2), guaranteed by R(M) = MᵀJ together with the admissibility filter of Section 3.4, and is therefore invertible over GF(2). A composition of bijections is itself a bijection, so each round, and hence the full 16-round map, is a permutation of {0,1}¹²⁸. □

For each round in reverse round order, the inverse cipher applies the five inverse operations: inverse routing (the same permutation, self-inverse), the inverse AES S-box, Mᵣ,_j⁻¹ over GF(2), XOR with rkᵣ (self-inverse), and finally rotr128 by K_VALUES[r mod 16].

3.9. Computational Complexity Analysis

The computational cost of Hill-Enigma-SPN is considered at three levels indicated in Table 4: (i) per-operation, (ii) per-round, and (iii) full encryption. All counts use elementary bit-level operations (AND, XOR, shift, table lookup); here a single “operation” denotes one such elementary step on a machine word, and an 8-bit table lookup is counted as one operation. These counts are an abstract instruction tally and are not intended to predict wall-clock time, which depends on the platform, word size, and compiler. This analysis establishes computational feasibility and quantifies the performance trade-off relative to AES-128.

8×8 GF(2) matrix-vector multiplication: Each of the 16 byte positions in Step 3 requires one matrix-vector multiply over GF(2). Using the popcount form of Section 3.1: v(y)[i] = popcount(m_i AND x) mod 2, i = 0,...,7. This requires 8 bitwise AND operations, 8 popcount operations, and 8 mod-2 reductions: approximately 24 bit-level operations per matrix multiply, with no field arithmetic beyond GF(2) (refer to Table 4).

Per-round and full-cipher cost: Step 3 (16 matrix multiplies) dominates at approximately 85% of per-round cost. The full 16-round encryption requires approximately 7,232 bit-level operations per 128-bit block, excluding key derivation (refer to Table 4).

AES-128 comparison: AES-128 (10 rounds) requires approximately 2,080 operations per block, estimated from the round structure specified in FIPS 197 [40] and the GF(2⁸) arithmetic analysis of Daemen and Rijmen [21], using a conservative upper estimate of approximately 160 operations for MixColumns per round.

Hill-Enigma-SPN: HESPN requires approximately 3.5× more operations [21,36,40]. This figure is approximate and implementation-dependent, reflecting the abstract instruction counts of Table 4 rather than any specific optimized build. The overhead is the deliberate cost of the rotor scheduling mechanism with 16 independent key-dependent matrix orientations per round versus AES’s single shared MixColumns, and is analogous to Serpent, which accepts lower throughput for stronger security margins.

Reference implementation performance: In the pure Python prototype at 16 rounds, throughput is approximately 27 bytes/s (36 bytes/s measured at 12 rounds). A vectorized NumPy implementation achieves approximately 6 MB/s (8 MB/s at 12 rounds). An optimized C implementation is projected at approximately 60 MB/s (a first-order extrapolation from the NumPy implementation's operation profile; no C benchmark has yet been performed), compared to 460–1,900 MB/s for AES-128-CBC and approximately 40–100 MB/s for Serpent [33] in software. PRESENT [34] is notably slower than HESPN’s C estimate in software due to its bit-oriented architecture.

4. Round Function and Full Cipher Specification

This section specifies the Hill-Enigma-SPN round function and full encryption/decryption, as implemented in the reference implementation (HESPNv4Rerun.py, available as supplementary material). Parameters: block size 128 bits (16 bytes); ROUNDS = 16; byte-local 8×8 binary matrices over GF(2); AES S-box; round-dependent routing permutation. Confusion in the sense of Shannon [1] is achieved by the AES S-box substitution layer (Step 4): with nonlinearity NL = 112 and algebraic degree 7, each ciphertext bit depends on the round key through a highly nonlinear composed function across all 16 rounds, obscuring the statistical relationship between key and ciphertext. The detailed scheme is shown in Figure 2.

4.1. State and Notation

Let state be a 128-bit value represented as 16 bytes (state [0],...,state [15]) in big-endian order. Let rotl128(b, k) denote left rotation of the 128-bit state b by k positions, where k ∈ {1,...,127} and no k in the schedule is a multiple of 8. Let rkᵣ be the r-th 16-byte round key. For round index r, the rotation amount is K_VALUES[r mod 16] with K_VALUES = [1,3,5,7].

4.2. Rotor Matrices per Round

Let S_j (j = 0,...,15) be the 16 admissible seed matrices produced during key setup (cached). In round r, byte position j uses Mᵣ,_j = R^{(r+j) mod 4}(S_j). As noted in Section 3.5, this schedule yields 256 matrix applications across 16 rounds, with the 64 distinct (seed, orientation) pairs each appearing exactly four times. All matrices in the schedule are admissible: invertible over GF(2) with B(Mᵣ,_j) ≥ 4 for all r and j.

4.3. Routing Permutation

Define a 4-mode permutation πᵣ on byte indices j ∈ {0,...,15} by swapping one selected bit of the 4-bit index according to mode r mod 4. Mode 0: identity. Mode 1: swap index bits 0 and 1. Mode 2: swap index bits 0 and 2. Mode 3: swap index bits 0 and 3. The mapping redistributes substituted bytes to achieve inter-byte diffusion each round. Each mode is self-inverse, so the inverse permutation equals the permutation itself. Remark: rotl128 provides cross-byte bit motion every round regardless, so no round is without inter-byte diffusion.

4.4. Round Function

At a high level, a single round mixes the block in five stages. It first rotates the whole 128-bit state to carry bits across byte boundaries, then adds a round key. Each byte is then passed independently through its scheduled GF(2) matrix, which provides intra-byte diffusion, and through the AES S-box, which provides nonlinearity. Finally, the byte positions are permuted to spread the result across the block. The formal specification follows.

Given 16-byte input state b, master key K, and round index r ∈ {0,...,15}:

Step 1. Rotate: state ← rotl128(b, K_VALUES[r mod 16]).

Step 2. Add round key: state ← state ⊕ rkᵣ.

Step 3. Per-byte linear layer: for each j = 0,...,15, y_j ← Mᵣ,_j(state[j]).

Step 4. S-box: for each j, z_j ← AES_SBOX[y_j].

Step 5. Routing: output byte at index πᵣ(j) ← z_j. Return new 16-byte state.

4.5. Encryption and Decryption

Encryption applies the round function sequentially for r = 0,...,ROUNDS − 1 with the schedule above.

Decryption applies the inverse operations in reverse order across all 16 rounds: inverse routing (same permutation, self-inverse), inverse AES S-box, inverse matrix Mᵣ,_j⁻¹ over GF(2) (which exists because all rotor orientations are invertible, proved via R(M) = MᵀJ and guaranteed by the admissibility filter; the inverse is computed as the GF(2) matrix inverse), XOR with rkᵣ (self-inverse), then rotr128 by K_VALUES[r mod 16].

4.6. Key Setup

From password P and salt σ, derive K via Argon2id (t = 3, m = 65,536 KiB, p = 2, ℓ = 32). Derive round keys as rkᵣ = SHA256(K ∥ “ROUNDKEY” ∥ r)[:16], with the round index r encoded as a 2-byte big-endian integer.

Derive each seed matrix S_j by hashing K with domain separator “MATRIX”, byte index j (encoded as one byte), and an incrementing counter (4-byte big-endian) until an admissible matrix family is found: all four orientations {Rᵏ(S) : k = 0,1,2,3} must be (i) invertible over GF(2) which was verified explicitly for defensive correctness and algebraically guaranteed by R(M) = MᵀJ, and (ii) satisfy B(Rᵏ(S)) ≥ 4. Cache the 16 seeds for reuse across all 16 rounds. Because admissible seeds are found by rejection sampling, key setup runs in variable time; this cost is incurred once per key, is independent of any plaintext or ciphertext, and only the accepted matrices, not the rejection counts, enter the cipher. A formal side-channel assessment of key setup is deferred to future work. In the revision-phase reruns, the measured rejection cost across 144 independent key instantiations was a mean of approximately 115 candidate matrices tested per seed (median 79, maximum 1,079), or roughly 1,830 candidates per full 16-seed key setup.

4.7. Security Claim and Round-Count Rationale

The design target for HESPN is resistance to key recovery below generic exhaustive search of a uniformly random 256-bit master key and resistance to permutation distinguishers below approximately 2¹²⁸ data/time. These targets are not established in this paper; the evidence supplied here is limited to the proved bijectivity and branch-number properties and bounded empirical tests, and the targets remain to be examined by third-party cryptanalysis. Table 3 records which supporting properties are formally established and which are empirical.

The round count of 16 is calibrated empirically. The analyses of Section 5 indicate statistical saturation of the classical single-input metrics (full avalanche, disappearance of sampled differential collisions, and saturation of the observable algebraic degree) by round 8 across independent sessions. Keystream randomness testing under NIST SP 800-22 (Section 5.6), however, showed that decorrelation of structurally related inputs completes later: 12-round counter-mode keystream retains a small but measurable correlation between ciphertexts of low-weight-difference inputs, and a round-count sweep places the disappearance of this effect at approximately 14 rounds. Sixteen rounds therefore provide a two-round margin over the most demanding empirically observed convergence point, and a 100% margin over round-8 saturation of the classical metrics, in the spirit of the conservative security margins adopted by AES and Serpent [21,33]. Because empirical convergence is not an attack bound, this margin remains heuristic; a trail-based justification of the round count is future work (Section 6).

4.8. Scope: Modes of Operation and Integrity

This paper specifies and analyzes the block primitive; deployment additionally requires a mode of operation and integrity protection, which are outside the present scope. As a 128-bit block cipher, HESPN can be used in standard modes such as CTR or CBC with a uniformly random 128-bit IV per message, composed with an encrypt-then-MAC integrity layer (for example, HMAC-SHA-256 under an independent key derived from K), or employed as the block cipher within a generic AEAD composition. Counter-mode keystream randomness for the final 16-round primitive is assessed directly in Section 5.6. The single-block test vector of Appendix A exercises the primitive only. The paper does not define a complete password-based encryption file format; it specifies only the block primitive and the Argon2id KDF profile.

5. Empirical Security Analysis

This section presents experiments conducted across four independent experimental sessions with distinct passwords and salts.¹ Session 1 produced the initial avalanche measurements and the differential distribution (Section 5.3) results using two independent runs (salts 5089... and cc02...) with the SHA-256 stub for computational efficiency.¹ Sessions 2 and 3 produced the primary linear-bias results (Section 5.4) at the final 16-round parameterization, each contributing 500 logged trials across two independent runs with distinct passwords and salts (prefixes 5CF0… and 6E2F…), using the Argon2id production implementation.² A preliminary linear-bias check (200 trials × 10,000 samples) served as an initial confirmation prior to the primary runs. Session 4 (revision phase) produced the high-precision avalanche results (Section 5.2), the algebraic-degree results (Section 5.5), the admissibility-filter statistics (Section 5.1), the NIST SP 800-22 keystream evaluation (Section 5.6), and the 16-round differential confirmation (Section 5.3, experiments [E] and [F]), using the Argon2id production implementation throughout. Test sizes are set to provide stable estimates at reasonable runtime. All experiments use the MSB-first convention of Section 3.1. Statistical properties are stable across sessions; the Argon2id production version is structurally identical to the stub with respect to the round function, rotor scheduling, S-box layer, and routing permutation.

5.1. Branch Number Summary

For each round r = [0,...,15], we instantiate the 16 rotor matrices Mᵣ,_j and compute branch numbers. Table 5 reports per-round min/max/mean and overall statistics.

The theoretical maximum for an 8×8 binary matrix is 9. All 256 matrix applications achieve branch number exactly 4, the minimum guaranteed by the admissibility filter and consistent with strong intra-byte diffusion per round. This is a consequence of the filter accepting the first seed satisfying B(Rᵏ(S)) ≥ 4 for all k, rather than maximizing branch number. A direct sampling experiment conducted during revision quantifies the filter: of 20,000 uniformly random candidate matrices, 28.86% were invertible (theory: |GL(8,2)|/2⁶⁴ ≈ 28.99%), and 3.24% of invertible families satisfied B ≥ 4 in all four orientations, corresponding to approximately 107 candidates per accepted seed, consistent with the measured key-setup cost reported in Section 4.6. No family achieved B ≥ 5 in all four orientations in the sample, indicating that raising the threshold to B ≥ 5 would multiply the rejection search by at least two orders of magnitude if such families exist at all; the exact density remains a combinatorial question for future analysis. Note that over GF(2⁸), MDS diffusion (B = 9) is achievable as in AES MixColumns; over GF(2), higher branch numbers require denser search.

5.2. Avalanche Effect

Table 6 reports plaintext and key avalanche results across round counts r ∈ {1, 2, 4, 5, 8, 12, 16} from the high-precision revision rerun (Session 4, Argon2id production implementation): 5,000 plaintext trials and 200 key trials per round count, reported with 95% confidence intervals. For plaintext avalanche, one random bit of the plaintext is flipped and the Hamming distance between the two ciphertexts is measured; for key avalanche, one random bit of the 256-bit master key is flipped, requiring a full key setup for the mutated key in each trial. The ideal target is 64 bits (50% of 128 bits), consistent with the avalanche criterion for substitution–permutation networks [41]. Two earlier independent 60/30-trial runs (Session 1, SHA-256 stub, salts 5089… and cc02…) agree with these values at all shared round counts; their plaintext means at r = 12 were 64.60 and 64.10.

Plaintext avalanche reaches 64.03 bits at 16 rounds (σ = 5.71, 95% CI [63.87, 64.19]), with the confidence interval containing the ideal value of 64. At 12 rounds the high-precision run resolves a small but statistically significant deficit (mean 63.45 bits against the ideal 64, 95% CI [63.29, 63.61]) that is invisible at 60-trial precision; this residual is the avalanche-side signature of the related-input correlation analyzed in Section 5.6 and is eliminated by the additional four rounds. Key avalanche is statistically indistinguishable from the ideal 64 bits at every round count from r = 1 onward, reflecting the fact that flipping one master-key bit rederives all seed matrices and round keys. This key-avalanche result is therefore best interpreted as a schedule-rekeying measurement rather than as progressive diffusion under a fixed round schedule. Substantial plaintext diffusion is observed by round 8 (mean 54.52), with full convergence at r = 16.

5.3. Differential Distribution Estimation

With 50,000 random pairs per experiment, no two output differences coincided at rounds 8 and 12 in either run; the resulting estimate, 2×10⁻⁵, is therefore the smallest value the sample size can resolve and should be read as a resolution limit rather than as a measured differential probability. At round 4, a few collisions remained (49,479 and 49,485 distinct differences across the two runs), indicating that diffusion was not yet complete. All sampled output differences are distinct from round 8 onward, and this behavior is sustained through round 12.

A 16-round confirmation was performed at the final parameterization across two independent Argon2id sessions (salts F7F0... and 4DD1...), extending the probe beyond the single representative difference to a panel of twelve chosen input differences spanning byte positions: single-bit differences at several bit and byte positions, single-byte (full 0xFF) differences at byte positions 0, 5, 10, and 15, and low-weight multi-byte differences (two-byte and four-byte, each of Hamming weight one per active byte). For every tested input difference in both sessions, all 50,000 output differences were distinct (experiments [E] and [F] in Table 7), so the largest multiplicity of any single output difference was one, at the 2×10⁻⁵ resolution limit. No output-difference clustering was observed at 16 rounds for any tested difference. This confirmation is a sampling screen of the same kind as the reduced-round experiments and carries the same interpretation: it rules out only differentials whose probability exceeds approximately 1/50,000, and does not substitute for the trail enumeration identified as future work.

Because this resolution limit was attained for experiments [B], [C], and [D] in both runs, the early onset of empirical diffusion saturation at round 8 on the tested metrics appears to be a stable structural property of the cipher rather than an artifact of any single key or salt. Random-pair sampling of this kind can detect only differentials whose probability exceeds approximately 1/50,000 (roughly 2⁻¹⁶); a trail-optimized differential characteristic of much lower probability would remain exploitable yet invisible to this experiment [42]. Establishing resistance against such characteristics requires explicit trail enumeration (for example, via MILP modeling of the S-box and linear layers [39]) and is identified as future work in Section 6.

5.4. Linear-Bias Probe

Linear bias was evaluated at the final 16-round parameterization across two independent sessions, each targeting 500 randomly selected input/output parity mask pairs (α, β ∈ {0,1}¹²⁸) assessed over 50,000 plaintext samples. Both sessions completed the full 500 trials, for a combined total of 1,000 trials × 50,000 samples, each session using a fresh password and a fresh random salt under the Argon2id production implementation.² For each mask pair, the reported statistic is the absolute deviation of the observed parity agreement rate from 1/2. The scale 1/√50,000 ≈ 0.00447 is used as a conservative screening reference, not as a formal multiple-comparison threshold. Because 896 mask pairs were tested, the maximum observed value is interpreted as a maximum over many null trials rather than as a binary pass/fail certificate. A preliminary run of 200 trials × 10,000 samples (screening scale 1/√10,000 = 0.010) served as an initial check.

Conclusion: The primary linear-bias probe comprised 1,000 logged trials × 50,000 samples across two independent 16-round sessions (Run 1, salt 5CF0…; Run 2, salt 6E2F…), each using a distinct password and salt under the Argon2id production implementation. Across the two sessions the pooled mean |bias| was 0.00177 and the overall maximum |bias| was 0.00752, against a per-sample standard error of σ = 0.5/√50,000 = 0.00224; 44 of the 1,000 trials (4.4%) exceeded the 2σ screening reference of 0.00447. Each of these figures matches its null expectation for an unbiased estimator at this sample size: the expected mean is σ√(2/π) = 0.00178, the expected maximum over 1,000 null trials is approximately 3.3σ = 0.0074, and the expected exceedance rate at 2σ is 4.55%. The distribution of the screen statistic is therefore indistinguishable from noise, and no structured linear correlation is revealed at the tested scale. This should not be read as a formal upper bound on all linear correlations, because the distribution of the maximum statistic depends on the number of masks tested and on the null distribution of the estimator. Random parity masks constitute a null-hypothesis screen rather than evidence of resistance to linear cryptanalysis: a practical linear attack employs masks constructed from single-round approximations propagated through the cipher as a linear trail [42], and the probability that randomly drawn masks coincide with a high-correlation trail is negligible. A trail-driven linear analysis is deferred to future work.

5.5. Algebraic Degree Lower Bounds

We estimate lower bounds on algebraic degree using restricted-variable Algebraic Normal Form (ANF) experiments with t = 12 active input bits per trial (4,096 encryptions per trial) and 16 trials per round count (Session 4). An earlier run with t = 6 active bits and 4 trials per round produced consistent but coarser results. Table 9 summarizes the t = 12 results.

Theoretical maximum algebraic degree = 12 (constrained by t = 12 active input bits per trial). Individual zero-degree trials at low round counts are Monte Carlo sampling artifacts in which random active-bit selections fail to witness high-degree monomials; with 16 trials per round, the best lower bound at r = 2 is 3, resolving the isolated zero recorded in the earlier t = 6 run.

Observable degree (theoretical max = t = 12) reaches its maximum by round 8 and remains saturated through rounds 12 and 16. These are restricted-variable lower bounds, not full-circuit degree characterizations; they should not be interpreted as directly comparable to AES degree saturation, which is established under a different experimental framework. The influence of the linear layer on algebraic-degree growth in SP networks is analyzed by Cid et al. [43]; the restricted-variable methodology used here yields lower bounds only and cannot detect the degree-stagnation phenomena studied there.

5.6. Keystream Randomness (NIST SP 800-22) and Round-Count Calibration

Keystream randomness was evaluated on HESPN in counter mode: the keystream is the concatenation of encryptions of sequential counter blocks 0, 1, 2, …, partitioned into 100 sequences of 10⁶ bits and assessed under the two-level methodology of NIST SP 800-22 [44]. This methodology combines a per-test pass proportion with a three-sigma acceptance region at α = 0.01 and χ² uniformity of the per-sequence p-values with threshold P_T ≥ 10⁻⁴. A core battery of seven tests (nine p-values), comprising frequency, block frequency, runs, longest run of ones, cumulative sums (forward and backward), serial (m = 2), and approximate entropy (m = 2), is implemented in the supplementary programs and was validated against the worked examples of SP 800-22; the generated keystream file is retained so that the full 15-test battery can be run under the official STS tool.

At 12 rounds this evaluation revealed a genuine structured-input bias. The frequency-sensitive family failed decisively: frequency 86/100 sequences passing, cumulative sums 84–86/100, serial 88/100, and approximate entropy 65/100, each with collapsed p-value uniformity (P_T < 10⁻⁴). The locally scoped tests, in contrast, passed (block frequency 99/100, longest run 97/100). This signature indicates inflated per-sequence ones-count variance (measured at 1.5–1.7× the ideal) arising from correlation between ciphertexts of related inputs rather than from any marginal nonuniformity: the same battery passes when the cipher encrypts independent random plaintexts, and diagnostic measurements place the mean pairwise Hamming distance between encryptions of counter values at fixed strides at 63.6–63.9 bits against the ideal 64.0, with deviations of up to 8.3 standard errors at 20,000 pairs per stride. The 12-round plaintext-avalanche deficit of Section 5.2 (63.45 bits, confidence interval [63.29, 63.61]) is the same phenomenon viewed through single-bit differences: at 12 rounds, decorrelation of low-weight input differences is measurably incomplete. In distinguisher terms, this constitutes a practical statistical distinguisher on the 12-round variant under known structured plaintexts at roughly 2²⁰ blocks of data, and it is the reason the 12-round parameterization was abandoned.

A round-count sweep (20 sequences per configuration, nine p-values each) localizes the convergence point: 12 rounds produced 9 failing tests of 180, with four per-test pass proportions below the acceptance region, while 14, 16, and 20 rounds each produced a single failing test of 180, nominal at α = 0.01 (expected count 1.8). The stride and sweep measurements, and the 300-sequence confirmation run, are exactly reproducible via the supplementary programs, which derive their keys deterministically; the confirmation run is regenerated by HESPNv4Confirm300.py under a fixed diagnostic key and was verified to reproduce identically across independent platforms (Windows with Python 3.14 and Linux with Python 3.12 producing the same master key and per-test counts). The round count was therefore set to 16 (Section 4.7). At 16 rounds, the 100-sequence core-battery run was broadly healthy but borderline in the runs/serial family: every p-value uniformity statistic was healthy (minimum P_T = 0.031), block frequency and longest run passed 100/100, approximate entropy 99/100, cumulative sums 98–99/100, and frequency 97/100, while runs and serial were 96/100 near the lower three-sigma boundary of 96.02. Because runs and serial (m = 2) are strongly correlated statistics, a single borderline sequence can depress all three proportions simultaneously. An extended confirmation run of 300 sequences under a fixed reproducible key showed fully nominal behavior: 34 failing tests of 2,700 against an expectation of 27, with every proportion inside its acceptance region (runs family 296/300 = 0.987 against a bound of 0.973). The 16-round avalanche mean of 64.03 with confidence interval containing 64 (Table 6) independently corroborates the completed decorrelation.

5.7. Interpretation and Limitations

The experiments support the structural diffusion claims of Section 3.6: (i) rapid avalanche convergence by round 8, with full convergence to 64.03 bits (confidence interval containing 64) at 16 rounds; (ii) low differential probabilities at 50,000 samples, consistent with ideal permutation behavior by round 8 in both Session-1 runs and confirmed at 16 rounds across a panel of twelve chosen input differences in two further sessions (Section 5.3); (iii) no structured correlation identified in the random-mask linear-bias screen at 16 rounds, with the screen-statistic distribution matching its null expectation at the tested scale; (iv) algebraic degree lower bounds saturating at the maximum observable value (t = 12) by round 8; and (v) counter-mode keystream randomness under the implemented NIST SP 800-22 core battery at 16 rounds, with borderline 100-sequence runs/serial results supported by a 300-sequence confirmation run and with the 12-round structured-input bias identified, characterized, and eliminated by the round-count increase (Section 5.6).

Further, results are drawn from four independent experimental sessions with distinct keys and salts: Session 1 produced the differential results (Table 7) and the initial avalanche measurements; Sessions 2 and 3 produced the linear-bias results (Table 8) at 16 rounds, contributing 500 logged trials each for a combined total of 1,000 trials × 50,000 samples; Session 4 (revision phase) produced the high-precision avalanche results (Table 6), the algebraic-degree results (Table 9), the admissibility statistics (Section 5.1), and the NIST SP 800-22 evaluation (Section 5.6).

These results are empirical signals rather than proofs: they describe observed behavior under bounded, single-key sampling, not guaranteed bounds. A formal trail-based analysis and dedicated structural cryptanalysis are deferred to the future work outlined in Section 6. Table 3 separates what the paper proves from what it only observes. The differential and linear measurements of Section 5.3 and Section 5.4 were performed at round counts up to 12 and therefore characterize reduced-round versions of the final 16-round cipher; each had already reached its observable ceiling (the sampling-resolution floor and the statistical noise floor, respectively) by rounds 8–12, and the additional four rounds apply further bijective mixing of the same form. The avalanche, degree, and keystream measurements (Section 5.2 and Section 5.5, and 5.6) were performed at the final round count.

¹ The Session-1 differential-distribution experiments (Section 5.3) and the initial avalanche runs were conducted using a SHA-256 stub in place of the full Argon2id derivation; the Session-4 experiments of Section 5.2 and Section 5.5, and 5.6 use the Argon2id production implementation. The stub was adopted due to the computational constraints of the reference hardware (Dell Precision 3581, Intel Core i5 13th gen, 4.60 GHz): Argon2id’s intentional memory hardness (t = 3, m = 65,536 KiB, p = 2) imposes a significant per-key cost that is prohibitive at the trial volumes required for these experiments. The stub replaces the Argon2id call with SHA-256(password ∥ salt) and produces a 256-bit master key of equivalent length. The round function, rotor scheduling, S-box layer, and routing permutation are identical in both versions. Key derivation method does not affect the structural properties measured here; the reported metrics reflect the cipher’s diffusion, differential, and algebraic properties independent of key derivation cost. The Argon2id production implementation (HESPNv4Rerun.py) is available as supplementary material.

² The linear-bias probe (Section 5.4) was conducted at the final 16-round parameterization using the Argon2id production implementation (t = 3, m = 65,536 KiB, p = 2), as the lower per-trial cost at this experiment’s sample size made full key derivation feasible. Sessions 2 and 3 used distinct passwords and salts (prefixes 5CF0… and 6E2F…); the round function and all cipher operations are structurally identical to the SHA-256 stub version used in Section 5.3 and in the initial Session-1 avalanche runs. The probe was generated by the supplementary script HESPNv4LinearBias16.py, which verifies itself against the Appendix A reference test vector at startup and writes a timestamped log recording the password, salt, and per-trial masks for each session.

6. Conclusions

This paper has presented Hill-Enigma-SPN (HESPN), a 128-bit substitution–permutation network block cipher that extends the classical Hill cipher through a novel integration of three structural components: (i) rotor-scheduled GF(2) matrix diffusion with formally verified branch number B ≥ 4, (ii) AES S-box substitution providing established cryptographic nonlinearity, and (iii) a routing permutation for inter-byte diffusion, together with (iv) a specified Argon2id memory-hard key-derivation profile for password-based operation.

Empirical security analysis across four independent experimental sessions indicates that in the HESPN architecture, measured properties align with those expected of a well-diffused substitution–permutation construction under the evaluated metrics. Plaintext avalanche reaches 64.03 bits at 16 rounds (95% confidence interval [63.87, 64.19], containing the ideal 64-bit target). No differential collisions were observed at rounds 8 and 12; the corresponding figure of 2×10⁻⁵ is the sample-size resolution floor and indicates the absence of a distinguisher at the tested scale rather than a bound on differential probability. A random-mask linear-bias screen at 16 rounds across 1,000 combined trials with two distinct Argon2id salts found a pooled mean |bias| of 0.00177 and a maximum of 0.00752, with the screen-statistic distribution matching its null expectation for an unbiased estimator at this sample size. Algebraic degree lower bounds saturate at the maximum observable value by round 8. These are empirical observations from bounded experiments; they do not establish resistance against trail-optimized differential or linear cryptanalysis (Section 5.3 and Section 5.4). A revision-phase NIST SP 800-22 core-battery evaluation of counter-mode keystream identified a structured-input correlation at 12 rounds, characterized its mechanism, and motivated the final round count of 16, at which the 100-sequence core-battery run is broadly healthy and a 300-sequence confirmation run shows fully nominal behavior (Section 5.6).

The rotor scheduling mechanism, inspired by the Enigma Encoding Machine [20] and prior work [6,7], and formalized through GF(2) admissible matrix families, provides 64 distinct (seed, orientation) matrix pairs, each appearing exactly four times across 16 rounds. This structural combination gives Hill-Enigma-SPN (HESPN) a more explicitly specified and empirically screened design basis than many prior Hill-cipher variants, while remaining below the level of assurance provided by standardized ciphers and still requiring independent cryptanalysis. Among the designs surveyed here, HESPN appears to be the only one combining a GF(2) rotor-scheduled matrix diffusion layer with formally verified branch number, AES S-box nonlinearity (DU = 4, NL = 112), and Argon2id memory-hard key derivation. In practical terms, HESPN is presented as a research construction rather than a deployment recommendation: settings lacking AES-NI are already well served by standardized software-oriented ciphers such as ChaCha20 [38], and independent third-party cryptanalysis must precede any practical use. The value of the construction lies in demonstrating that the rotor-scheduled GF(2) diffusion model can be integrated into a complete substitution–permutation network with sound statistical behavior on the evaluated metrics.

Future work specific to this construction includes (i) a wide-trail active-S-box bound for the per-byte GF(2) rotor layer, built on the B ≥ 4 floor of Section 3, for example via MILP modeling of the S-box and linear layers [39]; (ii) optimizing the rotor-matrix dimension and branch-number threshold; and (iii) a hardware throughput study. Conventional third-party cryptanalysis (related-key, integral, boomerang, and impossible-differential analysis, together with a side-channel evaluation of the rotor schedule) would then follow and are the subject of manuscripts in progress.

Supplementary Materials

The following supporting information is available from the journal repository, GitHub at https://github.com/ja9925ydbsu/Hill-Enigma-SPN-HESPN-COGGINS, or by contacting the corresponding author: hespn_test_vector_v4.py (generates and verifies the complete 16-round test vector, including all 16 round keys, all 16 seed matrices with branch numbers, per-round intermediate states, and a full decryption round-trip check; needs nothing beyond Python’s standard library); HESPNv4Rerun.py (reference implementation together with the full experimental suite of Section 5; Argon2id key derivation, with a table-driven fast path verified bit-exact against the reference code at startup); and HESPNv4Diagnostics.py (reproduces the structured-input diagnostics of Section 5.6, namely the control batteries, the stride Hamming-distance measurements, and the round-count sweep, under fixed diagnostic keys for exact reproducibility); HESPNv4LinearBias16.py (regenerates the two independent 16-round linear-bias sessions of Section 5.4, deriving a fresh password and salt per session under Argon2id, self-verifying against the Appendix A test vector at startup, and logging the session salt and per-trial masks); HESPNv4Differential16.py (regenerates the 16-round differential confirmation of Section 5.3 over the panel of chosen input differences, with the same per-session Argon2id derivation, startup self-verification, and logging); and HESPNv4Confirm300.py (regenerates the 300-sequence NIST SP 800-22 core-battery confirmation of Section 5.6 under a fixed diagnostic key, with the core battery validated against the SP 800-22 worked examples and the cipher self-verified against the Appendix A test vector at startup). The test-vector generator and the reference implementation realize the identical cipher (the generator substitutes a SHA-256 stub for Argon2id solely so that the vector is reproducible without third-party packages), and the normative byte-level encodings of the key expansion are stated in Section 4.6 and Appendix A.