Multiplication-Free Graph Spectral Embeddings: Bit-Sliced Repeated AX and A⊤ AX Products

1. Introduction

Spectral methods provide a powerful framework for analyzing the structure of graphs. Techniques like spectral clustering [2,3] and Laplacian Eigenmaps [1] have become standard tools in machine learning and data analysis for tasks such as community detection, dimensionality reduction, and manifold learning. These methods derive meaningful low-dimensional embeddings of the data by computing the eigenvectors of a graph’s adjacency matrix or, more commonly, its Laplacian matrix. The eigenvectors corresponding to the smallest eigenvalues capture the smoothest possible mappings of the graph onto a lower-dimensional space, thus revealing the underlying cluster or manifold structure.

A key computational step in these spectral methods is the calculation of these eigenvectors, which is often performed using iterative algorithms like the power method or the Lanczos algorithm [5]. These algorithms require repeated multiplications of the graph matrix (e.g., the adjacency matrix A or the Laplacian L) with a vector. For large graphs, these matrix-vector products, particularly operations like $AX$ and $A^{\top }AX$, dominate the computational cost. This paper introduces a bit-slicing approach that transforms these expensive matrix multiplications into a series of highly efficient Boolean operations, thereby offering a multiplication-free path to accelerating graph spectral embedding algorithms.

2. Method

The bit-slicing methodology is applied directly to the core matrix operations required by spectral embedding algorithms. The primary operations of interest are the matrix-vector products $AX$ and the symmetric product $A^{\top }AX$, where A is the graph’s adjacency matrix and X is a matrix of vectors.

Assuming the adjacency matrix A contains integer weights (with binary graphs being a special case where weights are 0 or 1), the product $AX$ is computed using the bitsliced_mm function described in the Appendix. This function decomposes both matrices A and X into their constituent bitplanes. It then computes the product by performing Boolean matrix multiplications on these bitplanes and accumulating the results with appropriate bit shifts. The product $A^{\top }AX$ is simply computed as a nested application of the same function: first computing $Y=AX$, and then computing $A^{\top }Y$.

For methods that use the graph Laplacian, $L=D-A$, where D is the diagonal degree matrix, we can compute the product $LX$ without explicitly forming L. The product is calculated as $LX=DX-AX$. The $AX$ term is computed as described above. The $DX$ product is a simple element-wise multiplication of the vectors in X by the corresponding diagonal entries of D, which is computationally inexpensive. The final result is obtained by subtracting the two resulting matrices. This avoids the need to perform bit-sliced multiplication on the potentially denser Laplacian matrix, leveraging the sparsity of the adjacency matrix instead.

3. Computational Complexity

The computational advantage of the bit-slicing approach stems from the complete elimination of scalar multiplications within its core loop. The standard method for computing a matrix-product like $AX$ is dominated by multiply-accumulate operations, with a complexity of $O\left(n^3\right)$ for general $n\times n$ matrices, or $O\left(n^2\right)$ for sparse matrices with an average of n non-zero entries per row [4].

In our proposed method, the computation is shifted to the bit-domain. The cost is determined by the number of bitwise operations (AND, POPCNT) and additions. For an $m\times k$ adjacency matrix A and a $k\times n$ vector matrix X, with data quantized to w bits, the complexity is proportional to $m·n·k·w^2$. While the dependency on $w^2$ might seem unfavorable, the extreme efficiency of bitwise operations on modern hardware, especially when processing 64 or 128 bits in parallel using SIMD instructions, leads to substantial performance gains in practice. The arithmetic consists only of bit shifts and additions to accumulate the final result, making the core of the algorithm entirely multiplication-free.

4. Numerical Example

To demonstrate the bit-slicing process for a graph kernel, let’s consider the computation of $A^{\top }AX$ with a simple binary adjacency matrix A and an integer vector X:

$A=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]$, $X=\left[\begin{array}{c}1\\ 2\end{array}\right]$

First, we compute the product $Y=AX$ using the bit-sliced method. The matrix A is already binary. We decompose X into its bitplanes. The maximum value in X is 2, so we need 2 bits ($w=2$):

$X_0=\left[\begin{array}{c}1\\ 0\end{array}\right]$ (LSB plane), $X_1=\left[\begin{array}{c}0\\ 1\end{array}\right]$ (MSB plane)

The Boolean products for each bitplane are:

• Bit 0: $A·X_0=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{c}1\\ 0\end{array}\right]=\left[\begin{array}{c}0\\ 1\end{array}\right]$

• Bit 1: $A·X_1=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{c}0\\ 1\end{array}\right]=\left[\begin{array}{c}1\\ 0\end{array}\right]$

We accumulate these results with the appropriate bit shifts to get $Y=AX$:

$Y=(2^0·\left[\begin{array}{c}0\\ 1\end{array}\right])+(2^1·\left[\begin{array}{c}1\\ 0\end{array}\right])=\left[\begin{array}{c}0\\ 1\end{array}\right]+\left[\begin{array}{c}2\\ 0\end{array}\right]=\left[\begin{array}{c}2\\ 1\end{array}\right]$

Next, we compute $A^{\top }Y$. Since A is symmetric, $A^{\top }=A$. We decompose our intermediate result Y into bitplanes:

$Y_0=\left[\begin{array}{c}0\\ 1\end{array}\right]$ (LSB plane), $Y_1=\left[\begin{array}{c}1\\ 0\end{array}\right]$ (MSB plane)

The Boolean products are:

• Bit 0: $A^{\top }·Y_0=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{c}0\\ 1\end{array}\right]=\left[\begin{array}{c}1\\ 0\end{array}\right]$

• Bit 1: $A^{\top }·Y_1=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{c}1\\ 0\end{array}\right]=\left[\begin{array}{c}0\\ 1\end{array}\right]$

Finally, we accumulate the results to get the final vector:

$A^{\top }AX=(2^0·\left[\begin{array}{c}1\\ 0\end{array}\right])+(2^1·\left[\begin{array}{c}0\\ 1\end{array}\right])=\left[\begin{array}{c}1\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 2\end{array}\right]=\left[\begin{array}{c}1\\ 2\end{array}\right]$

This result matches the standard integer matrix multiplication, and the process was completed without any scalar multiplications, as validated by the executable code in the Appendix.

5. Discussion

The bit-sliced approach to graph spectral embedding offers a compelling trade-off between computational efficiency and numerical precision. The use of an integer-based, multiplication-free core provides several advantages. For unweighted graphs, where the adjacency matrix is naturally binary, the computations are exact, avoiding the potential pitfalls of floating-point inaccuracies. This is a significant benefit for algorithms where the precise structure of the graph is critical.

For weighted graphs, the method requires the edge weights to be quantized to a fixed-point integer representation. The choice of the number of bits for quantization presents a trade-off: a higher number of bits allows for a more precise representation of the weights but increases the computational load and memory footprint of the bit-slicing algorithm. However, for many applications, a relatively low-precision representation is sufficient to capture the essential structure of the graph, making this a viable and attractive option.

While the bit-slicing approach can be memory-intensive due to the expansion of the input matrices into bitplanes, there are opportunities for optimization. For sparse graphs, many of the bitplanes will also be sparse. This sparsity can be exploited by using compressed data structures for the bitplanes, which would reduce memory consumption and could also accelerate the Boolean matrix multiplication. Furthermore, the inherent parallelism of the bit-sliced operations makes the method highly suitable for implementation on modern hardware with wide SIMD units and dedicated bit-manipulation instructions.

In conclusion, the multiplication-free approach presented in this paper provides a promising avenue for accelerating spectral graph algorithms. By shifting the computational burden from expensive multiplications to highly efficient bitwise operations, it opens the door to the analysis of larger graphs and the development of more efficient machine learning and data analysis pipelines.