I scaled a pure Spiking Neural Network (SNN) to 1.088B parameters from scratch. Ran out of budget, but here is what I found [R]

An 18-year-old indie developer scaled a pure Spiking Neural Network (SNN) to 1.088 billion parameters from scratch for language modeling, achieving loss convergence despite common beliefs about vanishing gradients. Key findings include maintaining 93% sparsity and the unexpected emergence of structurally correct Russian text, though the experiment was halted due to budget constraints.