Two $\operatorname{ReLU}$ Layers is All You Need

We prove that every $\operatorname{ReLU}$ network can be exactly simulated by a $\operatorname{ReLU}$ network with two hidden layers. We present two proofs: one that is based on fillings in homological algebra and a direct combinatorial proof. --- Source: arXiv:2610.28587v1 - http://arxiv.org/abs/2610.28587v1 PDF: https://arxiv.org/pdf/2610.28587v1 Original Link: http://arxiv.org/abs/2610.28587v1