Intrinsic Information-Theoretic Analysis of ReLU Nets
arXiv:2610.36864v1 Announce Type: new Abstract: Mutual information (MI) is fundamental to representation-learning analysis in deep networks, but in high dimensions MI estimators depend on hyperparameters whose bias often dominates the signal. We show that continuous piecewise-linear (CPWL) networks…
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-10-28 04:00 · arXiv stat.ML
Intrinsic Information-Theoretic Analysis of ReLU Nets