Could hierarchical latent representations be a practical alternative to next-token generation?
Been looking into Next-Concept Modeling (NCM). The approach has two main stages: A multiscale VQ-VAE learns coarse-to-fine representations of text. A transformer predicts each level of discrete codes in parallel, conditioned on previous levels. Once the hierarchy is complete, a decoder reconstructs…
Read the full story at r/deeplearning ↗
Timeline · 1 report
- 2026-10-23 07:29 · r/deeplearning
Could hierarchical latent representations be a practical alternative to next-token generation?