A note on Hao et al.'s Coconut: feeding a language model's last hidden state straight back as its next input embedding, so reasoning happens in continuous vectors instead of tokens.
Three interactive figures take apart the token bottleneck, the feed-back loop, and the latent breadth-first search that lets it out-plan chain-of-thought while staying illegible.
The full note has the complete argument and its interactive figures.
Read the full note