Token Compression: Reducing Attention Waste?
An experiment on compressing multiple LLM tokens into one representation for faster decoding and longer effective context.
Transformers
2 essays on this theme.
A quest for very long context: Part 1
Experiments on extending transformer context length, including training observations, tradeoffs, and lessons from long-context tuning.