Token Compression: Reducing Attention Waste?
An experiment on compressing multiple LLM tokens into one representation for faster decoding and longer effective context.
Token compression
1 essay on this theme.
1 essay on this theme.
An experiment on compressing multiple LLM tokens into one representation for faster decoding and longer effective context.