Tagged with

2 articles found

T5Gemma 2: Google’s Encoder-Decoder Revival Challenges the Decoder-Only Orthodoxy

Google’s T5Gemma 2 merges tied embeddings and unified attention to deliver efficient multimodal AI that outperforms decoder-only models at a fraction of the parameter cost.

#attention-mechanisms#encoder-decoder#gemma-3...

attention-mechanisms

Linear Attention’s Revenge: How Kimi Delta Attention Smashes the KV Cache Bottleneck

Moonshot AI’s hybrid architecture delivers 6x decoding speed with 75% less memory, making 1M-token contexts actually practical.

#attention-mechanisms#efficiency#LLM...