May 20262026 年 5 月
Transformers Are (Naively) Looped Transformers, HorizontallyTransformer 本身就是横向展开的循环 Transformer
The standard Transformer applies the same weights at every sequence position — making it a horizontally looped, time-recurrent architecture. The natural generalization assigns per-position weight matrices. TTT realizes this via gradient descent, and turns out to be LocoProp applied along the time axis.标准 Transformer 在每个序列位置用同一组权重——这让它成为一个横向循环、时间维递归的架构。最自然的推广是给每个位置配一组权重矩阵。TTT 用梯度下降实现了这一点，而它恰好就是沿时间轴施加的 LocoProp。
February 20262026 年 2 月
The ABC Research MindsetABC 研究心态
A simple debiasing trick for judging new research ideas. By reordering the timeline of proposals, we can strip away the novelty premium and compare ideas on their actual merits. Applied to latent reasoning and transformer architecture debates.一个评判新研究想法时去偏的小技巧：把提案的时间顺序调换，剥掉新颖性溢价，按真实价值比较。文中用它讨论了隐式推理和 Transformer 架构之争。
February 20262026 年 2 月
Rethinking Key-Value Relationships in Linear Attention重新审视线性注意力中的 key-value 关系
We tested how DeltaNet handles diverse key-value relationships from identity to exponentials, scaling up to 32,000 tokens. Under standard normalized conditions, DeltaNet stays remarkably stable whether values come from k² or e^k. Information theory explains the bounds: DeltaNet retains 99.6% of information for identity but only 1.2% for exponentials, revealing fundamental compression limits.我们测试了 DeltaNet 在各种 key-value 关系下的表现，从恒等映射到指数函数，序列最长到 32,000 token。在标准归一化条件下，无论 value 来自 k² 还是 e^k，它都稳得出奇。信息论解释了上界：恒等关系下保留 99.6% 的信息，指数关系下只剩 1.2%，这揭示了压缩能力的根本限制。