Breaking the Temporal Bottleneck in Linear Transformers To enable the development of scalable linear transformers capable of supporting infinite-context modeling, it is crucial to address two key challenges: achieving efficient parallelism across the time dimension and managing extremely large recurrent states. Here we examine an existing solution for accelerating the DeltaNet algorithm, which employs the […]