AI papers of the week - June 5th, 2024 - MMLU-Pro, Zipper, FineWeb, etc.

List of papers covered:

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark (special appearance by authors)

Improving the Training of Rectified Flows

Are Long-LLMs A Necessity For Long-Context Tasks?

Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities

FineWeb: decanting the web for the finest text data at scale

Offline Regularised Reinforcement Learning for Large Language Models Alignment

Transformers Can Do Arithmetic with the Right Embeddings

Guiding a Diffusion Model with a Bad Version of Itself

Self-Improving Robust Preference Optimization

Situational Awareness