Learning Multi-Level Features with Matryoshka Sparse Autoencoders

Bussmann, Nabeshima, Karvonen, Nanda (2025)

Read paper

Tags: architecture, matryoshka

Abstract

We propose Matryoshka SAEs that learn features at multiple levels of granularity simultaneously, inspired by Matryoshka representation learning.