Decomposing the Dark Matter of Sparse Autoencoders

Engels, Riggs, Tegmark (2024)

Read paper

Tags: representation-geometry, dark-matter

Abstract

We analyze the 'dark matter' of SAEs — the unexplained variance in reconstructions — identifying systematic patterns in what SAEs fail to capture.