Tanishq Mathew Abraham, Ph.D. @iScienceLuvr 2026-04-18
What are some SOTA alternatives to SAEs for mechanistic interpretability? Is Anthropic still using SAEs or have they moved on to something better now?
Jatin Nainani @jatin_n0 2026-04-18
Cross layer transcoders aim at replacing the MLPs https://transformer-circuits.pub/2025/attribution-graphs/biology.html… but it’s not an exact alternative to sae which are more used for residual streams, there’s also sae bench if you were talking about the sae architectures https://arxiv.org/abs/2503.09532
Burny - Effective Curiosity @burny_tech 2026-04-19
Steering vectors, cross layer transcoders
Eric Ho @ericho_goodfire 2026-04-19
depends on what you’re trying to do! if you DM me, we’re happy to provide a few pointers
Matthew Shu @mattshu04 2026-04-19
In our mech interp for life sci journal club we covered crosscoders that learn cross-layer features (SAEs learn features at a single layer). Crosscoders are cool because you can use them to diff models too. The recording is here at the bottom:
https://mattshu.dev/journal-club/journal-club-schedule…
Thin Signal @thin_signal 2026-04-18
PTS is an alternative -
Skye @skye7821 2026-04-18
IMO circuit activations are a lot easier to work with and also a tad more interpretable, but it depends on how low level you want to go. For feature level I think SAE is still the best.
Electrik Dreams @techdreamzai 2026-04-19
Anthropic’s still pushing SAEs: scaled to 1M features in March. I track transcoders, causal scrubbing, and geometric interpretability as the real alternatives.
Allen Schmaltz @Allen_Schmaltz 2026-04-19
Interpretability-by-exemplar via SDM estimators:
James Golden @James_R_Golden 2026-04-19
I am working on a method for equivalent linear representations, not from a big lab but paper in tmlr and a variant of this is at the top of the circuits interpretability benchmark leaderboard
Barak @barakwidawsky 2026-04-18
I think the answer to this really depends on what you’re looking to answer… I wouldn’t frame mech interp around SAEs, they’re just one tool
Linear probes work a lot better than SAEs in some scenarios, and logit diff amplification answers totally different types of questions
Kyle @kyle_mccleary 2026-04-18
Nanda was talking about activation oracles, although that’s still a bit early.
Veeraraju Elluru @VeerarajuE 2026-04-19
If you wanna just interp, probably activation oracles is better; for steering, SAEs are still actively used. Goodfire’s latest works on interp-ing LMs for bio is also good (it has a bit of probing, SAEs, everything).
Goncalo Paulo @GoncaloSPaulo 2026-04-19
I think you have to narrow your question. SOTA for what purpose?
Gabe @Gabogonzalez515 2026-04-18
what is the downstream use case? do you NEED linearity?