An AI model can produce a correct answer, yet no one — not even the engineers who built it — can explain, in human-readable terms, why it arrived at that answer. This gap between "capability" and "explainability" is the <a href="/en/glossary/interpretability/black-box-problem/">Black Box Problem</a>, and it's currently the most fundamental obstacle to auditing, debugging, and holding AI systems accountable.

A small trained neural network that decomposes an AI model's tangled, hard-to-interpret internal neuron activations into a large set of "features," each corresponding to a single, identifiable concept — currently one of the primary tools in mechanistic interpretability research.