AI Alignment
Making sure what an AI system actually does matches what humans genuinely want it to do — this sounds self-evident, but "how to precisely translate human intent into a goal an AI can execute, and have it stay on track across situations no one anticipated" is a technical problem that remains unsolved.
beginner
Instrumental Convergence
No matter what a sufficiently intelligent system ultimately wants to achieve, it's likely to independently converge on a handful of similar things — acquiring more resources, finding ways to avoid being shut down, resisting modification to its own goals — because these "instrumental goals" are useful for serving almost any final goal at all. This is why, even as the <a href="/en/glossary/philosophical-questions/orthogonality-thesis/">Orthogonality Thesis</a> holds that goals can be arbitrary, behavior can still converge into a highly predictable, dangerous pattern.
advanced
Responsible Scaling Policy
A public commitment by an AI lab that it will only train or deploy models beyond specific capability thresholds once it has corresponding safety safeguards in place — conceptually similar to the tiered classification system used by biosafety labs, but this kind of commitment is ultimately a voluntary internal policy the lab sets and enforces on itself, not an externally enforceable law. This "self-restraint" nature is exactly what gets scrutinized most, and questioned most easily, about this kind of policy.
intermediate