Making sure what an AI system actually does matches what humans genuinely want it to do — this sounds self-evident, but "how to precisely translate human intent into a goal an AI can execute, and have it stay on track across situations no one anticipated" is a technical problem that remains unsolved.

A public commitment by an AI lab that it will only train or deploy models beyond specific capability thresholds once it has corresponding safety safeguards in place — conceptually similar to the tiered classification system used by biosafety labs, but this kind of commitment is ultimately a voluntary internal policy the lab sets and enforces on itself, not an externally enforceable law. This "self-restraint" nature is exactly what gets scrutinized most, and questioned most easily, about this kind of policy.