What Is p(hack)? Two AI Security Risks Behind the Phrase
Unlike p(doom), “p(hack)” is not an established AI-safety term with one accepted definition. In this hub, it is a useful label for two different probabilities: that AI enables a successful cyberattack, and that an AI system itself is compromised or manipulated.
Always name the direction of attack. AI-enabled hacking and hacking an AI system involve different attackers, defenses, evidence, and consequences.
Meaning one: AI enables hacking
Advanced models can lower the cost of parts of cyber operations: researching targets, explaining vulnerabilities, drafting convincing messages, or adapting code. The relevant risk is conditional—what system, what access, which attacker, which target, and over what time period?
Human operators, security controls, model safeguards, and the difficulty of moving from plausible advice to a working exploit all matter. A single universal probability would hide those differences.
Meaning two: someone hacks the AI
The direction can be reversed. Attackers may try to steal model weights, poison training data, manipulate tools, inject hostile instructions, or gain control of the infrastructure around an AI system. A compromised model or agent could then leak data, take unauthorized actions, or become a force multiplier for the attacker.
This category overlaps with ordinary cybersecurity but introduces unusual surfaces: natural-language instructions, autonomous tool use, large training pipelines, and systems whose behavior is difficult to audit completely.
Do not confuse it with p-hacking or reward hacking
P-hacking is a statistics term for exploiting analytical flexibility until a statistically significant result appears. Reward hacking, by contrast, describes an AI satisfying the measurable objective it was given while violating the designer's real intent. Neither term means the probability of a cyberattack.
Reward hacking is still relevant to AI takeover theories because it demonstrates a general problem: optimizing a proxy can produce behavior that looks successful by the metric while failing in the world.
How to reason about p(hack)
A useful estimate should define a threat actor, capability, target, time horizon, success condition, and consequence. It should also separate demonstrated capability from plausible future capability. Without those details, p(hack) is rhetoric rather than analysis.