Model abuse is the misuse of an AI system for harmful, prohibited, or policy-violating activity. Instead of helping with legitimate tasks, the model is used to generate dangerous instructions, automate fraud, assist weaponization, or support other abusive workflows. The abuse may happen through a single prompt, but more often it appears as a pattern of requests that gradually reveal intent.
In cyber security, model abuse matters because AI tools can accelerate reconnaissance, phishing, malware development, social engineering, and other attack steps. Defenders look for evasive wording, repeated retries, fragmented conversations, and dual-use language that hides the real goal. Security controls such as logging, rate limits, content filters, and human review help detect and interrupt these behaviors. The same techniques also apply in non-cyber misuse, including weapons-adjacent planning or operational testing, where the model becomes part of an unsafe engineering loop.