AI red-teaming is structured testing of an AI system to uncover unsafe behavior, misuse paths, or guardrail failures. The goal is to behave like a determined adversary: probe prompts, inputs, tool use, and system rules to see where the model can be tricked into leaking data, generating harmful output, ignoring policy, or taking unintended actions.
It matters because AI systems are increasingly used in customer support, code generation, search, and agentic workflows. A weakness in the model or its surrounding controls can become a security, safety, or privacy problem. In practice, red-teamers may try prompt injection, jailbreaks, data exfiltration tricks, or adversarial inputs. Defenders use the findings to harden filters, tighten permissions, improve logging, and add safer fallback behavior.