Black-box extraction is a technique for inferring a model’s behavior by sending many queries and observing its outputs, without access to the model weights, training data, or internal code. Attackers may vary prompts, inputs, and sampling conditions to map patterns in responses, approximate decision boundaries, or reconstruct useful capabilities in a cheaper copy.
This matters in cyber security because a model can leak valuable behavior even when its internals are protected. In AI systems, extraction can support model stealing, prompt abuse, and the recovery of sensitive reasoning patterns or guardrail behavior. Defenders respond with rate limiting, anomaly detection, abuse correlation across accounts, output filtering, and restrictions on high-value responses. Strong logging and telemetry help identify automated probing, replay-style requests, and distributed querying that are designed to stay below detection thresholds.