A model extraction attack is an attempt to recreate a target AI model by repeatedly querying its interface and using the returned outputs as training data. Instead of breaking into the system, the attacker treats the model as a black box: they send many prompts, observe responses, and train a cheaper model to imitate its behavior, decisions, or style.
This matters because modern AI services often expose valuable behavior through public APIs. If enough queries are collected, an attacker may approximate a model closely enough to copy product features, bypass paid access, or build a competing system. In security operations, extraction traffic can look like normal benchmarking or heavy customer use, which makes it hard to detect. Defenses include rate limiting, account and identity controls, output monitoring, anomaly detection, and logging that can reveal suspicious query patterns. Legal and policy controls, such as terms of service, can also restrict reverse engineering and unauthorized imitation.