Token throughput is the rate at which an AI system generates or processes tokens under real workload conditions. A token is a unit of text the model reads or produces, so throughput measures practical output speed, not just benchmark performance. It depends on GPU capacity, model size, batching, memory bandwidth, network latency, storage speed, and orchestration overhead.
In cyber security, token throughput matters because slow or unstable AI services can create bottlenecks, raise costs, and expose weak points in shared infrastructure. Attackers may try to exhaust throughput with prompt flooding, inference denial-of-service, or resource contention that slows legitimate users. Defenders monitor throughput to detect abuse, size capacity, set rate limits, isolate workloads, and keep sensitive models responsive. In AI factories and other concentrated deployments, maintaining predictable token throughput is part of both resilience and operational security.