AI inference is the stage where a trained machine-learning model is run on new input to produce a prediction, score, label, or other decision. Training builds the model; inference uses it. In security environments, inference powers tools such as malware classification, phishing detection, log triage, anomaly scoring, and assistant features that summarize alerts or answer analyst queries.
It matters because inference turns model knowledge into operational output, often at scale and under time pressure. Defenders may run inference locally on GPUs or dedicated accelerators to keep data on-premises and reduce latency. Attackers can also abuse inference services by probing them for sensitive behavior, forcing excessive requests to drive up cost, or using model outputs to refine evasion. For organizations, budgeting for compute capacity, hardware lifecycle, and throughput is part of security planning, since limited inference resources can slow detection and analysis.