An inference model is the AI model that runs at deployment time to generate outputs from live prompts or inputs. In other words, it is the version doing the real work in production, as opposed to the model used only for training or fine-tuning. In enterprise systems, different inference models may be compared for speed, cost, accuracy, or safety.
In cyber security, the inference model matters because it sits on the attack surface. Prompt injection, prompt leakage, and data exfiltration attempts all target the runtime behavior of the deployed model, not just the code around it. Defenders evaluate inference models with controlled test sets, guardrails, and logging to spot unsafe responses, hidden instruction following, or unexpected data exposure. When organizations benchmark prompts across multiple inference models, they are testing how each runtime behaves under the same inputs before putting it into production.