An LLM proxy is a middleware service that brokers requests between applications and one or more large language model providers. Instead of every app calling each model directly, the proxy can centralize authentication, request routing, quotas, policy checks, and logging. This makes it easier to manage many teams and vendors through one control point.
In cyber security, that control point is highly sensitive. If an attacker compromises the proxy, they may steal API keys, redirect traffic to an unauthorized model, modify prompts or responses, or tamper with audit logs. A proxy can also become a supply-chain target if its code, build process, or container image is altered. Defenders protect it with RBAC, SSO, secrets management, signed releases, and tight monitoring of administrative changes and unusual routing behavior.