An inference framework is software that loads and runs a trained AI model. It processes inputs, manages outputs, and may optimize performance through techniques such as batching or hardware acceleration. Frameworks can run locally, in a data center, or as part of a hosted service.
In cybersecurity, the framework matters because it is a dependency in the AI supply chain and may affect how data is handled, where computation occurs, and which components need security updates. Attackers may target weaknesses in frameworks or their dependencies, while defenders should inventory them, verify their provenance, review permissions and data flows, and keep them patched. A branded AI application or API can conceal the framework underneath, so organizations may need suppliers to disclose which models and inference components their services use.