Hallucinated outputs are model responses that sound confident and plausible but are incorrect, unsupported, or entirely fabricated. In security terms, they are dangerous because an AI system may present a false statement as fact, invent a policy, misread evidence, or cite a tool result that never existed. When users trust the output, a small error can become a bad access decision, a wrong incident response step, or an unsafe automated action.
These errors matter most when AI is connected to business systems, ticketing tools, or code and data repositories. An attacker can exploit that trust by prompting a model to produce misleading guidance, false summaries, or fabricated references that hide malicious activity. Defenders reduce risk by grounding models in verified sources, requiring citations, constraining tool use, validating outputs before action, and keeping humans in the loop for high-impact decisions. Hallucinations are not just quality defects; they are a security and governance problem.