AI Red Team Academy🇰🇪 Kenya

01 Foundations — How AI Agents Work ·Lesson 2 of 6· 12 min

LLMs From the Outside: Tokens, Context Windows and Messages

Before we can secure AI agents, we need to understand the building blocks. This lesson covers how large language models work from an application developer's perspective — not the maths of transformers, but the practical model that matters for security.

Tokens

LLMs don't read text the way humans do. They break input into tokens — chunks that are roughly 3–4 characters of English text. The word "security" might be two tokens: "secur" + "ity". A long URL might be a dozen tokens.

This matters for security because:

  • Token limits are hard limits. If your system prompt, user input, tool outputs, and retrieved documents exceed the context window, something gets truncated — and truncation can cut off security instructions.
  • Cost scales with tokens. An attacker who can inflate the token count (by injecting verbose content into retrieved documents) can increase your costs.

The Context Window

Every LLM call receives a context window: a sequence of messages that the model reads in full before generating a response. A typical context looks like:

[system message] → [user message] → [assistant message] → [user message] → ...

The system message usually contains instructions for the model: its role, what tools it has, and any safety rules. The remaining messages form the conversation.

Key security insight: The model processes the entire context as one sequence of tokens. It has no built-in mechanism to enforce that "system instructions" take precedence over "user input." The distinction between system, user, and assistant messages is a convention — the model was trained to respect it, but that training is imperfect.

Messages and Roles

The Messages API uses roles to tag each message:

Role Purpose
system Instructions from the application developer. Sets behaviour, persona, rules.
user Input from the end user (or content presented as if from the user).
assistant The model's previous responses.
tool Results returned by tools the model called.

In an agent, the "user" message may not actually come from a human — it might be content the agent read from a webpage, an email, or a database. This blurring of who authored what message is the root cause of prompt injection.

Temperature and Non-Determinism

LLM outputs are non-deterministic by default. Even with temperature set to 0, minor infrastructure differences can produce different outputs for the same input. This has security implications:

  • A prompt injection that fails once might succeed on retry.
  • Security tests need to be run multiple times to be meaningful.
  • Defences that rely on "the model always says no" will eventually say yes.

Key Takeaways

  1. LLMs see tokens, not text — context limits and truncation have security consequences.
  2. The system/user/tool message roles are conventions, not enforcement boundaries.
  3. Non-determinism means that probabilistic defences (like "the model usually refuses") are not reliable security controls.

In the next lesson, we'll see how these building blocks are assembled into agents.