> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Context engineering

> How to structure context for chat, RAG, and coding agents.

export const VagueVsSpecificDiagram = () => {
  const CSS = `
  .learn-diagram .compare { display: grid; grid-template-columns: 1fr 1fr; gap: var(--space-md); }
  @media (max-width: 760px) { .learn-diagram .compare { grid-template-columns: 1fr; } }
  .learn-diagram .compare__col {
    background: var(--color-paper-2);
    border: 1px solid var(--color-rule-soft);
    border-top: 3px solid var(--color-rule);
    border-radius: var(--radius-md);
    padding: var(--space-sm);
    display: grid;
    gap: var(--space-xs);
    align-content: start;
  }
  .learn-diagram .compare__col--good { border-top-color: var(--tg-orange); }
  .learn-diagram .compare__head {
    font-family: var(--font-mono);
    font-size: 11px;
    font-weight: 500;
    letter-spacing: 0.1em;
    text-transform: uppercase;
    color: var(--color-ink-mute);
    display: flex;
    justify-content: space-between;
    gap: var(--space-2xs);
  }
  .learn-diagram .compare__col--good .compare__head { color: var(--tg-orange); }
  .learn-diagram .compare__prompt {
    font-family: var(--font-mono);
    font-size: 13.5px;
    line-height: 1.6;
    color: var(--color-ink);
    background: var(--color-paper);
    border: 1px solid var(--color-rule-soft);
    border-radius: var(--radius-sm);
    padding: var(--space-xs);
    min-height: 106px;
  }
  .learn-diagram .compare__specs { display: grid; gap: 6px; }
  .learn-diagram .compare__spec {
    display: flex;
    align-items: baseline;
    gap: var(--space-2xs);
    font-family: var(--font-mono);
    font-size: 11.5px;
  }
  .learn-diagram .compare__spec-key {
    letter-spacing: 0.08em;
    text-transform: uppercase;
    color: var(--color-ink-faint);
    width: 74px;
    flex: none;
  }
  .learn-diagram .compare__spec-val { color: var(--tg-orange-dark); }
  .learn-diagram .compare__spec-val--none { color: var(--color-ink-faint); font-style: italic; }
  .learn-diagram .compare__note {
    padding-top: var(--space-2xs);
    border-top: 1px solid var(--color-rule-soft);
    font-size: 13px;
    line-height: 1.55;
    color: var(--color-ink-mute);
  }
  `;
  const SPECS = [{
    key: "format",
    vague: "unspecified",
    specific: "3 bullet points"
  }, {
    key: "length",
    vague: "unspecified",
    specific: "one sentence each"
  }, {
    key: "audience",
    vague: "unspecified",
    specific: "non-technical reader"
  }, {
    key: "style",
    vague: "unspecified",
    specific: "no setup or transitions"
  }];
  return <DiagramFrame eyebrow="Prompting" title="Anything you leave unsaid, the model decides for you" caption="Both prompts get an answer. The difference is how many decisions the model has to make on your behalf. Every unspecified dimension is a place the output can drift between runs." css={CSS}>
    <div className="compare">
      <div className="compare__col">
        <div className="compare__head">
          <span>Vague</span>
          <span>4 open decisions</span>
        </div>
        <div className="compare__prompt">Summarize this article.</div>
        <div className="compare__specs">
          {SPECS.map(s => <div className="compare__spec" key={s.key}>
              <span className="compare__spec-key">{s.key}</span>
              <span className="compare__spec-val compare__spec-val--none">{s.vague}</span>
            </div>)}
        </div>
        <div className="compare__note">
          You will get a summary: of some length, in some tone, at some level of detail. Whatever
          the model happens to pick this time.
        </div>
      </div>

      <div className="compare__col compare__col--good">
        <div className="compare__head">
          <span>Specific</span>
          <span>0 open decisions</span>
        </div>
        <div className="compare__prompt">
          Summarize this article in 3 bullet points for a non-technical audience. Each bullet should
          be one sentence. Skip any setup or transition language.
        </div>
        <div className="compare__specs">
          {SPECS.map(s => <div className="compare__spec" key={s.key}>
              <span className="compare__spec-key">{s.key}</span>
              <span className="compare__spec-val">{s.specific}</span>
            </div>)}
        </div>
        <div className="compare__note">
          Format, audience, length and style are pinned down. The same prompt now returns the same
          shape of answer every time, a much smaller failure surface.
        </div>
      </div>
    </div>
  </DiagramFrame>;
};

export const Legend = ({items}) => <ul className="legend">
    {items.map((it, i) => <li className="legend__item" key={i}>
        <span className="legend__sw" style={it.style || ({
  background: it.color
})} />
        {it.label}
      </li>)}
  </ul>;

export const Slider = ({label, value, display, min, max, step = 1, onChange}) => <label className="slider-group">
    <span className="slider-group__label">
      {label}
      <span className="slider-group__value">{display === undefined ? value : display}</span>
    </span>
    <input type="range" min={min} max={max} step={step} value={value} onChange={e => onChange(parseFloat(e.target.value))} />
  </label>;

export const DiagramFrame = ({eyebrow, title, caption, css, controls, readout, children}) => {
  const BASE_CSS = `
.learn-diagram {
  /* ── Together AI brand palette ── */
  --tg-orange: #FC4C02;
  --tg-orange-dark: #972E02;
  --tg-orange-soft: #FEE8DF;
  --tg-orange-tint: #FFDCCD;
  --tg-pink: #EF2CC1;
  --tg-pink-soft: #FDE3F6;
  --tg-purple: #A373ED;
  --tg-purple-mid: #CAAEF5;
  --tg-purple-soft: #EDE4FC;
  --tg-blue: #3D99F5;
  --tg-blue-mid: #A3D1FF;
  --tg-blue-soft: #E5F3FF;
  --tg-slate: #5E86AE;
  --tg-slate-soft: #EEF3F6;
  --tg-navy: #010120;

  /* ── Surfaces ── */
  --color-paper:   #FFFFFF;
  --color-paper-2: #F7F8FA;
  --color-paper-3: #F0F1F4;
  --color-paper-4: #E2E4E9;
  /* ── Ink ── */
  --color-ink:       #090909;
  --color-ink-soft:  #414B58;
  --color-ink-mute:  #626B84;
  --color-ink-faint: #98A0B3;
  /* ── Lines ── */
  --color-rule:        #C4C9D4;
  --color-rule-soft:   #E2E4E9;
  --color-rule-strong: #98A0B3;
  /* ── Accent ── */
  --color-accent:      var(--tg-orange);
  --color-accent-soft: var(--tg-orange-soft);
  --color-empty:       #F0F1F4;

  /* ── Type ── */
  --font-display: 'Jost', 'Helvetica Neue', Arial, sans-serif;
  --font-body:    'The Future', 'Jost', 'Helvetica Neue', Arial, sans-serif;
  --font-mono:    'The Future Mono', ui-monospace, 'SFMono-Regular', 'Courier New', monospace;

  /* ── Spacing (4px base) ── */
  --space-3xs: 4px;
  --space-2xs: 8px;
  --space-xs:  12px;
  --space-sm:  16px;
  --space-md:  24px;
  --space-lg:  32px;

  /* ── Radius ── */
  --radius-sm: 4px;
  --radius-md: 8px;
  --radius-lg: 12px;

  /* ── Motion ── */
  --ease-out: cubic-bezier(0.22, 1, 0.36, 1);
  --dur-fast: 120ms;
  --dur-base: 200ms;
  --dur-slow: 360ms;
  --focus-ring: 0 0 0 3px rgba(252, 76, 2, 0.35);

  font-family: var(--font-body);
  font-size: 15px;
  line-height: 1.65;
  font-weight: 400;
  color: var(--color-ink);
  -webkit-font-smoothing: antialiased;
  text-wrap: pretty;
  margin: 24px 0;
}
.learn-diagram, .learn-diagram * { box-sizing: border-box; }

/* ============ SHELL ============ */
.learn-diagram .diagram {
  margin: 0;
  background: var(--color-paper);
  border: 1px solid var(--color-rule);
  border-radius: var(--radius-lg);
  padding: var(--space-md);
  box-shadow: 0 1px 2px rgba(9, 9, 9, 0.04), 0 2px 10px rgba(9, 9, 9, 0.03);
}
.learn-diagram .diagram__head { margin-bottom: var(--space-md); }
.learn-diagram .diagram__eyebrow {
  font-family: var(--font-mono);
  font-size: 11px;
  font-weight: 500;
  letter-spacing: 0.1em;
  text-transform: uppercase;
  color: var(--tg-orange);
  margin-bottom: 6px;
}
.learn-diagram .diagram__title {
  font-family: var(--font-display);
  font-size: 20px;
  font-weight: 600;
  line-height: 1.3;
  letter-spacing: -0.02em;
  color: var(--color-ink);
  margin: 0;
}
.learn-diagram .diagram__caption {
  font-size: 13px;
  line-height: 1.55;
  color: var(--color-ink-mute);
  margin: 6px 0 0;
  max-width: 68ch;
}
.learn-diagram .diagram__controls {
  display: flex;
  flex-wrap: wrap;
  gap: var(--space-2xs) var(--space-md);
  align-items: center;
  margin-top: var(--space-md);
  padding-top: var(--space-sm);
  border-top: 1px solid var(--color-rule-soft);
}
.learn-diagram .diagram__readout {
  font-family: var(--font-mono);
  font-size: 11px;
  letter-spacing: 0.08em;
  text-transform: uppercase;
  color: var(--color-ink-mute);
  margin-left: auto;
}
.learn-diagram .diagram__readout strong {
  color: var(--tg-orange);
  font-weight: 500;
  font-variant-numeric: tabular-nums;
}

/* ============ BUTTONS ============ */
.learn-diagram .btn {
  font-family: var(--font-body);
  font-size: 13px;
  font-weight: 500;
  padding: 9px 14px;
  border: 1px solid var(--color-rule);
  border-radius: var(--radius-md);
  background: var(--color-paper);
  color: var(--color-ink);
  cursor: pointer;
  white-space: nowrap;
  transition: background var(--dur-fast) ease, border-color var(--dur-fast) ease,
              color var(--dur-fast) ease;
}
.learn-diagram .btn:hover:not([disabled]) { background: var(--color-paper-3); }
.learn-diagram .btn:focus-visible { outline: none; box-shadow: var(--focus-ring); }
.learn-diagram .btn[disabled] { opacity: 0.4; cursor: not-allowed; }
.learn-diagram .btn--accent {
  background: var(--tg-orange);
  border-color: var(--tg-orange);
  color: #fff;
}
.learn-diagram .btn--accent:hover:not([disabled]) {
  background: var(--tg-orange-dark);
  border-color: var(--tg-orange-dark);
}
.learn-diagram .btn--on {
  background: var(--tg-orange-soft);
  border-color: var(--tg-orange);
  color: var(--tg-orange-dark);
}
.learn-diagram .btn-row { display: flex; flex-wrap: wrap; gap: var(--space-2xs); flex: 0 0 auto; }

/* ============ SLIDERS ============ */
.learn-diagram .slider-group {
  display: flex;
  flex-direction: column;
  gap: 2px;
  min-width: 190px;
  flex: 1 1 190px;
}
.learn-diagram .slider-group__label {
  font-family: var(--font-mono);
  font-size: 11px;
  letter-spacing: 0.08em;
  text-transform: uppercase;
  color: var(--color-ink-mute);
  display: flex;
  justify-content: space-between;
  gap: var(--space-2xs);
}
.learn-diagram .slider-group__value {
  color: var(--tg-orange);
  font-weight: 500;
  font-variant-numeric: tabular-nums;
  text-transform: none;
}
.learn-diagram input[type="range"] {
  -webkit-appearance: none;
  appearance: none;
  width: 100%;
  height: 2px;
  background: var(--color-rule);
  border-radius: 2px;
  outline: none;
  cursor: pointer;
  margin: 8px 0 2px;
}
.learn-diagram input[type="range"]::-webkit-slider-thumb {
  -webkit-appearance: none;
  appearance: none;
  width: 14px;
  height: 14px;
  background: var(--tg-orange);
  border-radius: 50%;
  border: 2px solid var(--color-paper);
  box-shadow: 0 1px 3px rgba(9, 9, 9, 0.18);
}
.learn-diagram input[type="range"]::-moz-range-thumb {
  width: 14px;
  height: 14px;
  background: var(--tg-orange);
  border-radius: 50%;
  border: 2px solid var(--color-paper);
}
.learn-diagram input[type="range"]:focus-visible { box-shadow: var(--focus-ring); }

/* ============ PANELS · STATS · LEGEND ============ */
.learn-diagram .panel {
  background: var(--color-paper-2);
  border: 1px solid var(--color-rule-soft);
  border-radius: var(--radius-md);
  padding: var(--space-sm);
}
.learn-diagram .panel__head {
  font-family: var(--font-mono);
  font-size: 11px;
  font-weight: 500;
  letter-spacing: 0.1em;
  text-transform: uppercase;
  color: var(--color-ink-mute);
  margin-bottom: var(--space-2xs);
}
.learn-diagram .stats {
  display: flex;
  flex-wrap: wrap;
  gap: var(--space-3xs) var(--space-md);
  font-family: var(--font-mono);
  font-size: 12px;
  color: var(--color-ink-mute);
}
.learn-diagram .stats strong {
  color: var(--tg-orange);
  font-weight: 500;
  font-variant-numeric: tabular-nums;
}
.learn-diagram .legend {
  display: flex;
  flex-wrap: wrap;
  gap: var(--space-3xs) var(--space-md);
  list-style: none;
  margin: var(--space-sm) 0 0;
  padding: 0;
  font-family: var(--font-mono);
  font-size: 11px;
  letter-spacing: 0.04em;
  color: var(--color-ink-soft);
}
.learn-diagram .legend__item { display: flex; align-items: center; gap: 6px; }
.learn-diagram .legend__sw {
  width: 12px;
  height: 12px;
  border-radius: 2px;
  flex: none;
  display: inline-block;
}
.learn-diagram .tag {
  font-family: var(--font-mono);
  font-size: 11px;
  letter-spacing: 0.06em;
  text-transform: uppercase;
  padding: 3px 8px;
  border-radius: var(--radius-sm);
  background: var(--color-paper-3);
  color: var(--color-ink-soft);
  white-space: nowrap;
}
.learn-diagram .tag--accent { background: var(--tg-orange-soft); color: var(--tg-orange-dark); }
.learn-diagram .tag--muted { background: transparent; border: 1px dashed var(--color-rule); color: var(--color-ink-faint); }
.learn-diagram .note {
  font-size: 13px;
  line-height: 1.55;
  color: var(--color-ink-mute);
}
.learn-diagram code, .learn-diagram .mono { font-family: var(--font-mono); font-size: 13px; }

@media (prefers-reduced-motion: reduce) {
  .learn-diagram *, .learn-diagram *::before, .learn-diagram *::after {
    transition-duration: 1ms !important;
    animation-duration: 1ms !important;
    animation-iteration-count: 1 !important;
  }
}
`;
  return <div className="learn-diagram">
    <style>{BASE_CSS}</style>
    {css ? <style>{css}</style> : null}
    <figure className="diagram">
      {(eyebrow || title || caption) && <div className="diagram__head">
          {eyebrow && <div className="diagram__eyebrow">{eyebrow}</div>}
          {title && <h3 className="diagram__title">{title}</h3>}
          {caption && <p className="diagram__caption">{caption}</p>}
        </div>}
      <div className="diagram__inner">{children}</div>
      {(controls || readout) && <div className="diagram__controls">
          {controls}
          {readout && <span className="diagram__readout">{readout}</span>}
        </div>}
    </figure>
  </div>;
};

**TL;DR:** The prompt you submit to a model is all that can see for a given call. Besides its weights, the model does not have any outside memory, or any hidden context. What you put in the prompt is the entire context for that call. Structuring that context so it carries every relevant detail and instruction is the main lever you have to impact output quality. For chat applications this includes the system prompt, previous `user` and `assistant` turns, as well as retrieved chunks in the case of retrieval-augmented generation (RAG). For coding agents, this includes the system prompt, your `AGENTS.md`, tools, memory, skills, previous turns of reasoning, tool calls, and user and assistant responses.

## What a prompt actually is

For a chat model, a "prompt" is the full list of messages you send to the API:

```text theme={null}
messages = [
  { role: "system",    content: "You are a helpful assistant..." },
  { role: "user",      content: "How do I revert a Git commit?" },
  { role: "assistant", content: "..." },
  { role: "user",      content: "And what if I already pushed?" }
]
```

Internally, all of those messages get concatenated into one long stream of tokens using the chat template, with role boundaries marked by special tokens (see [Tokens & tokenization](/learn/tokens-and-tokenization) for what those special tokens are). The model sees a single stream, not three separate messages. There is no memory or state between API calls; if you want the model to remember something from an earlier conversation, you have to include that information in the messages list.

By default, the model also has no live access to anything outside the prompt. It can't browse the web, read your files, see your screen, or check the current date. Everything the model knows comes either from its training data or from the current prompt. If you need the model to access context outside the immediate prompt, you give it tools (see [Function calling & tool use](/learn/function-calling-and-tool-use)).

## Specificity

The model cannot read your mind. If your prompt is vague, the model fills the gaps with plausible defaults, and those defaults will probably not be the ones you wanted. This is a common failure people experience when they give coding agents unspecific instructions.

Consider the difference between these two prompts:

<VagueVsSpecificDiagram />

The vague version will give you a summary of some length, in some tone, with some level of detail. The model chooses all of those for itself, and you have no way of predicting which choice it will make. The specific version, on the other hand, has a much smaller failure surface. The format is specified, the audience, length, and style are all specified, and the model should stay within those guardrails.

## Showing examples

Telling the model exactly what you want is good, but *showing* the model what you want, by including example inputs and outputs in the prompt, is even better. This technique is called **few-shot prompting**.

```text theme={null}
Classify the sentiment of each review as POSITIVE, NEGATIVE, or NEUTRAL.

Review: "Took forever to ship and the box was crushed."
Sentiment: NEGATIVE

Review: "Does exactly what it says on the tin."
Sentiment: POSITIVE

Review: "I bought it last Tuesday."
Sentiment: NEUTRAL

Review: "Was hoping for more but it works fine."
Sentiment:
```

A few examples can be more effective than a lot of instructions. The model picks up on patterns in the examples (format, tone, edge cases) and replicates those patterns in its own response. Two to five examples is enough for most tasks; you rarely need more than ten unless you are dealing with a complex extraction task with many possible structures.

This type of prompting will often work better for instruct/non-reasoning models, while specificity works better for reasoning models.

## Adding structure

For long prompts, adding some structure to the prompt helps the model find the right piece of context at the right time. There are two patterns that tend to pay off in practice.

### Use headings and delimiters

```text theme={null}
### CONTEXT
The user has uploaded a CSV with 3 columns: name, age, city.

### TASK
Write a Python function that returns a dict mapping city to the
average age in that city.

### CONSTRAINTS
- Use only the standard library.
- Handle empty input by returning {}.
- No type hints.
```

Heading-style sections, or XML tags like `<context>...</context>`, help the model treat instructions, examples, and data as distinct sections rather than as one continuous block of text. Structured prompts are also easier for *you* to maintain over time, because you can find and update the right section without rereading the whole thing.

### Put critical context near the top and the bottom

Models tend to pay more attention to the beginning and end of a long input than to the middle (see [Context windows](/learn/context-windows) for more on this). If there's something that has to be true about every output, it's worth mentioning that constraint in the system prompt, and also restating it right before the user's question. This redundancy is annoying when you write the prompt, but it tends to produce more reliable outputs.

## Ordering content for cache hits

Coding agents like Claude Code live inside long-running sessions and rack up many prompts per task. Each of those prompts is mostly the same: the same system prompt, the same tools, the same `AGENTS.md`, the same project files. The only thing that really grows is the conversation. This is exactly the thing that **prompt caching** is built for. Providers cache the prefix of the prompt, and as long as the prefix is identical to a previous call, you skip recomputing it and pay a fraction of the cost. When done well, 90–95% of the context can be a cache hit and thus cost less time and money. On Together, prompt caching is enabled by default for [dedicated endpoints](/docs/dedicated-endpoints/settings).

The trick is putting everything in the right order: **static content first, dynamic content last**. For a coding agent this usually looks like:

1. **Static system prompt** and tool definitions
2. **`AGENTS.md` / `CLAUDE.md`**, project-level instructions
3. **Session context**, skills, open files, recent edits, current working directory
4. **Conversation**, messages, reasoning traces, and tool calls

Anything that almost never changes goes at the top, so it can be cached across every session in the workspace. Anything that changes every turn goes at the bottom. The further down the prompt a change happens, the more of the prefix survives in the cache, and the more sessions end up sharing a cache hit.

This ordering is easy to break. Common pitfalls include:

* **Timestamps in the static prompt:** Including the current date/time at the top makes every call unique, killing cache hits. Put timestamps near the bottom if needed.
* **Unordered tools:** If tools are not sorted, their order can vary and break the prefix. Always sort tools by name before serializing.
* **Changing tool definitions:** Modifying or reordering tools changes the prefix for everyone. Only append new tools; don't alter or insert into the static list.

<Info>
  The most important thing to keep in mind is that the *earliest change* to your prompt invalidates all cached tokens afterwards. You want to keep the prefix as stable as possible to maximize the cache hit ratio. Only put volatile info (time, user message, current file) in the dynamic tail, not the static, cached head/prefix.
</Info>

## Next steps

<CardGroup cols={3}>
  <Card title="Structured outputs & JSON mode" icon="braces" href="/learn/structured-outputs">
    For when "output JSON" in the prompt is not enough.
  </Card>

  <Card title="Function calling & tool use" icon="tools" href="/learn/function-calling-and-tool-use">
    For letting the model access data beyond the prompt.
  </Card>

  <Card title="Fine-tune vs. prompt" icon="git-branch" href="/learn/finetune-vs-prompt">
    The next step when prompt engineering hits a ceiling in performance.
  </Card>
</CardGroup>