> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Prepare for FP4 inference

> Merge a trained LoRA adapter into its base model and quantize the result to FP4 for dedicated model inference.

Dedicated model inference serves merged models on GLM-5.3 and Kimi-K3 in FP4, a 4-bit weight format that fits these large models on fewer GPUs and serves them faster than full precision. Training doesn't run on those FP4 weights, so a low-rank adapter (LoRA) you train doesn't match the FP4 model you deploy. An FP4 preparation job gives you a trained model in the format production serving expects:

* **Serve at FP4 speed and cost:** An adapter served on top of its base model adds work to every request. A merged model has the adapter's changes built into its weights, so it serves like the FP4 base model, on the same hardware.
* **Keep the adapter's quality:** Merging an adapter straight into FP4 weights would apply its changes to weights that already carry quantization error. The job merges into the full-precision (BF16) weights first, then calibrates and quantizes the merged model to FP4, so the adapter's changes survive quantization.

Prepare adapters from [post-training](/docs/post-training/overview) sessions. A job takes 6 to 18 hours, depending on the base model, and produces one model, so start one after you've chosen a winning adapter. To compare several candidates first, [serve them on one deployment](/docs/post-training/multi-lora-deployment).

## Supported models

The adapter's base model must be one of the following:

| Base model | Adapter base ID | Typical running time |
| - | - | - |
| GLM-5.3 | `zai-org/GLM-5.3-FP8` or `zai-org/GLM-5.3-NVFP4` | About 6 hours |
| Kimi-K3 | `moonshotai/Kimi-K3` | About 18 hours |

The running time excludes any wait for GPU capacity before the job starts. Together sets the merge and quantization parameters for each model, and you can't change them. A GLM-5.3 adapter records `zai-org/GLM-5.3-FP8` as its base. Adapters saved before October 9, 2026 record `zai-org/GLM-5.3-NVFP4`. A Kimi-K3 job writes the model in the release's own MXFP4 format.

The adapter must be a LoRA adapter from a post-training session. Full-weight models aren't accepted.

## Find the adapter ID

A job takes the adapter's model registry ID (`ml_...`) and, optionally, a revision ID (`rv_...`). If you omit the revision, the job uses the adapter's latest revision. Read both from `registration.adapter` on the session's inference checkpoint, as [Find your model's ID](/docs/post-training/deployment) shows, or from `inference_registration.adapter` after the session ends, as [Find a checkpoint](/docs/post-training/checkpoints#find-a-checkpoint) shows.

Only a checkpoint that registers an adapter can be prepared. A checkpoint that registers a merged model [deploys directly](/docs/post-training/deployment#deploy-a-merged-model).

## Estimate the job

To check a job's price and running time without creating anything, estimate it with the adapter ID:

<CodeGroup>
  ```bash CLI theme={null}
  tg training fp4 estimate ml_abc123 --adapter-revision-id rv_abc123
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  estimate = client.post_training.prepare_for_fp4_inference.estimate_cost(
      inputs={
          "adapter_object_id": "ml_abc123",
          "adapter_revision_id": "rv_abc123",
      },
  )
  print(estimate.price_usd, estimate.time_hours, estimate.allowed_to_proceed)
  ```
</CodeGroup>

The CLI prints the price and the expected running time:

```text theme={null}
The estimated price of this job is $43.14 and it is expected to run for about 6.0h (excluding queue time).
```

The running time excludes the wait for capacity. It's an estimate, not a time limit, so a job that runs longer than estimated keeps running. If your credit can't cover the price, the CLI adds a warning, and in Python `allowed_to_proceed` is `False`.

A successful estimate also confirms that the adapter is eligible, because estimating and creating a job accept the same adapters.

## Calibrate on your own data

The job calibrates the quantized model on a general text corpus. To make calibration match your workload, add a conversation dataset with `--calibration-file` (`-c`). The job then uses your dataset for half of the calibration corpus and general text for the other half.

The dataset must be a JSONL file of conversations. Each row has `messages` and, optionally, `tools`, in the same chat format as [SFT data](/docs/post-training/sft#prepare-your-data). The file must be at most 64 MiB, and it must belong to the same project as the adapter.

<CodeGroup>
  ```bash CLI theme={null}
  # A local file: the CLI uploads it with purpose "calibration"
  tg training fp4 create ml_abc123 --calibration-file ./calibration.jsonl

  # A file you uploaded earlier
  tg training fp4 create ml_abc123 --calibration-file file-abc123
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  calibration = client.files.upload(
      file="calibration.jsonl", purpose="calibration"
  )

  job = client.post_training.prepare_for_fp4_inference.create(
      inputs={
          "adapter_object_id": "ml_abc123",
          "calibration_file_id": calibration.id,
      },
  )
  ```
</CodeGroup>

`estimate` accepts the same flag, and it checks that the file exists and passed validation before you start a job. It doesn't count tokens. Once the model's chat template is applied, your dataset must supply at least 598,016 tokens to fill its half of the calibration corpus, or the job fails with a `user_error`. Add more conversations and retry.

## Start the job

Create the job with the same adapter ID:

<CodeGroup>
  ```bash CLI theme={null}
  tg training fp4 create ml_abc123 --adapter-revision-id rv_abc123
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  job = client.post_training.prepare_for_fp4_inference.create(
      inputs={
          "adapter_object_id": "ml_abc123",
          "adapter_revision_id": "rv_abc123",
      },
  )
  print(job.id, job.status)
  ```
</CodeGroup>

Before it creates the job, the CLI shows the estimate and asks you to confirm. To skip the prompt in scripts, pass `--confirm` (`-y`). To wait for the job in the same command, add `--watch` (see [Monitor the job](#monitor-the-job)). For every flag, see the [`tg training` reference](/reference/cli/training).

<Warning>
  The CLI doesn't ask for confirmation when it doesn't run in an interactive terminal, for example in CI, with redirected input or output, or when it detects an AI agent. In these cases `create` starts the job at once. Run `estimate` first.
</Warning>

The new job has an ID in the form `shp-quant-...` and starts in `pending`. Save the ID to monitor or cancel the job.

Each request starts a new job, even for an adapter you've already prepared. A repeat job uploads its result to the same model as the earlier job, so reuse that model instead of starting another.

## Monitor the job

A job moves from `pending` to `queued` to `running` to `completed`. Watch it until it reaches a terminal status:

<CodeGroup>
  ```bash CLI theme={null}
  tg training fp4 get shp-quant-1a2b3c4d-5e6f --watch
  ```

  ```python Python theme={null}
  import time

  from together import Together

  client = Together()

  TERMINAL = {"completed", "cancelled", "error", "user_error"}

  job = client.post_training.prepare_for_fp4_inference.retrieve(
      "shp-quant-1a2b3c4d-5e6f"
  )
  while job.status not in TERMINAL:
      time.sleep(300)
      job = client.post_training.prepare_for_fp4_inference.retrieve(job.id)

  print(job.status)
  if job.status == "completed":
      print(job.results.api_model_object_id, job.results.api_model_revision_id)
  ```
</CodeGroup>

`--watch` prints each status change and new event until the job reaches a terminal status. The command exits with a non-zero code if the job failed. Stopping the command doesn't stop the job, so run it again to resume watching.

| Status | Meaning |
| - | - |
| `pending`, `queued` | Accepted and waiting for a GPU. |
| `running` | Merging, quantizing, or uploading the model. |
| `completed` | Done. `results` holds the new model's IDs. |
| `cancel_requested`, `cancelled` | You [cancelled the job](#cancel-a-job). |
| `user_error` | The job failed because of its input. The job's events say what to fix. |
| `error` | The job failed on Together's side. Retry, or contact support if it fails again. |

When the job reports `completed`, the prepared model is already validated for deployment. The CLI prints the model's ID (`ml_...`) and revision ID (`rv_...`). In Python, read them from `job.results.api_model_object_id` and `job.results.api_model_revision_id`.

`--watch` prints the job's events as they arrive. To read a job's events later, such as the reason it failed, list them:

<CodeGroup>
  ```bash CLI theme={null}
  tg training fp4 list-events shp-quant-1a2b3c4d-5e6f
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  events = client.post_training.prepare_for_fp4_inference.list_events(
      "shp-quant-1a2b3c4d-5e6f"
  )
  for event in events.data:
      print(event.created_at, event.type, event.message)
  ```
</CodeGroup>

A failed job ends with either a `SHAPING_USER_ERROR` event or a `SHAPING_ERROR` event. A `SHAPING_USER_ERROR` event's message says how to fix the input.

To list every FP4 preparation job in your project, run `tg training fp4 list` or call `client.post_training.prepare_for_fp4_inference.list()`.

## Deploy the prepared model

The prepared model is a private model in your project, registered against the base model that serves it in FP4. For GLM-5.3, that's `zai-org/GLM-5.3-NVFP4`, even when the adapter records `zai-org/GLM-5.3-FP8`. List its deployment profiles, then deploy it by its `model_object_id`:

```bash CLI theme={null}
tg beta models configs ml_def456

tg beta endpoints deploy ml_def456 \
  --endpoint my-prepared-model \
  --config cr_Cf3yApYdaWMrH6RsyVmh7 \
  --project your_project_id
```

Pass `--project` with your project ID from `tg whoami`, or set `TOGETHER_PROJECT_ID`, even for your default project. Without either, the CLI resolves a private model ID to its base model and deploys that instead. For `--config`, pass the config ID of an FP4 profile from the `configs` output, such as `cr_Cf3yApYdaWMrH6RsyVmh7` for GLM-5.3 or `cr_CfsRY8rFVfgH9cEWwywEa` for Kimi-K3. If the model has only one compatible profile, you can omit `--config`. For help picking a profile, see [Choose a deployment profile](/docs/dedicated-endpoints/configs).

The `deploy` command creates the endpoint, attaches a deployment, and routes all traffic to it. For the rest of the deployment lifecycle, see [Manage deployments](/docs/dedicated-endpoints/manage).

## Cancel a job

To stop a job before it completes, cancel it:

<CodeGroup>
  ```bash CLI theme={null}
  tg training fp4 cancel shp-quant-1a2b3c4d-5e6f
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  job = client.post_training.prepare_for_fp4_inference.cancel(
      "shp-quant-1a2b3c4d-5e6f"
  )
  print(job.status)
  ```
</CodeGroup>

The CLI asks you to confirm. Pass `--force` (`-y`) to skip the prompt. The job moves to `cancel_requested` right away and reaches `cancelled` once the run stops, so watch it to confirm. The CLI won't cancel a job that's already in `cancel_requested`. A job in a terminal status can't be cancelled, and the CLI prints that the job isn't cancellable.

## Troubleshooting

* **"Preparing an adapter on base model ... is not supported":** The adapter's base model isn't one of the [supported models](#supported-models).
* **"model ... is not an adapter" or "does not record a fine-tuning relation":** Pass the adapter's ID (`registration.adapter.id`, or `inference_registration.adapter.id` from the checkpoints API), not a merged model's ID. Full-weight models aren't accepted.
* **"model ... does not exist in this project":** The adapter belongs to a different project. Use an API key scoped to the project that trained the adapter.
* **"Calibration file ... not found":** The file ID is wrong, or the file belongs to another project. Upload the file in the adapter's project.
* **The estimate warns that the price exceeds your credit, or create fails with a `402` error:** Your credit balance can't cover the estimated price. Add credits and retry.
* **Create fails with a `403` error:** Your project is blocked from starting jobs. Contact support.
* **"Shaping job ... not found":** The job doesn't exist, or it belongs to another project.
* **Job ends in `user_error`:** List the job's events. The `SHAPING_USER_ERROR` event's message describes the problem with the input.

## Next steps

<CardGroup cols={2}>
  <Card title="Manage checkpoints" icon="device-floppy" href="/docs/post-training/checkpoints">
    Save inference checkpoints from a post-training session.
  </Card>

  <Card title="Multi-LoRA deployment" icon="stack-2" href="/docs/post-training/multi-lora-deployment">
    Compare candidate adapters before you prepare the winner.
  </Card>

  <Card title="Choose a deployment profile" icon="cpu" href="/docs/dedicated-endpoints/configs">
    Pick the hardware and quantization a deployment runs on.
  </Card>

  <Card title="Manage deployments" icon="settings" href="/docs/dedicated-endpoints/manage">
    Scale, stop, and delete endpoints and deployments.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.