Model configuration reference

Understand model identity, provider settings, generation, context accounting, execution routes, and the exact scope of per-step calibration.

On this page

A model profile describes one configured model: how to reach it, its physical context capacity, its execution route, and adjustments for particular calls. It does not contain the whole agent workflow or the credentials themselves.

Use this page when you need to know exactly what a field changes. For a guided experiment, start with Model calibration. For step mappings, reusable instruction files, timeouts, and context reserves, use the separate Request-runner configuration.

Examples below describe the published 1.8.1 contract. Placeholder model IDs must be replaced with a model actually available from your provider. Values are examples, not tested presets or recommended settings for every model.

Identity and the provider connection

The main runtime file registers provider connections and model profiles. This is a fragment, not a complete runtime.config.json:

{
  "models": {
    "providers": {
      "ollama-local": {
        "type": "ollama",
        "baseUrl": "http://127.0.0.1:11434"
      }
    },
    "profiles": {
      "local-assistant": {
        "configRef": "./models/local-assistant.config.json"
      }
    }
  }
}

The corresponding model file can contain:

{
  "label": "Local assistant",
  "provider": "ollama-local",
  "model": "replace-with-installed-model-id",
  "contextWindowTokens": 32768,
  "supportsThinking": false,
  "capabilities": {
    "inputModalities": ["text"],
    "outputModalities": ["text"]
  },
  "execution": {
    "policy": "supervisor-worker-v1"
  },
  "generation": {
    "temperature": 0.7,
    "topP": 0.95
  },
  "context": {
    "formatTokenAccounting": {
      "mode": "estimate"
    }
  }
}

The names have different jobs:

Field or key	Meaning
`local-assistant`	ABot’s profile ID. Runner settings and model selection refer to this ID.
`label`	Display name; defaults to the profile ID when omitted. Renaming it does not change the provider model.
`provider`	An existing key in `models.providers`, here `ollama-local`. It is not necessarily the adapter name.
`model`	Exact provider model identifier, including the local tag when applicable. It does not download a model.
`contextWindowTokens`	Positive safe integer describing physical capacity, not a desired answer length.

32768 is appropriate only if it matches the model and serving configuration. Raising it does not make a smaller model support a larger context. Ollama’s effective capacity is the smaller of this declaration and a valid positive integer options.num_ctx, when supplied.

Provider credentials and endpoints belong to the provider connection. For OpenAI, use apiKeyEnv to name the environment variable containing the key; do not paste the secret into a model profile. Adapter-owned connection settings belong under the provider’s settings, not under a calibration slot.

Referenced files versus inline profiles

configRef resolves relative to the main runtime configuration file, not your shell’s current directory. Its target must be a JSON object and a valid model definition in its own right, including its model and context capacity. It is not a recursively loaded chain of profile files.

Inline definitions under models.profiles.<id> are supported. They still need the required identity and capacity fields; they are not a different configuration language. Referenced files are easier to maintain independently, and are what the generated setup uses.

If a registration contains both configRef and inline settings, inline values override the referenced values. generation, options, capabilities, and context are merged one property deep; calibration is merged by slot. This is not a general deep merge. A supplied nested context.tokenEstimation object replaces that nested object, and arrays such as instructions replace rather than append. Prefer one clear owner for each setting so the UI file and effective configuration do not surprise you.

Capabilities declare support; they do not add it

capabilities.inputModalities and outputModalities accept arrays containing text, image, and audio. Omitted or empty modality lists resolve to text support.

For a genuinely image-capable model, use "inputModalities": ["text", "image"]. The model must support images and the provider adapter must implement image input. The gateway checks both for image attachments; its raw endpoint rejects image attachments. Merely adding image cannot turn a text-only model into a vision model.

Likewise, the schema accepting audio or an image output declaration is not a promise that the current UI and provider path provide that feature. Declare actual capabilities, then verify the intended end-to-end operation. Keep output as ["text"] for ordinary text generation; do not use modality flags as feature switches.

Generation and reasoning

The portable generation object supports three fields:

Field	Accepted configuration	Effect
`temperature`	Finite number	Sampling temperature when forwarded by the provider adapter.
`topP`	Finite number	Nucleus sampling; projected to the provider’s corresponding field.
`reasoningEffort`	`none`, `minimal`, `low`, `medium`, `high`, `xhigh`	Requested reasoning effort for a profile marked as supporting thinking.

The runtime does not impose a universal numerical range for temperature or top-p. The provider/model can reject an otherwise valid finite value. Omitted sampling values are not silently filled with a universal ABot preset; the provider’s behavior applies unless another configured overlay supplies them.

supportsThinking defaults to false. When false, the gateway does not send its resolved reasoning control. This does not prove the model performs no internal reasoning; it means ABot is not requesting a provider-specific thinking setting for that profile.

When true, reasoning resolves in this order:

A valid explicit reasoningOverride on the gateway request.
The effective profile’s generation.reasoningEffort, after applicable overlays.
The agent mode: fast → none, deep → high, otherwise medium.

Do not mark a profile as supporting thinking simply because it is a large model. Its provider and model must accept the requested controls. Accepted enum values in ABot do not imply every provider model supports every effort level.

Provider projection matters:

OpenAI: active reasoning on a thinking profile suppresses temperature and topP in the outgoing payload. When effort is none, those sampling settings can be sent again. A value visible in a resolved profile can therefore still be omitted from the actual request.
Ollama chat: none becomes think: false; other resolved efforts are passed as the thinking value. The raw payload path explicitly sends think: false, even if the profile otherwise supports thinking.

generation.maxOutputTokens is not a supported escape hatch: validation explicitly rejects it. Runner output reserves and provider output limits have different jobs; see Request-runner configuration.

Provider-native `options`

options is a JSON object on the model profile, but it is not a portable bag of arguments accepted by every provider.

The Ollama adapter begins with these native options, then applies effective context capacity as num_ctx; normalized generation.temperature and generation.topP override native temperature and top_p when present. For example, an Ollama-only profile fragment is:

{
  "options": {
    "num_ctx": 16384
  },
  "generation": {
    "temperature": 0.2
  }
}

For a profile declaring 32768 tokens, that num_ctx lowers the effective capacity to 16384. It is not merely a display preference. Ollama output-budget logic may derive or enforce num_predict for bounded steps; do not assume a native option always overrides runtime-owned limits.

The built-in OpenAI payload builder does not spread arbitrary profile options into the API request. Adding an unsupported OpenAI field there is not a supported way to enable it. Connection-level options and custom adapter settings also have their own ownership; check the adapter instead of copying an Ollama profile unchanged.

Context accounting

The model’s context object configures accounting, not physical capacity or conversation-history policy.

Structured-format overhead

formatTokenAccounting requires mode when the object is present:

Setting	Default and meaning
`mode: "estimate"`	Default when omitted. Account for the structured schema’s estimated token size.
`mode: "none"`	Do not charge structured-format overhead in this calculation. This does not disable JSON output.
`fixedOverheadTokens`	Non-negative integer; defaults to `0`. Extra format-related tokens in estimate mode when a structured format is present.

Use none only when the transport does not inject the structured schema into the model’s input context. Preserve a provider-specific onboarding value unless you have evidence to change it. A fixed overhead is not a reserve charged to every plain-text call.

Text estimation

tokenEstimation accepts positive finite numbers, including fractions. Zero and negative values are invalid; there is no documented upper bound. These are estimation parameters, not tokenizer settings:

Field	Default	Unit
`asciiCharactersPerToken`	`2`	ASCII characters per estimated token.
`nonAsciiBytesPerToken`	`2`	UTF-8 bytes of non-ASCII characters per estimated token.
`messageOverheadTokens`	`6`	Additional estimated tokens per message; tool-interaction envelopes have their own accounted fields.

Text estimates separately round up ASCII characters divided by their rate and non-ASCII UTF-8 bytes divided by their rate. At defaults, 100 ASCII characters estimate to 50 tokens before message overhead. Raising the divisor lowers the estimate, so an unjustified larger value makes admission less conservative—not the model more efficient.

An explicit default-valued accounting block is:

{
  "context": {
    "formatTokenAccounting": {
      "mode": "estimate",
      "fixedOverheadTokens": 0
    },
    "tokenEstimation": {
      "asciiCharactersPerToken": 2,
      "nonAsciiBytesPerToken": 2,
      "messageOverheadTokens": 6
    }
  }
}

This remains an estimate, not the provider’s billable tokenizer count. Where the runtime has a measured input count, admission uses that count instead of the estimated input count. Attachments and runner reserves are accounted for separately.

Do not add maxInputTokens, maxConversationMessages, maxToolObservationMessages, artifactContextMode, or compaction here: validation rejects those old or runtime-owned controls. Physical capacity stays on the profile; output/safety/attachment reserves belong to the runner. The published compaction trigger is runtime-owned at 70% of context capacity, not a model calibration parameter.

Execution route and plugin restrictions

execution.policy accepts supervisor-worker-v1 or execution-agent-v1. Omission selects Supervisor/Worker. This is explicit configuration, not automatic classification of model strength.

The canonical primary profile selects the route once per request. Changing a later calibration’s model does not switch the request to a different route halfway through. Route selection is runtime orchestration metadata and is not sent as a provider sampling parameter.

Model-level plugins.deny further restricts the globally available plugin/tool catalog. It accepts selectors such as a plugin ID, capability ID, qualified plugin/capability selector, or *. Empty or omitted deny adds no restriction. It cannot re-enable something globally disabled or denied.

The canonical primary model’s restriction applies to the request, including delegated calls and per-step model overrides. It is independent of execution route and does not replace user permissions. It also does not prevent plugin loading or preparation hooks; it is not a sandbox for untrusted installed code. See Plugins before installing community packages.

Calibration fields and their boundaries

A slot belongs to a specific model profile. It becomes relevant only when the runner maps an actual model step to that slot; adding an arbitrary key does not create a new step.

Slot field	Meaning
`name`, `description`	Optional non-empty display metadata. Not injected as instructions.
`profileId`	Optional existing model-profile ID to use for this call. Not a raw provider-model name.
`timeoutMs`	Positive integer milliseconds; overrides this step’s runner timeout.
`generation`	The same three normalized generation fields described above.
`context`	Format accounting and token estimation—not physical capacity.
`format`	`"json"` or a format object, used as a fallback to the caller’s explicit format.
`instructions`	Array of instruction strings; omitted means no extra slot instructions.

This fragment shows a focused adjustment, not a complete model file:

{
  "calibration": {
    "worker.decision": {
      "name": "Worker decision",
      "description": "Keep a structured decision focused on its evidence.",
      "timeoutMs": 90000,
      "generation": {
        "temperature": 0.2
      },
      "format": "json",
      "instructions": [
        "Do not report a tool action as completed unless its result confirms completion."
      ]
    }
  }
}

The instruction supplements the runtime contract; it does not grant tool permissions or replace required output schemas. Increasing the timeout changes latency tolerance, not context capacity or the number of allowed workflow steps. A runtime-supplied structured format takes priority over the slot’s fallback. Certain explicitly caller-owned text contracts can omit calibration instructions as well.

Keep structured decisions separate from natural-language output. supervisor.decision, worker.decision, planner.decision, planner.graph, reviewer.decision, execution.decision, and auditor.decision are not interchangeable with response steps. Do not force every response into JSON. The internal step tool_payload.raw maps to the slot toolPayload.raw; spelling them identically can leave your intended adjustment unused.

How inheritance actually works

The effective configuration is resolved in stages:

Load the registered profile and any referenced-file/inline overrides.
Select the applicable base profile. A valid client preference normally wins within its scope. overrideClientPreference: true reverses that preference. Ordinary Runtime requests use the runner’s step/default selection; catalog defaults.roles is not copied into their effective policy. The generic gateway can also use role fallback when its supplied policy actually contains roles. Runner schema v2 supplies default semantic-slot mappings; these slot names are not automatically different models.
Apply a selected named invocation profile’s generation/context overlay, if one was selected.
Resolve the mapped calibration slot on that selected model. If its mapping names an actual profile or named invocation profile, it is a direct selection instead, not a calibration-slot lookup.
If the slot has profileId, start from that target model’s base settings, then apply this slot’s generation/context/format. The target’s same-named calibration is not recursively applied. Without profileId, overlay the current profile.
Resolve request reasoning overrides and caller-owned format, then let the provider project supported fields.

Within a generation overlay, omitted fields inherit. Context objects merge only at the first level: setting tokenEstimation replaces that nested object, so omitted estimator members fall back to estimator defaults rather than the replaced object’s values. A missing slot leaves the selected base unchanged. A calibration timeout comes from the selected slot or falls back to the runner; it is not inherited by recursively looking through the target model’s slots.

A slot cannot directly redefine provider, model, options, supportsThinking, capabilities, contextWindowTokens, execution.policy, or plugin access. To use another model/capacity, register a real profile and reference its ID. That still does not change the request’s frozen route or its primary-profile plugin restrictions.

Verify a change before keeping it

Keep a backup, change one field, and start a fresh controlled request. Check these observations rather than asking the model which configuration it used:

Confirm the runtime loads the configuration without validation errors.
Inspect model.invocation.resolved in the runtime debug log: locate the relevant requestId and modelStep, then check resolved profile, provider, model, reasoning, generation, and context.
Where provider I/O tracing is enabled, inspect the outgoing payload to see what survived provider filtering. Pair its request/response records using their invocationId as well as request/step context. Logs can contain prompts, tool results, and personal data; keep them private and redact before sharing.
Inspect the actual outcome: valid structured decision, correct tool arguments, observed tool result, and usable final response. Compare the same task before and after.

If nothing changes, check the selected profile and slot mapping first. If a field disappears only in the provider request, check provider projection. If a request still runs out of room, inspect real capacity and runner reserves instead of making the estimator artificially optimistic. The calibration walkthrough turns this reference into a repeatable experiment.