Cognitio Infer

Overview #

Cognitio Infer answers the questions Cognitio would get wrong by answering too fast. You give it a conversation — and, if you like, a list of tools it may ask you to run — and it works the problem through before it replies: a short reasoning trace first, then the answer. Ask for more effort and it thinks longer. Use it when the reply has to be right rather than quick, or when the task needs several steps.

Cognitio Infer is the larger sibling of Cognitio in the Response family. Where Cognitio is a 1.5B-parameter base with a small persona adapter, Cognitio Infer is a 7.6B-parameter open-weight instruct model served as its vendor published it, on the same GPU, in the same process. Falcon adds no weights of its own: what makes it a Falcon model is the serving contract around it — a whole-conversation endpoint, prompted thinking at three efforts, tool calling parsed into a stable shape, and the two public routes that expose them.

Input is a conversation as a list of messages: system, user, assistant and tool turns, oldest first, plus optional tool specifications and a thinking effort. Output is one assistant message: the answer as content, the thinking as reasoning when it was asked for, and zero or more tool_calls for the host to run. The cognitio command line, released with this model, is the reference host: its code mode gives Cognitio Infer six tools over a repository and runs the loop.

Like Cognitio, it has no classification heads and does not decide whether a reply is safe to give; a host that needs that check runs an intention model such as ELIM before calling it.

Intended use #

• Questions with a right answer that takes a few steps — arithmetic, unit reasoning, a small plan — where the thinking trace is worth the seconds it costs.
• Tool-using tasks: the host offers functions, the model asks for them one or more at a time, the host runs them and sends the results back until the model finishes.
• Agentic coding over a repository through the cognitio code harness, which reads, searches, edits and runs with the user's confirmation.

Out of scope #

• Small talk and one-line replies; Cognitio does those in under a second and at a fraction of the cost.
• Anything that must be answered inside a second, or many requests at once: one generation runs at a time on the shared GPU.
• Current facts, retrieval or memory of earlier conversations; it knows only what the messages and the tool results tell it.

Choose Cognitio Infer when #

• The answer has to be worked out rather than recalled, or the task needs tools, and a wait of several seconds is acceptable.
• The host can hold the whole conversation and run the tool loop itself; the model only ever sees what it is sent.
• Otherwise pick Cognitio: the same route, an answer in under a second, and a friend style for casual chat.

Architecture #

Cognitio Infer is Qwen2.5-7B-Instruct: a decoder-only transformer with 28 layers, a hidden size of 3,584, an MLP intermediate width of 18,944 and grouped-query attention with 28 query heads over 4 key/value heads. Its token embedding and output head are separate matrices over a 152,064-entry vocabulary; the count of 7,615,616,512 parameters follows from the published configuration and matches the vendor's rounded 7.61B. The tokeniser is Qwen2 byte-pair encoding with the ChatML chat template, whose tool-call rendering the service relies on.

Nothing in the weights is Falcon's. The service loads the published safetensors in float16 beside Cognitio's 1.5B base and applies no adapter. Two behaviours are added by prompting and parsing rather than by training. Thinking is a fixed instruction, chosen by the requested effort, that asks the model to reason inside a marked block before answering, with a generation budget of 256, 640 or 1,280 tokens for the block; the service cuts the block out and returns it as reasoning, so content is only the answer. Tool calling uses the base's own format: the request's tool specifications are rendered into the system prompt by the chat template, and each call the model emits is parsed, its arguments decoded as JSON, and returned as an entry of tool_calls.

There are no heads, no label catalogue, no vision and no retrieval: the only fixed vocabulary is the ChatML role set (system, user, assistant, tool), and the model never sees anything but the rendered conversation.

Inputs & outputs #

Input #

The service accepts a JSON body with the whole conversation and renders it with the chat template: the tools, when given, go into the system turn; each message becomes a ChatML turn, assistant turns with their earlier tool calls and tool turns with their results; then the thinking instruction for the requested effort, and the assistant generation prompt. Only the user turns must be present. The Falcon API builds this body for you on /v1/chat, from text and history, and passes it through on /v1/agent.

{
  "model": "cognitio-infer",
  "think": "low",
  "tools": [
    { "type": "function", "function": { "name": "read_file", "description": "Read a file from the repository", "parameters": { "type": "object", "properties": { "path": { "type": "string" } }, "required": ["path"] } } }
  ],
  "messages": [
    { "role": "system", "content": "You are Cognitio Infer working as a coding assistant inside a software repository." },
    { "role": "user", "content": "What does bin/report.mjs print when it is given no arguments?" }
  ],
  "max_new_tokens": 640,
  "temperature": 0.3
}

Limits: at least one user turn; on the Falcon API at most 80 messages, 240,000 characters and 24 tools per request; at the service a rendered conversation of at most 14,000 tokens, refused with 413 above that. think is off, low, medium or high. max_new_tokens caps the answer on top of the thinking budget, and temperature has a floor of 0.05. There is no frame, no units and no structured input beyond the message list and the tool schemas.

Output #

A successful response is one assistant message plus usage. The generation is decoded without special tokens, the thinking block is split out, each tool call is parsed, and what remains is trimmed and returned as content.

{
  "message": {
    "role": "assistant",
    "content": "",
    "reasoning": "I have not seen the file yet, so I should read it before answering.",
    "tool_calls": [
      { "id": "call_1", "name": "read_file", "arguments": { "path": "bin/report.mjs" } }
    ]
  },
  "model": "cognitio-infer",
  "usage": { "prompt_tokens": 214, "completion_tokens": 58 },
  "latency_ms": 4100
}

The host runs read_file, appends this message and then a tool turn (tool_call_id: "call_1", the file text as content) to the conversation, and calls again; the next message answers in content with no tool calls. A turn with neither text nor tool calls is possible when the budget runs out inside the thinking block; the reference host asks the model to continue. There is no confidence, score or decision rule: Cognitio Infer returns a message or an error, and the reasoning trace is working, not a justification the host should rely on.

Training #

Cognitio Infer is not trained by Ducky Software. The model is Qwen2.5-7B-Instruct as its vendor published it: an open-weight instruct model whose pre-training and post-training data are the vendor's and are not disclosed in detail. No real user data of Falcon's was used, because no training took place; the served weights are byte-for-byte the published ones, loaded in float16.

What Falcon built is the serving contract around the weights. The thinking behaviour is a fixed instruction per effort with a token budget, chosen after trying the base with and without it on short reasoning questions; the tool-calling behaviour uses the format the base was already post-trained on, rendered by its own chat template; the splitting of reasoning, answer and tool calls is done by the service after generation. None of that changes the weights, and a new version of this model would come from a new published base or a new prompt, not from a training run.

Consequently there is no training job, no held-out set and no loss curve to report for this version, and no lineage: Cognitio Infer is not warm-started from Cognitio or from any other Falcon model, and Cognitio's persona adapter is never applied to it.

Evaluation #

The figures above are a small, reproducible check of the serving contract rather than a benchmark claim: a fixed random sample of 40 problems from the GSM8K test set, sent one at a time through the service's conversation endpoint with the default temperature, the last number in content compared with the reference answer. Cognitio (the 1.5B base, no thinking) is measured the same way as the baseline. Latency is the median wall-clock time of those calls on the shared L4, warm, with nothing else running. Two things stand out: the base already works step by step without being asked, so off and low land on the same score and low is the faster of the two because its answer is shorter once the working has moved into the thinking block; and medium buys two more correct answers for about four seconds more per call.

Known gaps: forty problems give a coarse figure, and a sampled generation can land differently on a second run. Tool calling and the code harness have no published accuracy measure; the harness's own tests exercise the loop against a scripted model, not the real one. Whole-conversation coherence over long tool loops and behaviour at the 14,000-token limit are untested beyond the reference host's own use. Estimates, never measurements, where the text says about.

API #

Cognitio Infer is exposed through two Falcon API routes, both metered in the response preview quota bucket. /v1/chat is the conversational route shared with Cognitio: send model: "cognitio-infer" and, optionally, think, and the reply carries text and reasoning. /v1/agent is the tool-calling route: a whole conversation and tool specifications in, one assistant message out. Envelope, authentication, rate limits and retry guidance are in API conventions. The service's own contract, which both routes wrap, follows.

POST /v1/chat HTTP/1.1
Authorization: Bearer $FALCON_API_KEY
Content-Type: application/json

{ "model": "cognitio-infer", "think": "medium", "text": "A shop sells pens at 3 for $2. How much do 21 pens cost, and what is the change from $20?" }

{ "ok": true, "engine": "cognitio-infer", "model": "cognitio-infer", "think": "medium", "text": "21 pens cost $14, so the change from $20 is $6.", "reasoning": "21 pens is 7 groups of 3 …", "usage": { "prompt_tokens": 96, "completion_tokens": 141 } }

POST /v1/agent HTTP/1.1
Authorization: Bearer $FALCON_API_KEY
Content-Type: application/json

{ "messages": [ { "role": "user", "content": "What does bin/report.mjs print with no arguments?" } ], "tools": [ { "type": "function", "function": { "name": "read_file", "parameters": { "type": "object", "properties": { "path": { "type": "string" } }, "required": ["path"] } } } ] }

{ "ok": true, "engine": "cognitio-infer", "model": "cognitio-infer", "think": "low", "message": { "role": "assistant", "content": "", "reasoning": "…", "tool_calls": [ { "id": "call_1", "name": "read_file", "arguments": { "path": "bin/report.mjs" } } ] }, "usage": { "prompt_tokens": 214, "completion_tokens": 58 }, "latency_ms": 4100 }

Route /v1/chat · quota bucket response · body limit 64,000 bytes on /v1/chat (text at most 2,000 characters, 8 history turns); 512,000 bytes on /v1/agent.

POST /v1/messages HTTP/1.1
Content-Type: application/json

{
  "model": "cognitio-infer",
  "think": "low",
  "messages": [ { "role": "user", "content": "What is 17 × 24?" } ],
  "max_new_tokens": 512,
  "temperature": 0.4
}

A 200 response carries message, model, usage and latency_ms as documented under Output. Any other status carries a single error string:

The health route reports both models and never blocks on generation:

GET /health HTTP/1.1

{
  "ok": true,
  "model": "cognitio",
  "models": ["cognitio", "cognitio-infer"],
  "ready": true,
  "infer": { "model": "cognitio-infer", "base_model": "Qwen/Qwen2.5-7B-Instruct", "ready": true, "loading": false, "error": null, "think": ["low", "medium", "high"], "max_prompt_tokens": 14000 }
}

models lists the names the service will accept, and infer.ready is true once this model's weights are in. The Falcon API polls this route and lists Cognitio Infer in its catalogue only while infer.ready is true.

Runtime & deployment #

Cognitio Infer runs inside the Cognitio service: one Python process, one threading HTTP server, two models. At start a background thread loads Cognitio's tokeniser, base and adapter, runs its warm-up, and then loads Cognitio Infer's tokeniser and weights from the image. Cognitio serves as soon as its own load is done; requests naming cognitio-infer receive 503 model still loading until the second load finishes, about two to three minutes later on the L4, and /health reports infer.loading: true meanwhile. If either load fails, /health reports the error for that model and its requests return 500 until the instance is replaced. Generation is serialised across both models with one lock, so a long thinking turn delays a Cognitio reply queued behind it.

The Falcon API behaves as follows:

• Probes /health at most once a minute and offers cognitio-infer on /v1/chat, /v1/agent and in the playground catalogue only while infer.ready is true; while the service is up but this model is not, those requests answer 503 chat_unavailable or agent_unavailable.
• POSTs to /v1/messages with a 150 s timeout, and on a 503 waits and retries once before answering 503 chat_warming or agent_warming with retry_after_s.
• Builds max_new_tokens for the service as the caller's answer cap plus the thinking budget of the requested effort.

The image is the Cognitio GPU build, running on a serverless container platform with one NVIDIA L4, 32 GiB of memory and 8 vCPUs, a 300 s request timeout, at most one instance and, since 2026-09-21, one instance kept warm at all times; the L4's 24 GB holds both models in float16 with room for a 14,000-token prompt. Because the instance stays up, a request after a quiet period answers at once; only a redeploy brings a start, during which the routes answer 503 with a retry hint until both models are in. Once warm, an answer takes a median of 8 s at low thinking and 12 s at medium on the evaluation sample, and a high-effort answer with a long trace runs longer; the figures are in the evaluation chapter. Both models' weights are baked into the image, so a cold start never fetches from the model hub. The CPU image does not carry this model.

Selection is a request field. On /v1/chat the host sends model: "cognitio-infer" and, optionally, think; on /v1/agent this model is the default. The host must hold the conversation, run any tool the model asks for and send the result back as a tool turn, decide when the loop ends, and apply its own retry policy beyond the API's single retry. The [cognitio command line](/docs/cli) is the reference host: cognitio "…" and cognitio chat use /v1/chat, cognitio code "…" runs the tool loop over a repository through /v1/agent, asking before every write and command, and stops after thirty steps.

Limits & safety #

Cognitio Infer does not see the host's files, shell, memory or user beyond what the messages and tool results carry: it reads a rendered conversation of at most 14,000 tokens and nothing else. It does not see images, locations, dates or the real time.

• Its reasoning is prompted, not trained: the trace is the base model thinking aloud under an instruction, it can be wrong while the answer is right and the reverse, and no effort level guarantees correctness.
• It is slow and single-file: seconds per turn, one generation at a time on a GPU shared with Cognitio, and a long tool loop is many turns.
• Its tools are whatever the host offers; it can ask for a call with wrong or harmful arguments, and it is the host that runs them. The reference harness confirms every write and command with the user for that reason, and refuses paths outside the repository.
• It has no retrieval, no current data and no memory across requests; long tool output crowds older turns out of the window, and the service refuses a conversation over 14,000 tokens rather than trimming it.
• It performs no input or output moderation of its own; an intention model such as ELIM, run before the call, is the intended check.
• It is non-deterministic at the default temperature, and the same conversation can produce a different trace and a different answer on a second call.
• The weights are a third party's published model; its knowledge, biases and licence are the vendor's, and Falcon distributes no weights for this model.

Out of scope: Cognitio Infer is a responder that can plan and ask for tools, not an autonomous agent. Decisions with consequences — spending, sending, deleting, releasing — belong to the host and its user, which is why the reference harness asks first; safety decisions belong to an intention model run before the call. Not medical, legal or crisis advice.

Fixed weights per version; the model does not learn from requests.

Versions #

Compatibility: a major version bump changes the request or response contract — the message and tool shapes on /v1/messages, the think levels, the fields of the returned message — or the base model family, since a different base changes tokenisation, the tool-call format and what the thinking budgets buy. A minor bump is a new published release of the same base family, or a new thinking or tool prompt, with the same contract; answers will differ in wording but not in shape. A patch bump changes metadata or runtime only — image, precision, deployment shape, budgets — with identical weights.

The current weights identifier is the vendor's published snapshot a09a35458c702b33eeacc393d103063234e8bc28 of Qwen/Qwen2.5-7B-Instruct, loaded at build time into the Cognitio image with no Falcon adapter on top; there is no training run to record. Falcon distributes no weights for this model, during the private preview or after; see Access and Status & versioning.