Cognitio

Overview #

Cognitio writes a short, friendly reply to small talk. You give it the latest message and a few earlier turns; it gives back one or two casual sentences. Use it when a conversation is a check-in or a quick question rather than a job, and a large language model would be slower and dearer than the moment deserves.

Cognitio is a conversational responder: given the current turn and a short window of prior turns, it returns one or two short sentences in a casual, warm register. It is the Response-family model a host application calls when a conversation is small talk rather than a job — a check-in, a quick opinion, a one-line question — and a full-size responder would be slower and more expensive than the moment deserves.

Cognitio is a fine-tune, not a model trained from scratch. It applies an 18.5M-parameter LoRA adapter, trained by Ducky Software, to Qwen2.5-1.5B-Instruct, an open-weight 1.5B-parameter instruct model. The adapter shapes the base model's voice and length; the base model supplies the language ability. Input is the current user turn plus up to eight prior turns, rendered by the service as a ChatML conversation behind a fixed system prompt; output is a single reply string capped at 96 new tokens.

Within the response family, Cognitio is the only member since ERM, the fixed-line reserve responder, was withdrawn on 2026-09-20: it generates free text on a GPU. Cognitio has no classification heads and no label catalogue; it does not decide whether a reply is safe to give. A host that needs that check runs an intention model such as ELIM before calling Cognitio.

Intended use #

• Short conversational replies — greetings, check-ins, casual opinions, quick how-tos — where one or two sentences are the right length.
• Keeping a consistent, brief voice across a conversation without prompting a large model for every turn.
• Serving as the ordinary responder in a host that already runs an intention check before it.

Out of scope #

• Long, technical, or multi-step answers; the 1.5B base and the 96-token cap both work against them.
• Anything that needs current facts, retrieval, or tools; Cognitio has none and will guess.
• Safety screening or content moderation of its own input or output.

Choose Cognitio when #

• The host needs a generated reply in a consistent casual register and can afford a GPU-backed HTTP call of a few seconds.
• The conversation window is short (at most nine turns) and the expected reply is one or two sentences.
• A host that needs a deterministic fallback while the service is unavailable holds a fixed line of its own; no Falcon model plays that part.

Architecture #

Cognitio is two artefacts: an open-weight base and a small adapter. The base is Qwen2.5-1.5B-Instruct, a decoder-only transformer with 28 layers, a hidden size of 1,536, an MLP intermediate width of 8,960 and grouped-query attention with key/value projections 256 wide (dimensions confirmed by the adapter's tensor shapes). Its tokeniser is Qwen2 byte-pair encoding, and the tokeniser files are saved beside the adapter so the service never needs the base's tokeniser separately.

The adapter is a PEFT LoRA of rank 16 with alpha 32 (a scale of 2.0) and dropout 0.05 during training. It targets all seven linear projections in every layer — the four attention projections q_proj, k_proj, v_proj, o_proj and the three MLP projections gate_proj, up_proj, down_proj — giving 392 low-rank tensor pairs and 18,464,768 trainable parameters, about 1.2% of the base. Per layer the budget is 49,152 parameters each for q_proj and o_proj, 28,672 each for k_proj and v_proj, and 167,936 each for the three MLP projections. No modules are fully fine-tuned and no layers are excluded.

The adapter is not merged into the base. At load the service reads the base weights, applies the adapter on top, and serves in float16 on GPU or float32 on CPU. The 4-bit NF4 quantisation used to fit training on one L4 is a training-time device only; inference runs the base at full precision for its dtype.

Cognitio has no heads and no label catalogue: it is free-text generation, and the only fixed vocabulary is the ChatML role set (system, user, assistant). There is no vision, no retrieval and no tool interface — it never sees anything but the rendered conversation text.

Inputs & outputs #

Input #

The service accepts a JSON body and renders it as a ChatML conversation: a fixed system prompt (a short persona line asking for brief, warm, human-sounding replies — baked into both training and serving), then the history turns in order, then the current prompt as the final user turn, then an assistant-turn generation prompt. Only the last eight history turns are kept; turns with empty content are dropped; the roles user and assistant map to their ChatML equivalents and model is accepted as an alias for assistant. The reference client sends the last six turns.

{
  "prompt": "you free later?",
  "history": [
    { "role": "user", "content": "hey" },
    { "role": "assistant", "content": "hey — what's up?" }
  ],
  "max_new_tokens": 64,
  "temperature": 0.7
}

The rendered conversation the model sees has this shape (the system line is fixed by the service and is not part of the request):

<|im_start|>system
…fixed persona line…<|im_end|>
<|im_start|>user
hey<|im_end|>
<|im_start|>assistant
hey — what's up?<|im_end|>
<|im_start|>user
you free later?<|im_end|>
<|im_start|>assistant

Limits: prompt must be non-empty after trimming; max_new_tokens is clamped to 1–96 (default 64); temperature has a floor of 0.05 (default 0.7). The service applies no explicit byte limit to the body, but the model was trained on sequences of at most 512 tokens and the base's own context window is far larger than any window the host should send. There is no frame, no units and no structured input beyond the turn list.

Output #

A successful response is a small JSON object. The reply is decoded with special tokens removed, then cut at the first blank line, end-of-text marker or legacy speaker-prefix line, then whitespace-trimmed. Typical replies are one or two short sentences, mostly lower-case, in the em-dash style of the training data.

{
  "text": "might be — what time were you thinking?",
  "model": "cognitio",
  "base_model": "Qwen/Qwen2.5-1.5B-Instruct",
  "backend": "friend-lora"
}

The reply shown is the human-written reference for this exact prompt in the 80-turn held-out set, given as an illustration of the target style; it is not a captured generation. Sampling at temperature 0.7 varies from call to call, and the same request can return a different reply on a second call.

There is no confidence, score or decision rule: Cognitio returns text or an error. The reference client strips any stray end-of-text marker, collapses whitespace, and treats an empty text as a failure rather than a valid reply. A host that wants deterministic output sets temperature to 0.05 or below.

Training #

Cognitio's adapter was trained by QLoRA supervised fine-tuning on 2,112 human-authored conversational exchanges. The seed set is 2,129 rows written by Ducky Software, each pairing a short user turn with a short reply — greetings, unit conversions, quick how-tos, polite drafts, reminder phrasing. No real user data was used: the rows are authored examples, not captured conversations. No teacher model was used either; an optional synthesis path exists in the trainer but the released run's row count matches the filtered human seed exactly, so the released adapter used the seed alone.

Preparation drops any seed row whose user text appears in the evaluation set, normalises whitespace, applies a reply filter, and de-duplicates on lower-cased user text. The filter keeps rows whose reply is 1–45 words and rejects assistant-register phrasing (“as an AI”, “I'm happy to help”, “in conclusion”, “furthermore”, “delve” and similar), a short list of inappropriate phrases, replies with more than three line breaks, and replies with three or more bullet lines. The result is the 2,112-row training set; 80 further human-authored turns form the held-out evaluation set (val_rows 80 in run-lora.json) and are excluded from training by user text.

Each row is rendered as a full ChatML conversation — the fixed system prompt, the user turn and the reply — and the loss is computed over the whole sequence; no completion-only masking is configured. The base was loaded in 4-bit NF4 with double quantisation and bfloat16 compute; the LoRA configuration is rank 16, alpha 32, dropout 0.05 on all seven projection families. Optimisation ran for 400 steps at batch size 2 with gradient accumulation 8 (16 sequences per step), learning rate 2e-4, cosine schedule with 3% warm-up, maximum sequence length 512, no packing, seed 7, evaluating every 100 steps. 400 steps is 3.03 epochs over 2,112 rows; the planned four epochs were cut by the step limit. Total compute was 3.1e15 FLOPs.

The run was Vertex AI custom job friend-lora-20260910-113709 on 2026-09-10, on a g2-standard-8 machine with one NVIDIA L4, using transformers 4.46.3, PEFT 0.13.2, TRL 0.11.4 and bitsandbytes 0.44.1. The adapter saved at step 400 is the one released; no best-checkpoint selection was applied. Cognitio is not warm-started from any other Falcon model. An earlier, archived Cognitio — a 124M-parameter GPT-style decoder pretrained on TinyStories and fine-tuned on the same seed pairs — is not part of this version and is not served.

Cognitio was not trained on multi-turn conversations (every training row is one exchange behind the system prompt), on long-form or technical answers, on tool or retrieval traces, or on any safety or refusal data.

Evaluation #

Evaluation is the loss on the 80-turn human-authored held-out set, measured every 100 steps by the trainer and recorded in trainer_state.json; the held-out turns are excluded from training by user text and were never seen by the adapter. Training loss at step 400 was 0.602. No human-preference, win-rate, latency or throughput measurement is recorded for this version; the project's stated success bar — preferring the adapter's replies over the archived Cognitio on at least 70% of the 80 held-out turns for brevity, warmth and helpfulness — has no recorded result and is therefore not claimed here.

Known gaps: held-out loss was lowest at step 100 (1.350) and rose steadily to 1.492 at step 400, while training loss fell from 1.07 to 0.60 over the same span. That is mild over-fitting to a 2,112-row set, and the released adapter is the step-400 one, not the step-100 one. The style is strongly tied to the training persona (lower-case, em dashes, Canadian references such as CAD, Celsius and local transit), which is a feature for the intended host and a limitation for any other. Evaluation covers single exchanges only; multi-turn behaviour with the eight-turn window is untested. Estimates, never measurements.

API #

Cognitio is exposed through the Falcon API at /v1/chat. Requests count against the complete preview quota bucket. The route forwards one turn, with up to eight prior turns of history, to the Cognitio service and returns the reply in the public envelope; 503 chat_unavailable is returned when the service is not configured, 413 text_too_long above 2,000 characters. Envelope, authentication, rate limits and retry guidance are in API conventions. The service's own contract, which the route wraps, follows.

POST /v1/chat HTTP/1.1
Authorization: Bearer $FALCON_API_KEY
Content-Type: application/json

{ "text": "long day. talk me through dinner ideas that take ten minutes?", "history": [ { "role": "user", "text": "hey" }, { "role": "assistant", "text": "hey — what's up?" } ], "temperature": 0.7 }

{ "ok": true, "engine": "cognitio", "text": "…" }

Route /v1/chat · quota bucket complete · body limit 64,000 bytes; text at most 2,000 characters; at most 8 history turns.

POST /v1/complete HTTP/1.1
Content-Type: application/json

{
  "prompt": "long day. talk me down",
  "history": [
    { "role": "user", "content": "hey" },
    { "role": "assistant", "content": "hey — what's up?" }
  ],
  "max_new_tokens": 64,
  "temperature": 0.7
}

A 200 response carries text, model, base_model and backend as documented under Output. Any other status carries a single error string:

The health route reports load state and never blocks on generation:

GET /health HTTP/1.1

{
  "ok": true,
  "model": "cognitio",
  "base_model": "Qwen/Qwen2.5-1.5B-Instruct",
  "backend": "friend-lora",
  "ready": true,
  "loading": false,
  "error": null
}

ready is true once the adapter has been applied; loading is true while the background load runs; error carries the load failure message when one occurred. Because Cognitio is not on a public route, it has no preview quota bucket and no per-request body limit beyond what the service reads.

Runtime & deployment #

Cognitio serves from a single Python process running a threading HTTP server on the platform's port. At start it launches the load in a background thread: tokeniser from the adapter directory, then the base weights, then the adapter applied on top, and since 2026-09-16 one warm-up generation, so that the first reply an instance serves costs the same as later ones. Until that finishes, every POST returns 503 model still loading (and triggers the load if it has not started), and /health reports ready: false, loading: true. If loading fails, /health reports the error and every POST returns 500 until the instance is replaced.

The reference client behaves as follows:

• POSTs to /v1/complete with Content-Type: application/json and a 60 s timeout.
• On HTTP 503 or a timeout, waits 2.5 s and retries exactly once.
• Treats any other non-2xx status, or an empty text, as an error and lets the host fall back to another responder for that turn.
• Sends the current turn plus the last six history turns, max_new_tokens 64 and temperature 0.7 unless the host overrides them.

Two container images exist. The production image is the GPU build, running on a serverless container platform with one NVIDIA L4, 16 GiB of memory and 4 vCPUs, a 300 s request timeout, at most one instance and no minimum instances, so a cold start after idle means one or more 503 responses before the first reply. Once the instance is ready a reply takes about 0.8 s measured through the Falcon API; before the start-up warm-up the first reply after a cold start was about 5 s slower than the rest. A CPU-only image runs the same service in float32 with 8 GiB of memory, 2 vCPUs, request concurrency 1 and at most two instances; it is correct but slow. Both images bundle the adapter and pre-download the base weights at build time, so a cold start never fetches from the model hub.

Selection is configuration, not a command. The host application marks a conversation as using model: "cognitio" and points its client at the service's base URL; when the base URL is unset, the host does not attempt the call. The host must provide the service base URL, the conversation window (current turn plus recent history in {role, content} form), its own retry and fallback policy beyond the client's single retry, and any intention check it wants to run before generation — Cognitio itself performs none. A hosted endpoint for this version has been serving since 2026-09-10.

Limits & safety #

Cognitio does not see the host application's tools, memory, user profile or any earlier part of the conversation: it reads at most nine turns rendered as ChatML text behind a fixed system prompt, and nothing else. It does not see images, locations, dates or the real time.

• It has no retrieval, no tools and no current data; questions about weather, prices, schedules or news receive a plausible guess, not an answer.
• It is small: long, technical or multi-step requests exceed what a 1.5B base and a 96-token cap can do well.
• It performs no input or output moderation of its own; the training filter removed assistant-register and inappropriate phrasing from the data, but nothing at inference checks the request or the reply.
• It is non-deterministic at the default temperature; the same request can produce different replies, and an empty or truncated reply is possible.
• Its voice is fixed by the training persona (lower-case, em dashes, Canadian references) and cannot be steered per request; there is no persona or style parameter.
• Its context is short: history beyond the last eight turns is silently dropped, and multi-turn coherence was not evaluated.
• The service has no authentication and no rate limiting; access control belongs to the deployment.

Out of scope: Cognitio is a conversational responder, not a task runner. Requests that need an action — a lookup, a booking, a reminder, a calculation that must be right — belong to the host application's own tools, and safety decisions belong to an intention model such as ELIM run before the call. Not medical, legal or crisis advice.

Fixed weights per version; the model does not learn from requests.

Versions #

Compatibility: a major version bump changes the request or response contract — the field names on /v1/complete, the role set accepted in history, the shape of the 200 body, or the base model family, since a different base changes tokenisation and the meaning of max_new_tokens. A minor bump is a retrain of the adapter on the same base with the same contract (more rows, more steps, a different checkpoint selection), and replies will differ in wording but not in shape. A patch bump changes metadata or runtime only — image, precision, deployment shape — with identical weights.

The current weights identifier is the adapter file adapter_model.safetensors (73,911,112 bytes, SHA-256 43f57ae7de6f3e6898ceb9b24d2a8d485d193b8694aa5103fa4e01181fbdda1e), applied at load to Qwen/Qwen2.5-1.5B-Instruct; the training run is recorded in run-lora.json and trainer_state.json from job friend-lora-20260910-113709. Weights are not distributed during the private preview; see Access and Status & versioning.