Waymark Extra

Overview #

Waymark Extra describes how someone would walk through a photographed scene. You give it a written sketch of the scene and a goal such as “walk to the door”, which can have a stop on the way or involve a person coming toward you; it gives back the walk step by step — where the person is at each moment, which way they face, what they do on the way — and a one-line summary you can show as it is.

Waymark Extra is the movement model of the Waymark series. It reads a Falcon scene sketch — the structured text a vision model emits for a photographed scene — together with a movement goal such as “walk to the stair flight, then over to the plaza paving”, and returns a timed trace of a person moving through that scene. Each line of the trace gives a time, a position in approximate metres, a heading, and short notes about what is ahead, beside or behind the mover. The Falcon API parses the trace into step objects and adds a plain-words summary built from the last step and the events along the way.

Where Waymark answers layout questions, Waymark Extra simulates walks of up to twelve steps. Its 1024-token window fits a long sketch and a long trace, and its training set covers two-leg routes with a waypoint, a person walking toward the viewer, a person heading off into the background, and a person who simply waits. Scenes in its data are busy — a second person, an extra road, a flight of stairs — so give-ways, road-edge stops and climbs are common rather than rare. It keeps Waymark’s layout question-and-answer ability in the same weights.

Everything it returns is an estimate reconstructed from the sketch’s depth and lateral bands. It never sees pixels, and its coordinates are approximate metres in a viewer-relative frame: x right, y ahead, z up, with the viewer at the origin.

Intended use #

• Turning a goal such as “walk to the doorway” into a plausible timed path through a sketched scene, with a summary a host can show or hand to its responder.
• Predicting where a named person in the sketch — a jogger, a cyclist, a dog walker — would plausibly end up, including approach toward the viewer, departure into the background and waiting.
• Producing steps in the viewer frame that an intention model such as LIM3D-XL can read for intent, hazards and advice when no tracker output is available.
• Two-leg routes with a stop on the way, and busy scenes with more than one person.

Out of scope #

• Navigation, accessibility or safety decisions; distances are estimates, not measurements.
• Predicting what a real person will actually do; a trace is a plausible simulation, not a forecast.
• Scenes or objects outside the fifty-kind sketch vocabulary, and sketches whose kind is not scene.
• Answering inside a tight latency budget: a trace takes seconds on a GPU, and a cold start takes minutes.

Choose Waymark Extra when #

• The question is about movement through the scene rather than static layout: how someone gets somewhere, and what they do on the way.
• The goal is a two-leg route, an approach toward the viewer, a walk into the background or a loiter, or the scene is long or busy.
• Only a layout question is being asked — choose Waymark, which is the smaller model built for exactly that and answers in well under a second.
• The host has an observed track rather than a goal — choose LIM3D-XL or LIM3D, which read steps and return intent, hazard and advice labels.

Specification #

Try it #

You send

A sketch of a path with a ramp, a crosswalk and a dog walker, and the goal “the dog walker walks toward you”.

You get back

A trace of the dog walker closing from about three metres away and arriving beside the viewer after a few seconds, plus a one-line summary.

The same exchange as the API sees it:

{
  "sketch": "kind: scene\nviewer: eye-level on path\ncapture: shot phone; orientation portrait, looking-down; height_agl eye-level ~1.5-1.8m\nobjects: roadway below (right midground); sidewalk (right midground, colour grey concrete); retaining wall (left foreground); ramp (center-left midground); concrete overpass (right background, colour grey concrete); crosswalk (left foreground); dog walker (center-right foreground)\nrelations: roadway below behind crosswalk; roadway below next to sidewalk; retaining wall in front of sidewalk; crosswalk left of retaining wall; ramp behind retaining wall; ramp left of sidewalk; roadway below right of center; crosswalk left of center\nrelative: crosswalk closer than roadway below; retaining wall closer than sidewalk; retaining wall closer than ramp\nuncertain: true distances not visible",
  "goal": "the dog walker walks toward you",
  "mover": "dog walker",
  "max_new_tokens": 400,
  "temperature": 0.2
}

{
  "ok": true,
  "engine": "waymark-extra",
  "frame": "approx m; x right, y ahead, z up; viewer at origin",
  "mover": "dog walker",
  "trace": "t=0.0s (0.9, 3.2, 0.0) back | you ~3m ahead | behind: sidewalk ~2m | left: roadway below ~2m | right: ramp ~2m\nt=1.5s (0.4, 1.8, 0.0) back | you ~1m ahead | ahead-right: retaining wall ~2m\narrive t=2.6s (0.1, 0.8, 0.0) you | retaining wall right | crosswalk right",
  "steps": [
    {"t": 0.0, "x": 0.9, "y": 3.2, "z": 0.0, "heading": "back", "notes": ["you ~3m ahead", "behind: sidewalk ~2m", "left: roadway below ~2m", "right: ramp ~2m"], "arrive": false},
    {"t": 1.5, "x": 0.4, "y": 1.8, "z": 0.0, "heading": "back", "notes": ["you ~1m ahead", "ahead-right: retaining wall ~2m"], "arrive": false},
    {"t": 2.6, "x": 0.1, "y": 0.8, "z": 0.0, "heading": "arrive", "target": "you", "notes": ["retaining wall right", "crosswalk right"], "arrive": true}
  ],
  "summary": "The dog walker reaches the you in about 3s, ending ~0m straight and ~1m ahead of where the photo was taken."
}

Limits & safety #

It does not see the image; it reads the sketch. Anything the sketch omits, mislabels or misplaces, the trace inherits, and the model will not add objects, exits or paths the sketch did not mention.

• Distances and positions are approximate metres reconstructed from depth and lateral bands. Estimates, never measurements. Nothing in a trace is GPS, surveyed or scaled to the real scene.
• Traces are capped at twelve lines and 480 generated tokens; long sketches lose their oldest lines to make room.
• Only the viewer stops at a road edge; simulated persons cross roads without stopping, and a road counts as crossable only if a crosswalk appears somewhere in the sketch. Even for the viewer, the model stopped in only 3 of 11 evaluated road-edge scenes.
• Avoidance is a geometric sidestep of at most three detours per trace, not path planning; the side word in sidestep and give way notes is inverted in the training data.
• Walks toward the viewer end about one metre in front of the origin and walks into the background end at 19–22 m by construction of the training data, whatever the scene.
• Loiter traces have no arrival line and carry a blank-label here note on every line.
• Where another person goes is a plausible simulation from templates, not a prediction of anyone’s behaviour.
• The model is sized for a GPU with a single instance that scales to zero: one busy cold start can delay every caller by a minute or two.

Related models #

Latest versions #