Waymark FlyTrack

Overview #

Waymark FlyTrack looks at drone footage that has already been recorded and says what vehicles and animals are in it, where each one is in every frame, and how each one moved. It is built for counting and flow: how many trucks used the haul road this morning, how many sheep crossed from one field to the other, where the vehicles on a site were at each second of a clip. Every detection is a class and a box; a track is a run of detections that the tracker judged to be the same object within one clip.

It is the fourth member of the Waymark series and the first to read pixels directly: Waymark and Waymark Extra read a scene sketch written from a photo, and Waymark Flight reads an airspace scene. FlyTrack is a small centre-point detector trained from scratch on openly licensed aerial imagery, paired with a deterministic tracker that has no weights at all.

What it deliberately does not do is identify. There is no person class, no face or appearance feature, no re-identification and no memory across clips. A track id means “the same car for the next few seconds of this clip” and nothing more. It works on saved files sent to the API, not on a camera feed.

Intended use #

• Traffic and haul-road counts from a drone clip: how many cars, trucks and buses passed, and in which direction.
• Herd and flock counts and movement from a survey flight over a field, a pen or a paddock.
• Site surveys: where the vehicles on a yard were at each moment of a clip, as paths in frame coordinates.

Out of scope #

• People: they are not a class, are never boxed and are never tracked. Footage in which the interest is a person is out of scope, whatever the consent arrangements.
• Identity of any kind: which truck, whose car, which animal. Track ids are per clip and carry nothing that links one clip to another.
• Camera feeds and anything time-critical: the route takes a saved clip and answers in tens of seconds.
• Equipment and other object classes: the first version knows four classes, and objects outside them are ignored or mislabelled as the nearest one.

Choose Waymark FlyTrack when #

• The footage is straight-down or steeply oblique drone video of vehicles or livestock, at the altitudes its training data covers (roughly 15 m for animals and 150 to 250 m for vehicles).
• A count per class and a rough path per object are the answer; if the question is about the layout of a scene, Waymark is the model.
• Otherwise there is no other member of the family for pixels; the held-out figures, once published, say how far to trust it.

Architecture #

The detector is a CenterNet-style network kept small enough to run on CPU. A stem of two convolutions halves the image, four residual stages take it to a thirty-second of its size, and a top-down path adds the finer feature maps back in until the output is at a quarter of the input resolution. At that resolution three heads answer per cell: a heatmap with one channel per class that peaks at object centres, the box width and height in pixels, and the fraction of a cell by which the true centre is offset. Decoding keeps the local maxima of the heatmap and reads a box at each; there is no anchor and no non-maximum suppression beyond the peak pick.

The tracker is code. Each open track predicts where its box will be next frame from a smoothed constant velocity. High-score detections are matched to predictions first, greedily by intersection over union; the tracks still unmatched then get a second chance against the low-score detections, which is what keeps a track alive through a frame of blur or partial occlusion. A track is confirmed after two consecutive hits and retired after eight frames without one, so a single spurious detection never becomes a track and a brief gap does not split one. Ids are integers assigned in order within a clip.

The service decodes the clip with ffmpeg at the requested frame rate, scales the frames to a long side of 1,024 pixels, runs the detector on each and the tracker across them, and returns everything in the pixels of the decoded frames.

Inputs & outputs #

Input #

Exactly one of the media fields is sent. A clip is read from its start for max_seconds at fps frames a second; a photo is one frame, so it returns detections and counts but no tracks.

{
  "video_base64": "data:video/mp4;base64,AAAAIGZ0eXBpc29t…",
  "fps": 3,
  "max_seconds": 10,
  "min_score": 0.25
}

Limits: the JSON body is at most 24,000,000 bytes, a clip at most 15,000,000 bytes and a photo at most 6,000,000 bytes; fps is clamped to 0.5–5, max_seconds to 0.5–12 and min_score to 0.05–0.95. Frames are scaled so the long side is 1,024 pixels, which sets the smallest object the detector can see at roughly ten pixels in the decoded frame.

Output #

A successful response carries the decoded media description, the per-frame detections, the tracks and the counts.

{
  "media": { "kind": "video", "width": 1920, "height": 1080, "duration_s": 6.2, "frames": 18, "fps": 3 },
  "frames": [
    { "t": 0, "detections": [ { "class": "truck", "score": 0.91, "box": [412.0, 220.5, 470.2, 251.0], "track": 1 } ] }
  ],
  "tracks": [
    { "id": 1, "class": "truck", "first_t": 0, "last_t": 5.67, "frames": 18, "max_score": 0.94, "path": [[0, 441.1, 235.8], [0.333, 448.0, 236.2]] }
  ],
  "counts": { "car": 0, "truck": 1, "bus": 0, "animal": 0 },
  "coordinates": "pixels of the decoded frame (long side ≤ 1024)",
  "latency_ms": 4200,
  "notice": "Class-level tracking of vehicles and animals in saved drone footage; no identity, no live use."
}

The example is illustrative of the shape. A detection with track: null was seen but not yet confirmed as a track, or belonged to a track that never reached two hits; counts counts confirmed tracks, so it is the number a host should report, not the number of boxes.

Training #

The detector is trained from scratch on four openly licensed sources, all Creative Commons Attribution 4.0 and attributed in meta.json: the Aerial Multi-Vehicle Detection Dataset (cars, buses and trucks in 9,048 stills at 150 to 250 m), the ICAERUS drone images of sheep and of goats (low-altitude flights over farms in France), and the New Zealand Cattle Detection tiles, whose point annotations become fixed 22-pixel boxes at 0.1 m per pixel. Sheep, goats and cattle share one animal class; no dataset with a research-only licence is used, and no person is a class.

Every image is cached once with a long side of 1,536 pixels. Training draws two 512 × 512 crops per image per epoch, biased toward objects two times in three, with scale jitter, flips and light colour jitter, and rasterises the centre heatmap, the sizes and the offsets at stride 4. The loss is the CenterNet focal loss on the heatmap plus L1 on the sizes and offsets at object centres; AdamW with a cosine schedule and bf16 autocast on one A100. The epoch with the best held-out average precision is kept.

The tracker is not trained. It is scored on four ICAERUS drone videos of sheep that carry per-frame boxes and consistent ids, which never enter training.

First run: the base size on the four sources, Vertex AI job flytrack-20260923-201621. Released once evaluated.

Evaluation #

Evaluation results have not been published for this version.

API #

Waymark FlyTrack is exposed through the Falcon API route /v1/track. Requests count against the spatial preview quota bucket, one unit per request whatever the clip length. Envelope, authentication, rate limits and retry guidance are in API conventions.

POST /v1/track HTTP/1.1
Authorization: Bearer $FALCON_API_KEY
Content-Type: application/json

{ "video_base64": "data:video/mp4;base64,AAAAIGZ0eXBpc29t…", "fps": 3, "max_seconds": 10 }

{ "ok": true, "engine": "waymark-flytrack", "media": { "kind": "video", "width": 1920, "height": 1080, "duration_s": 6.2, "frames": 18, "fps": 3 }, "frames": [ { "t": 0, "detections": [ { "class": "truck", "score": 0.91, "box": [412.0, 220.5, 470.2, 251.0], "track": 1 } ] } ], "tracks": [ { "id": 1, "class": "truck", "first_t": 0, "last_t": 5.67, "frames": 18, "max_score": 0.94, "path": [[0, 441.1, 235.8]] } ], "counts": { "car": 0, "truck": 1, "bus": 0, "animal": 0 }, "latency_ms": 4200, "notice": "Class-level tracking of vehicles and animals in saved drone footage; no identity, no live use." }

Route /v1/track · quota bucket spatial · body limit 24,000,000 bytes; a clip of at most 15,000,000 bytes or a photo of at most 6,000,000 bytes.

The service behind the route takes the same body on its own /v1/track and answers /health with readiness, the class list and the limits.

Runtime & deployment #

The service is a Python container on CPU with the detector weights, ffmpeg and the tracker. A ten-second clip at three frames a second is thirty detector passes at a long side of 1,024 pixels, about a second each on four cores, so a request answers in roughly half a minute; a photo answers in about a second. The container scales to zero when idle and the Falcon API turns its loading state into 503 track_warming with a retry hint.

• The host sends the file; nothing is fetched from a URL, and nothing is kept after the response.
• Frame rate and duration are the cost knobs: fewer frames answer faster and lose short-lived tracks.
• The same trainer package serves and trains, so the detector, the decoding and the tracker in the service are the code the evaluation used.

Limits & safety #

FlyTrack sees only the pixels it is sent, at a long side of 1,024. It does not know the altitude, the camera, the ground sampling distance or the location, and it does not see any other clip.

• People are not a class. The model has not been trained to see them and returns nothing for them; a person in the footage is not a detection, not a track and not a count. The scope of the model is vehicles and animals, and the notice in every response says so.
• No identity, ever: no re-identification, no appearance features, no linking across clips, no face processing. A track id is an integer valid inside one response.
• Saved footage only. The route has no streaming form and answers in tens of seconds, which rules out real-time tracking by design.
• It knows four classes at the altitudes of its training data. Objects it has not seen the like of, clips filmed much lower or higher, night footage, thermal footage and heavy motion blur are outside what the held-out figures describe.
• Counts are counts of confirmed tracks, which a long occlusion can split and a crowd can merge; the per-frame detections are there to be checked.
• The footage the host sends may show private property or people; the host is responsible for having the right to process it. The model cannot make that judgement and does not try.

Out of scope: FlyTrack does not identify, recognise, follow across clips or watch in real time, and it is not a safety system. Not a substitute for a survey, and no help with a clip whose subject is a person.

Fixed weights per version; the model does not learn from requests.

Versions #

Compatibility: a major version bump changes the request or response contract — a field added, removed or renamed, a class added to or removed from the list — or the geometry the service decodes at. A minor bump is a retrain with the same contract; the figures on this page change, the shapes do not.

No weights are released yet; this page records the training run and is updated with the bundle's identifier and hash when it is. Weights are not distributed during the private preview; see Access and Status & versioning.