Perceptron Mk1.5 adds audio, video tracking and tools for embodied agents

Perceptron introduced Mk1.5 on 25 September 2026 as a model for agents that interpret their surroundings and choose actions through tools. Its model card lists text, images, video and audio as inputs, with text, spatial and temporal annotations and function calls as outputs. The emphasis is on connecting perception to an application’s control loop: a developer supplies the camera feed, tools, execution rules and checks; the model proposes observations and calls.

That distinction matters for the company’s drone, quadruped, smart-glasses and phone demonstrations. They show Mk1.5 operating within configured systems, and Perceptron says the same model was used across these platforms without platform-specific retraining. They do not, by themselves, establish general autonomous control, safe operation in unfamiliar environments or a reusable robot policy independent of the surrounding software.

What the release adds

Capability	What the model returns	What the application still has to do
Native audio input	Text answers, transcription or time-localized audible events	Supply a recording; explicitly enable a video’s soundtrack
Video object tracking	Timestamped points or boxes grouped into an object track	Validate positions, handle occlusions and render an overlay
Function calling	A function name and arguments in an OpenAI-style chat response	Validate and execute the function, then return its result
Sub-agent workflows	The launch shows parallel task delegation	Build and govern the orchestration around model calls

For tracking, a <track> groups observations of the same target across time. This is different from returning a separate detection for every frame: the application receives a time-indexed trajectory it can use for an overlay or subsequent analysis. The tracking guide warns that observations can be sparse. Filling gaps by interpolation is a client decision, and a missing position during occlusion should not be treated as a model observation. It also recommends checking timestamps, coordinates and completion status before using the result.

Audio is an input modality, not generated speech. According to the audio guide, Mk1.5 accepts WAV, MP3 and FLAC recordings for transcription, questions and sound-event localization. A video’s soundtrack is ignored unless the request explicitly enables it. Streaming the text response does not make the endpoint a live microphone stream. These details matter when an application is expected to synchronize what the camera sees with what a microphone hears.

The tools story also needs a precise boundary. The public function-calling API lets Mk1.5 request functions described by the developer. The caller validates arguments, executes the function and returns a result. Naming a function search does not activate a hosted web-search service. Perceptron’s launch demos include search, page reading, zoom, reverse-image search and parallel sub-agents; those demonstrations depend on tools and orchestration supplied around the model. In a physical system, permission checks, action budgets and emergency stops belong in that surrounding control layer.

What the reported evaluations show

Perceptron’s launch evaluation reports a lead on three of four tracking benchmarks; MolmoPoint-8B leads on MeViS valid_u. Some competitor scores come from published papers. The chart below preserves the original Molmo2-Track comparison.

For an application builder, the useful question is whether that advantage survives their own camera, lighting and occlusions. A benchmark ranking alone does not tell us how many objects an inspection system will lose over an entire shift, or whether it can recover the correct identity afterward. Those should be measured separately before connecting a track to an action.

The company describes roughly 2–5× faster end-to-end completion than Mk1, with up to 4.7× in its chart. The reported median comes from three runs on one H100 for selected chat, image-question and video workloads. That does not establish the speed of a whole robot action loop: capture, network transfer, tool execution and actuator response are additional steps. We have not independently run these benchmarks or tested a robot with Mk1.5.

Access, limits and cost

Perceptron says Mk1.5 is available through its platform and updated SDK. The API model ID is perceptron-mk1.5; the quickstart uses the chat-completions endpoint and Python package version 0.4.0 or later.

Published specification	Value and qualification
Standard input / output	$0.15 / $1.50 per million tokens
Cached input	$0.0375 per million tokens on the current model card
Context	Launch post: 32K multimodal tokens; current model card: 36,864 tokens
Maximum output	8,192 tokens on the current model card
Audio per item	16,384 audio tokens, about 21.8 minutes at the documented estimate

The context figures in Perceptron’s own sources do not match. For integration planning, use the current model card and verify the effective limit through the API; do not assume the launch-post shorthand is the enforced value. Audio, video frames, text, tool definitions, reasoning and output all share the context budget. The tokenization guide cautions that video cost cannot be inferred from file size or simply multiplying frame rate by duration. Measure reported usage on representative clips. The audio estimate is likewise a planning figure; the server’s token count controls acceptance.

Where this is most useful

Mk1.5 is most interesting when an application needs to connect what happened and where to a bounded action: follow a component through inspection footage, locate a spoken alarm in a video, or ask a tool for the maintenance record of an observed part. A practical evaluation would use footage from the intended camera, lighting and motion conditions; compare track continuity and missed events; and record the cost and latency of the complete tool loop. For any movement or consequential action, independently validate the model’s suggested tool call before execution.

The release broadens the interface for perception-driven agents. How reliably that interface works on a particular device or site still has to be measured there.

Sources checked on 26 September 2026. Benchmark and hardware-deployment claims are attributed to Perceptron; no independent model or robot evaluation was conducted for this article.