Google Adds Agentic Video Processing to the Gemini API, Cutting Token Use by Up to 88%

Google has released agentic video understanding for the Gemini API, replacing fixed-rate video inspection with a query-directed process that can choose which parts of a recording to examine. Google reports that the mode uses up to 88% fewer tokens, lowers analysis costs by up to 66%, and improves accuracy by up to 7% on its video-analysis benchmarks.

The feature launched on September 1, 2026, for Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite across both the Interactions and GenerateContent APIs. It is available for uploaded files and public YouTube videos through Google AI Studio and the Gemini Enterprise Agent Platform.

Google released Gemini 3.8 Flash one day later. As of September 3, the live developer documentation also lists that model as supporting agentic video understanding, even though the original announcement and September 1 release note name only the three earlier Flash models.

1. Gemini No Longer Has to Process Every Video at a Fixed Rate

Gemini’s default static video-processing mode extracts frames at a fixed rate of one frame per second and places the resulting video context into the model in a single pass. Developers can adjust the sampling rate, but they must choose it before the model answers the question.

That creates a tradeoff for long recordings. A high sampling rate consumes more context and input tokens, while a low rate can miss brief actions, editing boundaries, visual defects, or other events that occur between sampled frames. Developers can build their own preprocessing pipelines to search transcripts, divide videos into segments, and resample promising intervals, but that work sits outside the model.

Agentic processing moves those decisions into Gemini’s inference loop. The model can navigate the timeline in response to the prompt, request selected frames or audio, inspect a transcript, and revisit relevant intervals at a different frame rate or resolution. It loads the evidence it determines is necessary instead of automatically placing a fixed representation of the full video into context.

This is a form of internal tool use rather than a new video-generation system. Gemini is still answering questions about supplied footage; the change concerns how it gathers visual and audio evidence before producing that answer.

Google identifies sub-second moment retrieval, anomaly detection, repeated-action counting, object counting, and searches across multi-hour recordings as target applications. Dynamic resampling is particularly relevant when an initial scan identifies a short interval containing rapid movement or a subtle visual change.

2. Developers Can Select the Mode for Each Video

In the Interactions API, developers enable the feature by setting a video input’s processing field to "agentic". The equivalent GenerateContent configuration uses the mediaProcessing or media_processing setting, depending on the SDK.

The selection is made per video rather than per request. An application comparing several recordings can therefore use agentic processing for a long lecture while retaining static processing for a short experiment in the same prompt.

That distinction matters because Google does not present agentic mode as universally faster. Its documentation recommends static processing for latency-sensitive clips shorter than five minutes and for work that requires consistent frame-level coverage across the entire recording. Agentic navigation may increase time to first token on short clips because the model performs internal reasoning and tool round trips before generating its response.

The Interactions API exposes evidence that this navigation occurred. Responses can contain processing_call and processing_result steps before the final model output. Applications may use those steps to show a progress trace, although they do not need to send a reply for each internal call. GenerateContent responses similarly preserve MEDIA_PROCESSING tool-call and response parts for multi-turn reasoning context.

Video context can also be retained across turns in a stateful interaction. This allows a user to ask follow-up questions without the application manually reconstructing the model’s previous inspection process.

Existing Gemini video-input limits still apply. Google’s documentation says Gemini 2.5 and later models can receive as many as 10 videos in one request. YouTube inputs must be public rather than private or unlisted, and free-tier users are limited to eight hours of YouTube video uploads per day.

3. The Efficiency Claims Depend on the Video and the Question

Google’s headline measurements are maximum improvements, not fixed reductions for every request. The company reports up to 88% lower token consumption, up to 66% lower analysis cost, and up to 7% higher accuracy across its standard video-analysis benchmarks.

The claimed gains are most relevant to long-form material, including 10-minute instructional videos, 90-minute lectures, and recordings lasting several hours. For these inputs, a narrowly targeted question may require only a transcript passage and a few visual intervals rather than a fixed-rate representation of the complete recording.

Token consumption becomes dependent on query complexity and the depth of dynamic sampling. A detailed question may cause Gemini to inspect more segments or sample selected portions above one frame per second. Developers therefore cannot assume that every request will reach the advertised 88% reduction.

Google has not added a separate feature charge for agentic video understanding. Requests use the relevant model’s standard Gemini API token prices. The reported cost reduction comes from processing fewer tokens, not from a discounted agentic-video rate.

The 66% maximum cost reduction is also smaller than the 88% maximum token reduction. Those figures should not be combined as if they described one guaranteed workload: they are benchmark maxima, and a complete request can include dynamically loaded media, reasoning, and output tokens charged under the selected model’s pricing.

Google’s figures remain vendor-reported. The announcement describes results across video-analysis benchmarks and demonstrates Gemini 3.7 Flash on long-video understanding, fast-action counting, and needle-in-a-haystack retrieval, but it does not establish that every production workload will improve by the same amount. Teams adopting the mode will need matched tests using their own videos, prompts, latency requirements, and scoring criteria.

4. The Immediate Impact Is on Long-Video Application Design

The change reduces the need for developers to construct separate transcript-search and keyframe-selection systems before sending a long video to Gemini. An application can provide the recording and question, allow the model to decide which evidence to retrieve, and inspect the returned processing steps to confirm that agentic navigation was used.

It also changes cost estimation. Static processing ties input size more directly to video duration and the chosen sampling configuration. Agentic processing can make a targeted query substantially cheaper, but its token use varies with what the model decides to load. Production systems will need to measure usage by query class rather than relying only on video length.

The current support matrix covers Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite, according to the updated developer guide. Static mode remains the default, so existing integrations do not automatically switch processing behavior.

The API release is the first deployment stage. Google says agentic video understanding will subsequently reach Flash and Flash-Lite models in the consumer Gemini app. The company also plans to use it for YouTube’s “Ask YouTube” feature in the coming months, with answers grounded more closely in a video’s visual content. Google did not provide exact rollout dates for either consumer integration.

Frequently Asked Questions

What is agentic video understanding?

It is a Gemini processing mode that dynamically selects video segments, frames, audio, and transcript passages based on the user’s question instead of processing the entire recording at a fixed frame rate.

Which Gemini models support it?

Google’s current documentation lists Gemini 3.8 Flash, Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite. The original September 1 launch covered the latter three models.

Does it always reduce token use by 88%?

No. Google says “up to” 88%. Actual consumption depends on the video, query complexity, and how extensively the model resamples relevant segments.

Is agentic processing more expensive to enable?

There is no separate feature fee. Standard Gemini API token pricing applies, although the number of tokens consumed varies by request.

Should developers use it for short videos?

Not necessarily. Google recommends static processing for latency-sensitive clips under five minutes and tasks requiring uniform frame-level inspection of the whole video.

Sources

• Original announcement by Logan Kilpatrick on X
• Google: Introducing agentic video understanding with Gemini
• Gemini API video-understanding documentation
• Gemini API release notes
• Gemini Developer API pricing
• Android Authority: Gemini now analyzes your videos more accurately, and at a lower cost