Neural networks have proven they can meaningfully improve video compression. The catch is that larger, more capable models can also demand significantly more processing power — a real constraint inside the codecs that run on billions of devices.

So, Dolby researchers asked a different question: what if the architecture itself could become smarter, without the computational budget growing at all?

The work focuses on the LOP3 neural loop filter being studied by the Joint Video Experts Team — a component that refines compressed video frames before they're displayed. The researchers replaced its sequential 1×3 and 3×1 convolution blocks with a parallel design that processes the same information more efficiently. This way the number of multiply-accumulate operations per pixel stays essentially unchanged. The computational cost doesn't increase. What changes is how that computation is organized.

We observed that BD-Rate improvements reached approximately 3% — including -3.01% for floating-point processing and -3.03% for 16-bit integer processing — without adding a single extra operation to the budget.

We also experimented on compression techniques. A slow, static landscape and a fast-moving sports sequence are fundamentally different problems — different amounts of motion, different spatial complexity, different demands on the encoder. Yet traditional bitrate ladders treat them with the same predetermined combinations of bitrate and resolution, regardless of what's on screen.

Per-title encoding was a meaningful step forward: test multiple encodes, find the best options for each specific piece of content. But those repeated encodes are expensive — in time, in cloud computing costs, and in the energy required to run them.

Dolby researchers developed a content-driven encoding model that sidesteps the problem entirely. Rather than encoding a clip multiple times to discover the right bitrate, the system predicts the complete quality curve directly from the source video — before a single encode takes place.

The model was trained on 5,336 two-second 1080p clips drawn from 16 titles, built around a SlowFast neural architecture that analyzes both spatial detail and motion simultaneously. At inference, it processes a two-second segment in approximately 0.27 seconds on a CPU, predicting video quality with an average error of around ±7 VMAF points.

This resulted in fewer trial encodes and lower cloud-computing costs. And bitrate decisions shaped by the actual content — not by a predetermined table that treats every video the same. However, challenges remain, particularly for complex, unpredictable motion like fire and smoke, where prediction errors can increase. But the direction is clear: the encoder of the future won't just compress video. It will understand it.

The other challenge was efficiency and quality.

Consider what happens when video moves through a complex pipeline — captured at high speed, converted to a different frame rate, processed for motion analysis, or reconstructed from an event-driven camera. At each step, the relationship between when a frame was originally captured and when it's eventually displayed can shift, blur, or be lost entirely.

For most consumer videos this might appear to be a minor concern, but for scientific imaging, machine vision, time-lapse, reverse video, and high-speed capture, it can make a stark difference.

The Source Picture Timing Information (SPTI) Supplemental Enhancement Information message addresses this directly — preserving the actual temporal distance between source pictures independently of when decoded pictures are output. It can also indicate whether a coded picture represents a genuine original frame or a synthetically generated one: an increasingly important distinction as AI-generated content enters professional workflows.

SPTI is being incorporated into the Versatile Supplemental Enhancement Information standard, with support for VVC, HEVC, and AVC — ensuring that timing information survives the full journey from capture to display, regardless of what happens in between.

Three decisions. One smarter pipeline.

These three research directions are each answering a different question that every video delivery system must ask:

How efficiently should we process each frame? How many bits does this content need? What timing information must survive the journey?

Today, those decisions are often made independently. The opportunity ahead is to connect them — combining perceptual quality, content understanding, efficient neural processing, and richer metadata into a pipeline that makes all three decisions together and makes them well.

For Dolby, that means looking beyond compression as a single algorithmic challenge. The bigger ambition is building video systems that genuinely understand where computation adds value, where bandwidth matters most, and what information must be preserved to deliver the experience as it was intended.

Smarter Video, Fewer Bits: Rethinking How We Encode and Deliver Video

{{ pageEducationalSlot.headingContent[index] }}