Is AI Enabling Next-Generation Codecs? Three Years On

This article asked the question in March 2023 and could not answer it. Three and a half years later, it can.

In 2023, "AI codec" meant a research paper and a promise.

It now means a published international standard, an acquisition that looks well timed, and a set of techniques already running inside the apps you use.

Here is what actually happened.

JPEG AI Became a Real Standard

The most concrete answer is a format. JPEG AI is a learned image codec, meaning a neural network performs the compression itself rather than tuning a conventional one, and ISO and IEC published the standard in October 2024.

It is royalty-free, which matters more than the technology for anything hoping to reach the open web. Reported coding gains run to around 28.5% over VVC in all-intra mode.

That is the first time a learned codec has come through a standards body rather than a research group, and it is why the question this article originally asked now has a real answer rather than a hopeful one.

Apple Bought Its Way In, the Same Month This Was Written

In March 2023, Apple quietly acquired WaveOne, a neural video compression startup whose founders came out of Meta's AI research division.

WaveOne's published work claimed around 44% bitrate savings against H.264 on a standard test set, using a neural network in place of a conventional encoder. The whole team went to Apple.

Three years on, Apple Machine Learning Research has published its results. Apple's own figures claim a learned image codec delivering two to three times the bitrate saving of AV1, VVC and JPEG AI.

The genuinely interesting figure is speed rather than ratio. Apple reports encoding a 12-megapixel image in roughly 230 milliseconds on a recent iPhone, which is the barrier that made neural compression impractical in 2023.

What Is Already Running Today

The version of this that quietly shipped years ago is far less glamorous, and you have been watching its output all week.

Per-title and per-scene encoding uses machine learning to decide how much bitrate each shot deserves, rather than applying one setting across a whole film. A dark, static dialogue scene gets a fraction of what a fight sequence gets.

This runs on conventional codecs. There is no neural network in the decoder, no new format, and nothing for you to install - the intelligence sits in the encoding decision.

Netflix's VMAF is a large part of how the quality half of that gets measured, and unusually for this field it is free and downloadable.

Upscaling on the display end is the third strand. Televisions reconstructing detail locally means less has to be sent, which is compression by another route.

What Has Not Happened

No neural video codec has reached a consumer device as a delivery format. JPEG AI is images, and Apple's work is images.

Video remains harder by a wide margin, because motion between frames is where the compression happens and where the computational cost explodes.

Decoding is the other unsolved half. A codec is only useful if the receiving device can decode it cheaply, and neural decoders are expensive in exactly the way that flattens phone batteries.

Conventional codecs are not being replaced. HEVC and AV1 still carry essentially all delivery, and our look at where codecs are heading covers why that is unlikely to change quickly.

What It Means for You

Nothing immediate, which is the honest answer.

You cannot download a neural codec, no player supports one, and nothing you own will decode JPEG AI. The practical choice is still between the video codecs that exist today, and which codec to stream in compares the current four.

What has changed is the direction of travel. In 2023 this was a research curiosity; it is now a standard with a version number and a trillion-dollar company's acquisition behind it.

If you want the mechanics rather than the headlines, how codecs work explains what a codec actually does, and our piece on Apple's work on AI video compression goes further into that side.

For the stranger end of the field, hybrid neural avatars covers what happens when the thing being transmitted stops being video at all.

Quick questions

The 2023 question has an answer: yes for images, not yet for video, and already true in ways that never needed a new codec at all.

Check what your own machine decodes today with our Codec Checker, or tell us whether you think neural codecs land this decade.