Always cool to see new visual compression libraries hit the scene. That said I think the hardest part isn't the math of it, but the adoption of it.
Likely the format with the best chance of overthrowing the jpg/gif/png incumbents is AVIF. Since it's based on AV1, you'd get hardware acceleration for decoding/encoding once it starts becoming a standard, and browser support will be trivial to add once AV1 has wide support.
Compression wise AVIF is performing at about the same level as FLIF (15-25% better than webp, depending on the image), and is also royalty free. The leg it has upon FLIF is the Alliance for Open Media[1] is behind it, which is a consortium of companies including: "Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel Corporation, Microsoft, Mozilla, Netflix, Nvidia, Samsung Electronics and Tencent."
I'm really excited for it and I hope it actually gets traction. It'd be lovely to have photos / screenshots / gifs all able to share a common format.
>Likely the format with the best chance of overthrowing the jpg/gif/png incumbents is AVIF
I used to think the same as well, however I now think Jpeg XL is poised to be the 'winner' among next gen image codecs. It's royalty free, great lossy and lossless compression which is said to beat the competition, as well as providing a perfect upgrade path for existing jpeg's as it can losslessly recompress them into the jpeg XL format with a ~20% size decrease (courtesy of the PIK project).
It's slated for standardisation within a couple of weeks, it will be very interesting to see large-scale comparisons of this codec against the likes of AVIF and HEIF.
It may not be the best name, but it sure is better than the names of the projects it was based on: PIK and FUIF. At least if you speak Dutch. A "pik" is a penis and a "fuif" is a party, so the combination would be the "penis party" image codec. I prefer "JPEG XL".
The etymology of the name "XL" is as follows: JPEG has called all its new standards since j2k something that starts with an X: XR, XT, XS (S for speed, since it is very fast and ultra-low-latency), and now XL. The L is supposed to mean Long term, since the goal is to make something that can replace the legacy JPEG and last as long as it did.
There might be an explanation around it, but I find it not much better than Apple with their iPhone XS. I agree that XL is a bad name, and I don’t think that “pik fuif” would matter a lot (I’m also Dutch).
I hope that JPEG XL will be simpler than the competitors. If it's compression ratio is similar to AVIF and it can do HDR, then I'll all for it!
AVIF (and its image sequences) seems to be fairly complicated. Here's few comments [1] about it:
"Given all that I'm also beginning to understand why some folks want something simpler like webp2 :)"
"Despite authoring libavif (library, not the standard) and being a big fan of the AV1 codec, I do occasionally find HEIF/AVIF as a format to be a bit "it can do anything!", which is likely to lead to huge or fragmented/incomplete implementations."
Anyway, instead of adding FLIF, AVIF, BPG, and lots of similar image formats to web browsers, I think only one good format is enough and JPEG XR might be it. After something has been added to web browsers, it can't be removed.
Safari hasn't added support for WebP (which is good, there's no need for WebP after AVIF/JPEG XR is out) and it hasn't added support for HEIF (which is weird, considering Apple is using it on iOS), but maybe they know that there's no need to rush.
It is complementary to AVIF (which targets photo capture more than delivery over the web).
WebP2 is like WebP: keep it simple, with only useful and efficient features, no fancy stuff. And aim at very low bitrate, it's the current trend, favoring image size over pristine quality.
Progressive rendering uses ~3x more resources on client side. So, instead, better have efficient 300-bytes thumbnails in the header.
The example you linked to is pretty telling, because not only do the BPG images decode more slowly than natively supported images, the javascript decoding approach apparently breaks the browser's (Firefox's) color management. I think native support is needed for newer codecs to be viable for more than simple demos.
Chrome appears to interpret the canvas as sRGB and to convert from that, but that means that images decoded that way are effectively limited to sRGB until the canvas API allows specifying other colorspaces.
In Firefox, these canvas images appear stretched into the full gamut of the monitor (oversaturated), even though I have a color profile and have full color management enabled in about:config.
I'd by far rather see several full article-appropriately sized sideline image at only progression step 1 or 2 out of 5 or 6, than several empty boxes all showing spinners while the images load in.
Plus I'd love the ability to say "I'm on a low bandwidth, $$$ per megabyte network, stop loading any image after the first iteration until I indicate I want to see the full one" because you almost never need full-progression-loaded images. They just make things pretty. Having rudimentary images is often good enough to get the experience of the full article.
(whether that's news, or a tutorial, or even an educational resource. Load the text and images, and as someone who understands the text I'm reading, I can decide whether or not I need the rest of that image data after everything's already typeset and presentable, instead of having a DOM constantly reflow because it's loading in more and more images)
Progressive mode is better than a loading spinner in the same way that PWAs are better than a loading spinner: By getting mostly usable content in as little time as possible to the user you decrease perceived wait, you decrease time to interactive and you increase perceived loading speed (even though time to full load might be the same or slightly increased).
Progressive photos always irritate me. The photo comes on my screen all blurry and out of focus and I'm disappointed that the moment I thought I had captured didn't turn out. Then, 3 seconds later, the picture changes and gets slightly better. Then I'm hopeful, but disappointed again. Then i think, "maybe it's not loaded yet", so i wait and hope. Then 2 seconds later it changes again. Is it done? Is it loaded now? Will it get better? Is my computer just dog slow? How long should I wait before I assume it's as good as it's going to get?
I know it's a small thing and doesn't really matter, but I don't like progressive photos.
Edit: This is just one context. There are plenty of other contexts where progressive is very useful.
On the other hand, it also doesn't constantly change the DOM, moving you across the page because images above what you're reading are getting swapped in and now you're looking at the paragraph above the one you were reading.
The progressive mode in Jpeg and JPEG XL is quite different, because the quality is so much better your perception of it changes. Where Progressive Jpeg are literally useless before it finish loading, JPEG XL provides decent quality.
In the common case that you don't actually care about the picture, you decrease actual wait, not just perceived. Progressive mode lets you ignore it before it's fully loaded.
true progressive mode actually gives you something usable before it's fully loaded, to the point where "not fully loading" can be entirely acceptable. If all images load "first iteration" first, you have a stable DOM that won't need to reflow in any way as each image goes through the rest of its progressive layers. And having a nice setting that says "only load up to X on slow networks" and "require image click to load past first iteration" would be godsend, even on gigabit connections. If I could bring the size and load time for modern webpages back down to "maybe a few 10s of kb" and "immediate" rather than "2.9MB and a second or two", I'd happily turn those options on.
How so? Progressive JPEGs load in multiple passes. The first pass trades fidelity for speed, successive passes add quality over time. Seems pretty much in line with what PWA is all about.
Imagine this: you set your browser to download only the first N bytes of each image, showing you a decent preview. If you want more detail on a particular image, you tap it, and the browser requests the next N bytes (or the rest of the file, if you prefer).
And to enable this, the site only needed to create one high-resolution image.
Seems like a victory for loading speeds, for low-bandwidth, and for content creation.
Agreed, but is it likely? Does any browser implement the even simpler feature "Show a placeholder for images. Tap the placeholder to load the actual image"?
Systems that render low res can download the prefix of the file and show an image, and stop there. Many modern image formats support this for precisely this reason. If you then want higher quality for some particular image, you can download more, not having to start over with a separate image file.
The talk is by the FLIF author. One of the big marketing points for FLIF is its progressive mode. Of course every other codec will be criticized for not having one.
>Interesting. Where have you seen that adoption will be swifter with JPEG XL instead of, say, AV1/AVIF?
Well, it's not finalized as of yet (though it is imminent), so rate of adoption is just pure guesswork at this stage. However, things I deem necessary for a new image codec to become the next 'de facto' standard are:
royalty free
major improvements over the current de facto standards
Both AVIF and Jpeg XL tick these boxes, however Jpeg XL has another strong feature which is that it offers a lossless upgrade path for existing jpeg's with significantly improved compression as a bonus.
Yes, it losslessly recompresses existing jpeg's into the jpeg XL format, while also making the files ~20% smaller, key point being lossless. Thus it is the 'perfect' upgrade path from jpeg which is the current lossy standard, as you will get better compression and no loss in quality when shifting to Jpeg XL.
This being a 'killer feature' of course relies on Jpeg XL being very competitive with AVIF in terms of lossy/lossless compression overall.
I'm assuming this is bidirectional? You can go back from XL to jpeg losslessly as well? If thats the case, I'm having trouble imagining a scenario where you're not correct; it'd be an utterly painless upgrade path
>A lightweight lossless conversion process back to JPEG ensures compatibility with existing JPEG-only clients such as older generation phones and browsers. Thus it is easy to migrate to JPEG XL, because servers can store a single JPEG XL file to serve both JPEG and JPEG XL clients.
>You can go back from XL to jpeg losslessly as well
I don't think so, but I don't quite see the point unless you are thinking of using it as a way to archive jpeg's, but in that case there are programs specifically for that, like PackJPG, Lepton etc.
Decompressing and recompressing a zip gives a lossless copy of the actual data, but there's no way to reconstruct the same exact zip you started with.
The same thing can be done with image data. For something like jpeg you can keep the coefficients but store them in a more compact form.
For what it's worth JPEG XL claims that it's 'reversible', but I'm not sure if that means you get your original jpeg back, byte for byte, or you get an equivalent jpeg back.
> Decompressing and recompressing a zip gives a lossless copy of the actual data, but there's no way to reconstruct the same exact zip you started with.
I don't think getting the same JPEG is the goal here but getting a JPEG that decodes to the same image data.
Btw, if memory serves right, Google Photos deliberately gives you back the same pixels but not the same JPEG codestream bits under some circumstances. That's too make it harder to exploit vulnerabilities with carefully crafted files.
Is it possible to get a JPEG back without any loss when recompressing a JPEG to another JPEG (after discarding the original, ie JPEG -> bitmap -> JPEG)?
If you can go both directions, you can store the more efficient jpegXL format but still have perfectly transparent support for clients that don't support jpegXL.
If you can't produce the exact same original jpeg, then you can still have some issues during the global upgrade process -- eg your webserver database of image hashes for deduplication has to be reconstructed.
A relatively minor problem to be sure, but afaict if jpegXL does support this (which apparently it does), the upgrade process is really as pain-free as I could imagine. I can't really think of anything more you could ask for out of a new format. Better & backwards+forward compatibility
Sounds like it will just take an existing JPEG and reduce the size, but not re-compress it - so even though the original JPEG is lossy, there will be no additional loss introduced, whereas another format not based on JPEG would require a re-encode pass that would lose additional information.
Does it have the feature where the file can be truncated to any size to form its own thumbnail? That would be an incredibly useful feature in so many applications.
Some wavelet formats (SPIHT and EZW e.g) have the property that truncating the lossless version at 10% of the size gives you the 10:1 lossy rendering of that picture. That property holds not just for 10% but for any fraction of the file.
It's not quite the same thing, but a very highly compressed high-res picture would probably look okay in thumbnail format.
Good news: the reference software (http://gitlab.com/wg1/jpeg-xl) can decode about 50 Megapixels/s on a single Skylake core, with 3.1-3.7x speedup on 4 cores.
Encoding throughput is configurable from ~1 MP/s to about 50 per core. That's somewhat slower than JPEG, but can be faster with multiple cores.
> It'd be lovely to have photos / screenshots / gifs all able to share a common format.
Given the way GIF is used nowadays the only reason it exists is it let's you put a video on the most of the places where you are only allowed to put pictures. For some weird reason many of such websites still won't let you upload an MP4 file despite they auto-convert the gifs you upload to MP4/AVC. From the technical point of view there hardly is a reason for GIF to be used at all.
Objectively worse are infinite CSS-animations with opacity:0 set on them that Blink can't figure out are indeed invisible and don't need repainting at 60fps. Cue me noticing my CPU is at 100% and 69°C after about 2 minutes...
(ThinkPad T400s are really bad at thermal management.)
Exactly what you are saying: FLIF is not new (2015) but is still not adopted in any browser I know.
I also believe this was due to some license choices in the past.
But in this time of responsive websites it is a great format!
Just download the amount of pixels you need to form a sharp image and then skip loading the rest.
What it will take for something to overthrow JPEG is for both Google and Apple to adopt it. If Android, Chrome, iOS, macOS and Safari all support it (and the phones shoot in it natively like the iPhone does with HEIF right now), it will take over.
Think of it Google favoring sites with AVIF, JPEG XL, whatever they settle on in their page rank? Within half a year 80% of the web sites will have converted.
Worth noting that it's not really "new" as such, being over 4 years old. It still doesn't seem to have hugely caught on...
That said, lossless codecs have it easier because you can freely transcode between them as the mood takes you, with no generation loss, so there's less lock-in. For example, FLAC is dominant in audio, but there are a variety of other smaller lossless formats which still see use. Nobody much minds.
Would this really be lovely? Isn't there some value in having 3 distinct formats (still picture, moving picture, moving picture with audio)? Each one is a subset of the next, but they are 3 different things that can easily be identified by what they (only potentially, but probably) are.
It's a win from the implementation side because you can support all three use cases with one format, reducing code duplication and at least hypothetically leveraging the same improvements across the three use cases with no additional effort.
The UX concern you're describing doesn't necessarily have to have anything to do with the implementation details themselves, as demonstrated by sites like imgur rebranding silent mp4/webms as "gifv". As long as the new format makes it reasonably easy to access data about its content (e.g., header field indicating animation/movement), there shouldn't be any issue with collapsing the use cases into a single implementation and simultaneously addressing the UX concern you mention.
It’s faster (especially to encode), and in my opinion, better tuned for high quality. H.265 can give a “usable” image in a relatively small file size, but it sometimes takes a surprising amount of additional bytes to get rid of the somewhat oversmoothed aspect, and that additional amount can vary a lot from one image to the next, so with the current implementations, you have little choice but to verify visually. At least, that’s my experience with the reference encoder. I haven’t had a chance to experiment with Apple’s implementation.
In contrast, with JPEG XL, I can simply use the high quality setting once per image and be done with it, trusting that I will get a faithful image at a reasonable size.
Likely the format with the best chance of overthrowing the jpg/gif/png incumbents is AVIF. Since it's based on AV1, you'd get hardware acceleration for decoding/encoding once it starts becoming a standard, and browser support will be trivial to add once AV1 has wide support.
Compression wise AVIF is performing at about the same level as FLIF (15-25% better than webp, depending on the image), and is also royalty free. The leg it has upon FLIF is the Alliance for Open Media[1] is behind it, which is a consortium of companies including: "Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel Corporation, Microsoft, Mozilla, Netflix, Nvidia, Samsung Electronics and Tencent."
I'm really excited for it and I hope it actually gets traction. It'd be lovely to have photos / screenshots / gifs all able to share a common format.
[1] https://en.wikipedia.org/wiki/Alliance_for_Open_Media