Guetzli: A New Open-Source JPEG Encoder(research.googleblog.com) |
Guetzli: A New Open-Source JPEG Encoder(research.googleblog.com) |
echo Drag and drop one or multiple jpg or png files onto this batch file, to compress with google guetzli using a psychovisual model
if [%1]==[] goto :eof
:loop
echo compressing...
guetzli_windows_x86-64.exe -quality 84 %1 "%~dpn1_guetzlicompressed%~x1"
shift
if not [%1]==[] goto loop
echo DONE
Looking at the code to that, it looks like 1500 lines of this:
double MaskDcB(double delta) {
PROFILER_FUNC;
static const double extmul = 0.349376011816;
static const double extoff = -0.894711072781;
static const double offset = 0.901647926679;
static const double scaler = 0.380086095024;
static const double mul = 18.0373825149;
static const std::array<double, 512> lut =
MakeMask(extmul, extoff, mul, offset, scaler);
return InterpolateClampNegative(lut.data(), lut.size(), delta);
}
A constant lookup table is used for determining the importance of a change vs distance. Seperate tables are used for vertical and horizontal distances (I guess eyes might be slightly more sensitive to vertical edges than horizontal ones?).
Those tables are wildly different in magnitude:
static const double off = 1.4103373714040413; // First value of Y lookup table
static const double off = 11.38708334481672; // First value of X lookup table
static const double xmul = 0.758304045695;
static const double ymul = 2.28148649801;
http://disp.ee.ntu.edu.tw/meeting/%E7%B6%AD%E6%AF%85/An%20In...
and also read here:
https://en.wikipedia.org/wiki/YUV
That is my quick guess on how to roughly derive the constants (because it is new, probably there are some fancy modifications tho :) )
I don't know if this code is related to that but just pointing out that seemingly nonsensical constants may appear more than one would thing.
Careful what you wish for!
What are you on these days ? image codecs still ?
The original libjpeg code was written to try and change the Usenet News binary pictures groups over from GIF to JPEG (so that more images would fit down the rather narrow transatlantic pipe that I had at the time). The choice of license turned out to be a good one (it predated the GPL V2) -- who knows what would have happened if we (the precursor to the IJG) had chosen that one.
I believe using a full blown FFT and complex IQA metrics is too much. I have great results with custom quantization matrices, Mozjpeg trellis quantization, and a modification of PSNR-HVS-M, and there's still a lot of room for improvement.
...and generates a solution that uses far less bandwidth, especially after thousands or millions of hits, which is the real point of the exercise.
Cloud computing companies love this. They've got a lot of bored hardware to put to use. It's absolutely no surprise to see solutions like this coming from Google. Spending a dollar of compute time to save $1000 in bandwidth is a no-brainer win for a company with a million servers.
Specifically looking at the cat's eye example, in the bottom of the pupil area there's a bit of green (reflection?) in the lower pupil. In the original it is #293623 (green) - in the libjpeg it is #2E3230 (still green, slightly muted). But in the Guetzil encoded image it is #362C35 - still slightly green but quite close to grey.
In my experience people love to see colors "pop" in photos (and photography is where JPEG excels) - hopefully this is just an outlier and the majority of compressions with this tool don't lose color like this.
I suspect that if you give this algorithm twice the file size as a budget, that green color will come back.
It would definitely have its uses as such. Or maybe it's great all around and I just found one bad example?
TLDR:
> We didn't do a full human rater study between guetzli and mozjpeg, but a few samples indicated that mozjpeg is closer to libjpeg than guetzli in human viewing.
I can see that if you try to render an incomplete file you might end up "wasting" effort blitting it to the screen and stuff before the rest of the data is decoded. But if thats a concern, one can simply rearrange the data back to scanline order and decode as normal?
I'm still hoping for a Google Now that understands Swiss German :)
(Full disclosure: I am a programmer and I try to match programmers with Zurich's startups for a living.)
So, if you want to move to Zurich, you find my e-mail address in my HN-handle. Read more about Switzerland in my semi-famous blogpost "8 reasons why I moved to Switzerland to work in tech": https://medium.com/@iwaninzurich/eight-reasons-why-i-moved-t...)
Someone out there must have tried this.
[1]:https://blogs.dropbox.com/tech/2016/07/lepton-image-compress...
https://agileblaze.com/google-guetzli-image-compression-setu...
But I get a gflags linking error with 2.1 and 2.2 with -DGFLAGS_NAMESPACE=google. This is atrocious. "google::SetUsageMessage(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)" Without this it works fine.
Guess I still have some old incompat gflags headers around.
EDIT: The fix on macports with 2.2 installed default into /usr/local is: make CXX=g++-mp-6 verbose=0 LDFLAGS="-L/usr/local/lib -L/opt/local/lib -v" CFLAGS=-I/usr/local/include
i.e. enforce gflags 2.2 over the system 2.1
The real problem, as far as I can see, is that JPEG2000 is really slow to decode due to its complexity.
The blog makes it sound like that's the target but the paper has this line:
"Our results are only valid for high-bitrate compression, which is useful for long-term photo storage."
Do the author's think the size/quality benefits still show up when targetting lower bitrates/qualities that are more common on the web? Do they intend to try to prove it?
Another limitation is that Guetzli runs very slowly. This gives a further limiting axis: Guetzli in its current form cannot be applied to a huge corpus of images. Perhaps this covers half of the images on the internet.
So, let's say that Guetzli is 25% relevant to the web pages.
1) YUV420 vs YUV444. Guetzli practically always goes for YUV444.
2) Choosing quantization matrices.
3) After normal quantization, choose even more zeros. JPEG encodes zeros very efficiently.
When doing the above, increase the errors where it matters least (certain RGB values hide errors in certain components, and certain types of visual noise hides other kind of noise).
On top of that, you could tweak the quantized values themselves to make them more compressible.
The more bits you lose during quantization, the more ringing and artifacts you can expect after the IDCT process.
So the tradeoff is quite literally artifacts for smaller size.
this compressor seems to be cleverer about where to lose data than libjpeg.
https://github.com/google/guetzli/releases
and I'm getting "Invalid input JPEG file" from a lot of images unfortunately.
https://github.com/google/guetzli/issues/40#issuecomment-287...
No directly-linkable PDF :P
http://manpages.ubuntu.com/manpages/xenial/man1/netpbm.1.htm...
http://www.streamingmedia.com/Articles/Editorial/-110383.asp...
But I'm still skeptical because HEVC might be more widespread with hardware decoders everywhere and people might just not care enough to move to the new standard. Unless MPEG-LA exploits its dominance really bad with license fees, then we can expect MPEG codecs to die off. Although I think x264 will still live.
[1]: https://fosdem.org/2017/schedule/event/om_av1/attachments/sl...
(4th slide. This codec will be standardized after the experiments are removed and it is frozen to test software.)
As an example of the latter:
I think Opera Mini (which I ran the development of for its first decade) still has somewhere around 150-200 million monthly unique users, down from a peak of 250M. Pretty much all of those users would be quite happy to receive this image quality improvement for free, I think. (Assuming the incremental encoding CPU cost isn't prohibitive for the server farms.) Opera Mini was a "launch user" of webp (smartphone clients only) for this particular reason.
Many of those users devices are Nokia/Sony Ericsson/etc J2ME devices with no realistic way of ever getting system-level software updates. They are still running some circa 2004 libjpeg version to actually decode the images. It's still safe because the transcoding step in Opera Mini means that they aren't exposed to modern bitstream-level JPEG exploits from current web, but it underscores why any improvements targetting formats like JPEG is still quite useful.
Opera Mini for J2ME actually includes a very tiny (like 10k iirc) Java-based JPEG decoder since quite a few devices back then didn't support JPEG decoding inside the J2ME environment. It's better than having to use PNG for everything, but because it's typically like 5x-10x slower than the native/C version even in a great JVM of the time it really only makes sense to use as a fallback.)
There is no way we will start paying royalties to show images on the web.
(thanks for the answer btw)
edit: second read-through I found the paper [1] which explains it. The answer is basically "yes", where the large scale distortion function is basically activity masking. Normally this would be implemented with delta-QPs, but because JPEG doesn't have that, Guetzli uses runs of zeroes instead.
Butteraugli uses a 8x8 FFT, but computes this every 3x3 pixel creating coverage at block boundaries. In later stages of butteraugli calculation values are aggregated from an even larger area. Block boundary artefacts are taken into account by this and impact quantization decisions.
Butteraugli operates neither in RGB nor YUV. It has a new color space that is a hybrid of tri-chromatic colors and opponent colors. Black-to-yellow and red-to-green are opponent, but blue is modeled closer to tri-chromatic. In more simple explanation it is possible to think of it as follows: first apply inverse gamma correction, second apply a 3x4 transform for rgb, third apply gamma correction, fourth calculate r - g, r + g and keep blue separate.
I actually prefer x264 encoded video even though it results in much larger file sizes. Although HEVC has lower bitrate for supposedly same quality, my needs aren't constrained enough where I have to go for HEVC.
I hope AV1 doesn't lower the quality aspect since it is focused on being a primarily streaming codec with most of the companies being streaming focused (Google, Netflix, Amazon and Vidyo) and them focusing on better compression rate.
If I recall correctly HEVC was also meant to be a streaming codec and I feel like that lead to the lower quality compared to h264. It just doesn't feel that sharp although it is supposed to be 1080p. The blurring aspect is especially bad.
I don't think AOMedia guys will blow it though. I feel like they have a lot more expertise since there are people from multiple codecs (VP9, Daala, etc.) contributing to this.
Most notably:
* The gamma value of 0.38 (when most studies suggest 1.5 - 2.5 for human eye gamma)
* The significant difference in the vertical and horizontal constants (when as far as I know human eyes are equally sensitive to most distortions independant of angle).
In this use the gamma is the inverse of something close to 2.6. Butteraugli needs both gamma correction and inverse gamma correction.
The FFT co-efficients only look weird, but they should actually lead to a symmetric result if our math is correct. In a future version we move away from the FFT, so I don't encourage anyone to actually debug that too much.
Do you have any idea when that will land in the open? Say, before 2018? Or maybe a little sooner?
This is even more the case with software. If this cost 20 or 20 million man hours is irrelevant in the long run because eventually it will be used enough to offset costs if it is as good as every one says. Of course short term cost hurt now, but sounds like you still had computers do much of the heavy lifting.
One question: as the top-level comment in this thread noted, this algorithm may be specialized to certain kinds of images.
Can you release info about the kind of images/datsets this approach will pathologically fail with? That would be really really awesome.
PDFs with JPEG2000 images contain a Postscript library to decode these images. That is the code that gets to show them.
It seems like that's where most of the magic lies. Also peculiarities of human vision are one of my oddball interests, after compression of course. :)