← Notes

Notes

Interactive

Compression Laboratory

13 min read computer-visioncompressioninteractive

JPEG is thirty years old and still the default way a photograph moves around the world. It survives because it makes one very good bet: that the eye notices a smeared edge far less than it notices a shifted colour, and notices fine detail far less than it notices overall brightness. Everything else in the format is bookkeeping around that bet.

This page takes the format apart. The first bay is the one that makes the rest make sense: a single 8×8 block, followed all the way from pixels to coefficients to quantised integers and back, with the error it accumulated. Every number is computed here, in this tab, from the image you are looking at.

Compression bench

Eight by eight, and everything that follows

08 bays
live · client-side
F(u,v) the DCT coefficient — how much of one basis pattern is in this block
Q(u,v) the quantisation step for that frequency — the whole perceptual model
F̂ a small integer, and usually zero. This rounding is the loss
round the only irreversible step in the entire format
[ 01 ]

One 8×8 block, end to end

live

Click anywhere in the thumbnail to choose a block. The strip below follows it through the entire pipeline: the original luma samples, the DCT coefficients they produce, those coefficients divided by the quantisation table and rounded, the reconstruction after multiplying back and inverting the transform, and finally the error. Everything JPEG throws away is thrown away in the middle panel.

Quantisation

quality
flat step override

The flat override replaces the standard table with one step for every frequency. Turn it up and compare: the standard table is not uniform, and the shape of that non-uniformity is the entire perceptual model.

coefficients kept—
block RMSE—
DC value—
DC step—
entropy cost of this block—bits

Imageor drop your own

green = positive, blue = negative, dot = exactly zeroluma channel only

  • Pick a block in flat sky. Nearly every coefficient rounds to zero and only DC survives — a whole 64-pixel block stored in a handful of bits.
  • Now pick a block on a hard edge. The coefficients spread across the whole grid, few of them vanish, and the error panel lights up along the edge. Detail is expensive and edges are the most expensive detail there is.
  • Drag quality down and watch the third panel empty out from the bottom-right corner. High frequencies go first, because the table quantises them hardest.
Takeaway The DCT itself is lossless — it is an orthonormal change of basis, and you can invert it exactly. All of JPEG's loss happens in one round(). Everything before it is preparation to make that rounding cheap, and everything after it is bookkeeping to store the result compactly.
[ 02 ]

Sixty-four patterns, and nothing else

live

Every 8×8 block that has ever existed is a weighted sum of the 64 patterns on the left. That is not an approximation — they form a complete orthonormal basis, so the DCT just measures how much of each one is present. Frequency rises to the right and downward, so the top-left pattern is flat (DC) and the bottom-right alternates every pixel.

the 64 basis patterns
rebuilt from the first N

Keepzigzag order

coefficients

Coefficients are taken in the zigzag scan order, which walks outward from DC. The greyed-out patterns on the left are the ones currently discarded.

of the block kept—
reconstruction RMSE—

Imageor drop your own

  • Start at 1. One coefficient gives you the block's average brightness and nothing else — that is DC, and it is why heavily compressed images look like flat tiles.
  • Move to 6. On most photographic blocks the reconstruction is already close, from under a tenth of the data. That gap between 6/64 and looking right is the whole reason the format works.
  • Push to 64. RMSE goes to zero exactly, because the basis is complete and nothing has been quantised yet.
Takeaway The DCT does not compress anything. It rearranges the block so that most of the energy lands in a few coefficients, which makes the subsequent quantisation and entropy coding cheap. Energy compaction is a property of natural images, not of the transform — feed it noise and the coefficients stay stubbornly spread out, which is exactly why noise does not compress.
[ 03 ]

The whole codec, with every dial

live

The full pipeline, and the price of every decision. The difference image on the right is |original − result|, amplified — it shows you precisely where the bits went and where they did not. Watch it concentrate on edges and texture while flat regions stay black.

reconstruction
difference, amplified

Quantisation

quality

Chroma

Colour space

Coding RGB directly costs far more for the same quality: the three channels are strongly correlated, so you pay for the same structure three times over, and you cannot subsample any of them.

Before encoding

resolution
bit depth
difference gain

Imageor drop your own

compressed—
uncompressed—
ratio—
bits / pixel—
PSNR—
SSIM—

size is the entropy of the coded symbol stream — see the note at the endmetrics computed on luma

  • Switch colour space to RGB at the same quality. The file gets substantially bigger for a result that is no better — that comparison is the reason JPEG converts to YCbCr at all.
  • At quality 40, drag 4:4:4 → 4:2:0. On a photograph the size drops noticeably and PSNR barely moves. Now load the Chroma target and do it again.
  • Pull resolution to 50% and raise quality to compensate. Sometimes that is the better trade — bay 07 makes the comparison properly.
Takeaway Quality is not a percentage of anything. It is an index into a formula that scales one fixed table, which is why "quality 80" from two different encoders can differ by a factor of two in size. The only numbers that transfer between tools are bits per pixel and a distortion measure.
[ 04 ]

Throwing away colour that nobody sees

live

The retina has far fewer chroma-sensitive receptors than luminance-sensitive ones, and the visual system low-passes colour aggressively. So JPEG stores colour at half resolution, or a quarter, and almost nobody notices — until the image is built from pure chroma edges, which is exactly what the Chroma target is. Then the failure is impossible to miss.

original
subsampled

Mode

Show

samples stored—
samples saved—
luma PSNR—
chroma RMSE—

Imageor drop your own

  • Note the luma PSNR stays at infinity in every mode. Subsampling does not touch luminance at all — it is a pure chroma operation, and that is the whole trick.
  • Load Photo and switch to 4:1:1. You will struggle to see a difference at a third of the colour data. Now switch back to Chroma: the red and blue bars smear into mush.
  • Choose Chroma only to see the colour planes on their own. That blur is present in every JPEG you have ever looked at.
Takeaway This is the only stage that exploits human vision directly rather than image statistics — and it is why saturated red text on a dark background looks terrible as a JPEG while the same text in black looks fine. If your image is a screenshot, a diagram, or anything with coloured type, 4:2:0 is actively working against you.
[ 05 ]

Blocking, ringing, and where they come from

live

Drag on the left image to move the loupe. The right panel shows the original above and the compressed version below, at the same crop, with the 8×8 lattice drawn over the lower half. Every artefact you can see has a cause you can point at: blocking is neighbouring blocks disagreeing about their DC value, and ringing is a sharp edge being rebuilt from too few sinusoids.

drag to move the loupe
original / compressed

Compression

quality
magnification
PSNR—
SSIM—
bits / pixel—

Imageor drop your own

  • Park the loupe on the yellow diagonal in the Edges target. The speckle around it is mosquito noise — high-frequency coefficients that were quantised to the wrong value, smeared over the whole block.
  • Find a flat region and drop quality to 5. The lattice becomes visible as flat tiles with visible steps between them. That is DC quantisation, and nothing else.
  • Load the Step target, which has one vertical edge deliberately off the 8-pixel grid. Ringing appears on both sides of it and stops dead at the block boundary — artefacts cannot cross blocks, because blocks are coded independently.
Takeaway JPEG's artefacts are structured, not random, and the structure is always the 8×8 grid. That is what makes them so recognisable — and it is why every later codec (JPEG 2000, WebP, AVIF) spends real effort on either larger, variable-sized blocks or an explicit deblocking filter.
[ 06 ]

The curve every codec is judged on

live

Compression has exactly two axes: how many bits you spend, and how wrong the result is. This sweeps quality from 5 to 95 and plots the pair. The shape is the important part — it is steeply diminishing. Doubling the bitrate at the right-hand end buys almost nothing, which is why saving photographs at quality 100 is nearly always wasted space.

rate vs distortion

Distortion measure

The two disagree, and the disagreement is the point. PSNR counts squared error and is blind to where it lands; SSIM compares local structure and punishes blocking far more harshly.

Marker

quality
sweep—
our estimate at this quality—
browser's real JPEG—
difference—

Imageor drop your own

4:2:0 4:4:4 current quality
  • The 4:2:0 curve sits above and to the left of 4:4:4 — better quality for fewer bits. A curve that dominates another like this is the only honest way to say one codec beats another.
  • Switch to SSIM. The knee moves: SSIM keeps improving where PSNR has already flattened, because it is still registering structural damage that squared error has stopped counting.
  • Load the Gradient target. Its curve is almost vertical — a smooth image is nearly free to encode, and adding bits does essentially nothing.
Takeaway A single quality number tells you nothing without the image it was measured on. The same encoder at the same setting produces 0.2 bits per pixel on a gradient and 4 on noise. Compare codecs by overlaying whole curves on a corpus, never by comparing two files.
[ 07 ]

Fewer pixels, or worse pixels?

live

Given a fixed bit budget, you can spend it on full resolution at low quality, or on half the pixels at a much higher quality each. Both panels below are encoded to the same bits per pixel; the quality setting for each is solved for by bisection. Which one wins depends entirely on the image and on where you set the budget.

full resolution
half resolution, upscaled

Budget

bits per pixel

Full resolution

quality—
bpp—
PSNR—
SSIM—

Half resolution

quality—
bpp—
PSNR—
SSIM—
better SSIM at this budget—

Imageor drop your own

  • Start at a very low budget (0.1 bpp). Downscaling usually wins: at that rate, full resolution is being destroyed block by block, while half resolution still has enough bits per remaining pixel to look coherent.
  • Raise the budget past about 0.6 bpp. Full resolution takes over — there are now enough bits to encode the real detail, and throwing pixels away is pure loss.
  • Try the Edges target. Downscaling is punished hard, because the detail it discards is exactly the structure the image is made of.
Takeaway "Save at a lower quality" and "save fewer pixels" are the same decision viewed from different angles, and the crossover between them is real and image-dependent. This is why image CDNs resize before they compress, rather than serving one large file at a punishing quality setting.
[ 08 ]

How few levels can you get away with?

live

Bit depth is quantisation without any transform: just fewer available levels per channel. The failure mode is banding — visible steps where a smooth gradient runs out of levels to describe itself. The fix is counter-intuitive: add noise. Dithering scatters the rounding error so the eye averages it away, trading a visible artefact for an invisible one.

quantised
luma histogram

Depth

bits per channel
levels / channel—
total colours—
PSNR—
SSIM—
uncompressed size at this depth—

Imageor drop your own

  • At 4 bits on the Gradient target the bands are obvious, and the histogram collapses into isolated spikes — one per surviving level, with nothing in between.
  • Tick dither. The bands dissolve while PSNR gets slightly worse. That is the clearest demonstration on this page that PSNR is not the same thing as looking better.
  • Load Photo and drop to 4 bits without dithering. Photographic texture hides banding almost completely — the noise that was already there was doing the dithering for you.
Takeaway Banding is not caused by too few bits, it is caused by too few bits and too little noise. This is why cinema and HDR pipelines work in 10 or 12 bits and why raw sensor noise, usually an enemy, quietly saves you here. A perfectly clean 8-bit gradient bands; a slightly noisy one does not.

— end of bench —

What is actually running here

jpeg-sci.js implements the pipeline: the orthonormal 8×8 DCT and its inverse, the Annex K quantisation tables and libjpeg’s quality scaling, the zigzag scan, chroma subsampling, the JPEG symbol stream (DC differences, run/size AC pairs, ZRL and EOB), PSNR, and SSIM with the standard 11×11 Gaussian window. Its test suite runs 62 assertions — DCT round-trips, Parseval, basis orthonormality, quality-table monotonicity, and the bound that a dequantised coefficient lies within half a step of the original.

One honest caveat about the file sizes. This does not emit a real JPEG bitstream. It builds the exact symbol stream a real encoder would, then costs it at the empirical entropy of that stream plus the raw magnitude bits — the bound a well-fitted coder approaches, not the output of one specific set of Huffman tables.

Bay 06 shows the estimate next to a real file from the browser’s own encoder, and they differ substantially — the real file is several times larger on these small test images. That gap is not an error, and it is worth understanding:

  • Every JPEG carries roughly 600 bytes of headers before a single pixel: two quantisation tables, four Huffman tables, and the frame and scan headers. On a 192×128 image that is most of the file.
  • The browser uses the standard fixed Huffman tables, which are not fitted to your image; the estimate here is the entropy bound those tables only approach.
  • The browser maps its own quality slider to its own tables, and chooses its own subsampling. “Quality 50” in the two encoders is simply not the same setting — which is exactly the point bay 03 makes.

Use the estimate for comparing settings against each other, which is what it is good for. Do not read it as a prediction of what a real file will weigh.

One test caught a genuine misconception of mine rather than a bug. I had asserted that a coarse quantiser zeroes most coefficients, using a random block — and it does not, because energy compaction is a property of natural images, not of the transform. Noise spreads its energy evenly across all 64 frequencies and there is nothing to concentrate. The suite now asserts both halves of that: smooth blocks compact, noise does not.

Everything runs on your machine. Images you drop in are decoded in this tab and never leave it.

This is the third of these benches, after the Camera Geometry Playground and the Colour Science Playground; all three share the same instrument chrome.