Lossy vs Lossless Image Formats, Explained Step by Step

Following a 1920 x 1080 picture through JPEG

The raw picture holds 1920 x 1080 = 2,073,600 pixels, each with three bytes, for 6,220,800 bytes. A JPEG encoder shrinks that in four moves, and only two of them lose anything.

1. Change the color model. Red-green-blue becomes one brightness layer (luma) and two color-difference layers (chroma). Nothing is lost here; the numbers are only re-expressed.
2. Subsample the color. Eyes resolve brightness detail much better than color detail, so the usual setting (4:2:0) keeps luma at full size but shrinks each chroma layer to half the width and half the height: 960 x 540 = 518,400 samples each. Total samples: 2,073,600 + 2 x 518,400 = 3,110,400, exactly half of the 6,220,800 you started with, before any real compression. This step is lossy.
3. Cut into 8 x 8 blocks and transform. The luma layer becomes 240 x 135 = 32,400 blocks; each chroma layer about 8,160 (540 does not divide by 8, so the last row is padded). Each block's 64 values are re-expressed as 64 strengths of pattern, from the plain block average up to the finest checkerboard. Still reversible, apart from tiny rounding.
4. Quantise. Each strength is divided by a number from a table and rounded to a whole number. This is the lossy heart of JPEG, and the quality setting is simply a choice of table.

A final lossless stage packs the resulting numbers, many now zeros, into compact code.

Quantisation with real numbers

I read the actual tables that Pillow wrote into the files. At quality 50 the table starts at 16 for the block average and ends at 99 for the finest-detail corner (its largest entry anywhere is 121). At quality 90 those same two positions hold 3 and 20; at quality 20, 40 and 248. The mean entry is 11.5 at quality 90, 57.6 at 50 and 141.5 at 20.

Suppose a block has a fine-detail strength of 37. At quality 50 the divisor for that position might be 99: 37 / 99 = 0.37, which rounds to 0. The detail is gone and costs nothing to store. At quality 90 the divisor might be 20: 37 / 20 = 1.85, which rounds to 2, and on decoding becomes 2 x 20 = 40, an error of 3 out of a possible range of hundreds. That arithmetic is an illustration with chosen numbers, but it is exactly the mechanism: higher quality means smaller divisors, so less rounding error and more bytes.

What the damage looks like

Two artifacts come straight from the steps above. Rounding fine-pattern strengths produces ringing, faint speckle around sharp edges, because an edge needs many fine patterns to describe. Cutting into blocks produces blocking, visible 8 x 8 squares in flat areas at low quality.

Chroma subsampling has a signature of its own. I drew a one-pixel pure red line (255, 0, 0) on a blue background and saved it as JPEG quality 90 with the default 4:2:0 setting. On the line itself the decoded pixel came back as (124, 28, 187), a purple that is neither red nor blue; the neighbor on one side was (27, 0, 216) and on the other (77, 0, 139). With subsampling off (4:4:4), the same pixel came back as (252, 1, 0), but the file grew from 4,105 to 6,506 bytes. That is why thin red text on blue, or logo edges between saturated colors, look muddy in JPEG while ordinary photographs look fine: natural scenes rarely hold one-pixel lines of pure opposing colors.

How lossless formats shrink without rounding

PNG guesses each pixel from its neighbors (to the left, above, or a mix of them) and stores only the difference from the guess; if the guess was right, the stored value is zero. A general-purpose packing method then squeezes the long runs of zeros and repeats. Nothing is rounded, so decoding reproduces the original exactly.

The result depends entirely on how guessable the picture is. My flat test picture (color blocks and text) went from 6,220,800 bytes to 135,263 as PNG, 46 times smaller. A picture of gradients with fine noise went to 3,587,826, only 1.7 times smaller, because noise is by definition not guessable. A lossless format cannot beat the randomness inside the picture; a lossy one can, by deciding the randomness does not matter. Those two pictures are the test images of this series: synthetic, 1920 x 1080, with real photos landing somewhere between them.

Lossless does not mean undamaged

Lossless describes the save, not the history. If a picture was damaged by an earlier JPEG save, a PNG made from it stores the damage faithfully. I encoded the flat test picture as JPEG at quality 85 (190,580 bytes) and converted the result to PNG: 482,158 bytes, 2.5 times larger and with exactly the same flaws. The same happens to the noisy picture, whose PNG from a JPEG was 1,993,234 bytes. This is the usual reason a converted file grows, covered with more cases in why a converted image can get bigger.

Likewise, lossy does not mean visibly worse. At a high setting a JPEG is within a few levels of the original almost everywhere. The practical question is not "lossy or lossless" but "will the loss be visible where this picture is shown", which is the subject of image quality after converting.

Three common misunderstandings

"Saving at 100 percent restores the quality." It does not. Quality 100 in JPEG still rounds (and, with Pillow's default settings, still subsamples color; encoders differ in when they do); it just rounds finely. In my test, Picture B at quality 100 took 1,534,583 bytes, 4.6 times the quality 90 file, and it was still not an exact copy. Even more important, it cannot restore detail an earlier save removed.

"Lossless is always the larger file." Usually for photographs, but not always: the flat test picture was 135,263 bytes as PNG and 219,783 as JPEG at quality 90. Content decides.

"Lossy formats are the low-quality formats." Newer lossy formats such as lossy WebP and AVIF use the same basic idea with better prediction and filtering, and at moderate settings the loss is below what viewers can see at normal size. The loss is real and measurable, which matters when you edit or reprint; it is not necessarily visible.

Picking a side in practice

• Photographs for viewing: lossy. The savings are tenfold or more, as measured in JPEG vs PNG.
• Screenshots, text, line art: lossless, for crisp edges and often a smaller file.
• Something you will edit and save repeatedly: lossless working copies, lossy only for the final export.
• Transparency required: a format that supports it, either lossless (PNG) or lossy (WebP).