JPEG

Revision as of 13:44, 28 April 2016

JPEG is a popular raster image format well-suited to photographic images. It usually uses lossy DCT compression. It is named after the Joint Photographic Experts Group, the organization which developed the format. It is sometimes called JPEG1, JPEG-1, or JPEG 1992 to help disambiguate it. (Also see JPEG 2000 and JP2.)

Terminology

(In which we discuss the tired old question of whether JPEG is a file format, or a compression standard)

In the original specification (ITU-T Rec. T.81), the term JPEG refers only to an organization. The serialized form of the compression format it defines is referred to only as the "interchange format".

The followup ITU-T Rec. T.84 document says that JPEG can also refer to a set of standards, including at least Rec. T.81, Rec. T.83, and Rec. T.84. It also informally uses JPEG to refer to the Rec. T.81 interchange format.

ITU-T Rec. T.851 refers to the interchange format as JPEG-1.

The interchange format is almost a full-fledged portable file format, lacking only standard conventions and/or standard metadata elements to tell how to interpret it as an image. The missing pieces can be added in various ways. Some file formats, including JFIF, do so in such a way that the interchange format is used directly as a file format. In practice, the term JPEG often refers to the family of such file formats, and this usage does not seem unreasonable, especially since there is no other suitable name. The term JIF (for JPEG Interchange Format) has been tossed around, but it is an ambiguous term that doesn't necessarily refer to a file format, and it's unusable in practice due to similarity to GIF.

It could be that JPEG is sometimes misused to mean JFIF, but such an accusation might be relying on the incorrect assumption that .jpg files always use JFIF format.

Portability

In and of itself, JPEG is not really a suitable format for image interchange, for several reasons:

• It essentially only defines a way to store one or more arrays of numbers. It does not say how to interpret those numbers as an image. The decoder will just have to guess.
• It is large and general, and it might be unrealistic to expect every decoder to support all of it.
• It defines no standard metadata elements.

To address these issues, a number of JPEG subformats have been invented. JFIF is by far the most popular of them, though a significant minority of JPEG files use Exif instead. The SPIFF file format was intended as a replacement for JFIF, but never caught on.

In practice, a portable JPEG file is pretty much "whatever the libjpeg software supports". This includes some varieties (such as RGB color) that do not conform to JFIF, and excludes some (such as lossless JPEG) that do.

Format

A JPEG file consists largely of a sequence of tagged segments. Each segment begins with a two-byte "marker". (The term "marker" is often used to refer to the entire segment.) The first byte of a marker is 0xff. The second may have any value except 0x00 or 0xff, and indicates the type of data stored in the segment. Segment types are assigned names; for example, 0xd9 is "SOI", and 0xe1 is "APP1".

Segment types 0x01, and 0xd0 through 0xd9, consist entirely of the two-byte marker. All other markers are followed by a two-byte integer indicating the size of the segment, followed by the payload data contained in the segment.

The image data is the exception to this segmented format. It appears following an "SOS" (0xda) segment, but there is no prefix to indicate its size. Instead, any 0xff bytes in the image data are escaped as 0xff 0x00, so that they won't be mistaken for markers. (Note that some JPEG-like formats, such as JPEG-LS and JPEG 2000 codestream, use different rules for escaping 0xff bytes.)

Application segments

There are 16 segment types reserved for application-specific data: 0xe0 ("APP0") through 0xef ("APP15").

An APP segment's data usually begins with NUL-terminated string to identify the type of data contained in it. The actual payload data then begins after the NUL byte. This convention was not part of the original JPEG specification, but was later standardized by ISO/IEC 10918-4:1999 (see ITU-T Rec. T.86).

Identifiers

Identification

JPEG files begin with bytes ff d8 ff.

However, this does not distinguish them from JPEG-LS. To do that reliably, one may have to scan the file to look for one of the valid SOF segment types.

Types of JPEG files

The following types are not disjoint. For example, a progressive JPEG may use arithmetic coding.

Some JPEG images do not belong to any of these types. An SOF1 segment is used if no other SOF segment applies.

Baseline

JPEGs with an SOF0 segment are known as Baseline JPEGs. They are always lossy, not progressive, use Huffman coding, and have a bit depth of 8. Every application that supports JPEG is supposed to at least support Baseline JPEG.

Progressive

Progressive JPEG rearranges the image data, so that the the first part of it represents a very low quality version of the entire image, rather than a high quality version of a small part of the image.

A progressive JPEG is identified by the presence of an SOF2, SOF6, SOF10, or SOF14 segment.

Non-interleaved

Color JPEG images may be either interleaved or non-interleaved.

In an interleaved JPEG, all the color components (e.g. Y, Cb, Cr) for a pixel are stored close together in the file.

In a non-interleaved JPEG, the image is separated into its color components, and each component is stored separately in the file.

Interleaving is not a simple yes/no option, because a single image may use both interleaved and non-interleaved scans (SOS segments) – in fact, progressive JPEGs usually do just that. Some JPEG decoders do not support non-interleaved images unless they use progressive encoding.

Arithmetic coding

Even lossy JPEG makes use of a lossless compression algorithm. The lossless algorithm is usually Huffman coding, but arithmetic coding may be used instead. JPEG's arithmetic coding usually results in a smaller file size, but it is not as widely supported as one might hope, probably because it used to be encumbered by patents.

An arithmetic-coded JPEG is identified by the presence of an SOF9, SOF10, SOF11, SOF13, SOF14, or SOF15 segment.

12-bit

Except for Baseline JPEG, all lossy types of JPEG may use a bit depth of either 8 or 12 bits per sample. However, few applications support anything other than 8 bits.

Starting with version 9a, IJG libjpeg also supports bit depths of 9, 10, and 11. These depths are nonstandard, and libjpeg's implementation is nearly unusable in practice, because it only supports a single bit depth, which must be selected at compile time.

Lossless JPEG

Main article: Lossless JPEG (original)

JPEG supports true lossless compression, but it is used so rarely that JPEG is commonly thought of as strictly a lossy format.

A lossless JPEG is identified by the presence of an SOF3, SOF7, SOF11, or SOF15 segment.

See also the Lossless JPEG disambiguation page, for other uses of the term.

Hierarchical

Also called "differential", hierarchical JPEG is vaguely similar to progressive JPEG, but geared toward storing multiple sizes of the same image, such that the decoder can select the size it prefers. Hierarchical JPEGs are, to a close approximation, nonexistent.

A hierarchical JPEG is identified by the presence of an SOF5, SOF6, SOF7, SOF13, SOF14, or SOF15 segment.

Color format

There are five JPEG color types that are reasonably portable:

1. Grayscale
2. YCbCr
3. RGB
4. YCCK
5. CMYK

To make the compression more effective, RGB images are almost always transformed to YCbCr format when they are written to a JPEG file. However, many applications will still report that such images use "RGB" color. This may be because their authors weren't aware of the transformation, or because they considered it to be an internal part of the compression algorithm as opposed to a different colorspace. Unfortunately, this inconsistent terminology can make it hard to distinguish YCbCr JPEGs from the rare JPEGs that really do use RGB color.

YCCK is a transformed version of CMYK, and the same terminology confusion exists as with YCbCr and RGB.

The JPEG format does not have any clear way to indicate the color type of an image. Decoders usually determine the color type based on several factors:

• The number of color components
• The presence of a JFIF application segment
• The "color transform" field of the APP14 "Adobe" segment, if present
• The component ID numbers ({82, 71, 66} suggests RGB)

SmartScale

Starting with version 7 or 8, the IJG libjpeg software has been adding nonstandard "SmartScale" scaling and color transform features of debatable merit^[1][2].

Notably, it is possible to achieve lossless DCT compression by setting the DCT block size to 1, and using RGB color if necessary.

Version 9 introduced a "reversible color transform" feature that can improve the compression of RGB images. Files with this feature contain a JPG8 (0xf8) segment.

Related Formats

• JFIF is a subformat and extension, and uses APP0 "JFIF" and APP0 "JFXX" segments.
• SPIFF is a subformat and extension, and uses an APP8 "SPIFF" segment.
• JPEG-HDR is an extension, and uses APP11 segments.
• JPEG XT is an extension.
• JPS is an extension, and uses an APP3 "_JPSJPS_" segment.
• Multi-Picture Format is an extension, and uses an APP2 "MPF" segment.
• The Exif standard uses an APP1 "Exif" segment.
• FlashPix data is contained in APP2 "FPXR" segments in Exif-compliant JPEGs. Refer to the Exif specification.
• Photoshop Image Resources is contained in an APP13 "Photoshop 3.0" segment.
• IPTC metadata often appears in JPEG files, embedded in Photoshop Image Resources.
• XMP metadata is contained in an APP1 "http://ns.adobe.com/xap/1.0/" segment.
• ICC profile data is contained in an APP2 "ICC_PROFILE" segment.
• Many other file formats, such as TIFF, MNG/JNG, and PDF, can contain JPEG-compressed data or encapsulated JPEG files.
• (And we heard you like JPEG...) JPEG files themselves often contain thumbnail images in the form of embedded JPEG files, via formats such as Exif, Photoshop Image Resources, or JFIF.

Formats that are not compatible with JPEG include JPEG-LS, JPEG 2000, and JPEG XR.

Specifications

• ITU-T Rec. T.81 (originally CCITT Rec. T.81): The JPEG standard
• ITU-T Rec. T.83: Compliance testing
• ITU-T Rec. T.84: Extensions
• ITU-T Rec. T.86: Registration of JPEG Profiles, etc.
• ITU-T Rec. T.851: (JPEG-1)-based still-image coding using an alternative arithmetic coder
• ITU-T Rec. T.872: Application to printing systems
• ExifTool: The APP14 "Adobe" segment
• ISO/IEC 10918: Digital compression and coding of continuous-tone still images
  • ISO/IEC 10918-1:1994 - Requirements and guidelines
    • ISO/IEC 10918-1:1994/Cor 1:2005 - Patent information update
  • ISO/IEC 10918-2:1995 - Compliance testing
  • ISO/IEC 10918-3:1997 - Extensions
    • ISO/IEC 10918-3:1997/Amd 1:1999 - Refer to SPIFF
  • ISO/IEC 10918-4:1999 - Registration of JPEG profiles, ...
    • ISO/IEC 10918-4:1999/Amd 1:2013 - Application specific marker list
  • ISO/IEC 10918-5:2013 - Refer to JFIF
  • ISO/IEC 10918-6:2013 - Application to printing systems

Metaformat files

• Synalysis grammar file (for Hexinator / Synalize It!; more details)

Software

• The Independent JPEG Group's software, commonly known as libjpeg or IJG libjpeg
• Konvertor
• libjpeg-turbo
• Thomas Richter's libjpeg: C++ library that supports all of the JPEG standard
• mozjpeg: Improved JPEG encoding library from Mozilla
  • Info on mozjpeg 3.0
• Bad Peggy: scans images for problems

Notes

When saving a JPEG image from Twitter in Google Chrome / Chromium, it will save with the default extension .jpg-large. This is due to a known bug (and one which shows no signs of being solved any time soon) in the way the browser sanitises Twitter image URLs (the filename is determined to be e.g. Xyzxyzxyz.jpg:large, which is sanitised to Xyzxyzxyz.jpg-large).

References

1. ↑ A Study on the Usefulness of DCT Scaling and SmartScale
2. ↑ IJG swings again, and misses

External links

• JPEG (Wikipedia)
• ExifTool JPEG Tags
• Malware Hidden Inside JPG EXIF Headers
• What Is a JPEG? The Invisible Object You See Every Day
• Project L0SS, a collection of images demonstrating the glitches and artifacts caused by repeatedly recompressing JPEG images
• Mozilla launches JPEG encoder mozjpeg to reduce webpage loads, results show up to 10% decrease in file size
• Improving JPEG image encoding
• We don’t need new image formats: Mozilla works to build a better JPEG
• Explanation of JPEG compression algorithm
• Lossless JPEG decompression
• Online demonstrator- shows quality differences between similarly-sized images in JPEG, JP2, WebP and BPG formats
• Method to improve JPEG compression that was the subject of a now-expired patent
• How JPG Works
• JPEG for the horseshoe crabs