Compression

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
m (Transparent file compression)
(31 intermediate revisions by 4 users not shown)
Line 9: Line 9:
 
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive.
 
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive.
  
== Compression in general ==
+
== Compression algorithms and compressed data formats ==
  
(including specific implementations used in multiple formats)
+
(excluding formats usually used as file formats)
  
 
* [[Arithmetic coding]]
 
* [[Arithmetic coding]]
Line 17: Line 17:
 
* [[Burrows–Wheeler transform]]
 
* [[Burrows–Wheeler transform]]
 
* [[DEFLATE]]
 
* [[DEFLATE]]
 +
* [[DivANS]]
 
* [[Huffman coding]]
 
* [[Huffman coding]]
 
* [[LZ4]]
 
* [[LZ4]]
 
* [[LZ77]]
 
* [[LZ77]]
 
* [[LZ78]]
 
* [[LZ78]]
 +
* [[LZFSE]]
 
* [[LZHAM]]
 
* [[LZHAM]]
 
* [[LZMA]]
 
* [[LZMA]]
Line 27: Line 29:
 
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski)
 
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski)
 
* [[LZW]]
 
* [[LZW]]
 +
* [[MSZIP]]
 +
* [[PKWARE DCL Implode]]
 
* [[PPMd]]
 
* [[PPMd]]
 
* [[Run-length encoding]]
 
* [[Run-length encoding]]
 +
** [[RLE90]]
 
* [[Shannon–Fano coding]]
 
* [[Shannon–Fano coding]]
 
* [[Snappy]] (Google-created compression also used by Apple in iWork formats)
 
* [[Snappy]] (Google-created compression also used by Apple in iWork formats)
 
* [[zlib]]
 
* [[zlib]]
 
* [[Zstandard]] (Zstd)
 
* [[Zstandard]] (Zstd)
 +
** [[Zstandard dictionary]]
  
== Graphics compression ==
+
=== Graphics compression ===
 
See [[Graphics#Compression]] for compression formats used primarily with graphics.
 
See [[Graphics#Compression]] for compression formats used primarily with graphics.
  
Line 41: Line 47:
 
(multi-file compressors are in [[Archiving]])
 
(multi-file compressors are in [[Archiving]])
  
* [[7z]]
 
 
* [[9CDR]] (Amiga FileImploder Clone)
 
* [[9CDR]] (Amiga FileImploder Clone)
 
* [[BARF]] (.x, .x??)
 
* [[BARF]] (.x, .x??)
 +
* [[BWTC32Key]] (.B3K)
 
* [[bzip]] (.bz)
 
* [[bzip]] (.bz)
 
* [[bzip2]] (.bz2)
 
* [[bzip2]] (.bz2)
 +
* [[Compact (Unix)|compact]] (.C)
 
* [[compress]] (.Z)
 
* [[compress]] (.Z)
 
* [[CrLZH]] (.?y?)
 
* [[CrLZH]] (.?y?)
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
 
* [[Crunch]] (.?z?)
 
* [[Crunch]] (.?z?)
* [[DiskDoubler]]
 
 
* [[DUPA]] (Amiga FileImploder Clone)
 
* [[DUPA]] (Amiga FileImploder Clone)
 
* [[Error Code Modeler]] (.ecm)
 
* [[Error Code Modeler]] (.ecm)
Line 56: Line 62:
 
* [[Freeze/Melt]] (Unix) (.F)
 
* [[Freeze/Melt]] (Unix) (.F)
 
* [[gzip]] (.gz)
 
* [[gzip]] (.gz)
 +
* [[Hammer (compression)|Hammer]]
 +
* [[ICE (Michael Lamanuzzi)|ICE]] (.ice, by Michael Lamanuzzi)
 
* [[Inflate]] (.infl) - parody format that actually increases size
 
* [[Inflate]] (.infl) - parody format that actually increases size
 +
* [[JAM (W. Jiang)]]
 
* [[JCalG1]] (.jc, Commodore Amiga)
 
* [[JCalG1]] (.jc, Commodore Amiga)
 
* [[lrzip]] (.lrz)
 
* [[lrzip]] (.lrz)
Line 63: Line 72:
 
* [[lzop]] (.lzo)
 
* [[lzop]] (.lzo)
 
* [[LZX]]
 
* [[LZX]]
 +
* [[Mozilla LZ4]] (.jsonlz4)
 
* [[MS-DOS installation compression]] (.??_)
 
* [[MS-DOS installation compression]] (.??_)
 
* [[Oodle Data Compression]]
 
* [[Oodle Data Compression]]
 
* [[pack]] (.z)
 
* [[pack]] (.z)
* [[Pack-Ice]]
+
* [[PowerPacker]] (.pp)
 +
* [[Pro-Pack]]
 +
* [[RAC]]
 
* [[RK]] (WinRK)
 
* [[RK]] (WinRK)
 
* [[rzip]] (.rz)
 
* [[rzip]] (.rz)
 +
* [[SCO compress LZH]]
 
* [[SHARC]] (density)
 
* [[SHARC]] (density)
 
* [[Softdisk Text Compressor]] (.ctx)
 
* [[Softdisk Text Compressor]] (.ctx)
Line 76: Line 89:
 
* [[tzip]]
 
* [[tzip]]
 
* [[Tzip (Text ZIP)]]
 
* [[Tzip (Text ZIP)]]
 +
* [[WWPACK]]
 
* [[XZ]] (.xz)
 
* [[XZ]] (.xz)
 +
* [[Yaz0]]
 +
 +
== Transparent file compression ==
 +
(An obsolete technique in which a background process hooks into the OS and decompresses files transparently when they are read. ''Compression'' is usually done manually. The line between this and ''Disk compression'' can be blurry.)
 +
 +
* [[CFS (Computer Concepts Compression)]]
 +
* [[DIET (compression)|DIET]]
 +
* [[Pack-Ice]]
 +
* [[SLIM (Dominic Herity)|SLIM]]
 +
* [[XPACK]]
 +
 +
== Disk compression ==
 +
(Transparent disk compression software and the like, excluding filesystem formats.)
 +
 +
* [[DiskDoubler]]
 +
* [[DriveSpace]] and DoubleSpace
 +
* [[JAM (disk compression)]]
 +
* [[Stacker]]
 +
 +
For filesystem formats that feature compression, see [[Filesystem]].
 +
 +
== Executable compression ==
 +
 +
See [[Executable compression]].
  
 
== See also ==
 
== See also ==
  
 +
* [[Compression research]]
 
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression)
 
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression)
 
* [[Graphics#Compression|Graphics]] (for image-specific compression)
 
* [[Graphics#Compression|Graphics]] (for image-specific compression)

Revision as of 14:01, 10 February 2020

File Format
Name Compression
Ontology

{{{caption}}}

Contents

Stream compression formats

A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like tar can be combined with an archive format to produce a proper compressed archive.

Compression algorithms and compressed data formats

(excluding formats usually used as file formats)

Graphics compression

See Graphics#Compression for compression formats used primarily with graphics.

Specific file formats/programs

(multi-file compressors are in Archiving)

Transparent file compression

(An obsolete technique in which a background process hooks into the OS and decompresses files transparently when they are read. Compression is usually done manually. The line between this and Disk compression can be blurry.)

Disk compression

(Transparent disk compression software and the like, excluding filesystem formats.)

For filesystem formats that feature compression, see Filesystem.

Executable compression

See Executable compression.

See also

Links

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox