Compression

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
(→‎Specific file formats/programs)
(Relocate LZ4 page to Stream compression formats section.)
 
(23 intermediate revisions by 3 users not shown)
Line 7: Line 7:
 
== Stream compression formats ==
 
== Stream compression formats ==
  
−
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive.
+
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] or the more archaic variant [[shar]] can be combined with an archive format to produce a proper compressed archive.
 +
 
 +
* [[gzip]] (.gz)
 +
* [[bzip2]] (.bz2)
 +
* [[LZ4]]
 +
* [[LZMA Alone|LZMA_Alone]] (.lzma)
 +
* [[pack (Unix)]] (.z)
 +
* [[XZ]] (.xz)
 +
* [[Zstandard]] (.zstd)
 +
** [[Zstandard dictionary]]
  
 
== Compression algorithms and compressed data formats ==
 
== Compression algorithms and compressed data formats ==
Line 19: Line 28:
 
* [[DivANS]]
 
* [[DivANS]]
 
* [[EXEPACK2]]
 
* [[EXEPACK2]]
 +
* [[FastLZ]]
 
* [[Fractal compression]]
 
* [[Fractal compression]]
 
* [[HPACK (HTTP)|HPACK]] (used to compress HTTP headers)
 
* [[HPACK (HTTP)|HPACK]] (used to compress HTTP headers)
Line 24: Line 34:
 
** [[Adaptive Huffman coding]]
 
** [[Adaptive Huffman coding]]
 
** [[Canonical Huffman code]]
 
** [[Canonical Huffman code]]
 +
* [[InstaCompOne]]
 
* [[Lempel–Ziv–Stac]] (LZS)
 
* [[Lempel–Ziv–Stac]] (LZS)
−
* [[LZ4]]
 
 
* [[LZ77]]
 
* [[LZ77]]
 
* [[LZ77 with Huffman coding]]
 
* [[LZ77 with Huffman coding]]
Line 35: Line 45:
 
* [[LZMW]]
 
* [[LZMW]]
 
* [[LZO]] (Lempel–Ziv–Oberhumer)
 
* [[LZO]] (Lempel–Ziv–Oberhumer)
 +
* [[LZRW]]
 
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski)
 
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski)
 
* [[LZW]]
 
* [[LZW]]
Line 49: Line 60:
 
* [[Snappy]] (Google-created compression also used by Apple in iWork formats)
 
* [[Snappy]] (Google-created compression also used by Apple in iWork formats)
 
* [[zlib]]
 
* [[zlib]]
−
* [[Zstandard]] (Zstd)
 
−
** [[Zstandard dictionary]]
 
  
 
=== Graphics compression ===
 
=== Graphics compression ===
Line 62: Line 71:
 
* [[ABC (Advanced Blocksorting Compressor)]]
 
* [[ABC (Advanced Blocksorting Compressor)]]
 
* [[ABComp]] (.abp)
 
* [[ABComp]] (.abp)
 +
* [[Aldus LZW]]
 +
* [[Aldus PKZP]]
 
* [[BARF]] (.x, .x??)
 
* [[BARF]] (.x, .x??)
 
* [[BGZF]]
 
* [[BGZF]]
 
* [[BWTC32Key]] (.B3K)
 
* [[BWTC32Key]] (.B3K)
 
* [[bzip]] (.bz)
 
* [[bzip]] (.bz)
−
* [[bzip2]] (.bz2)
 
 
* [[BZip3]] (.bz3)
 
* [[BZip3]] (.bz3)
 
* [[CAZIP]]
 
* [[CAZIP]]
Line 73: Line 83:
 
* [[compact (Unix)]] (.C)
 
* [[compact (Unix)]] (.C)
 
* [[compress (Unix)]] (.Z)
 
* [[compress (Unix)]] (.Z)
 +
* [[CMP (NEW-DOS)]]
 
* [[CrLZH]] (.?y?)
 
* [[CrLZH]] (.?y?)
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
Line 78: Line 89:
 
* [[Crunch-Mania]]
 
* [[Crunch-Mania]]
 
* [[DACT]]
 
* [[DACT]]
 +
* [[DitPack]]
 
* [[DUPA]] (Amiga FileImploder Clone)
 
* [[DUPA]] (Amiga FileImploder Clone)
 +
* [[EBZ|EBZip]] (.ebz)
 
* [[EDI Install archive]]
 
* [[EDI Install archive]]
 
* [[EDI Install packed file]]
 
* [[EDI Install packed file]]
 
* [[EDI LZSSLib]]
 
* [[EDI LZSSLib]]
 +
* [[EmmaCompress]]
 
* [[Error Code Modeler]] (.ecm)
 
* [[Error Code Modeler]] (.ecm)
 
* [[File Imploder]] (Amiga) (.imp)
 
* [[File Imploder]] (Amiga) (.imp)
 
* [[FlySpeed st/exp]]
 
* [[FlySpeed st/exp]]
 
* [[Freeze/Melt]] (Unix) (.F)
 
* [[Freeze/Melt]] (Unix) (.F)
−
* [[gzip]] (.gz)
 
 
* [[Hammer (compression)|Hammer]]
 
* [[Hammer (compression)|Hammer]]
−
* [[HAP]] v2.x
+
* [[HAP]] (v2.x)
 
* [[ICE (Michael Lamanuzzi)|ICE]] (.ice, by Michael Lamanuzzi)
 
* [[ICE (Michael Lamanuzzi)|ICE]] (.ice, by Michael Lamanuzzi)
 
* [[Inflate]] (.infl) - parody format that actually increases size
 
* [[Inflate]] (.infl) - parody format that actually increases size
 
* [[InstallIt!]]
 
* [[InstallIt!]]
 
* [[JAM (W. Jiang)]]
 
* [[JAM (W. Jiang)]]
−
* [[The JAM Packer]]
+
* [[The JAM Packer|JAM Packer, The]]
 
* [[JCalG1]] (.jc, Commodore Amiga)
 
* [[JCalG1]] (.jc, Commodore Amiga)
 
* [[Kboom11]]
 
* [[Kboom11]]
 +
* [[Logitech Compress]]
 
* [[lrzip]] (.lrz)
 
* [[lrzip]] (.lrz)
 +
* [[LZ5 and Lizard]]
 
* [[LZARI]]
 
* [[LZARI]]
 
* [[LZHUF]]
 
* [[LZHUF]]
 
* [[Lzip]] (.lz)
 
* [[Lzip]] (.lz)
−
* [[LZMA Alone|LZMA_Alone]] (.lzma)
 
 
* [[lzop]] (.lzo)
 
* [[lzop]] (.lzo)
 
* [[LZS221]]
 
* [[LZS221]]
Line 106: Line 120:
 
* [[LZWCOM]]
 
* [[LZWCOM]]
 
* [[LzwPak]]
 
* [[LzwPak]]
 +
* [[Microsoft KWAJ installation compression]]
 
* [[Microsoft SZ installation compression]]
 
* [[Microsoft SZ installation compression]]
 
* [[Mozilla LZ4]] (.jsonlz4)
 
* [[Mozilla LZ4]] (.jsonlz4)
 
* [[MS-DOS installation compression]] (.??_)
 
* [[MS-DOS installation compression]] (.??_)
 
* [[MULTIKIT COMPRESS]]
 
* [[MULTIKIT COMPRESS]]
 +
* [[NetWare Packed File]]
 
* [[NOAH]]
 
* [[NOAH]]
 
* [[Nova Compress]]
 
* [[Nova Compress]]
Line 115: Line 131:
 
* [[NRV (Not Really Vanished)]]
 
* [[NRV (Not Really Vanished)]]
 
* [[Oodle Data Compression]]
 
* [[Oodle Data Compression]]
−
* [[pack (Unix)]] (.z)
 
 
* [[PGNPack]]
 
* [[PGNPack]]
 
* [[PowerPacker]] (.pp)
 
* [[PowerPacker]] (.pp)
 +
* [[PPMZ]]
 +
* [[PPMZ2]]
 
* [[Pro-Pack]]
 
* [[Pro-Pack]]
 
* [[PUT and GET (Scott Jennings)]]
 
* [[PUT and GET (Scott Jennings)]]
 
* [[Quasijarus Strong Compression]]
 
* [[Quasijarus Strong Compression]]
 
* [[RAC]]
 
* [[RAC]]
 +
* [[Reduq]]
 
* [[RKUC]]
 
* [[RKUC]]
 
* [[rzip]] (.rz)
 
* [[rzip]] (.rz)
 +
* [[S2]] (.s2, .snappy)
 
* [[SCO compress LZH]]
 
* [[SCO compress LZH]]
 
* [[SEMONE]] (.one)
 
* [[SEMONE]] (.one)
 +
* [[sfArk]]
 
* [[SFDN Packer]]
 
* [[SFDN Packer]]
 +
* [[SFPack]]
 
* [[SHARC]] (density)
 
* [[SHARC]] (density)
 
* [[Softdisk Text Compressor]] (.ctx)
 
* [[Softdisk Text Compressor]] (.ctx)
Line 146: Line 167:
 
* [[WWPACK]]
 
* [[WWPACK]]
 
* [[XPK]]
 
* [[XPK]]
−
* [[XZ]] (.xz)
 
 
* [[Yaz0]]
 
* [[Yaz0]]
 
* [[zchunk]]
 
* [[zchunk]]
Line 159: Line 179:
 
* [[CFS (Computer Concepts Compression)]]
 
* [[CFS (Computer Concepts Compression)]]
 
* [[DIET (compression)|DIET]]
 
* [[DIET (compression)|DIET]]
 +
* [[Diet Disk]]
 +
* [[DiskDoubler]]
 
* [[Pack-Ice]]
 
* [[Pack-Ice]]
 
* [[SLIM (Dominic Herity)|SLIM]]
 
* [[SLIM (Dominic Herity)|SLIM]]
Line 167: Line 189:
 
(Transparent disk compression software and the like, excluding filesystem formats.)
 
(Transparent disk compression software and the like, excluding filesystem formats.)
  
−
* [[DiskDoubler]]  
+
* [[DoubleDisk]]
−
* [[DriveSpace]] and DoubleSpace
+
* [[DoubleSpace and DriveSpace]]
 
* [[JAM (disk compression)]]
 
* [[JAM (disk compression)]]
 
* [[Stacker]]
 
* [[Stacker]]
 +
* [[SuperStor]]
  
 
For filesystem formats that feature compression, see [[Filesystem]] (especially [[Filesystem#Compressed filesystems]]).
 
For filesystem formats that feature compression, see [[Filesystem]] (especially [[Filesystem#Compressed filesystems]]).
Line 189: Line 212:
 
== Links ==
 
== Links ==
 
* [http://boingboing.net/2014/03/04/data-compression-with-playing.html Data-compression with playing cards]
 
* [http://boingboing.net/2014/03/04/data-compression-with-playing.html Data-compression with playing cards]
−
* [http://www.quora.com/What-is-the-most-compressed-file-ever/answer/Kelly-Martin?srid=uejz&share=1 What is the most compressed file ever?]
+
* [https://web.archive.org/web/20201108101659/https://bestofquora.wordpress.com/2014/06/14/what-is-the-most-compressed-file-ever/ What is the most compressed file ever?]  
−
* [http://ieeeghn.org/wiki/index.php/History_of_Lossless_Data_Compression_Algorithms History of Lossless Data Compression Algorithms]
+
* [https://ethw.org/History_of_Lossless_Data_Compression_Algorithms History of Lossless Data Compression Algorithms]
 +
* [https://www.mattmahoney.net/dc/dce.html Data Compression Explained - Matt Mahoney]

Latest revision as of 08:37, 16 August 2025

File Format
Name Compression
Ontology

{{{caption}}}

Contents

[edit] Stream compression formats

A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like tar or the more archaic variant shar can be combined with an archive format to produce a proper compressed archive.

[edit] Compression algorithms and compressed data formats

(excluding formats usually used as file formats)

[edit] Graphics compression

See Graphics#Compression for compression formats used primarily with graphics.

[edit] Specific file formats/programs

(multi-file compressors are in Archiving)

[edit] Transparent file compression

(A mostly-obsolete technique in which a background process hooks into the OS and decompresses files transparently when they are read. Compression might be done manually. The line between this and Disk compression can be blurry.)

[edit] Disk compression

(Transparent disk compression software and the like, excluding filesystem formats.)

For filesystem formats that feature compression, see Filesystem (especially Filesystem#Compressed filesystems).

[edit] Executable compression

See Executable compression.

[edit] See also

[edit] Links

Retrieved from "http://fileformats.archiveteam.org/index.php?title=Compression&oldid=50841"
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox