Compression
From Just Solve the File Format Problem
(Difference between revisions)
Dan Tobias (Talk | contribs) (→See also) |
Dan Tobias (Talk | contribs) (→Specific file formats/programs: Change Yaz0 link to be consistent with page rename and avoid redirection) |
||
(23 intermediate revisions by 2 users not shown) | |||
Line 9: | Line 9: | ||
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive. | A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive. | ||
− | == Compression | + | == Compression algorithms and compressed data formats == |
− | ( | + | (excluding formats usually used as file formats) |
* [[Arithmetic coding]] | * [[Arithmetic coding]] | ||
+ | * [[Brotli]] | ||
+ | * [[Burrows–Wheeler transform]] | ||
* [[DEFLATE]] | * [[DEFLATE]] | ||
+ | * [[DivANS]] | ||
* [[Huffman coding]] | * [[Huffman coding]] | ||
− | |||
* [[LZ4]] | * [[LZ4]] | ||
* [[LZ77]] | * [[LZ77]] | ||
* [[LZ78]] | * [[LZ78]] | ||
+ | * [[LZFSE]] | ||
+ | * [[LZHAM]] | ||
* [[LZMA]] | * [[LZMA]] | ||
* [[LZMA2]] | * [[LZMA2]] | ||
+ | * [[LZO]] (Lempel–Ziv–Oberhumer) | ||
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski) | * [[LZSS]] (Lempel–Ziv–Storer–Szymanski) | ||
* [[LZW]] | * [[LZW]] | ||
+ | * [[PKWARE DCL Implode]] | ||
+ | * [[PPMd]] | ||
* [[Run-length encoding]] | * [[Run-length encoding]] | ||
+ | * [[Shannon–Fano coding]] | ||
+ | * [[Snappy]] (Google-created compression also used by Apple in iWork formats) | ||
* [[zlib]] | * [[zlib]] | ||
+ | * [[Zstandard]] (Zstd) | ||
+ | |||
+ | === Graphics compression === | ||
+ | See [[Graphics#Compression]] for compression formats used primarily with graphics. | ||
== Specific file formats/programs == | == Specific file formats/programs == | ||
Line 53: | Line 66: | ||
* [[lzop]] (.lzo) | * [[lzop]] (.lzo) | ||
* [[LZX]] | * [[LZX]] | ||
+ | * [[Mozilla LZ4]] (.jsonlz4) | ||
* [[MS-DOS installation compression]] (.??_) | * [[MS-DOS installation compression]] (.??_) | ||
+ | * [[Oodle Data Compression]] | ||
* [[pack]] (.z) | * [[pack]] (.z) | ||
+ | * [[Pack-Ice]] | ||
+ | * [[PowerPacker]] (.pp) | ||
+ | * [[Pro-Pack]] | ||
* [[RK]] (WinRK) | * [[RK]] (WinRK) | ||
* [[rzip]] (.rz) | * [[rzip]] (.rz) | ||
+ | * [[SHARC]] (density) | ||
* [[Softdisk Text Compressor]] (.ctx) | * [[Softdisk Text Compressor]] (.ctx) | ||
* [[Squash (RISC OS)|Squash]] - single file compression on RISC OS | * [[Squash (RISC OS)|Squash]] - single file compression on RISC OS | ||
Line 64: | Line 83: | ||
* [[Tzip (Text ZIP)]] | * [[Tzip (Text ZIP)]] | ||
* [[XZ]] (.xz) | * [[XZ]] (.xz) | ||
+ | * [[Yaz0]] | ||
== See also == | == See also == | ||
+ | * [[Compression research]] | ||
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression) | * [[Archiving]] (for formats that place multiple files together in one file, with or without compression) | ||
* [[Graphics#Compression|Graphics]] (for image-specific compression) | * [[Graphics#Compression|Graphics]] (for image-specific compression) | ||
* [[Video]] (for video-specific compression) | * [[Video]] (for video-specific compression) | ||
* [[Audio and Music]] (for audio-specific compression) | * [[Audio and Music]] (for audio-specific compression) | ||
+ | * [[Executables#Executable compression|Executables]] (for executable-specific compression) | ||
== Links == | == Links == | ||
* [http://boingboing.net/2014/03/04/data-compression-with-playing.html Data-compression with playing cards] | * [http://boingboing.net/2014/03/04/data-compression-with-playing.html Data-compression with playing cards] | ||
+ | * [http://www.quora.com/What-is-the-most-compressed-file-ever/answer/Kelly-Martin?srid=uejz&share=1 What is the most compressed file ever?] | ||
+ | * [http://ieeeghn.org/wiki/index.php/History_of_Lossless_Data_Compression_Algorithms History of Lossless Data Compression Algorithms] |
Revision as of 12:46, 10 April 2019
Contents |
Stream compression formats
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like tar can be combined with an archive format to produce a proper compressed archive.
Compression algorithms and compressed data formats
(excluding formats usually used as file formats)
- Arithmetic coding
- Brotli
- Burrows–Wheeler transform
- DEFLATE
- DivANS
- Huffman coding
- LZ4
- LZ77
- LZ78
- LZFSE
- LZHAM
- LZMA
- LZMA2
- LZO (Lempel–Ziv–Oberhumer)
- LZSS (Lempel–Ziv–Storer–Szymanski)
- LZW
- PKWARE DCL Implode
- PPMd
- Run-length encoding
- Shannon–Fano coding
- Snappy (Google-created compression also used by Apple in iWork formats)
- zlib
- Zstandard (Zstd)
Graphics compression
See Graphics#Compression for compression formats used primarily with graphics.
Specific file formats/programs
(multi-file compressors are in Archiving)
- 7z
- 9CDR (Amiga FileImploder Clone)
- BARF (.x, .x??)
- bzip (.bz)
- bzip2 (.bz2)
- compress (.Z)
- CrLZH (.?y?)
- CRN (.crn) - compressed text files used for PC-Write manual
- Crunch (.?z?)
- DiskDoubler
- DUPA (Amiga FileImploder Clone)
- Error Code Modeler (.ecm)
- File Imploder (Amiga) (.imp)
- Freeze/Melt (Unix) (.F)
- gzip (.gz)
- Inflate (.infl) - parody format that actually increases size
- JCalG1 (.jc, Commodore Amiga)
- lrzip (.lrz)
- Lzip (.lz)
- LZMA_Alone (.lzma)
- lzop (.lzo)
- LZX
- Mozilla LZ4 (.jsonlz4)
- MS-DOS installation compression (.??_)
- Oodle Data Compression
- pack (.z)
- Pack-Ice
- PowerPacker (.pp)
- Pro-Pack
- RK (WinRK)
- rzip (.rz)
- SHARC (density)
- Softdisk Text Compressor (.ctx)
- Squash - single file compression on RISC OS
- Squeeze/SQ (.?q?)
- TTComp archive
- tzip
- Tzip (Text ZIP)
- XZ (.xz)
- Yaz0
See also
- Compression research
- Archiving (for formats that place multiple files together in one file, with or without compression)
- Graphics (for image-specific compression)
- Video (for video-specific compression)
- Audio and Music (for audio-specific compression)
- Executables (for executable-specific compression)