Compression
From Just Solve the File Format Problem
(Difference between revisions)
Dan Tobias (Talk | contribs) (→Links) |
(→Specific file formats/programs) |
||
(24 intermediate revisions by 3 users not shown) | |||
Line 9: | Line 9: | ||
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive. | A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive. | ||
− | == Compression | + | == Compression algorithms and compressed data formats == |
− | ( | + | (excluding formats usually used as file formats) |
* [[Arithmetic coding]] | * [[Arithmetic coding]] | ||
+ | * [[Brotli]] | ||
* [[Burrows–Wheeler transform]] | * [[Burrows–Wheeler transform]] | ||
* [[DEFLATE]] | * [[DEFLATE]] | ||
+ | * [[DivANS]] | ||
* [[Huffman coding]] | * [[Huffman coding]] | ||
* [[LZ4]] | * [[LZ4]] | ||
* [[LZ77]] | * [[LZ77]] | ||
* [[LZ78]] | * [[LZ78]] | ||
+ | * [[LZFSE]] | ||
+ | * [[LZHAM]] | ||
* [[LZMA]] | * [[LZMA]] | ||
* [[LZMA2]] | * [[LZMA2]] | ||
Line 25: | Line 29: | ||
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski) | * [[LZSS]] (Lempel–Ziv–Storer–Szymanski) | ||
* [[LZW]] | * [[LZW]] | ||
+ | * [[PKWARE DCL Implode]] | ||
* [[PPMd]] | * [[PPMd]] | ||
* [[Run-length encoding]] | * [[Run-length encoding]] | ||
+ | ** [[RLE90]] | ||
+ | * [[Shannon–Fano coding]] | ||
* [[Snappy]] (Google-created compression also used by Apple in iWork formats) | * [[Snappy]] (Google-created compression also used by Apple in iWork formats) | ||
* [[zlib]] | * [[zlib]] | ||
+ | * [[Zstandard]] (Zstd) | ||
+ | ** [[Zstandard dictionary]] | ||
+ | |||
+ | === Graphics compression === | ||
+ | See [[Graphics#Compression]] for compression formats used primarily with graphics. | ||
== Specific file formats/programs == | == Specific file formats/programs == | ||
Line 34: | Line 46: | ||
(multi-file compressors are in [[Archiving]]) | (multi-file compressors are in [[Archiving]]) | ||
− | |||
* [[9CDR]] (Amiga FileImploder Clone) | * [[9CDR]] (Amiga FileImploder Clone) | ||
* [[BARF]] (.x, .x??) | * [[BARF]] (.x, .x??) | ||
Line 43: | Line 54: | ||
* [[CRN]] (.crn) - compressed text files used for PC-Write manual | * [[CRN]] (.crn) - compressed text files used for PC-Write manual | ||
* [[Crunch]] (.?z?) | * [[Crunch]] (.?z?) | ||
+ | * [[DIET (compression)|DIET]] | ||
* [[DiskDoubler]] | * [[DiskDoubler]] | ||
* [[DUPA]] (Amiga FileImploder Clone) | * [[DUPA]] (Amiga FileImploder Clone) | ||
Line 49: | Line 61: | ||
* [[Freeze/Melt]] (Unix) (.F) | * [[Freeze/Melt]] (Unix) (.F) | ||
* [[gzip]] (.gz) | * [[gzip]] (.gz) | ||
+ | * [[Hammer (compression)|Hammer]] | ||
* [[Inflate]] (.infl) - parody format that actually increases size | * [[Inflate]] (.infl) - parody format that actually increases size | ||
* [[JCalG1]] (.jc, Commodore Amiga) | * [[JCalG1]] (.jc, Commodore Amiga) | ||
Line 56: | Line 69: | ||
* [[lzop]] (.lzo) | * [[lzop]] (.lzo) | ||
* [[LZX]] | * [[LZX]] | ||
+ | * [[Mozilla LZ4]] (.jsonlz4) | ||
* [[MS-DOS installation compression]] (.??_) | * [[MS-DOS installation compression]] (.??_) | ||
+ | * [[Oodle Data Compression]] | ||
* [[pack]] (.z) | * [[pack]] (.z) | ||
+ | * [[Pack-Ice]] | ||
+ | * [[PowerPacker]] (.pp) | ||
+ | * [[Pro-Pack]] | ||
* [[RK]] (WinRK) | * [[RK]] (WinRK) | ||
* [[rzip]] (.rz) | * [[rzip]] (.rz) | ||
+ | * [[SHARC]] (density) | ||
* [[Softdisk Text Compressor]] (.ctx) | * [[Softdisk Text Compressor]] (.ctx) | ||
* [[Squash (RISC OS)|Squash]] - single file compression on RISC OS | * [[Squash (RISC OS)|Squash]] - single file compression on RISC OS | ||
Line 67: | Line 86: | ||
* [[Tzip (Text ZIP)]] | * [[Tzip (Text ZIP)]] | ||
* [[XZ]] (.xz) | * [[XZ]] (.xz) | ||
+ | * [[Yaz0]] | ||
+ | |||
+ | == Executable compression == | ||
+ | |||
+ | See [[Executable compression]]. | ||
== See also == | == See also == | ||
+ | * [[Compression research]] | ||
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression) | * [[Archiving]] (for formats that place multiple files together in one file, with or without compression) | ||
* [[Graphics#Compression|Graphics]] (for image-specific compression) | * [[Graphics#Compression|Graphics]] (for image-specific compression) |
Revision as of 14:50, 15 September 2019
Contents |
Stream compression formats
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like tar can be combined with an archive format to produce a proper compressed archive.
Compression algorithms and compressed data formats
(excluding formats usually used as file formats)
- Arithmetic coding
- Brotli
- Burrows–Wheeler transform
- DEFLATE
- DivANS
- Huffman coding
- LZ4
- LZ77
- LZ78
- LZFSE
- LZHAM
- LZMA
- LZMA2
- LZO (Lempel–Ziv–Oberhumer)
- LZSS (Lempel–Ziv–Storer–Szymanski)
- LZW
- PKWARE DCL Implode
- PPMd
- Run-length encoding
- Shannon–Fano coding
- Snappy (Google-created compression also used by Apple in iWork formats)
- zlib
- Zstandard (Zstd)
Graphics compression
See Graphics#Compression for compression formats used primarily with graphics.
Specific file formats/programs
(multi-file compressors are in Archiving)
- 9CDR (Amiga FileImploder Clone)
- BARF (.x, .x??)
- bzip (.bz)
- bzip2 (.bz2)
- compress (.Z)
- CrLZH (.?y?)
- CRN (.crn) - compressed text files used for PC-Write manual
- Crunch (.?z?)
- DIET
- DiskDoubler
- DUPA (Amiga FileImploder Clone)
- Error Code Modeler (.ecm)
- File Imploder (Amiga) (.imp)
- Freeze/Melt (Unix) (.F)
- gzip (.gz)
- Hammer
- Inflate (.infl) - parody format that actually increases size
- JCalG1 (.jc, Commodore Amiga)
- lrzip (.lrz)
- Lzip (.lz)
- LZMA_Alone (.lzma)
- lzop (.lzo)
- LZX
- Mozilla LZ4 (.jsonlz4)
- MS-DOS installation compression (.??_)
- Oodle Data Compression
- pack (.z)
- Pack-Ice
- PowerPacker (.pp)
- Pro-Pack
- RK (WinRK)
- rzip (.rz)
- SHARC (density)
- Softdisk Text Compressor (.ctx)
- Squash - single file compression on RISC OS
- Squeeze/SQ (.?q?)
- TTComp archive
- tzip
- Tzip (Text ZIP)
- XZ (.xz)
- Yaz0
Executable compression
See also
- Compression research
- Archiving (for formats that place multiple files together in one file, with or without compression)
- Graphics (for image-specific compression)
- Video (for video-specific compression)
- Audio and Music (for audio-specific compression)