Crunch

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
(bit more detail on lbrate)
(9 intermediate revisions by one user not shown)
Line 4: Line 4:
 
|extensions={{ext|?z?}}, {{ext|zzz}}
 
|extensions={{ext|?z?}}, {{ext|zzz}}
 
}}
 
}}
 +
:''This article is about the CP/M compressed file format. See the [[#Disambiguation|disambiguation section]] for other "Crunch" formats.''
  
[[Crunch]] was a method of compressing single files popular on [[CP/M]], devised by Steve Greenberg circa 1986. It superseded [[Squeeze]] and was succeeded by [[CrLZH]], and crunched files were common in [[LBR]] archives. The underlying compression uses the [[LZW]] algorithm.
+
[[Crunch]] was a method of compressing single files popular on [[CP/M]], devised by Steve Greenberg circa 1986. It superseded [[Squeeze]] and was succeeded by [[CrLZH]], and crunched files were common in [[LBR]] archives. The underlying compression uses the [[LZW]] algorithm, combined with [[run-length encoding]].
  
 
Similar to [[Squeeze]], crunched files were signified in CP/M's 8.3 filename format by replacing the middle letter of the extension with Z (.?Z?), with the extension .ZZZ used for corner cases such as a blank extension.
 
Similar to [[Squeeze]], crunched files were signified in CP/M's 8.3 filename format by replacing the middle letter of the extension with Z (.?Z?), with the extension .ZZZ used for corner cases such as a blank extension.
 +
 +
There are two versions of the compressed data format, and not all decompressors support both. The new format is apparently more common.
 +
 +
== Disambiguation ==
 +
Not to be confused with:
 +
* The "crunched" compression methods used in [[ARC (compression format)|ARC]] format. These methods are possibly related to those used by CP/M Crunch, but the file formats are not otherwise similar.
 +
* [[Crunch-Mania]] - An Amiga file compression utility
 +
* [[Cruncher]] - An executable compression utility for DOS, by Ori Berger [http://cd.textfiles.com/simtel/simtel20/MSDOS/EXECOMP/CRUNCH10.ZIP]
 +
* CRUNCH - A compression optimization utility for DOS, by Bruce Gavin. It does not appear to have any native file formats. [http://cd.textfiles.com/20mnn/ARCHIVE/CRUNCH10.ZIP]
 +
* Crunch - An old [[ARC (compression format)|ARC]] compression utility by Richard P. Byrne [http://cd.textfiles.com/rbbsv3n1/d86v/crunch.zip]
 +
* CRUNCH - A PKARC automation utility by Chuck Zulker [http://cd.textfiles.com/megarom/megarom1/ARC_LBR/CRUNCH.ZIP]
 +
 +
== See also ==
 +
* [[Squeeze]] - predecessor
 +
* [[CrLZH]] - successor
 +
* [[LBR]] - container
 +
* [[ZSQ (LZW compression)‎]] - Possibly related
 +
 +
== Format details ==
 +
Note that, as explained in the format documentation, the "filename" field contains not only the filename, but also extension data. If extension data exists, the filename extension is padded with spaces until it is exactly three characters long.
 +
 +
One quirk that does not seem to be documented is that the high bit of every byte in the filename field must be ignored, and treated as if it were 0. It's unclear why, and it's unclear if this rule also applies to the terminating 0x00 (/0x80) byte.
 +
 +
== Identification ==
 +
Files begin with bytes {{magic|76 fe}}.
 +
 +
== Specifications ==
 +
 +
* The file header is described in the text file LZDEF20.DOC shipped with [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/sigm/vol294/crunch20.lbr CRUNCH20.LBR].
 +
** An extracted copy is provided [[Crunch/LZDEF20.DOC|here]].
 +
** Note that the file header follows a similar/compatible structure to [[CrLZH]]. It was derived from [[Squeeze]], but bears only a little resemblance to it.
 +
* FIXME: is the exact compression algorithm documented anywhere?
 +
** CRUNCH20.DOC shipped in CRUNCH20.LBR says: ''It embodies all of the concepts employed in the UNIX COMPRESS / ARC512 algorithm, but is additionally enhanced by a "metastatic code reassignment" facility. This is one of several concepts I am developing as part of an effort to advance data compression techniques beyond current performance limits. I believe this is the first time this principle has been proposed or implemented.''
 +
** See also "Technical Abstract" by Steven Greenberg, 16 November 1986: [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crunch.abs CRUNCH.ABS]
  
 
== Tools ==
 
== Tools ==
Line 16: Line 51:
 
** The canonical tools were CRUNCH and UNCR. Possibly Greenberg's last version (Feb 1988) is v2.4: [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crunch24.lbr CRUNCH24.LBR], [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crnch24s.lbr CRNCH24S.LBR] (source code).
 
** The canonical tools were CRUNCH and UNCR. Possibly Greenberg's last version (Feb 1988) is v2.4: [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crunch24.lbr CRUNCH24.LBR], [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crnch24s.lbr CRNCH24S.LBR] (source code).
 
** The later LT31 deals with extracting from all of [[Squeeze]], [[Crunch]], [[CrLZH]] and [[LBR]] formats. Widely available in CP/M archives, e.g. [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/arc-lbr/lt31.lbr LT31.LBR]
 
** The later LT31 deals with extracting from all of [[Squeeze]], [[Crunch]], [[CrLZH]] and [[LBR]] formats. Widely available in CP/M archives, e.g. [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/arc-lbr/lt31.lbr LT31.LBR]
 +
* [[The Unarchiver]]
  
== References ==
+
== Sample files ==
 +
* [http://www.classiccmp.org/cpmarchives/ftp.php?b=cpm/mirrors/oak.oakland.edu/pub/cpm/ OAK CP/M archive] → .../*.?z?
 +
* Found in many [[LBR#Sample files|LBR]] files. Note that you may have to tell your LBR utility not to decompress them (e.g. <code>lbrate -n</code>).
  
* The file header is described in the text file LZDEF20.DOC shipped with [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/sigm/vol294/crunch20.lbr CRUNCH20.LBR].
+
[[Category:File formats with too many extensions]]
** An extracted copy is provided [[Crunch/LZDEF20.DOC|here]].
+
[[Category:CP/M]]
** Note that the file header follows a similar/compatible structure to [[Squeeze]] and [[CrLZH]].
+
* FIXME: is the exact compression algorithm documented anywhere?
+
** CRUNCH20.DOC shipped in CRUNCH20.LBR says: ''It embodies all of the concepts employed in the UNIX COMPRESS / ARC512 algorithm, but is additionally enhanced by a "metastatic code reassignment" facility. This is one of several concepts I am developing as part of an effort to advance data compression techniques beyond current performance limits. I believe this is the first time this principle has been proposed or implemented.''
+
** See also "Technical Abstract" by Steven Greenberg, 16 November 1986: [http://www.classiccmp.org/cpmarchives/cpm/mirrors/oak.oakland.edu/pub/cpm/squsq/crunch.abs CRUNCH.ABS]
+

Revision as of 13:37, 27 September 2020

File Format
Name Crunch
Ontology
Extension(s) .?z?, .zzz
This article is about the CP/M compressed file format. See the disambiguation section for other "Crunch" formats.

Crunch was a method of compressing single files popular on CP/M, devised by Steve Greenberg circa 1986. It superseded Squeeze and was succeeded by CrLZH, and crunched files were common in LBR archives. The underlying compression uses the LZW algorithm, combined with run-length encoding.

Similar to Squeeze, crunched files were signified in CP/M's 8.3 filename format by replacing the middle letter of the extension with Z (.?Z?), with the extension .ZZZ used for corner cases such as a blank extension.

There are two versions of the compressed data format, and not all decompressors support both. The new format is apparently more common.

Contents

Disambiguation

Not to be confused with:

  • The "crunched" compression methods used in ARC format. These methods are possibly related to those used by CP/M Crunch, but the file formats are not otherwise similar.
  • Crunch-Mania - An Amiga file compression utility
  • Cruncher - An executable compression utility for DOS, by Ori Berger [1]
  • CRUNCH - A compression optimization utility for DOS, by Bruce Gavin. It does not appear to have any native file formats. [2]
  • Crunch - An old ARC compression utility by Richard P. Byrne [3]
  • CRUNCH - A PKARC automation utility by Chuck Zulker [4]

See also

Format details

Note that, as explained in the format documentation, the "filename" field contains not only the filename, but also extension data. If extension data exists, the filename extension is padded with spaces until it is exactly three characters long.

One quirk that does not seem to be documented is that the high bit of every byte in the filename field must be ignored, and treated as if it were 0. It's unclear why, and it's unclear if this rule also applies to the terminating 0x00 (/0x80) byte.

Identification

Files begin with bytes 76 fe.

Specifications

  • The file header is described in the text file LZDEF20.DOC shipped with CRUNCH20.LBR.
    • An extracted copy is provided here.
    • Note that the file header follows a similar/compatible structure to CrLZH. It was derived from Squeeze, but bears only a little resemblance to it.
  • FIXME: is the exact compression algorithm documented anywhere?
    • CRUNCH20.DOC shipped in CRUNCH20.LBR says: It embodies all of the concepts employed in the UNIX COMPRESS / ARC512 algorithm, but is additionally enhanced by a "metastatic code reassignment" facility. This is one of several concepts I am developing as part of an effort to advance data compression techniques beyond current performance limits. I believe this is the first time this principle has been proposed or implemented.
    • See also "Technical Abstract" by Steven Greenberg, 16 November 1986: CRUNCH.ABS

Tools

  • CFX (DOS/Unix)
  • lbrate by Russell Marks, c. 2001 (Unix, GPL2)
  • On CP/M (or emulators):
    • The canonical tools were CRUNCH and UNCR. Possibly Greenberg's last version (Feb 1988) is v2.4: CRUNCH24.LBR, CRNCH24S.LBR (source code).
    • The later LT31 deals with extracting from all of Squeeze, Crunch, CrLZH and LBR formats. Widely available in CP/M archives, e.g. LT31.LBR
  • The Unarchiver

Sample files

  • OAK CP/M archive → .../*.?z?
  • Found in many LBR files. Note that you may have to tell your LBR utility not to decompress them (e.g. lbrate -n).
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox