WARC

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
(Other links and references)
Line 38: Line 38:
 
* [https://github.com/lintool/warcbase/wiki Warcbase Wiki]
 
* [https://github.com/lintool/warcbase/wiki Warcbase Wiki]
 
* [http://kris-sigur.blogspot.com/2015/08/the-warc-format-11.html Discussion of WARC format 1.1, under development]
 
* [http://kris-sigur.blogspot.com/2015/08/the-warc-format-11.html Discussion of WARC format 1.1, under development]
 +
* [http://qanda.digipres.org/1155/merging-&-deduping-warc-files?show=1159#a1159 Merging & Deduping WARC files]
 
* [http://library.gwu.edu/scholarly-technology-group/posts/harvesting-twitter-streaming-api-warc-files Harvesting the Twitter Streaming API to WARC files]
 
* [http://library.gwu.edu/scholarly-technology-group/posts/harvesting-twitter-streaming-api-warc-files Harvesting the Twitter Streaming API to WARC files]
 
* [http://www.digitalstudies.org/ojs/index.php/digital_studies/article/view/325/412 The great WARC adventure: Using SIPS, AIPS, and DIPS to document SLAAPs]
 
* [http://www.digitalstudies.org/ojs/index.php/digital_studies/article/view/325/412 The great WARC adventure: Using SIPS, AIPS, and DIPS to document SLAAPs]

Revision as of 13:16, 8 June 2018

File Format
Name WARC
Ontology
Extension(s) .warc, .warc.gz
MIME Type(s) application/warc, application/warc-fields
PRONOM fmt/289

WARC is the successor to the ARC (Internet Archive) format. Standardized as ISO 28500:2009, Information and documentation -- WARC file format. Developed under the auspices of the International Internet Preservation Consortium. WARC was developed as an extension to ARC in part to provide better capabilities for managing Web archives for the long term, allowing for capture of more metadata about the circumstances of archiving.

WARC files are often compressed using gzip, resulting in a .warc.gz extension.

There is also a specification for a Web Archive Metadata File. Another metadata format used with WARC files is CDX.

Contents

Specifications

Sample files

Tools

Other links and references

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox