Just Solve the File Format Problem - User contributions [en]

BMP

2021-04-16T04:20:01Z

Cypherpunks: add note about negative height

{{FormatInfo
|formattype=electronic
|subcat=Graphics
|extensions={{ext|bmp}}, {{ext|rle}}, {{ext|dib}}, others
|mimetypes={{mimetype|image/bmp}}
|locfdd={{LoCFDD|fdd000189}}
|pronom={{PRONOM|fmt/116}}, others
|wikidata={{wikidata|Q192869}}
|kaitai struct=bmp
|released=1987
}}
'''BMP''' is a family of raster image file formats primarily used on Microsoft Windows and OS/2 operating systems. The format is sometimes known as '''Device-Independent Bitmap''' ('''DIB'''), since, when loaded into memory using Windows software, the image is held as a DIB structure.

Though seemingly a simple format, it is complicated by its many different versions, lack of an official specification, lack of any version control process, and ambiguities and contradictions in the documentation.

== Discussion ==
The term ''DIB'' can mean several different things:
* A synonym for BMP file format.
* An in-memory DIB object, with no file header. The header data and the bitmap data do not have to be stored contiguously. Some Win32 API functions use this format.
* A "packed DIB" memory object or file component, with no file header. The header data and the bitmap data are stored contiguously. This is a standard [[Windows clipboard|clipboard format]], for example.
* A "packed DIB" stored in a file by itself.

A number of Windows-centric formats contain some nonstandard modified or compressed form of BMP/DIB, intended to be reconstructed as a DIB at runtime.

== Format details ==

=== Compression ===
Images are usually uncompressed, but [[Run-length encoding|RLE]] compression can be used under some conditions. [[JPEG]], [[PNG]], and [[Modified Huffman|Huffman 1D]] compression are also theoretically possible, but rarely supported.

=== File structure ===
A BMP file starts with a 14-byte "BITMAPFILEHEADER" structure.

Immediately after that is another header which we'll refer to as the "info header", though some versions of it are named "core header" instead. There are a number of different versions and sub-versions of it. It starts with 4-byte integer indicating its size, which ''mostly'' reveals its version.

The pixel data is pointed to by a field in BITMAPFILEHEADER. There can be other data segments, e.g. for a color palette, before (and, rarely, after) the pixel data.

Pixel data is usually stored from bottom up (but is top-down if the header indicates a negative height). If uncompressed, each row is padded to a multiple of 4 bytes.

== Identifiers ==
No MIME type has been officially registered. Strings found in practice are: image/bmp; image/x-bmp; image/x-ms-bmp

The usual filename extension is '''.bmp'''. Extensions '''.rle''' (for RLE-compressed images) and '''.dib''' (which sometimes indicates that the file lacks a ''file header'') are also sometimes seen. Many other extensions have been used by various applications.

== Identification ==
BMP files start with the ASCII signature "{{magic|BM}}".

The bytes at the beginning of the file match one of these two patterns:

42 4d ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? 00 00 0c 00
00 00 ?? ?? ?? ?? 01 00 ?? 00

or

42 4d ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? 00 00 ?? 00
00 00 ?? ?? ?? ?? ?? ?? ?? ?? 01 00 ?? 00

The "reserved" bytes at offset 6 to 9 are usually all zero, but a few OS/2 BMP files have nonzero "hotspot coordinates" there.

Compare to [[VBM (VDC BitMap)]], an obscure format with a similar signature.

== Well-known versions ==

=== Windows BMP v2 ===
* Info header size: 12
* Info header name: BITMAPCOREHEADER
* PRONOM: {{PRONOM|fmt/115}}

See also OS/2 BMP 1.0, below.

=== Windows BMP v3 ===
* Info header size: 40
* Info header name: BITMAPINFOHEADER
* PRONOM: {{PRONOM|fmt/116}}, {{PRONOM|fmt/117}}

This is by far the most widely used version of BMP. It was introduced with Windows 3.x. Windows NT extended it to support 16 and 32 bits/pixel.

Windows CE also extended it, for example to allow 2 bits/pixel (see [[Pocket PC Bitmap]]), but its extensions were not migrated to the BMP v4 and v5 formats.

Microsoft's GDI+ reportedly supports BMP images with 64 bits/pixel[https://docs.microsoft.com/en-us/windows/win32/gdiplus/-gdiplus-types-of-bitmaps-about], but no technical documentation of this extension has been located.

It is apparently possible for OS/2 BMP 2.0 format to masquerade as Windows BMP v3. The upshot is that if the compression type is 3 and the bit depth is 1, or the compression type is 4 and the bit depth is 24, then the file should be treated as OS/2 BMP 2.0.

=== Windows BMP v4 ===
* Info header size: 108
* Info header name: BITMAPV4HEADER
* PRONOM: {{PRONOM|fmt/118}}

Introduced with Windows 95. Adds support for transparency and colorimetry.

=== Windows BMP v5 ===
* Info header size: 124
* Info header name: BITMAPV5HEADER
* PRONOM: {{PRONOM|fmt/119}}

Introduced with Windows 98. Adds support for [[ICC profile|ICC profiles]].

== Other versions ==

=== Windows BMP v1 ===
* PRONOM: {{PRONOM|fmt/114}}

Also called DDB, this is the bitmap file format used by Windows 1.0. It's not really a BMP format. Refer to [[Windows DDB]].

=== OS/2 BMP 1.0 ===
:''See also the [[OS/2 BMP|OS/2 BMP disambiguation page]].''
* Info header size: 12
* Info header name: BITMAPCOREHEADER or OS21XBITMAPHEADER
* PRONOM: {{PRONOM|x-fmt/25}} (deprecated), {{PRONOM|fmt/115}}

For practical purposes, OS/2 BMP 1.0 is identical to Windows BMP v2. But there can be at least one small difference: In OS/2 formats, the "Size" field at offset 2 (variously named "cbSize", "FileSize", or "bfSize") is sometimes set to the size of the headers, instead of the size of the file. For v1, that means it will be 26 (14+12). For v2, it can range from 30 to 78.

=== OS/2 BMP 2.0 ===
* Info header size: 16–64 (16, 24, 40, 48, and 64 may be most common)
* Info header name: BITMAPCOREHEADER2 or OS22XBITMAPHEADER
* PRONOM: {{PRONOM|x-fmt/270}}

OS/2 BMP 2.0 defines several file subtypes; here we are describing only the "Bitmap" subtype (files with a signature of "BM"). For other subtypes, see [[OS/2 bitmap family]].

The header size can be reduced from its full size of 64 bytes. Omitted fields are assumed to have a value of zero.

The fields in the first 40 bytes of the header are (nearly) identical to those in Windows BMP v3, v4, and v5. The remaining fields are different.

OS/2 BMP 2.0 supports compression types "Huffman 1D" and "RLE24", unlike any other version of BMP.

=== BITMAPV2INFOHEADER ===
* Info header size: 52
* Info header name: BITMAPV2INFOHEADER

Uncertain; possibly an abbreviated V4/V5 header.

=== BITMAPV3INFOHEADER ===
* Info header size: 56
* Info header name: BITMAPV3INFOHEADER

Uncertain; possibly an abbreviated V4/V5 header.

=== Packed DIB file ===
Same as the common BMP formats, but omits the 14-byte ''file header''.

=== OS/2 Bitmap Array ===
Sometimes, an [[OS/2 Bitmap Array]] file containing one or more bitmaps is considered to be a kind of BMP file. Such a file begins with an extra 14-byte header, with signature "<code>BA</code>". (But note that deleting this header is not quite enough to get a valid BMP file.)

== Symbol definitions ==
Here are the definitions, from the Windows SDKs, of some of the symbols used in the BMP documentation.

All integers use little-endian [[Endianness|byte order]].

{| class="wikitable"
! Symbol
! Definition
|-
|WORD || unsigned 16-bit integer
|-
|DWORD || unsigned 32-bit integer
|-
|LONG || signed 32-bit integer
|-
|BI_RGB || 0
|-
|BI_RLE8 || 1
|-
|BI_RLE4 || 2
|-
|BI_BITFIELDS || 3
|-
|(Huffman 1D) || 3
|-
|BI_JPEG || 4
|-
|(24-bit RLE) || 4
|-
|BI_PNG || 5
|-
|BI_ALPHABITFIELDS || 6
|-
|BI_SRCPREROTATE || 0x8000 (?)
|-
|LCS_CALIBRATED_RGB || 0
|-
|LCS_sRGB || <code>'sRGB'</code> = 0x73524742
|-
|LCS_WINDOWS_COLOR_SPACE || <code>'Win '</code> = 0x57696e20
|-
|PROFILE_LINKED || <code>'LINK'</code> = 0x4c494e4b
|-
|PROFILE_EMBEDDED || <code>'MBED'</code> = 0x4d424544
|-
|LCS_GM_BUSINESS || 1
|-
|LCS_GM_GRAPHICS || 2
|-
|LCS_GM_IMAGES || 4
|-
|LCS_GM_ABS_COLORIMETRIC || 8
|}

== Specifications ==
* Most of the format (but not the BITMAPFILEHEADER) is defined in the [https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-wmf/4813e7fd-52d0-4f42-965f-228c8b7488d2 Windows Metafile Specification] (as "DeviceIndependentBitmap")
* Defined in the [https://www.aelius.com/njh/wavemetatools/doc/riffmci.pdf RIFF specification] (as Device Independent Bitmap File Format)
* [https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapfileheader BITMAPFILEHEADER]
* [https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapcoreheader BITMAPCOREHEADER]
* [https://docs.microsoft.com/en-us/previous-versions/dd183376(v=vs.85) BITMAPINFOHEADER]
* [https://docs.microsoft.com/en-us/previous-versions/windows/embedded/aa452885(v=msdn.10) BITMAPINFOHEADER (Windows CE 5.0)]
* [https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv4header BITMAPV4HEADER]
* [https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv5header BITMAPV5HEADER]
* [http://www.digicamsoft.com/bmp/bmp.html Another site with format descriptions]
* [http://www.fileformat.info/format/os2bmp/spec/902d5c253f2a43ada39c2b81034f27fd/view.htm OS/2 Presentation Manager Programming Reference Guide, Vol III]
* [http://whatis.rest7.com/how-to-open-bmp-file Specification of BMP headers]
* [http://zig.tgschultz.com/bmp_file_format.txt BMP format information by T. Schultz]

== Metaformat files ==

* {{Synalysis|bitmap}}

== Software ==
BMP is widely supported by graphics software, including web browsers. Software listed here has been arbitrarily selected.
* [[Netpbm]]: bmptopnm, ppmtobmp
* [[ImageMagick]]
* [[XnView]]
* [[Tom's Viewer]]

== Sample files ==
=== Windows BMP v2 ===
This format is fairly common, but examples of it can be hard to spot amidst all the BMPv3 files.
* [https://samples.libav.org/image-samples/money-2-%28os2%29.bmp money-2-(os2).bmp], [https://samples.libav.org/image-samples/money-16-%28os2%29.bmp money-16-(os2).bmp], [https://samples.libav.org/image-samples/money-256-%28os2%29.bmp money-256-(os2).bmp], [https://samples.libav.org/image-samples/money-24bit-os2.bmp money-24bit-os2.bmp]
* {{CdTextfilesURL|hobbesos29709/disk2/MULTIMED/BMP/}} → BMPS*.ZIP, OS2ORBIT.ZIP, REALMERL.BMP

=== Windows BMP v3 ===
* {{CdTextfilesURL|mmplatinum/IMAGES/BMP/}}
* {{CdTextfilesURL|monstmedia/WIN/ICONS/}}
* {{CdTextfilesURL|maxx/tothemaxww/BMPMISC/}}

=== OS/2 BMP 2.0 ===
* [http://kimludvigsen.dk/os2/os2/graphx/teamlog2.zip teamlog2.zip] → TEAMcol.bmp, teamcol2.bmp
* [https://samples.libav.org/image-samples/bmp-files/test4os2v2.bmp test4os2v2.bmp]
* {{CdTextfiles|hobbesos29709/disk2/MULTIMED/BMP/CARDBMPS.ZIP|CARDBMPS.ZIP}} - Most of the files are OS/2 BMP 2.0
* {{CdTextfiles|hobbesos29709/disk1/GAMES/MAKMAN10.ZIP|MAKMAN10.ZIP}} - Some of the files are files OS/2 BMP 2.0
* {{CdTextfiles|monstmedia/IMAGES/JUR_OS2.BMP|JUR_OS2.BMP}}
* {{CdTextfilesURL|pier/pier09/cdrom/036/}} → os2bmp*.zip

=== Various ===
* [http://wvnvms.wvnet.edu/vmswww/bmp.html Example BMP images (all Windows v3 except as indicated)]
* [https://archive.org/details/OS2BBS OS/2 BBS Files Archive] → 13-Bitmap.zip - Lots of OS/2 BMP 1.0 & 2.0 files
* [https://entropymine.com/jason/bmpsuite/bmpsuite/html/bmpsuite.html BMP Suite Image List]
* https://telparia.com/fileFormatSamples/image/bmp/

== See also ==
* [[CUR]]
* [[ICO]]
* [[OS/2 bitmap family]]
* [[OS/2 Bitmap Array]]
* [[packPNM]]
* [[Pocket PC Bitmap]]
* [[Poser Bump Map]]
* [[Segmented Hypergraphics]]
* [[Windows DDB]]
* [[Winzle Puzzle]]

== Links ==
* [[Wikipedia:BMP file format|Wikipedia: BMP file format]]
* {{EGFF|bmp|Microsoft Windows Bitmap File Format Summary}}, from the [[Encyclopedia of Graphics File Formats]]
* {{EGFF|os2bmp|OS/2 Bitmap File Format Summary}}, from the [[Encyclopedia of Graphics File Formats]]
* [http://coptr.digipres.org/Bad_Peggy Bad Peggy: scans images for problems]
* [https://web.archive.org/web/20190919034602/http://forensicswiki.org/wiki/BMP ForensicsWiki entry] (not much useful info except for a link to an iOS tool for reading metadata)

[[Category:Microsoft]]
[[Category:Windows]]
[[Category:OS/2]]

RAR

2020-12-22T22:42:34Z

Cypherpunks: The Unarchiver can read RAR files

{{FormatInfo
|formattype=electronic
|subcat=Archiving
|extensions={{ext|rar}}
|mimetypes={{mimetype|application/vnd.rar}}, {{mimetype|application/x-rar-compressed}}
|pronom={{PRONOM|x-fmt/264}}, {{PRONOM|fmt/411}}, {{PRONOM|fmt/613}}
|kaitai struct=rar
|released=1993
}}
'''R'''oshal '''AR'''chive ('''RAR''') is a proprietary file format used by the compression software WinRAR. They make the decompression code available for use in other programs and allow its distribution, but with a license provision that "You cannot use the unrar source to re-create the RAR compression algorithm, which is proprietary."

== File identification ==

Current versions start their files with (hex) {{magic|52 61 72 21 1A 07}}, followed by {{magic|00}} or {{magic|01}}. Early versions (for which no format documentation is known) start with {{magic|52 45 7E 5E}}.

== Software ==
* [http://www.rarlab.com/ WinRAR website]
** [http://www.rarlab.com/rar/UnRARDLL.exe Free UnRAR.dll]
** [http://www.rarlab.com/rar/unrarsrc-4.2.4.tar.gz UnRAR source]
* [[Konvertor]]
* [[7-Zip]]
* [[The Unarchiver]]

== Sample files ==
* http://cd.textfiles.com/ataricompendium/FILES/MISC/PRTFOLIO/
* http://cd.textfiles.com/thebbsorg/max_files/
* https://telparia.com/fileFormatSamples/archive/rar/ → *.rar

== Links ==
* [[Wikipedia:RAR (file format)|Wikipedia article]]
* [http://www.forensicswiki.org/wiki/RAR Forensics Wiki article] (has lots of format details)
* [https://imgur.com/a/MtQZv#0 Mini RAR poster]

Finance and Accounting

2020-11-17T06:25:27Z

Cypherpunks: /* Interchange formats: open */ link to Plain text accounting journals

{{FormatInfo
|formattype=electronic
|thiscat=Finance and Accounting
|image=Money grabber.png
}}

==Accounting software==

* [[Exact]]
* [[Peachtree]]
* [[QuickBooks]] (.qbo, .qbw)
* [[M.Y.O.B.]]

==Interchange formats: open==

* [[GnuCash XML Format]]
* [[OFX]]
* [[QIF]]
* [[Tax XML]] (used by various governmental entities for e-filing)
* [[TXF]] (Tax Interchange Format; used by TurboTax)
* [[Plain text accounting journal]] (a family of formats used by Ledger and "Ledger-likes")

==Interchange formats: proprietary==

* [[BACS Standard 18]]
* [[QFX]]

== Market data ==

* [[ITCH]] (NASDAQ)

==Personal finance software==

* [[Banktivity]]
* [[Budget (Apple II)]]
* [[GnuCash]]
* [[Lemon]] (app/website tracking cards and receipts)
* [[Microsoft Money]]
* [[MoneyTree]]
* [[Quicken]] (Creates .qdf, .qph, .qel, .qsd, Imports .qif, .qfx)

==Tax software==

* [[H&R Block]] (.t11, etc.; changes every year)
* [[TurboTax]] (.tax, .tax2010, etc.; changes every year)

== Naming and numbering systems ==

* [[Routing transit number]]

== See also ==

* [[Currency]]

== Links ==
* [http://www.theguardian.com/politics/2014/apr/18/hmrc-to-sell-taxpayers-data HMRC to sell taxpayers' financial data]
* [https://www.federalreserve.gov/apps/fof/ Info on Federal Reserve download data]

Plain text accounting journal

2020-11-17T06:22:54Z

Cypherpunks: add links to journal syntaxes

{{FormatInfo
|subcat=Finance and Accounting
}}

Plain text accounting journals are a method of encoding financial transactions, intended to be readable by humans and also parsable by various programs. The first such format was designed for Ledger by John Wiegley in 2003. Other tools now exist to deal with the same basic format, sometimes differing in details.

== Links ==
* [https://plaintextaccounting.org/ Plain text accounting portal]
* [https://www.ledger-cli.org/3.0/doc/ledger3.html#Basic-format Ledger 3 basic format]
* [https://hledger.org/journal.html hledger journal format]
* [https://beancount.github.io/docs/beancount_language_syntax.html Beancount syntax]

L5X

2020-07-02T04:05:34Z

Cypherpunks: link to reference manual

{{FormatInfo
|formattype=electronic
|subcat=Industrial
|extensions={{ext|l5x}}
}}

L5X is an XML-based file format for configuring Logix 5000 programmable logic controllers from Allen-Bradley, now owned by Rockwell Automation. Their RSLogix 5000 software can import and export this format, or the equivalent non-XML text-based [[L5K|L5K format]].

== Links ==

* [https://literature.rockwellautomation.com/idc/groups/literature/documents/rm/1756-rm084_-en-p.pdf Allen-Bradley reference manual]

L5K

2020-07-02T04:04:29Z

Cypherpunks: link to reference manual

{{FormatInfo
|formattype=electronic
|subcat=Industrial
|extensions={{ext|l5k}}
}}

L5K is a custom text-based file format for configuring Logix 5000 programmable logic controllers from Allen-Bradley, now owned by Rockwell Automation. Their RSLogix 5000 software can import and export this format, or the equivalent XML-based [[L5X|L5X format]].

== Links ==

* [https://literature.rockwellautomation.com/idc/groups/literature/documents/rm/1756-rm084_-en-p.pdf Allen-Bradley reference manual]

XDP

2020-06-09T04:11:18Z

Cypherpunks: link to specification

{{FormatInfo
|formattype=electronic
|extensions={{ext|xdp}}
|mimetypes={{mimetype|application/vnd.adobe.xdp+xml}}
}}
[[XDP]] ('''XML Data Package''') is an XML-based container for [[PDF]] and [[XFA]] data.

== Specifications ==
* [https://web.archive.org/web/20160329143634if_/http://partners.adobe.com/public/developer/en/xml/xdp_2.0.pdf XML Data Package Specification, Version 2.0 (2003-10)]

[[Category:XML based file formats]]
[[Category:Adobe]]

XFDF

2020-06-09T03:56:17Z

Cypherpunks: link to other spec versions

{{FormatInfo
|formattype=electronic
|subcat=Serialization
|extensions={{ext|xfdf}}
|mimetypes={{mimetype|application/vnd.adobe.xfdf}}
}}
[[XFDF]] ('''XML Forms Data Format''') is a format for storing and transferring data entered into a form in a [[PDF]] file. It is an [[XML]]-based alternative to [[FDF]].

== Specifications ==
* [https://web.archive.org/web/20160408204348if_/https://partners.adobe.com/public/developer/en/xml/XFDF_Spec_3.0.pdf XML Forms Data Format Specification, Version 3.0 (2009-08)]
* [https://web.archive.org/web/20160303233354if_/http://partners.adobe.com/public/developer/en/xml/xfdf_2.0.pdf XML Forms Data Format Specification, Version 2.0 (2007-09)]
* [https://web.archive.org/web/20091007170953if_/http://partners.adobe.com:80/public/developer/en/xml/xfdf_2.0_draft.pdf XML Forms Data Format Specification Version 2.0 draft (2003-07)]
* [https://web.archive.org/web/20120422084810if_/http://partners.adobe.com:80/public/developer/en/xml/xfdf.xsd XML schema]

[[Category:XML based file formats]]
[[Category:Adobe]]

XFA

2020-06-09T03:46:16Z

Cypherpunks: fix 2.1 draft 3 link

{{FormatInfo
|formattype=electronic
}}
[[XFA]] ('''XML Forms Architecture''') is a deprecated format used in [[PDF]] files. Most PDF viewers are incompatible and will show only a message such as "The document you are trying to load requires Adobe Reader 8 or higher."

== Specifications ==
* [https://web.archive.org/web/20160329145223if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_3_3.pdf XML Forms Architecture (XFA) Specification, Version 3.3 (2012-01-09)]
* [https://web.archive.org/web/20160329152047/http://partners.adobe.com:80/public/developer/en/xml/xfa_spec_3_1.pdf XML Forms Architecture (XFA) Specification, Version 3.1 (2009-11-16)]
* [https://web.archive.org/web/20160329152422if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_3_0.pdf XML Forms Architecture (XFA) Specification, Version 3.0 (2009-03-12)]
* [https://web.archive.org/web/20160329150533if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_8.pdf XML Forms Architecture (XFA) Specification, Version 2.8 (2008-10-23)]
* [https://web.archive.org/web/20160329150908if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_7.pdf XML Forms Architecture (XFA) Specification, Version 2.7 (2008-02-29)]
* [https://web.archive.org/web/20160329150938if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_6.pdf XML Forms Architecture (XFA) Specification, Version 2.6 (2008-01-25)]
* [https://web.archive.org/web/20160329151210if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_5.pdf XML Forms Architecture (XFA) Specification, Version 2.5 (2007-06-08)]
* [https://web.archive.org/web/20160329152255if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_4.pdf XML Forms Architecture (XFA) Specification, Version 2.4 (2006-09-11)]
* [https://web.archive.org/web/20160329152024if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec.2.2.pdf XML Forms Architecture (XFA) Specification, Version 2.2 (2005-06-07)]
* [https://web.archive.org/web/20160807001610if_/http://partners.adobe.com:80/public/developer/en/xml/xfapecification_2.2_draft.pdf XFA Specification, Version 2.2 draft (2004-10-01)]
* [https://web.archive.org/web/20150706043727if_/http://partners.adobe.com/public/developer/en/xml/xfapecification_2.1_draft3.pdf XFA Specification, Version 2.1 draft 3 (2004-07-01)]

[[Category:XML based file formats]]
[[Category:Adobe]]

XFA

2020-06-09T03:43:45Z

Cypherpunks: link to other spec versions

{{FormatInfo
|formattype=electronic
}}
[[XFA]] ('''XML Forms Architecture''') is a deprecated format used in [[PDF]] files. Most PDF viewers are incompatible and will show only a message such as "The document you are trying to load requires Adobe Reader 8 or higher."

== Specifications ==
* [https://web.archive.org/web/20160329145223if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_3_3.pdf XML Forms Architecture (XFA) Specification, Version 3.3 (2012-01-09)]
* [https://web.archive.org/web/20160329152047/http://partners.adobe.com:80/public/developer/en/xml/xfa_spec_3_1.pdf XML Forms Architecture (XFA) Specification, Version 3.1 (2009-11-16)]
* [https://web.archive.org/web/20160329152422if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_3_0.pdf XML Forms Architecture (XFA) Specification, Version 3.0 (2009-03-12)]
* [https://web.archive.org/web/20160329150533if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_8.pdf XML Forms Architecture (XFA) Specification, Version 2.8 (2008-10-23)]
* [https://web.archive.org/web/20160329150908if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_7.pdf XML Forms Architecture (XFA) Specification, Version 2.7 (2008-02-29)]
* [https://web.archive.org/web/20160329150938if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_6.pdf XML Forms Architecture (XFA) Specification, Version 2.6 (2008-01-25)]
* [https://web.archive.org/web/20160329151210if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_5.pdf XML Forms Architecture (XFA) Specification, Version 2.5 (2007-06-08)]
* [https://web.archive.org/web/20160329152255if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_4.pdf XML Forms Architecture (XFA) Specification, Version 2.4 (2006-09-11)]
* [https://web.archive.org/web/20160329152024if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec.2.2.pdf XML Forms Architecture (XFA) Specification, Version 2.2 (2005-06-07)]
* [https://web.archive.org/web/20160807001610if_/http://partners.adobe.com:80/public/developer/en/xml/xfapecification_2.2_draft.pdf XFA Specification, Version 2.2 draft (2004-10-01)]
* [https://web.archive.org/web/20150916221516if_/http://partners.adobe.com/public/developer/en/xml/xfapecification_2.1_draft3.pdF XFA Specification, Version 2.1 draft 3 (2004-07-01)]

[[Category:XML based file formats]]
[[Category:Adobe]]

XFA

2020-06-09T03:12:24Z

Cypherpunks: fix link to "PDF" page

XFA

2020-06-09T03:11:40Z

Cypherpunks: link to spec via archive.org

{{FormatInfo
|formattype=electronic
}}
[[XFA]] ('''XML Forms Architecture''') is a deprecated format used in [PDF] files. Most PDF viewers are incompatible and will show only a message such as "The document you are trying to load requires Adobe Reader 8 or higher."

== Specifications ==
* [https://web.archive.org/web/20160329151210if_/http://partners.adobe.com/public/developer/en/xml/xfa_spec_2_5.pdf XML Forms Architecture (XFA) Specification, Version 2.5]

[[Category:XML based file formats]]
[[Category:Adobe]]

XFDF

2020-06-09T02:48:12Z

Cypherpunks: fix broken link

SimTower saved game

2016-09-27T03:03:09Z

Cypherpunks: /* util.py */ remove trailing newline

{{FormatInfo
|formattype=electronic
|subcat=Saved Games
|extensions={{ext|tdt}}
|image=Simtower.png
}}

== Links ==

* [https://raw.githubusercontent.com/fabianschuiki/OpenSkyscraper/90e6e92a2d46869a4052332e48633dd1545256e4/doc/simtower/TDT_format.txt Format description from the OpenSkyscraper project]
* [[Wikipedia:SimTower|Wikipedia article on SimTower]]

== Code ==

The following code can parse the format; and even for those who don't know Python it should mostly be understandable as a format description.

Integers are named as <code>(sign)(endian)Int(bits)</code>, like <code>ULInt32</code>. TDT uses little-endian. Exact size and signedness wasn't verified for most fields.

* This requires the Construct package for Python (tested with Construct 2.5.2 and Python 3.5.2).
* It's meant to handle binary-identical roundtripping even with bad values.

=== Usage example ===

If both files below are installed to the current directory, you can run the following at the command line to make all elevators carry 42 people: 
<nowiki>$ python3
>>> import tdt_cons
>>> f = open('SIMTOWER/TDT/TOWER.TDT','rb')
>>> tower = tdt_cons.Tower_Struct.parse_stream(f)
>>> for el in tower.elevators: el.capacity = 42
...
>>> f = open('SIMTOWER/TDT/TOWEROUT.TDT','wb')
>>> tdt_cons.Tower_Struct.build_stream(tower, f)
>>> </nowiki>

=== tdt_cons.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns
import util as _util

#### File header

File_Header_Struct = _cns.Struct('file_header',
# Anchor records the current file position, for sanity checks (Assert).
# Put _cns.Probe() between fields to dump data for debugging.
_cns.Anchor('hdr_start_'),

# SimTower doesn't seem to care about the first byte, but the second
# must be 0x24. Larger values produce a "wrong version" warning.
_cns.Padding(1),
_cns.Const(_cns.ULInt8('version'), 0x24),

_cns.ULInt8('stars'), # 1-5 stars, or 6 for "tower"
_util.Unknown('_hdr3', 1),
_cns.ULInt32('cash_balance'), # units of $100
_cns.ULInt32('other_income'),
_cns.ULInt32('construction_costs'),
_cns.ULInt32('last_quarter_cash'),
_cns.ULInt16('frame_time'),
_cns.ULInt32('day'),
_util.Unknown('_hdr26', 2),
_cns.ULInt8('lobby_height'),
_util.Unknown('_hdr29', 9),
_cns.ULInt16('viewport_x'),
_cns.ULInt16('viewport_y'),
_util.Unknown('_hdr42', 14),
_cns.ULInt8('named_people_count'),

_util.Unknown('_hdr57', 503),
_cns.Anchor('hdr_end_'),
_util.Assert(lambda ctx: ctx.hdr_end_ == ctx.hdr_start_ + 560),
)

#### Tenants

Tenant_Type = _util.NumToSym(_cns.ULInt8('type'), {
0:'floor',
3:'hotel_single',
4:'hotel_double',
5:'hotal_suite',
6:'restaurant',
7:'office',
9:'condo',
10:'shop',
11:'parking_space',
12:'fast_food',
13:'medical',
14:'security',
15:'housekeeping',
18:'movie_theater_floor2', 19:'movie_theater',
20:'recycling_floor2', 21:'recycling',
24:'lobby',
29:'hall_floor2', 30:'hall',
33:'metro',
34:'movie_screen_floor2', 35:'movie_screen',
36:'cathedral_floor5', 37:'cathedral_floor4',
38:'cathedral_floor3', 39:'cathedral_floor2',
40:'cathedral',
44:'parking_ramp',
45:'metro_tunnel',
48:'burned',
})

Tenant_Struct = _cns.Struct('tenant',
# Horizontal positions are given with half-open ranges [left,right)
# in units of 8 pixels.
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('type', Tenant_Type),
_cns.ULInt8('status'),
_cns.ULInt8('pertype_idx'),
_util.Unknown('_unk7', 1),
_cns.ULInt32('people_offset'),
_cns.ULInt8('id'),
_util.Unknown('_unk13', 3),
_cns.ULInt8('rent_class'),
_util.Unknown('_unk17', 1),
)
assert(Tenant_Struct.sizeof() == 18)

#### Floors

def _floor_below_grade(raw_floor):
return raw_floor < 10

def floor_id_from_raw(raw_floor):
# Raw floor values {0..9} are basements: {-10..-1} or {B10..B1}.
# Values {10..119} are floors {1..110}.
if _floor_below_grade(raw_floor):
return raw_floor-10
else:
return raw_floor-9

def _floor_could_be_lobby(raw_floor):
return floor_id_from_raw(raw_floor) in (1, 15, 30, 45, 60, 75, 90)

Floor_Struct = _cns.Struct('floor',
_cns.ULInt16('tenant_count'),
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('tenants',
_cns.Array(lambda ctx: ctx.tenant_count, Tenant_Struct)),
_cns.Array(94, _cns.ULInt16('tenant_id_to_index')),
)

#### People

Person_Struct = _cns.Struct('person',
_cns.ULInt8('tenant_floor'),
_cns.ULInt8('tenant_index'),
_cns.ULInt16('number_in_tenant'),
_cns.Rename('tenant_type', Tenant_Type),
_cns.ULInt8('status'),
_cns.SLInt8('current_floor'),
_util.Unknown('_unk6', 5),
_cns.ULInt16('stress'),
_cns.ULInt16('eval'),
)
assert(Person_Struct.sizeof() == 16)

#### Retail

Retail_Struct = _cns.Struct('retail',
_cns.ULInt8('floor'),
_util.Unknown('_unk1', 10),
_cns.ULInt8('type'),
_util.Unknown('_unk12', 6),
)
assert(Retail_Struct.sizeof() == 18)

#### Stairs

Stairs_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'escalator', 1:'standard'})

Stairs_Struct = _cns.Struct('stairs',
_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Stairs_Type),
_cns.ULInt16('xpos'),
_cns.ULInt8('bottom_floor'),
_util.Unknown('_unk5', 5),
)
assert(Stairs_Struct.sizeof() == 10)

#### Elevators

Elevator_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'express', 1:'standard', 2:'service'})

def _potential_floors_served(ctx):
if ctx.type == 'express':
count = 0
for i in range(ctx.bottom_floor, ctx.top_floor + 1):
if _floor_below_grade(i) or _floor_could_be_lobby(i):
count += 1
else:
count = 1 + (ctx.top_floor - ctx.bottom_floor)
return count

Elevator_Struct = _cns.Struct('elevator',
_cns.Anchor('start_'),

_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Elevator_Type),

# Capacity > 42 may crash SimTower.
# 42 seems to work for non-express elevators.
_cns.ULInt8('capacity'),
_cns.ULInt8('car_count'),

# There are 4 "schedule" arrays of size 14: 7 weekday periods, followed
# by 7 weekend periods. The GUI shows 6 periods per day; the 7th is
# not used. The entire first array looks unused too. The regex
# '\x01{14}......\x05......\x05' can reliably find elevator data.
_cns.Array(14, _cns.ULInt8('#unknown_sched')),
_cns.If(lambda ctx: ctx.present,
_util.Assert(lambda ctx: ctx['#unknown_sched'] == [1]*14)),
_cns.Array(14, _cns.ULInt8('response_distances')),
_cns.Array(14, _cns.ULInt8('express_modes')),
_cns.Array(14, _cns.ULInt8('departure_delays')),

_util.Bool(_cns.ULInt8('shaft_visible')),
_util.Unknown('_unk60', 1),
_cns.ULInt16('left_edge'),
_cns.ULInt8('top_floor'),
_cns.ULInt8('bottom_floor'),
_cns.Value('potential_floors_served_', _potential_floors_served),

# For each of 120 floors, value is 1 if served, otherwise 0.
# Adding express elevator stops on improper floors (above ground and
# not a multiple of 15) may crash SimTower, probably because there's
# an array of size potential_floors_served.
_cns.Array(120, _cns.ULInt8('floor_stops')),

# Resting floor for each car.
_cns.Array(8, _cns.ULInt8('resting_floors')),

_cns.Anchor('fixed_end_'),
_util.Assert(lambda ctx: ctx.fixed_end_ == ctx.start_ + 194),

_cns.If(lambda ctx: ctx.present, _util.AnonEmbed(
_util.Unknown('_unk194', 3488),

# Information about people waiting for elevators
# is likely to be in the following array.
_cns.Array(lambda ctx: ctx.potential_floors_served_,
_util.Unknown('_unk3682', 324)),
)),
_cns.Anchor('end_'),
)

#### Finances/population block

Finances_Struct = _cns.Struct('finances',
_cns.Array(10, _cns.ULInt32('tenant_populations')),
_cns.ULInt32('tower_population'),
_cns.Array(10, _cns.ULInt32('tenant_incomes')),
_cns.ULInt32('tower_income'),
_cns.Array(10, _cns.ULInt32('tenant_maintenance')),
_cns.ULInt32('tower_maintenance'),
)

#### Overall file structure

Tower_Struct = _cns.Struct('tower',
_cns.Embedded(File_Header_Struct),

_cns.Rename('floors', _cns.Array(120, Floor_Struct)),

_cns.ULInt32('people_count'),
_cns.Rename('people',
_cns.Array(lambda ctx: ctx.people_count, Person_Struct)),

_cns.Rename('retail', _cns.Array(512, Retail_Struct)),

_cns.Anchor('elevators_start_'),
_cns.Rename('elevators', _cns.Array(24, Elevator_Struct)),
_cns.Anchor('elevators_end_'),

_util.Unknown('_unk_after_elev', 88),
_cns.Anchor('finances_start_'),
_cns.Embedded(Finances_Struct),
_cns.Anchor('finances_end_'),

_util.Unknown('_unk_after_finpop', 1102),
_cns.Anchor('stairs_start_'),
_cns.Rename('stairs', _cns.Array(64, Stairs_Struct)),
_cns.Anchor('stairs_end_'),

_util.Unknown('_unk_after_stairs', 17242),
_cns.Rename('named_people',
_cns.Array(lambda ctx: ctx.named_people_count,
_util.ZStr(None, 16))),

_cns.Terminator # must reach end of file
)</nowiki>

=== util.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns

#### Utility functions

Unknown = _cns.Field

class Assert(_cns.Construct):
def __init__(self, fn):
_cns.Construct.__init__(self, None)
self.fn = fn
def _sizeof(self, context): return 0
def _assert(self, stream, context): assert self.fn(context)
def _parse(self, stream, context): self._assert(stream, context)
def _build(self, obj, stream, context): self._assert(stream, context)

def AnonEmbed(*subcons):
return _cns.Embedded(_cns.Struct(None, *subcons))

def NumToSym(subcon, decoding):
return _cns.MappingAdapter(subcon, decoding = decoding,
encoding = dict((v,k) for (k,v) in decoding.items()),
decdefault = _cns.Pass, encdefault = _cns.Pass)

def Bool(subcon):
return NumToSym(subcon, {0:False, 1:True})

class _PaddedBytes(bytes):
def __new__(cls, raw=b'', padding=None):
b = super(_PaddedBytes, cls).__new__(cls, raw)
b.padding = padding
return b

class _ZStrAdapter(_cns.StringAdapter):
def __init__(self, subcon):
_cns.StringAdapter.__init__(self, subcon)
def _decode(self, obj, context):
obj = _PaddedBytes(*obj.split(b'\0', 1))
return _cns.StringAdapter._decode(self, obj, context)
def _encode(self, obj, context):
out = _cns.StringAdapter._encode(self, obj, context)
out += b'\0'
try:
if obj.padding is not None:
out += obj.padding
except AttributeError:
pass

pad = self._sizeof(context) - len(out)
if pad < 0:
raise ValueError("string too long")
out += b'\0' * pad

return out

def ZStr(name, length, encoding=None):
fld = _cns.Field(name, length)
return _ZStrAdapter(_cns.StringAdapter(fld, encoding=encoding))</nowiki>

SimTower saved game

2016-09-27T03:01:53Z

Cypherpunks: /* File Format */ add image

File:Simtower.png

2016-09-27T03:00:04Z

Cypherpunks: SimTower screenshot

SimTower screenshot

SimTower saved game

2016-09-27T01:53:54Z

Cypherpunks: /* tdt_cons.py */ fix comment

{{FormatInfo
|formattype=electronic
|subcat=Saved Games
|extensions={{ext|tdt}}
}}

== Links ==

* [https://raw.githubusercontent.com/fabianschuiki/OpenSkyscraper/90e6e92a2d46869a4052332e48633dd1545256e4/doc/simtower/TDT_format.txt Format description from the OpenSkyscraper project]
* [[Wikipedia:SimTower|Wikipedia article on SimTower]]

== Code ==

The following code can parse the format; and even for those who don't know Python it should mostly be understandable as a format description.

Integers are named as <code>(sign)(endian)Int(bits)</code>, like <code>ULInt32</code>. TDT uses little-endian. Exact size and signedness wasn't verified for most fields.

* This requires the Construct package for Python (tested with Construct 2.5.2 and Python 3.5.2).
* It's meant to handle binary-identical roundtripping even with bad values.

=== Usage example ===

If both files below are installed to the current directory, you can run the following at the command line to make all elevators carry 42 people: 
<nowiki>$ python3
>>> import tdt_cons
>>> f = open('SIMTOWER/TDT/TOWER.TDT','rb')
>>> tower = tdt_cons.Tower_Struct.parse_stream(f)
>>> for el in tower.elevators: el.capacity = 42
...
>>> f = open('SIMTOWER/TDT/TOWEROUT.TDT','wb')
>>> tdt_cons.Tower_Struct.build_stream(tower, f)
>>> </nowiki>

=== tdt_cons.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns
import util as _util

#### File header

File_Header_Struct = _cns.Struct('file_header',
# Anchor records the current file position, for sanity checks (Assert).
# Put _cns.Probe() between fields to dump data for debugging.
_cns.Anchor('hdr_start_'),

# SimTower doesn't seem to care about the first byte, but the second
# must be 0x24. Larger values produce a "wrong version" warning.
_cns.Padding(1),
_cns.Const(_cns.ULInt8('version'), 0x24),

_cns.ULInt8('stars'), # 1-5 stars, or 6 for "tower"
_util.Unknown('_hdr3', 1),
_cns.ULInt32('cash_balance'), # units of $100
_cns.ULInt32('other_income'),
_cns.ULInt32('construction_costs'),
_cns.ULInt32('last_quarter_cash'),
_cns.ULInt16('frame_time'),
_cns.ULInt32('day'),
_util.Unknown('_hdr26', 2),
_cns.ULInt8('lobby_height'),
_util.Unknown('_hdr29', 9),
_cns.ULInt16('viewport_x'),
_cns.ULInt16('viewport_y'),
_util.Unknown('_hdr42', 14),
_cns.ULInt8('named_people_count'),

_util.Unknown('_hdr57', 503),
_cns.Anchor('hdr_end_'),
_util.Assert(lambda ctx: ctx.hdr_end_ == ctx.hdr_start_ + 560),
)

#### Tenants

Tenant_Type = _util.NumToSym(_cns.ULInt8('type'), {
0:'floor',
3:'hotel_single',
4:'hotel_double',
5:'hotal_suite',
6:'restaurant',
7:'office',
9:'condo',
10:'shop',
11:'parking_space',
12:'fast_food',
13:'medical',
14:'security',
15:'housekeeping',
18:'movie_theater_floor2', 19:'movie_theater',
20:'recycling_floor2', 21:'recycling',
24:'lobby',
29:'hall_floor2', 30:'hall',
33:'metro',
34:'movie_screen_floor2', 35:'movie_screen',
36:'cathedral_floor5', 37:'cathedral_floor4',
38:'cathedral_floor3', 39:'cathedral_floor2',
40:'cathedral',
44:'parking_ramp',
45:'metro_tunnel',
48:'burned',
})

Tenant_Struct = _cns.Struct('tenant',
# Horizontal positions are given with half-open ranges [left,right)
# in units of 8 pixels.
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('type', Tenant_Type),
_cns.ULInt8('status'),
_cns.ULInt8('pertype_idx'),
_util.Unknown('_unk7', 1),
_cns.ULInt32('people_offset'),
_cns.ULInt8('id'),
_util.Unknown('_unk13', 3),
_cns.ULInt8('rent_class'),
_util.Unknown('_unk17', 1),
)
assert(Tenant_Struct.sizeof() == 18)

#### Floors

def _floor_below_grade(raw_floor):
return raw_floor < 10

def floor_id_from_raw(raw_floor):
# Raw floor values {0..9} are basements: {-10..-1} or {B10..B1}.
# Values {10..119} are floors {1..110}.
if _floor_below_grade(raw_floor):
return raw_floor-10
else:
return raw_floor-9

def _floor_could_be_lobby(raw_floor):
return floor_id_from_raw(raw_floor) in (1, 15, 30, 45, 60, 75, 90)

Floor_Struct = _cns.Struct('floor',
_cns.ULInt16('tenant_count'),
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('tenants',
_cns.Array(lambda ctx: ctx.tenant_count, Tenant_Struct)),
_cns.Array(94, _cns.ULInt16('tenant_id_to_index')),
)

#### People

Person_Struct = _cns.Struct('person',
_cns.ULInt8('tenant_floor'),
_cns.ULInt8('tenant_index'),
_cns.ULInt16('number_in_tenant'),
_cns.Rename('tenant_type', Tenant_Type),
_cns.ULInt8('status'),
_cns.SLInt8('current_floor'),
_util.Unknown('_unk6', 5),
_cns.ULInt16('stress'),
_cns.ULInt16('eval'),
)
assert(Person_Struct.sizeof() == 16)

#### Retail

Retail_Struct = _cns.Struct('retail',
_cns.ULInt8('floor'),
_util.Unknown('_unk1', 10),
_cns.ULInt8('type'),
_util.Unknown('_unk12', 6),
)
assert(Retail_Struct.sizeof() == 18)

#### Stairs

Stairs_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'escalator', 1:'standard'})

Stairs_Struct = _cns.Struct('stairs',
_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Stairs_Type),
_cns.ULInt16('xpos'),
_cns.ULInt8('bottom_floor'),
_util.Unknown('_unk5', 5),
)
assert(Stairs_Struct.sizeof() == 10)

#### Elevators

Elevator_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'express', 1:'standard', 2:'service'})

def _potential_floors_served(ctx):
if ctx.type == 'express':
count = 0
for i in range(ctx.bottom_floor, ctx.top_floor + 1):
if _floor_below_grade(i) or _floor_could_be_lobby(i):
count += 1
else:
count = 1 + (ctx.top_floor - ctx.bottom_floor)
return count

Elevator_Struct = _cns.Struct('elevator',
_cns.Anchor('start_'),

_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Elevator_Type),

# Capacity > 42 may crash SimTower.
# 42 seems to work for non-express elevators.
_cns.ULInt8('capacity'),
_cns.ULInt8('car_count'),

# There are 4 "schedule" arrays of size 14: 7 weekday periods, followed
# by 7 weekend periods. The GUI shows 6 periods per day; the 7th is
# not used. The entire first array looks unused too. The regex
# '\x01{14}......\x05......\x05' can reliably find elevator data.
_cns.Array(14, _cns.ULInt8('#unknown_sched')),
_cns.If(lambda ctx: ctx.present,
_util.Assert(lambda ctx: ctx['#unknown_sched'] == [1]*14)),
_cns.Array(14, _cns.ULInt8('response_distances')),
_cns.Array(14, _cns.ULInt8('express_modes')),
_cns.Array(14, _cns.ULInt8('departure_delays')),

_util.Bool(_cns.ULInt8('shaft_visible')),
_util.Unknown('_unk60', 1),
_cns.ULInt16('left_edge'),
_cns.ULInt8('top_floor'),
_cns.ULInt8('bottom_floor'),
_cns.Value('potential_floors_served_', _potential_floors_served),

# For each of 120 floors, value is 1 if served, otherwise 0.
# Adding express elevator stops on improper floors (above ground and
# not a multiple of 15) may crash SimTower, probably because there's
# an array of size potential_floors_served.
_cns.Array(120, _cns.ULInt8('floor_stops')),

# Resting floor for each car.
_cns.Array(8, _cns.ULInt8('resting_floors')),

_cns.Anchor('fixed_end_'),
_util.Assert(lambda ctx: ctx.fixed_end_ == ctx.start_ + 194),

_cns.If(lambda ctx: ctx.present, _util.AnonEmbed(
_util.Unknown('_unk194', 3488),

# Information about people waiting for elevators
# is likely to be in the following array.
_cns.Array(lambda ctx: ctx.potential_floors_served_,
_util.Unknown('_unk3682', 324)),
)),
_cns.Anchor('end_'),
)

#### Finances/population block

Finances_Struct = _cns.Struct('finances',
_cns.Array(10, _cns.ULInt32('tenant_populations')),
_cns.ULInt32('tower_population'),
_cns.Array(10, _cns.ULInt32('tenant_incomes')),
_cns.ULInt32('tower_income'),
_cns.Array(10, _cns.ULInt32('tenant_maintenance')),
_cns.ULInt32('tower_maintenance'),
)

#### Overall file structure

Tower_Struct = _cns.Struct('tower',
_cns.Embedded(File_Header_Struct),

_cns.Rename('floors', _cns.Array(120, Floor_Struct)),

_cns.ULInt32('people_count'),
_cns.Rename('people',
_cns.Array(lambda ctx: ctx.people_count, Person_Struct)),

_cns.Rename('retail', _cns.Array(512, Retail_Struct)),

_cns.Anchor('elevators_start_'),
_cns.Rename('elevators', _cns.Array(24, Elevator_Struct)),
_cns.Anchor('elevators_end_'),

_util.Unknown('_unk_after_elev', 88),
_cns.Anchor('finances_start_'),
_cns.Embedded(Finances_Struct),
_cns.Anchor('finances_end_'),

_util.Unknown('_unk_after_finpop', 1102),
_cns.Anchor('stairs_start_'),
_cns.Rename('stairs', _cns.Array(64, Stairs_Struct)),
_cns.Anchor('stairs_end_'),

_util.Unknown('_unk_after_stairs', 17242),
_cns.Rename('named_people',
_cns.Array(lambda ctx: ctx.named_people_count,
_util.ZStr(None, 16))),

_cns.Terminator # must reach end of file
)</nowiki>

=== util.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns

#### Utility functions

Unknown = _cns.Field

class Assert(_cns.Construct):
def __init__(self, fn):
_cns.Construct.__init__(self, None)
self.fn = fn
def _sizeof(self, context): return 0
def _assert(self, stream, context): assert self.fn(context)
def _parse(self, stream, context): self._assert(stream, context)
def _build(self, obj, stream, context): self._assert(stream, context)

def AnonEmbed(*subcons):
return _cns.Embedded(_cns.Struct(None, *subcons))

def NumToSym(subcon, decoding):
return _cns.MappingAdapter(subcon, decoding = decoding,
encoding = dict((v,k) for (k,v) in decoding.items()),
decdefault = _cns.Pass, encdefault = _cns.Pass)

def Bool(subcon):
return NumToSym(subcon, {0:False, 1:True})

class _PaddedBytes(bytes):
def __new__(cls, raw=b'', padding=None):
b = super(_PaddedBytes, cls).__new__(cls, raw)
b.padding = padding
return b

class _ZStrAdapter(_cns.StringAdapter):
def __init__(self, subcon):
_cns.StringAdapter.__init__(self, subcon)
def _decode(self, obj, context):
obj = _PaddedBytes(*obj.split(b'\0', 1))
return _cns.StringAdapter._decode(self, obj, context)
def _encode(self, obj, context):
out = _cns.StringAdapter._encode(self, obj, context)
out += b'\0'
try:
if obj.padding is not None:
out += obj.padding
except AttributeError:
pass

pad = self._sizeof(context) - len(out)
if pad < 0:
raise ValueError("string too long")
out += b'\0' * pad

return out

def ZStr(name, length, encoding=None):
fld = _cns.Field(name, length)
return _ZStrAdapter(_cns.StringAdapter(fld, encoding=encoding))
</nowiki>

SimTower saved game

2016-09-27T01:36:58Z

Cypherpunks: /* Usage example */ add section to change elevator capacity

{{FormatInfo
|formattype=electronic
|subcat=Saved Games
|extensions={{ext|tdt}}
}}

== Links ==

* [https://raw.githubusercontent.com/fabianschuiki/OpenSkyscraper/90e6e92a2d46869a4052332e48633dd1545256e4/doc/simtower/TDT_format.txt Format description from the OpenSkyscraper project]
* [[Wikipedia:SimTower|Wikipedia article on SimTower]]

== Code ==

The following code can parse the format; and even for those who don't know Python it should mostly be understandable as a format description.

Integers are named as <code>(sign)(endian)Int(bits)</code>, like <code>ULInt32</code>. TDT uses little-endian. Exact size and signedness wasn't verified for most fields.

* This requires the Construct package for Python (tested with Construct 2.5.2 and Python 3.5.2).
* It's meant to handle binary-identical roundtripping even with bad values.

=== Usage example ===

If both files below are installed to the current directory, you can run the following at the command line to make all elevators carry 42 people: 
<nowiki>$ python3
>>> import tdt_cons
>>> f = open('SIMTOWER/TDT/TOWER.TDT','rb')
>>> tower = tdt_cons.Tower_Struct.parse_stream(f)
>>> for el in tower.elevators: el.capacity = 42
...
>>> f = open('SIMTOWER/TDT/TOWEROUT.TDT','wb')
>>> tdt_cons.Tower_Struct.build_stream(tower, f)
>>> </nowiki>

=== tdt_cons.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns
import util as _util

#### File header

File_Header_Struct = _cns.Struct('file_header',
_cns.Anchor('hdr_start_'), # to debug, change Anchor to Probe

# SimTower doesn't seem to care about the first byte, but the second
# must be 0x24. Larger values produce a "wrong version" warning.
_cns.Padding(1),
_cns.Const(_cns.ULInt8('version'), 0x24),

_cns.ULInt8('stars'), # 1-5 stars, or 6 for "tower"
_util.Unknown('_hdr3', 1),
_cns.ULInt32('cash_balance'), # units of $100
_cns.ULInt32('other_income'),
_cns.ULInt32('construction_costs'),
_cns.ULInt32('last_quarter_cash'),
_cns.ULInt16('frame_time'),
_cns.ULInt32('day'),
_util.Unknown('_hdr26', 2),
_cns.ULInt8('lobby_height'),
_util.Unknown('_hdr29', 9),
_cns.ULInt16('viewport_x'),
_cns.ULInt16('viewport_y'),
_util.Unknown('_hdr42', 14),
_cns.ULInt8('named_people_count'),

_util.Unknown('_hdr57', 503),
_cns.Anchor('hdr_end_'),
_util.Assert(lambda ctx: ctx.hdr_end_ == ctx.hdr_start_ + 560),
)

#### Tenants

Tenant_Type = _util.NumToSym(_cns.ULInt8('type'), {
0:'floor',
3:'hotel_single',
4:'hotel_double',
5:'hotal_suite',
6:'restaurant',
7:'office',
9:'condo',
10:'shop',
11:'parking_space',
12:'fast_food',
13:'medical',
14:'security',
15:'housekeeping',
18:'movie_theater_floor2', 19:'movie_theater',
20:'recycling_floor2', 21:'recycling',
24:'lobby',
29:'hall_floor2', 30:'hall',
33:'metro',
34:'movie_screen_floor2', 35:'movie_screen',
36:'cathedral_floor5', 37:'cathedral_floor4',
38:'cathedral_floor3', 39:'cathedral_floor2',
40:'cathedral',
44:'parking_ramp',
45:'metro_tunnel',
48:'burned',
})

Tenant_Struct = _cns.Struct('tenant',
# Horizontal positions are given with half-open ranges [left,right)
# in units of 8 pixels.
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('type', Tenant_Type),
_cns.ULInt8('status'),
_cns.ULInt8('pertype_idx'),
_util.Unknown('_unk7', 1),
_cns.ULInt32('people_offset'),
_cns.ULInt8('id'),
_util.Unknown('_unk13', 3),
_cns.ULInt8('rent_class'),
_util.Unknown('_unk17', 1),
)
assert(Tenant_Struct.sizeof() == 18)

#### Floors

def _floor_below_grade(raw_floor):
return raw_floor < 10

def floor_id_from_raw(raw_floor):
# Raw floor values {0..9} are basements: {-10..-1} or {B10..B1}.
# Values {10..119} are floors {1..110}.
if _floor_below_grade(raw_floor):
return raw_floor-10
else:
return raw_floor-9

def _floor_could_be_lobby(raw_floor):
return floor_id_from_raw(raw_floor) in (1, 15, 30, 45, 60, 75, 90)

Floor_Struct = _cns.Struct('floor',
_cns.ULInt16('tenant_count'),
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('tenants',
_cns.Array(lambda ctx: ctx.tenant_count, Tenant_Struct)),
_cns.Array(94, _cns.ULInt16('tenant_id_to_index')),
)

#### People

Person_Struct = _cns.Struct('person',
_cns.ULInt8('tenant_floor'),
_cns.ULInt8('tenant_index'),
_cns.ULInt16('number_in_tenant'),
_cns.Rename('tenant_type', Tenant_Type),
_cns.ULInt8('status'),
_cns.SLInt8('current_floor'),
_util.Unknown('_unk6', 5),
_cns.ULInt16('stress'),
_cns.ULInt16('eval'),
)
assert(Person_Struct.sizeof() == 16)

#### Retail

Retail_Struct = _cns.Struct('retail',
_cns.ULInt8('floor'),
_util.Unknown('_unk1', 10),
_cns.ULInt8('type'),
_util.Unknown('_unk12', 6),
)
assert(Retail_Struct.sizeof() == 18)

#### Stairs

Stairs_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'escalator', 1:'standard'})

Stairs_Struct = _cns.Struct('stairs',
_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Stairs_Type),
_cns.ULInt16('xpos'),
_cns.ULInt8('bottom_floor'),
_util.Unknown('_unk5', 5),
)
assert(Stairs_Struct.sizeof() == 10)

#### Elevators

Elevator_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'express', 1:'standard', 2:'service'})

def _potential_floors_served(ctx):
if ctx.type == 'express':
count = 0
for i in range(ctx.bottom_floor, ctx.top_floor + 1):
if _floor_below_grade(i) or _floor_could_be_lobby(i):
count += 1
else:
count = 1 + (ctx.top_floor - ctx.bottom_floor)
return count

Elevator_Struct = _cns.Struct('elevator',
_cns.Anchor('start_'),

_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Elevator_Type),

# Capacity > 42 may crash SimTower.
# 42 seems to work for non-express elevators.
_cns.ULInt8('capacity'),
_cns.ULInt8('car_count'),

# There are 4 "schedule" arrays of size 14: 7 weekday periods, followed
# by 7 weekend periods. The GUI shows 6 periods per day; the 7th is
# not used. The entire first array looks unused too. The regex
# '\x01{14}......\x05......\x05' can reliably find elevator data.
_cns.Array(14, _cns.ULInt8('#unknown_sched')),
_cns.If(lambda ctx: ctx.present,
_util.Assert(lambda ctx: ctx['#unknown_sched'] == [1]*14)),
_cns.Array(14, _cns.ULInt8('response_distances')),
_cns.Array(14, _cns.ULInt8('express_modes')),
_cns.Array(14, _cns.ULInt8('departure_delays')),

_util.Bool(_cns.ULInt8('shaft_visible')),
_util.Unknown('_unk60', 1),
_cns.ULInt16('left_edge'),
_cns.ULInt8('top_floor'),
_cns.ULInt8('bottom_floor'),
_cns.Value('potential_floors_served_', _potential_floors_served),

# For each of 120 floors, value is 1 if served, otherwise 0.
# Adding express elevator stops on improper floors (above ground and
# not a multiple of 15) may crash SimTower, probably because there's
# an array of size potential_floors_served.
_cns.Array(120, _cns.ULInt8('floor_stops')),

# Resting floor for each car.
_cns.Array(8, _cns.ULInt8('resting_floors')),

_cns.Anchor('fixed_end_'),
_util.Assert(lambda ctx: ctx.fixed_end_ == ctx.start_ + 194),

_cns.If(lambda ctx: ctx.present, _util.AnonEmbed(
_util.Unknown('_unk194', 3488),

# Information about people waiting for elevators
# is likely to be in the following array.
_cns.Array(lambda ctx: ctx.potential_floors_served_,
_util.Unknown('_unk3682', 324)),
)),
_cns.Anchor('end_'),
)

#### Finances/population block

Finances_Struct = _cns.Struct('finances',
_cns.Array(10, _cns.ULInt32('tenant_populations')),
_cns.ULInt32('tower_population'),
_cns.Array(10, _cns.ULInt32('tenant_incomes')),
_cns.ULInt32('tower_income'),
_cns.Array(10, _cns.ULInt32('tenant_maintenance')),
_cns.ULInt32('tower_maintenance'),
)

#### Overall file structure

Tower_Struct = _cns.Struct('tower',
_cns.Embedded(File_Header_Struct),

_cns.Rename('floors', _cns.Array(120, Floor_Struct)),

_cns.ULInt32('people_count'),
_cns.Rename('people',
_cns.Array(lambda ctx: ctx.people_count, Person_Struct)),

_cns.Rename('retail', _cns.Array(512, Retail_Struct)),

_cns.Anchor('elevators_start_'),
_cns.Rename('elevators', _cns.Array(24, Elevator_Struct)),
_cns.Anchor('elevators_end_'),

_util.Unknown('_unk_after_elev', 88),
_cns.Anchor('finances_start_'),
_cns.Embedded(Finances_Struct),
_cns.Anchor('finances_end_'),

_util.Unknown('_unk_after_finpop', 1102),
_cns.Anchor('stairs_start_'),
_cns.Rename('stairs', _cns.Array(64, Stairs_Struct)),
_cns.Anchor('stairs_end_'),

_util.Unknown('_unk_after_stairs', 17242),
_cns.Rename('named_people',
_cns.Array(lambda ctx: ctx.named_people_count,
_util.ZStr(None, 16))),

_cns.Terminator # must reach end of file
)</nowiki>

=== util.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns

#### Utility functions

Unknown = _cns.Field

class Assert(_cns.Construct):
def __init__(self, fn):
_cns.Construct.__init__(self, None)
self.fn = fn
def _sizeof(self, context): return 0
def _assert(self, stream, context): assert self.fn(context)
def _parse(self, stream, context): self._assert(stream, context)
def _build(self, obj, stream, context): self._assert(stream, context)

def AnonEmbed(*subcons):
return _cns.Embedded(_cns.Struct(None, *subcons))

def NumToSym(subcon, decoding):
return _cns.MappingAdapter(subcon, decoding = decoding,
encoding = dict((v,k) for (k,v) in decoding.items()),
decdefault = _cns.Pass, encdefault = _cns.Pass)

def Bool(subcon):
return NumToSym(subcon, {0:False, 1:True})

class _PaddedBytes(bytes):
def __new__(cls, raw=b'', padding=None):
b = super(_PaddedBytes, cls).__new__(cls, raw)
b.padding = padding
return b

class _ZStrAdapter(_cns.StringAdapter):
def __init__(self, subcon):
_cns.StringAdapter.__init__(self, subcon)
def _decode(self, obj, context):
obj = _PaddedBytes(*obj.split(b'\0', 1))
return _cns.StringAdapter._decode(self, obj, context)
def _encode(self, obj, context):
out = _cns.StringAdapter._encode(self, obj, context)
out += b'\0'
try:
if obj.padding is not None:
out += obj.padding
except AttributeError:
pass

pad = self._sizeof(context) - len(out)
if pad < 0:
raise ValueError("string too long")
out += b'\0' * pad

return out

def ZStr(name, length, encoding=None):
fld = _cns.Field(name, length)
return _ZStrAdapter(_cns.StringAdapter(fld, encoding=encoding))
</nowiki>

SimTower saved game

2016-09-27T01:23:24Z

Cypherpunks: add notes

{{FormatInfo
|formattype=electronic
|subcat=Saved Games
|extensions={{ext|tdt}}
}}

== Links ==

* [https://raw.githubusercontent.com/fabianschuiki/OpenSkyscraper/90e6e92a2d46869a4052332e48633dd1545256e4/doc/simtower/TDT_format.txt Format description from the OpenSkyscraper project]
* [[Wikipedia:SimTower|Wikipedia article on SimTower]]

== Code ==

The following code can parse the format; and even for those who don't know Python it should mostly be understandable as a format description.

Integers are named as <code>(sign)(endian)Int(bits)</code>, like ULInt32. TDT uses little-endian. Exact size and signedness wasn't verified for most fields.

* This requires the Construct package for Python (tested with Construct 2.5.2 and Python 3.5.2).
* It's meant to handle binary-identical roundtripping even with bad values.
 

=== tdt_cons.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns
import util as _util

#### File header

File_Header_Struct = _cns.Struct('file_header',
_cns.Anchor('hdr_start_'), # to debug, change Anchor to Probe

# SimTower doesn't seem to care about the first byte, but the second
# must be 0x24. Larger values produce a "wrong version" warning.
_cns.Padding(1),
_cns.Const(_cns.ULInt8('version'), 0x24),

_cns.ULInt8('stars'), # 1-5 stars, or 6 for "tower"
_util.Unknown('_hdr3', 1),
_cns.ULInt32('cash_balance'), # units of $100
_cns.ULInt32('other_income'),
_cns.ULInt32('construction_costs'),
_cns.ULInt32('last_quarter_cash'),
_cns.ULInt16('frame_time'),
_cns.ULInt32('day'),
_util.Unknown('_hdr26', 2),
_cns.ULInt8('lobby_height'),
_util.Unknown('_hdr29', 9),
_cns.ULInt16('viewport_x'),
_cns.ULInt16('viewport_y'),
_util.Unknown('_hdr42', 14),
_cns.ULInt8('named_people_count'),

_util.Unknown('_hdr57', 503),
_cns.Anchor('hdr_end_'),
_util.Assert(lambda ctx: ctx.hdr_end_ == ctx.hdr_start_ + 560),
)

#### Tenants

Tenant_Type = _util.NumToSym(_cns.ULInt8('type'), {
0:'floor',
3:'hotel_single',
4:'hotel_double',
5:'hotal_suite',
6:'restaurant',
7:'office',
9:'condo',
10:'shop',
11:'parking_space',
12:'fast_food',
13:'medical',
14:'security',
15:'housekeeping',
18:'movie_theater_floor2', 19:'movie_theater',
20:'recycling_floor2', 21:'recycling',
24:'lobby',
29:'hall_floor2', 30:'hall',
33:'metro',
34:'movie_screen_floor2', 35:'movie_screen',
36:'cathedral_floor5', 37:'cathedral_floor4',
38:'cathedral_floor3', 39:'cathedral_floor2',
40:'cathedral',
44:'parking_ramp',
45:'metro_tunnel',
48:'burned',
})

Tenant_Struct = _cns.Struct('tenant',
# Horizontal positions are given with half-open ranges [left,right)
# in units of 8 pixels.
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('type', Tenant_Type),
_cns.ULInt8('status'),
_cns.ULInt8('pertype_idx'),
_util.Unknown('_unk7', 1),
_cns.ULInt32('people_offset'),
_cns.ULInt8('id'),
_util.Unknown('_unk13', 3),
_cns.ULInt8('rent_class'),
_util.Unknown('_unk17', 1),
)
assert(Tenant_Struct.sizeof() == 18)

#### Floors

def _floor_below_grade(raw_floor):
return raw_floor < 10

def floor_id_from_raw(raw_floor):
# Raw floor values {0..9} are basements: {-10..-1} or {B10..B1}.
# Values {10..119} are floors {1..110}.
if _floor_below_grade(raw_floor):
return raw_floor-10
else:
return raw_floor-9

def _floor_could_be_lobby(raw_floor):
return floor_id_from_raw(raw_floor) in (1, 15, 30, 45, 60, 75, 90)

Floor_Struct = _cns.Struct('floor',
_cns.ULInt16('tenant_count'),
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('tenants',
_cns.Array(lambda ctx: ctx.tenant_count, Tenant_Struct)),
_cns.Array(94, _cns.ULInt16('tenant_id_to_index')),
)

#### People

Person_Struct = _cns.Struct('person',
_cns.ULInt8('tenant_floor'),
_cns.ULInt8('tenant_index'),
_cns.ULInt16('number_in_tenant'),
_cns.Rename('tenant_type', Tenant_Type),
_cns.ULInt8('status'),
_cns.SLInt8('current_floor'),
_util.Unknown('_unk6', 5),
_cns.ULInt16('stress'),
_cns.ULInt16('eval'),
)
assert(Person_Struct.sizeof() == 16)

#### Retail

Retail_Struct = _cns.Struct('retail',
_cns.ULInt8('floor'),
_util.Unknown('_unk1', 10),
_cns.ULInt8('type'),
_util.Unknown('_unk12', 6),
)
assert(Retail_Struct.sizeof() == 18)

#### Stairs

Stairs_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'escalator', 1:'standard'})

Stairs_Struct = _cns.Struct('stairs',
_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Stairs_Type),
_cns.ULInt16('xpos'),
_cns.ULInt8('bottom_floor'),
_util.Unknown('_unk5', 5),
)
assert(Stairs_Struct.sizeof() == 10)

#### Elevators

Elevator_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'express', 1:'standard', 2:'service'})

def _potential_floors_served(ctx):
if ctx.type == 'express':
count = 0
for i in range(ctx.bottom_floor, ctx.top_floor + 1):
if _floor_below_grade(i) or _floor_could_be_lobby(i):
count += 1
else:
count = 1 + (ctx.top_floor - ctx.bottom_floor)
return count

Elevator_Struct = _cns.Struct('elevator',
_cns.Anchor('start_'),

_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Elevator_Type),

# Capacity > 42 may crash SimTower.
# 42 seems to work for non-express elevators.
_cns.ULInt8('capacity'),
_cns.ULInt8('car_count'),

# There are 4 "schedule" arrays of size 14: 7 weekday periods, followed
# by 7 weekend periods. The GUI shows 6 periods per day; the 7th is
# not used. The entire first array looks unused too. The regex
# '\x01{14}......\x05......\x05' can reliably find elevator data.
_cns.Array(14, _cns.ULInt8('#unknown_sched')),
_cns.If(lambda ctx: ctx.present,
_util.Assert(lambda ctx: ctx['#unknown_sched'] == [1]*14)),
_cns.Array(14, _cns.ULInt8('response_distances')),
_cns.Array(14, _cns.ULInt8('express_modes')),
_cns.Array(14, _cns.ULInt8('departure_delays')),

_util.Bool(_cns.ULInt8('shaft_visible')),
_util.Unknown('_unk60', 1),
_cns.ULInt16('left_edge'),
_cns.ULInt8('top_floor'),
_cns.ULInt8('bottom_floor'),
_cns.Value('potential_floors_served_', _potential_floors_served),

# For each of 120 floors, value is 1 if served, otherwise 0.
# Adding express elevator stops on improper floors (above ground and
# not a multiple of 15) may crash SimTower, probably because there's
# an array of size potential_floors_served.
_cns.Array(120, _cns.ULInt8('floor_stops')),

# Resting floor for each car.
_cns.Array(8, _cns.ULInt8('resting_floors')),

_cns.Anchor('fixed_end_'),
_util.Assert(lambda ctx: ctx.fixed_end_ == ctx.start_ + 194),

_cns.If(lambda ctx: ctx.present, _util.AnonEmbed(
_util.Unknown('_unk194', 3488),

# Information about people waiting for elevators
# is likely to be in the following array.
_cns.Array(lambda ctx: ctx.potential_floors_served_,
_util.Unknown('_unk3682', 324)),
)),
_cns.Anchor('end_'),
)

#### Finances/population block

Finances_Struct = _cns.Struct('finances',
_cns.Array(10, _cns.ULInt32('tenant_populations')),
_cns.ULInt32('tower_population'),
_cns.Array(10, _cns.ULInt32('tenant_incomes')),
_cns.ULInt32('tower_income'),
_cns.Array(10, _cns.ULInt32('tenant_maintenance')),
_cns.ULInt32('tower_maintenance'),
)

#### Overall file structure

Tower_Struct = _cns.Struct('tower',
_cns.Embedded(File_Header_Struct),

_cns.Rename('floors', _cns.Array(120, Floor_Struct)),

_cns.ULInt32('people_count'),
_cns.Rename('people',
_cns.Array(lambda ctx: ctx.people_count, Person_Struct)),

_cns.Rename('retail', _cns.Array(512, Retail_Struct)),

_cns.Anchor('elevators_start_'),
_cns.Rename('elevators', _cns.Array(24, Elevator_Struct)),
_cns.Anchor('elevators_end_'),

_util.Unknown('_unk_after_elev', 88),
_cns.Anchor('finances_start_'),
_cns.Embedded(Finances_Struct),
_cns.Anchor('finances_end_'),

_util.Unknown('_unk_after_finpop', 1102),
_cns.Anchor('stairs_start_'),
_cns.Rename('stairs', _cns.Array(64, Stairs_Struct)),
_cns.Anchor('stairs_end_'),

_util.Unknown('_unk_after_stairs', 17242),
_cns.Rename('named_people',
_cns.Array(lambda ctx: ctx.named_people_count,
_util.ZStr(None, 16))),

_cns.Terminator # must reach end of file
)</nowiki>

=== util.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns

#### Utility functions

Unknown = _cns.Field

class Assert(_cns.Construct):
def __init__(self, fn):
_cns.Construct.__init__(self, None)
self.fn = fn
def _sizeof(self, context): return 0
def _assert(self, stream, context): assert self.fn(context)
def _parse(self, stream, context): self._assert(stream, context)
def _build(self, obj, stream, context): self._assert(stream, context)

def AnonEmbed(*subcons):
return _cns.Embedded(_cns.Struct(None, *subcons))

def NumToSym(subcon, decoding):
return _cns.MappingAdapter(subcon, decoding = decoding,
encoding = dict((v,k) for (k,v) in decoding.items()),
decdefault = _cns.Pass, encdefault = _cns.Pass)

def Bool(subcon):
return NumToSym(subcon, {0:False, 1:True})

class _PaddedBytes(bytes):
def __new__(cls, raw=b'', padding=None):
b = super(_PaddedBytes, cls).__new__(cls, raw)
b.padding = padding
return b

class _ZStrAdapter(_cns.StringAdapter):
def __init__(self, subcon):
_cns.StringAdapter.__init__(self, subcon)
def _decode(self, obj, context):
obj = _PaddedBytes(*obj.split(b'\0', 1))
return _cns.StringAdapter._decode(self, obj, context)
def _encode(self, obj, context):
out = _cns.StringAdapter._encode(self, obj, context)
out += b'\0'
try:
if obj.padding is not None:
out += obj.padding
except AttributeError:
pass

pad = self._sizeof(context) - len(out)
if pad < 0:
raise ValueError("string too long")
out += b'\0' * pad

return out

def ZStr(name, length, encoding=None):
fld = _cns.Field(name, length)
return _ZStrAdapter(_cns.StringAdapter(fld, encoding=encoding))
</nowiki>

SimTower saved game

2016-09-27T01:14:12Z

Cypherpunks: add some code to parse the format (TODO: add usage notes)

{{FormatInfo
|formattype=electronic
|subcat=Saved Games
|extensions={{ext|tdt}}
}}

== Links ==

* [https://raw.githubusercontent.com/fabianschuiki/OpenSkyscraper/90e6e92a2d46869a4052332e48633dd1545256e4/doc/simtower/TDT_format.txt Format description from the OpenSkyscraper project]
* [[Wikipedia:SimTower|Wikipedia article on SimTower]]

== Code ==
 

=== tdt_cons.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns
import util as _util

#### File header

File_Header_Struct = _cns.Struct('file_header',
_cns.Anchor('hdr_start_'), # to debug, change Anchor to Probe

# SimTower doesn't seem to care about the first byte, but the second
# must be 0x24. Larger values produce a "wrong version" warning.
_cns.Padding(1),
_cns.Const(_cns.ULInt8('version'), 0x24),

_cns.ULInt8('stars'), # 1-5 stars, or 6 for "tower"
_util.Unknown('_hdr3', 1),
_cns.ULInt32('cash_balance'), # units of $100
_cns.ULInt32('other_income'),
_cns.ULInt32('construction_costs'),
_cns.ULInt32('last_quarter_cash'),
_cns.ULInt16('frame_time'),
_cns.ULInt32('day'),
_util.Unknown('_hdr26', 2),
_cns.ULInt8('lobby_height'),
_util.Unknown('_hdr29', 9),
_cns.ULInt16('viewport_x'),
_cns.ULInt16('viewport_y'),
_util.Unknown('_hdr42', 14),
_cns.ULInt8('named_people_count'),

_util.Unknown('_hdr57', 503),
_cns.Anchor('hdr_end_'),
_util.Assert(lambda ctx: ctx.hdr_end_ == ctx.hdr_start_ + 560),
)

#### Tenants

Tenant_Type = _util.NumToSym(_cns.ULInt8('type'), {
0:'floor',
3:'hotel_single',
4:'hotel_double',
5:'hotal_suite',
6:'restaurant',
7:'office',
9:'condo',
10:'shop',
11:'parking_space',
12:'fast_food',
13:'medical',
14:'security',
15:'housekeeping',
18:'movie_theater_floor2', 19:'movie_theater',
20:'recycling_floor2', 21:'recycling',
24:'lobby',
29:'hall_floor2', 30:'hall',
33:'metro',
34:'movie_screen_floor2', 35:'movie_screen',
36:'cathedral_floor5', 37:'cathedral_floor4',
38:'cathedral_floor3', 39:'cathedral_floor2',
40:'cathedral',
44:'parking_ramp',
45:'metro_tunnel',
48:'burned',
})

Tenant_Struct = _cns.Struct('tenant',
# Horizontal positions are given with half-open ranges [left,right)
# in units of 8 pixels.
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('type', Tenant_Type),
_cns.ULInt8('status'),
_cns.ULInt8('pertype_idx'),
_util.Unknown('_unk7', 1),
_cns.ULInt32('people_offset'),
_cns.ULInt8('id'),
_util.Unknown('_unk13', 3),
_cns.ULInt8('rent_class'),
_util.Unknown('_unk17', 1),
)
assert(Tenant_Struct.sizeof() == 18)

#### Floors

def _floor_below_grade(raw_floor):
return raw_floor < 10

def floor_id_from_raw(raw_floor):
# Raw floor values {0..9} are basements: {-10..-1} or {B10..B1}.
# Values {10..119} are floors {1..110}.
if _floor_below_grade(raw_floor):
return raw_floor-10
else:
return raw_floor-9

def _floor_could_be_lobby(raw_floor):
return floor_id_from_raw(raw_floor) in (1, 15, 30, 45, 60, 75, 90)

Floor_Struct = _cns.Struct('floor',
_cns.ULInt16('tenant_count'),
_cns.ULInt16('left_edge'),
_cns.ULInt16('right_edge'),
_cns.Rename('tenants',
_cns.Array(lambda ctx: ctx.tenant_count, Tenant_Struct)),
_cns.Array(94, _cns.ULInt16('tenant_id_to_index')),
)

#### People

Person_Struct = _cns.Struct('person',
_cns.ULInt8('tenant_floor'),
_cns.ULInt8('tenant_index'),
_cns.ULInt16('number_in_tenant'),
_cns.Rename('tenant_type', Tenant_Type),
_cns.ULInt8('status'),
_cns.SLInt8('current_floor'),
_util.Unknown('_unk6', 5),
_cns.ULInt16('stress'),
_cns.ULInt16('eval'),
)
assert(Person_Struct.sizeof() == 16)

#### Retail

Retail_Struct = _cns.Struct('retail',
_cns.ULInt8('floor'),
_util.Unknown('_unk1', 10),
_cns.ULInt8('type'),
_util.Unknown('_unk12', 6),
)
assert(Retail_Struct.sizeof() == 18)

#### Stairs

Stairs_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'escalator', 1:'standard'})

Stairs_Struct = _cns.Struct('stairs',
_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Stairs_Type),
_cns.ULInt16('xpos'),
_cns.ULInt8('bottom_floor'),
_util.Unknown('_unk5', 5),
)
assert(Stairs_Struct.sizeof() == 10)

#### Elevators

Elevator_Type = _util.NumToSym(_cns.ULInt8('type'),
{0:'express', 1:'standard', 2:'service'})

def _potential_floors_served(ctx):
if ctx.type == 'express':
count = 0
for i in range(ctx.bottom_floor, ctx.top_floor + 1):
if _floor_below_grade(i) or _floor_could_be_lobby(i):
count += 1
else:
count = 1 + (ctx.top_floor - ctx.bottom_floor)
return count

Elevator_Struct = _cns.Struct('elevator',
_cns.Anchor('start_'),

_util.Bool(_cns.ULInt8('present')),
_cns.Rename('type', Elevator_Type),

# Capacity > 42 may crash SimTower.
# 42 seems to work for non-express elevators.
_cns.ULInt8('capacity'),
_cns.ULInt8('car_count'),

# There are 4 "schedule" arrays of size 14: 7 weekday periods, followed
# by 7 weekend periods. The GUI shows 6 periods per day; the 7th is
# not used. The entire first array looks unused too. The regex
# '\x01{14}......\x05......\x05' can reliably find elevator data.
_cns.Array(14, _cns.ULInt8('#unknown_sched')),
_cns.If(lambda ctx: ctx.present,
_util.Assert(lambda ctx: ctx['#unknown_sched'] == [1]*14)),
_cns.Array(14, _cns.ULInt8('response_distances')),
_cns.Array(14, _cns.ULInt8('express_modes')),
_cns.Array(14, _cns.ULInt8('departure_delays')),

_util.Bool(_cns.ULInt8('shaft_visible')),
_util.Unknown('_unk60', 1),
_cns.ULInt16('left_edge'),
_cns.ULInt8('top_floor'),
_cns.ULInt8('bottom_floor'),
_cns.Value('potential_floors_served_', _potential_floors_served),

# For each of 120 floors, value is 1 if served, otherwise 0.
# Adding express elevator stops on improper floors (above ground and
# not a multiple of 15) may crash SimTower, probably because there's
# an array of size potential_floors_served.
_cns.Array(120, _cns.ULInt8('floor_stops')),

# Resting floor for each car.
_cns.Array(8, _cns.ULInt8('resting_floors')),

_cns.Anchor('fixed_end_'),
_util.Assert(lambda ctx: ctx.fixed_end_ == ctx.start_ + 194),

_cns.If(lambda ctx: ctx.present, _util.AnonEmbed(
_util.Unknown('_unk194', 3488),

# Information about people waiting for elevators
# is likely to be in the following array.
_cns.Array(lambda ctx: ctx.potential_floors_served_,
_util.Unknown('_unk3682', 324)),
)),
_cns.Anchor('end_'),
)

#### Finances/population block

Finances_Struct = _cns.Struct('finances',
_cns.Array(10, _cns.ULInt32('tenant_populations')),
_cns.ULInt32('tower_population'),
_cns.Array(10, _cns.ULInt32('tenant_incomes')),
_cns.ULInt32('tower_income'),
_cns.Array(10, _cns.ULInt32('tenant_maintenance')),
_cns.ULInt32('tower_maintenance'),
)

#### Overall file structure

Tower_Struct = _cns.Struct('tower',
_cns.Embedded(File_Header_Struct),

_cns.Rename('floors', _cns.Array(120, Floor_Struct)),

_cns.ULInt32('people_count'),
_cns.Rename('people',
_cns.Array(lambda ctx: ctx.people_count, Person_Struct)),

_cns.Rename('retail', _cns.Array(512, Retail_Struct)),

_cns.Anchor('elevators_start_'),
_cns.Rename('elevators', _cns.Array(24, Elevator_Struct)),
_cns.Anchor('elevators_end_'),

_util.Unknown('_unk_after_elev', 88),
_cns.Anchor('finances_start_'),
_cns.Embedded(Finances_Struct),
_cns.Anchor('finances_end_'),

_util.Unknown('_unk_after_finpop', 1102),
_cns.Anchor('stairs_start_'),
_cns.Rename('stairs', _cns.Array(64, Stairs_Struct)),
_cns.Anchor('stairs_end_'),

_util.Unknown('_unk_after_stairs', 17242),
_cns.Rename('named_people',
_cns.Array(lambda ctx: ctx.named_people_count,
_util.ZStr(None, 16))),

_cns.Terminator # must reach end of file
)</nowiki>

=== util.py ===

<nowiki># To the extent possible under law, the author(s) have dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.
#
# See <http://creativecommons.org/publicdomain/zero/1.0/>.

import construct as _cns

#### Utility functions

Unknown = _cns.Field

class Assert(_cns.Construct):
def __init__(self, fn):
_cns.Construct.__init__(self, None)
self.fn = fn
def _sizeof(self, context): return 0
def _assert(self, stream, context): assert self.fn(context)
def _parse(self, stream, context): self._assert(stream, context)
def _build(self, obj, stream, context): self._assert(stream, context)

def AnonEmbed(*subcons):
return _cns.Embedded(_cns.Struct(None, *subcons))

def NumToSym(subcon, decoding):
return _cns.MappingAdapter(subcon, decoding = decoding,
encoding = dict((v,k) for (k,v) in decoding.items()),
decdefault = _cns.Pass, encdefault = _cns.Pass)

def Bool(subcon):
return NumToSym(subcon, {0:False, 1:True})

class _PaddedBytes(bytes):
def __new__(cls, raw=b'', padding=None):
b = super(_PaddedBytes, cls).__new__(cls, raw)
b.padding = padding
return b

class _ZStrAdapter(_cns.StringAdapter):
def __init__(self, subcon):
_cns.StringAdapter.__init__(self, subcon)
def _decode(self, obj, context):
obj = _PaddedBytes(*obj.split(b'\0', 1))
return _cns.StringAdapter._decode(self, obj, context)
def _encode(self, obj, context):
out = _cns.StringAdapter._encode(self, obj, context)
out += b'\0'
try:
if obj.padding is not None:
out += obj.padding
except AttributeError:
pass

pad = self._sizeof(context) - len(out)
if pad < 0:
raise ValueError("string too long")
out += b'\0' * pad

return out

def ZStr(name, length, encoding=None):
fld = _cns.Field(name, length)
return _ZStrAdapter(_cns.StringAdapter(fld, encoding=encoding))
</nowiki>

Help:Editing

2016-09-27T01:02:53Z

Cypherpunks: link to mediawiki.org

[https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Formatting MediaWiki markup details]

Ogg

2016-09-24T05:27:23Z

Cypherpunks: s/OGG/ogg/ «Ogg is not an acronym and should not be mentioned as "OGG".»

{{FormatInfo
|subcat=Audio and Music
|extensions={{ext|ogg}}, {{ext|ogx}}, {{ext|ogv}}, {{ext|oga}}, {{ext|spx}}, {{ext|ogm}}, {{ext|opus}}
|mimetypes={{mimetype|application/ogg}} {{mimetype|audio/ogg}} {{mimetype|video/ogg}}
|released=1993
}}
'''Ogg''' is a multimedia container format, most commonly used with [[Vorbis]] and other codecs developed by the Xiph.Org Foundation. Ogg files start with the magic number {{magic|"OggS"}}.

Although the initial specification and IANA registration called for the .ogg extension and application/ogg MIME type, regardless of the type of media in the container, the September 2008 RFC 5334 changed that recommendation and registered audio/ogg and video/ogg MIME types with corresponding .oga and .ogv extensions for content that is primarily audio or video respectively. The .ogx extension was introduced for use with content that incorporated the Ogg Skeleton and for which application/ogg remained appropriate. The .ogg extension was grandfathered to refer to Ogg audio with a Vorbis audio encoding. The .spx extension should be used for an Ogg audio file using the Speex codec.

Because Ogg formats are free and open-source, not proprietary as many other formats are, they are preferred by many "free-media" projects including Wikipedia, but this causes some issues for people attempting to view/listen to them, since some devices (e.g., Apple's iOS devices) don't support the Ogg formats, and others (e.g., Windows PCs) don't have "out-of-the-box" support until you install codecs, plug-ins, or software for it. Some of the proprietary formats have wider support in consumer devices in their default configurations.

==Streams that can be placed in Ogg==
* Audio
** [[Speex]]
** [[Vorbis]]
** [[FLAC]]
* Video
** [[Dirac]]
** [[Theora]]
* Other
** [[Kate]]

== Software ==
* [[Konvertor]]
* [http://www.xiph.org/dshow/ Windows Media Player codecs for some Ogg formats]
* [http://xiph.org/quicktime/ Ogg QuickTime components (OSX, Windows)]
* [http://www.mirovideoconverter.com/ Miro Video Converter] (watch out for attempts to install annoying toolbars in installer, as is regretfully common these days)

== Utilities ==
* [http://www.oggconvert.com/ Online audio format converter]

== References ==
* [http://www.xiph.org/ogg/doc/ Ogg documentation from xiph.org]
* [https://wiki.xiph.org/Ogg xiph.org wiki page for Ogg]
* [http://www.xiph.org/ogg/doc/rfc3533.txt RFC 3533: The Ogg Encapsulation Format Version 0]
* [http://www.xiph.org/ogg/doc/rfc5334.txt RFC 5334: Ogg Media types] Redefines application/ogg and registers video/ogg and audio/ogg.
* [[Wikipedia: Ogg]]
* [http://www.digitalpreservation.gov/formats/fdd/fdd000026.shtml Ogg File Format, from Library of Congress resource on Sustainability of Digital Formats]
* [http://hardwarebug.org/2010/03/03/ogg-objections/ Ogg objections] and [https://people.xiph.org/~xiphmont/lj-pseudocut/o-response-1.html Chris Montgomery's response]

[[Category:Video]]

Ogg

2016-09-24T05:25:34Z

Cypherpunks: /* References */ link to xiph wiki

{{FormatInfo
|subcat=Audio and Music
|extensions={{ext|ogg}}, {{ext|ogx}}, {{ext|ogv}}, {{ext|oga}}, {{ext|spx}}, {{ext|ogm}}, {{ext|opus}}
|mimetypes={{mimetype|application/ogg}} {{mimetype|audio/ogg}} {{mimetype|video/ogg}}
|released=1993
}}
'''Ogg''' is a multimedia container format, most commonly used with [[Vorbis]] and other codecs developed by the Xiph.Org Foundation. Ogg files start with the magic number {{magic|"OggS"}}.

Although the initial specification and IANA registration called for the .ogg extension and application/ogg MIME type, regardless of the type of media in the container, the September 2008 RFC 5334 changed that recommendation and registered audio/ogg and video/ogg MIME types with corresponding .oga and .ogv extensions for content that is primarily audio or video respectively. The .ogx extension was introduced for use with content that incorporated the OGG Skeleton and for which application/ogg remained appropriate. The .ogg extension was grandfathered to refer to OGG audio with a Vorbis audio encoding. The .spx extension should be used for an Ogg audio file using the Speex codec.

Because OGG formats are free and open-source, not proprietary as many other formats are, they are preferred by many "free-media" projects including Wikipedia, but this causes some issues for people attempting to view/listen to them, since some devices (e.g., Apple's iOS devices) don't support the OGG formats, and others (e.g., Windows PCs) don't have "out-of-the-box" support until you install codecs, plug-ins, or software for it. Some of the proprietary formats have wider support in consumer devices in their default configurations.

==Streams that can be placed in Ogg==
* Audio
** [[Speex]]
** [[Vorbis]]
** [[FLAC]]
* Video
** [[Dirac]]
** [[Theora]]
* Other
** [[Kate]]

== Software ==
* [[Konvertor]]
* [http://www.xiph.org/dshow/ Windows Media Player codecs for some OGG formats]
* [http://xiph.org/quicktime/ OGG QuickTime components (OSX, Windows)]
* [http://www.mirovideoconverter.com/ Miro Video Converter] (watch out for attempts to install annoying toolbars in installer, as is regretfully common these days)

== Utilities ==
* [http://www.oggconvert.com/ Online audio format converter]

== References ==
* [http://www.xiph.org/ogg/doc/ Ogg documentation from xiph.org]
* [https://wiki.xiph.org/Ogg xiph.org wiki page for Ogg]
* [http://www.xiph.org/ogg/doc/rfc3533.txt RFC 3533: The Ogg Encapsulation Format Version 0]
* [http://www.xiph.org/ogg/doc/rfc5334.txt RFC 5334: Ogg Media types] Redefines application/ogg and registers video/ogg and audio/ogg.
* [[Wikipedia: Ogg]]
* [http://www.digitalpreservation.gov/formats/fdd/fdd000026.shtml Ogg File Format, from Library of Congress resource on Sustainability of Digital Formats]
* [http://hardwarebug.org/2010/03/03/ogg-objections/ Ogg objections] and [https://people.xiph.org/~xiphmont/lj-pseudocut/o-response-1.html Chris Montgomery's response]

[[Category:Video]]

TDT

2016-09-24T05:09:23Z

Cypherpunks: redirect to SimTower saved game

#REDIRECT [[SimTower saved game]]

SimTower saved game

2016-09-24T05:07:29Z

Cypherpunks: link to OpenSkyscraper document