Byte Order Mark - Revision history

SamSamuel: /* Byte patterns of common BOMs */

2025-03-26T01:11:02Z

‎Byte patterns of common BOMs

Jsummers at 22:01, 4 March 2016

2016-03-04T22:01:08Z

Jsummers: Rewrote & expanded

2013-02-22T20:22:39Z

Rewrote & expanded

Dan Tobias at 14:49, 9 February 2013

2013-02-09T14:49:45Z

Dan Tobias at 01:49, 11 January 2013

2013-01-11T01:49:29Z

Dan Tobias: /* Introduction */

2012-11-28T01:10:53Z

‎Introduction

Dan Tobias: Add infobox

2012-11-27T13:38:54Z

Add infobox

Dan Tobias: /* Introduction */

2012-11-22T18:56:36Z

‎Introduction

Dan Tobias: /* Introduction */

2012-11-22T18:56:00Z

‎Introduction

Dan Tobias at 16:21, 22 November 2012

2012-11-22T16:21:24Z

← Older revision		Revision as of 01:11, 26 March 2025
Line 24:		Line 24:

	<code>0xEF 0xBB 0xBF</code> suggests [[UTF-8]].		<code>0xEF 0xBB 0xBF</code> suggests [[UTF-8]].
		+
		+	<code>0x0E 0xFE 0xFF</code> suggests [[SCSU]].

	== UTF-8 ==		== UTF-8 ==

@@ Line 1: / Line 1: @@
 {{FormatInfo
 |formattype=electronic
-|subcat=Character Encodings
+|subcat=Character encoding
 }}
-==Introduction==
+== Discussion ==
-−
-A '''Byte Order Mark''' ('''BOM''') is a strategically-placed U+FEFF (ZERO WIDTH NO-BREAK SPACE) character at the beginning of a [[Unicode]] text file, or other block of Unicode text.
 There are two main schools of thought as to its purpose:

@@ Line 6: / Line 6: @@
 ==Introduction==
-The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings|character encoding]]. It was designed to deal with the [[Endianness|"big-endian vs. little-endian"]] problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set (officially designated "Zero-width non-break space", which is completely invisible when printed and has no effect on rendering, spacing, or breaking of adjacent characters), where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
+A '''Byte Order Mark''' ('''BOM''') is a strategically-placed U+FEFF (ZERO WIDTH NO-BREAK SPACE) character at the beginning of a [[Unicode]] text file, or other block of Unicode text.
-Hence, if you examine the raw bytes of a file you believe to be a 16-bit-encoded text file and its first two bytes are FE and FF in that order, this indicates that the order is big-endian, while if the first two bytes are FF then FE, it is little-endian.
+There are two main schools of thought as to its purpose:
-Some [[UTF-8]] files (including [[CSV]] files) are written with a prepending BOM consisting of 3 bytes: <code>EF BB BF</code>.
+The idea of a BOM is undeniably a ''hack'', but its benefits sometimes outweigh its drawbacks.
-"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature." [http://en.wikipedia.org/wiki/Byte-order_mark#cite_note-2]
+To make false positives less likely, the U+FFFE code point is permanently reserved, and will never be a meaningful code point.
-== References ==
+Other usage of the U+FEFF character is deprecated, and U+2060 WORD JOINER is suggested instead.
-* [http://en.wikipedia.org/wiki/Byte-order_mark Byte-order mark (Wikipedia)]
 [[Category:File format details]]

@@ Line 6: / Line 6: @@
 ==Introduction==
-The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings|character encoding]]. It was designed to deal with the [[Endianness|"big-endian vs. little-endian"]] problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set, where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
+The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings|character encoding]]. It was designed to deal with the [[Endianness|"big-endian vs. little-endian"]] problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set (officially designated "Zero-width non-break space", which is completely invisible when printed and has no effect on rendering, spacing, or breaking of adjacent characters), where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
 Hence, if you examine the raw bytes of a file you believe to be a 16-bit-encoded text file and its first two bytes are FE and FF in that order, this indicates that the order is big-endian, while if the first two bytes are FF then FE, it is little-endian.

@@ Line 6: / Line 6: @@
 ==Introduction==
-The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings|character encoding]]. It was designed to deal with the "big-endian vs. little-endian" problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set, where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
+The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings|character encoding]]. It was designed to deal with the [[Endianness|"big-endian vs. little-endian"]] problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set, where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
 Hence, if you examine the raw bytes of a file you believe to be a 16-bit-encoded text file and its first two bytes are FE and FF in that order, this indicates that the order is big-endian, while if the first two bytes are FF then FE, it is little-endian.

← Older revision		Revision as of 01:10, 28 November 2012
Line 7:		Line 7:

	The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings\|character encoding]]. It was designed to deal with the "big-endian vs. little-endian" problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set, where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.		The '''Byte Order Mark''' (BOM) is a header that is added sometimes to some type of textual formats such as [[CSV]] to have applications recognize the right [[Character Encodings\|character encoding]]. It was designed to deal with the "big-endian vs. little-endian" problem of expressing multi-byte numeric data, where some systems put the highest-order byte first and others put it last. This affects 16-bit character encodings. The BOM has been allocated a character position (U+FEFF) in the [[Unicode]] character set, where the corresponding character with the two bytes of the 16-bit code point reversed (U+FFFE) is reserved and guaranteed against having a different meaning allocated by Unicode. This means that if the reversed version is encountered, the file is known to be in the opposite byte order than was previously assumed, and processing should restart at the beginning of the file with the proper order set. Usually it's only processed in this manner if it's the first character of the file, or else processing could get in an infinite loop if both versions appear somewhere in the file.
		+
		+	Hence, if you examine the raw bytes of a file you believe to be a 16-bit-encoded text file and its first two bytes are FE and FF in that order, this indicates that the order is big-endian, while if the first two bytes are FF then FE, it is little-endian.

	Some [[UTF-8]] files (including [[CSV]] files) are written with a prepending BOM consisting of 3 bytes: <code>EF BB BF</code>.		Some [[UTF-8]] files (including [[CSV]] files) are written with a prepending BOM consisting of 3 bytes: <code>EF BB BF</code>.

← Older revision		Revision as of 13:38, 27 November 2012
Line 1:		Line 1:
		+	{{FormatInfo
		+	\|formattype=electronic
		+	\|subcat=Character Encodings
		+	}}
		+
	==Introduction==		==Introduction==