Atari BASIC tokenized file - Revision history

Dexvertbot: Change telparia.com samples link to template

2023-12-28T04:08:52Z

Change telparia.com samples link to template

Sembiance: Added another detokenizer and sample files

2021-01-16T15:55:22Z

Added another detokenizer and sample files

Maury Markowitz at 17:30, 28 February 2020

2020-02-28T17:30:01Z

Maury Markowitz at 17:27, 28 February 2020

2020-02-28T17:27:20Z

Dan Tobias: /* References */

2014-02-13T00:39:17Z

‎References

Dan Tobias: /* References */

2013-04-08T03:11:21Z

‎References

Dan Tobias: /* References */

2013-03-13T04:21:12Z

‎References

Dan Tobias: /* References */

2013-01-14T04:39:37Z

‎References

Dan Tobias at 19:33, 6 January 2013

2013-01-06T19:33:10Z

Dan Tobias at 12:40, 24 December 2012

2012-12-24T12:40:13Z

@@ Line 20: / Line 20: @@
 == Sample files ==
-* https://telparia.com/fileFormatSamples/text/gfaBASICAtari/
+* {{DexvertSamples|other/gfaBASICAtari}}
 == References ==

@@ Line 17: / Line 17: @@
 == Software ==
 * [http://joyfulcoder.com/memopad/ Atari Memopad: detokenizes BASIC programs among other functions]
 == References ==

@@ Line 7: / Line 7: @@
 '''Atari BASIC''' was used on the Atari 400 and 800 computers, among the many systems competing for the home computer market in the late '70s and early '80s. While Atari considered using an adapted Microsoft BASIC like some other manufacturers, they ultimately used an independently-developed BASIC instead, meaning that many characteristics of this BASIC (including its manner of tokenization) differ greatly from the other BASICs of the time.
-To understand the differences in the file format, a quick review of other BASICs will be helpful. Some BASICs, notable TinyBASIC, performed very little conversion of the program code typed in by the user. All it did was read the line number, if present, and converted that to an 8-bit integer value. This made it easier to search for in GOTOs and such. The rest of the line was stored exactly as it had been typed, in ASCII format with a trailing CR. Microsoft-like BASICs went slightly further, first converting the line number to a 16-bit value, and then converting the first word on the line to an 8-bit value, the token. This makes it easier to look up the runtime code associated with the instruction, it doesn't have to convert from text form. The rest of the line, as in Tiny, was left as the original text.
+To understand the differences in the file format, a quick review of other BASICs will be helpful. Some BASICs, notably TinyBASIC, performed very little conversion of the program code typed in by the user. All it did was read the line number, if present, and converted that to an 8-bit integer value. This made it easier to search for in GOTOs and such. The rest of the line was stored exactly as it had been typed, in ASCII format with a trailing CR. Microsoft-like BASICs went slightly further, first converting the line number to a 16-bit value, and then converting the first word on the line to an 8-bit value, the token. This makes it easier to look up the runtime code associated with the instruction, it doesn't have to convert from text form. The rest of the line, as in Tiny, was left as the original text.
 In contrast, Atari BASIC tokenized every item in the line to an internal format, thereby eliminating any runtime parsing. For instance, any numbers in the code were converted to their five-byte floating-point format and put into memory in that form, with a lead token to indicate it's a numeric constant, eliminating the need to do any conversion at runtime. So while one can read the tokenized form of Tiny or MS with some ability to understand what is going on, Atari BASIC files are completely binary. The only thing that could be read as-is were string literals. String constants were marked by the byte 0F (hex), followed by a byte giving the string length (0-255), then the characters of the string itself. Numeric constants were marked by 0E (hex), followed by six bytes holding a floating-point value.

@@ Line 7: / Line 7: @@
 '''Atari BASIC''' was used on the Atari 400 and 800 computers, among the many systems competing for the home computer market in the late '70s and early '80s. While Atari considered using an adapted Microsoft BASIC like some other manufacturers, they ultimately used an independently-developed BASIC instead, meaning that many characteristics of this BASIC (including its manner of tokenization) differ greatly from the other BASICs of the time.
-While most BASICs used some byte values to represent literal characters (in string constants and variable names, for instance) and others (often in the "high-bit" range of #128-#255) for tokenized keywords, Atari BASIC used a more complicated scheme whereby values all throughout the 256 possibilities were used for tokens of multiple classes. Context determined the interpretation of a value. If it was the first token of a line, its meaning was taken from a list of Statement Name Tokens. At other positions, a different token list was used which included values representing functions, operators, or variables. The variables in the program were itemized in a variable table stored at the beginning of the program, so that references to a variable in the program used only the single-byte token, representing the name. There were 128 positions in the token list for variables (comprising the high-bit values), meaning that only 128 different variables could be used in a program.
+To understand the differences in the file format, a quick review of other BASICs will be helpful. Some BASICs, notable TinyBASIC, performed very little conversion of the program code typed in by the user. All it did was read the line number, if present, and converted that to an 8-bit integer value. This made it easier to search for in GOTOs and such. The rest of the line was stored exactly as it had been typed, in ASCII format with a trailing CR. Microsoft-like BASICs went slightly further, first converting the line number to a 16-bit value, and then converting the first word on the line to an 8-bit value, the token. This makes it easier to look up the runtime code associated with the instruction, it doesn't have to convert from text form. The rest of the line, as in Tiny, was left as the original text.
-String constants were marked by the byte 0F (hex), followed by a byte giving the string length (0-255), then the characters of the string itself. Numeric constants were marked by 0E (hex), followed by six bytes holding a floating point value.
+In contrast, Atari BASIC tokenized every item in the line to an internal format, thereby eliminating any runtime parsing. For instance, any numbers in the code were converted to their five-byte floating-point format and put into memory in that form, with a lead token to indicate it's a numeric constant, eliminating the need to do any conversion at runtime. So while one can read the tokenized form of Tiny or MS with some ability to understand what is going on, Atari BASIC files are completely binary. The only thing that could be read as-is were string literals. String constants were marked by the byte 0F (hex), followed by a byte giving the string length (0-255), then the characters of the string itself. Numeric constants were marked by 0E (hex), followed by six bytes holding a floating-point value.
 Literal characters are in [[ATASCII]], Atari's not-quite-ASCII character set.

@@ Line 21: / Line 21: @@
 * [http://archive.org/details/ataribooks Books on Atari programming] in Internet Archive
 * [http://archive.org/details/bitsavers_atari40080mputerTechnicalReferenceNotes1982_20170986 Atari technical manual] - Lots of tech details about Atari 400/800
 [[Category:Atari computers]]

← Older revision		Revision as of 19:33, 6 January 2013
Line 10:		Line 10:

	String constants were marked by the byte 0F (hex), followed by a byte giving the string length (0-255), then the characters of the string itself. Numeric constants were marked by 0E (hex), followed by six bytes holding a floating point value.		String constants were marked by the byte 0F (hex), followed by a byte giving the string length (0-255), then the characters of the string itself. Numeric constants were marked by 0E (hex), followed by six bytes holding a floating point value.
		+
		+	Literal characters are in [[ATASCII]], Atari's not-quite-ASCII character set.

	== Software ==		== Software ==

← Older revision		Revision as of 12:40, 24 December 2012
Line 16:		Line 16:
	== References ==		== References ==
	* [http://users.telenet.be/kim1-6502/6502/absb.html Atari BASIC Source Book] - has assembly source and some descriptions of what it does, from which a complete token table and file format spec could be puzzled out, though the required info is spread out somewhat.		* [http://users.telenet.be/kim1-6502/6502/absb.html Atari BASIC Source Book] - has assembly source and some descriptions of what it does, from which a complete token table and file format spec could be puzzled out, though the required info is spread out somewhat.
		+
		+	[[Category:Atari computers]]