The lexical analysis is independent of the syntax parsing and the semanticanalysis. The lexical analyzer splits the source text into tokens. Thelexical grammar describes the syntax of these tokens. The grammar is designed tobe suitable for high-speed scanning and to facilitate the implementation of a correctscanner. It has a minimum of special case rules and there is only onephase of translation.
SourceFile:ByteOrderMarkModuleoptShebangModuleoptModuleopt
ByteOrderMark:\uFEFFShebang:#!CharactersoptEndOfShebangEndOfShebang:\u000AEndOfFile
Source text can be encoded as any one of the following:
One of the following UTFBOMs (Byte Order Marks) can be present at the beginning of the source text:
| Format | BOM |
|---|---|
| UTF-8 | EF BB BF |
| UTF-16BE | FE FF |
| UTF-16LE | FF FE |
| UTF-32BE | 00 00 FE FF |
| UTF-32LE | FF FE 00 00 |
| ASCII | no BOM |
If the source file does not begin with a BOM, then the first character mustbe less than or equal to U+0000007F.
The source text is decoded from its source representation into UnicodeCharacters. TheCharacters are further divided into:WhiteSpace,EndOfLine,Comments,SpecialTokenSequences, andTokens, with the source terminated by anEndOfFile.
The source text is split into tokens using the maximal munch algorithm,i.e., the lexical analyzer assumes the longest possible token. For example,>> is a right-shift token rather than two greater-than tokens. There are twoexceptions to this rule:
Character:any Unicode character
EndOfFile:physical end of the file\u0000\u001A
The source text is terminated by whichever comes first.
EndOfLine:\u000D\u000A\u000D\u000A\u2028\u2029EndOfFile
WhiteSpace:SpaceSpaceWhiteSpaceSpace:\u0020\u0009\u000B\u000C
Comment:BlockCommentLineCommentNestingBlockCommentBlockComment:/*Charactersopt*/LineComment://CharactersoptEndOfLineNestingBlockComment:/+NestingBlockCommentCharactersopt+/NestingBlockCommentCharacters:NestingBlockCommentCharacterNestingBlockCommentCharacterNestingBlockCommentCharactersNestingBlockCommentCharacter:CharacterNestingBlockCommentCharacters:CharacterCharacterCharacters
There are three kinds of comments:
The contents of strings and comments are not tokenized. Consequently, comment openings occurring within a string do not begin a comment, and string delimiters within a comment do not affect the recognition of comment closings and nested/+ comment openings. With the exception of/+ occurring within a/+ comment, comment openings within a comment are ignored.
a =/+ // +/ 1;// parses as if 'a = 1;'a =/+ "+/" +/ 1";// parses as if 'a = " +/ 1";'a =/+ /* +/ */ 3;// parses as if 'a = */ 3;'
Comments cannot be used as token concatenators, for example,abc/**/defis two tokens,abc anddef, not oneabcdef token.
Tokens:TokenTokenTokensToken:{}TokenNoBracesTokenNoBraces:IdentifierStringLiteralInterpolationExpressionSequenceCharacterLiteralIntegerLiteralFloatLiteralKeyword//=......&&=&&||=||--=--++=++<<=<<<<=>>=>>=>>>=>>>>>!!=()[]?,;:$===**=%%=^^=^^^^=~~=@=>
Identifier:IdentifierStartIdentifierStartIdentifierCharsIdentifierChars:IdentifierCharIdentifierCharIdentifierCharsIdentifierStart:_LetterUniversalAlphaIdentifierChar:IdentifierStart0NonZeroDigit
Identifiers start with a letter,_, or universal alpha, and arefollowed by any number of letters,_, digits, or universal alphas.Universal alphas are as defined in ISO/IEC 9899:1999(E) Appendix D of the C99 Standard.Identifiers can be arbitrarily long, and are case sensitive.
StringLiteral:WysiwygStringAlternateWysiwygStringDoubleQuotedStringDelimitedStringTokenStringHexString
A string literal is either a wysiwyg quoted string, a double quotedstring, a delimited string, a token string, or a hex string.
In all string literal forms, anEndOfLine is regarded as a single\n character.
WysiwygString:r"WysiwygCharactersopt"StringPostfixoptAlternateWysiwygString:`WysiwygCharactersopt`StringPostfixoptWysiwygCharacters:WysiwygCharacterWysiwygCharacterWysiwygCharactersWysiwygCharacter:CharacterEndOfLine
Wysiwyg ("what you see is what you get") quoted strings can be defined using either of two syntaxes.
In the first form, they are enclosed betweenr" and". All characters between ther" and" are part of the string. There are no escape sequences inside wysiwyg strings.
r"I am Oz"r"c:\games\Sudoku.exe"r"ab\n"// string is 4 characters,// 'a', 'b', '\', 'n'
Alternatively, wysiwyg strings can be enclosed by backquotes, using the ` character.
`the Great and Powerful.``c:\games\Empire.exe``The "lazy" dog``a"b\n`// string is 5 characters,// 'a', '"', 'b', '\', 'n'
See alsoInterpolatedWysiwygLiteral
DoubleQuotedString:"DoubleQuotedCharactersopt"StringPostfixoptDoubleQuotedCharacters:DoubleQuotedCharacterDoubleQuotedCharacterDoubleQuotedCharactersDoubleQuotedCharacter:CharacterEscapeSequenceEndOfLine
Double quoted strings are enclosed by "".EscapeSequences can be embedded in them.
"Who are you?""c:\\games\\Doom.exe""ab\n"// string is 3 characters,// 'a', 'b', and a linefeed"ab"// string is 3 characters,// 'a', 'b', and a linefeed
See alsoInterpolatedDoubleQuotedLiteral
DelimitedString:q"DelimiterWysiwygCharactersoptMatchingDelimiter"StringPostfixoptq"(ParenDelimitedCharactersopt)"StringPostfixoptq"[BracketDelimitedCharactersopt]"StringPostfixoptq"{BraceDelimitedCharactersopt}"StringPostfixoptq"<AngleDelimitedCharactersopt>"StringPostfixoptDelimiter:IdentifierMatchingDelimiter:IdentifierParenDelimitedCharacters:WysiwygCharacterWysiwygCharacterParenDelimitedCharacters(ParenDelimitedCharactersopt)BracketDelimitedCharacters:WysiwygCharacterWysiwygCharacterBracketDelimitedCharacters[BracketDelimitedCharactersopt]BraceDelimitedCharacters:WysiwygCharacterWysiwygCharacterBraceDelimitedCharacters{BraceDelimitedCharactersopt}AngleDelimitedCharacters:WysiwygCharacterWysiwygCharacterAngleDelimitedCharacters<AngleDelimitedCharactersopt>
Delimited strings use various forms of delimiters. The delimiter, whether a character or identifier, must immediately follow the " without any intervening whitespace. The terminating delimiter must immediately precede the closing " without any intervening whitespace. Anesting delimiter nests, and is one of the following characters:
| Delimiter | Matching Delimiter |
|---|---|
| [ | ] |
| ( | ) |
| < | > |
| { | } |
q"(foo(xxx))"// "foo(xxx)"q"[foo{]"// "foo{"
If the delimiter is an identifier, the identifier must be immediately followed by a newline, and the matching delimiter must be the same identifier starting at the beginning of the line:
writeln(q"EOSThisis a multi-lineheredoc stringEOS");The newline following the opening identifier is not part of the string, but the last newline before the closing identifier is part of the string. The closing identifier must be placed on its own line at the leftmost column.
Otherwise, the matching delimiter is the same as the delimiter character:
q"/foo]/"// "foo]"// q"/abc/def/" // error
TokenString:q{TokenStringTokensopt}StringPostfixoptTokenStringTokens:TokenStringTokenTokenStringTokenTokenStringTokensTokenStringToken:TokenNoBraces{TokenStringTokensopt}
Token strings open with the charactersq{ and close with the token}. In between must be valid D tokens. The{ and} tokens nest. The string is formed of all the characters between the opening and closing of the token string, including comments.
q{this is the voice of}// "this is the voice of"q{/*}*/ }// "/*}*/ "q{ world(q{control}); }// " world(q{control}); "q{ __TIME__ }// " __TIME__ "// i.e. it is not replaced with the time// q{ __EOF__ } // error// __EOF__ is not a token, it's end of file
See alsoInterpolatedTokenLiteral
HexString:x"HexStringCharsopt"StringPostfixoptHexStringChars:HexStringCharHexStringCharHexStringCharsHexStringChar:HexDigitWhiteSpaceEndOfLine
Hex strings allow string literals to be created using hex data. The hex data need not form valid UTF characters.
x"0A"// same as "\x0A"x"00 FBCD 32FD 0A"// same as "\x00\xFB\xCD\x32\xFD\x0A"
Whitespace and newlines are ignored, so the hex data can be easily formatted. The number of hex characters must be a multiple of 2.
StringPostfix:cwd
The optionalStringPostfix character gives a specific type to the string, rather than it being inferred from the context. The types corresponding to the postfix characters are:
| Postfix | Type | Alias |
|---|---|---|
| c | immutable(char)[] | string |
| w | immutable(wchar)[] | wstring |
| d | immutable(dchar)[] | dstring |
"hello"c// string"hello"w// wstring"hello"d// dstring
The string literals are assembled as UTF-8 char arrays, and the postfix is applied to convert to wchar or dchar as necessary as a final step.
EscapeSequence:\'\"\?\\\0\a\b\f\n\r\t\v\xHexDigitHexDigit\OctalDigit\OctalDigitOctalDigit\OctalDigitOctalDigitOctalDigit\uHexDigitHexDigitHexDigitHexDigit\UHexDigitHexDigitHexDigitHexDigitHexDigitHexDigitHexDigitHexDigit\NamedCharacterEntityOctalDigit:01234567
| Sequence | Meaning |
|---|---|
| \' | Literal single-quote:' |
| \" | Literal double-quote:" |
| \? | Literal question mark:? |
| \\ | Literal backslash:\ |
| \0 | Binary zero (NUL, U+0000). |
| \a | BEL (alarm) character (U+0007). |
| \b | Backspace (U+0008). |
| \f | Form feed (FF) (U+000C). |
| \n | End-of-line (U+000A). |
| \r | Carriage return (U+000D). |
| \t | Horizontal tab (U+0009). |
| \v | Vertical tab (U+000B). |
| \xnn | Byte value in hexadecimal, wherenn is specified as two hexadecimal digits. For example:\xFF represents the character with the value 255. See also:std.conv.hexString. |
| \n \nn \nnn | Byte value in octal. For example:\101 represents the character with the value 65 ('A'). Analogous to hexadecimal characters, the largest byte value is\377 (=\xFF in hexadecimal or255 in decimal) See also:std.conv.octal. |
| \unnnn | Unicode character U+nnnn, wherennnn are four hexadecimal digits. For example,\u03B3 represents the Unicode character γ (U+03B3 - GREEK SMALL LETTER GAMMA). |
| \Unnnnnnnn | Unicode character U+nnnnnnnn, wherennnnnnnn are 8 hexadecimal digits. For example,\U0001F603 represents the Unicode character U+1F603 (SMILING FACE WITH OPEN MOUTH). |
| \name | Named character entity from the HTML5 specification. These names begin with& and end with;, e.g.,€. SeeNamedCharacterEntity. |
CharacterLiteral:'SingleQuotedCharacter'SingleQuotedCharacter:CharacterEscapeSequence
Character literals are a single character or escape sequence enclosed bysingle quotes.
'h'// the letter h'\n'// newline'\\'// the backslash character
A character literal resolves to one of typechar,wchar, ordchar (seeBasic Data Types).
Otherwise, it resolves to the type with the smallest size it will fit into.
IntegerLiteral:IntegerIntegerIntegerSuffixInteger:DecimalIntegerBinaryIntegerHexadecimalIntegerIntegerSuffix:LuULuLUuLUL
DecimalInteger:0UnderscoresoptNonZeroDigitNonZeroDigitDecimalDigitsUSUnderscores:_Underscores_NonZeroDigit:123456789DecimalDigits:DecimalDigitDecimalDigitDecimalDigitsDecimalDigitsUS:DecimalDigitUSDecimalDigitUSDecimalDigitsUSDecimalDigitsNoSingleUS:DecimalDigitsUSoptDecimalDigitDecimalDigitsUSoptDecimalDigitsNoStartingUS:DecimalDigitDecimalDigitDecimalDigitsUSDecimalDigit:0NonZeroDigitDecimalDigitUS:DecimalDigit_
BinaryInteger:BinPrefixBinaryDigitsNoSingleUSBinPrefix:0b0BBinaryDigitsNoSingleUS:BinaryDigitsUSoptBinaryDigitBinaryDigitsUSoptBinaryDigitsUS:BinaryDigitUSBinaryDigitUSBinaryDigitsUSBinaryDigit:01BinaryDigitUS:BinaryDigit_
HexadecimalInteger:HexPrefixHexDigitsNoSingleUSHexDigits:HexDigitHexDigitHexDigitsHexDigitsUS:HexDigitUSHexDigitUSHexDigitsUSHexDigitsNoSingleUS:HexDigitsUSoptHexDigitHexDigitsUSoptHexDigitsNoStartingUS:HexDigitHexDigitHexDigitsUSHexDigit:DecimalDigitHexLetterHexDigitUS:HexDigit_HexLetter:abcdefABCDEF
Integers can be specified in decimal, binary, or hexadecimal.
10// decimal0b1010// binary0xA// hex
Integers can have embedded ‘_’ characters after a digit to improve readability, which are ignored.
20_000// leagues under the sea867_5309// number on the wall1_522_000// thrust of F1 engine (lbf sea level)0xBAAD_F00D// magic number for debugging
Integers can be immediately followed by one ‘L’ or one of ‘u’ or ‘U’ or both. Note that there is no ‘l’ suffix.
The type of the integer is resolved as follows:
| Literal | Type | Usual decimal notation |
|---|---|
| 0 .. 2_147_483_647 | int |
| 2_147_483_648 .. 9_223_372_036_854_775_807 | long |
| 9_223_372_036_854_775_808 .. 18_446_744_073_709_551_615 | ulong | Explicit suffixes |
| 0L .. 9_223_372_036_854_775_807L | long |
| 0U .. 4_294_967_295U | uint |
| 4_294_967_296U .. 18_446_744_073_709_551_615U | ulong |
| 0UL .. 18_446_744_073_709_551_615UL | ulong | Hexadecimal notation |
| 0x0 .. 0x7FFF_FFFF | int |
| 0x8000_0000 .. 0xFFFF_FFFF | uint |
| 0x1_0000_0000 .. 0x7FFF_FFFF_FFFF_FFFF | long |
| 0x8000_0000_0000_0000 .. 0xFFFF_FFFF_FFFF_FFFF | ulong | Hexadecimal notation with explicit suffixes |
| 0x0L .. 0x7FFF_FFFF_FFFF_FFFFL | long |
| 0x8000_0000_0000_0000L .. 0xFFFF_FFFF_FFFF_FFFFL | ulong |
| 0x0U .. 0xFFFF_FFFFU | uint |
| 0x1_0000_0000U .. 0xFFFF_FFFF_FFFF_FFFFU | ulong |
| 0x0UL .. 0xFFFF_FFFF_FFFF_FFFFUL | ulong |
An integer literal may not exceed these values.
FloatLiteral:FloatSuffixoptIntegerFloatSuffixImaginarySuffixoptIntegerRealSuffixoptImaginarySuffixFloat:DecimalFloatHexFloatDecimalFloat:LeadingDecimal.DecimalDigitsNoStartingUSoptLeadingDecimal.DecimalDigitsNoStartingUSDecimalExponent.DecimalDigitsNoStartingUSDecimalExponentoptLeadingDecimalDecimalExponentDecimalExponent:DecimalExponentStartDecimalDigitsNoSingleUSDecimalExponentStart:eEe+E+e-E-HexFloat:HexPrefixHexDigitsNoSingleUS.HexDigitsNoStartingUSHexExponentHexPrefix.HexDigitsNoStartingUSHexExponentHexPrefixHexDigitsNoSingleUSHexExponentHexPrefixHexExponentHexPrefix:0x0XHexExponent:HexExponentStartDecimalDigitsNoSingleUSHexExponentStart:pPp+P+p-P-Suffix:FloatSuffixImaginarySuffixoptRealSuffixImaginarySuffixoptImaginarySuffixFloatSuffix:fFRealSuffix:LImaginarySuffix:iLeadingDecimal:DecimalInteger0DecimalDigitsNoSingleUS
Floats can be in decimal or hexadecimal format, and must have at least one digit and either a decimal point, an exponent, or aFloatSuffix.
Decimal floats can have an exponent which ise orE followed by a decimal number serving as the exponent of 10.
-1.01e2// 100.01e-2// 0.01-1.175494351e-38F// float.min
Hexadecimal floats are preceded by a0x or0X and the exponent is ap orP followed by a decimal number serving as the exponent of 2.
0xAp0// 10.00x1p2// 4.00x1.FFFFFFFFFFFFFp1023// double.max0x1p-52// double.epsilon
Floating literals can have embedded_ characters after a digit to improve readability, which are ignored.
2.645_7516.022140857E+236_022.140857E+206_022_.140_857E+20_
0.0// double0F// float0.0L// real
The literal may not exceed the range of the type. The literal is rounded to fit into the significant digits of the type.
If a floating literal has a. and a type suffix, at least one digit must be in-between:
1f;// OK, float1.f;// error1.;// OK, double
Keywords are reserved identifiers.
Keyword:abstractaliasalignasmassertautobodyboolbreakbytecasecastcatchcdoublecentcfloatcharclassconstcontinuecrealdchardebugdefaultdelegatedeletedeprecateddodoubleelseenumexportexternfalsefinalfinallyfloatforforeachforeach_reversefunctiongotoidoubleififloatimmutableimportininoutintinterfaceinvariantirealislazylongmacromixinmodulenewnothrownulloutoverridepackagepragmaprivateprotectedpublicpurerealrefreturnscopesharedshortstaticstructsuperswitchsynchronizedtemplatethisthrowtruetrytypeidtypeofubyteucentuintulongunionunittestushortversionvoidwcharwhilewith__FILE____FILE_FULL_PATH____FUNCTION____LINE____MODULE____PRETTY_FUNCTION____gshared__parameters__rvalue__traits__vector
These tokens are replaced with other tokens according to the following table:
| Special Token | Replaced with |
|---|---|
| __DATE__ | string literal of the date of compilation "mmm dd yyyy" |
| __EOF__ | tells the scanner to ignore everything after this token |
| __TIME__ | string literal of the time of compilation "hh:mm:ss" |
| __TIMESTAMP__ | string literal of the date and time of compilation "www mmm dd hh:mm:ss yyyy" |
| __VENDOR__ | Compiler vendor string |
| __VERSION__ | Compiler version as an integer |
SpecialTokenSequence:# lineIntegerLiteralFilespecoptEndOfLine# line__LINE__FilespecoptEndOfLine
Filespec:"DoubleQuotedCharactersopt"
Special token sequences are processed by the lexical analyzer, may appear between any other tokens, and do not affect the syntax parsing.
Special token sequences are terminated by the first newline that follows the first# token at the beginning of the sequence.
There is currently only one special token sequence,#line.
This sets the line number of the next source line toIntegerLiteral, and optionally the current source file name toFilespec, beginning with the next line of source text.
For example:
int #line 6 "pkg/mod.d"x;// this is now line 6 of file pkg/mod.d