pub enum TokenTy {
Show 120 variants
LeftCurly,
RightCurly,
LeftBracket,
RightBracket,
LeftParen,
RightParen,
Plus,
PlusEq,
Star,
StarEq,
Div,
DivEq,
Xor,
XorEq,
Mod,
ModEq,
Bang,
BangEq,
Minus,
MinusEq,
SingleArrow,
Eq,
EqEq,
DoubleArrow,
Lt,
LtEq,
LtLt,
LtLtEq,
Gt,
GtEq,
GtGt,
GtGtEq,
And,
AndEq,
AndAnd,
Or,
OrEq,
OrOr,
Colon,
ColonEq,
ColonColon,
Dot,
DotDot,
DotDotEq,
At,
Tilde,
Semi,
Comma,
Hash,
Question,
Dollar,
Underscore,
Identifier,
LineComment,
BlockComment,
OuterDocComment,
OuterBlockDocComment,
InnerDocComment,
InnerBlockDocComment,
UnterminatedBlockComment,
KwRecord,
KwType,
KwEnum,
KwUnion,
KwFunc,
KwPure,
KwNaked,
KwUnsafe,
KwRepr,
KwImpl,
KwConstrain,
KwConstraint,
KwReferences,
KwTrait,
KwUse,
KwAs,
KwConst,
KwMod,
KwIf,
KwElse,
KwMatch,
KwFor,
KwIn,
KwWhile,
KwTrue,
KwFalse,
KwLoop,
KwWhere,
KwPub,
KwReturn,
KwBreak,
KwContinue,
KwLet,
KwVar,
KwBool,
KwU8,
KwI8,
KwU16,
KwI16,
KwU32,
KwI32,
KwF32,
KwU64,
KwI64,
KwF64,
KwChar,
IntegerLiteral,
FloatLiteral,
StringLiteral,
FormatStringLiteral,
CharLiteral,
UnterminatedStringLiteral,
UnterminatedFormatStringLiteral,
UnterminatedCharLiteral,
Whitespace,
Unknown,
EOF,
EOFExpr,
EOFTy,
Error,
}lexer only.Expand description
The different types of tokens in wright source.
This enum is also used directly as the terminal type of the generated parser (see crate::parser), which is why
every variant is field-less and the names line up with the token names in src/parser/wright.llw.
Variants§
LeftCurly
RightCurly
LeftBracket
RightBracket
LeftParen
RightParen
Plus
PlusEq
Star
StarEq
Div
DivEq
Xor
XorEq
Mod
ModEq
Bang
BangEq
Minus
MinusEq
SingleArrow
Eq
EqEq
DoubleArrow
Lt
LtEq
LtLt
LtLtEq
Gt
GtEq
GtGt
GtGtEq
And
AndEq
AndAnd
Or
OrEq
OrOr
Colon
ColonEq
ColonColon
Dot
DotDot
DotDotEq
At
Tilde
Semi
Comma
Hash
Question
Dollar
Underscore
Identifier
LineComment
A non-documenting // ... comment.
BlockComment
A non-documenting /* ... */ comment.
OuterDocComment
A /// ... comment documenting the item that follows it.
OuterBlockDocComment
A /** ... */ comment documenting the item that follows it.
InnerDocComment
A //! ... comment documenting the enclosing item (or file).
InnerBlockDocComment
A /*! ... */ comment documenting the enclosing item (or file).
UnterminatedBlockComment
Indicates a block style comment without termination. Separate from the other comment variants to indicate that unterminated comments will be handled differently (they produce errors).
KwRecord
KwType
KwEnum
KwUnion
KwFunc
KwPure
KwNaked
KwUnsafe
KwRepr
KwImpl
KwConstrain
KwConstraint
KwReferences
KwTrait
KwUse
KwAs
KwConst
KwMod
KwIf
KwElse
KwMatch
KwFor
KwIn
KwWhile
KwTrue
KwFalse
KwLoop
KwWhere
KwPub
KwReturn
KwBreak
KwContinue
KwLet
KwVar
KwBool
KwU8
KwI8
KwU16
KwI16
KwU32
KwI32
KwF32
KwU64
KwI64
KwF64
KwChar
IntegerLiteral
FloatLiteral
StringLiteral
FormatStringLiteral
CharLiteral
UnterminatedStringLiteral
A string literal that reached the end of the source before its closing ".
UnterminatedFormatStringLiteral
A format string literal that reached the end of the source before its closing `.
UnterminatedCharLiteral
A character literal that reached the end of the source before its closing '.
Whitespace
Whitespace counts as a token.
Unknown
Unknown character in lexer fragment.
EOF
End of input. Never produced by the Lexer – used internally by the parser.
EOFExpr
End of input when parsing a single expression. Never produced by the Lexer – used internally by the parser.
EOFTy
End of input when parsing a single type signature. Never produced by the Lexer – used internally by the parser.
Error
A token that could not be lexed. Never produced by the Lexer – the parser converts TokenTy::Unknown tokens into this variant, which is always skipped during parsing.
Implementations§
Source§impl TokenTy
impl TokenTy
Sourcepub const fn is_trivia(self) -> bool
pub const fn is_trivia(self) -> bool
Is this token whitespace or a comment of some kind (including doc comments and unterminated comments)?
Sourcepub const fn is_outer_doc_comment(self) -> bool
pub const fn is_outer_doc_comment(self) -> bool
Is this token a documentation comment that documents the item following it (/// or /** */)?
Sourcepub const fn is_inner_doc_comment(self) -> bool
pub const fn is_inner_doc_comment(self) -> bool
Is this token a documentation comment that documents the enclosing item (//! or /*! */)?
Trait Implementations§
impl Copy for TokenTy
impl Eq for TokenTy
impl StructuralPartialEq for TokenTy
Auto Trait Implementations§
impl Forget for TokenTy
impl Freeze for TokenTy
impl RefUnwindSafe for TokenTy
impl Send for TokenTy
impl Sync for TokenTy
impl Unpin for TokenTy
impl UnsafeUnpin for TokenTy
impl UnwindSafe for TokenTy
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more