Available on crate feature
lexer only.Expand description
First pass lexer that gets run on the source code and returns a series of tokens with their associated Fragments.
Every byte of the source ends up in exactly one token – whitespace and comments are produced as tokens too (see TokenTy::is_trivia), which lets the parser build a lossless syntax tree.
Modules§
- comments
- Implementation of comment token lexing.
- identifier
- Implementation related to parsing keywords and identifiers.
- integer_
literal - Implementation for lexing integer and float literals.
- quoted
- Lexing implementation for quoted literals.
- token
- Token models.
- trivial
- Trivial tokens and their implementation.
Structs§
- Lexer
- The lexical analyser for wright. This produces a series of tokens that make up the larger elements of the language.