`mojolint.tokenize`

Logical lines: Mojo source with comments removed, string contents blanked, and bracketed continuations joined.

This is all the "parsing" mojolint does. A rule sees a Line — one statement or header, its first physical line number, its indent, the code text, and the identifier words in it — and matches on text. No AST, no types, no dataflow. That ceiling is deliberate; see the README.

Structs

`Line`

struct Line

One logical line of code.

Implements Copyable, Movable

Fields: lineno: Int: 1-based number of the first physical line.; indent: Int: Leading spaces of the first physical line.; code: String: The text, comments stripped, every string literal reduced to "".; words: List[String]: Identifier tokens of code, in order.; allow: List[String]: Rule ids named in a # lint: allow(...) comment on or just above it.; starts: List[Int]: Offset in code where each physical line begins; starts[0] is 0.; indents: List[Int]: Leading spaces of each physical line, parallel to starts.

`locate`

def locate(self, at: Int) -> Tuple[Int, Int]

The 1-based physical line and 1-based column of offset at in code, so a finding on a bracketed continuation points into the physical line it sits on.

`has_word`

def has_word(self, word: String) -> Bool

`allows`

def allows(self, rule: String) -> Bool

Functions

`handed_whole`

def handed_whole(code: String, name: String) -> Bool

Does code pass the local name as a whole value somewhere?

True when an occurrence of name is followed by ) or , (an argument) or . (a method call on it). False when it is only read through: as the base of name[...], or not present at all. Ctx.to(totals) hands totals; Int(cells[unsafe_offset=2]) merely reads a cell of cells.

`words_of`

def words_of(code: String) -> List[String]

The identifier tokens of code, in order.

Numbers are skipped, so 1_000 never becomes a word; string contents are already gone by the time this runs.

`logical_lines`

def logical_lines(source: String) -> List[Line]

Split source into logical lines.

Comments are removed first, then string literals are blanked, then physical lines are joined while brackets are open. Blank lines are dropped. A # lint: allow(...) comment on a line applies to that line; on a line of its own it applies to the next code line.