logo

class

compiler::Tokenize

sys::Obj
  compiler::CompilerSupport
    compiler::CompilerStep
      compiler::Tokenize

Tokenize is responsible for parsing all the source files into a a list of tokens. Each source file is mapped to a CompilationUnit and stored in the PodDef.units field:

Compiler.srcFiles -> Compiler.pod.units

Slots

make

new make(Compiler compiler)

Constructor takes the associated Compiler

run

override Void run()

Run the step on the list of source files

runSource

Void runSource(Location location, Str src)

Run the step on the specified source string

tokenize

CompilationUnit tokenize(Location location, Str src)

Tokenize the source into a CompilationUnit