Fan

 

class

compiler::Tokenize

sys::Obj
  compiler::CompilerSupport
    compiler::CompilerStep
      compiler::Tokenize

Tokenize is responsible for parsing all the source files into a a list of tokens. Each source file is mapped to a CompilationUnit and stored in the PodDef.units field:

Compiler.srcFiles -> Compiler.pod.units

Slots

makeSource

new make(Compiler compiler)

Constructor takes the associated Compiler

runSource

override Void run()

Run the step on the list of source files

runSourceSource

Void runSource(Location location, Str src)

Run the step on the specified source string

tokenizeSource

CompilationUnit tokenize(Location location, Str src)

Tokenize the source into a CompilationUnit