Index

Package: Tokens

Description

package Tokens is

Classes

Token

type Token is new Limited_Object with private;

Ancestors:

Immediate Children:

Primitive operations:

Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A Token represents a recognized element of syntax. It is a terminal element in a script's context free grammar.

Identifier_Token

type Identifier_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
An Identifier_Token represents an identifier or variable name.

Number_Token

type Number_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A Number_Token represents a literal number, either decimal or integer.

String_Token

type String_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A String_Token represents a literal string.

Types

Token_Type

type Token_Type is (
        TK_AND,
        TK_BANG,
        TK_COMMA,
        TK_EQUALS,
        TK_GREATER,
        TK_GREATER_EQUALS,
        TK_LEFT_PARENTHESIS,
        TK_LESS,
        TK_LESS_EQUALS,
        TK_MINUS,
        TK_NOT_EQUALS,
        TK_OR,
        TK_PERCENT,
        TK_PLUS,
        TK_RIGHT_PARENTHESIS,
        TK_SLASH,
        TK_STAR,

        TK_IDENTIFIER,
        TK_NUMBER,
        TK_STRING,
        TK_EOF
    );
Each Token_Type value represents a token in a grammar.

Token_Location

type Token_Location is
        record
            line : Natural := 0;
            col  : Natural := 0;
        end record;
Encapsulates a token's position in an input stream. A token's location is determined by the location of it's first character.

A_Token

type A_Token is access all Token'Class;

A_Identifier_Token

type A_Identifier_Token is access all Identifier_Token'Class;

A_Number_Token

type A_Number_Token is access all Number_Token'Class;

A_String_Token

type A_String_Token is access all String_Token'Class;

Subprograms & Entries

To_String

function To_String
( loc: Token_Location ) return String;
Returns a string representation in the format 'line:column'.

Create_Token

function Create_Token
( text: String;
loc: Token_Location ) return A_Token;
Creates a new Token of the type matching 'text'. If 'text' is not recognized as a valid token, null will be returned.

Get_Location

function Get_Location
( this: not null access Token'Class ) return Token_Location;
Returns the location of the token in the input stream.

Get_Type

function Get_Type
( this: not null access Token'Class ) return Token_Type;
Returns the token's type.

Delete

procedure Delete
( this: in out A_Token );
Deletes the Token.

Get_Name

function Get_Name
( this: not null access Identifier_Token'Class ) return String;
Returns the name of the identifier specified by the token.

Get_Value

function Get_Value
( this: not null access Number_Token'Class ) return Long_Float;
Returns the numeric value of the token.

Get_Value

function Get_Value
( this: not null access String_Token'Class ) return String;
Returns the string value of the token.