Index

Package: Tokens

Description

package Tokens is
Copyright (c) 2012 Kevin Wellwood All rights reserved. This source code is distributed under the Modified BSD License. For terms and conditions, see license.txt.

Classes

Token

type Token is new Limited_Object with private;

Ancestors:

Immediate Children:

Primitive operations:

Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A Token represents a recognized element of syntax. It is a terminal element in a script's context free grammar.

Identifier_Token

type Identifier_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
An Identifier_Token represents an identifier or variable name.

Number_Token

type Number_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A Number_Token represents a literal number, either decimal or integer.

String_Token

type String_Token is new Token with private;

Ancestors:

Primitive operations:

Construct (Inherited)
Construct
Objects.Construct (Inherited)
Objects.Delete (Inherited)
Objects.To_String (Inherited)
A String_Token represents a literal string.

Types

Token_Type

type Token_Type is (
        TK_AND,
        TK_BANG,
        TK_COMMA,
        TK_EQUALS,
        TK_GREATER,
        TK_GREATER_EQUALS,
        TK_LEFT_PARENTHESIS,
        TK_LESS,
        TK_LESS_EQUALS,
        TK_MINUS,
        TK_NOT_EQUALS,
        TK_OR,
        TK_PERCENT,
        TK_PLUS,
        TK_RIGHT_PARENTHESIS,
        TK_SEMICOLON,
        TK_SLASH,
        TK_STAR,

        TK_IDENTIFIER,
        TK_NUMBER,
        TK_STRING,
        TK_EOF
    );
Each Token_Type value represents a token in a grammar.

Static_Token_Type

subtype Static_Token_Type is Token_Type range TK_AND..TK_STAR;

Token_Location

type Token_Location is
        record
            line : Natural := 0;
            col  : Natural := 0;
        end record;
Encapsulates a token's position in an input stream. A token's location is determined by the location of it's first character.

A_Token

type A_Token is access all Token'Class;

A_Identifier_Token

type A_Identifier_Token is access all Identifier_Token'Class;

A_Number_Token

type A_Number_Token is access all Number_Token'Class;

A_String_Token

type A_String_Token is access all String_Token'Class;

Subprograms & Entries

To_String

function To_String
( loc: Token_Location ) return String;
Returns a string representation in the format 'line:column'.

Create_Token

function Create_Token
( text: String;
loc: Token_Location ) return A_Token;
Creates a new Token of the type matching 'text'. If 'text' is not recognized as a valid token, null will be returned.

Get_Location

function Get_Location
( this: not null access Token'Class ) return Token_Location;
Returns the location of the token in the input stream.

Get_Type

function Get_Type
( this: not null access Token'Class ) return Token_Type;
Returns the token's type.

Delete

procedure Delete
( this: in out A_Token );
Deletes the Token.

Get_Name

function Get_Name
( this: not null access Identifier_Token'Class ) return String;
Returns the name of the identifier specified by the token.

Get_Value

function Get_Value
( this: not null access Number_Token'Class ) return Long_Float;
Returns the numeric value of the token.

Get_Value

function Get_Value
( this: not null access String_Token'Class ) return String;
Returns the string value of the token.