Index

Package: Scanners

Description

package Tokens.Scanners is

Classes

Token_Scanner

type Token_Scanner is new Limited_Object with private;

Ancestors:

Primitive operations:

Delete (overriding Objects.Delete)
Objects.Construct (Inherited)
Objects.To_String (Inherited)
A Token_Scanner parses and scans a text stream for specific types of tokens. Tokens can be optionally accepted from the stream, or expected. If a specific token type is expected but a different type is found next, a Parse_Exception is raised and the token remains.

Types

A_Token_Scanner

type A_Token_Scanner is access all Token_Scanner'Class;

Constants & Global variables

Parse_Exception

Parse_Exception : exception;
Raised when an unexpected token is found.

Subprograms & Entries

Create_Token_Scanner

function Create_Token_Scanner return A_Token_Scanner;
Creates a new Token_Scanner.

Accept_Binary_Op

function Accept_Binary_Op
( this: not null access Token_Scanner'Class ) return A_Token;
Accepts a valid binary expression operator, as defined by the class Binary_Operator. If a valid binary operator is not found, no tokens are taken from the stream and null is returned. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Accept_Identifier

function Accept_Identifier
( this: not null access Token_Scanner'Class ) return A_Identifier_Token;
Optionally accepts an identifier token. Returns the token if the next in the stream is an identifier, otherwise null is returned. This is the same as calling Accept_Token( TK_IDENTIFIER ). Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Accept_Token

function Accept_Token
( this: not null access Token_Scanner'Class;
tokenType: Token_Type ) return A_Token;
Optionally accepts a token of a specific type. Returns the next token from the stream if it matches 'tokenType', otherwise the token remains in the stream and null is returned. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Accept_Unary_Op

function Accept_Unary_Op
( this: not null access Token_Scanner'Class ) return A_Token;
Optionally accepts a valid unary expression operator, as defined by the class Unary_Operator. If a valid unary operator is not found, no tokens are taken from the stream and null is returned. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Expect_Identifier

function Expect_Identifier
( this: not null access Token_Scanner'Class ) return A_Identifier_Token;
Requires an identifier token to be next. If the next token in the stream is not an identifier then Parse_Exception will be raised. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Expect_Token

function Expect_Token
( this: not null access Token_Scanner'Class;
tokenType: Token_Type ) return A_Token;
Requires the next token to be of a specific type. Returns the next token from the stream if it matches 'tokenType', otherwise the token remains in the stream and Parse_Exception is raised. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Expect

procedure Expect
( this: not null access Token_Scanner'Class;
tokenType: Token_Type );
Requires the next token to be of a specific type. If the next token does not match 'tokenType', it will remain in the stream and a Parse_Exception will be raised. Otherwise, the expected token will be consumed and the procedure will return normally. Token_Exception will be raised if an unrecognized or malformed token is encountered; The bad token will be discarded and not returned again.

Get_Location

function Get_Location
( this: not null access Token_Scanner'Class ) return Token_Location;
Returns the current location of the scanner in the input stream.

Set_Input

procedure Set_Input
( this: not null access Token_Scanner'Class;
stream: Stream_Access );
Sets the input stream for reading characters. If 'stream' is null, the scanner's input will be cleared.

Delete

procedure Delete
( this: in out A_Token_Scanner );
Deletes the Token_Scanner;