The core of our parser is the Reader class. This class extends the
StandardTokenParsers of Scala. This is the backbone for our parser. We start by creating the reserved words used by the language, as follows:
class Reader extends StandardTokenParsers {
lexical.reserved += ("DIM", "PRINT", "IF", "SUB", "THEN",
"FUNCTION", "SUB", "MAIN", "RETURN", "END FUNCTION")
lexical.delimiters += ("*", "/", "%", "+", "-", "(", ")",
"=", "<", ">", "==", "!=", "<=", ">=", ",", ":")
You can see that we first define all the reserved words and delimiters that will be used by the Scala parser. These are the words we can use to tokenize our language.
124
As noted earlier, we can use different techniques to create our AST, so we have to define the rules we must apply to our language. These rules are used to create the tree we can use later to translate the language. To compile the rules, what we need to do is translate the EBFN that we have defined in Scala syntax.
The first rule we want to establish is the entry point for our program. In BASIC, this is usually the SUB MAIN. The code for this is as follows:
def mainPoint: Parser[Program] = (rep(function) <~ ("SUB" ~
"MAIN")) ~ block ^^ {
case f ~ c => new Program(f, c) }
We can stop for a moment to analyze this first function. We can see that this function uses the element that I introduced in Chapter 6.
(rep(function) <~ ("SUB" ~ "MAIN")) ~ block
The program has a number of functions that can be repeated. After the word SUB MAIN, we find a block. This is essentially another part of the rules, and it identifies the block of code we can write inside the main function.
We can now continue to write other simple rules for defining the language.
/*
with this function we define the node "function" this node is created when we found the word FUNCTION(), we build essentially a complex node, we have FUNCTION, and some argument, after the () we expect a "block" and optiona a return statement, this function use the Helper parser define in the chapter 6, the function want and END FUNCTION for close it. The block is something like that:
FUNCTION Test() PRINT "Test"
END FUNCTION
125
*/
def function: Parser[Function] = ("FUNCTION" ~> ident) ~ ("("
~> arguments) ~ (")" ~> block) ~ opt(returnStatement) <~ "END FUNCTION" ^^ {
case a ~ b ~ c ~ None => new Function(a, b, c, Number(0)) case a ~ b ~ c ~ d => new Function(a, b, c, d.get)
}
//With this function we define the RETURN word, used in the FUNCTION method
def returnStatement: Parser[Expr] = "RETURN" ~> expr ^^ { e => e
}
def arguments: Parser[Map[String, Int]] = repsep(ident, ",") ^^
{
argumentList => {
(for (a <- argumentList) yield (a -> 0)) toMap }
}
//This function defines a block, the block is a set of statements used to define the functionality of the code
def block: Parser[List[Statement]] = rep(statement) ^^ { a => a }
def statement: Parser[Statement] =
positioned(variableAssignment | outStatement | ifStatement | executeFunction | outStatement) ^^ { a => a }
//This defines the reserved word DIM used to define the variable
def variableAssignment: Parser[VariableDefinition] = "DIM" ~>
ident ~ "=" ~ positioned(executeFunction | expr) ^^ { case a ~
"=" ~ b => { new VariableDefinition(a, b) } }
126
def outStatement: Parser[PrintStatement] = "PRINT" ~>
positioned(expr) ^^ { case a => new PrintStatement(a) }
//This defines the statement if, this means when the code finds an if, now we can see we found the code conditional ~ block this means we must use the function conditional to define the word to use, this is essentially a node, with a definition of child inside
def ifStatement: Parser[IfStatement] = conditional ~ block ^^ { case a ~ b ~ c => {
//This defines a conditional statement used in the block of code, now we can see in the word IF( ) THEN we can define some condition, the code can be IF(TRUE)THEN
def conditional: Parser[Condition] = "IF" ~ "(" ~> condition <~
")" ~ "THEN"
//the condition, is used to define the operator we can use in the if , for example ==, > or < the if now can be write //like that:
// IF (VALUE==TRUE)THEN
def condition: Parser[Condition] = positioned(expr) ~ ("<" |
">" | "==" | "!=" | "<=" | ">=") ~ positioned(expr) ^^ { case a ~ b ~ c => {
new Condition(b, a, c) }
}
127 def iterations: Parser[Int] = numericLit ^^ { _ toInt }
//This essentially is responsible for parsing the FUNCTION, what we do is use the Parser Helper from Scala and call the function involved, this helps to translate the code in functionality
def executeFunction: Parser[CallFunctionMethod] = ((ident) <~
"(") ~ callFunctionMethod <~ ")" ^^ {
case a ~ l => new CallFunctionMethod(a, l) }
def functionCallArguments: Parser[Map[String, Expr]] = repsep(functionArgument, ",") ^^ {
_ toMap }
def functionArgument: Parser[(String, Expr)] = (ident <~ "=") ~ expr ^^ {
case a ~ b => (a, b) }
//This function executes a parser of the operation, apply the operation + or – to a term, the term is a number used to create the operation, we can see in this case another little piece of parser kept alive
def expr: Parser[Expr] = term ~ rep(("+" | "-") ~ term) ^^ { case a ~ List() => a
case a ~ b => {
def appendExpression(c: Operator, p: Operator): Operator = {
p.left = c p
}
128
var root: Operator = new Operator(b.head._1, a, b.head._2)
for (f <- b.tail) { var parent = f._1 match {
case "+" => new Operator("+", null, f._2) case "-" => Operator("-", null, f._2) }
root = appendExpression(root, parent) }
root } }
//This function defines a term, essentially identify every single part of an expression
def term: Parser[Expr] = multiplydividemodulo ^^ { l => l } | factor ^^ {
a => a }
def multiplydividemodulo: Parser[Expr] = factor ~ rep(("*" |
"/" | "%") ~ factor) ^^ {
case a ~ List() => a case a ~ b => {
def appendExpression(e: Operator, t: Operator): Operator = {
t.left = e.right e.right = t t
}
129 var root: Operator = new Operator(b.head._1, a, b.head._2) var current = root
for (f <- b.tail) { var rightOperator = f._1 match {
case "*" => Operator("*", null, f._2) case "/" => Operator("/", null, f._2) case "%" => Operator("%", null, f._2) }
current = appendExpression(current, rightOperator) }
root } }
def factor: Parser[Expr] = numericLit ^^ { a => Number(a.toInt) } |
"(" ~> expr <~ ")" ^^ { e => e } | ident ^^ { new Identifier(_) }
The preceding code represents all the code for the Reader. We write the code to parse every element of the language, and we introduce the concept we used to create the parser. Essentially, we split every single command, for example, the if or the function, in a piece of code used to create a token. We essentially tokenize the element. With this element, we can create the AST. We can now define the last function for the language and close the reader. We must implement the function parseAll from the trait we have extended for starting the parser, as follows:
def parseAll[T](p: Parser[T], in: String): ParseResult[T] = { phrase(p)(new lexical.Scanner(in))
}
130
The function parseAll calls the lexical.Scanner and creates the parser for our language. This function creates the token for every element of the language.
This piece of code introduces some elements we must develop for creating the language. Basically, we create a call from some structure we need to define to generate the output of the reader. This code solves the first step of our parser and addresses the first requirement of the creation of our own language. The next step is to create the translator. This is not a simple class but a branch of the class used to translate the language. First, to describe the translator, we must define how the code builds the AST.