Headline

Parsing and tree walking with ANTLR

Characteristics

Both concrete and abstract syntax is defined for companies. To this end, we use an ANTLR-based parser description. The resulting context-free parsing implements Feature:Parsing. The abstract syntax is based on ANTLR's homogeneous tree format. A constructed tree can be walked with an ANTLR tree grammar to total all salaries or to cut salaries in half.

Illustration

The data model is implemented as plain textual files:

company "ACME Corporation" { department "Research" { manager "Craig" { address "Redmond" salary 123456 } employee "Erik" { address "Utrecht" salary 12345 } employee "Ralf" { address "Koblenz" salary 1234 } } department "Development" { manager "Ray" { address "Redmond" salary 234567 } department "Dev1" { manager "Klaus" { address "Boston" salary 23456 } department "Dev1.1" { manager "Karl" { address "Riga" salary 2345 } employee "Joe" { address "Wifi City" salary 2344 } } } } }

A Parser for Feature:Company will be generated by Technology:ANTLR using src/main/antlr/Company.g:

A Compony is defined by:

company :
  'company' STRING '{' department* '}'
  -> ^(COMPANY STRING department*)
  ;
  
department :
  'department' name=STRING '{' 
    manager
    ('employee' employee)*
    department*
  '}'
  -> ^(DEPT $name manager employee* department*)
  ;
    
manager : 
  'manager' employee 
  -> ^(MANAGER employee)
  ;   
    
employee :
  n=STRING '{'
    'address' a=STRING
    'salary' s=FLOAT
  '}'
  -> ^(EMPLOYEE $n $a $s)
  ;

STRING  :   '"' (~'"')* '"';
FLOAT   :   ('0'..'9')+ ('.' ('0'..'9')+)?;
WS      :   (' '|'\r'? '\n'|'\t')+ {skip();};

Feature:Parsing is implemented using the generated Parser:

package org.softlang.company.features;

import static org.softlang.company.antlr.CompanyParser.*;

import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.*;

import java.io.IOException;

public class Parsing {

    public static CommonTree parse(String s)
            throws IOException, RecognitionException {
        /*
         * Parsing is implemented in the generated Parser
         * See src/main/antlr/Company.g
         */
        CommonTree t = (CommonTree)parseCompany(s).getTree();
        return t;
    }

}

Feature:Total and Feature:Cut are implemented using classes generated by ANTLR tree grammars (see TotalCompany.g and CutCompany.g):

tree grammar TotalCompany;

options { 
  tokenVocab=Company;
  ASTLabelType=CommonTree;
}

@header {
package org.softlang.company.antlr;
}

@members {

public double total = 0;

}

company :
  ^(COMPANY STRING dept*)
  ;
  
dept :
  ^(DEPT STRING manager employee* dept*)
  ;
    
manager : 
  ^(MANAGER employee)
  ;   
    
employee :
  ^(EMPLOYEE STRING STRING FLOAT)
  { total += Double.parseDouble($FLOAT.text); }
  ;
tree grammar CutCompany;

options { 
  tokenVocab=Company;
  ASTLabelType=CommonTree;
  output = AST;
  filter=true;
  backtrack=true;
}

@header {
package org.softlang.company.antlr;
}
    
// START: strategy
topdown : employee;
// END: strategy
        
employee :
  ^(EMPLOYEE STRING STRING s=FLOAT)
  -> ^(EMPLOYEE STRING STRING FLOAT[Double.toString(Double.parseDouble($s.text) / 2.0d)])
  ;

    public static double total(CommonTree tree)
            throws IOException, RecognitionException {
        /*
         * Total is implemented in the generated Parser
         * See src/main/antlr/TotalCompany.g
         */
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
        TotalCompany total = new TotalCompany(nodes);
        total.company();
        return total.total;
    }

    public static CommonTree cut(CommonTree tree)
            throws IOException, RecognitionException {
        /*
         * Cut is implemented in the generated Parser
         * See src/main/antlr/CutCompany.g
         */
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(tree);
        CutCompany cut = new CutCompany(nodes);
        CommonTree cutTree = (CommonTree)cut.downup(tree);
        return cutTree;
    }

Test cases are implemented for all Namespace:Features. There is also an invalid input:

This is not a company.

Relationships

This contribution shows Feature:Total and Feature:Cut with tree walking using Technology:ANTLR and Language:Java.

For a ANTLR4 version with Listener pattern see Contribution:antlr4ParseTreeListener.

For a ANTLR4 version with Visitor pattern see Contribution:antlr4ParseTreeListener.

For plain syntax checking see Contribution:antlrAcceptor.

For a basic example of lexing/tokenization see Contribution:antlrLexer.

For an implementation of Feature:Parsing with semantic actions see Contribution:antlrParser.

For Object-Text mapping with Technology:ANTLR see Contribution:antlrObjects.

Architecture

The contribution follows a standardized structure:

  • inputs contains input files for tests
  • src/main/antlr contains grammar files for Technology:ANTLR.
  • src/main/java contains the following packages:
  • src/test/java contains the following packages:

Usage

This contribution uses Technology:Gradle for building. Technology:Eclipse is supported.

See https://github.com/101companies/101simplejava/blob/master/README.md

Metadata


There are no revisions for this page.

User contributions

    This user never has never made submissions.

    User edits

    Syntax for editing wiki

    For you are available next options:

    will make text bold.

    will make text italic.

    will make text underlined.

    will make text striked.

    will allow you to paste code headline into the page.

    will allow you to link into the page.

    will allow you to paste code with syntax highlight into the page. You will need to define used programming language.

    will allow you to paste image into the page.

    is list with bullets.

    is list with numbers.

    will allow your to insert slideshare presentation into the page. You need to copy link to presentation and insert it as parameter in this tag.

    will allow your to insert youtube video into the page. You need to copy link to youtube page with video and insert it as parameter in this tag.

    will allow your to insert code snippets from @worker.

    Syntax for editing wiki

    For you are available next options:

    will make text bold.

    will make text italic.

    will make text underlined.

    will make text striked.

    will allow you to paste code headline into the page.

    will allow you to link into the page.

    will allow you to paste code with syntax highlight into the page. You will need to define used programming language.

    will allow you to paste image into the page.

    is list with bullets.

    is list with numbers.

    will allow your to insert slideshare presentation into the page. You need to copy link to presentation and insert it as parameter in this tag.

    will allow your to insert youtube video into the page. You need to copy link to youtube page with video and insert it as parameter in this tag.

    will allow your to insert code snippets from @worker.