APG Docs
Documentation for APG
apg-lite Documentation
Working on JavaScript parsers for ABNF grammars it became clear that for many applications the full machinery of apg-js was not needed and that a stripped-down version could be built that would have a very small footprint and no external dependencies. This led to the development of apg-lite, a light-weight parser of ABNF grammars. It is a parser only and relies on apg-js, version 4.3.0, with the new –lite option to generate ESM grammar objects. It is designed to fit snugly into a web page without the need for dependencies and bundlers. apg-lite features:
- Parses only JavaScript strings, whereas apg-js can parse any arbitrary array of positive integers.
- Retains only three of the eight SABNF superset operators of apg-js.
- UDT, (User-Defined Terminals) Handwritten, phrase-matching code snippets.
- AND, The positive look-ahead operator.
- NOT, The negative look-ahead operator.
- The AST has been simplified.
- Tracing (debugging) the parse tree has been simplified.
- Statistics collection (profiling) has been simplified.
- apg-lite is fully contained in a single JavaScript file.
- Can be scripted into a web page without bundlers.
- Can be copied and pasted into application code to eliminate importing.
Python APG is written completely in Python and is very similar in functionality to the JavaScript version. It has a stand-alone generator of grammar objects suitable for importing into parsers and also has an API for generating grammar objects on the fly within the parser project. It also comes with a pattern-matching engine which uses SABNF for the pattern syntax and APG as the parser to match the patterns.
It is available from both the GitHub and PyPI repositories. The GitHub repository contains a large set of examples which demonstrate most all of the Python APG features and a large set of unit tests.
A quick start guide to building a parser from the GitHub repository can be found here.
A quick start guide to building a parser from the PyPI repository can be found here.
Python APG is licensed with the permissive 2-Clause BSD license.
python-ini is an INI file parser and writer. It is simple and easy to use and yet it retains most of the stable and varying features of this WikiPedia article. The INI file syntax is very liberal.
- allows semicolon(;) and hash(#) for comment characters
- allows equals(=), colon(:) and space for key/value delimiters
- allows comma(,) and space as value delimiters in multi-value mode
- allows integer, floating point, boolean and string values
- allows unquoted, single-quoted and double-quoted strings
- allows escaped characters in strings for:
- special characters
- control characters
- hexadecimal values
- 16-bit Unicode characters
- 32-bit Unicode characters
It supports both single-value and multi-value modes, allowing keys to have only one value or a list of multiple values. The parser supports default values for section keys that do not exist in the INI file. The writer supports configuration options for delimiters, booleans and the comment tab for comments on key/value lines.
Its chief omission is that it does not directly support subsections. However, since the period is a valid section name character, section names such as [A], [A.B] and [A.B.C] are unique section names and could be used to simulate subsections.
The parser is implemented with a single class and five member “getter” functions, it is easy to use with a short learning curve. The writer is implemented with a single class with three functions to write comments, key/value pairs and sections.
python-ini is available from both the GitHub and PyPI repositories. The GitHub repository contains a set of examples which demonstrate most all of the python-ini features and a set of unit tests.
GitHub quick-start guides can be found here:
PyPI quick-start guides can be found here:
python-ini is licensed with the permissive 2-Clause BSD license.
Version 7.0 is written entirely in C and extensively documented. This is the latest version of APG with a significant number of advantages and features not available in any previous versions or languages. It can be downloaded from its GitHub repository and is licensed with the permissive 2-Clause BSD license.
- It exposes an Application Programming Interface, API, for generating parsers within the user’s code on the fly.
- generates parsers for 8-, 16-, 32-, or 64-bit wide alphabet characters.
- It re-introduces Partially-Predictive Parsing Tables (PPPT), first introduced in version 5.0 in 2007, but never continued in future versions or languages.
- It adds a Pattern-Matching Engine, apgex:
- replaces the cryptic pattern-matching syntax of regex with SABNF
- more powerful than regex
- full recursion for matching deeply nested pairs
- 2 modes of back referencing – introduces a new “parent-mode” back referencing
- allows handwritten code snippets for complex patterns
- all phrases are uniquely named and easily referenced
- AST exposed for custom phrase translations
- tracing optionally exposed for pattern debugging
- It includes an RFC8259-compliant JSON parser and builder.
- It includes a standards-compliant XML, non-validating parser.
- It includes a number of utilities, commonly needed and used by parsing applications:
- Unicode data encoding and decoding
- display of unprintable, non-ASCII data in hexdump-like format
- line separation and handling for ASCII and Unicode data
- a message logging facility
- plus an extensive tool box of utility functions for system information, pretty printing of APG information and more
- and a large number of examples demonstrating all aspects of use
(Depricated) Version 6.3 is written entirely in C but has facilities for C++ wrappers. This version has been used by a number of Telecom companies in their protocol stacks. It has look-ahead operators and User-Defined Terminals (UDTs) but none of the many other new features of version 7.0. It can be downloaded from its GitHub repository.
Java APG Version 1.1.0 makes no operational changes to the previous version 1.0 but updates it in a couple of ways. Primarily, the licensed is changed from GPL to the more permissive 2-Clause BSD License. Also, it gives some hints and documentation about how to use it in maven projects.
Version 1.0 introduced for the first time a number of innovations.
- User-Defined Terminals (UDTs) were defined which allow difficult-to-match phrases to be hand coded and included in the grammar like any of the other terminal phrases.
- Single-quoted case-sensitive strings were introduced before the publication of RFC 7405 and have been grandfathered in, even though RFC 7405 is now implemented as well.
- Multiple SABNF files may be specified, all of which are then concatenated for the final grammar.
All of these features are retained in Version 1.1.0 and, in fact, are now standard in all current versions for all languages.
Java APG Version 1.1.0, along with a large set of examples of how to implement it, can be downloaded from its GitHub repository.
Java APG has been instrumental in creating the ABNF grammars for the OASIS Open Data Protocol (OData) and in testing the developed grammars.
apg-js
apg-js is and updated version of JavaScript APG. It obsoletes all previous JavaScript APG libraries and applications. It changes them in two significant ways.
- It fixes a major problem in the attributes algorithm. Previous versions could fail on large grammars with many or large sets of mutually-recursive rules.
- It combines all libraries and applications into a single, easier to manage and maintain package.
This package contains four libraries
- apg-exp – the pattern-matching engine library. See the user’s guide and this tutorial at sitepoint.com.
- apg-api – the API for the parser generator
- apg-lib – the library of parsing functions required by all APG parsers
- apg-conv-api – the API for data conversion operations
The libraries have inter-dependencies and each of the listed libraries depends on each and all of the libraries below it in the list.
These libraries are meant to assist other parsing applications and are normally not installed by themselves. They are, rather, installed along with those other applications as in the examples described below.
apg-js does, however, provide access to two, node.js applications.
- apg – this is the desktop parser generator. It generates a “grammar” function that can be used to initialize a JavaScript APG parser.
- apg-conv – this is a desktop data conversion tool
apg-js has been documented in the code with docco. The documentation can be generated with the script ./bin/docco-gen.sh or viewed here.
apg-js is licensed with the 2-Clause BSD license and is available for download from npm and GitHub.
apg-js-examples
apg-js-examples contains a large set of examples of using apg-js. Each example is in its own directory and in that directory is a script, main.sh, which will execute the example. Many examples have multiple options. In these cases, running the script with no arguments will display a description of the example and a detailed list of the options available.
npm scripts are also available for each of these examples in the package.json file. For example,
src/ast/main.sh
&
npm run ast
both run the AST test.
If executing the scripts directly or in with a debugger, they should be executed from the repository directory for correct relative path name alignment.
Each of the four libraries, apg-conv-api, apg-lib, apg-api and apg-exp, has been bundled for browser use. Consequently, the examples for each of these libraries contains an example of browser usage. The browser-based example file names will have a “web-” prefix.
apg-js-examples has been documented in the code with docco. The documentation can be generated with the script ./bin/docco-gen.sh or viewed here.
apg-js-examples is licensed with the 2-Clause BSD license and is available for download from npm and GitHub.
Note that apg-js and apg-js-examples obsolete all previous versions of JavaScript APG.
