A Short History of APG
Version Number
Release Date
Description
1.0
05/12/2005
Needless to say, it was pretty clunky. It had lots and lots of inefficiencies which have been, for the most part, methodically eliminated over the following releases. But it worked. And its use of the ABNF-defined operators as the basic phrase-recognition building blocks is still the foundation of APG.
3.0
3.1
Versions 2.0 – 3.1 mostly just cleaned up many of the most glaring inefficiencies.
4.0
06/02/2006
This was the last version written in C++ and had the ambitious goal of building a C++ parser. This challenge was only partially successful – a nearly complete pre-parser was developed – but it led to several important APG inovations. Most notably:
- The first new operator was added to the ABNF-defined set. This was the first “Superset ABNF” or SABNF grammar. The operator, repeat-until, was a modification of the repetition operator. It would only repeat until a specified phrase was encountered. This was later recognized as a special case of the syntactic predicate AND and NOT operators and has since been replaced by them.
- This was also the first version to used “first success disambiguation”. Ambiguities were resolved by simply accepting the first successful match in a list of alternate expressions and ignoring the remaining alternates. The Parsing Expression Grammars (PEG) method terms this “prioritized-choice” which is now the preferred phrase.
5.0
10/15/2007
The first version written entirely in C. It still generated C++ parsers, but only as C++ wrappers for the underlying C code. This was the first and only version to utilize Partially-Predictive Parsing Tables. This is the recognition that any grammar can be preparsed to the extent that pre-defined responses can be prepared in advance for each possible single-character look ahead in the language alphabet. These are often, but not always, deterministic – hence the term “partially-predictive”. This was an interesting development and did, in fact, improve parsing speeds by factors of 2-4. However, its implementation was complicated – difficult to write and difficult to maintain. A great idea, but it was never carried forward into future versions.
5.0.1
11/25/2007
This was the first Linux port and the first version to be released under the GNU General Public License.
5.1
03/01/2009
The Linux build system was changed to the automake standard. The ABNF standard line ending \r\n was relaxed to accept just \n, the Linux standard, as well.
6.0
06/17/2009
In this version, the rule callback functions were allowed to override the ABNF parsing rules, disallowing matched phrases or even doing their own, hand-written phrase recognition. The precurser to UDTs. This version also cleaned up the messy business of selecting which rule names to include in the AST. Previous versions used grammar notation for this. In this version, these selections were moved to a runtime set up model.
JavaScript Interactive APG
12/02/2009
I wanted an interactive web page for testing ABNF grammars. I wanted a tool that would not only validate the grammars but the grammar-defined language sentences as well. This required a JavaScript version of APG.
6.1
12/02/2009
Updated version 6.0 to also generate JavaScript parsers.
APG 1.0
10/23/2011
Inspired by the need for Java in mobile apps, I took up the challenge of learning it. What better way than to create a Java version of APG? I never got around to doing a mobile app, but I did get a version of APG with both the generator and the generated parsers written entirely in Java. Java APG introduced for the first time User-Defined Terminals (UDTs.)
6.2
This was version 6.1 converted to run on 64-bit operating systems. All previous C/C++ versions were restricted to 32-bit OSs. Released only under commercial licensing.
6.3
06/29/2012
A complete re-write of version 6.1, its primary new features are:
- 64-bit operating systems.
- Variable-width alphabet character codes – 8-, 16-, 32- and 64-bits wide.
- User-Defined Terminals (UDTs).
- Additional profiler optimizations.
- Improved attributes algorithm.
JavaScript APG 2.0
10/31/2015
A complete re-write of JavaScript 1.0. Written with the main objectives in mind:
- Updating its features to the full APG 6.3 and Java APG 1.0 complement.
- Creating a user-local version of the parser generator as well as the parsing library (the version 1.0 generator was only available here as Interactive APG).
- Developing it as node.js modules.
- Publishing it to the npm registry for easy installation and use.
apg-exp
03/06/2016
apg-exp is a regex-like pattern-matching engine that uses SABNF syntax for pattern definitions and APG to generate and execute the pattern-matching parsers.
apg.html
02/15/2017
apg.html is a self-contained, stand-alone web page GUI for development and testing of SABNF grammars. With apg.html, the user gets detailed visual information about the grammar, rules, attributes and a page for inputting data to test the generated parser. It replaces the “Interactive APG” page which has now been retired.
7.0
02/15/2021
A complete re-write in C featuring:
- API for generating parsers on the fly
- vairable-width alphabet character codes, 8-, 16-, 32-, and 64-bits wide
- Partially-Predictive Parsing Tables (PPPT)
- pattern-matching engine, apgex
- XML parser
- JSON parser and builder
- data encoding/decoding utility
- hexdump-like data formatting utility
- plus a large tool chest of additional utilities
- and a large set of examples illustrating all of the most important features
apg-js
04/26/2021
apg-js oboletes apg-js2, apg-js2-lib, apg-js2-exp, apg-js2-api, apg-conv and apg-conv-api. It changes them is several ways.
- Most significantly, it fixes a major problem in the attributes algorithm. Previous versions could fail on large grammars with many or large sets of mutually-recursive rules.
- It combines all of those packages into a single package that is easier to use and maintain.
- The license is changed to the 2-Clause BSD License.
apg-js-examples
04/26/2021
apg-js-examples replaces apg-js2-examples. apg-js2-examples only existed on GitHub and was never an npm package. apg-js-examples is now an npm package with a single dependency on apg-js.
The examples are all the same, but much more accessible. They can be run with npm run commands or through scripts that are easily run in an IDE with a debugger.
Java
APG 1.1.0
05/08/2021
Version 1.1.0 updates version 1.0 in a couple of ways. Primarily, the license is changed to the more permissive 2-Clause BSD license. Also, in the installation section below are instructions on how to include Java APG in a maven project.
Python
APG 1.0
12/24/2022
A few months ago I noticed that Python had become the most popular programming language. Curious to try it out I naturally thought that a Python version of APG was in order. Like the JavaScript version, it comes with a pattern-matching engine and a large set of examples of how to use it. You can find it both on GitHub and PyPI.
apg-lite
11/27/2023
Working on JavaScript parsers for ABNF grammars it became clear that for many applications the full machinery of apg-js was not needed and that a stripped-down version could be built that would have a very small footprint with no external dependencies. This led to the development of apg-lite. By way of demonstration, a complete, well-tested URI parser has been developed and included that is contained in a single file that weighs in at 78K – 38K for the minified version.
