PEGO

Ready-made parsers

Parsers for common languages, written in PEGO and generated into Go with pego gen. Each is a Go module of its own that depends only on the standard library: import it and parse, with nothing to generate or compile at run time.

Language Module Specification Checked against
CEL github.com/ornew/pego/parsers/cel The CEL language definition, as cel-go parses it All 2,527 conformance tests of cel-spec; cel-go v0.32.0 (trees, offsets and literal values on 32,794 vendored expressions; fuzz tests)
CSV github.com/ornew/pego/parsers/csv RFC 4180, with the leniencies of encoding/csv 73 edge cases, encoding/csv (differential and fuzz tests)
CUE github.com/ornew/pego/parsers/cue The CUE Language Specification, as cue/parser v0.17.1 parses it cue/parser: 4,061 sources of the CUE repository, 32,488 mutants, 15,000 generated inputs and 4,126 literals (trees, offsets and comments; no difference); fuzz tests
DuckDB SQL github.com/ornew/pego/parsers/duckdb DuckDB 1.5.6's SQL dialect DuckDB 1.5.6 on 158,605 texts (acceptance, statement splitting and the AST of SELECT statements; 13 known deviations)
Go github.com/ornew/pego/parsers/golang The Go Programming Language Specification (go1.27), as go/parser parses it; ParseFile returns go/ast go/parser: every file of GOROOT/src (whole trees, positions included), the programs embedded in its tests, mutations and fuzz tests (no difference)
JSON github.com/ornew/pego/parsers/json RFC 8259 JSONTestSuite (every y_ and n_ case), encoding/json (differential and fuzz tests)
Python github.com/ornew/pego/parsers/python Python 3.14, as CPython 3.14.0 parses it CPython's ast.dump, positions included, on the standard library, Lib/test and its 61,540 code snippets (no difference); mutation and fuzz tests
TypeScript github.com/ornew/pego/parsers/typescript TypeScript 5.9 and TSX, as the TypeScript 5.9.3 compiler parses them The compiler's trees on its own test cases (14,799 of 14,804 files agree), its lib declaration files and mutated sources
XML github.com/ornew/pego/parsers/xml XML 1.0 (Fifth Edition), non-validating, with Namespaces in XML 1.0 W3C XML Conformance Test Suite (no failure among the XML 1.0 tests that need no external entity), encoding/xml (differential and fuzz tests)
YAML github.com/ornew/pego/parsers/yaml YAML 1.2.2, with the core schema for Go values yaml-test-suite (all 402 tests: events and JSON values) 
go get github.com/ornew/pego/parsers/json

import "github.com/ornew/pego/parsers/json"

v, err := json.Decode(`{"name": "pego", "tags": ["peg", "pratt"]}`) // map[string]any{...}
ast, err := json.ParseAST(src)                                      // typed values with positions

Every parser has the API of a parser generated with pego gen -types -recognize (see code generation), plus functions written for its language:
ParseAST(input, unit...) The document as the Go types the grammar declares, each with its Span in the input
Parse(input, unit...) The document as a tree of *Node, as the engine returns it
Recognize(input, unit...) Only checks the input, without building anything
*SyntaxError The position (Line, Col, Pos) and the expected tokens of a syntax error

Positions are in code points by default; pass Bytes for byte offsets.

Layout

Each directory holds:

File Contents
<language>.pego The grammar. It can be used with the engine too (pego.CompileSource), for streaming, incremental parsing or the pego tools.
parser.go The parser generated from the grammar. Do not edit it.
other .go files Code written for the language: documentation, conversions to Go values, helpers
*_test.go Tests against the language's specification, conformance suites and reference implementations; examples; benchmarks
testdata/*.txt, *.golden Inputs and the expected trees and errors, checked on the generated parser here and on every backend of the engine by parsers_test.go
go.mod, LICENCE The module

Development

go generate ./parsers                 # regenerate every parser.go (after changing a grammar or the generator)
go test ./parsers                     # parser.go up to date; golden files on every backend of the engine
go test ./parsers -update             # rewrite the golden files from the engine
parsers/test.sh                       # both, and the tests of every module
cd parsers/json && go test ./...      # the tests of one module

The generated parser.go declares unexported names of its runtime (such as parse, parser and rule) in the package, so hand-written code must avoid them.

The generation commands are the go:generate lines of generate.go; the up-to-date test reads them, so a new parser needs only a new line there. The modules are not part of the main module's go test ./...: run parsers/test.sh after changing the engine or the generator.