YAML

github.com/ornew/pego/parsers/yaml parses YAML as defined by YAML 1.2.2, the whole syntax: streams of documents, directives, block and flow collections, every scalar style, properties, aliases and comments, with a parser generated by PEGO from yaml.pego. It depends only on the standard library.

go get github.com/ornew/pego/parsers/yaml

#Use

// Go values, with the YAML 1.2 core schema: map[string]any, []any, string, int64, float64, bool, nil.
v, err := yaml.Load("name: pego\ntags: [peg, pratt]\nstars: 42\n")
docs, err := yaml.LoadAll("--- a\n--- [b, c]\n") // one value per document

// Typed values with positions, tags, anchors and styles.
s, err := yaml.ParseAST("key: &x value # comment\nlist:\n- 'single'\n- *x\n")
for _, p := range s.Documents[0].Root.(*yaml.Mapping).Pairs {
	k := p.Key.(*yaml.Scalar)
	fmt.Printf("%s at %d-%d: %T\n", k.Value(), k.Start, k.End, p.Value)
}
// key at 0-3: *yaml.Scalar
// list at 24-28: *yaml.Sequence

// The event stream of the yaml-test-suite.
events, err := yaml.Events("--- !!map\nkey: [a, 'b']\n")
// +STR, +DOC ---, +MAP <tag:yaml.org,2002:map>, =VAL :key, +SEQ [], =VAL :a, =VAL 'b, -SEQ, -MAP, -DOC, -STR

// Only check.
ok := yaml.Valid("a: b: c") // false

A node's Props holds its tag and anchor; Mapping.Flow and Sequence.Flow tell the flow style ({}, []) from the block style; Scalar.Style is Plain, SingleQuoted, DoubleQuoted, Literal or Folded, and Scalar.Indent is the indentation of the content of a block scalar. An empty node (key:) is an empty plain *Scalar, and a pair of a flow sequence ([a: b]) a flow *Mapping of one pair. Positions are in code points by default; yaml.ParseAST(src, yaml.Bytes) counts bytes.

#Checks that are not syntax

The parser checks the syntax. Check (and Valid, Events, Load and LoadAll) report what the syntax cannot show: a tag shorthand whose handle the document does not declare, more than one %YAML directive in a document or more than one %TAG directive for a handle, a %YAML version whose major version is not 1, and an alias of an anchor that no earlier node of the document has. Valid, Events, Load and LoadAll also reject input that is not UTF-8, which the parser reads as U+FFFD.

#Go values

Load composes a document with the YAML 1.2 core schema (10.3):

• a mapping is a map[string]any if all of its keys are strings, and a map[any]any otherwise; a key that is a mapping or a sequence is an error, and so is a key that occurs twice;
• a sequence is a []any;
• a plain scalar without a tag is nil (null, Null, NULL, ~ or empty), a bool (true, True, TRUE, false, False, FALSE), an int64 (decimal, 0o octal or 0x hexadecimal), a float64 (with .inf, -.inf and .nan in the three cases) or a string; an integer outside the range of int64 is an error;
• a quoted or block scalar, and a scalar with the tag ! or !!str, is a string;
• the tags !!null, !!bool, !!int and !!float require a value that resolves to their type, and !!map and !!seq a mapping and a sequence; other tags do not change the value (there are no YAML 1.1 types such as !!binary or !!timestamp, nor merge keys <<);
• an alias is the value of its anchor, the same map or slice for a collection; an alias inside the node it refers to is an error.

#Conformance

The tests run the yaml-test-suite, data release 2022-01-17, which is vendored in testdata/yaml-test-suite (MIT License): for each of the 402 tests, a valid input must give the events of test.event and the values of in.json where it exists, and an invalid input must be rejected by Events and LoadAll. All 402 pass: 308 valid inputs (279 with JSON values) and 94 invalid ones, 92 of which the parser rejects and 2 Check (an undeclared tag handle, QLJ7, and a second %YAML directive, SF5V). ParseAST and Recognize agree on every input.

TestSuiteSource runs the source definitions of the suite's main branch instead, where YAML_TEST_SUITE_SRC names its src directory (git clone https://github.com/yaml/yaml-test-suite; read with this package and converted as the suite's bin/suite-to-data.pl does). At commit da267a5 (2025-12-25) they give the same 402 tests, identical in input, events and JSON, and all pass.

Other tests decode every scalar style, compose values with the core schema, check the errors of Load and Check, nest deeply, and fuzz the parser (go test -fuzz FuzzParse: ParseAST and Recognize agree, Valid and Events agree, nothing panics).

Known deviations and choices where the specification leaves room:

• A carriage return alone is a line break, but the line after it does not start a line for the rules that test the start of a line (a PEGO ^ matches after a line feed only): there, a comment or a document marker at the start of the line is rejected ("a: 1\r# c\rb: 2"). Line feeds and CR LF are not affected.
• The 1024-character limit of implicit keys is not enforced.
• Encodings: the input is UTF-8; UTF-16 and UTF-32 are not detected. ParseAST, Parse and Recognize read invalid UTF-8 as U+FFFD; the other functions reject it.
• Duplicate keys are kept by ParseAST and Events, as the suite expects (2JQS), and rejected by Load.
• ? and : of block mappings (explicit keys and values) must be followed by white space or a line break, like -; otherwise they start a plain scalar (?x: y is the key ?x).
• An indentation indicator at the top level adds to the indentation -1 of the top-level node, as the productions say (l-bare-document is s-l+block-node(-1,block-in)): the content of --- |2 is indented by 1, where libyaml takes 2. The suite has no test of it; automatic detection at the top level allows content in column 0, as the suite expects (DK3J).
• A last line without a line break in a block scalar ends with one, as in the suite's reference parser (JEF9/02, L24T/01).
• Nesting is limited by the generated parser's depth limit of 100,000 rule calls: about 12,000 levels of flow sequences, 10,000 of flow mappings, 25,000 of compact block sequences (- - - x) and 12,000 of indented block mappings. Deeper input fails with an error instead of exhausting the stack.

#Grammar

yaml.pego follows the productions of the specification, numbered as there. The parameters of a production are variables of the grammar: n (the indentation, an int) and c (the context: "block-out", "block-in", "flow-out", "flow-in", "block-key" or "flow-key"). A production called with other arguments defines them in a rule of its own, since a definition lasts until the rule that made it returns; the auto-detected indentations of block collections and block scalars are read by a lookahead and defined the same way. Where a parameter takes one value or two in practice, the production is written once per value (ns_plain_one_out for ns-plain-one-line(c) with c = block-key), and s-indent(n) reads all the spaces and compares their number with n, since what follows it never starts with a space. The recursion of the double- and single-quoted next lines is unrolled into repetitions.

The grammar differs from the productions where a PEG needs it or where it measurably pays, each commented in the grammar:

• A PEG commits to the first alternative that matches, where the specification may try another: optional properties of a block collection are retried without the second property or without any, and a JSON-like key of a flow mapping is tried before a YAML key that would be just its properties.
• Alternatives are tried only where they can match, by the next characters: a block node is a flow node unless an indicator of a block scalar, a property, a comment or a line break follows; plain scalars and keys without properties, the common case, are read by rules that do not need the context; the context is defined only where it changes.

A differential test against the first version of the grammar, which followed the productions without these changes (git show 4d9d1bc:parsers/yaml/yaml.pego), found no difference on the suite and on 150,000 random mutations of its inputs: both grammars accepted the same 70,639 and gave the same trees. (It ran on the engine, which this module cannot import, and is not part of its tests.)

#Performance

On a stream of Kubernetes-like manifests of 256 KB (deployments, services and config maps; go test -bench ., Apple M3 Max):

The standard library has no YAML parser; decoding the same values from JSON with encoding/json takes a quarter of the time of LoadAll. Recognize builds nothing but is slower than ParseAST, whose rules the generator compiles into direct code (see the code generation guide). The first version of the grammar, which tried every alternative of the specification in order, took 39 ms for ParseAST; most of the difference is calls that could not match and the memoization they caused.

#Development

go generate ./parsers                 # regenerate parser.go after changing yaml.pego (from the repository root)
go test ./parsers                     # parser.go up to date; golden files on every backend of the engine
cd parsers/yaml && go test ./...      # the suite and the other tests
YAML_TEST_SUITE_SRC=/path/to/yaml-test-suite/src go test -run TestSuiteSource ./...
go test -fuzz FuzzParse               # fuzz the parser
go test -bench .                      # benchmarks