PEGO

Validate a YAML file and point at the problem

Problem. Your program reads a YAML configuration. You want to reject a file that is not YAML, a file that repeats a key, and a file whose port is not a port, and to tell the user the line and the column of each problem.

Do not write a YAML grammar: parsers/yaml is a complete YAML 1.2.2 parser, a module of its own that depends only on the standard library. Use yaml.Load for the syntax and the checks of the core schema, and yaml.ParseAST for your own checks, which gives every node its position.

go get github.com/ornew/pego/parsers/yaml
// main.go
package main

import (
	"errors"
	"fmt"
	"strconv"

	"github.com/ornew/pego/parsers/yaml"
)

// lineCol converts an offset in code points, the unit of the spans, to a line and a column.
func lineCol(src string, offset int) (line, col int) {
	line, col = 1, 1
	for i, r := range []rune(src) {
		if i == offset {
			break
		}
		if r == '\n' {
			line, col = line+1, 1
		} else {
			col++
		}
	}
	return line, col
}

// check validates a configuration: it must be YAML, and the top-level key port must be an integer from 1 to 65535.
func check(src string) error {
	// Load parses the YAML (a *yaml.SyntaxError, with a line and a column) and checks what the syntax cannot show,
	// such as a key that occurs twice (a *yaml.SemanticError, with a line and a column).
	if _, err := yaml.Load(src); err != nil {
		return err
	}

	// Our own rules need the positions, which Load does not keep: use the typed values.
	stream, err := yaml.ParseAST(src)
	if err != nil {
		return err
	}
	root, ok := stream.Documents[0].Root.(*yaml.Mapping)
	if !ok {
		return errors.New("the configuration must be a mapping")
	}
	for _, p := range root.Pairs {
		if k, ok := p.Key.(*yaml.Scalar); !ok || k.Value() != "port" {
			continue
		}
		span := p.Span
		if v, ok := p.Value.(*yaml.Scalar); ok {
			span = v.Span // point at the value, not at the whole pair
			if n, err := strconv.Atoi(v.Value()); err == nil && n >= 1 && n <= 65535 {
				return nil
			}
		}
		line, col := lineCol(src, span.Start)
		return fmt.Errorf("%d:%d: port must be an integer from 1 to 65535", line, col)
	}
	return errors.New("port is missing")
}

func main() {
	for _, src := range []string{
		"name: api\nport: 8080\n",
		"name: api\nport: 80a80\n",
		"name: api\nport: 70000\n",
		"name: \"api\nport: 8080\n",
		"name: api\nport: 8080\nname: web\n",
		"- just\n- a list\n",
	} {
		fmt.Printf("%-40q %v\n", src, check(src))
	}
}
"name: api\nport: 8080\n"                <nil>
"name: api\nport: 80a80\n"               2:7: port must be an integer from 1 to 65535
"name: api\nport: 70000\n"               2:7: port must be an integer from 1 to 65535
"name: \"api\nport: 8080\n"              2:1: syntax error: expected " ", "\n", "\r", "\r\n"
"name: api\nport: 8080\nname: web\n"     yaml: 3:1: duplicate key "name"
"- just\n- a list\n"                     the configuration must be a mapping

How it works

  • A ready-made parser is a package. go get it and import it; there is nothing to generate or compile. Each parser of parsers/ is generated by PEGO from a grammar, checked against the conformance suite and reference implementation of its language, and has the API of a generated parser (ParseAST, Parse, Recognize, *SyntaxError) plus functions written for the language. The README of each lists them.
  • Two levels of API. Load gives Go values (map[string]any, []any, int64, ...) with the YAML core schema, and reports what the syntax cannot show: here a repeated key, with a line and a column. ParseAST gives the typed tree, with a Span (start and end offsets) on every node and the style of every scalar.
  • Positions are offsets. A Span counts code points (bytes with yaml.ParseAST(src, yaml.Bytes)), so lineCol turns an offset into the line and column that people read. Report an error with its source line and a caret prints the line and a caret under it.
  • The YAML syntax error is verbose on purpose: it lists what could have come at the farthest position, as every PEGO parser does. Show it as it is, or trim it, as you like; the line and the column are fields of *yaml.SyntaxError.
  • Value() decodes. A scalar's Text is the source (with its quotes), and Value() is its content after the escapes of the quoting style are decoded.

Variations

  • Other formats: JSON (json.Decode, json.ParseAST), CSV, XML, CEL, CUE, Go, Python, TypeScript and DuckDB SQL work the same way. The Go parser also returns go/ast trees, for use with go/printer, go/format and the other tools of the standard library.
  • Only a check. yaml.Valid(src) or yaml.Recognize(src) build nothing.
  • A format with no ready-made parser: write the grammar, as in Read a configuration file into Go structs, and ship it as generated code.
  • The grammar of a ready-made parser is a .pego file in its directory (yaml.pego), and works with the engine too: for streaming, incremental parsing and the pego tools such as pego trace.