Skip to content

Deliver the whole document to a root Unmarshaler (#994) - #1083

Merged
pelletier merged 1 commit into
v2from
fix-994-root-unmarshaler
Jun 24, 2026
Merged

pelletier merged 1 commit into
v2from
fix-994-root-unmarshaler

Conversation

@pelletier

Copy link
Copy Markdown
Owner

Fixes #994.

Problem

When the decode target itself implements unstable.Unmarshaler (with EnableUnmarshalerInterface()), UnmarshalTOML was not receiving the document. Top-level key-values were delegated one value at a time, and a document split into pieces (top-level keys followed by tables) arrived as disconnected fragments instead of a single document:

Input Before After
foo = "bar" called once with "bar" (just the value) called once with foo = "bar"\n
two top-level keys called twice, once per value called once, whole doc
title=… then [owner]… called twice with fragments called once, whole doc

This made a root Unmarshaler unusable for its main purpose — re-parsing the full document, e.g. to preserve key ordering, which is exactly what the issue asked for. PR #996 only delegated unmatched top-level keys individually and, as noted when it was merged, never handled piece-wise tables.

Fix

The rewritten decoder already accumulates a whole table (including split and array tables) into a single raw "capture" that is delivered to UnmarshalTOML once. This routes the root through the same machinery: when the root target implements the interface, a capture spanning the entire document is opened up front. Top-level key-values then flow through the existing capture branch, and any tables attach to it via resumeCapture, so UnmarshalTOML is called exactly once with a self-contained TOML document that re-parses to the original structure.

The change is gated on the opt-in interface, so it adds no work to the default decode path.

Example: preserving key order (the issue's use-case)

// OrderedConfig records top-level keys in document order. Because it is the
// decode target and implements unstable.Unmarshaler, it receives the whole
// document and re-parses it with the unstable parser.
type OrderedConfig struct {
	Keys []string
}

func (c *OrderedConfig) UnmarshalTOML(data []byte) error {
	var p unstable.Parser
	p.Reset(data)
	for p.NextExpression() {
		e := p.Expression()
		if e.Kind != unstable.KeyValue {
			continue
		}
		it := e.Key()
		it.Next() // first part of the (possibly dotted) key
		c.Keys = append(c.Keys, string(it.Node().Data))
	}
	return p.Error()
}

func main() {
	doc := `gamma = 1
alpha = 2
beta  = 3`

	var cfg OrderedConfig
	if err := toml.NewDecoder(strings.NewReader(doc)).
		EnableUnmarshalerInterface().
		Decode(&cfg); err != nil {
		panic(err)
	}
	fmt.Println(cfg.Keys) // [gamma alpha beta]
}

Before this change, cfg.Keys would have been empty (the unmarshaler was handed only the value 1, twice).

Tests

  • Updated TestIssue994_OK, which had baked in the old partial-value result.
  • Added TestIssue994_WholeDocument (single/multiple keys, keys-then-table, keys-then-array-table, only-table, dotted keys, empty doc): asserts UnmarshalTOML is called exactly once and that the bytes round-trip back to the same structure as the original input.
  • Added TestIssue994_ErrorPropagation for the multi-expression capture path.

Full suite passes (go test ./...).

Performance

No regression. Interleaved A/B benchstat on linux/amd64 (go1.26.4, n=12), all benchmarks not statistically significant — allocs/op byte-identical:

Unmarshal/SimpleDocument/struct-8   335.2n ± 2%   335.6n ± 1%   ~ (p=0.385 n=12)
Unmarshal/SimpleDocument/map-8      386.4n ± 3%   387.9n ± 1%   ~ (p=0.854 n=12)
Unmarshal/ReferenceFile/struct-8    36.51µ ± 0%   36.46µ ± 1%   ~ (p=1.000 n=12)
Unmarshal/ReferenceFile/map-8       29.72µ ± 1%   29.88µ ± 2%   ~ (p=0.478 n=12)
Unmarshal/HugoFrontMatter-8         5.226µ ± 1%   5.192µ ± 2%   ~ (p=0.347 n=12)

🤖 Generated with Claude Code

When the decode target itself implements unstable.Unmarshaler (with the
interface enabled), top-level key-values were handed to UnmarshalTOML one
value at a time, and a document split into pieces (top-level keys followed by
tables) was delivered as disconnected fragments rather than as one document.
That made the root unmarshaler unusable for its main purpose: re-parsing the
whole document (e.g. to preserve key ordering), which is what issue #994 asked
for. PR #996 only delegated unmatched top-level keys individually and, as
noted at the time, never handled piece-wise tables.

The rewritten decoder already accumulates a whole table (including split and
array tables) into a single raw "capture" delivered to UnmarshalTOML once.
When the root target implements the interface, open such a capture spanning
the entire document up front: top-level key-values then flow through the
existing capture branch and any tables attach to it via resumeCapture, so
UnmarshalTOML is called exactly once with a self-contained TOML document that
re-parses to the original structure. Gated on the opt-in interface, this adds
no work to the default decode path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pelletier pelletier added the bug Issues describing a bug in go-toml. label Jun 24, 2026
@pelletier
pelletier merged commit 21f8286 into v2 Jun 24, 2026
13 checks passed
@pelletier
pelletier deleted the fix-994-root-unmarshaler branch June 24, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Issues describing a bug in go-toml.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UnmarshalTOML isn't called when expected

1 participant