Skip to content

Fix zero attribute value length for malformed UTF-8 - #9054

Merged
pellared merged 6 commits into
open-telemetry:mainfrom
pellared:fix-9053-zero-length-invalid-utf8
Oct 1, 2026
Merged

pellared merged 6 commits into
open-telemetry:mainfrom
pellared:fix-9053-zero-length-invalid-utf8

Conversation

@pellared

@pellared pellared commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Fixes #9053.

Fix zero attribute value length truncation for a single malformed UTF-8 byte in the shared SDK normalizer and the independent auto-instrumented trace path.

Benchmark

benchstat on AMD EPYC 7571 with Go 1.27, -cpu=1 -benchtime=200ms -count=8, comparing upstream main with this branch (ns/op; no allocation changes):

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/internal/attrnorm
cpu: AMD EPYC 7571
                    โ”‚ /tmp/otel-9053-sdk-final-before.bench โ”‚ /tmp/otel-9053-sdk-final-after.bench โ”‚
                    โ”‚                sec/op                 โ”‚     sec/op      vs base              โ”‚
Truncate/Zero                                   6.784n ยฑ 0%      7.207n ยฑ 0%  +6.22% (p=0.000 n=8)
Truncate/ASCII                                  13.49n ยฑ 0%      13.86n ยฑ 0%  +2.78% (p=0.000 n=8)
Truncate/ValidUTF-8                             25.53n ยฑ 1%      27.14n ยฑ 1%  +6.31% (p=0.000 n=8)
geomean                                         13.27n           13.94n       +5.09%

                    โ”‚ /tmp/otel-9053-sdk-final-before.bench โ”‚ /tmp/otel-9053-sdk-final-after.bench โ”‚
                    โ”‚                 B/op                  โ”‚     B/op      vs base                โ”‚
Truncate/Zero                                  0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ASCII                                 0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ValidUTF-8                            0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
geomean                                                   ยฒ                 +0.00%               ยฒ
ยน all samples are equal
ยฒ summaries must be >0 to compute geomean

                    โ”‚ /tmp/otel-9053-sdk-final-before.bench โ”‚ /tmp/otel-9053-sdk-final-after.bench โ”‚
                    โ”‚               allocs/op               โ”‚  allocs/op    vs base                โ”‚
Truncate/Zero                                  0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ASCII                                 0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ValidUTF-8                            0.000 ยฑ 0%       0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
geomean                                                   ยฒ                 +0.00%               ยฒ
ยน all samples are equal
ยฒ summaries must be >0 to compute geomean
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/trace
cpu: AMD EPYC 7571
                    โ”‚ /tmp/otel-9053-trace-controlled-before.bench โ”‚ /tmp/otel-9053-trace-controlled-after.bench โ”‚
                    โ”‚                    sec/op                    โ”‚        sec/op         vs base               โ”‚
Truncate/Zero                                          7.987n ยฑ 0%            6.802n ยฑ 1%  -14.85% (p=0.000 n=8)
Truncate/ASCII                                         16.32n ยฑ 0%            15.17n ยฑ 0%   -7.05% (p=0.000 n=8)
Truncate/ValidUTF-8                                    30.71n ยฑ 0%            29.54n ยฑ 1%   -3.83% (p=0.000 n=8)
geomean                                                15.88n                 14.50n        -8.69%

                    โ”‚ /tmp/otel-9053-trace-controlled-before.bench โ”‚ /tmp/otel-9053-trace-controlled-after.bench โ”‚
                    โ”‚                     B/op                     โ”‚        B/op          vs base                โ”‚
Truncate/Zero                                         0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ASCII                                        0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ValidUTF-8                                   0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
geomean                                                          ยฒ                        +0.00%               ยฒ
ยน all samples are equal
ยฒ summaries must be >0 to compute geomean

                    โ”‚ /tmp/otel-9053-trace-controlled-before.bench โ”‚ /tmp/otel-9053-trace-controlled-after.bench โ”‚
                    โ”‚                  allocs/op                   โ”‚      allocs/op       vs base                โ”‚
Truncate/Zero                                         0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ASCII                                        0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
Truncate/ValidUTF-8                                   0.000 ยฑ 0%              0.000 ยฑ 0%       ~ (p=1.000 n=8) ยน
geomean                                                          ยฒ                        +0.00%               ยฒ
ยน all samples are equal
ยฒ summaries must be >0 to compute geomean

@codecov

codecov Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codecov Report

โœ… All modified and coverable lines are covered by tests.
โœ… Project coverage is 89.1%. Comparing base (75b1fe3) to head (2de8fd1).

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #9054   +/-   ##
=====================================
  Coverage   89.1%   89.1%           
=====================================
  Files        338     338           
  Lines      22274   22280    +6     
=====================================
+ Hits       19855   19861    +6     
  Misses      2419    2419           
Files with missing lines Coverage ฮ”
sdk/internal/attrnorm/truncate.go 100.0% <100.0%> (รธ)
sdk/log/internal/attrnorm/truncate.go 100.0% <100.0%> (รธ)
trace/auto.go 98.0% <100.0%> (+<0.1%) โฌ†๏ธ

... and 1 file with indirect coverage changes

๐Ÿš€ New features to boost your workflow:
  • โ„๏ธ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread CHANGELOG.md Resolved
@pellared
pellared marked this pull request as ready for review September 30, 2026 16:19
@pellared
pellared requested a lite review from Copilot September 30, 2026 16:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

๐ŸŸก Changes recommended

Update the changelog to include the root trace package.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Fixes zero-length attribute truncation for malformed UTF-8 across SDK and auto-instrumented trace paths.

Changes:

  • Corrects zero-limit handling.
  • Adds regression tests.
  • Updates the changelog.
File Description
trace/โ€‹auto.go Fixes auto-trace truncation.
trace/โ€‹auto_test.go Adds regression tests.
sdk/โ€‹log/โ€‹internal/โ€‹attrnorm/โ€‹truncate.go Updates generated log normalizer.
sdk/โ€‹log/โ€‹internal/โ€‹attrnorm/โ€‹truncate_test.go Adds log normalizer tests.
sdk/โ€‹internal/โ€‹attrnorm/โ€‹truncate.go Updates generated SDK normalizer.
sdk/โ€‹internal/โ€‹attrnorm/โ€‹truncate_test.go Adds SDK normalizer tests.
internal/โ€‹shared/โ€‹attrnorm/โ€‹truncate.go.tmpl Updates shared truncation template.
internal/โ€‹shared/โ€‹attrnorm/โ€‹truncate_test.go.tmpl Updates shared test template.
CHANGELOG.md Documents the fix.

๐Ÿ’ก Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread CHANGELOG.md Resolved
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

@ps-mir ps-mir left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

truncate's count doesn't look accurate for direct use of \uFFFD, which has unexpected results with non zero limits.

The fix works for limit = 0.

@pellared
pellared merged commit 55899e3 into open-telemetry:main Oct 1, 2026
50 checks passed
@pellared
pellared deleted the fix-9053-zero-length-invalid-utf8 branch October 1, 2026 18:39
pellared added a commit that referenced this pull request Oct 2, 2026
This release contains the first stable release of the OpenTelemetry Go
Logs API and SDK.
Our project stability guarantees now apply to the following modules:

- `go.opentelemetry.io/otel/log`
- `go.opentelemetry.io/otel/sdk/log`

See our [versioning
policy](https://github.com/open-telemetry/opentelemetry-go/blob/main/VERSIONING.md)
for more information about these stability guarantees.

### Added

- Add `String` method for `KeyValue` type in
`go.opentelemetry.io/otel/attribute`. (#8205)
- Add `String` method for `Set` type in
`go.opentelemetry.io/otel/attribute`. (#8347)
- Add `WithMaxExportBatchSize` to `go.opentelemetry.io/otel/sdk/metric`
to configure the maximum export batch size for `PeriodicReader`. (#8960)
- Add `WithAttributeValueDepthLimit`, `DefaultAttributeValueDepthLimit`,
and `SpanLimits.AttributeValueDepthLimit` in
`go.opentelemetry.io/otel/sdk/trace` to configure the maximum depth of
span, event, link, and instrumentation scope attribute values. (#8938)
- Add `WithAttributeValueDepthLimit` in
`go.opentelemetry.io/otel/sdk/log` to configure the maximum depth of log
record and instrumentation scope attribute values. (#8938)
- Add `WithMaxResponseSize` to
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`,
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`, and
  `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8941)
- Add experimental `ProbabilitySampler` in
`go.opentelemetry.io/otel/sdk/trace/x` that conforms to the
[OpenTelemetry specification's threshold-based sampling
algorithm](https://opentelemetry.io/docs/specs/otel/trace/sdk/#probabilitysampler).
(#8123)
- Add experimental `*Binder` extension interfaces in
`go.opentelemetry.io/otel/metric/x` for instruments that support binding
attributes ahead of time. (#8760)
- Add the experimental `Finisher` synchronous metric instrument
extension interface to `go.opentelemetry.io/otel/metric/x`. (#8906)

### Changed

- Use the [OpenTelemetry Attribute Collection representation for
non-OTLP
protocols](https://opentelemetry.io/docs/specs/otel/common/#attribute-collection-representation-for-non-otlp)
for metric data diffs in
`go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest`. (#8993)
- Apply a maximum depth of 64 by default to span, event, link, log
record, and instrumentation scope attribute values in
`go.opentelemetry.io/otel/sdk/trace` and
`go.opentelemetry.io/otel/sdk/log`. (#8938)
- Reduce allocations when applying attribute value length limits in
`go.opentelemetry.io/otel/sdk/trace` and
`go.opentelemetry.io/otel/sdk/log` by rebuilding composite values only
when truncation is needed. (#8912)
- Decode `traceparent` using a hex lookup table in
`go.opentelemetry.io/otel/propagation`, which rejects the
specification-disallowed upper-case characters without a separate scan
of the header. (#8739)
- Decode all characters at constant indices in `TraceIDFromHex` and
`SpanIDFromHex` in `go.opentelemetry.io/otel/trace` to eliminate bounds
checks. (#8740)

### Fixed

- Count valid U+FFFD replacement characters toward attribute value
length limits in `go.opentelemetry.io/otel/trace`,
`go.opentelemetry.io/otel/sdk/trace`, and
`go.opentelemetry.io/otel/sdk/log`.
- Truncate string attribute values to empty strings when the configured
length limit is zero, including malformed UTF-8, in
`go.opentelemetry.io/otel/trace`, `go.opentelemetry.io/otel/sdk/trace`,
and `go.opentelemetry.io/otel/sdk/log`. (#9054)
- Fix a data race in `NewPeriodicReader` where `r.inst` was assigned
after the background goroutine was launched in
`go.opentelemetry.io/otel/sdk/metric`. (#9028)
- Prevent precision loss for large `int64` values in `Sum`, `Histogram`,
and `ExponentialHistogram` aggregations in
`go.opentelemetry.io/otel/sdk/metric`. (#8981)
- Ensure `grpc.DialOption` values passed via `WithDialOption` take
precedence over conflicting internally-computed defaults in
  `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`,
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`, and
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#8805, #8834, #8836)
- Treat overflowing `OTEL_METRIC_EXPORT_INTERVAL` and
`OTEL_METRIC_EXPORT_TIMEOUT` values as invalid in
`go.opentelemetry.io/otel/sdk/metric`. (#8800)
- Prevent a panic in `NewFixedSizeReservoir` and
`FixedSizeReservoirProvider` when given a negative size in
`go.opentelemetry.io/otel/sdk/metric/exemplar`; negative sizes are now
clamped to zero, consistent with a size of zero. (#8832)
- Preserve exponential histogram `ZeroThreshold` during OTLP metric
export in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. (#8801)
- Ensure metric helpers in `go.opentelemetry.io/otel/semconv/v1.32.0`,
`go.opentelemetry.io/otel/semconv/v1.33.0`, and
`go.opentelemetry.io/otel/semconv/v1.34.0` record measurements only once
when no attributes are provided. (#8849)
- Treat empty `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG` values
as unset in `go.opentelemetry.io/otel/sdk/trace`. (#8873)
- Normalize global `OTEL_EXPORTER_OTLP_ENDPOINT` path joining in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` so
trailing-slash base URLs do not produce double-slash log export paths.
(#8864)
- Prevent a deadlock when formatting an error passed to `RecordError`
calls back into the span in `go.opentelemetry.io/otel/sdk/trace`.
(#8815)
- Prevent `SimpleSpanProcessor.Shutdown` from panicking when constructed
with a nil exporter in `go.opentelemetry.io/otel/sdk/trace`. (#8844)
- Propagate invalid exponential histogram scale errors to Prometheus
exporter self-observability metrics in
`go.opentelemetry.io/otel/exporters/prometheus`. (#8839)
- Retain instrument advisory attributes configured with
`go.opentelemetry.io/otel/metric/x.WithDefaultAttributes` when a
matching View in `go.opentelemetry.io/otel/sdk/metric` does not specify
an attribute filter. (#8859)
- Ignore HTTP URL paths when building OTLP/gRPC metric exporter targets
from `OTEL_EXPORTER_OTLP_ENDPOINT` and
`OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, while preserving `unix://` and
`unix-abstract://` targets; treat `unix-abstract://` endpoints as
insecure in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` and
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`.
(#8862)
- Ignore HTTP(S) paths when deriving gRPC trace exporter endpoints from
`OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`
in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#8852)
- Bound memory used while parsing dense W3C property metadata by storing
it compactly and materializing `Property` values on demand in
`go.opentelemetry.io/otel/baggage` and
`go.opentelemetry.io/otel/propagation`.

### Removed

- Remove the experimental `OTEL_GO_X_METRIC_EXPORT_BATCH_SIZE`
environment variable in `go.opentelemetry.io/otel/sdk/metric`.
  Use the `WithMaxExportBatchSize` option instead. (#8960)
@pellared pellared added this to the v1.47.0 milestone Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zero attribute value length limit preserves a single invalid UTF-8 byte

5 participants