Repository navigation
Support BYTESLICE attributes across trace and exporter paths - #8153
Conversation
Codecov Reportโ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8153 +/- ##
=====================================
Coverage 82.2% 82.2%
=====================================
Files 309 309
Lines 24211 24235 +24
=====================================
+ Hits 19906 19931 +25
+ Misses 3927 3925 -2
- Partials 378 379 +1
๐ New features to boost your workflow:
|
Handle attribute.BYTESLICE in trace/auto attribute conversion. This keeps byte slice attributes from being dropped when auto-instrumented spans are converted into internal telemetry values. Add a regression test that verifies byte slice attributes are converted into telemetry bytes values.
Handle attribute.BYTESLICE in OTLP trace and metric attribute transforms. This updates the OTLP trace transform and both metric transforms to map byte slice attributes to AnyValue_BytesValue instead of falling through to the invalid default case. Add regression coverage for the trace transform and for the shared metric transform template and generated outputs.
Handle attribute.BYTESLICE explicitly in Zipkin tag serialization. Zipkin tag values are strings, so byte slices need a stable textual form. Serialize them as JSON arrays of byte values instead of falling back to the base64 string returned by Value.Emit. Add a regression test for byte slice tag serialization.
b973ecd to
d60d853
Compare
d60d853 to
373935f
Compare
Handle attribute.BYTESLICE in the OTLP log attribute transforms. This updates the shared otlplog transform template and the generated gRPC and HTTP outputs so standard attribute values are encoded as AnyValue_BytesValue instead of falling through to the invalid default case. Add regression coverage for both otlplog transform packages and record the package support in the changelog.
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end support for attribute.BYTESLICE so byte-slice attributes are preserved/encoded correctly across the remaining trace and exporter conversion paths (trace/auto, OTLP trace/log/metric transforms, and Zipkin), with regression tests.
Changes:
- Preserve
BYTESLICEattributes intrace/autoby converting them into internal telemetry bytes values. - Encode
BYTESLICEas OTLPAnyValue_BytesValueacross trace/log/metric transforms (including shared templates) and add regression tests. - Serialize Zipkin
BYTESLICEtags as JSON arrays of byte values (instead of base64 strings) and add a regression test.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| trace/auto.go | Add BYTESLICE handling to span attribute value conversion. |
| trace/auto_test.go | Add regression test for convAttrValue byte-slice conversion. |
| internal/shared/otlp/otlpmetric/transform/attribute.go.tmpl | Add OTLP bytes value encoding in metric attribute template. |
| internal/shared/otlp/otlpmetric/transform/attribute_test.go.tmpl | Add template test coverage for bytes attribute transformation. |
| internal/shared/otlp/otlplog/transform/log.go.tmpl | Add OTLP bytes value encoding in log attribute template. |
| internal/shared/otlp/otlplog/transform/attr_test.go.tmpl | Add template test coverage for bytes attribute transformation. |
| exporters/zipkin/model.go | Add explicit Zipkin serialization for BYTESLICE as JSON numeric arrays. |
| exporters/zipkin/model_test.go | Add regression test for Zipkin byte-slice tag serialization. |
| exporters/otlp/otlptrace/internal/tracetransform/attribute.go | Add OTLP trace BYTESLICE โ AnyValue_BytesValue mapping. |
| exporters/otlp/otlptrace/internal/tracetransform/attribute_test.go | Add regression coverage for OTLP trace bytes attributes. |
| exporters/otlp/otlpmetric/otlpmetrichttp/internal/transform/attribute.go | Add OTLP metric (HTTP) bytes attribute mapping. |
| exporters/otlp/otlpmetric/otlpmetrichttp/internal/transform/attribute_test.go | Add regression coverage for OTLP metric (HTTP) bytes attributes. |
| exporters/otlp/otlpmetric/otlpmetricgrpc/internal/transform/attribute.go | Add OTLP metric (gRPC) bytes attribute mapping. |
| exporters/otlp/otlpmetric/otlpmetricgrpc/internal/transform/attribute_test.go | Add regression coverage for OTLP metric (gRPC) bytes attributes. |
| exporters/otlp/otlplog/otlploghttp/internal/transform/log.go | Add OTLP log (HTTP) bytes attribute mapping. |
| exporters/otlp/otlplog/otlploghttp/internal/transform/attr_test.go | Add regression coverage for OTLP log (HTTP) bytes attributes. |
| exporters/otlp/otlplog/otlploggrpc/internal/transform/log.go | Add OTLP log (gRPC) bytes attribute mapping. |
| exporters/otlp/otlplog/otlploggrpc/internal/transform/attr_test.go | Add regression coverage for OTLP log (gRPC) bytes attributes. |
| CHANGELOG.md | Document BYTESLICE support fixes across trace and exporters. |
๐ก Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Do not we need to also implement attribute value limits for the new type?
Something like
opentelemetry-go/trace/auto.go
Lines 352 to 363 in 73876cb
There was a problem hiding this comment.
Fixes #8162 Follow-up to #8153 for `attribute.SLICE`. Add end-to-end exporter handling for `attribute.SLICE` in the remaining paths that still treated it as invalid or relied on fallback formatting. Changes: - encode `attribute.SLICE` as OTLP `AnyValue_ArrayValue` for trace, log, and metric transforms - serialize Zipkin `SLICE` attributes using the non-OTLP AnyValue string representation - add trace-side coverage for recursive `convAttrValue` slice conversion
โฆtelemetry#8153) Fixes open-telemetry#8164 Supersedes open-telemetry#8042 which does not address the `trace` or `zipkin` packages, and has unrelated support changes. Add end-to-end handling for `attribute.BYTESLICE` in the remaining trace and exporter paths that still dropped, invalidated, or stringified byte slice attributes. This change: - preserves byte slice attributes in `trace/auto` - encodes byte slice attributes as OTLP `AnyValue_BytesValue` in trace, log, and metric transforms - serializes Zipkin byte slice attributes as JSON arrays of byte values - adds regression tests for each updated path ## Problem `attribute.BYTESLICE` is public, but several downstream conversions still did not handle it correctly: - `trace/auto` dropped byte slice attributes during conversion - OTLP trace, log, and metric transforms fell through to their invalid default handling - Zipkin fell back to `Value.Emit()`, which produced a base64 string rather than an explicit byte-array representation That made `BYTESLICE` unusable or inconsistent depending on the export path. ## Changes ### Trace - Handle `attribute.BYTESLICE` in `trace/auto` by converting it to an internal telemetry bytes value. - Add a regression test covering byte slice conversion. ### OTLP - Handle `attribute.BYTESLICE` in: - trace attribute transform - log gRPC attribute transform - log HTTP attribute transform - metric HTTP attribute transform - metric gRPC attribute transform - Update the shared log and metric transform templates so generated outputs stay aligned. - Add regression tests for the trace transform, both log transform outputs, and both metric transform outputs. ### Zipkin - Handle `attribute.BYTESLICE` explicitly in Zipkin tag serialization. - Serialize byte slices as JSON arrays of byte values instead of base64 text. - Add a regression test for Zipkin byte slice serialization. --------- Co-authored-by: Robert Pajฤ k <pellared@hotmail.com>
Fixes open-telemetry#8162 Follow-up to open-telemetry#8153 for `attribute.SLICE`. Add end-to-end exporter handling for `attribute.SLICE` in the remaining paths that still treated it as invalid or relied on fallback formatting. Changes: - encode `attribute.SLICE` as OTLP `AnyValue_ArrayValue` for trace, log, and metric transforms - serialize Zipkin `SLICE` attributes using the non-OTLP AnyValue string representation - add trace-side coverage for recursive `convAttrValue` slice conversion
### Added - Add `ByteSlice` and `ByteSliceValue` functions for new `BYTESLICE` attribute type in `go.opentelemetry.io/otel/attribute`. (#7948) - Apply attribute value limit to the `KindBytes` attribute type in `go.opentelemetry.io/otel/sdk/log`. (#7990) - Apply attribute value limit to the `BYTESLICE` attribute type in `go.opentelemetry.io/otel/sdk/trace`. (#7990) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/trace`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlptrace`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlplog`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/zipkin`. (#8153) - Add `String` method for `Value` type in `go.opentelemetry.io/otel/attribute`. (#8142) - Add `Slice` and `SliceValue` functions for new `SLICE` attribute type in `go.opentelemetry.io/otel/attribute`. (#8166) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlptrace`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlplog`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/zipkin`. (#8216) - Apply `AttributeValueLengthLimit` to `attribute.SLICE` type attribute values in `go.opentelemetry.io/otel/sdk/trace`, recursively truncating contained string values. (#8217) - Add `Error` field on `Record` type in `go.opentelemetry.io/otel/log/logtest`. (#8148) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8157) - Add `Settable` to `go.opentelemetry.io/otel/metric/x` to allow reusing attribute options. (#8178) - Add experimental support for splitting metric data across multiple batches in `go.opentelemetry.io/otel/sdk/metric`. Set `OTEL_GO_X_METRIC_EXPORT_BATCH_SIZE=<max_size>` to enable for all periodic readers. See `go.opentelemetry.io/otel/sdk/metric/internal/x` for feature documentation. (#8071) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc/internal/x` for feature documentation. (#8192) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp/internal/x` for feature documentation. (#8194) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/stdout/stdoutlog`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/stdout/stdoutlog/internal/x` for feature documentation. (#8263) - Add `WithDefaultAttributes` to `go.opentelemetry.io/otel/metric/x` to support setting default attributes on instruments. (#8135) - Add `go.opentelemetry.io/otel/semconv/v1.41.0` package. The package contains semantic conventions from the `v1.41.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.41.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.40.0`. (#8324) - Add Observable variants of instruments to `go.opentelemetry.io/otel/semconv/v1.41.0` package. (#8350) - Generate explicit histogram bucket boundaries from weaver configuration for HTTP and RPC duration instruments in `go.opentelemetry.io/otel/semconv/v1.41.0`. (#8002) ### Changed -โ ๏ธ **Breaking Change:** `go.opentelemetry.io/otel/sdk/metric` now applies a default cardinality limit of 2000 to comply with the Metrics SDK specification recommendation. New attribute sets are dropped when the cardinality limit is reached. The measurement of these sets are aggregated into a special attribute set containing `attribute.Bool("otel.metric.overflow", true)`. This can break users who relied on the previous unlimited default. Set `WithCardinalityLimit(0)` or the deprecated `OTEL_GO_X_CARDINALITY_LIMIT=0` environment variable to preserve unlimited cardinality. Note that support for `OTEL_GO_X_CARDINALITY_LIMIT` may be removed in a future release. (#8247) - `ErrorType` in `go.opentelemetry.io/otel/semconv` now unwraps errors created with `fmt.Errorf` when deriving the `error.type` attribute. (#8133) - `go.opentelemetry.io/otel/sdk/log` now unwraps error chains created with `fmt.Errorf` when deriving the `error.type` attribute from errors on log records. (#8133) - `Set.MarshalLog` method in `go.opentelemetry.io/otel/attribute` now uses `Value.String` formatting following the [OpenTelemetry AnyValue representation for non-OTLP protocols](https://opentelemetry.io/docs/specs/otel/common/#anyvalue). (#8169) - Optimize `go.opentelemetry.io/otel/sdk/metric` to return a drop reservoir and short-circuit `Offer` calls to the exemplar reservoir when `exemplar.AlwaysOffFilter` is configured. (#8211) (#8267) - Optimize `go.opentelemetry.io/otel/sdk/metric` to return a drop reservoir for asynchronous instruments when `exemplar.TraceBasedFilter` is configured. (#8286) ### Deprecated - Deprecate `Value.Emit` method in `go.opentelemetry.io/otel/attribute`. Use `Value.String` instead. (#8176) ### Fixed - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Fix gzipped request body replay on redirect in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#8135) - Fix gzipped request body replay on redirect in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8152) - `go.opentelemetry.io/otel/exporters/prometheus` now uses `Value.String` formatting for label values following the [OpenTelemetry AnyValue representation for non-OTLP protocols](https://opentelemetry.io/docs/specs/otel/common/#anyvalue). (#8170) - Propagate errors from the exporter when calling `Shutdown` on `BatchSpanProcessor` in `go.opentelemetry.io/otel/sdk/trace`. (#8197) - Fix stale status code reporting on self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` and `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8226) - Fix a concurrent `Collect` data race and potential panic in `go.opentelemetry.io/otel/exporters/prometheus` when `WithResourceAsConstantLabels` option is used. (#8227) - Fix race condition in `FixedSizeReservoir` in `go.opentelemetry.io/otel/sdk/metric/exemplar` by reverting #7447. (#8249) - Fix `FixedSizeReservoir` in `go.opentelemetry.io/otel/sdk/metric/exemplar` to safely handle zero size. A capacity check in the constructor initializes the reservoir safely and skips initialization for zero-cap; early returns in `Offer()` and `Collect()` ensure no-op behavior. (#8295) - Fix counting of spans and logs in self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`, `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`, `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`, and `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8254) - Drop conflicting scope attributes named `name`, `version`, or `schema_url` from metric labels in `go.opentelemetry.io/otel/exporters/prometheus`, preserving the dedicated `otel_scope_name`, `otel_scope_version`, and `otel_scope_schema_url` labels. (#8264) - Close schema files opened by `ParseFile` in `go.opentelemetry.io/otel/schema/v1.0` and `go.opentelemetry.io/otel/schema/v1.1`. ([GHSA-995v-fvrw-c78m](GHSA-995v-fvrw-c78m)) - Enforce the 8192-byte baggage size limit during extraction/parsing, changing behavior when the limit is exceeded in `go.opentelemetry.io/otel/baggage` and `go.opentelemetry.io/otel/propagation`. (#8222) - Fix `go.opentelemetry.io/otel/semconv/v1.41.0` to include `Attr*` helper methods for required attributes on observable instruments. (#8361) - Limit baggage extraction error reporting in `go.opentelemetry.io/otel/propagation` to prevent malformed or oversized baggage headers from flooding logs. ([GHSA-5wrp-cwcj-q835](GHSA-5wrp-cwcj-q835))
Fixes #8164
Supersedes #8042 which does not address the
traceorzipkinpackages, and has unrelated support changes.Add end-to-end handling for
attribute.BYTESLICEin the remaining trace and exporter paths that still dropped, invalidated, or stringified byte slice attributes.This change:
trace/autoAnyValue_BytesValuein trace, log, and metric transformsProblem
attribute.BYTESLICEis public, but several downstream conversions still did not handle it correctly:trace/autodropped byte slice attributes during conversionValue.Emit(), which produced a base64 string rather than an explicit byte-array representationThat made
BYTESLICEunusable or inconsistent depending on the export path.Changes
Trace
attribute.BYTESLICEintrace/autoby converting it to an internal telemetry bytes value.OTLP
attribute.BYTESLICEin:Zipkin
attribute.BYTESLICEexplicitly in Zipkin tag serialization.