Repository navigation
fix(specs): BREAKING CHANGE - make AB test pValue optional - #7066
Conversation
π₯ Breaking changes detected !!Either this PR or a previous PR not released yet introduced breaking changes, be careful when merging. βοΈ Code generated!
π Benchmark resultsBenchmarks performed on the method using a mock server, the results might not reflect the real-world performance.
|
571a47d to
d31d2e7
Compare
fdaf09f to
8352939
Compare
55bd737 to
3f77878
Compare
There was a problem hiding this comment.
π’ Approval recommended
The focused schema change correctly represents the documented runtime behavior without introducing inconsistencies.
Pull request overview
Aligns the A/B Testing v3 schema with API responses that may omit pValue.
Changes:
- Makes
metricResult.pValueoptional. - Documents omission behavior and updates the example.
File summaries
| File | Description |
|---|---|
specs/abtesting-v3/common/schemas/Variant.yml |
Updates the pValue schema contract and documentation. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Balanced
π‘ Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
βΉοΈ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with π while any review is running, comments if it has suggestions, and reacts with π once all reviews finish with no findings. |
d31d2e7 to
c7daff8
Compare
3f77878 to
1543400
Compare
c7daff8 to
57555d2
Compare
57555d2 to
8154217
Compare
1543400 to
bb6471b
Compare
bb6471b to
a948787
Compare
a948787 to
48ef6bb
Compare
Summary by Aikido
β‘ Enhancements
|
|
@Fluf22 thank you for all the reviews! π Because the Github stack required rebasing after each PR merge, I've merged the stack into this single feature branch that will need your sign-off one more time. |
β¦enerated) [skip ci] Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
algolia/api-clients-automation#7066 Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com> Co-authored-by: Steven Evans <steven.evans@algolia.com>
π§ What and Why
The A/B Testing v3 API has allowed
metricResult.pValueto be omitted for some time when no Frequentist result is available. The OpenAPI spec still marks it as required, so generated clients do not accurately represent the existing API behavior and may reject valid responses.This PR brings the spec and generated public model types back in line with the runtime contract. Bayesian tests exposed the mismatch because Bayesian-only responses omit Frequentist inference fields, but the mismatch predates that work and is not specific to Bayesian testing.
Changes included:
metricResult.pValueoptional.pValue.This changes generated public model types from required to optional or nullable across client languages and should be handled as a breaking client release.
Changelog
metricResult.pValueis now optional in generated A/B Testing v3 clients, matching API responses where no p-value is available for a metric. This is a breaking type change: code that assumespValueis always present must handle an absent or nullable value.π§ͺ Tests
Green CI