Skip to content

feat(specs): add decision (declared winner) to abtesting-v3 ABTest - #6732

Merged
Fluf22 merged 4 commits into
mainfrom
feat/abtesting-v3-decision
Aug 11, 2026
Merged

Fluf22 merged 4 commits into
mainfrom
feat/abtesting-v3-decision

Conversation

@cdhawke

@cdhawke cdhawke commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

PR Stack:

What

Add an optional decision object to the abtesting-v3 ABTest schema, so it appears on getABTest and listABTests:

decision:
  winnerVariantId: integer   # declared winning variant (control = 1)
  declaredAt: string         # RFC 3339

Present: the test has been decided (a winner declared). Absent: no winner declared. Modelled as an object (not a bare flag) so future decision signals, e.g. Bayesian probability-to-be-better, can join the same container, covering both frequentist and Bayesian outcomes.

Why

The decided state lives only in the abtests settings table server-side, so the dashboard otherwise needs a separate /settings request per test to know it, which is prohibitive on the list view. Surfacing it on the A/B test object lets the dashboard read it from the single retrieval call it already makes.

How

Only the hand-written spec (specs/abtesting-v3/common/schemas/ABTest.yml) is committed here; per this repo's workflow, CI regenerates the clients. Verified locally that yarn cli generate javascript abtesting-v3 produces the expected Decision model and ABTest.decision?: Decision field. yarn cli build specs abtesting-v3 passes.

Dependencies

  • algolia/go#26540 (the server field) has merged, so this is unblocked. Kept as a draft on purpose: it is parked for a possible future winner/decision affordance in the dashboard, not required by any shipping product yet.
  • Decoupled from the dashboard Evidence status work (algolia/AlgoliaWeb#29384). That PR is frequentist-only and does not consume decision; it answers "is there enough data?", which is separate from "was a winner declared?". A future dashboard winner affordance can bump the regenerated @algolia/abtesting and consume decision then.

preview:
image
(this has been rolled out and is being returned by the api)

Adds an optional `decision` object to the abtesting-v3 ABTest schema, surfaced
on getABTest and listABTests. It reflects a declared winner and is a container
for future decision signals (e.g. Bayesian probability-to-be-better):

    decision: { winnerVariantId: integer, declaredAt: string }

Present => the test has been decided; absent => no winner declared.

Regenerated the @algolia/abtesting JS client (the model only). Backed by the
server change in algolia/go#26540.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@algolia-api-clients-automation-bot

algolia-api-clients-automation-bot Bot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

✔️ Code generated!

Name Link
🪓 Triggered by 82bc1e663cbeeb22b4764a103e2a5b3d6e9f9bc5
🍃 Generated commit 2c3087487501dbcf0282e3a379b76666649f703f
🌲 Generated branch generated/feat/abtesting-v3-decision
📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language Req/s
javascript 2242
go 1754
php 1697
csharp 1472
python 1447
swift 1142
java 1100
ruby 926
scala 657

Drop the speculative language about future Bayesian signals from the decision field description. The description ships into the generated clients, so it should state what the field is, not internal roadmap intent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cdhawke
cdhawke marked this pull request as ready for review July 27, 2026 17:20
@cdhawke
cdhawke requested a review from a team as a code owner July 27, 2026 17:20
@cdhawke

cdhawke commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

@Fluf22 @MarioAlexandruDan could I get a review on this one please?

@Fluf22
Fluf22 enabled auto-merge (squash) August 11, 2026 21:23
@Fluf22
Fluf22 merged commit 65cc642 into main Aug 11, 2026
37 checks passed
@Fluf22
Fluf22 deleted the feat/abtesting-v3-decision branch August 11, 2026 21:43
algolia-api-clients-automation-bot Bot added a commit that referenced this pull request Aug 11, 2026
…6732) (generated) [skip ci]

Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-csharp that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-dart that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-go that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-java that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-javascript that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-kotlin that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-php that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-python that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-ruby that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-scala that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
algolia-api-clients-automation-bot Bot added a commit to algolia/algoliasearch-client-swift that referenced this pull request Aug 11, 2026
…enerated)

algolia/api-clients-automation#6732

Co-authored-by: algolia-api-clients-automation-bot[bot] <288895823+algolia-api-clients-automation-bot[bot]@users.noreply.github.com>
Co-authored-by: Christopher Hawke <69921547+cdhawke@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants