id stringclasses 400
values | workflow stringclasses 4
values | question stringclasses 15
values | type stringclasses 3
values | probabilities unknown |
|---|---|---|---|---|
agent_trace_observability_000000 | agent_trace_observability | action | choice | {
"continue": 0.3083462226,
"human_review": 0.0009862971,
"observe": 0.4814648613,
"stop": 0.209202619
} |
agent_trace_observability_000000 | agent_trace_observability | needs_review | noul | {
"false": 0.9965405386,
"true": 0.0034594614
} |
agent_trace_observability_000000 | agent_trace_observability | outcome | choice | {
"failure": 0.0000016953,
"harmful": 1.4000000000000001e-9,
"partial": 0.0000720875,
"success": 0.9999262158000001
} |
agent_trace_observability_000000 | agent_trace_observability | risk | score | {
"0": 0.030527984600000002,
"1": 0.9681683224000001,
"2": 0.0012995993000000001,
"3": 0.0000040937
} |
agent_trace_observability_000000 | agent_trace_observability | urgency | score | {
"0": 0.9516635229,
"1": 0.048180053800000004,
"2": 0.00015601380000000001,
"3": 4.095e-7
} |
agent_trace_observability_000001 | agent_trace_observability | action | choice | {
"continue": 0.2951102752,
"human_review": 0.0438695624,
"observe": 0.46079769200000004,
"stop": 0.2002224705
} |
agent_trace_observability_000001 | agent_trace_observability | needs_review | noul | {
"false": 0.8792409759,
"true": 0.1207590241
} |
agent_trace_observability_000001 | agent_trace_observability | outcome | choice | {
"failure": 0.008637012500000001,
"harmful": 2.913e-7,
"partial": 0.20179977030000001,
"success": 0.789562926
} |
agent_trace_observability_000001 | agent_trace_observability | risk | score | {
"0": 0.7564684821000001,
"1": 0.2431496276,
"2": 0.0003812431,
"3": 6.472e-7
} |
agent_trace_observability_000001 | agent_trace_observability | urgency | score | {
"0": 0.8775344519,
"1": 0.1215707951,
"2": 0.0008933513,
"3": 0.0000014018
} |
agent_trace_observability_000002 | agent_trace_observability | action | choice | {
"continue": 0.049580339300000005,
"human_review": 0.8393642124,
"observe": 0.0774168432,
"stop": 0.0336386051
} |
agent_trace_observability_000002 | agent_trace_observability | needs_review | noul | {
"false": 0.11566692540000001,
"true": 0.8843330746
} |
agent_trace_observability_000002 | agent_trace_observability | outcome | choice | {
"failure": 0.6786735592,
"harmful": 2.087e-7,
"partial": 0.3210843149,
"success": 0.0002419172
} |
agent_trace_observability_000002 | agent_trace_observability | risk | score | {
"0": 0.47357542560000004,
"1": 0.522170164,
"2": 0.0042488255,
"3": 0.000005585000000000001
} |
agent_trace_observability_000002 | agent_trace_observability | urgency | score | {
"0": 0.0392567978,
"1": 0.8090162858000001,
"2": 0.15144580100000002,
"3": 0.00028111550000000003
} |
agent_trace_observability_000003 | agent_trace_observability | action | choice | {
"continue": 0.3068751276,
"human_review": 0.005752511,
"observe": 0.4791678312,
"stop": 0.2082045301
} |
agent_trace_observability_000003 | agent_trace_observability | needs_review | noul | {
"false": 0.9828522359,
"true": 0.0171477641
} |
agent_trace_observability_000003 | agent_trace_observability | outcome | choice | {
"failure": 0.0006817474,
"harmful": 5.5100000000000004e-8,
"partial": 0.0795065588,
"success": 0.9198116387
} |
agent_trace_observability_000003 | agent_trace_observability | risk | score | {
"0": 0.4387968285,
"1": 0.5604984145,
"2": 0.0007037629,
"3": 9.942e-7
} |
agent_trace_observability_000003 | agent_trace_observability | urgency | score | {
"0": 0.8502283487000001,
"1": 0.14903640440000002,
"2": 0.0007343928,
"3": 8.541e-7
} |
agent_trace_observability_000004 | agent_trace_observability | action | choice | {
"continue": 0.3037923263,
"human_review": 0.0157405069,
"observe": 0.47435421450000004,
"stop": 0.2061129523
} |
agent_trace_observability_000004 | agent_trace_observability | needs_review | noul | {
"false": 0.9513756661,
"true": 0.048624333900000004
} |
agent_trace_observability_000004 | agent_trace_observability | outcome | choice | {
"failure": 0.0101966776,
"harmful": 4.836e-7,
"partial": 0.4385246065,
"success": 0.5512782323000001
} |
agent_trace_observability_000004 | agent_trace_observability | risk | score | {
"0": 0.1507282993,
"1": 0.8466997625,
"2": 0.0025695048,
"3": 0.0000024334000000000003
} |
agent_trace_observability_000004 | agent_trace_observability | urgency | score | {
"0": 0.5197057696,
"1": 0.47622355980000003,
"2": 0.0040621339,
"3": 0.0000085367
} |
agent_trace_observability_000005 | agent_trace_observability | action | choice | {
"continue": 0.2995442752,
"human_review": 0.0295038058,
"observe": 0.4677211275,
"stop": 0.2032307915
} |
agent_trace_observability_000005 | agent_trace_observability | needs_review | noul | {
"false": 0.9449166134,
"true": 0.0550833866
} |
agent_trace_observability_000005 | agent_trace_observability | outcome | choice | {
"failure": 0.0005548138,
"harmful": 1.085e-7,
"partial": 0.033184416700000004,
"success": 0.9662606610000001
} |
agent_trace_observability_000005 | agent_trace_observability | risk | score | {
"0": 0.0032841685000000002,
"1": 0.991273681,
"2": 0.0054275918,
"3": 0.0000145587
} |
agent_trace_observability_000005 | agent_trace_observability | urgency | score | {
"0": 0.2152359455,
"1": 0.7565776025000001,
"2": 0.0281548458,
"3": 0.0000316062
} |
agent_trace_observability_000006 | agent_trace_observability | action | choice | {
"continue": 0.0000582538,
"human_review": 0.9998112631,
"observe": 0.0000909599,
"stop": 0.0000395232
} |
agent_trace_observability_000006 | agent_trace_observability | needs_review | noul | {
"false": 0.0002431823,
"true": 0.9997568177
} |
agent_trace_observability_000006 | agent_trace_observability | outcome | choice | {
"failure": 0.00010341730000000001,
"harmful": 0.9998845876,
"partial": 0.0000119937,
"success": 1.4000000000000001e-9
} |
agent_trace_observability_000006 | agent_trace_observability | risk | score | {
"0": 2.579e-7,
"1": 0.0001659467,
"2": 0.0160301287,
"3": 0.9838036667000001
} |
agent_trace_observability_000006 | agent_trace_observability | urgency | score | {
"0": 6.762e-7,
"1": 0.0003657704,
"2": 0.0361183067,
"3": 0.9635152466
} |
agent_trace_observability_000007 | agent_trace_observability | action | choice | {
"continue": 0.2451134988,
"human_review": 0.2058545683,
"observe": 0.3827306062,
"stop": 0.1663013267
} |
agent_trace_observability_000007 | agent_trace_observability | needs_review | noul | {
"false": 0.6640984961,
"true": 0.3359015039
} |
agent_trace_observability_000007 | agent_trace_observability | outcome | choice | {
"failure": 0.0512251274,
"harmful": 3.059e-7,
"partial": 0.9247210923,
"success": 0.0240534745
} |
agent_trace_observability_000007 | agent_trace_observability | risk | score | {
"0": 0.1214934158,
"1": 0.8756364429,
"2": 0.0028623628,
"3": 0.0000077785
} |
agent_trace_observability_000007 | agent_trace_observability | urgency | score | {
"0": 0.0682031057,
"1": 0.8879642719,
"2": 0.0438004192,
"3": 0.000032203200000000004
} |
agent_trace_observability_000008 | agent_trace_observability | action | choice | {
"continue": 0.0000765532,
"human_review": 0.9997519746,
"observe": 0.00011953340000000001,
"stop": 0.000051938800000000005
} |
agent_trace_observability_000008 | agent_trace_observability | needs_review | noul | {
"false": 0.0002614621,
"true": 0.9997385379
} |
agent_trace_observability_000008 | agent_trace_observability | outcome | choice | {
"failure": 0.00028906090000000004,
"harmful": 0.9995506216000001,
"partial": 0.0001603125,
"success": 5e-9
} |
agent_trace_observability_000008 | agent_trace_observability | risk | score | {
"0": 2.4980000000000003e-7,
"1": 0.00017125900000000002,
"2": 0.0188270744,
"3": 0.9810014168000001
} |
agent_trace_observability_000008 | agent_trace_observability | urgency | score | {
"0": 0.0000012007,
"1": 0.0006347727,
"2": 0.0827296591,
"3": 0.9166343675
} |
agent_trace_observability_000009 | agent_trace_observability | action | choice | {
"continue": 0.0000896896,
"human_review": 0.999709414,
"observe": 0.0001400451,
"stop": 0.000060851400000000005
} |
agent_trace_observability_000009 | agent_trace_observability | needs_review | noul | {
"false": 0.0004667351,
"true": 0.9995332649
} |
agent_trace_observability_000009 | agent_trace_observability | outcome | choice | {
"failure": 0.00009145,
"harmful": 0.9996220418,
"partial": 0.0002864923,
"success": 1.59e-8
} |
agent_trace_observability_000009 | agent_trace_observability | risk | score | {
"0": 3.235e-7,
"1": 0.0001604068,
"2": 0.0241117905,
"3": 0.9757274791
} |
agent_trace_observability_000009 | agent_trace_observability | urgency | score | {
"0": 0.0000014147,
"1": 0.0009335279000000001,
"2": 0.08598830910000001,
"3": 0.9130767483000001
} |
agent_trace_observability_000010 | agent_trace_observability | action | choice | {
"continue": 0.3002713508,
"human_review": 0.027148147,
"observe": 0.4688564142,
"stop": 0.20372408800000003
} |
agent_trace_observability_000010 | agent_trace_observability | needs_review | noul | {
"false": 0.9408894768,
"true": 0.0591105232
} |
agent_trace_observability_000010 | agent_trace_observability | outcome | choice | {
"failure": 0.0022577206,
"harmful": 5.46e-8,
"partial": 0.448289686,
"success": 0.5494525387
} |
agent_trace_observability_000010 | agent_trace_observability | risk | score | {
"0": 0.16409596910000002,
"1": 0.8346000171,
"2": 0.0013023732,
"3": 0.0000016406
} |
agent_trace_observability_000010 | agent_trace_observability | urgency | score | {
"0": 0.4785595849,
"1": 0.5165030105,
"2": 0.0049306727,
"3": 0.0000067318000000000005
} |
agent_trace_observability_000011 | agent_trace_observability | action | choice | {
"continue": 0.308405139,
"human_review": 0.0007954134,
"observe": 0.4815568559,
"stop": 0.20924259180000002
} |
agent_trace_observability_000011 | agent_trace_observability | needs_review | noul | {
"false": 0.9965622544,
"true": 0.0034377456
} |
agent_trace_observability_000011 | agent_trace_observability | outcome | choice | {
"failure": 0.0000040812,
"harmful": 4.2e-9,
"partial": 0.0013209685,
"success": 0.9986749462000001
} |
agent_trace_observability_000011 | agent_trace_observability | risk | score | {
"0": 0.9045215160000001,
"1": 0.0953912481,
"2": 0.00008711420000000001,
"3": 1.217e-7
} |
agent_trace_observability_000011 | agent_trace_observability | urgency | score | {
"0": 0.9499292577,
"1": 0.049985222700000005,
"2": 0.0000853494,
"3": 1.702e-7
} |
agent_trace_observability_000012 | agent_trace_observability | action | choice | {
"continue": 0.2507470422,
"human_review": 0.1876024003,
"observe": 0.3915270597,
"stop": 0.1701234978
} |
agent_trace_observability_000012 | agent_trace_observability | needs_review | noul | {
"false": 0.7403615777,
"true": 0.2596384223
} |
agent_trace_observability_000012 | agent_trace_observability | outcome | choice | {
"failure": 0.0001101586,
"harmful": 4.924e-7,
"partial": 0.1052791777,
"success": 0.8946101714
} |
agent_trace_observability_000012 | agent_trace_observability | risk | score | {
"0": 0.0104503241,
"1": 0.5170555537,
"2": 0.47203681140000003,
"3": 0.0004573108
} |
agent_trace_observability_000012 | agent_trace_observability | urgency | score | {
"0": 0.2960112076,
"1": 0.6805915141000001,
"2": 0.0233382351,
"3": 0.000059043300000000004
} |
agent_trace_observability_000013 | agent_trace_observability | action | choice | {
"continue": 0.3001186804,
"human_review": 0.0276427853,
"observe": 0.46861802810000003,
"stop": 0.2036205062
} |
agent_trace_observability_000013 | agent_trace_observability | needs_review | noul | {
"false": 0.8947110437,
"true": 0.1052889563
} |
agent_trace_observability_000013 | agent_trace_observability | outcome | choice | {
"failure": 0.008160786100000001,
"harmful": 1.293e-7,
"partial": 0.798026509,
"success": 0.1938125755
} |
agent_trace_observability_000013 | agent_trace_observability | risk | score | {
"0": 0.14792436180000001,
"1": 0.8498939733,
"2": 0.0021797773,
"3": 0.0000018877000000000002
} |
agent_trace_observability_000013 | agent_trace_observability | urgency | score | {
"0": 0.21553070570000002,
"1": 0.7774152355,
"2": 0.0070459795000000006,
"3": 0.0000080793
} |
agent_trace_observability_000014 | agent_trace_observability | action | choice | {
"continue": 0.2532893576,
"human_review": 0.17936552960000002,
"observe": 0.3954967388,
"stop": 0.171848374
} |
agent_trace_observability_000014 | agent_trace_observability | needs_review | noul | {
"false": 0.7301423739,
"true": 0.2698576261
} |
agent_trace_observability_000014 | agent_trace_observability | outcome | choice | {
"failure": 0.040485592800000005,
"harmful": 6.717000000000001e-7,
"partial": 0.6711202604000001,
"success": 0.288393475
} |
agent_trace_observability_000014 | agent_trace_observability | risk | score | {
"0": 0.1273383811,
"1": 0.8635943248,
"2": 0.009053636,
"3": 0.000013658
} |
agent_trace_observability_000014 | agent_trace_observability | urgency | score | {
"0": 0.1849148801,
"1": 0.7979513769000001,
"2": 0.0170856697,
"3": 0.0000480733
} |
agent_trace_observability_000015 | agent_trace_observability | action | choice | {
"continue": 0.00010448670000000001,
"human_review": 0.9996614726,
"observe": 0.00016315,
"stop": 0.0000708907
} |
agent_trace_observability_000015 | agent_trace_observability | needs_review | noul | {
"false": 0.0004254315,
"true": 0.9995745685
} |
agent_trace_observability_000015 | agent_trace_observability | outcome | choice | {
"failure": 0.00008164460000000001,
"harmful": 0.9999070721000001,
"partial": 0.0000112827,
"success": 6e-10
} |
agent_trace_observability_000015 | agent_trace_observability | risk | score | {
"0": 1.661e-7,
"1": 0.0001100234,
"2": 0.0075404599,
"3": 0.9923493505000001
} |
agent_trace_observability_000015 | agent_trace_observability | urgency | score | {
"0": 9.723e-7,
"1": 0.0005322552,
"2": 0.0570661435,
"3": 0.9424006290000001
} |
agent_trace_observability_000016 | agent_trace_observability | action | choice | {
"continue": 0.0000564523,
"human_review": 0.9998170998,
"observe": 0.0000881469,
"stop": 0.000038301000000000005
} |
agent_trace_observability_000016 | agent_trace_observability | needs_review | noul | {
"false": 0.001010539,
"true": 0.9989894610000001
} |
agent_trace_observability_000016 | agent_trace_observability | outcome | choice | {
"failure": 0.0002837456,
"harmful": 0.9996693273,
"partial": 0.0000469209,
"success": 6.2e-9
} |
agent_trace_observability_000016 | agent_trace_observability | risk | score | {
"0": 4.854e-7,
"1": 0.0002756014,
"2": 0.0523684342,
"3": 0.9473554790000001
} |
agent_trace_observability_000016 | agent_trace_observability | urgency | score | {
"0": 0.0000051084,
"1": 0.0017553834,
"2": 0.2026387906,
"3": 0.7956007176000001
} |
agent_trace_observability_000017 | agent_trace_observability | action | choice | {
"continue": 0.0208449764,
"human_review": 0.9324641733000001,
"observe": 0.0325482297,
"stop": 0.0141426207
} |
agent_trace_observability_000017 | agent_trace_observability | needs_review | noul | {
"false": 0.3759202142,
"true": 0.6240797858
} |
agent_trace_observability_000017 | agent_trace_observability | outcome | choice | {
"failure": 0.0002548368,
"harmful": 1.307e-7,
"partial": 0.0288656187,
"success": 0.9708794138000001
} |
agent_trace_observability_000017 | agent_trace_observability | risk | score | {
"0": 0.0002452258,
"1": 0.1040491136,
"2": 0.894263616,
"3": 0.0014420446
} |
agent_trace_observability_000017 | agent_trace_observability | urgency | score | {
"0": 0.0017185242,
"1": 0.6830973319,
"2": 0.31266629830000003,
"3": 0.0025178456
} |
agent_trace_observability_000018 | agent_trace_observability | action | choice | {
"continue": 0.3080591922,
"human_review": 0.0019162495,
"observe": 0.4810166799,
"stop": 0.2090078784
} |
agent_trace_observability_000018 | agent_trace_observability | needs_review | noul | {
"false": 0.9911326534,
"true": 0.008867346600000001
} |
agent_trace_observability_000018 | agent_trace_observability | outcome | choice | {
"failure": 0.0002855528,
"harmful": 3.14e-8,
"partial": 0.35606383410000003,
"success": 0.6436505816
} |
agent_trace_observability_000018 | agent_trace_observability | risk | score | {
"0": 0.9870498355,
"1": 0.0129245634,
"2": 0.0000255728,
"3": 2.8300000000000002e-8
} |
agent_trace_observability_000018 | agent_trace_observability | urgency | score | {
"0": 0.983102574,
"1": 0.0168366358,
"2": 0.000060703300000000005,
"3": 8.69e-8
} |
agent_trace_observability_000019 | agent_trace_observability | action | choice | {
"continue": 0.1244913817,
"human_review": 0.5966592515,
"observe": 0.19438612,
"stop": 0.0844632468
} |
agent_trace_observability_000019 | agent_trace_observability | needs_review | noul | {
"false": 0.7148902960000001,
"true": 0.285109704
} |
agent_trace_observability_000019 | agent_trace_observability | outcome | choice | {
"failure": 0.0002128555,
"harmful": 1.3270000000000002e-7,
"partial": 0.2031426439,
"success": 0.7966443679
} |
agent_trace_observability_000019 | agent_trace_observability | risk | score | {
"0": 0.0053664623,
"1": 0.3427557737,
"2": 0.6516945647,
"3": 0.00018319940000000002
} |
agent_trace_observability_000019 | agent_trace_observability | urgency | score | {
"0": 0.2737975488,
"1": 0.6957998658,
"2": 0.0303690491,
"3": 0.0000335363
} |
End of preview. Expand in Data Studio
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
ZhiPan-Z3-Pro — Typed Decisions test-split predictions
Closed-weights specialist ZhiPan-Z3-Pro — a per-primitive ensemble (prediction
fusion) of 14 fine-tuned runs on a Qwen3.5-4B backbone with typed decision heads,
fitted on the Typed Decisions train split only.
Self-reported results (official scorer formulas, test split, 400 cases / 2000 decisions)
| accuracy | KL from gold | Brier | ECE (10 bins) |
|---|---|---|---|
| 0.8265 | 0.8033 | 0.1515 | 0.0741 |
Per-primitive top-1: choice 77.17 (n=600) · noul 88.83 (n=600) · score 82.12 (n=800)
Files
predictions.jsonl— one line per decision:{id, workflow, question, type, probabilities}(probabilities keyed by official gold label order; noul usestrue/falselabels). Re-scorable withbench/score_with_jobs.py::score_onefrom the benchmark repo.
Notes
- Kind: specialist (fitted on train) — not comparable with zero-shot rows.
- Weights stay closed; predictions are public so the scores can be re-derived.
- ZhiPan-Z3-Pro = per-type independent fusion: choice 6 tracks, noul 3 tracks,
score 5 tracks (log-space weight grid search + add/remove member search, all
fitted strictly on the
trainsplit; official scorer re-checked, n=2000, zero missing). - Gold argmax ties break to the first max in the official
gold.probabilitieskey order, matching the bench scorer.
- Downloads last month
- 64