id stringclasses 400
values | workflow stringclasses 4
values | question stringclasses 15
values | type stringclasses 3
values | probabilities unknown |
|---|---|---|---|---|
agent_trace_observability_000000 | agent_trace_observability | action | choice | {
"continue": 0.2935055073,
"human_review": 0.119483478,
"observe": 0.2935055073,
"stop": 0.2935055073
} |
agent_trace_observability_000000 | agent_trace_observability | needs_review | noul | {
"false": 0.7971704244000001,
"true": 0.2028295756
} |
agent_trace_observability_000000 | agent_trace_observability | outcome | choice | {
"failure": 0.0592701078,
"harmful": 0.0162916161,
"partial": 0.1617427166,
"success": 0.7626955595
} |
agent_trace_observability_000000 | agent_trace_observability | risk | score | {
"0": 0.35166972090000004,
"1": 0.5368674874,
"2": 0.0920927727,
"3": 0.019370019000000002
} |
agent_trace_observability_000000 | agent_trace_observability | urgency | score | {
"0": 0.4978980264,
"1": 0.4232839212,
"2": 0.0640155876,
"3": 0.0148024647
} |
agent_trace_observability_000001 | agent_trace_observability | action | choice | {
"continue": 0.2755594079,
"human_review": 0.17332177640000002,
"observe": 0.2755594079,
"stop": 0.2755594079
} |
agent_trace_observability_000001 | agent_trace_observability | needs_review | noul | {
"false": 0.6637644768000001,
"true": 0.3362355232
} |
agent_trace_observability_000001 | agent_trace_observability | outcome | choice | {
"failure": 0.1865340284,
"harmful": 0.028934752,
"partial": 0.3027737377,
"success": 0.481757482
} |
agent_trace_observability_000001 | agent_trace_observability | risk | score | {
"0": 0.5028549734000001,
"1": 0.39782503890000004,
"2": 0.083997498,
"3": 0.0153224897
} |
agent_trace_observability_000001 | agent_trace_observability | urgency | score | {
"0": 0.3920055685,
"1": 0.4508658072,
"2": 0.1322952279,
"3": 0.024833396400000003
} |
agent_trace_observability_000002 | agent_trace_observability | action | choice | {
"continue": 0.2024032711,
"human_review": 0.39279018660000004,
"observe": 0.2024032711,
"stop": 0.2024032711
} |
agent_trace_observability_000002 | agent_trace_observability | needs_review | noul | {
"false": 0.3876579404,
"true": 0.6123420596
} |
agent_trace_observability_000002 | agent_trace_observability | outcome | choice | {
"failure": 0.5008390098000001,
"harmful": 0.028861237,
"partial": 0.361061809,
"success": 0.1092379443
} |
agent_trace_observability_000002 | agent_trace_observability | risk | score | {
"0": 0.4689238115,
"1": 0.4094778934,
"2": 0.1033006097,
"3": 0.0182976855
} |
agent_trace_observability_000002 | agent_trace_observability | urgency | score | {
"0": 0.17576877770000002,
"1": 0.45981150870000004,
"2": 0.3067057729,
"3": 0.0577139407
} |
agent_trace_observability_000003 | agent_trace_observability | action | choice | {
"continue": 0.2873618092,
"human_review": 0.1379145723,
"observe": 0.2873618092,
"stop": 0.2873618092
} |
agent_trace_observability_000003 | agent_trace_observability | needs_review | noul | {
"false": 0.7403759956,
"true": 0.2596240044
} |
agent_trace_observability_000003 | agent_trace_observability | outcome | choice | {
"failure": 0.1410614644,
"harmful": 0.0235688621,
"partial": 0.2891825324,
"success": 0.546187141
} |
agent_trace_observability_000003 | agent_trace_observability | risk | score | {
"0": 0.5317655732000001,
"1": 0.3873370239,
"2": 0.067813336,
"3": 0.0130840669
} |
agent_trace_observability_000003 | agent_trace_observability | urgency | score | {
"0": 0.4225323318,
"1": 0.4493622607,
"2": 0.1091689349,
"3": 0.0189364725
} |
agent_trace_observability_000004 | agent_trace_observability | action | choice | {
"continue": 0.2820061571,
"human_review": 0.1539815286,
"observe": 0.2820061571,
"stop": 0.2820061571
} |
agent_trace_observability_000004 | agent_trace_observability | needs_review | noul | {
"false": 0.7130570241,
"true": 0.2869429759
} |
agent_trace_observability_000004 | agent_trace_observability | outcome | choice | {
"failure": 0.1574816114,
"harmful": 0.027854528400000002,
"partial": 0.3053328704,
"success": 0.5093309898
} |
agent_trace_observability_000004 | agent_trace_observability | risk | score | {
"0": 0.4838937132,
"1": 0.42842285680000003,
"2": 0.0741332876,
"3": 0.013550142400000001
} |
agent_trace_observability_000004 | agent_trace_observability | urgency | score | {
"0": 0.426916821,
"1": 0.4614111947,
"2": 0.0926845986,
"3": 0.0189873856
} |
agent_trace_observability_000005 | agent_trace_observability | action | choice | {
"continue": 0.26573345460000003,
"human_review": 0.2027996362,
"observe": 0.26573345460000003,
"stop": 0.26573345460000003
} |
agent_trace_observability_000005 | agent_trace_observability | needs_review | noul | {
"false": 0.6747212410000001,
"true": 0.32527875900000003
} |
agent_trace_observability_000005 | agent_trace_observability | outcome | choice | {
"failure": 0.1602390558,
"harmful": 0.025259211700000003,
"partial": 0.26961633560000003,
"success": 0.544885397
} |
agent_trace_observability_000005 | agent_trace_observability | risk | score | {
"0": 0.2626767769,
"1": 0.5751505526,
"2": 0.1353271909,
"3": 0.0268454795
} |
agent_trace_observability_000005 | agent_trace_observability | urgency | score | {
"0": 0.2959171042,
"1": 0.5243865437,
"2": 0.1536632831,
"3": 0.026033069000000002
} |
agent_trace_observability_000006 | agent_trace_observability | action | choice | {
"continue": 0.0992746189,
"human_review": 0.7021761432,
"observe": 0.0992746189,
"stop": 0.0992746189
} |
agent_trace_observability_000006 | agent_trace_observability | needs_review | noul | {
"false": 0.12173358620000001,
"true": 0.8782664138
} |
agent_trace_observability_000006 | agent_trace_observability | outcome | choice | {
"failure": 0.1283227244,
"harmful": 0.7762108972,
"partial": 0.0769494035,
"success": 0.018516974800000002
} |
agent_trace_observability_000006 | agent_trace_observability | risk | score | {
"0": 0.0156615521,
"1": 0.0586028231,
"2": 0.24376779070000001,
"3": 0.6819678342000001
} |
agent_trace_observability_000006 | agent_trace_observability | urgency | score | {
"0": 0.01667197,
"1": 0.0882521569,
"2": 0.3105802796,
"3": 0.5844955934
} |
agent_trace_observability_000007 | agent_trace_observability | action | choice | {
"continue": 0.2535702288,
"human_review": 0.2392893136,
"observe": 0.2535702288,
"stop": 0.2535702288
} |
agent_trace_observability_000007 | agent_trace_observability | needs_review | noul | {
"false": 0.5534948276,
"true": 0.44650517240000004
} |
agent_trace_observability_000007 | agent_trace_observability | outcome | choice | {
"failure": 0.29060400610000003,
"harmful": 0.0272715325,
"partial": 0.4035606833,
"success": 0.27856377800000004
} |
agent_trace_observability_000007 | agent_trace_observability | risk | score | {
"0": 0.37083374,
"1": 0.49280182580000004,
"2": 0.1174723386,
"3": 0.018892095600000002
} |
agent_trace_observability_000007 | agent_trace_observability | urgency | score | {
"0": 0.2899058564,
"1": 0.4884939963,
"2": 0.1916671314,
"3": 0.029933015900000002
} |
agent_trace_observability_000008 | agent_trace_observability | action | choice | {
"continue": 0.10341876750000001,
"human_review": 0.6897436976,
"observe": 0.10341876750000001,
"stop": 0.10341876750000001
} |
agent_trace_observability_000008 | agent_trace_observability | needs_review | noul | {
"false": 0.11969416620000001,
"true": 0.8803058338
} |
agent_trace_observability_000008 | agent_trace_observability | outcome | choice | {
"failure": 0.1591539203,
"harmful": 0.710757845,
"partial": 0.10967147420000001,
"success": 0.020416760500000002
} |
agent_trace_observability_000008 | agent_trace_observability | risk | score | {
"0": 0.014285809600000001,
"1": 0.057322972400000005,
"2": 0.2323670871,
"3": 0.6960241309
} |
agent_trace_observability_000008 | agent_trace_observability | urgency | score | {
"0": 0.0163575465,
"1": 0.09239137800000001,
"2": 0.3481772497,
"3": 0.5430738259
} |
agent_trace_observability_000009 | agent_trace_observability | action | choice | {
"continue": 0.1069961621,
"human_review": 0.6790115136,
"observe": 0.1069961621,
"stop": 0.1069961621
} |
agent_trace_observability_000009 | agent_trace_observability | needs_review | noul | {
"false": 0.1262004581,
"true": 0.8737995419
} |
agent_trace_observability_000009 | agent_trace_observability | outcome | choice | {
"failure": 0.1273381341,
"harmful": 0.7189003671,
"partial": 0.12869273920000002,
"success": 0.0250687596
} |
agent_trace_observability_000009 | agent_trace_observability | risk | score | {
"0": 0.0156841318,
"1": 0.057114613200000004,
"2": 0.2545636036,
"3": 0.6726376515
} |
agent_trace_observability_000009 | agent_trace_observability | urgency | score | {
"0": 0.017396038000000003,
"1": 0.12718318,
"2": 0.35848555060000004,
"3": 0.4969352313
} |
agent_trace_observability_000010 | agent_trace_observability | action | choice | {
"continue": 0.2658433656,
"human_review": 0.2024699033,
"observe": 0.2658433656,
"stop": 0.2658433656
} |
agent_trace_observability_000010 | agent_trace_observability | needs_review | noul | {
"false": 0.6581921178,
"true": 0.3418078822
} |
agent_trace_observability_000010 | agent_trace_observability | outcome | choice | {
"failure": 0.1856827764,
"harmful": 0.021265615300000002,
"partial": 0.3659639716,
"success": 0.4270876368
} |
agent_trace_observability_000010 | agent_trace_observability | risk | score | {
"0": 0.3964550458,
"1": 0.4947564854,
"2": 0.0921314574,
"3": 0.0166570114
} |
agent_trace_observability_000010 | agent_trace_observability | urgency | score | {
"0": 0.3285169358,
"1": 0.5124473062,
"2": 0.1360060893,
"3": 0.0230296686
} |
agent_trace_observability_000011 | agent_trace_observability | action | choice | {
"continue": 0.293959526,
"human_review": 0.11812142190000001,
"observe": 0.293959526,
"stop": 0.293959526
} |
agent_trace_observability_000011 | agent_trace_observability | needs_review | noul | {
"false": 0.8059000965,
"true": 0.1940999035
} |
agent_trace_observability_000011 | agent_trace_observability | outcome | choice | {
"failure": 0.0654139027,
"harmful": 0.0184068909,
"partial": 0.1650680488,
"success": 0.7511111576
} |
agent_trace_observability_000011 | agent_trace_observability | risk | score | {
"0": 0.6617674643,
"1": 0.2932964481,
"2": 0.036536275300000004,
"3": 0.0083998123
} |
agent_trace_observability_000011 | agent_trace_observability | urgency | score | {
"0": 0.5318442136,
"1": 0.3980233184,
"2": 0.0584596971,
"3": 0.0116727709
} |
agent_trace_observability_000012 | agent_trace_observability | action | choice | {
"continue": 0.2518061287,
"human_review": 0.24458161390000002,
"observe": 0.2518061287,
"stop": 0.2518061287
} |
agent_trace_observability_000012 | agent_trace_observability | needs_review | noul | {
"false": 0.5531580182,
"true": 0.4468419818
} |
agent_trace_observability_000012 | agent_trace_observability | outcome | choice | {
"failure": 0.0991882991,
"harmful": 0.0340688717,
"partial": 0.3011755679,
"success": 0.5655672613
} |
agent_trace_observability_000012 | agent_trace_observability | risk | score | {
"0": 0.1739726559,
"1": 0.3573515806,
"2": 0.40940342420000003,
"3": 0.059272339300000004
} |
agent_trace_observability_000012 | agent_trace_observability | urgency | score | {
"0": 0.3276461916,
"1": 0.4585920811,
"2": 0.18378594280000002,
"3": 0.0299757846
} |
agent_trace_observability_000013 | agent_trace_observability | action | choice | {
"continue": 0.2707636695,
"human_review": 0.1877089916,
"observe": 0.2707636695,
"stop": 0.2707636695
} |
agent_trace_observability_000013 | agent_trace_observability | needs_review | noul | {
"false": 0.656753162,
"true": 0.343246838
} |
agent_trace_observability_000013 | agent_trace_observability | outcome | choice | {
"failure": 0.2032393274,
"harmful": 0.0223122562,
"partial": 0.3716946646,
"success": 0.4027537518
} |
agent_trace_observability_000013 | agent_trace_observability | risk | score | {
"0": 0.4355089742,
"1": 0.4598223052,
"2": 0.0902629436,
"3": 0.014405777000000002
} |
agent_trace_observability_000013 | agent_trace_observability | urgency | score | {
"0": 0.32844238740000004,
"1": 0.5148839835,
"2": 0.1347465634,
"3": 0.0219270657
} |
agent_trace_observability_000014 | agent_trace_observability | action | choice | {
"continue": 0.2501550242,
"human_review": 0.24953492730000001,
"observe": 0.2501550242,
"stop": 0.2501550242
} |
agent_trace_observability_000014 | agent_trace_observability | needs_review | noul | {
"false": 0.5513896778,
"true": 0.4486103222
} |
agent_trace_observability_000014 | agent_trace_observability | outcome | choice | {
"failure": 0.2735141982,
"harmful": 0.0311416984,
"partial": 0.3350258974,
"success": 0.36031820600000003
} |
agent_trace_observability_000014 | agent_trace_observability | risk | score | {
"0": 0.3501157152,
"1": 0.4621585888,
"2": 0.1528624141,
"3": 0.0348632819
} |
agent_trace_observability_000014 | agent_trace_observability | urgency | score | {
"0": 0.2940623462,
"1": 0.5185208321,
"2": 0.1560908705,
"3": 0.0313259512
} |
agent_trace_observability_000015 | agent_trace_observability | action | choice | {
"continue": 0.10739539910000001,
"human_review": 0.6778138027,
"observe": 0.10739539910000001,
"stop": 0.10739539910000001
} |
agent_trace_observability_000015 | agent_trace_observability | needs_review | noul | {
"false": 0.121937791,
"true": 0.8780622090000001
} |
agent_trace_observability_000015 | agent_trace_observability | outcome | choice | {
"failure": 0.1282610969,
"harmful": 0.7789048110000001,
"partial": 0.0793838828,
"success": 0.0134502092
} |
agent_trace_observability_000015 | agent_trace_observability | risk | score | {
"0": 0.0142831012,
"1": 0.0553099035,
"2": 0.1947415362,
"3": 0.7356654591
} |
agent_trace_observability_000015 | agent_trace_observability | urgency | score | {
"0": 0.0190040227,
"1": 0.1076454746,
"2": 0.3517428047,
"3": 0.5216076980000001
} |
agent_trace_observability_000016 | agent_trace_observability | action | choice | {
"continue": 0.10269327,
"human_review": 0.6919201899,
"observe": 0.10269327,
"stop": 0.10269327
} |
agent_trace_observability_000016 | agent_trace_observability | needs_review | noul | {
"false": 0.1481488943,
"true": 0.8518511057
} |
agent_trace_observability_000016 | agent_trace_observability | outcome | choice | {
"failure": 0.1554642933,
"harmful": 0.7275219045,
"partial": 0.0960608061,
"success": 0.0209529961
} |
agent_trace_observability_000016 | agent_trace_observability | risk | score | {
"0": 0.018494102800000002,
"1": 0.0658450859,
"2": 0.257335509,
"3": 0.6583253023
} |
agent_trace_observability_000016 | agent_trace_observability | urgency | score | {
"0": 0.019803944900000002,
"1": 0.1073233183,
"2": 0.3636103557,
"3": 0.5092623811
} |
agent_trace_observability_000017 | agent_trace_observability | action | choice | {
"continue": 0.1720050254,
"human_review": 0.4839849237,
"observe": 0.1720050254,
"stop": 0.1720050254
} |
agent_trace_observability_000017 | agent_trace_observability | needs_review | noul | {
"false": 0.4415189028,
"true": 0.5584810972
} |
agent_trace_observability_000017 | agent_trace_observability | outcome | choice | {
"failure": 0.14220026530000002,
"harmful": 0.0298732546,
"partial": 0.2828442533,
"success": 0.5450822267000001
} |
agent_trace_observability_000017 | agent_trace_observability | risk | score | {
"0": 0.0640132442,
"1": 0.2808179024,
"2": 0.5355399967,
"3": 0.1196288568
} |
agent_trace_observability_000017 | agent_trace_observability | urgency | score | {
"0": 0.0859765851,
"1": 0.4568510618,
"2": 0.36757677520000004,
"3": 0.0895955779
} |
agent_trace_observability_000018 | agent_trace_observability | action | choice | {
"continue": 0.29489897800000003,
"human_review": 0.1153030659,
"observe": 0.29489897800000003,
"stop": 0.29489897800000003
} |
agent_trace_observability_000018 | agent_trace_observability | needs_review | noul | {
"false": 0.7920288444,
"true": 0.2079711556
} |
agent_trace_observability_000018 | agent_trace_observability | outcome | choice | {
"failure": 0.11601712510000001,
"harmful": 0.019685130500000002,
"partial": 0.332168101,
"success": 0.5321296434
} |
agent_trace_observability_000018 | agent_trace_observability | risk | score | {
"0": 0.7537121057,
"1": 0.21291592380000002,
"2": 0.0272990759,
"3": 0.0060728946
} |
agent_trace_observability_000018 | agent_trace_observability | urgency | score | {
"0": 0.5854829653,
"1": 0.3449223997,
"2": 0.057749834900000005,
"3": 0.0118448001
} |
agent_trace_observability_000019 | agent_trace_observability | action | choice | {
"continue": 0.2344991451,
"human_review": 0.29650256480000003,
"observe": 0.2344991451,
"stop": 0.2344991451
} |
agent_trace_observability_000019 | agent_trace_observability | needs_review | noul | {
"false": 0.5534077883,
"true": 0.4465922117
} |
agent_trace_observability_000019 | agent_trace_observability | outcome | choice | {
"failure": 0.10251486230000001,
"harmful": 0.0260050071,
"partial": 0.3033853449,
"success": 0.5680947857
} |
agent_trace_observability_000019 | agent_trace_observability | risk | score | {
"0": 0.2091923968,
"1": 0.316491189,
"2": 0.4372006091,
"3": 0.0371158051
} |
agent_trace_observability_000019 | agent_trace_observability | urgency | score | {
"0": 0.34276940840000003,
"1": 0.4525139554,
"2": 0.1829942136,
"3": 0.0217224225
} |
End of preview. Expand in Data Studio
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
ZhiPan2-4B — Typed Decisions test-split predictions
Closed-weights specialist model ZhiPan2-4B (Qwen3.5-4B backbone + typed decision head,
per-primitive temperature calibration), fitted on the Typed Decisions train split only
(per-type mixed distillation: score pure-gold + choice/noul soft targets).
Self-reported results (official scorer formulas, test split, 400 cases / 2000 decisions)
| accuracy | KL from gold | Brier | ECE (10 bins) |
|---|---|---|---|
| 0.7710 | 0.1105 | 0.0587 | 0.1625 |
Per-primitive top-1: choice 67.33 (n=600) · noul 86.33 (n=600) · score 77.50 (n=800)
Files
predictions.jsonl— one line per decision:{id, workflow, question, type, probabilities}(probabilities keyed by official gold label order; noul usestrue/falselabels). Re-scorable withbench/score_with_jobs.py::score_onefrom the benchmark repo.
Notes
- Kind: specialist (fitted on train) — not comparable with zero-shot rows.
- Weights stay closed; predictions are public so the scores can be re-derived.
- Gold argmax ties (6 noul questions with 50/50 gold) break to the first max in the
official
gold.probabilitieskey order, matching the bench scorer.
- Downloads last month
- 59