jev-benchmarks
AbdelStark/jev-benchmarks
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
Python
- 星标
- 28
- 派生
- 5
收录依据
基准测试 README 说明它将 TypeSafe Jev 与另一分类器在零样本单标签判断和逐标签概率上进行比较。
AbdelStark/jev-benchmarks
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
基准测试 README 说明它将 TypeSafe Jev 与另一分类器在零样本单标签判断和逐标签概率上进行比较。
Local playground and test bench for TypeSafe's Jev System One model: typed questions, calibrated-probability answers, and side-by-side comparison with an LLM baseline.
共同主题: calibration, evaluation
Preregistered, bias-corrected test of TypeSafe Jev's calibration under human disagreement (ChaosNLI, 100 labels per item)
共同主题: calibration, evaluation
First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs
共同主题: calibration, evaluation