全部项目

jev-benchmarks

AbdelStark/jev-benchmarks

Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

Python
星标
28
派生
5

收录依据

基准测试 README 说明它将 TypeSafe Jev 与另一分类器在零样本单标签判断和逐标签概率上进行比较。

相关项目

  • here-we-go-jev

    Local playground and test bench for TypeSafe's Jev System One model: typed questions, calibrated-probability answers, and side-by-side comparison with an LLM baseline.

    共同主题: calibration, evaluation

  • jevbench

    Preregistered, bias-corrected test of TypeSafe Jev's calibration under human disagreement (ChaosNLI, 100 labels per item)

    共同主题: calibration, evaluation

  • sysone-bench

    First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs

    共同主题: calibration, evaluation