All projects

jev-benchmarks

AbdelStark/jev-benchmarks

Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

Python
Stars
28
Forks
5

Why it is listed

The benchmark README describes a comparison of TypeSafe Jev with another classifier on zero-shot single-label decisions and per-label probabilities.

Related projects

  • here-we-go-jev

    Local playground and test bench for TypeSafe's Jev System One model: typed questions, calibrated-probability answers, and side-by-side comparison with an LLM baseline.

    Shared topics: calibration, evaluation

  • jevbench

    Preregistered, bias-corrected test of TypeSafe Jev's calibration under human disagreement (ChaosNLI, 100 labels per item)

    Shared topics: calibration, evaluation

  • sysone-bench

    First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs

    Shared topics: calibration, evaluation