All projects

decision-model-benchmark

nibzard/decision-model-benchmark

Independent, reproducible benchmark: a decision model (jev), eight constrained LLMs, and deterministic baselines on typed decisions - accuracy, calibration, latency, cost, failure modes

HTML
Stars
12
Forks
1

Why it is listed

An independent benchmark comparing TypeSafe Jev typed decisions with structured-output LLMs and deterministic baselines, including calibration, failures, latency and cost. Listed for its evaluation methodology and artifacts, not endorsement of a model or certification of reported rankings.

Related projects

  • typed-decision-bench

    Bench of typed decision models: Jev vs OpenJev vs Laya, small local LLMs and cheap hosted LLMs on the same zero-shot classification tasks, in English and French.

    Shared topics: benchmark, calibration, llm-evaluation

  • jev-rag-benchmark

    Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.

    Shared topics: benchmark, calibration, llm-evaluation

  • system-one-playground

    Run the new wave of decision models (Laya, Decider, Kev, Jev) side by side on your Mac. Structured input in, calibrated probabilities out, with honest accuracy, calibration, latency and memory numbers.

    Shared topics: benchmark, calibration, decision-models