全部项目

decision-model-benchmark

nibzard/decision-model-benchmark

Independent, reproducible benchmark: a decision model (jev), eight constrained LLMs, and deterministic baselines on typed decisions - accuracy, calibration, latency, cost, failure modes

HTML
星标
12
派生
1

收录依据

对比 TypeSafe Jev 类型化决策、结构化输出 LLM 与确定性基线,考察校准、失败、延迟和费用的独立基准。收录其评估方法与产物,不为模型或所报排名背书。

相关项目

  • typed-decision-bench

    Bench of typed decision models: Jev vs OpenJev vs Laya, small local LLMs and cheap hosted LLMs on the same zero-shot classification tasks, in English and French.

    共同主题: benchmark, calibration, llm-evaluation

  • jev-rag-benchmark

    Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.

    共同主题: benchmark, calibration, llm-evaluation

  • system-one-playground

    Run the new wave of decision models (Laya, Decider, Kev, Jev) side by side on your Mac. Structured input in, calibrated probabilities out, with honest accuracy, calibration, latency and memory numbers.

    共同主题: benchmark, calibration, decision-models