すべてのプロジェクト

decision-model-benchmark

nibzard/decision-model-benchmark

Independent, reproducible benchmark: a decision model (jev), eight constrained LLMs, and deterministic baselines on typed decisions - accuracy, calibration, latency, cost, failure modes

HTML
スター
12
フォーク
1

掲載理由

TypeSafe Jev の型付き判断を構造化出力 LLM と固定ベースラインに比較し、較正・失敗・遅延・費用を扱う独立評価。方法と成果物を資料として収録し、モデルや掲載順位を保証しない。

関連プロジェクト

  • typed-decision-bench

    Bench of typed decision models: Jev vs OpenJev vs Laya, small local LLMs and cheap hosted LLMs on the same zero-shot classification tasks, in English and French.

    共通トピック: benchmark, calibration, llm-evaluation

  • jev-rag-benchmark

    Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.

    共通トピック: benchmark, calibration, llm-evaluation

  • system-one-playground

    Run the new wave of decision models (Laya, Decider, Kev, Jev) side by side on your Mac. Structured input in, calibrated probabilities out, with honest accuracy, calibration, latency and memory numbers.

    共通トピック: benchmark, calibration, decision-models