judge-jev
carlaiau/judge-jev
Reproducible experiments evaluating Jev as an automated judge across benchmarks and tasks.
TypeScript
- スター
- 0
- フォーク
- 0
掲載理由
Jev の Choice・Score・Noul を文章関連性の判定に使い、人手ラベルと公開 LLMJudge 提出を比較する研究。公開ラベルを用いた事後比較で、盲検課題ではない。
carlaiau/judge-jev
Reproducible experiments evaluating Jev as an automated judge across benchmarks and tasks.
Jev の Choice・Score・Noul を文章関連性の判定に使い、人手ラベルと公開 LLMJudge 提出を比較する研究。公開ラベルを用いた事後比較で、盲検課題ではない。
Find where an API router confuses look-alike operations, rewrite the wording, and rerun. Runs offline in mock mode; live mode uses TypeSafe's Jev.
共通トピック: evals
Does a System One model actually beat keyword matching? A reproducible benchmark on catching disguised duplicate thesis titles. 24 cases, real production baseline, raw data and charts included.
共通トピック: information-retrieval
Evidence-gated registry of Jev question packs — curated questions, golden cases and measured evidence for Jev-compatible decision endpoints
共通トピック: evals