jev-llm-judge
nadheesh/jev-llm-judge
Can a decision model replace an LLM as an eval judge? Same rubrics, scored by Jev and by an LLM, side by side.
- スター
- 0
- フォーク
- 0
掲載理由
エージェント軌跡の五つの評価尺度を Jev の Score に移し、共通 EvalResult 契約でチャット判定と比較する研究。Jev に説明文を生成させない。
nadheesh/jev-llm-judge
Can a decision model replace an LLM as an eval judge? Same rubrics, scored by Jev and by an LLM, side by side.
エージェント軌跡の五つの評価尺度を Jev の Score に移し、共通 EvalResult 契約でチャット判定と比較する研究。Jev に説明文を生成させない。
Evidence-gated registry of Jev question packs — curated questions, golden cases and measured evidence for Jev-compatible decision endpoints
共通トピック: evals, llm-evaluation
RAG evaluation with typed decisions: sentence-level hallucination verdicts with offsets and citations, passage and answer relevance, in one Jev call per answer. Benchmarked against Ragas, DeepEval, an LLM judge and HHEM.
共通トピック: llm-as-a-judge, llm-evaluation
What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.
共通トピック: evals, llm-evaluation