jev-llm-judge
nadheesh/jev-llm-judge
Can a decision model replace an LLM as an eval judge? Same rubrics, scored by Jev and by an LLM, side by side.
- 星标
- 0
- 派生
- 0
收录依据
评估研究把五项代理轨迹量表移植为 Jev 类型化 Score 问题,通过共同 EvalResult 契约与聊天模型评委对比,不生成 Jev 解释。
nadheesh/jev-llm-judge
Can a decision model replace an LLM as an eval judge? Same rubrics, scored by Jev and by an LLM, side by side.
评估研究把五项代理轨迹量表移植为 Jev 类型化 Score 问题,通过共同 EvalResult 契约与聊天模型评委对比,不生成 Jev 解释。
Evidence-gated registry of Jev question packs — curated questions, golden cases and measured evidence for Jev-compatible decision endpoints
共同主题: evals, llm-evaluation
RAG evaluation with typed decisions: sentence-level hallucination verdicts with offsets and citations, passage and answer relevance, in one Jev call per answer. Benchmarked against Ragas, DeepEval, an LLM judge and HHEM.
共同主题: llm-as-a-judge, llm-evaluation
What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.
共同主题: evals, llm-evaluation