Projects tagged with “llm-evaluation”

Browse Jev GitHub projects tagged with llm-evaluation. Filter, sort and explore their source repositories.

44awesome-jev-gallery
🔥🔥 Papers, open reproductions and independent evaluations behind System One models and Jev.
OmniJev/awesome-jev-gallery · JavaScript
50232
awesomeawesome-listcalibrated-probabilitiescalibrationdecision-modelllmllm-evaluation
115jevals
Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.
openlayer-ai/jevals · Python
1029
ai-agentsevaluationguardrailsllmllm-evaluationragas
204jev-capability-atlas
Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
Zaious/jev-capability-atlas · Python
276
ai-agentsbenchmarkcalibrationllm-evaluationmachine-learningzh-tw
291snapjudge
Typed decisions (choice / score / yes-no) from local Qwen models on Apple Silicon. Probabilities come straight from the logits, no text generation. TypeSafe-compatible HTTP API, runs on MLX.
Micha0827/snapjudge · Python
140
apple-siliconclassificationdecision-makingfastapillmllm-evaluationlocal-llm
317decision-model-benchmark
Independent, reproducible benchmark: a decision model (jev), eight constrained LLMs, and deterministic baselines on typed decisions - accuracy, calibration, latency, cost, failure modes
nibzard/decision-model-benchmark · HTML
110
benchmarkcalibrationdecision-modelllm-evaluationreproducibilityhtml
377jev-state
Build and regression-test conversational state machines powered by Jev. Inspect decisions, capture failing conversations as tests, and export runnable TypeScript for your app.
priyankark/jev-state · TypeScript
80
ai-agentscode-generationconversational-aideveloper-toolsevaluationllm-evaluationlocal-first
391what-the-jev
What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.
keta1930/what-the-jev · Python
81
benchmarkdecision-modelevaluationllm-evaluation
412jevloop
A Jev-driven general-purpose agent harness for faster, lower-cost execution, with built-in side-by-side experiments against LLM-only agents.
parkavenue9639/jevloop · Python
70
agent-runtimeai-agentsdockerllm-evaluationobservabilitypythonsandbox
417pytest-jev
Semantic assertions for pytest: test what your LLM app's output means, judged by TypeSafe's Jev.
allebee/pytest-jev · Python
71
llmllm-evaluationpytestpytest-pluginpythontesting
435jev-baselines-eval
Pre-registered independent eval of TypeSafe Jev against a nano-class LLM, a frontier LLM, and a supervised encoder (Banking77 + CLINC150 zero-shot)
ickma2311/jev-baselines-eval · Python
60
intent-classificationllm-cascadellm-evaluationpreregistrationpython
472jev-2048
An instrumented 2048 web lab where every move is a Jev (TypeSafe AI System One) Choice, with no heuristic fallback | 用 Jev 决策模型驱动每一步的 2048 网页实验台，概率、置信度、延迟与成本全部摊开可见，且刻意不做启发式兜底
ARCJ137442/jev-2048 · TypeScript
51
2048llm-evaluationtyped-decisionstypescript
630jev-skills
Practical agent skills and examples for building with Jev. API setup, routing, ranking, and evidence checks.
laguagu/jev-skills
31
agent-pluginsagent-skillsclaude-codecodexevidence-verificationjavascriptllm-evaluation
640jevalyzer
Grade the agent sessions already on your disk. Claude Code, Codex, opencode, Gemini CLI and Antigravity, scored with Jev for cents.
killerz3/jevalyzer · TypeScript
30
ai-agentsbunclaude-codeclicodexgemini-clillm-evaluation
764jev-rag-benchmark
Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.
emretheus/jev-rag-benchmark · Python
20
benchmarkcalibrationinformation-retrievalllm-evaluationnvidiaopenjevrag
773jev-synthetic-survey
Jev vs GPT-4.1 as synthetic survey respondents on Twin-2K-500. How you ask mattered more than which model you used.
jjd-lab/jev-synthetic-survey · Python
20
behavioral-economicsbenchmarkcalibrationdigital-twindigital-twinsllmllm-evaluation
775jev-system-one-reference
Independent Jev / System One reference with API examples, implementation guidance and reusable prompts for engineers and coding assistants.
sebastianbennis/jev-system-one-reference
20
ai-agentsapi-referenceclassificationcoding-assistantllmllm-evaluationprompt-engineering
831tally
Turn a pile of comments into an auditable distribution: Jev judges each item, code counts the percentages. React workspace + Python engine with pluggable data sources.
adlternative/tally · Python
20
data-analysisi18nllmllm-evaluationpythonreact
883decide
Bulk decisions for AI agents. Jev classifies files and logs; your agent reviews exceptions. Reproducible cost and quality benchmarks.
alsoleg89/decide · Python
10
ai-agentsclassificationclaude-codecodexcontext-engineeringllm-evaluationmcp
984jev-github-quality-gate
A Jev-powered quality gate for GitHub issues, pull requests, and commits.
Bnymn1306/jev-github-quality-gate · TypeScript
10
ai-evaluationdeveloper-toolsgithubllm-evaluationquality-gate
1003jev-lab
Hands-on research lab for TypeSafe's Jev (System One model): reproducible benchmarks of Noul/Choice/Score primitives, confidence gating, fan-out latency, agent control — plus a living audit of the Jev ecosystem.
llt22/jev-lab · Python
10
ai-agentsbenchmarkconfidence-calibrationevaluationllmllm-evaluationllm-observability
1031jev-report
发明 RLHF 的人，这次做了个不会说话的模型：Jev 独立研究报告。52 页 PDF + 50 条中文实测复现包 + 143 条可回溯数据表
HackSing/jev-report · Python
10
pythonai-researchchinesellm-evaluation

1–43 of 43