タグ:benchmark
benchmark タグの付いた Jev GitHub プロジェクトを検索・並べ替えし、元のリポジトリを確認できます。
- 2225jev-kitGet coding agents to write Jev questions that work the first time: rules measured on 57 labelled suites.
BarakChamo/jev-kit · JavaScript
00agent-skillsai-agentsbenchmarkcalibrationclaude-codeclaude-code-pluginclaude-code-skill - 2288jev-no-enemReproducible benchmark evaluating TypeSafe AI's Jev (System One paradigm) on Brazil's ENEM 2025 standardized exam. Evaluates typed decision-making, domain-specific accuracy, and RLCD uncertainty calibration against open LLM baselines with an interactive GitHub Pages dashboard.
patryckalves/jev-no-enem · Python
00benchmarkbrasilenemllmrlcdpython - Does ORDER BY over a Jev probability put rows in a defensible order? Independent ranking, calibration and invariant measurements of TypeSafe AI's Jev: passes six pre-registered gates on 360 labeled rows, fails four of six on graded product relevance.
yodablocks/jev-orderby-bench · Python
01benchmarkcalibrationduckdbinformation-retrievalllm-evaluationpythonranking - 2353jev-quiz-pilotPut Jev in the pilot's seat of a web quiz. A Python CLI that lets TypeSafe's Jev navigate quizzes in your Chrome and logs every pick, so you can measure how it does.
juanfabrega/jev-quiz-pilot · Python
00benchmarkbrowser-automationclillm-evaluationopenrouterplaywrightpython - Reproducible Chinese RAG reranking benchmark for Jev (TypeSafe System One) — private during build, protocol + scripts + full logs. EN/中文.
vicfei/jev-rag-zh-benchmark · Python
00benchmarkchineseragreproducible-researchrerankingt2ranking - 2358jev-ragcheckRAG evaluation with typed decisions: sentence-level hallucination verdicts with offsets and citations, passage and answer relevance, in one Jev call per answer. Benchmarked against Ragas, DeepEval, an LLM judge and HHEM.
gabazureus/jev-ragcheck · Python
00benchmarkfaithfulnesshallucination-detectionllm-as-a-judgellm-evaluationportuguesepython - 2460jev-tool-callingApplying Jev to zero-shot tool calling on the Berkeley Function Calling Leaderboard (BFCL)
will-rice/jev-tool-calling · Python
01benchmarkbfclevaluationfunction-callingllm-agentstoken-classificationtool-calling - Does a System One model actually beat keyword matching? A reproducible benchmark on catching disguised duplicate thesis titles. 24 cases, real production baseline, raw data and charts included.
devnolife/jev-vs-tfidf-benchmark · TypeScript
00benchmarkindonesiainformation-retrievalllmplagiarism-detectionstructured-outputtf-idf - 2664mars-gateA semantic-decision benchmark comparing Jev with general-purpose LLMs in a fictional Mars entry checkpoint.
komzweb/mars-gate · TypeScript
00benchmarkllmreacttypescript - 2716open-system-oneAn independent, reproducible benchmark of TypeSafe's Jev against open, CPU-only alternatives — 10,000 decisions, all raw results published.
zhlei07/open-system-one · Python
00benchmarkcpu-inferencecross-encoderembeddingstext-classificationzero-shot-classification - How many decisions can a model make in a minute, and what do they cost? Jev, a System One decision model, raced against six LLMs on the same Snake boards.
angelgalvisc/snake-arena-jev-vs-llms · Python
00anthropicbenchmarkcalibrationdecision-makinglatencyllmmoonshot - Jev vs Laya on Bulgarian: two System One models suggest a category for each blog post. A local BgGPT names new topics.
SciScend/system-one-categorizer-demo · Python
00ai-agentsbenchmarkbggptbulgarianlayallmmodel-comparison - Run the new wave of decision models (Laya, Decider, Kev, Jev) side by side on your Mac. Structured input in, calibrated probabilities out, with honest accuracy, calibration, latency and memory numbers.
jeffclimbsbikes/system-one-playground · TypeScript
01apple-siliconbenchmarkcalibrationclassificationdecision-modelevaluationlaya - Large typed decision model evaluation benchmark and novel calibration standard (`calibration.json`) for any typed decision inference system, covering 275 distinct use-cases and 27,598 individual decisions for Jev-like open typed decision models.
kyr0/typed-decision-bench · Python
00benchmarkbenchmark-frameworkcalibrationjev-alternativejev-api - Bench of typed decision models: Jev vs OpenJev vs Laya, small local LLMs and cheap hosted LLMs on the same zero-shot classification tasks, in English and French.
4nt0ineB/typed-decision-bench · Python
00benchmarkcalibrationenglishfrenchlayallm-evaluationmassive-dataset - Score TypeSafe's Jev on water-supply triage reports: a zero-dependency Python benchmark that answers urgency, scope, cause, priority and escalation questions and checks them against your team's expected policy.
hariprasad2512/water-supply-prioritizer · Python
00benchmarkllm-evaluationpythonwater-supply
66 件中 51–66 件