jev-hanko
matu79go/jev-hanko
Measuring TypeSafe AI's Jev on 41-clause contract review (CUAD, 20,500 decisions) against fast, cheap LLMs — latency, cost and F1
Python
- スター
- 0
- フォーク
- 0
掲載理由
契約ページを41種の条項に分類する課題で、Jevと他モデルを比較します。
matu79go/jev-hanko
Measuring TypeSafe AI's Jev on 41-clause contract review (CUAD, 20,500 decisions) against fast, cheap LLMs — latency, cost and F1
契約ページを41種の条項に分類する課題で、Jevと他モデルを比較します。
How many decisions can a model make in a minute, and what do they cost? Jev, a System One decision model, raced against six LLMs on the same Snake boards.
共通トピック: benchmark, latency
Measured: asking TypeSafe Jev N questions in one call bills the state once. 2,976 real requests, raw data, exact billing check.
共通トピック: benchmark, openrouter
Put Jev in the pilot's seat of a web quiz. A Python CLI that lets TypeSafe's Jev navigate quizzes in your Chrome and logs every pick, so you can measure how it does.
共通トピック: benchmark, openrouter