All projects

jev-certify

nikkoxgonzales/jev-certify

Finite-sample guarantees for Jev (TypeSafe's System One). Conformal risk control turns calibrated probabilities into certified routing thresholds; prediction-powered inference audits them. 2,412 decisions on CLINC150 for $0.23 — including the shift and prevalence cases where the guarantee breaks.

Python
Stars
1
Forks
0

Why it is listed

A research tool applies conformal risk-control methods to turn Jev choices into routing thresholds; its reported experiment is based on logged Jev decisions.

Related projects

  • jev-measured

    Measured cost, latency, accuracy and raw output: TypeSafe's Jev and 12 other decision models (OpenAI Decisions API, Clef, Decider, d1…) — reproducible

    Shared topics: decision-models, llm-routing

  • jev-fanout-bench

    Measured: asking TypeSafe Jev N questions in one call bills the state once. 2,976 real requests, raw data, exact billing check.

    Shared topics: openrouter

  • JEV

    JEV 调研报告:TypeSafe System One 决策模型与开源对标(含原始核实记录)

    Shared topics: llm-routing