What does fast get us?

Notes from the first few days of playing with Jev, a model that answers instead of writes, around three ideas: compression, tagging and ranking.

2 minute read Source

Jev doesn't write anything. You give it some state and a list of narrow questions, and it gives back numbers. TypeSafe puts its response time at 70–500 ms, and most of my calls came back in under half a second.

I wanted to know what that speed is good for. If a judgment costs about as much as a network request, where does it go? I spent the first few days playing around, and the same three things kept turning up: compression, tagging and ranking.

Three ways to ask

Jev takes three kinds of question:

  • Yes or no, which the API calls noul, comes back as a probability that the answer is yes.
  • Choice picks one of the options you name, with odds for each and a confidence.
  • Score puts the answer on an ordered scale you write, like none, some, most, all. It returns a position between the ends.

Every answer is a number or a spread of numbers, never prose. That's the whole trick. The model makes the judgment, and plain code can sort, threshold, group and count it.

Fig. 1. One call. State and questions go in; numbers come out in 70–500 ms (TypeSafe’s figure), and code does the rest.

Tagging

A choice is a tag. In one demo, every post in a long discussion gets a role: proposal, objection, decision, reversal. Then you can lift the decisions out of the thread and see which objections nobody answered. In another, every commit in a history gets a category from its actual diff, not its message. When Jev isn't confident, the commit lands in needs review instead of a guess.

Fig. 2. Which answer does which job.

Ranking

A yes-or-no probability is a sort key. Semantic Search asks one question of each of 2,414 functions from Hono, TanStack Query and Vite: does this catch an error and carry on? The list reorders as each batch comes back. The first results showed up in about 300 ms, and the whole corpus in under nine seconds.

Score is the obvious ranking tool, and I barely needed it. A probability already sorts.

Fig. 3. Compression is ranking with a budget. Move the budget and code reselects, with no new call.

Compression

Compression is choosing what to keep. The context demo asks two questions about each turn of an agent's thread: is it relevant, and would dropping it lose something? It blends the answers, always keeps the two turns it can't do without, drops the ones with little signal, and fills a token budget from the top of the ranking down, skipping any turn too big for the room left. Choice compresses too. For each bug report, the duplicates demo picks the one passage that best explains how it relates to yours.

Fig. 4. Pairs. Most of the interesting demos sit on an edge.

Pairs

The fun started when two of these met:

  • Tag and rank: review lenses tag each hunk of a diff by concern, with a probability. A threshold folds the quiet ones away.
  • Rank and compress: context selection ranks turns and then fills the budget from the top. A new budget only means choosing again from the same ranking, so it doesn't need a new call.
  • Tag and compress: for a changelog, Jev sorts the commits into categories, and a writing model gets the groups instead of the raw log.

None of this makes the model smarter. Fast just makes a judgment cheap enough to treat like any other value: compute it, show it, throw it away when its input changes, and ask again.