Jev plus Opus gives near-frontier accuracy at 4x lower cost

Hightouch reported that pairing the decision model Jev with Claude Opus gets close to Opus-level accuracy when grading identity resolution, at more than four times lower cost. The finding matters because identity resolution is expensive to check by hand, and a cheaper judge that knows when it is unsure could make routine auditing far faster without giving up frontier-level quality.

What did Hightouch test, and why identity resolution?

According to Hightouch, identity resolution is the process of mapping data records to real people, creating a set of identities so that each identity corresponds to exactly one person and no two identities correspond to the same person. The company describes this as foundational for outreach, personalisation and analytics, because a business cannot, for example, calculate a customer's lifetime value unless all of that customer's purchases can be mapped to a single identity.

The hard case, according to Hightouch, is over-merging, where one identity incorrectly covers multiple real people. People share emails, move addresses, use nicknames, and records contain typos, misread characters, swapped dates and blank fields. On real data there is no answer key to consult after records have been merged, so teams must sample output identities and review them by hand.

Hightouch says human sampling is slow, at roughly 50 identities per hour, and that estimating a rare over-merging rate precisely takes more samples than many teams expect, with more than 1,000 samples needed to bring the error on the over-merging rate to under plus or minus one percent. The question behind the test was whether a language model could replicate that human inspection, so that many identities could be graded in parallel.

How did the models perform on the first benchmark?

Hightouch built a benchmark using pseudopeople, a tool from the Institute for Health Metrics and Evaluation at the University of Washington that simulates a fictional US town and generates records of residents from multiple sources, with additional realistic data errors added. From those records the team built 1,000 identities, half representing one person with noisy records and half representing over-merges.

The prompt asked the model to audit a profile produced by an identity resolution system, decide how many distinct people the records describe, and, only when over-merging was judged true, provide a justification naming the specific values behind the decision. The prompt also listed common kinds of noise, such as typos, nicknames and dates with month and day swapped, and explained that the records came from US government sources covering 2020 to 2040.

With the final prompt, according to Hightouch, Claude Opus 5.5 reached 99.0% accuracy, Gemini 3.1 Pro reached 98.3%, and GPT-6 Sol reached 98.3%. The decision models were close behind on that first test: GPT-6 Luna reached 97.3% and Jev reached 95.6% at a 50% confidence cutoff. The cost and speed gap was much larger. Opus cost $3.86 per 1,000 identities and took 2.2 seconds per identity, while Luna cost $0.10 and took 0.17 seconds, and Jev cost $0.05 and took 0.15 seconds.

Hightouch then adjusted the confidence threshold. The team settled on 60% for Jev and 70% for Luna, which maximised accuracy in each case. At those thresholds Jev reached 95.7% and Luna remained at 97.3%, according to Hightouch. The cutoff mattered a lot for Jev and very little for Luna.

Why did Jev look weaker alone but stronger in a hybrid?

Hightouch reports that Luna performed better than Jev out of the box, but that the gap disappears on harder cases. The more important difference was calibration. Jev returns a confidence score from 0 to 1 for whether a specific condition is true, and Hightouch found that most of Jev's mistakes came with low confidence that it was right. Most of Luna's mistakes, by contrast, came with high confidence.

That pattern matters for a hybrid evaluator. If a cheap model is usually unsure when it is wrong, the system can send only those uncertain cases to a frontier model and accept the cheap model's answer when its confidence is high. A model that is confidently wrong is harder to correct in that way, because the system has no signal that a second opinion is needed.

Hightouch describes Jev and Luna as members of a new class of decision models that return a confidence score, and says they are far faster and cheaper than frontier models. In the company's summary, decision models are up to 90 times cheaper and more than 10 times faster out of the box, costing between $0.10 and $0.50 per 1,000 identities in the summary text and answering in about 0.15 seconds, while trailing Opus accuracy by about six percentage points on harder benchmarks. The table in the article gives the specific per-model figures quoted above.

What does pairing Jev with Opus change?

The central result, according to Hightouch, is that a hybrid model combining Jev with Opus achieves frontier accuracy at more than four times lower cost than using Opus alone. The article's headline frames this as a Jev model that can ask Opus for help giving Opus accuracy at four times lower cost, and the text says the pairing gets close to Opus-level accuracy for much less.

Hightouch is clear that frontier models are still more accurate than decision models on their own. The hybrid does not claim that Jev alone equals Opus. Instead, the benefit comes from using Jev as a first pass and reserving the expensive model for the cases where Jev's confidence is low. That is why calibration, rather than raw solo accuracy, became the deciding factor in the team's conclusion.

For teams that grade large volumes of identities, the practical implication is a different cost structure. Instead of paying frontier prices for every identity, they pay decision-model prices for most identities and frontier prices only for the uncertain subset. Hightouch argues this could enable much faster iteration when setting up identity resolution and make maintenance easier, because a sample of 100,000 identities could be evaluated in seconds rather than through slow manual sampling.

How could you explore model comparison in Grafy?

This story is about comparing models on the same task and learning when a cheaper model is safe to trust. That is a natural fit for Grafy Chat, where readers can chat with many AI models side by side and compare how different models handle the same prompt, including cases where one model is unsure and another is more reliable. A reader could use that side-by-side setup to test a small auditing prompt, vary the instructions, and see where answers diverge before committing to a production approach.

For deeper reading, Grafy Reader can help a reader work through PDFs and papers with AI, which is useful if they want to study evaluation methods or identity resolution literature in more detail. Two Grafy News Desk pieces are also relevant background: one on lowest-cost model routing and one on asking several AI models at once with a judge. Neither replaces the Hightouch benchmark, but together they show how routing and judging choices affect cost and quality.

Frequently asked questions

Does Jev match Claude Opus on its own in Hightouch's test?

No. According to Hightouch, Jev reached 95.7% accuracy at its best threshold, while Claude Opus 5.5 reached 99.0%. Jev was much cheaper and faster, at $0.05 per 1,000 identities and 0.15 seconds per identity, but it did not match Opus alone. The near-frontier result came from the hybrid pairing, not from Jev by itself.

Why can Jev work well in a hybrid despite lower solo accuracy?

Because its confidence is better calibrated, according to Hightouch. Most of Jev's mistakes came with low confidence, so a hybrid system can send uncertain cases to Opus and accept Jev's answer when confidence is high. Luna's mistakes more often came with high confidence, which makes it harder to know when a second opinion is needed.

How much cheaper and faster were the decision models than the frontier models?

Hightouch says decision models are up to 90 times cheaper and more than 10 times faster out of the box. In the table, Jev cost $0.05 per 1,000 identities and took 0.15 seconds per identity, compared with $3.86 and 2.2 seconds for Claude Opus 5.5. Luna cost $0.10 and took 0.17 seconds per identity.

---

Source: Hacker News — front page — A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost