Jev for Designers

A model that tells you how sure it is, and why it matters to us.

On the 15th of September a company called TypeSafe released a model called Jev, and engineers have found it pretty exciting, but what’s in it for designers? If you’ve seen the name everywhere this fortnight and wondered whether it’s hype or worth the time, this post is for you. I think it does, and I built a small demo so you can see why.

---

What is Jev?

Most of the AI we use writes and chats. We ask the chat model for something and it writes back, a paragraph, some code, a plan. Jev doesn’t write at all. You give it some information or ask a question, you say which answers are allowed, and it returns a decision.

Specifically, it answers three kinds of question:

• A Choice picks one option from a list you give it, like whether an email is a complaint, a question or an invoice.

• A Score places something on a scale, like how urgent a message is from one to five.

• A Noul is TypeSafe’s name for a yes-or-no question, and it returns the probability that the answer is yes.

Every answer comes with a number which is how likely each option is.

It’s also fast and very cheap. Most calls come back in about a tenth of a second, input costs $0.042 per million tokens and output is free. Karo Zieminski measured a four-question request at $0.000126, which is why developers have piled in so quickly. The speed and price aren’t the only reasons I think designers should pay attention.

---

Why that matters

Reinforcement learning from human feedback (RLHF), rewards the answers people prefer, and people tend to prefer answers that sound confident and agree with them. This is why so many models are very good at sounding sure.

Jev was trained differently. TypeSafe calls its method “Reinforcement Learning for Calibrated Decisions”, and it rewards probabilities that are honest rather than answers that please. The aim is that when Jev says 70%, it’s right about seventy times in a hundred. TypeSafe’s founder, Diogo Almeida, was one of the people who helped invent RLHF and InstructGPT at OpenAI.

On TypeSafe’s own benchmark Jev gets around 68% right, close to mid-tier language models. Honest uncertainty is exactly the material you need when designing though. In design things are rarely binary.

---

A bit of fun with Jev

HeyStefan built a lovely demo where you type a concept and watch Jev sort a pile of emoji against it. I wanted to build on it and add the “maybe zone”, which is what Jev does when it isn’t sure.

There are 166 emoji in the pile, from a pencil and a paperclip to a rotary phone, the London Underground sign, a rubber duck and an egg. Type a concept, or pick one of the starters like “things with iconic design” or “things you’d find in a studio”, and Jev judges them.

Each emoji floats to a height that matches how sure Jev is. The ones Jev can’t decide on hang in the middle.

Then you get your say: Drag any emoji to where you think it belongs and a line appears between Jev’s answer and yours.

---

A few limits

Jev only reads text. It can’t look at a screenshot, so you have to describe your design in words, which will teach you a lot about how clearly your design is documented.

It also reads your question literally, and TypeSafe publishes a page listing where the current model breaks. Writing the question well is design work too.

You can try it this week. There’s a waitlist for direct access, but it’s available through Vercel’s AI Gateway without one.

---

Thanks to HeyStefan, whose emoji demo made me smile in the first place (and the various others that were super fun by Nelson), and to Karo Zieminski, whose explainer was awesome. There’s a growing list of use cases and fun stuff in here.

Go have a play!