To The Moon

Seems like the two hottest trends these days are personal AIs (Instinct, $Meta Platforms, Inc.(META)$ Muse, $SpaceX(SPCX)$ Grok Bot, Dots from OpenAI, etc) and Decision Models like Jev. I wanted to write a bit today about decision models!

Jev (from the company TypeSafe) has taken the world by storm (it launched mid September, so very recently). Within 24 hours of launching on Vercel’s AI Gateway, Jev was used by ~13% of paid teams. This made it the fastest adopted model in the gateway history! The Information reported they’re talking to investors about raising a round at a $10b valuation. Then this week OpenAI announced their Decisions API, Databricks announced their ai_decide function (powered by a decision model), and Perplexity announced their open weight decision model. In many ways decision models is a continuation of the trend of “choice” when it comes to model types. We started with only frontier models. Then we had open weight models. Are decision models the next shoe to drop?

So what is a decision model and what is all the hype? Decision models take unstructured input and instead of giving you text as an output, they give you a “decision.” (and yes, I know I just used the word in it’s own definition…). A decision could be a binary yes/no, it could be a classifier (pick one of N categories), or a score on an ordered scale (on scale of 1-5 how urgent is this request). On top of the “decision” the model also provides a probability of how sure it is of it’s decision (this can be expressed in a percentage). Here’s an example of each. The input is a support ticket. The decision model output could be:

• binary: is this customer threatening to churn? answer: yes, 87%

• Classifier: who should I route this ticket to / which team should handle this ticket? Billing team, 92%

• Ordered Scale: on a scale of 1-10 how urgent is this ticket? 3, 78%

There are so many other ways decision models can be used. Here are some of my favorite:

• Model Routing: Take a prompt as an input and decide which model to route it to

• Agent Controls: Take an agent’s proposed next step as an input and decide if it should keep going, stop, retry, or route to a human for their input

• AI Judge: Take a LLMs output as an input and decide if if follow’s your companies brand guidelines or company policy

• Document Review: Take a customer contract as a input and decide if it contains non-standard terms that need a further legal review

As you can imagine, the possibilities start to become endless.

And the kicker on these decision models is they’re incredibly cheap! I saw an article that quoted Jev pricing at $0.042 per million input tokens (free output tokens). That’s >100x cheaper than leading model prices! It’s fair to push back on this though - the Jev pricing is obviously significantly less expensive than frontier models, but how does it compare to something like GPT6-Luna? Closer to 4x cheaper (big difference to the 100x+ figure).

And something like GPT6-Luna can also be suited for more simple tasks like classification / routing. However - just simply looking at cost doesn’t tell the full picture. There’s also accuracy, latency, and then the “scoring” calibration of the decision models. On the accuracy piece, the bet is that a purpose-built decision model will be far more accurate for tasks like binary classification than a general purpose LLM. If Jev is closer to frontier accuracy at Luna prices, it’s probably more fair to compare the price to the frontier price.

Then there’s latency. Think of a workflow made up of many if/then type statements (where the output of each is a classifier). You string tons of these together, you’ll really care about the latency. And the promise of the decision models is that they will be way faster than general purpose LLMs. Then finally, there’s the “confidence interval” the decision models spit out (which general purpose LLMs like Luna wouldn’t). It’s hard to make an apples to apples comparison and definitively answer “how much cheaper is Jev” because it’s hard to know where on the spectrum of frontier to Luna you should compare it to. Factoring in cost, accuracy, and latency I think it’s very fair to say it’s “significantly” cheaper than current alternatives today.

The natural question becomes - what does this mean for the broader model ecosystem? What percent of tokens generated today can be replicated with decision models? I asked Claude that exact question, and it guessed 10-15%. So not a trivial amount!

What’s maybe more interesting about all of this is looking at the Price x Quantity equation for token spend TAM. Everyone knows that the quantity of tokens generated is going up by many many orders of magnitude. We’re just so early. Everyone also knows P will come down eventually, but it’s more of a theoretical vs “and here’s how".” Open weight models were a great real stake in the ground for how models would get cheaper (ie how the P would go down). Decision models could be the next shoe to drop. P going down is great - it will certainly drive even more adoption and drive the Q up even faster.

Markets are always moving - and sometimes, the best move is knowing what works for you.

With Treasury yields, oil prices and rate expectations keeping markets on edge this week, investors are once again thinking carefully about where to position next. There’s no one-size-fits-all choice in investing — and the same goes for Tiger Merch. This month’s hot picks are in, featuring the Tiger Toiletry Bag, Universal Travel Adapter, Tiger Umbrella and more favourites chosen by fellow Tigers.

Explore the Monthly Hot Picks in Tiger Coin Mall, now 12% OFF for a limited time.

https://laohu8.com/J/redeemGift?goodID=100561&type=delivery