ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
93% Positive
Analyzed from 5816 words in the discussion.
Trending Topics
#model#output#jev#llm#structured#llms#models#text#more#https
Discussion (191 Comments)Read Original on HackerNews
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm not the gatekeeper of who gets to call themselves a frontier model, but I don't think most people would count Jev in that group. It sounds false.
If their specific claims hold up, then it would make more sense to say something like:
"Advanced the speed/cost frontier for structured decisions"
I think AlphaFold and related are also frontier models.
Being an LLM does not seem like the qualifier for frontier.
System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
https://softwaredoug.com/blog/2026/08/10/hypothetical-classi...
You could do that or you could just... use a model that will never produce unreliable outputs in the first place.
My first thought was that it would be ideal for robotics? As in control of limbs, general planning, route finding, etc.
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
(To be clear, this one raw model does not support outputing a full AST directly - it wants to output "choice" among fixed options, "score" on a sliding scale, or a true/false answer (all of these with confidence scores attached), so building the AST/structure would be a code-driven (or even perhaps outside LLM-driven in some more challenging cases) multi-step affair where the model would essentially be playing a "game" of building the structured output step by step and getting a revised partial state back. But one could expect this to lead to interesting results.)
- @CompleteSkeptic
Very strange.
I foresee this to be the path moving forward - giving AI models understanding of the computation directly(as well as compositional rules) This feels like a short path towards total software in many areas.
For those who don't know (which is probably everyone but me), I ported the design-by-contract pattern in Python and combined it with LLMs. This was early 2025. I originally wrote about it here: https://leoveanu.com/2025-03-01-dbc/ . Contracts are a core feature of SymbolicAI ever since. The community seems to have loved it too (https://news.ycombinator.com/item?id=44399234).
I think I'm starting to glimpse the implications and it's gonna change agentic workloads if it holds up to scrutiny. It's too early for me to tell anything other than jot down some rough thoughts.
In short, you get blazingly fast semantic branching you can use in control flows. For contracts, I can now directly take the data model that you have to design and convert it into Jev's expected format. Or I can use Jev for semantic branching in postconditions.
If my understanding is correct, that should be doable, but I need to think more about it. It could be that with Jev I can finally “compile contracts” and better chain them into workflows, which is something I always wanted but didn't know how to do properly.
Eager to test. On the waiting list.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
[0]:https://docs.typesafe.ai/concepts/system-one
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson
(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
e.g.: https://docs.typesafe.ai/primitives/advanced#structured-inst...
> not sure if multi-modal
just JSON... for now :)
> outputs the question's answers as appropriate
correct!
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they emphasize parallel generation, so is it like a text diffusion model?
We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had
1. "unstructured state in, probabilistic decisions out" 2. "orders of magnitude faster and more efficient"
What was hard there was to train the model head without ML expertise, and considerable amount of data.
This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.
If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
this is complex, but generating text is highly complicated and requires mode dropping to make long cohesive text
Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.
So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?
I guess you can choose your provider still? But isn't the point that the lowest bidder is doing inference?
So, assuming this is not vaporware, this would raise the tide for everyone because it shows what's possible.
Typesafe.AI sounds like some typescript/structured output type of tool…
What even is “system one” ?
IMO the product/tech is really there, just needs better communication.
I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.
The latency and cost - yes, those are super interesting.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
I'd love to do research on this when I have the time.
That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA
we do think there's a lot of potential though and do want coding themed releases soon
As you said, if it worked, they would have demoed it haha
I like the idea of System one models but all LLMs so far work as system 1 thinking because humans generate speech subconsciously with system 1.
System 2 thinking requires consciousness which AI does not have, so even reasoning models are still system 1 thinking as system 1 in humans has reasoning with heuristics.
Its limited but most people navigate the world with it completely, so it's enough for AI.
The doom demo is quite cool
https://x.com/completeskeptic/status/2099925687465570372
I suppose this is the same video as the one from the parent comment, but I don't know for sure - I don't have a twitter account and the above link doesn't work for me.
I can see the individual tweets in the browser while not signed in though.
It's in the parent article under a section named "Doom" in case that asset URL ever changes.
OpenAI has been teasing how fast computer use is with their models running on Cerebras chips but the difference here is a burning hole in your pocket.
Edit: never mind, found https://docs.typesafe.ai/introduction/quickstart by now
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO
[0] https://goodstartlabs.com/research/verification-is-the-bottl...
Given that, can't you just replicate Jev by telling Astra "here is the question, you must make a multiple choice decision / output a score between 1-10, please answer directly in a single word, no reasoning allowed"?
(Edit: Ok, Jev is much cheaper in input tokens so these two aren't directly comparable at all)
we actually use astra (and fable) in this way for our evals: evals.typesafe.ai
someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol
https://github.com/typesafeainate/dspy-typesafeify
Is there anything published on how it maintains calibration? Or when you say "outputs calibrated probabilities" you mean "as calibrated as frontier LLM models, just cheaper" - which is a different claim; as LLM's aren't particularly well calibrated
- is the capital of france paris?
- it is august. is it raining in paris?
(forgive the examples; they're probably not semantically the sort of thing jev is trained to work on. but i figure the point translates to various kinds of questions that come up in "inner loop of agentic pid controller" contexts)
a normal text-generating model if asked to produce a number will also do that just fine. i assume in jev's case it was actually rled to essentially learn to express priors over things using its implicit world model, which definitely ought to help, but can we say more?
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
[1]: www.expressionsofchange.org
You could theoretically ask “what is the next appropriate character?” and add the entire ascii charset but i doubt it’d work well and you’d be implementing autoregressive churn across network latency…
The goal is going to be to use llms to distill operations down to some dsl, and pass it into something like Jev.
https://substack.com/home/post/p-215252866
it is a structured data model, but technically not a language model (it doesn't generate language)
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
Except this, this thing looks like revolution.
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
lol, I bet they would publish them if their score on those benchmarks were good.
This website does not inspire confidence at all, it all sounds like a marketing piece. I wish it was true, some kind of text-prompted classifier with LLM performance would be cool, but I can't trust it with what we are given.
Either way, really cool and impressive.
> we assume there is a correct compute graph (a “workflow” represented in code) and use the predictions of the largest, smartest, and most expensive external models as reference probabilities. ... Rephrased: every model gets the same workflow. We test how they compare to the average of the smartest models (in this case, Astra and Fable).
They assume there is a correct graph, but they don't compare to that, they compare to the average of the smarts models? So the smartest models are getting it wrong but you compare that anyway as a benchmark? So the outcome is "how much of a Fable am I getting" etc. Why not compare the actually correct thing?
But then even on this hand constructed eval, the first plot is showing Jev at less than Sonnet 5 accuracy. It is barely better than Luna. There are two Opus 5's and two Sonnet 5's without explanation. What is the plot showing?
I gave up.
(though ideally everyone gets off the waitlist and can try it out for themselves )
Yes, that’s the kind of attitude I want to see in these model releases
Does this imply it's a very small model? I couldn't find anything about the model itself.
I think we'll see a bunch of different architectures over the next five years.
Ouh! Any open weights models that can do this yet?? If not, how much longer? I have a Mac Studio coming soon.
https://x.com/CompleteSkeptic/status/2099925682726002904
we also thought the voice at the end was AI-ish, but apparently that's a real voice actor but slightly sped up
Reminds me of this: https://www.reddit.com/r/ITcrowd/comments/tg05j1/i_cant_beli...
forget LLM benchmaxxing sidequests, I'm sold on the real benchmark
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
edit: looks like a framer export where there is a text stroke being applied :|
https://docs.typesafe.ai/concepts/use-case-map
> LLMS > Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.
> Jev > Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
I mean, this isn't even remotely comparable to LLMs so why compare? Also, why are they bringing up AGI given there approach is so restrictive that what they're building literally cannot have the creativity required for AGI? The video is 100% marketing slop...
The bulk of the application of LLMs is that they generate reasonably reliable text which doesn't need to be defined in advanced. I'm sure there is a niche for this and congrats to the team, but please let's not hype this as if it's the next big thing in AI...
I'd use this as a tool an LLM can use for specialized tasks. It's not AI in itself.