AI systems built to work in the real world.
When a system’s job is to decide, not explain, an AI decision engine is the right tool. A language model is the wrong choice for it. An engineer does not need a paragraph to answer one clear question. Does this patient need an ER? Is this ticket toxic? Will this subscriber leave? Is this transaction anomalous? For those cases, a single-pass model returns a numeric answer in about 33 ms on a GPU. There is no text to parse and no tokens to hallucinate.
Key takeaways
- A decision engine answers typed questions in a single pass. It does not stream tokens, so it is fast and it cannot “hallucinate” an answer.
- In triage, moderation, churn, and fraud, you need a branchable score, not a prose answer.
- Laya is an open system of three checkpoints, 421M and 322M weights. On an Nvidia T4, per-question latency is about 33 ms (39.5 ms English, 32.8 ms multilingual).
- The base checkpoints are near-chance at zero shot. Real accuracy comes from fine-tuning on your own labeled questions.
- Self-hosted, a checkpoint costs nothing per token. By contrast, the leading closed decision API charges about $0.042 per million input tokens.
Where a yes-or-no answer is the whole job
Most AI projects start by reaching for a chat model. That makes sense when the job is to write, summarize, or translate. It stops making sense the moment the job is to act on a discrete question. The four cases below share one shape: a short input arrives, and a downstream system needs a label, a score, or a yes/no to choose the next step.
- Medical triage: “Does the described symptom require immediate emergency-room intervention?”
- Legal and safety: “Does this message contain an explicit threat of physical violence?”
- Customer support: “Does the customer explicitly state they want to cancel their subscription?”
- Finance: “Does this transaction log show patterns of unauthorized access?”
Each is a guardrail: a fast gate that decides whether a human or another system acts. That gate is exactly what an AI decision engine does. It takes a state, such as an email, a ticket, or a JSON document, plus one or more typed questions. It returns a label, a score, or a probability, never a sentence.
The same shape appears in Jimatech’s sector work. Its infrastructure AI turns street imagery into maintenance scores, while its healthcare data curation and financial anomaly detection pipelines feed the same kind of discrete decision. In short, wherever the next step is a discrete decision, an AI decision engine can answer it in a single pass.
How Laya is not a language model
Be clear about one point: this is not a generative LLM. A large language model predicts the next token over and over. Its output grows token by token, so latency scales with answer length. Laya instead reads each input once and scores typed questions against it. There is no generation, so there is nothing to stream and nothing to parse afterward.
One pass, not a stream
Laya is built on bidirectional encoders (ModernBERT-large and mmBERT-base) rather than a decoder. The output is strictly numbers: a top label, a probability distribution, or a calibrated P(true) between 0.0 and 1.0. Because there is no token-by-token generation, the answer length never changes the latency.
Why honest confidence replaces fake confidence
That matters for safety. A chat model that prints “confidence: 0.95” is just emitting tokens that sound confident. The number has no statistical bearing on correctness. Laya is trained with reinforcement learning against proper scoring rules. Its author called this System 1 reasoning: instant, reflex decisions with honest odds, in tens of milliseconds. A model earns the right to act only when its probability reflects real belief.
The three decision primitives
The primitives map directly onto the four cases above. First, the choice primitive picks one option and returns per-option odds, for routing and intent. Second, the score primitive places the input on a ranked scale, for urgency, frustration, or harm severity. Third, the noul primitive returns a calibrated P(true), for phishing, spam, churn, or threat flags. Because the output space is just numbers, broken JSON is physically impossible.
The router that picks the checkpoint before inference
A router inspects the input first. So it can dispatch the input to the right checkpoint in under a millisecond, before any model runs. Inputs in Latin-script English go to the English checkpoint. The multilingual checkpoint covers 100+ languages and runs about twice as fast. The typed-decisions checkpoint is tuned for the choice/score/noul workflows. Routing exists for a reason. The English checkpoint collapses to near-zero accuracy on non-Latin scripts while staying confidently wrong. So routing catches that case before inference.
How fast an AI decision engine is, and what it needs
The project reports the headline figure on a single Nvidia T4. One question takes about 33 ms. Once batched, it is 7.2 ms per question, so 10 questions finish in about 72 ms. Batched throughput reaches 103 to 332 questions per second on that same GPU.
| Checkpoint | Encoder | Params | Context | Best for |
|---|---|---|---|---|
| laya | ModernBERT-large | 421M | 512 | English |
| laya-multilingual | mmBERT-base | 322M | 1024 | 100+ languages, about 2x faster |
| laya-typed-decisions | ModernBERT-large | 421M | 1024 | typed-decisions workflows |
Those sizes make the compute story simple. These are encoders, not a billion-parameter decoder. A single T4 is enough. A CPU is usable for lower volumes, at 193 to 464 ms per question. Compare that with a chat-style decision API at about $0.042 per million input tokens. Laya answers a single question roughly 6 to 7 times faster, so self-hosted it is $0 per token. The only real constraint is memory. Preload the checkpoints you actually serve, because a language switch otherwise forces a rebuild that costs seconds.
The catch: not plug-and-play at zero shot
The figures above come from a checkpoint fine-tuned on the typed-decisions benchmark. The base checkpoints are not a general-purpose triage brain. They sit below the majority-class baseline at zero shot. The base scores 0.362 and 0.342, below a 0.461 majority baseline and a 0.318 random baseline. Only the English checkpoint is useful outside English. The project documents these honest limits. In short, the capability comes from fine-tuning, not from the base weights.
If you adopt a decision engine, you bring the labels that matter for your domain. Train against your real yes/no questions, then gate on confidence. Act automatically above 0.85 confidence, and escalate the rest. The upside is worth the work. The fine-tuned checkpoint reaches 0.766 accuracy on the typed-decisions benchmark, above the 0.727 published for the leading closed alternative. Its calibration gap is an order of magnitude smaller.
Frequently asked questions
Is a decision engine just a small classifier?
Not quite. A classifier bolted onto a frozen embedding can be fast, but it is two pieces: an encoder fixed at training time plus a head you retrain. A decision engine is one end-to-end model built for calibration as well as accuracy. It answers arbitrary typed questions at runtime without re-encoding. That is why confidence is a branching signal, not an afterthought.
Can I run this on a CPU?
Yes. On a CPU, a single question takes about 193 to 464 ms depending on the checkpoint. On an Nvidia T4, it is about 33 ms. For anything beyond low volume, preload the checkpoints you serve, so a language flip does not rebuild a model on every request.
Does “not an LLM” mean it cannot handle nuance?
It means it returns a number, not a paragraph. Nuance lives in how you design the question and the criteria, and in choosing the right confidence threshold. Honest calibration lets you escalate the genuinely ambiguous cases instead of papering over them.
Should I replace my chat-style triage now?
Only where the decision is stable, labeled, and gated by confidence. If the next action is to write a human-readable explanation, keep a language model in the loop. Decision engines earn their keep when the action is to route, flag, block, or escalate.
What is the real cost?
Self-hosted, the GPU you already use. The open checkpoint versus about $0.042 per million input tokens for the leading closed decision API tilts toward open hardware the moment you reach steady volume.
A practical next step
If your stack already moderates content, routes tickets, or flags payments, the question is worth asking: is this a decision or a draft? Where the answer is a discrete decision with a clear next action, a small, fine-tuned AI decision engine fits. It gives you calibrated confidence and millisecond speed on hardware you control. Jimatech helps infrastructure, healthcare, and financial teams prepare data so their AI systems can act accurately and effectively. To compare a decision-first design with your current flow, contact Jimatech to discuss a focused evaluation.
Better data. Smarter AI. Real-world impact.
