Ollaya: Run Jev-Style Decision Models Locally at Millisecond Speeds

In the rapidly evolving landscape of AI, a new open-source project is making waves by bringing decision models—a specialized class of AI that provides typed, calibrated answers—to local hardware. Ollaya, which recently hit the front page of Hacker News, is being described as “Ollama for open-source, Jev-style decision models.” The project promises to let developers run sophisticated decision intelligence on their own machines, with latencies measured in milliseconds, all while keeping data private.

What is Ollaya?

Ollaya is a runtime and model hub that downloads and serves open decision models locally. Decision models, unlike generative LLMs, answer structured questions like “What is the intent of this customer message?” or “Is this request urgent?” with a probability score. Instead of generating tokens one by one, these models perform a single forward pass, making them exceptionally fast.

The project is currently in beta and offers a desktop app, a command-line tool, and a Docker image for servers. It supports macOS, Windows, Linux, and even WSL 2, with NVIDIA GPU acceleration available on supported platforms. The runtime is Apache-2.0 licensed, and all model weights are pulled directly from the authors’ Hugging Face repositories, pinned to specific commits and verified with sha256 hashes.

Blazing Fast Performance

Ollaya’s headline feature is its speed. On a single NVIDIA RTX 4090, a five-question request to the Laya model takes about 8–10 milliseconds end-to-end through the HTTP API. This is a significant improvement over hosted alternatives. For instance, the TypeSafe Jev hosted API has a median request latency of 236–276 ms in third-party benchmarks, which includes network overhead. Even smaller models like decider:0.8b run in about 155 ms locally, while the larger decider:2b takes roughly 190 ms.

This performance advantage is a game-changer for real-time applications. In customer support, fraud detection, or any scenario where decisions need to be made instantly, sub-20ms inference opens the door to inline processing that would be impossible with slower, token-generating models.

Drop-in TypeSafe Compatibility

One of Ollaya’s most strategic moves is its compatibility with TypeSafe’s API. The project serves /v1/systemone and /v1/models endpoints with exact request and response shapes that match TypeSafe’s. This means the official TypeSafe Python SDK (version 0.7.1) works unchanged against a local Ollaya server.

Developers can simply set TYPESAFE_BASE_URL=http://localhost:11435, use any dummy API key, and point their default model to laya. This drop-in compatibility lowers the barrier to adoption, allowing teams to switch from a hosted service to a local, private solution without rewriting their code.

Open Models, Ready to Pull

Ollaya ships with a curated set of open-weight models, each optimized for different decision tasks:

• Laya (from Convai Innovations): Available in English and a 100+ language multilingual version, fine-tuned for typed decisions. Sizes: 322M and 421M parameters.
• decider (by Mapika): Based on Qwen3.5, this model reads answer from option-letter logits in a single forward pass. It is touted as the most accurate open decision model Ollaya ships. Sizes: 0.75B and 1.9B.
• nli (by Moritz Laurer): Zero-shot classifiers that score each option as a hypothesis for entailment. Sizes: 396M and 435M.
• gliclass (by Knowledgator): An instruction-following zero-shot classifier that scores all options of a question in one pass, so cost barely grows with the number of options. Size: 439M.

More models are planned, including von and GGUF LLM-based decision models via llama.cpp, which could further expand the ecosystem.

Privacy and Calibration

Ollaya positions itself as a privacy-first alternative. Since everything runs locally, sensitive data like tickets, emails, and user messages never leave the machine. The server listens on 127.0.0.1 by default, and there are no per-token fees or metering.

Calibration is another key focus. The project claims Laya’s calibration error (ECE) is 0.081 after temperature fitting, compared to 0.246 for Jev. This means the probabilities returned by Laya are more trustworthy, allowing developers to set thresholds with confidence.

Platform Support and Installation

Ollaya aims to be easy to install. A single binary and the command ollaya run laya gets you started. Here’s a quick breakdown of platform support:

NVIDIA GPUs require driver R580 or newer. The installers automatically fetch CUDA libraries only when a compatible GPU is detected.

Why It Matters

Ollaya’s emergence signals a broader trend in AI: the move from centralized, hosted models to local, open alternatives. By combining the speed of decision models with the privacy of local execution, it addresses two major pain points for enterprises—latency and data sovereignty.

The project has already captured the attention of the developer community, hitting #1 on Hacker News. Its drop-in compatibility with TypeSafe could accelerate adoption, as teams can test the waters without committing to a full migration.

While Ollaya is still in beta, its roadmap and execution suggest it could become a staple in the decision intelligence toolkit. For developers looking to build responsive, private, and cost-effective decision systems, Ollaya is a project worth watching.