Skip to content
KOLDOS
Sign in
KOLDOS

How KOLDOS works

A KOLDOS turn runs through the kernel, the Decision Engine, the Context Broker, the model and the tools, all on your computer. The map shows what runs locally, what reaches the internet and what is only planned.

System map

Your computer. Runs on your hardware.
Location not specified. Not defined yet, or depends on the feature.
Outside your computer. Needs a service over the network.

Local chat

Available

A normal turn: a question or a coding task. The model, the tools and the memory run on your own hardware.

  1. 01You send a request from the app, the CLI or the local API.On your computer
  2. 02The Decision Engine picks a route and the Context Broker assembles the prompt within its budget.On your computer
  3. 03llama.cpp runs the model on your GPU.On your computer
  4. 04Tools, verification and memory run on your computer too, and the turn is traced to disk.On your computer

What leaves your computer

Nothing. KOLDOS is designed with no outgoing telemetry and no cloud mode.

Parts marked with a status square are not usable yet. The diagram shows how they are designed to connect; implementation details will be documented as each one ships.

Optional components

Local, external or not specified

Each optional component with its status and where it runs. Nothing is marked local unless that has been stated.

  • Memory

    Experimental

    On your computer

    Stored in a SQLite database on your disk, one per project.

  • Files

    Available

    On your computer

    Attachments are copied to a folder inside the workspace on your disk.

  • Vision

    Experimental

    On your computer

    OCR and the vision model run on the CPU of your computer. The image becomes text before it reaches the main model.

  • Tools

    Available

    On your computer

    Every tool call goes through the Tool Manager, the only boundary to disk and network, and is limited to the workspace by a capability token.

  • Web search

    Experimental

    Outside your computer

    The search runs through a SearxNG instance started by KOLDOS on your computer, which sends the query to public search engines.

  • KOLDOS API

    Planned

    Outside your computer

    Requests would be processed on hosted infrastructure. The design exists; the service has not started.

Full descriptions are on the capabilities page.

Local AI

Why a model can run on your computer at all

A few years ago this needed a data center. Smaller models, quantization and fast GPU memory changed that. It also comes with limits worth knowing.
Privacy
With local inference, prompts and replies are processed in your computer's memory. Whether anything leaves the machine depends on what you enable: web search sends queries to a search provider, and the KOLDOS API processes requests outside your computer. Local does not mean private in every configuration.
Control
You choose which model to run, how compressed it is and how much context it gets. The model files sit on your disk, so the model you tested is the model you keep using.
Your own hardware
Your GPU and RAM do the work. There is no per-request fee for local inference; the cost is the hardware you already have and the electricity it uses.
Less dependence on outside services
Local inference doesn't need a remote server to be up. Downloads, updates and features like web search still need a connection.
Quantized models
Storing weights with about 4 to 8 bits instead of 16 shrinks a model to a fraction of its size. That is what lets a 7B model fit on an ordinary graphics card, at a small cost in quality.
GPU use
Generating text means reading the model's weights for every token. Graphics memory is much faster than system RAM, so a model that fits in VRAM generates text far faster.
RAM use
When a model doesn't fit in VRAM, part of it can run from system RAM on the CPU. It works on more machines, but those layers slow down every token.
Runtime profiles
KOLDOS picks quantization, KV cache precision and context window per GPU. On a 6 GB laptop GPU it runs Q5_K_M with an 8-bit cache and a 24K window; with 10 GB or more the profile moves to Q6_K with a 32K window.

Data residency

What stays on your computer

The answer depends on the feature. This table covers each kind of data in each mode.
DataLocal chatWith web searchHosted API, planned
Your prompts and the repliesStays on your computerStays on your computerSent to the KOLDOS API
Search queries and the pages readNot involvedSent to public search engines through SearxNGNot involved
Your code and attached filesStays on your computerStays on your computerSent to the KOLDOS API
Memory databaseWhere memory would live for the hosted service has not been decided.Stays on your computerStays on your computerNot specified. Not published yet
Model weightsStays on your computerStays on your computerNot involved
Turn traces and diagnosticsStays on your computerStays on your computerSent to the KOLDOS API

Components

The layers, top to bottom

What each layer is responsible for. Implementation details that haven't been published are marked as pending.
  1. L1

    Interfaces

    Where requests come in: the desktop app, the CLI with its own chat, and the local REST API with streaming.

    Desktop app
    Electron
    Local API
    REST and server-sent events on 127.0.0.1
  2. L2

    Kernel

    Opens each turn, assigns its budgets, issues the capability token and runs the message bus and scheduler.

    Concurrency
    One process, one model in VRAM
    Scheduler
    Interactive turns always first
  3. L3

    Cognition

    Decision Engine, Planner, Executor, Verifier and Critic. None of them talks to the model runtime or the disk directly.

    Decision Engine
    Rules, then the model with a grammar, then a fallback
    Verifier
    Deterministic: compile, tests, schema
  4. L4

    Resources

    The Context Broker assembles every prompt within budget, the Memory Manager recalls and forgets, the Tool Manager is the only boundary to disk and network.

    Active context target
    Under 10K tokens
    Tools
    25, exposed a few per turn
  5. L5

    Ports and adapters

    Interfaces the cognition layer depends on, and the adapters that implement them. Swapping the inference engine means writing one adapter.

    Inference
    llama.cpp with CUDA
    Storage
    SQLite, one database per project
    Model file
    GGUF
  6. L6

    Your hardware

    GPU and VRAM for the model, CPU and RAM for everything else, disk for the model file, memory and traces.

    Sized with the hardware checker.