Some data should never leave the building. Bank statements, payment rails, customer records, anything you would not paste into a hosted chatbot. We wanted an assistant that can reason over exactly that kind of material and call real tools against it, with a simple rule: no third party ever sees the prompt or the data. The only network traffic the models make is to localhost.

The constraint that makes this interesting is the hardware. One consumer GPU with 8GB of VRAM (an RTX 3070 Ti), a six-core CPU (Ryzen 5 5600X), and 64GB of system RAM. That is not a lot of VRAM for an agent that both plans and generates. The trick was to stop treating "the model" as one thing.

We run two models with different roles, and we put them on different hardware so they never fight for memory.

The first is the router. It is the tool-manager: it reads the request, decides which tools to call, calls them, reads the results, and decides what to do next. For that job we use LFM2.5-8B-A1B, a mixture-of-experts model from Liquid AI. It has 8.3B total parameters but only about 1.5B are active on any given token, and it does native tool calling. The "only 1.5B active" part matters, because it means the model is fast even on a CPU. So the router lives mostly in system RAM, and we offload about half of its layers to the GPU as a slice. The slice is tuned to a hard limit: enough to quicken the router, but small enough that the worker still stays fully resident on the same card, so the two never swap.

The second is the worker. It does the actual generation and analysis, including anything a hosted model tends to refuse for no good reason (security research, adversarial examples, blunt financial reasoning). For that we use an uncensored build of Gemma-4-E2B, quantized to a high-quality 8-bit and paired with its vision projector. It sits fully on the GPU, sharing the card with the router's slice but with room for both.

The router calls the worker as just another tool. Ask a question that needs open-ended generation and the router emits a tool call to the worker, gets the text back, and folds it into its answer. Everything runs on the same box.

you -> router (LFM2.5, CPU + GPU slice, tool-manager)
          |
          +-- read_file / list_dir / search_files   (sandboxed)
          +-- ask_uncensored -> worker (Gemma, GPU)
          +-- http_get (https only, allowlist, approval gate)

Because the target is finance, the tool layer is built defensively from the start. Tools are Python functions with pydantic-typed arguments, so every call is validated before it runs. File tools are sandboxed: a path is resolved to its real location and rejected if it escapes the workspace root, so a symlink or a ../../ cannot walk out. Anything that touches the network or changes state is gated behind an explicit approval step, and it fails closed. Run the agent non-interactively and a network tool is denied automatically, because there is no human present to approve it. Secrets come from environment variables, are never written to disk by the tool, and are redacted from the audit log. Every tool call is logged, redacted, to a local file.

Adding a tool is deliberately boring: write a function, annotate its arguments with a pydantic model, and it appears in the registry with a generated JSON schema the router can call. A bank API connector is the same shape as the built-in file reader, plus an allowlist and an approval gate. The connector for a specific bank is the one piece we do not ship, because that needs a specific API. The secure pattern around it is done.

The whole thing also exposes itself over the Model Context Protocol, so our coding agent can call the local stack as a tool without any of the data leaving the machine.

The tool layer above is the fence around our own code. The next step was to run tools we did not write, safely, and to be able to prove afterward exactly what ran. So the router now drives WasmBox, a small launcher for sandboxed WebAssembly tools. Think of it as a Flatpak for .wasm: install a tool, verify it, and run it with only the capabilities it declares and nothing more.

We wired the installed wasmbox CLI in as first-class router tools, so the LFM2.5 router can call them on-device. Each one is a capability-zero wasm binary, hash-verified before it runs and sandboxed to stdin and stdout only: no filesystem, no network, no clock. The set we exposed is small and practical: jfmt, secretscan, b64, compact, epoch, hashit, yamlfmt, diffsummary, errparse, and worldid-verify. A tool that can only read stdin and write stdout cannot exfiltrate anything, by construction, whatever it claims to do.

The reason this matters beyond safety is the record it leaves. WasmBox keeps a policy of which tools are approved and a signed, append-only audit log of every run. That is the shape the EU AI Act asks for: traceability of what the system did, and a human in the loop for changes to it. Installing a new tool is approval-gated, the same fail-closed step the network tools already pass through, and the log is Ed25519-signed, so an entry cannot be quietly altered after the fact.

The stack now has two security layers that compose rather than duplicate. localai brings the approval gate, the redacted audit log, and the path sandbox around our own tools. WasmBox brings per-tool capability grants, hash verification of each binary before it runs, and signed compliance reports for the tools it hosts. A sensitive run passes through both.

To make sure this was real and not just a diagram, we built the use case it is meant for. We loaded a reference dataset of European UCITS ETFs (IWDA, VWCE, SXR8, AGGH) and a sample portfolio, and exposed them as local tools: etf_quote and portfolio_value to read and value the book, and portfolio_buy and portfolio_sell to trade it. Reading is ungated; every trade is a state change, so it passes through the same approval gate as a network call, and it is refused outright when the cash is not there.

Then we let the router run it, not us. From a single instruction, value the portfolio, buy twenty shares of one ETF, sell a hundred of another, and value it again, the LFM2.5 router made the four tool calls in the right order and approved each trade once at the gate, the whole sequence on-device in about a minute. It read a book worth €30,643 against €5,000 of cash; the buy took cash down to €2,896, the sale brought it to €3,381, and the total held at €30,643, because trading at the reference mark only shifts value between cash and holdings. Every call landed in the signed log, and nothing left the machine. The one piece we still do not ship is a live price feed; the reference dataset stands in for it, and the fence around it is built.

One honest limit. The WasmBox tools run through the CLI as a subprocess rather than in-process, so there is a small wrapper cost on each call. The models are still the local two-model stack described above, unchanged. And as before, nothing here reaches a third party: the sandbox, the hash check, and the signed log all run on the same machine.

Measured on the hardware above, both models resident at the same time.

Worker (Gemma-4-E2B uncensored, 8-bit, GPU):

• Generation around 118 tokens per second.

• Prompt ingestion around 2,000 to 2,700 tokens per second.

• 2.7GB of VRAM resident. Cold load about half a second.

• Vision confirmed with a control test: red, green, and blue test images were each identified correctly, so the projector is reading pixels rather than guessing a plausible colour.

Router (LFM2.5-8B-A1B, 4-bit, CPU with a GPU slice):

• On the CPU alone it generates around 31 tokens per second on six cores, respectable for an 8B-class model precisely because only 1.5B parameters are active. Offloading about half its layers to the GPU lifts the reasoning pass while keeping both models resident.

• About 5.6GB in total, now split roughly half on the GPU and half in system RAM. Cold load about 0.2 seconds.

• Native tool calls parse correctly out of the box.

The GPU slice is sized so both models stay loaded together: about 6.2GB of the 8GB card in use, the worker fully resident and the router half-offloaded, with roughly 1.6GB to spare and no model swapping between turns. A full two-hop agent task (router decides, worker generates, router reads the result and answers) still completes in roughly thirty seconds, because that time is dominated by the router's reasoning pass and the cold model loads rather than by where the layers sit; the slice helps most when the router has a long chain of tokens to work through.

On refusals: the worker's author reports zero refusals across their own test battery. We did not reproduce that number, but in our own use for legitimate security work it answered questions a hosted model routinely declines, which is the entire reason it exists in this stack.

This is a small setup, and the honest limits are the useful part.

The worker is a 2B model. It is quick and it is uncensored, but it is not a frontier reasoner. Long multi-step logic, subtle nuance, and very long coherent output are not its strengths. Treat it as a fast local generator, not as a replacement for a large hosted model when the task needs one.

The router's latency is real. A two-hop task lands in the low thirties of seconds, fine for a background finance job and slow for a snappy interactive chat. lThe obvious lever, and the one we took, is to give the router a slice of the GPU rather than pin it fully to the CPU: about half its layers offload, which quickens its reasoning while leaving the worker fully resident, with no swapping. That slice is deliberately capped. Push much past half on this 8GB card and the load runs out of memory, or the worker gets evicted and the two begin swapping, which is worse than a slower router. So the real way to make this fast is a bigger card, not a bigger slice.

Vision works today because the current local runtime accepts a separate projector alongside the model. Older runtimes ignore it. If yours does, the fallback is to run the weights directly with a projector flag, and the setup notes exactly that.

Finally, the bank connector itself is not written. What is written is the fence around it: sandbox, allowlist, approval, redaction, audit. That is the part that is easy to get wrong, so that is the part we built first.

None of this is exotic. It is ollama, a few hundred lines of typed Python, and a decision to split one model into two. The payoff is an agent that can work on the numbers you cannot send anywhere else.

Feel free to reach out to me if you have questions