Flat per-tenant licensing works because marginal cost is close to zero. One more user is a few rows and some bandwidth. Put an LLM feature in the product and that stops being true: every conversation consumes inference, retrieval and sometimes tool calls, and one enthusiastic tenant can cost many times more to serve than another on the identical plan.
If you cannot attribute that spend, you cannot price the feature, and you cannot even tell which customers are profitable. Here is a design that makes the numbers visible and keeps them bounded.

Put one gateway in front of every model call
Everything depends on a single choke point. Feature code should never call a provider SDK directly. It calls an internal gateway that knows the tenant, picks the model, checks the budget, records usage and handles credentials.
const result = await llm.complete({
ctx, // verified tenant + user
task: "classify_ticket", // routing key, not a model name
feature: "support_triage", // the line item for reporting
input,
});
Note that callers pass a task, never a model. Model selection becomes a configuration concern, which you will want when a supplier changes pricing or retires a model.
Meter everything that costs money
Tie model tokens, embeddings and tool calls to the tenant that incurred them. Write an immutable event per billable operation and aggregate later. Storing raw units alongside a calculated cost lets you re-rate history when prices change.
CREATE TABLE llm_usage_event (
id uuid PRIMARY KEY,
tenant_id uuid NOT NULL,
user_id uuid,
feature text NOT NULL,
task text NOT NULL,
kind text NOT NULL, -- completion | embedding | tool_call
provider text NOT NULL,
model text NOT NULL,
input_tokens integer,
output_tokens integer,
cached_tokens integer,
key_source text NOT NULL, -- platform | tenant_byok
cost_micros bigint,
price_version text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
Include two practical details. Streaming responses must still emit a final event when a client disconnects midway, because the provider may bill for what was generated. Retries need to be recorded too, since a retry storm is a real cost even when the user saw only one answer.
Background work counts as well. Nightly re-embedding for one tenant's document library belongs on that tenant's ledger. Otherwise, it disappears into a shared overhead number.
Route by task
Simple classification and short replies rarely need the most capable model, and sending everything to the largest one is the most common source of avoidable spend. Keep routing in data so it can be changed per tenant and per plan without a deploy.
tasks:
classify_ticket: { tier: small }
short_reply: { tier: small }
summarize_doc: { tier: medium }
agent_plan: { tier: large }
tiers:
small: { primary: provider-a/small-model, fallback: provider-b/small-model }
medium: { primary: provider-a/medium-model, fallback: provider-b/medium-model }
large: { primary: provider-a/large-model, fallback: provider-a/medium-model }
Treat a routing change like a prompt change. A cheaper model may help one tenant's use case and harm another's, so run per-tenant evaluation before you move a task down a tier.
Budgets need a defined behavior at the limit
Each tenant gets a cap. The interesting part of the design is deciding what happens as the cap approaches. A hard failure in the middle of a user's conversation is the worst option available.
function resolveBudgetAction(state: BudgetState, task: Task): BudgetAction {
const used = state.spendMicros / state.capMicros;
if (state.keySource === "tenant_byok") return { action: "allow" };
if (used < 0.8) return { action: "allow" };
if (used < 1.0) return { action: "allow", notifyAdmin: true };
if (task.degradable) return { action: "downgrade", tier: "small" };
return { action: "block", userMessage: "assistant_limit_reached" };
}
The thresholds above are illustrative. Whatever you choose, document the behavior so the tenant can read it, and notify the tenant's administrator before their users notice anything. A shift to a smaller model or a clear message to the user are both acceptable outcomes. Silent quality loss without any notice is not.
Check budgets against a fast counter, such as a per-tenant running total in an in-memory store, and reconcile it against the event table. Decide in advance how much overshoot you accept, because concurrent requests will all pass the check at once. Per-user sub-limits also stop one person from consuming a whole tenant's allowance.
Price so that revenue follows cost
Once you can see usage, the commercial options become straightforward. The goal is a model where margin survives your heaviest tenant.
-
Bundled allowance plus overage. Easy to sell. Set the allowance from observed usage, not from hope.
-
Credits. One abstract unit across tasks of varying cost. This hides model churn from customers and lets you reprice internally.
-
Usage tiers. Predictable for buyers, though you need a clear rule for what happens on crossing a tier.
-
Bring-your-own-key (BYOK). Large clients pay their provider directly, and you charge for the platform.
BYOK has more edges than it appears to
Store tenant keys in a secrets manager, encrypted, and fetch them only at the gateway. Never log them. Then deal with the implications:
-
Keep metering on. Record
key_source: tenant_byokwith token counts. You still need the usage data for support, abuse detection and capacity planning, even at zero cost to you. -
Budget checks change. The tenant's spend is the tenant's concern, but your own rate limits still apply. A noisy BYOK tenant can still saturate shared infrastructure.
-
Provider choice constrains features. State which features depend on capabilities a given provider may lack, and fail clearly when a tenant's chosen provider cannot serve a task.
-
Tenant-scoped caching still applies. Cache hits save money only if the keys respect tenant boundaries. A shared cache across BYOK tenants would also mean one tenant paying for an answer another receives.
Show tenants their own numbers
Expose a usage dashboard broken down by feature, plus alerts at the thresholds you defined. Billing disputes tend to shrink when customers can see the same ledger you do. The buyers on the other side of this will be asking whether AI usage is bundled, what happens past the allowance, and whether they can choose the provider, so publish those answers.
Metering is one piece of a wider set of platform and commercial decisions, which we cover in our full guide to white label apps. If you need a gateway like this designed around your product, our LLM integration team works on exactly this overlap between cost and architecture.
Frequently Asked Questions
What should be recorded per tenant for LLM cost tracking?
Record tenant, user, feature, task, provider, model, input and output tokens, cached tokens and whether the platform or tenant key paid. Include embeddings, tool calls, retries and background jobs, so that the total reflects the real cost.
What should happen when a tenant reaches its LLM budget?
Follow a documented path: warn the administrator as the limit nears, move degradable tasks to a smaller model and show users a clear message when a task cannot continue. Avoid unannounced failures or quiet quality drops.
When does bring-your-own-key pricing make sense?
Mainly for large clients with high volume or an existing provider relationship. It removes inference cost from your margin, but you still meter usage and enforce rate limits to protect shared infrastructure.
How does model routing reduce LLM spend in a multi-tenant app?
Routing sends simple work such as classification and short replies to smaller models and reserves large ones for tasks that need them. Keeping the routing table in configuration lets you adjust it per plan, subject to per-tenant evaluation.