
Every time you ask an AI coding assistant to "find where useAppStore is defined," it launches a multi-step expedition: glob for files, grep for patterns, read candidates one by one. For a modest 30-file React project, that's easily 5–10 tool calls and 15,000+ tokens of file content ingested — just to answer a question that has a three-word answer: src/stores/appStore.ts:23.
We built a dead-simple fix: a static JSON index of your codebase that the AI reads once instead of exploring repeatedly. Here's what happened when we iterated it down to the essentials.
AI coding tools (Claude Code, Cursor, Copilot, etc.) don't have a mental model of your project. Every session starts from zero. When they need context, they:
Glob for file patterns (**/*.ts, **/*.tsx)
Grep for symbol names across matches
Read each candidate file to confirm
Repeat for every follow-up question
For our palm reading app (28 source files, React + TypeScript + Zustand), a typical "understand the codebase" exploration consumed ~50,000 input tokens across 15–20 tool calls. That's before the assistant writes a single line of code.
We built Codebase Indexer, a lightweight Go tool that uses tree-sitter ASTs to extract three things:
dependencyGraph — who imports whom
symbols — every exported function, type, and variable with file + line number
files — file listing with language and export metadata
The AI reads this once at conversation start, then knows exactly where everything is.
codebase-indexer -root . -out .context/relationships.json
The first version indexed everything: every local variable, every function call, every destructured binding. It worked, but it was bloated. We went through five rounds of pruning:
Version | Lines | What Changed |
|---|---|---|
v1: Everything | 2,970 | Calls, locals, destructurings, duplicates — all indexed |
v2: Remove calls + dedup | 2,255 | Dropped |
v3: Filter locals | 314 | Only exported symbols, stripped all local |
v4: Fix type exports | 470 | Re-added |
v5: Full export detection | 570 |
|
The key insight at each stage: what does the AI actually need to navigate? Not local variables. Not call expressions. Not destructured const [foo, setFoo] bindings. It needs:
Where is this symbol defined? → symbols
What depends on what? → dependencyGraph
What does this file export? → files[x].exports
Everything else is noise that costs tokens without aiding navigation.
Scenario | Without Index | With Index | Savings |
|---|---|---|---|
"Find where X is defined" | ~15,000 tokens (5-10 tool calls) | ~2,000 tokens (1 read) | 87% |
"Understand the codebase" | ~50,000 tokens (15-20 tool calls) | ~4,000 tokens (1 read) | 92% |
Typical feature implementation | ~80,000 tokens exploration overhead | ~12,000 tokens | 85% |
Impact analysis ("what breaks if I change X?") | ~25,000 tokens | ~2,000 tokens | 92% |
The index file itself costs ~4,000 tokens to ingest. It pays for itself on the first question.
A developer working with an AI assistant typically runs 20–40 conversations per day. Conservatively:
Without index: 30 sessions × 50,000 token overhead = 1,500,000 tokens/day on exploration
With index: 30 sessions × 6,000 token overhead = 180,000 tokens/day
Daily savings: ~1,320,000 tokens → ~1.3M tokens/day
Saved: ~29M tokens/month
At typical API pricing (~$3/M input tokens for Claude Sonnet): ~$87/month saved
For Opus-class models (~$15/M input tokens): ~$435/month saved
Saved: ~290M tokens/month
Sonnet pricing: ~$870/month
Opus pricing: ~$4,350/month
AI inference has a real carbon footprint. The exact numbers vary by provider, hardware, and energy mix, but published research gives us reasonable estimates.
Based on Luccioni et al. (2023) and infrastructure reports from major cloud providers:
Large language model inference: ~0.3–0.5g CO2 per 1,000 tokens (varies by model size and datacenter)
Using a midpoint of 0.4g CO2 per 1,000 tokens
29M tokens saved/month × 0.4g/1,000 = 11.6 kg CO2/month
139 kg CO2/year — equivalent to driving ~560 km in an average car
290M tokens saved/month × 0.4g/1,000 = 116 kg CO2/month
1,392 kg CO2/year — equivalent to a one-way transatlantic flight (NYC → London)
11,600 kg CO2/month = 139 metric tons CO2/year
Equivalent to ~30 cars driven for a year
These are conservative estimates. They don't account for output tokens (which cost more energy), retries from failed explorations, or the compound effect of the AI making better decisions faster with good context.
You might wonder: why not use vector embeddings or retrieval-augmented generation?
Zero infrastructure. A Go binary and a JSON file. No vector database, no embedding model, no chunking strategy to tune.
Deterministic. The same codebase always produces the same index. No relevance scoring to debug.
Complete. Every export is indexed. Embedding-based retrieval can miss symbols that don't appear in semantically similar contexts.
Fast. The indexer runs in <1 second for most projects. It can run on every commit via a git hook.
Readable. The AI (and you) can read the JSON directly. Try that with a vector database.
git clone https://tangled.org/metaend.eth.xyz/codebase-indexer/
cd codebase-indexer
go build -o ~/go/bin/codebase-indexer .
Make sure ~/go/bin is in your PATH.
codebase-indexer -root . -out .context/relationships.json
Add to CLAUDE.md or AGENTS.md:
Before exploring unfamiliar parts of the codebase, consult `.context/relationships.json`
This file contains:
- **symbols**: Global symbol-to-location index (find any function/var instantly)
- **dependencyGraph**: Which files depend on which
- **files**: Per-file exports and defines
# Git hook
cp post-commit .git/hooks/ && chmod +x .git/hooks/post-commit
# Or watch mode
./watch-index.sh
The most effective optimization isn't a smarter model or a bigger context window. It's not sending tokens you don't need to send.
A 570-line JSON file, generated in under a second, eliminates 85% of the exploration overhead that AI coding assistants burn through on every session. That's real money saved, real latency reduced, and real carbon not emitted.
The best part: it took five iterations to get from "index everything" to "index only what matters." The same principle applies to the index itself — less is more, as long as it's the right less.
Codebase Indexer is open source. Supports JavaScript, TypeScript, Go, and Python via tree-sitter. Adding a new language is ~50 lines of Go.

Every time you ask an AI coding assistant to "find where useAppStore is defined," it launches a multi-step expedition: glob for files, grep for patterns, read candidates one by one. For a modest 30-file React project, that's easily 5–10 tool calls and 15,000+ tokens of file content ingested — just to answer a question that has a three-word answer: src/stores/appStore.ts:23.
We built a dead-simple fix: a static JSON index of your codebase that the AI reads once instead of exploring repeatedly. Here's what happened when we iterated it down to the essentials.
AI coding tools (Claude Code, Cursor, Copilot, etc.) don't have a mental model of your project. Every session starts from zero. When they need context, they:
Glob for file patterns (**/*.ts, **/*.tsx)
Grep for symbol names across matches
Read each candidate file to confirm
Repeat for every follow-up question
For our palm reading app (28 source files, React + TypeScript + Zustand), a typical "understand the codebase" exploration consumed ~50,000 input tokens across 15–20 tool calls. That's before the assistant writes a single line of code.
We built Codebase Indexer, a lightweight Go tool that uses tree-sitter ASTs to extract three things:
dependencyGraph — who imports whom
symbols — every exported function, type, and variable with file + line number
files — file listing with language and export metadata
The AI reads this once at conversation start, then knows exactly where everything is.
codebase-indexer -root . -out .context/relationships.json
The first version indexed everything: every local variable, every function call, every destructured binding. It worked, but it was bloated. We went through five rounds of pruning:
Version | Lines | What Changed |
|---|---|---|
v1: Everything | 2,970 | Calls, locals, destructurings, duplicates — all indexed |
v2: Remove calls + dedup | 2,255 | Dropped |
v3: Filter locals | 314 | Only exported symbols, stripped all local |
v4: Fix type exports | 470 | Re-added |
v5: Full export detection | 570 |
|
The key insight at each stage: what does the AI actually need to navigate? Not local variables. Not call expressions. Not destructured const [foo, setFoo] bindings. It needs:
Where is this symbol defined? → symbols
What depends on what? → dependencyGraph
What does this file export? → files[x].exports
Everything else is noise that costs tokens without aiding navigation.
Scenario | Without Index | With Index | Savings |
|---|---|---|---|
"Find where X is defined" | ~15,000 tokens (5-10 tool calls) | ~2,000 tokens (1 read) | 87% |
"Understand the codebase" | ~50,000 tokens (15-20 tool calls) | ~4,000 tokens (1 read) | 92% |
Typical feature implementation | ~80,000 tokens exploration overhead | ~12,000 tokens | 85% |
Impact analysis ("what breaks if I change X?") | ~25,000 tokens | ~2,000 tokens | 92% |
The index file itself costs ~4,000 tokens to ingest. It pays for itself on the first question.
A developer working with an AI assistant typically runs 20–40 conversations per day. Conservatively:
Without index: 30 sessions × 50,000 token overhead = 1,500,000 tokens/day on exploration
With index: 30 sessions × 6,000 token overhead = 180,000 tokens/day
Daily savings: ~1,320,000 tokens → ~1.3M tokens/day
Saved: ~29M tokens/month
At typical API pricing (~$3/M input tokens for Claude Sonnet): ~$87/month saved
For Opus-class models (~$15/M input tokens): ~$435/month saved
Saved: ~290M tokens/month
Sonnet pricing: ~$870/month
Opus pricing: ~$4,350/month
AI inference has a real carbon footprint. The exact numbers vary by provider, hardware, and energy mix, but published research gives us reasonable estimates.
Based on Luccioni et al. (2023) and infrastructure reports from major cloud providers:
Large language model inference: ~0.3–0.5g CO2 per 1,000 tokens (varies by model size and datacenter)
Using a midpoint of 0.4g CO2 per 1,000 tokens
29M tokens saved/month × 0.4g/1,000 = 11.6 kg CO2/month
139 kg CO2/year — equivalent to driving ~560 km in an average car
290M tokens saved/month × 0.4g/1,000 = 116 kg CO2/month
1,392 kg CO2/year — equivalent to a one-way transatlantic flight (NYC → London)
11,600 kg CO2/month = 139 metric tons CO2/year
Equivalent to ~30 cars driven for a year
These are conservative estimates. They don't account for output tokens (which cost more energy), retries from failed explorations, or the compound effect of the AI making better decisions faster with good context.
You might wonder: why not use vector embeddings or retrieval-augmented generation?
Zero infrastructure. A Go binary and a JSON file. No vector database, no embedding model, no chunking strategy to tune.
Deterministic. The same codebase always produces the same index. No relevance scoring to debug.
Complete. Every export is indexed. Embedding-based retrieval can miss symbols that don't appear in semantically similar contexts.
Fast. The indexer runs in <1 second for most projects. It can run on every commit via a git hook.
Readable. The AI (and you) can read the JSON directly. Try that with a vector database.
git clone https://tangled.org/metaend.eth.xyz/codebase-indexer/
cd codebase-indexer
go build -o ~/go/bin/codebase-indexer .
Make sure ~/go/bin is in your PATH.
codebase-indexer -root . -out .context/relationships.json
Add to CLAUDE.md or AGENTS.md:
Before exploring unfamiliar parts of the codebase, consult `.context/relationships.json`
This file contains:
- **symbols**: Global symbol-to-location index (find any function/var instantly)
- **dependencyGraph**: Which files depend on which
- **files**: Per-file exports and defines
# Git hook
cp post-commit .git/hooks/ && chmod +x .git/hooks/post-commit
# Or watch mode
./watch-index.sh
The most effective optimization isn't a smarter model or a bigger context window. It's not sending tokens you don't need to send.
A 570-line JSON file, generated in under a second, eliminates 85% of the exploration overhead that AI coding assistants burn through on every session. That's real money saved, real latency reduced, and real carbon not emitted.
The best part: it took five iterations to get from "index everything" to "index only what matters." The same principle applies to the index itself — less is more, as long as it's the right less.
Codebase Indexer is open source. Supports JavaScript, TypeScript, Go, and Python via tree-sitter. Adding a new language is ~50 lines of Go.
>400 subscribers
>400 subscribers
Share Dialog
Share Dialog
1 comment
How a 570-Line JSON File Saves 85% of AI Token Waste (and the CO2 That Goes With It) https://paragraph.com/@metaend/codebase-indexer-saving-85-percent-ai-tokens