Blurring Programs and AI:
Today's AI models are a bunch of matrices of numbers interpreted through an inference process. This is one side.
On the other side, we have machine code generated by compilers that is evaluated by hardware.
In between, we have interpreted programs (like Python or JavaScript) which is some compiled interpreter evaluating another language back into machine instructions. On the other side of the in-between we have AI agents doing "tool calling", which is inference and a harness that calls normal programs and takes the output back into inference.
What's next? What if programs and AI fuse further together?
Imagine we can detect regular repeated structures in AI models' matrices that can be short-circuited into a "precompiled" program.
Instead of relying on the model itself to introspect and decide to use a "tool" for math, what if we could track all of the weights that get activated when we do math, and replace that with a trigger in the precompile process that bypasses the weights entirely and just evaluates the math in traditional machine instructions. Sort of like a cyborg equivalent of AI (except instead of a cybernetic organism being a human-machine hybrid, it's an AI-program hybrid--what's a cool name for this? softmodel?).
Okay but you know what would be crazier? What if we could convert some subsets of inference-based models into precompiled machine instructions? Today's models are general purpose, but what if we just need an extremely narrow use of them?
The first step is to use a process like distillation or ablation to remove any weights that aren't necessary for our narrow use case. Consider a simple example: Let's say we're making a game with 500 words of dialogue and we want to take a general text-to-speech model and strip it down so that it only knows how to encode/decode what we need in our game's dialogue.
What if we develop another kind of inference process where instead of producing final tokens, it produces a kind of "bytecode" equivalent of tokens where it's still somewhat dynamic but has a lot of information pre-inferred?
I expect the next 5-10 years we'll see a lot of exploration that blurs the line between what is a program and what is an inferred model. In many ways, models are already programs that are expressed in algebra with floating points, and we know we can express Turing Complete programs this way because that's how ZK circuits work too! It's just matrices of floats with a cryptographic inference process.
We'll have programs inside of inference, and inference inside of programs, and probably different kinds of inference processes for different kinds of encodings, all mixed together.
There is a kind of "purity" in today's AI models: We throw a bunch of human content at them, close our eyes for a few weeks, and out comes a magical dump truck of numbers that we can sift through to emulate intelligence. I think it's the purity of it that is scary for a lot of people.
Perhaps once the lines thoroughly blur between AI models and programs, machines and humans, consciousness and state, then perhaps it won't be scary anymore.
