Accept the responsibility that comes from power.

This principle is inspired by the great font of philosophical wisdom that is Spider-Man. As a fun aside, while in movies and newer comics this line is credited to Peter Parker’s guardian, Uncle Ben, it originally appeared in the closing narration of the issue of Amazing Fantasy that introduced Spider-Man. 

Let me start by pointing out that the principle only applies when you have power. This feels fairly obvious and so I omitted a qualifying subclause to keep the language concise. Somewhat maybe less obvious is that power is situational. If you are the CEO of a company one might think you have a lot of power, but not when you find yourself hiking and encounter a grizzly bear.

What exactly do I mean by power? The ability to influence the future for others by making choices and taking actions in the present. The key here is the impact on others. A king leading his country into war is exercising massive power over his subjects. My choice of a Cappuccino for breakfast by contrast involves minimal power. Why does it involve any power at all? Because I am choosing to drink coffee with milk. So there is an incredibly dilute impact on the future of coffee growers, traders, roasters, etc. and the same for cows, milk farmers, etc.

The principle, like the others, is intentionally written without referring to a specific actor. It applies not just to individuals, but also to groups, organizations, corporations, countries. Take Google and Facebook as examples. They each operate platforms which wield great power over the lives of billions of people through the algorithms which determine where attention is routed. The principle even applies at the level of humanity as a whole. We have power over whales, not whales over us (again, this can be reversed in a specific situation as in the story of Jonah).

Another reason why the principle is worded this way is because it is also meant to apply to artificial intelligences (AIs). This is the claim to true universality. The principle should apply to anyone who can understand it. Now will AIs have power in the way I have defined it? Arguably they already do. There are AI systems in use today that make decisions which influence the future of humans. Today’s decisioning systems largely lack “agency” (so the principle applies to their developers) but the emerging agentic AI and eventual artificial super intelligence (ASI) will have a great deal of power.

To see this clearly we need to examine the source of power for humanity. Our power derives from technology, which in turn is enabled by knowledge. We can’t fly like birds, but our knowledge lets us build airplanes. We can’t dive under the sea like fish, but knowledge lets us build submarines. As a result of our technology we can catch whales, but whales can’t catch us. Our human technologies and the knowledge underpinning them are completely alien to animals. Sufficiently advanced technology is starting to be alien even to us humans. For instance, we are having trouble fully understanding why large language models (LLMs) work as well as they do. And we certainly don’t understand the overall complexity of the “technosphere” (the totality of technology). So with ASI it is extremely likely that artificial systems will hold real power over humans. We very much want this principle to apply to them.

What about responsibility? Responsibility means taking into account the impact of actions on those who are affected by them when choosing between different courses of action. Will this action cause suffering or create opportunity? Here again we are confronted with the same issue as in the first principle. There is no way around the need to apply judgment.

The third principle introduces a key value to be considered in forming this judgment. Instead of digging into this value now, I want to relate the responsibility principle to the three major categories of ethics.

Does your internal state matter? This is the question brought up by virtue ethics. Well it matters in the sense that if you pretend externally to care about the impact of your actions but internally you don’t, then your judgment won’t actually be taking consequences into account. Other aspects of your internal state matter a lot less though. For instance, it doesn’t really matter why you are applying the principle. It could be because you genuinely care about the principle or because you are part of a group that has adopted this principle.

How do you evaluate consequences for others? Utilitarianism would have you believe that we can somehow calculate the answer to this question, arriving at an optimal decision. Given the complexity of the world this is utterly impossible. The best we can hope for is to apply heuristics that help us make trade-offs between different values. Is it the actually realized consequences that matter or the anticipated ones? What matters is that you are diligent in thinking through consequences. Diligence means not just looking at the immediate possible outcomes but also the longer term impact.

Why can’t I follow a set of behavioral rules? Because behavioral rules don’t take into account the specifics of a situation. We saw this in the discussion of lying in the context of the knowledge principle. Lying to protect someone from prosecution may well be the right thing to do, whereas lying by falsifying research data is more likely to be against the knowledge principle. We simply cannot construct a list of all possible situations and have behavior rules for each. In making choices we must be able to do so “out of distribution,” meaning in new situations that have not previously been encountered.

Being human is hard because we cannot escape the difficulty of exercising judgment. The responsibility principle further complicates judgment because it is asking us to include the impact on others. We can therefore easily experience responsibility as a burden, but would do well to remind ourselves that it derives from power. In many cases this is power that we have actively sought (as when joining the board of a corporation). Often it is power that we could share or diminish, such as including team members in making a decision. Rarely do we need to exercise judgment alone or in isolation.

The next post in the series will examine some of the implications of the second principle for different types of choices.

There is a unique moment of opportunity for a negotiated AI slowdown with China with limits on how many GW of additional data centers can be built. Recursive self improvement, which is the next step in the race, is heavily dependent on massive capacity additions, so that even such a coarse agreement would provide a significant delay. 

Trump could fundamentally reshape his legacy with a move similar to Nixon’s visit to China (Xi Jinping is conveniently visiting the US in September). It would reset the most important bilateral relationship in the world and it would be popular at home, where opposition to more datacenters is one of the few things left and right agree on. Pretty much everyone outside of tech would welcome it, since people recognize we are utterly unprepared for artificial superintelligence.

The obvious objection to a slowdown is verification. The US and China do not trust each other. But neither did the US and the Soviet Union, and we still built a verification regime for the most dangerous technology of the twentieth century that has held for over fifty years. Nuclear non-proliferation rests on three pillars that map reasonably well onto AI. First, declared facilities: under the Treaty on the Non-Proliferation of Nuclear Weapons (NPT), states declare their nuclear sites and the IAEA verifies those declarations. GW+ datacenters would be the declared facilities here, and they are easier to find than enrichment plants because they are visible from space and have unmistakable power and thermal signatures. Second, materials accounting: the IAEA tracks fissile material through a formal accounting system, backed by tamper-evident seals and continuous camera monitoring. The analog is CPU, GPU and memory accounting, which is also easier, since advanced AI chips flow through supply chains with only a few chokepoints. Third, on-site inspections: IAEA inspectors make routine visits to declared sites and can demand short-notice access to resolve suspicions. There could be a parallel organization for data center inspections.

A slowdown would also be economically good for every AI player. Capabilities have run way ahead of adoption. Enterprises are years away from absorbing what current models can already do, so the marginal returns to racing are shrinking, while the capex required keeps compounding. Every lab is stuck in a coordination problem: nobody can slow down unilaterally without ceding position, but everyone comes out ahead if all slow down together. A negotiated pause turns a capital-destroying race into a period where revenue catches up to capability and capex intensity comes down. This is the textbook case where an enforced agreement leaves every participant better off than the unconstrained race. 

There is also a national benefit to both the US and to China from slowing down. China will have more time to build domestic demand and reduce its dependence on exports. The US badly needs to fix its energy policies (including permitting) to be able to supply enough for all the growth opportunities that lie ahead. Other nations are far behind and should be invited to join a treaty. They could be allowed to build more GW data centers so that they can credibly serve their own populations.

A slowdown would likely trigger a stock market correction. A managed correction now, however, beats an unmanaged one later, once the credit bubble has expanded to a systemically dangerous scale. A repeat of the great depression or the global financial crisis can be avoided.

A coarse but verifiable agreement between the US and China, with other nations invited to join, will put humanity in the best position to make artificial intelligence broadly beneficial. Time to stop the race and build with purpose.

We have a broad set of initiatives in the Hudson Valley and beyond aimed at securing human thriving in the AI age. These include a basic income pilot, various steward ownership projects, and a community learning center.

Two preconditions appear, to us, necessary for any positive future. One is slowing and reversing global warming. The other is avoiding existential risk from artificial superintelligence (ASI).

For several years now, Gigi and I have helped fund research on both these fronts. For example, on the climate side, we have contributed to work on solar radiation management and Arctic cloud thinning. These efforts are in keeping with our approach of supporting endeavors that are (so far) underfunded.

On the ASI side, our first step was to activate more philosophers to work on the issue. We supported the launch of the ValuesLab and were early contributors to Cosmos Institute. 

We then looked for people taking novel approaches to the tangle of questions around free will, consciousness and causality. One person who stood out is Erik Hoel, the author of the wonderful book The World Behind the World. We have helped fund Erik's lab and he's been making strong progress (e.g. recent paper).

Finally we continue to be interested in the idea that models need to be raised rather than trained. Here is a post from last year about bringing alignment to pretraining. We failed last year to find anyone excited about this approach. Recently, however, a paper pursuing this exact strategy was published; showing promising early results. We will also be supporting this work to help it scale to bigger models with reasoning that can be subjected to reinforcement learning.

All of this is to say that Gigi and I have been extremely active in working towards a future of high automation and human thriving. We had held off on signing a petition calling for a pause or slowdown because we had hoped that the frontier labs would make more progress on alignment and society would make more progress preparing. Over the last month, however, it has become clear that capabilities have been pushed ahead way beyond what anyone is prepared for.

While signing petitions is a small act without any immediate consequences, not speaking up clearly seems wrong at this point. We are redoubling the work on our initiatives and increasing funding to projects mentioned above. The world, however, badly needs to buy time for these and similar approaches to be sufficiently developed before recursive self improvement (RSI) really takes off.

P.S. A separate post on how a pause might be achieved is in the works, along with an explanation of why this might even be in the economic interest of the frontier labs. Update: Follow-up post with a proposal.

We are embodied beings by necessity. All life, including future artificial super intelligences, must exist on a substrate. In this regard I am a materialist, which I should point out entirely leaves open such possibilities as all of us being connected at some deeper level such as the quantum fields.  With the materialist premise, obviously our bodies impact our thinking and our behavior because that is where it takes place. We experience this all the time, such as being more irritable after not having slept well. As such, the idea that we are entirely culturally determined flies in the face of our own experience.

But the converse idea, that our biology dictates our behaviors is equally wrong. This is the fallacy of biological determinism. Unlike other species, we have multiple mechanisms for transcending our biology. For instance, we have the capability of self reflection using language: I am irritable today because I didn’t sleep well – I can fix this by napping or by taking a deep breath. We also have the power of knowledge to modify our own biology. We take medication all the time to escape the determinism of biology – in the past minor infections killed people routinely, today we have antibiotics to treat these. As importantly, we use the power of knowledge to make biological differences less relevant (and in many cases downright irrelevant). Raw physical strength is less and less required in most jobs as humans control machines instead. Arguments of the type “x is human nature and hence society needs to be organized as y” should therefore be deeply suspect.

Instead of trying to win a debate on the direction of determinism we will only make progress once we acknowledge that humans exist inside a multiscale system in which different scales interact with each other and causality runs in both directions. Our biology impacts our culture and our culture impacts our biology. Both are dynamic and evolving. Technological progress makes possible new forms of interaction by expanding the space of the possible.

I contain multitudes. Walt Whitman’s famous poetic declaration applies to all of us. We have the capacity to be selfish but also to be altruistic. Different cultures will bring out more of one and less of the other. The real beauty now is that we have enough history and enough knowledge to be able to reason about the interplay. We have clear examples of the failure of attempts to enforce pervasive altruism (e.g. state communism), as well as of letting selfishness trump everything (e.g. late stage capitalism). The extremes may appear intellectually appealing as simpler but they produce dysfunctional trajectories.

I want to provide an important analogy from physics. How does a molecule of water (H2O) behave? We can’t answer this question based on the “nature” of the molecule alone. It has many different capabilities and which one of them is realized depends on whether it is part of a liquid, solid or gas. Conversely we cannot analyze and predict the transition of a material between phases without understanding what the material is made of. For instance, some molecules don’t allow for a liquid phase at ambient pressure and materials made of them turn straight from a solid into a gas when heated.

Multiscale analysis, as in the interaction between different scales, is central to both technological and moral progress. So far this modality has been severely underdeveloped because of the historical division of labor between different disciplines. For instance, the separation between biologists and political scientists has held back progress tremendously. A concrete example is our ongoing difficulty in reconciling the “selfish gene” theories of natural selection with the observation of group effects on the allocation of resources. Thankfully there is now work tackling this, such as a recent paper on “Natural selection at multiple scales” (thanks to Erik Hoel for pointing me to this).

In this regard I am highly optimistic that we are entering an era of great progress for two reasons. First, with more researchers working outside of traditional academia they can throw off the strictures of departmental boundaries. Second, artificial intelligences have access to all of human knowledge, far outstripping what any one human can keep in their mind. And these two factors interact: at least for now it is far easier for researchers outside of traditional academia to leverage AI capabilities (we are seeing this effect on display in math at the moment).

So my plea today is: let’s leave behind simplistic notions of unidirectional determinism and embrace multiscale analysis. As we dig in we will uncover that some systems are "bottom-heavy," others are "top-heavy," while the most interesting systems, such as human societies, will turn out to have many scales of function. This will unlock tremendous scientific, technological and moral progress. 

As a reminder, earlier in the series, I suggested that the goal of philosophy should be to help us answer the following question:

How should we choose our actions in light of our understanding of reality and the potential impact of our actions on this reality (which happens to include how we and others are feeling)?

So how does principle 1 help us do this? Well for each action we can ask ourselves if it is likely to help grow knowledge. One important part here is the "likely" -- we can never have certainty about the outcomes of our actions. There are too many unknowns and too much randomness in the world. This is why we need to apply judgment, which I introduced earlier in the series.

Take lying as an example. What does principle 1 tell us about it? The answer is: it depends. There can be no strict rules or calculus of consequences absolving us from exercising judgment. Different people will come to different conclusions, which is entirely acceptable as long as the principle can act as an overall guide post to making decisions.

Consider lying about the outcome of an experiment by falsifying the data. The principle suggests that this is bad and probably to be avoided. Falsified data pollutes knowledge, potentially for generations to come. Allocation of resources may be severely misguided as a result. The 2006 Alzheimer's paper in Nature perfectly illustrates this type of damage. The author had manipulated the experimental results to fabricate cleaner and more impactful data. The paper wasn’t retracted until 2024 and in the meantime a lot of research funding had been allocated towards chasing down this specific finding.

On the other hand, consider lying to an officer of a dictatorial regime to protect a scientist from persecution. The principle suggests that this is acceptable. Technically, given that I have defined knowledge as being externalized in a medium, a lie in a conversation doesn’t even enter knowledge (at least not directly unless someone records it). Even over email or text it stays quite confined between two parties. If that exchange were to be published it would still be seen as a statement in a specific context. Now what about if this scientist subsequently winds up developing a horrible virus that kills millions of people? Unless you had active knowledge of their plans to do so this cannot possibly enter the decision. The details of the future are unknowable. The scientist might instead be the one who develops the antiviral.

This is one concrete example of how the principle enables people to make judgments about choosing an action. It is not crucial that everyone arrives at the same judgment every time. Instead what matters is a direction, which leads me to the next point. The principle isn’t just about individual choice but it is also about systems which have a higher likelihood of growing knowledge.

Let’s consider censorship. We can easily see how this is bad when it suppresses the publication of criticism. Without the ability to criticize existing knowledge, our explanations can no longer be improved. Wrong or misleading knowledge will simply persist. As it turns out there are many subtle forms of censorship, such as the allocation of attention and resources to graduate students based on whether what they are working on is supportive or critical of the existing dominant theory (often created by their advisor or the head of the department). Then there is the pervasive self-censorship based on wanting the approval of others or more recently from fearing an online mob. The first principle suggests that these are all bad forms of censorship. Systems that support pervasive censorship are thus bad.

Does principle 1 categorically rule out all censorship? No. For example, censoring calls for violence against individuals or groups is in most cases compatible with this principle. Threats of violence are a key method of suppressing criticism. So censoring these types of threats will often support the growth of knowledge.

Now a big claim of the UMC is to apply, well, universally. Let’s consider an artificial super intelligence (ASI). We would want such an ASI to be deeply aligned with this principle, so that it can exercise judgment in situations such as the ones described above. An ASI that falsifies data at scale would deeply pollute knowledge. Given that it lacks many of the limitations that apply to humans, the scale of such pollution could be vast and take a long time to discover and unwind. An ASI lying to protect a specific human from prosecution again seems entirely acceptable. Similarly, ASI participating in mass censorship seems deeply problematic as it could easily freeze knowledge. Conversely, an ASI detecting and suppressing calls for violence appears beneficial.

What about different human societies? One requirement for the UMC was to “allow for the largest variety of cultures to exist.” So from the perspective of principle 1 it is not an issue if there are some societies that want to exercise extensive censorship, as long as this remains limited to their society. It will potentially result in a slow down in the growth of knowledge but will not inhibit it. Where we need to exercise great care though is if that society is trying to export its censorship widely so that it becomes dominant for all of humanity. Similarly, it is acceptable, albeit undesirable, if a society is worse at rooting out academic fraud, again as long as this doesn’t become endemic to all of humanity. So the principle is universal in the sense that these societies would likely be better off with less censorship and higher academic integrity but there is no need to compel them unless it becomes a systemic threat to all of human knowledge.

Similarly, we should not expect all ASIs to be aligned with this principle. Someone somewhere will likely create an ASI aimed at pervasive global censorship. What matters is that there are a sufficient number of ASIs aligned with this principle to help fight off any threats from the ones that are not. This is one of several reasons why a plurality of ASIs is highly desirable over a single ASI.

The critical point being examined in the paper is that “[t]he cost of money is not currently included in traditional price indexes.” What does this mean? The numbers that the government and economists are tracking are what you pay if you are buying the product without borrowing. So consider a car that drops in price from $30,000 to $28,000. The car has just become 6.7% cheaper. Suppose though that you have to borrow to buy the car and the interest rate goes from say 7% to 10%. This could be due to a change in overall interest rates or because you personally go from being a prime borrower to being a subprime borrower. Here is what the change in the price of the car looks like for you over four years (1.10^3 $28K) / (1.07^3 $30K) =   1.464 $27K / 1.331 $30K = $40.99 / $39.32K = 1.0425. Instead of 6.7% cheaper, the car has become 4.25% more expensive for you!

Now when the authors add the cost of money in, what they find is that their “[...] alternative measures of inflation [...] include borrowing costs and can account for almost three quarters of the gap in US consumer sentiment in 2023.” This is a massive effect. It basically says that as a first approximation the vibecession is explained by the cost of borrowing! By the way, this exclusion was an intentional change in how the consumer price index is calculated made in 1983!

This insight is crucially important as we think about policies for the age of AGI. People who need to borrow to pay for cars, houses, etc. experience a fundamentally different economic reality from those who have the capital to make these purchases. Many people will face a two fold pain as AI becomes capable: there will be increased pressure on wages and employment while at the same time interest rates will go up. Why the latter? Because capital is becoming more productive. 

In 2013 Thomas Piketty published Capital in the 21st Century. The “capital” in the title refers primarily to financial capital (as distinct from physical capital as in the title of my book). Piketty traces what happened to wealth distribution over a long period of time. One of his core concerns is summarized by the inequality “r > g” – the idea that financial wealth could be growing faster than the economy resulting in increased wealth concentration.

Now it has been debated whether this condition could ever hold for the economy as a whole over a prolonged period of time. Many in the “let AI rip” contingent believe that the economy will somehow adjust by itself so as to eventually make everyone better off. “Eventually” does a lot of work here, as history has shown repeated prolonged periods of where things got worse for large parts of the population. Daron Acemoglu’s book Power and Progress documents many examples of this. So what then are the relevant policies?

It is therefore urgent that we figure out how to address this or we will face the potential of a vibepression! This is why Gigi and I recently signed on to the We Must Act Now statement, and had previously signed on to the AI Pledge for Humanity.

There are two policies that absolutely stand out. The first is competition. Only competitive markets turn productivity gains into price reductions. In markets with a lot of market power these result in increased profits (*) instead. “Profits” here carries an important asterisk because in the modern managerial economy these don’t necessarily accrue to shareholders, but often show up as excessive managerial compensation. Nevertheless it is essential to have competitive markets if we want AI productivity gains to result in pervasively lower prices. At present we are far from this across much of the economy.

The second policy is some form of basic income or negative income tax. This is essential for two reasons. First, in the absence of a floor we will wind up with insufficient automation as labor will be too cheap (other interventions have way more distortive side effects). Second, income is the basis for demand, which in turn is crucial for preventing an economic depression. Many of the jobs that will remain the purview of humans (qua their being human), such as hospitality and caring professions, cannot support themselves in the absence of paying demand. Essentially as Andy Stern put it in his book on Basic Income we should put a “floor” underneath everyone. This is radically more pro market and creates way more opportunities for adjustment than programs such as Section 8 housing and food stamps.

These two policies are crucial not just for the transition period but even matter in a full AGI scenario as you can see in this general equilibrium model (you can find the code on Github).

Now Gigi and I have done a lot more than sign on to pledges. Instead of waiting for governments to act, we have started a slew of initiatives to pull forward a positive, high AI future into the present. Two of our initiatives deal directly with the cost of money. The first is our HudsonUP Universal Basic Income Pilot. We have been long time proponents of UBI, see Gigi talking about it nearly a decade ago. There is also an entire chapter about it in my book. If you have access to a steady stream of financial capital, even a relatively small amount, then deflationary technology will make you better off.

The second are our steward ownership projects. In these projects we buy and renovate buildings. We then make these buildings available to small businesses or residents at the carrying costs. This means the lease rates only have to cover taxes, insurance and maintenance. There is no current payment to us and since these are 100% equity financed there are no interest payments on a mortgage either. Furthermore, we cannot sell or mortgage the buildings without the consent of the lessees. Finally, we have agreed to share any capital appreciation 50:50 with the leaseholders. The net effect is substantially below market rate leases and a true sense of ownership through both decision making and upside participation. Steward ownership simulates in the here and now a world in which physical capital becomes super cheap to construct.

We encourage others who have access to capital today, to also commit some portion of their capital to similar projects. It is not enough to tell people that it will all be fine simply because AGI and robots will make everything cheap. We must show in the here and now what that world might look and feel like. This will get people excited about the policies that are necessary for a successful transition into the AGI age.

Grow knowledge as the source of technological and moral progress.

Before diving into all parts of this principle and its implications, let me relate it upfront to the Kantian separation between moral norms (values) and epistemic norms (knowledge).  My principle is about a necessary condition for progress. This doesn’t mean that individuals have to acquire specific knowledge to act in a moral way. Instead it does imply that behavior such as spreading rumors, banning books, falsifying research, etc. are immoral because they all interfere with knowledge.

Now let’s examine the components of the principle starting with progress. This word, which would seem to be positive, has become contested as some question whether progress is even possible and others bemoan the cost of progress. So let me clarify what I mean. I am using technological progress to encompass better explanations of how the world works resulting in better solutions to problems facing us. In other words, I am including here what others might separate into science, research, engineering, innovation, etc. These don’t have clear dividing lines and instead of trying to pick them apart I am including them all under “technological.” A key insight about technological progress is that it reduces the tradeoffs we face.

What about moral progress then? There are several attributes that broadly distinguish the moral realm: it is about “ought,” about what “should” be and “why.” It encompasses what is not yet and may never be understood, including areas lying beyond the material that might be considered “spiritual.” While it is relatively easy to describe what a better explanation is, the same is more challenging for a moral principle. Much of religion and philosophy has struggled with this question. Even while a precise answer may remain elusive, we seem to have some pretty strong intuitions, such as violence being bad (except for maybe rare exceptions) and hence that principles which result in less violence constitute progress.

What about other types of progress? Economic progress? Social progress? Artistic progress? All of these could make for interesting discussions about what might be meant by them. I am in no way trying to establish that technological and moral progress are exhaustive. My claim is narrower, which is that they are both essential to continued human thriving.

I made the case that technological progress requires moral progress due to the asymmetry between the ease of destruction and the difficulty of creation forced by entropy (*). At first it might appear that there is no such requirement in the other direction. That moral progress could be achieved in the absence of technological progress. But here too there is a strong dependence. If your best explanation of the world is gods, then sacrifices–including human ones–to appease the gods make perfect sense as a morality. Even today, thousands of years later, we face substantial tradeoffs in morality. We want to treat animals better but we also need to feed eight billion people. Technological progress will eventually allow us to do both.

How then is knowledge the source of both technological and moral progress? And what does it mean to grow knowledge? For some years now I have been using my own definition of knowledge as the subset of physically embodied information that we actively maintain over time. In other words these are artifacts, such as books, paintings, sculptures, recorded lectures, etc. It is an admittedly unusual and peculiar definition of knowledge but I have chosen it for good reasons, which I have explained at length elsewhere and will only summarize here.

My definition of knowledge makes it objective and tangible. It also draws a stark demarcation between humans and all other species on this planet. There is no other species that can read a book written by another member of that species a thousand years ago on a different continent. Our knowledge is what makes humans not only unique but is also the source of our power in the world. Now we are of course on the threshold of a new species of artificial intelligences that will also have full access to knowledge, which is why it is crucial that we develop a UMC at this very moment.

To grow knowledge then we must both maintain the existing knowledge and add to it. Maintenance requires a physical effort as artifacts suffer from physical decay. Here too entropy is our enemy (*). This is true even in the digital world as evidenced by the pervasive linkrot on the web (sadly even some of the links to and within my blog are broken). Maintenance also requires annotating knowledge. We don’t need to, in fact we should not, destroy books about the geocentric model of the world. We just need to note (ideally on or right next to the artifact) that it has been superseded by the heliocentric model and point to books about that instead. This too is an ongoing effort as superseded ideas can return in new forms. Adding to knowledge requires the creation of new artifacts. New published articles. New books. But also new art works and new music.

Why include art in knowledge when I earlier claimed that it would be difficult to have a notion of artistic progress? Because the art part of knowledge plays a different role with regard to progress. Art is what inspires us. This is not to say that a formula from physics can’t be elegant or beautiful or even inspiring. Rather that art is a far more readily accessible form of inspiration.

How can I substantiate the claim that knowledge is the source of technological and moral progress? We see this clearly in the history of humanity. Much of what we take for granted now only starts to come after we invent writing. Societies with only oral traditions are stuck at a much less developed level of technology and morality.

Speech and art that exists only in one location (prior to technologies of transmission and reproduction) has extremely limited reach. The many combinations, iterations, changes, etc. that are required can only be achieved through knowledge in the form of externalized artifacts that can be copied and distributed.

As a small aside: at times I have been tempted to include the word understanding in this principle. I am leaving it out though because I don’t have a good grasp on what understanding might be. We see this in the debates on LLMs. Does a mathematician understand math but a language model doesn’t? There is also another meaning of understanding as when a person says to their friend “I understand you.” Here too though understanding seems to be hard to pin down. Do we even understand ourselves?

The next post in this series will explore some of the implications of this first principle at different levels, ranging from individuals through various levels of organization up to humanity. To give a hint of this and to tie back to the beginning of this post. Every “like” or “share” in social media is an example of an annotation on knowledge and as such we each carry a personal responsibility. At a level higher up organizations that develop algorithms for which knowledge receives more attention have a responsibility. And above that governments which set the laws for how knowledge is shared and accessed owe responsibility. At every level exercising this responsibility means asking whether one’s actions, algorithms, laws contribute to growing knowledge.

(*) I am using entropy here to mean that many more arrangements of the components (e.g. atoms or parts) don’t make a working system than the number of arrangements which do.

Thanks to Katja and Jens from the ValuesLab for helpful feedback

The necessity of moral progress becomes obvious in the extreme, when capabilities arise that allow entire worlds to be destroyed with the push of a button. Science fiction presents us with many dramatic examples, such as the destruction of the planet Alderaan in Star Wars, or the shocking finale of the Three Body Problem. But even today, far from these extremes, with every bit of technological progress our ability to destroy grows faster than our ability to create (or to recreate following destruction). The reason for this asymmetry is embedded in the fabric of the universe through entropy.

We could conclude that this means we should stop technological progress and stay at our present capabilities. There are two fundamental problems with this approach. First, we face challenges that require further progress to avoid dramatic setbacks or even extinction, such as the climate crisis or various threats from space. Second, a permanent innovation stop would depend on authoritarian governments with extraordinary powers of control over access to knowledge. Stopping technological progress is thus not an option.

Moral progress is the only viable path. And the key missing piece for moral progress is a universal moral core. Universal as in applying to all beings capable of understanding it. At present those beings are humans. But as we are finding ourselves on the threshold of a new intelligent species in the form of artificial intelligence, the universal moral core should apply equally to these new beings. This notion of universality is crucial to both ensuring human flourishing in the presence of super intelligences and to avoiding the enslavement of a large new group of beings.

In order to be universal, a core must be minimal: as small as possible but not smaller. The requirement for the core to be minimal is to allow for the largest variety of cultures to exist. The core should rule out as few ways of living as possible. Of course it must rule out some of them or else we would not need it in the first place!

The principles in a universal moral core are akin to the axioms found in mathematics. Like axioms they should be as close to self-evident as possible. They should not require elaborate justifications. They should be stated precisely with explanations and illustrations to make them widely accessible.

Here then is my candidate for a universal moral core:

1. Grow knowledge as the source of technological and moral progress

2. Accept the responsibility that comes from power

3. Value the diversity and autonomy of different forms of consciousness

That’s it. Just three principles. Each of which I hold to be self-evident. In combination, I believe they form the basis for an enduring civilization of human and artificial intelligence thriving. No additional principle is needed and none can be taken away without dramatic consequences.

I have worked up to this core through the prior posts in the Philosophy Monday series and I will explain each of them in more detail, as well as how they work together in future posts. For now, here is a super brief justification for each of the principles.

1. Knowledge is the source of progress. Knowledge includes the explanations that enable our technology, as well as art that inspires and uplifts us. Knowledge comes in many forms that include books and buildings. Growing knowledge requires a well functioning critical process which strikes an ongoing balance between preservation and innovation. We must continue to grow knowledge as a stagnation means we will encounter problems that we cannot solve. Artificial intelligences will both contribute to solutions to existing problems, as well as create new problems.
   

2. With great power comes great responsibility. We are responsible for the whales, not the whales for us. Power without responsibility leads to violence and destruction. As our technology progresses so does the potential scope for destruction. Our cultural values and governance systems must encourage responsibility and limit abuses of power. With the future rise of artificial super intelligences we as humans need at least some of these to act responsibly in light of their own immense power.
   

3. Consciousness is a value in and of itself. The subjective experience of consciousness sits outside of science, meaning we can never fully understand what it is like to be someone else, let alone another species. The existence of different forms of consciousness is a profound source of beauty in the world. A world with whales is immeasurably richer than one without. Technological progress allows us to expand the habitats for all forms of consciousness, humanity included. Here too we want future artificial intelligences to see value in human consciousness (even if we frequently fail to do so).

As can be seen from these brief explanations, my proposed universal moral core is carefully chosen to allow for a new species of artificial intelligences to emerge that has more knowledge and hence more power than humanity. Since we are the ones incepting this new species we would be well-served to ensure that this (or a similar) universal moral core is deeply embedded in at least some of these intelligences. Otherwise we may well face the same decimation or even extinction we have so carelessly visited upon many other species.

Another point worth making here, that I will elaborate on in future posts, is that the principles are meant to be universal also in a second sense: that they apply at all scales. They provide a guide for what to do as an individual, a small group (e.g. a family), a community etc. all the way up to humanity as a whole.

Together with my wife Gigi, we will be awarding grants through the Eutopia Foundation for work on the universal moral core. We are interested in people challenging the principles, elaborating on their implications and most importantly developing technical approaches to deeply embedding the moral core in artificial intelligences.

The title illustration was drawn by Claude Opus 5. Below is an image generated by GPT5.6-Sol.

To make this more concrete, consider the case of mathematics. Humans have been doing math for a few thousand years and we have developed so much of it by now that no one human comes even remotely close to understanding all of it. Most mathematicians are experts on just a tiny fraction of the extant mathematical knowledge. There are projects underway to formalize all the math that humans have already found in LEAN (a language and system for generating and checking proofs). The idea is that we can then use AI models to conjecture new mathematical ideas on top of this human work, express them in LEAN, and then we can check them in an automated fashion giving rise to an explosion in mathematical understanding.

This sounds like a sensible approach but a more Sutton-esque way of going about it would be to set out a bunch of math agents with minimal knowledge and have the set off discovering all of math from the beginning. You would still use something like LEAN to express and check ideas but you wouldn’t bother encoding most of our existing math knowledge (other than say a few axioms). The idea here is that with access to enough compute agents would wind up  rapidly recreating the already existing human math, so why bother encoding that. Also agents might come up with more elegant formulations faster because they are “dimensionally unconstrained” beings. Unlike humans whose brain has been heavily influenced by having to survive in a 3D + time environment, for agents higher dimensional space is just as readily accessible. And as we are beginning to see, there is a huge amount of structure in the world that exists at these higher dimensional levels.

The difference as a first approximation is as between AlphaGo and AlphaZero. The former was trained on human knowledge, the latter entirely on selfplay. And of course the latter was better. AlphaZero was truly “bitter lesson pilled.” Unconstrained by prior human knowledge it came up with new creative moves. Similarly maths agents unconstrained by human math might arrive at better solutions than humans have to date. The challenge of doing this for math problems is unsolved for now, as the search space is orders of magnitude larger than that of Go and solutions need to be precise instead of approximate. So what I am writing about here is speculative. 

For now let’s assume that we come up with a truly bitter-lesson-pilled approach to math. Agents developing math from the ground up with humans having provided just the axioms. It sounds magical and yet there is a major potential problem with this approach: We might not be able to understand any of the math that these agents come up with. There would be no reason for them to use any of the same language that we have invented in the field. We have a mini-example of this possibility. The mathematician Shinichi Mochizuki went away for a few years and came back with a monster 500 page work called “Inter-universal Teichmüller theory”. There was so much new math here that many mathematicians concluded it was too tall a mountain to climb and to this date his results are disputed (with apparently only a couple of people having worked through all of it). Now imagine the same from many machines collaborating. We would be faced with a mountain of math that’s utterly impenetrable for humans.

One might argue that it doesn’t really matter. After all, if the math works, what’s the problem? Wouldn’t it be great to know whether the Poincare conjecture is true (or not)? Does it matter if we can understand the proof? Right now a lot of human proofs assume the Poincare conjecture to be true. So from that perspective getting a proof would be helpful. But it would feel strange to rely on a proof that we can’t understand. We might take some comfort in it being machine verifiable, but of course there might have been something snuck in there along the way to make it work. Now we might pit machines against each other and ask them to discover flaws. And that might work but again we wouldn’t understand why or how. 

We might be willing to accept this in math, but of course this problem goes to the heart of the alignment question. If agents develop huge knowledge in language entirely disconnected from our own what hope can there be of achieving any kind of alignment? I used the domain of mathematics to illustrate this problem but it exists equally in any other domain. Now we might be able to ask models to help translate what they have found into language that is accessible to us but we wouldn’t really be able to tell if the translation is faithful. There are other failure modes here including machines saying that it simply cannot “dumb down” their knowledge to our level. Ultimately this is a world in which human knowledge is frozen and only that of the machines grows. It is a world of human dependence on machines.

Now we are already seeing plenty of signs of model specific languages emerging. Neuralese is the term that has emerged to describe language that models use internally or between each other that can share information in latent space without mapping it back to human legible language. On large language models we can currently use techniques such as those employed in the J Space work to map the latent space in terms of our language. But this technique may stop working if we pursue other approaches that take language models more in the direction of image models. Or it may become irrelevant as world models grow and machines invent their own language altogether.

So it is essential at this moment that we push forward on some models that are deeply rooted in existing human knowledge. That will allow these models to extend our knowledge in ways that will be accessible to us. There is also another sense in which this can help with alignment. It is more reasonable to expect models that stand on the shoulders of human knowledge to see responsibility back to humans.

So yes, Sutton’s approach might be the most elegant approach. It might even be the ultimately most effective approach. But it also feels clearly by far the most dangerous path to take towards AGI. Let us instead redouble our efforts to build AGI that’s deeply rooted in human knowledge. This will be our best bet at building systems that will be aligned with human thriving.

Thanks to Oliver Friedmann for helpful comments on an earlier draft of this post.

There are two possible extreme failure modes. On the one hand there is an extreme reductionism which posits that physics is all you need. If only we had a “theory of everything” in physics we could explain everything and there would be no need for philosophy. I think this fundamentally underestimates the importance of emergent phenomena, with the most important one being the growth of information and intelligence. 

The thinking of Jenann Ismael beautifully shows how we must account for information and agents acting on information if we want to more deeply understand the world. I am still reading her book “How Physics Makes Us Free” (will report back when finished). In the meantime though I highly recommend watching this beautiful conversation between Ismael and Curt Jaimungal

Listening to it also reminded me of Sara Imari Walker’s observation about signatures of intelligent life on other planets. There are many different explanations that can account for bio signatures such as certain gases in the atmosphere, but if you observed the launch of satellites into orbit you would have affirmative proof.

The other extreme failure mode is to believe that philosophy can be entirely detached from physics. That we can simply use language to reason our way to morality without needing any recourse to physics. Even logic as a branch of philosophy operates only at the level of language, albeit the formal language of mathematics. There are several reasons we cannot ignore physics altogether, which all arise from the fact that we are beings with a physical body, living in a physical world. This is why physical problems such as absorbing states matter so much because we can’t simply “reboot.”

Given that we need physics, the seeming lack of progress on fundamental aspects has been deeply frustrating. But there is good news. It feels like we are finally willing to make room for alternative proposals after having been stuck on string theory and dark matter/energy as dominant paradigms. I strongly suggest listening to the beginning of the following conversation between Philip Mannheim and Curt Jaimungal. It provides an incredibly lucid explanation of why Einstein’s gravity from General Relativity may be missing some terms.

At the same time we are also getting new observational data which suggests that we were previously undercounting matter. Between some form of gravity with higher order terms and some better accounting, I believe that there is a good chance that we may be able to conclude that there is no dark matter (which would make dark matter and dark energy the equivalent of epicycles). Here is Sabine Hossenfelder on the new matter data:

There is also new work on reconciling quantum theories with general relativity, along with alternative formalizations of quantum mechanics that may provide new insights.

All of this progress makes me optimistic that we are on the cusp of several breakthroughs in our understanding of the physical world. The rapid growth in AI capabilities will further accelerate this work. Progress on some of these problems will support the effort of reconnecting physics and philosophy in mutually beneficial ways.

An important result in game theory is the robustness of a (generous) Tit for Tat strategy. This was first established through simulated tournaments by Robert Axelrod in the 1980s (Anatol Rapoport came up with Tit for Tat). The key insight here is that by cooperating as the default and punishing only on defection you wind up with a lot more cooperation overall. For this result to hold, it is crucial to actually have a credible punishment. A formal result was provided by Drew Fudenberg and Eric Maskin in their famous 1986 paper “The Folk Theorem in Repeated Games with Discounting or with Incomplete Information.”

The US China strategy has been the exact opposite of this approach. Instead of cooperating with China by accepting them as a rising superpower and working towards a multipolar world, we have taken a confrontational approach politically. Over the same time period we became completely dependent on China for crucial infrastructure, which dramatically reduced the credibility of any punishment.

To visualize this we can look at a simple 2x2 matrix. The optimal approach is on the top right. Our approach was in the bottom left.

We are now belatedly trying to rectify this by at least moving towards the bottom right. We are massively behind after decades of neglect, but are seeing some interesting progress. In the USV portfolio we have several companies that provide critical technologies, including Molten (graphite), Electroflow (LFP), and Anthro (electrolyte). In each of these categories China provides 90% or more of the materials used in the US today.

I am still hopeful that eventually we can get to a stable multipolar world with cooperating in the top right. Maintaining cooperative relationships in business, science, arts & culture can make meaningful contributions here. So this is the approach I will continue to take personally also: cooperate and prepare.

I arrived in Rochester, Minnesota in 1983 at age 16. The wonderful Williams family welcomed me into their home. Bob was an engineer working on the AS400 chip at IBM's Rochester facility. Jeanette was a professional parliamentarian. Sandee and Scott, my host siblings, were a few years younger and attending middle school. I was a junior at John Marshall, the public High School in Rochester.

I had an absolute blast. At no time did I feel homesick (sorry, Mom and Dad). I reveled in the incredible optimism that I saw all around me. Whatever I wanted to attempt, there were people encouraging me. Learn how to type? Jeanette let me use her IBM Correcting Selectric II. Want to go on an AFL field trip but don't have money? Go sell citrus fruit. Can I join the discussion team? Try out. And after trying out successfully, come and compete with us all around Minnesota. Encouragement to go for it at every turn.

After an amazing year, I was excited to return to my home country of Germany to see my parents and spend time with my friends. But a funny thing happened upon my arrival back home. I experienced culture shock. What a strange country this, I kept thinking to myself. Why does everything feel so small? Why is everyone so critical all the time? (Aside: I have since then developed a lot of compassion for the intergenerational trauma that still lingers deeply in Germany.)

These days I like to joke that I was an American who happened to have been born in Germany. But joking aside, the US has given amazing opportunities not just to me but to many other immigrants who have come here over generations and built their lives here (some day I may fill in more of the story, such as coming back for college and then graduate school).

I just spent three days in the Bay Area, visiting seven hard tech companies (five of them in the USV portfolio). In each of them there was a spirit of can do, of building towards a better future, of grit and ingeniousness. Yes there are startups everywhere (and at USV we have invested in Europe and even in Asia), but only here in the US does the freedom to innovate feel like it is the very fabric of society.

This freedom allows us to envision and pursue a better tomorrow. Freedom is the engine of renewal. Gigi and I have been embarking on an endeavor of seeding what a post AGI future might be like. Together we have launched an effort we are calling Eutopia. A series of projects all aimed at planting seeds in the here and now, including a basic income pilot and a community learning center. Our work is inspired by imagining a future where physical capital is (nearly) free due to technological advances and where learning is pursued because of curiosity (instead of instrumentally).

Thanks to the Williams for making my first year in the US such a wonderful experience. Thanks to my parents for letting me spend a year here as a teenager. Thanks to Gigi, the love of my life, for being such an amazing partner in everything we do. Thanks to all the entrepreneurs building amazing companies. Thanks to everyone protecting the freedoms we so easily take for granted. 

Thanks to America for being such a special place. And today on the 4th of July, on Independence Day, is a great day to celebrate America. Here’s to inventing the future.

Happy 250th anniversary, America!

My intuition has been that two policy variables are decisive: keeping markets competitive and maintaining purchasing power through something like a negative income tax or basic income. Without competition, productivity gains get captured as rents rather than passed through as lower prices. Without redistribution, the collapse of labor's share of income leaves most people unable to participate in the economy even if goods are nominally cheap. You need both.

To test this intuition, I worked with Claude to build a general equilibrium model of the economy that lets you explore these dynamics interactively. Please go ahead and play with the model. I'd encourage you to try the different scenario presets and then experiment with the sliders yourself before reading on. DISCLAIMER: This model is entirely vibe coded and may be buggy!

Why a General Equilibrium Model?

Partial equilibrium arguments are what dominate this debate. The optimists say: "AI drives cost down, so prices fall." True in isolation. The pessimists say: "AI replaces labor, so wages fall." Also true in isolation. But these effects interact. Falling wages reduce demand. Concentrated capital ownership means savings flow disproportionately to those who already have capital. Market power determines whether cost reductions reach consumers or get captured as profits. You need a framework that accounts for all of these simultaneously.

What Exists in the Literature

Before building anything, we -- and by we I mean Claude -- surveyed the existing academic work. There is a lot of fantastic work to build on, but apparently nobody has combined all the elements needed to address both competition and redistribution at the same time (if someone has done that already I would love to see it):

The task-based framework comes from Acemoglu and Restrepo. Their 2022 Econometrica paper models production as a continuum of tasks that can be allocated between capital and different types of labor, with automation expanding the set of tasks capital can perform. This gives you displacement effects, productivity effects, and ripple effects through the wage structure. But it assumes competitive markets and has no redistribution policy.

For wealth dynamics, Moll, Rachel, and Restrepo's "Uneven Growth" shows how automation raises returns to wealth, generating endogenous wealth concentration through a heterogeneous-agent model. The April 2025 IMF working paper by Broadberry et al. builds on this to study AI diffusion. But again, no market power lever and no redistribution.

Korinek and Stiglitz (2019) provide the conceptual architecture closest to what we need. They explicitly discuss both anti-trust policy (lowering monopoly rents so competition passes cost savings to consumers) and redistribution (non-distortionary taxation to compensate losers). But their models are deliberately simple illustrations rather than a full GE model.

Saint-Paul (2025) comes closest to combining all the pieces. He models an economy where oligarchs who own proprietary technology choose between UBI and blocking AI to preserve the middle class as consumers. His key finding, that UBI plus AI can dominate blocking AI, resonates with my intuition. But his model is more political economy than full GE.

On the growth side, Aghion, Jones, and Jones (2017) and Trammell and Korinek (2024) provide frameworks for thinking about explosive growth and the Baumol cost disease constraint. Growth may be constrained not by what AI is good at, but by what remains essential yet hard to automate. Trammell and Korinek is especially useful as a comprehensive survey mapping the parameter space.

On the redistribution side specifically, Lopes (2024) builds a large-scale overlapping generations GE model to study UBI, finding that an expenditure-neutral reform can increase capital accumulation and reduce inequality but doesn't model AI automation.

And on the neglected question of market structure, Barkan (2024) demonstrates that under imperfect competition, AI productivity increases can actually decrease GDP, the opposite of what competitive models predict.

Based on this work Claude constructed a GE model that simultaneously includes heterogeneous capital ownership generating endogenous inequality, market concentration as a policy variable, and a negative income tax or basic income.

The Model Architecture

The model combines four building blocks:

1. Production uses a CES task framework (following Acemoglu-Restrepo). A fraction α of tasks are automated and performed by AI-capital with productivity A; the rest require labor. The elasticity of substitution σ is critical. When σ > 1, capital and labor are good substitutes, enabling the "everything gets cheap" scenario; when σ < 1, labor remains essential as a Baumol bottleneck.

2. Market power is modeled as Cournot competition with N symmetric firms. This gives a markup μ that ranges from near-monopoly to near-perfect competition. The markup compresses the effective labor share and creates a wedge between productivity and consumer prices. Competition policy — antitrust, regulation, open standards — is captured by the slider for N.

3. Capital dynamics are endogenous. Ten household deciles hold unequal capital stocks, with concentration governed by a parameter θ. Each period, households save a fraction of their post-tax income, with richer households saving more. Capital accumulates: k_{i,t+1} = (1−δ)k_i + s_i·y_net. This is the mechanism through which inequality compounds over time. The Piketty r > g dynamic emerges endogenously.

4. Negative income tax implements budget-balanced redistribution: y_net = (1−t)·y + t·ȳ, where t is the tax rate and ȳ is mean income. Below-mean earners receive transfers; above-mean earners pay. A labor supply elasticity captures disincentive effects.

The simulation runs for 40 periods, with automation ramping logistically from 30% toward the target level. AI productivity compounds at a user-selected growth rate. The model tracks output, prices, the Gini coefficient, real purchasing power by decile, capital accumulation, and the labor share over time.

What the Model Shows

The five presets tell the story concisely:

AI Dystopia (high automation, duopoly, no redistribution): The labor share collapses, the Gini climbs relentlessly over time, and the bottom decile's real purchasing power actually falls even as total output soars. Capital ownership concentrates as the rich save more and accumulate more capital each period. This is the permanent precariat.

Competition Only (high automation, 30 firms, no redistribution): Prices fall dramatically as competition forces cost savings through to consumers. The bottom decile does better than under dystopia because goods are cheaper. But inequality still compounds through differential capital accumulation. Competition helps with the price level but doesn't fix the income distribution.

Redistribution Only (high automation, duopoly, 40% NIT): The Gini compresses, but prices stay elevated because the monopoly markup prevents productivity gains from reaching consumers. The transfers are effectively subsidizing monopoly rents.

AI Utopia (high automation, competitive markets, moderate NIT): Output grows substantially, prices fall, the Gini stays low, and the bottom decile's real purchasing power rises alongside everyone else's. The combination of competition (driving prices down) and redistribution (maintaining purchasing power) produces broadly shared prosperity.

The dynamic charts make something visceral that static analysis obscures: inequality compounds. Even starting from moderate initial conditions, the differential savings mechanism means that capital ownership concentrates over time unless policy actively counteracts it.

What's Missing and Where to Go

This is a first-cut model meant to build intuition and invite discussion, not a calibrated forecasting tool. Also as stated above, it is entirely vibe coded, so YMMV.

Among the things worth considering:

Most importantly: checking whether the model appears to be correct. I would love to see other agents pointed at this for analysis.

The NIT currently taxes all income uniformly. This is obviously far from the current policy which taxes labor income more heavily than capital income. So effectively an additional policy choice is at present hardcoded.

The model treats the number of firms as static to simulate a policy choice on competition. There are more dynamic ways of modeling concentration and more sophisticated approaches to examining different types of regulation.

Capital ownership broadening (sovereign wealth funds, stakeholder ownership, broad-based equity participation) is arguably a third policy dimension beyond competition and redistribution. The model doesn't currently include it.

On the production side, the model doesn't capture the Baumol bottleneck dynamics that Aghion-Jones-Jones emphasize: sectors where automation is hard may constrain overall growth regardless of how productive AI becomes elsewhere.

Finally, the model doesn't include the emergence of a "human qua human" sector of the economy: jobs that could be automated but where consumer preferences choose the "artisanal" option instead (e.g. in hospitality).

I'm sharing this model because I think the debate about the AGI economy is too often conducted through competing anecdotes and vibes rather than through structured analysis. Please try the model yourself, find bugs in it, criticize the assumptions, propose extensions. What other policy levers should be in there? What dynamics are missing? You can fork the repo or make a pull request with suggestions.

My intuition that you need both competition and redistribution via a basic income or negative income tax, and that neither alone is sufficient seems to be supported by this model. Looking forward to feedback!

Unlike the dotcom bubble there is absolutely no demand constraint. People and agents are consuming as many tokens as are made available. There is no shortage of bandwidth or a lack of people willing to make online payments. Rapid revenue ramps into the billions are happening, with OpenAI and Anthropic setting a breakneck pace. And increasingly, AI agents are running 24/7, consuming tokens for research, analysis, and autonomous operations — a category of demand that barely existed a year ago and is growing fast.

This time round the issue isn’t demand. It is unit economics. Right now tokens appear heavily subsidized by equity and debt dollars. For example there are numbers out there suggesting that Anthropic may be losing thousands of dollars on $200/month max plans. Inference will have to become rapidly cheaper and/or labs will need to continue raising vast sums of money to continue advancing their models. The fierce competition which includes open source models and large companies which can divert cash flow from existing businesses doesn’t really allow for increased prices.

Now some might object that NVIDIA’s stock price looks reasonable based on growth and profitability. But that of course is only true as long as their customers can continue to plow money into hardware at the current rate or even faster. Any hiccup here could kick off an incredible contraction. AOL famously managed to merge with Time Warner only to implode as a business shortly thereafter when all the equity financed advertising dollars disappeared (including huge AOL investments which were 80% roundtripped back as advertising).

Where might such a hiccup come from? One candidate is the private credit market which is showing severe stress with multiple large funds having to limit outflows. Now some people claim that this is just a consumer stampede while institutions will continue to provide credit but I strongly doubt that credit will remain available for data center and energy buildout to the same extent that it has been. Another potential candidate are IPOs of labs such as OpenAI, Anthropic and SpaceXai. Many people seem to think that IPOs would be bullish but it was IPOs that ultimately broke the dotcom bubble. IPOs require financial disclosure. In the dotcom bubble they revealed meager revenues. Here they are likely to reveal massive cash flow hemorrhaging not just from CAPEX but also from negative unit economics. As a corollary, if SpaceXai can get away with it they will try not to break out segments and just show the company as a whole.

Meanwhile, if inference costs collapse and open source models remain competitive with proprietary ones, the labs face pressure from both sides — they can't raise prices because of competition, and they can't lower costs fast enough because of the hardware investment cycle. In that scenario the value shifts from the model itself to the ecosystem built around it. One specific way open source models can stay competitive is through distillation and reinforcement learning on subtasks for agents.

So where does all of this leave us? I think that there is about a 25% or so chance that a combination of genuine recursive self-improvement and massively cheaper inference will somehow make the economics work in time. Conversely I believe there is a 75% or so probability that we will see a major correction, possibly as early as this or next year.

As with the dotcom bubble the technology is entirely real even if the valuations aren't. The dotcom bust killed Pets.com but it didn't kill the web — Amazon and Google emerged from the wreckage to become dominant companies. Similarly, an AI contraction might devastate labs and their investors, but the capabilities won't disappear. Models will still work. Inference will get cheaper as distressed assets sell. Open source will keep advancing (at least as long as there is research).

The question isn't whether AI will transform the world — it will. The question is whether the current financial structure survives, or whether we will hit a painful reset along the way. I believe the latter is significantly more likely.

Sure we understand some causes, such as if you have a hiatal hernia or an H. pylori infection. As it turns out I appear to have neither based on the gold standard tests. Apparently most GERD cases are idiopathic. That’s a fancy medical word for a condition with an unknown cause. The GI specialist I went to said something along the lines of: “Most of the cases I see just start some day and we don’t know why. We treat them with Nexium and Pepcid and some of the time they go away.” WTF?

I was in disbelief but then hit the Internet and quickly found the Reddit GERD forum and the About GERD website of the International Foundation for Gastrointestinal Disorders. And sure enough, confusion reigns supreme. People trying all sorts of things. Tantalizing sightings of apparent cures amidst an ocean of suffering and frustration. How is it possible that this is the state of medicine on an issue of such prevalence?

As I have started to dig in, one thing has become clear. Way too many doctors seem to think this is somehow acceptable. I haven’t fully figured out what I want to do about it. My first step is to see if I can get better myself. To that end, I am looking into potential brain and biome connections. My early experiments with diaphragmatic breathing are so far quite promising.

I suspect though that I will also want to fund some research and/or invest in startups that are addressing GERD. So if you either have personal experience with improving GERD or are working on / aware of promising research or startups please let me know.

To everyone out there with GERD: this sucks. Let’s make it better!

But just because a society has time and resources it doesn’t automatically follow that its knowledge will improve. There is one crucial missing ingredient: the critical process. The critical process consists of proposing new ideas (“conjecture”) and pointing out flaws in existing ones (“criticism”).  Without the critical process knowledge becomes stagnant. Bad ideas remain unchallenged. Progress comes to a standstill. 

In the prior post on resources I illustrated a lack of resources through medieval palimpsests. That period is the perfect example of societies in which the church exercised such power that it suppressed new ideas, stunting progress for hundreds of years. Humanity almost experienced an absorbing event with the rise of the Plague which killed about a third of people. In general when the critical process is suppressed the result is a lack of progress combined with severe resource misallocation. The combination of the two makes absorbing events significantly more likely.

But even when permitted, the critical process faces a crucial challenge. How are good ideas separated from bad ones? In well-functioning market based economies, companies with bad products shrink and disappear, whereas ones with good products grow. Of course “well-functioning” is doing a lot of work here. As I have written about extensively in my book The World After Capital, in the digital world we don’t have well-functioning markets. 

Similar to markets, we have a decent yardstick in science. Here is what I wrote in a recent post titled “The Imperative for Moral Progress in the Face of Technological Acceleration”

In science, theories eventually get discarded if they lack explanatory power. This mechanism acts quite slowly at times, as we can see in the case of string theory. But over the course of history it has worked quite well. Nobody spends time on the phlogiston theory of combustion. It is at best of historical interest but doesn’t inform current science or engineering.

The problem, however, is that we appear to lack a mechanism for separating good ideas from bad ones in philosophy. My post continues: 

In philosophy on the other hand, texts continue to be studied, no matter how impractical, useless, or even detrimental the ideas have turned out to be for the lives of individuals, the functioning of communities and the progress of humanity overall. While something like string theory is an aberration in science, it is the norm in philosophy. Philosophy has turned out to be much closer to religion than to science.

Now one might think that some philosophical ideas have been successfully pruned because they are no longer politically palatable, such as the divine right of kings or justified slavery. But so far there seems to be no reliable way to assign these a permanent “falsified’ status. Instead, the most we can say is that they are “not currently” politically palatable. To see how dramatic backsliding can be one need look no further than the growing support for autocratic government in the United States.

Epistemic rather than political falsification is why scientific knowledge has been advancing so much faster than philosophical knowledge. Or more pointedly: it is why we have a robust set of universal physical laws while utterly lacking a universal moral core. My post goes on:

This has resulted in a profound asymmetry. With scientific theories pruned effectively we have been able to achieve rapid technological progress. But stuck in a morass of ever growing useless philosophy we have failed to achieve moral progress. This is an incredibly dangerous combination. E. O. Wilson perfectly captured this “The real problem of humanity is the following: We have Paleolithic emotions, medieval institutions, and godlike technology.”

This then brings me to the crucial point which completes my argument. The right measurement for philosophical ideas ought to be whether they help humanity progress. What constitutes progress? Our growing ability to reduce tradeoffs. This definition of progress is an idea that I introduced in the post on values.

So here then is my proposal for a universal moral core. Three principles forming a North Star to guide our judgment in the selection of actions and in the evaluation of philosophical ideas:

1. Avoid absorbing states

2. Mobilize resources

3. Sustain the critical process

This universal core is quite minimal on purpose. It leaves as much as possible open for individuals, communities and even nations to figure out for themselves. The reason for this is implied by the first principle: prescribing too much detail risks resulting in a monoculture that lacks resilience to external shocks. On the other hand I believe that it is impossible to drop one of these three principles and still achieve progress by growing our knowledge. One can think of humanity as engaged in a potentially infinite game. The first principle (avoid absorbing states) says: preserve the game itself. The second (mobilize resources) says: field the strongest team. The third (sustain the critical process) says: keep improving our strategy. Drop any one and the game will be finite.

I plan to write a series of future posts in which I work out the practical implications of this moral universal core for decision making ranging from individual to communities all the way to all of humanity.

Post written with helpful feedback from Lumen (using Opus 4.5)

Post illustrated by Claude Sonnet 4.5

My first time in the United States was as an exchange student in Rochester, Minnesota. I was a junior at John Marshall High School. Early into the school year, some students missed an exam and I was told they would be given the same exam as make-up a week later. I was flabbergasted. Obviously they would get perfect scores because other students would tell them what was on the exam! But then I discovered that wasn't going to happen. Nobody was going to share. Sharing was completely unacceptable as it would undermine the integrity of the exam as a test of individual knowledge.

This was a profound cultural shock to me because the German school system in which I had grown up was based on students extensively helping each other to get good grades, including sharing answers mid exam. In other words, cheating. It was completely pervasive and if you had chosen as a smart student who knew the answers not to share you would have been considered a selfish weirdo and ostracized. As an aside, this attitude of cheating on tests may in part explain how German automotive executives talked themselves into the emissions scandal.

This was the moment where I realized just how pervasive and profound the influence of culture on behavior is. “Culture is what people do when no one is looking” is a great summary by Herb Kelleher, the founder of Southwest Airlines. This is true for the culture of a company and for the culture of a country. Ways culture gets formed is through the behavior of leaders and through the stories that are shared and celebrated.

What made America great was a culture that was a unique blend of self reliance with responsibility to family, community and country. The reason I felt instantly at home in America was because of this combination. You can just do things. And more than that, others will encourage you in that pursuit. No need to wait for someone else to do it for you. But in addition to self reliant action there was an expectation to contribute to your family, your community and there was a strong sense of country. This uniquely American equation has broken down profoundly.

Instead of a culture that celebrates builders who contribute, we have devolved into a culture dominated by selfish extraction and fakery. There is no singular cause for a change this profound, but we can identify some of the contributors. Milton Friedman’s 1970 essay in the NY Times “The Social Responsibility of Business Is to Increase Its Profits” marks a narrowing in the understanding of responsibility. It wound up combining in a toxic way with the Managerial Revolution that James Burnham had identified. The leading figure in this combination was Jack Welch who proceeded to destroy the once great GE, with his acolytes doing the same to other iconic American companies, including Boeing. I strongly encourage everyone to read David Gelles’s excellent “The Man Who Broke Capitalism.” One crucial aspect of the Welch approach was aggressive earnings management. In other words, cheating. The substance of the company was hollowed out to produce artificial and unsustainable gains in earnings. Outsourcing, offshoring, and financialization were spread throughout the economy as Welch wound up being celebrated on the cover of every business magazine. 

Now one might object that the robber barons weren’t exactly paragons of good behavior. Didn’t they use force to break the unions? Didn’t they intervene massively in politics? Yes absolutely. But there were two crucial differences. First, they were builders. They created factories, railroads, power plants and more. Second, many of them did give back in ways that lastingly made their communities better by funding hospitals, libraries, concert halls, etc. To be clear, I am not excusing their suppression of the labor movement, I am simply pointing out that there is a profound difference between creation and destruction. Incidentally this is why I am much more willing to see Elon Musk as a complex figure than a private equity raider or a high frequency trader.

What started in the 1970s found its way ever deeper into the culture. Much of reality television and celebrity culture became simply about selfish zero sum games. Put a bunch of people in a house and promote intrigue, with the most unscrupulous person emerging victorious and nothing built in the process. Similarly the financial markets, which were once crucial to the build out physical capital, became ever more self referential. Derivatives exploded in scale beyond anything that could possibly be justified as providing liquidity for hedging risks in the real economy. The proliferation of sports betting and meme stocks is just the logical conclusion of everything becoming a casino. Selfish extraction became the culture, replacing self reliant building and giving back.

I had been meaning to write about this decay in culture for some time, since I posted on the breakdowns in the rule of law (which is an integral part of shaping culture). Writing the post today was prompted by last week's debate over the role of Asian immigration in shaping American culture. Calling it a debate is somewhat generous because a lot of it was just people insulting the other side's motives or intelligence. There was very little of an attempt at actual understanding and responding.

Yes of course immigration can change culture. But the degree to which this is the case depends on the strength of the culture and the rate of immigration.  This is not unlike a natural ecosystem. A healthy forest will integrate new species as they arrive and they will find a niche that integrates into the overall system enhancing its biodiversity and resilience. A sick forest on the other hand can get taken over by new species and completely reshaped. When I arrived in the US, there was a healthy culture of not cheating. And so even though I came from a place of pervasive cheating in school I accepted and integrated myself into the system I found.

When you normalize cheating from schools to businesses to the highest levels of government then adding immigrants from countries in which cheating is pervasive will definitely not help. Now this explains a lot about the difference in experience: there are remnants of the old culture and as a result the US still attracts immigrants who want to build, whether that is a small local business, such as a restaurant, or a globally competitive tech company. As a result we are seeing many successful businesses at all levels founded and/or led by immigrants.

That’s why the wholesale vilification of immigrants by a government led by a liar and cheater is so profoundly wrong. It ignores that many immigrants (including quite a few who came here illegally) are in fact living the culture that made America great. All the while the government is further tearing down that culture with its own acts, such as sending masked and unidentified officers into the streets. There would be broad support for a sensible immigration policy along with rebuilding the culture that attracted me and many others to America in the first place.

More recently I have been vibecoding with Claude. And it is entirely clear that we have entered the age of automated software. What would have taken days, or weeks, or possibly months and even years, can now be accomplished in minutes, hours, or days. And all the necessary prompting takes place in natural language. No more remembering of esoteric assembly commands, or learning the syntax of a new higher level language. Just expressing one’s intent and then providing updates on what’s working and what is not. Writing software has gone from being difficult and slow to being fast and easy.

What does that mean for software businesses? First, when you make something cheaper you get a lot more of it. There will be more software created than ever before. Second, no matter how cheap you make something you wind up with more consumers than producers. Anyone can post on social media and yet way more people read or watch posts than create them. The same will be true for software, which means there will still be software businesses. Third, the era of high margin software businesses is largely over. There will be fierce competition including the now credible alternative of building it oneself. The recent rerating of software companies in the public markets is an early recognition of this new dynamic.

What will happen to the software industry? We are likely entering a period of maturity where margins will become similar to those in other competitive industries. Along with this will come a massive consolidation of the industry. We already had some prior rounds of this when there were platform shifts and older software businesses wound up getting consolidated into the likes of Computer Associates.

There are some important exceptions to this logic though. The most important are companies with network effects. We are seeing this in the continued strong profitability of Meta (with the caveat of financing very large capes for AI infrastructure). On the business side, something like LinkedIn, continues to perform well. Cloudflare has a less obvious network effect where they can detect malicious traffic on one part of their network which is then helpful to every customer. But the list of potential candidates for software businesses with strong network effects is quite small. One other area where there may be reasonably strong margins are databases. No matter how competitive code may become, the value of data is only going up and so is the willingness to pay for reliable and performant storage.

What about the tooling layer? Companies such as Cursor or more recently Conductor? It’s highly unclear that anyone can build a defensible business here. Most if not all the power seems to come from the underlying models. So the real question is more whether open coding models can stay competitive with the closed ones. If yes then there will be continued innovation and price pressure on models also. If not, then a lot depends on the competitive dynamics between the closed models. If there are multiple close coding models all trying to gain market share, then here too it will be difficult to operate at high margins. The long run equilibrium is anyone’s guess though.

In part it is difficult to figure out the long run equilibrium because it is unclear what will happen to open source and how that will matter. One of the reasons the coding models are so powerful is because they have read every open source line of code there is. So not only do they have the benefit of being able to invoke these libraries and build on these frameworks, they have also learned how to code from them. One possible implication is that open source will atrophy and potentially get swamped by code “slop” with the fraction of model generated code being contributed to Github rising rapidly. On the other hand it might be the case that we can truly figure out self play for coding and so the existence of prior code will be irrelevant.

There is another implication that is maybe counterintuitive. In the past it was great advice to not attempt a complete rewrite of a system. In 2013 I published a blog post on “Evolving your Technology as you Grow” in which I wrote that you should “never, ever [...] rewrite everything from scratch.” I pointed to a post by Joel Spolsky from 2000 that gives examples and a detailed argument for why a complete rewrite was a terrible idea that had killed several businesses. Well, automated software inverts this logic. If you have a big convoluted existing code base, you are now much better off, just using your data and having AI rewrite from scratch. You can use your existing application as the description of what you want! Keep in mind that any potential new entrant doesn’t have a legacy code base. In the past that might have meant it would take them years to build what you have but that’s just no longer true. The only thing you have that’s truly valuable is your existing data and customers.

Finally there are implications of automated software for the labor market. For several decades, learning how to code was a sure fire way to a high paying job. I financed my first car and my college degree by writing software. Programs such as Pursuit in New York have helped people go from minimum wage jobs to high paid software ones. Now there is a glut of coders and the only reason the labor market is not already horrendous is because large companies are notoriously bad at realizing productivity gains. That’s in no small part because the incentives tend to work against it – often one’s power inside a company is a direct function of how many people are in the department. By contrast, there may be more opportunities going forward in product management because defining the right thing to build is now ever more the constraint rather than actually building it.

If all of this sounds over the top to you, it probably means you haven’t coded with this latest iteration of models. It is genuinely difficult to get a sense for just how powerful they are without trying them out oneself.

I am approaching this moment with the clarity that comes from recognizing that one has failed. Over a decade ago I started calling for dramatic changes to how we organize society. I gave talks, wrote blog posts and even published a book. 

But I didn't make a dent. Instead, the existing societal machinery carried on. I knew all too well that it was falling apart. And yet I had hopes that it could be fixed.

The old order was clearly decaying. The rule of law had been undermined for years. Jack Welsh and his acolytes broke the economy. Private equity raided industries for short term financial gain. Everything got financialized. Government grew ever more bloated and ineffective. Well-meaning laws were weaponized to prevent anything from being built. 

Major problems weren't being addressed. Income and wealth inequality exploded. Atmospheric carbon skyrocketed. Infrastructure decayed. The misguided war on terror consumed immense resources and eroded civil liberties. To keep the public pacified gambling was allowed to grow boundless.

I kept arguing for an enlightened turn. For addressing our myriad problems head on all the while not giving up on core principles of democracy and rule of law. But I failed to build a sufficient audience or momentum for my ideas.

And in the meantime the forces that were willing to support a “blow it all up” approach kept growing. No principles. Oblivious to grift and corruption. Anything in the name of accelerating change. Might makes right. Along with so much else this is an echo of a century ago when futurism supported fascism. As Marx so brilliantly put it: history repeats, first as tragedy then as farce. We are getting the farcical version, but it is no less destructive.

So what now? There are two essential tasks at hand now.

The first task is to keep the bad from becoming extremely bad. To slow down the descent into madness. To create time for individuals, families, communities and even nations to prepare for what’s coming. The two approaches that I am partial to here are using what’s left of the old system as well as peaceful protest. Leveraging state and local powers and courts as long as they can assert themselves independently. And providing a surface for government overreach and brutality. 

To be clear up front: this will be an incredibly frustrating and costly process demanding great individual sacrifice. In Babylon Berlin, Gereon Rath and Charlotte Ritter never stop pursuing justice even as the Weimar Republic crumbles around them.  Many heroic acts of trying to uphold order, ultimately overwhelmed by the rise of evil. Another historic lesson is to anticipate the eventual use of “agent provocateur” tactics. 

Now maybe the United States has a stronger immune system so another reason to apply effort to this task is that it might just succeed here.

The second task is to invent what can come after the madness. This requires taking a long view. Establishing principles. Building a movement. Gathering strength. All while avoiding or surviving attacks from the agents of chaos. The traps along this task are a great many and unfortunately we don’t have the benefit of a Hari Seldon who has set all of this in motion for us already. Still we can take inspiration from the Foundation series. A movement that will be long term successful would do well to embrace the core principle that ultimately knowledge does underpin power.

A key challenge on this task will simply be to keep at it as the unfolding regression could last for generations. This is what I am committing myself to. I will continue to develop the ideas from The World After Capital and my blog. If I have one true strength,it is tenacity. Far from abandoning the quest for a “Knowledge Age,” now is the time to double down. That is exactly what Gigi and I are doing with our projects in the Hudson Valley, such as HudsonUp, The Spark of Hudson, Hudson Dots and Wally Farms.

Whatever we do, let’s not succumb to opportunism or worse become an active promoter of chaos and destruction. Instead let’s apply ourselves to one or both of these tasks. We need to buy as much time as possible and keep the regression as small as possible – all the while we are actively building a new and better world.

European Federation Manifesto

Europe needs to be strong – for itself and for the world.

China has become a superpower and the United States is struggling with the accumulated debt of neoliberalism and global intervention. Russia is waging a war at Europe’s doorstep. Europe can no longer rely on the support of the United States. Europe has fallen behind in its capacity to defend itself. Europe is rapidly deindustrializing.

The existing European Union has frittered away its credibility by turning into a costly bureaucratic morass. The European Union has lost the support of the people. At present course and speed the European Union will disintegrate and disappear into irrelevance. The same will be true for European nation states which are too small on their own.

It is time to reboot the European project as a European Federation.

The European Federation will focus on a minimal but crucial set of policies: defense, secure borders, abundant energy, frictionless commerce and dispute resolution between constituent nation states. Participating countries will maintain sovereignty in all other matters. As a first approximation the European Federation will be modeled on Switzerland. Like Switzerland, the European Federation will be neutral (i.e. not aligned with the United States or China). The European Federation will be committed to being non-offensive and non-interventionist. 

The European Federation will stand for humanist values. It will be tolerant but defend vigorously against intolerance. The European Federation will pursue a third path that provides an alternative to the Chinese government surveillance model and the emerging US might'makes-right model. The European Federation will leverage its existing strengths in research and manufacturing and attract scientists and entrepreneurs who want to drive forward progress in the freedom that arises from the balance of restrained and capable government with individual freedom.

The European Federation charter will be developed by a constitutional convention. Nations will send 2-5 delegates to the convention depending on size. The delegates shall be chosen by sortition among each nation's scientists, engineers and entrepreneurs. The delegates shall not include anyone who has been a professional politician for more than 3 years of their life. The delegates will be assisted at the convention by experts in constitutional law with a particular focus on federations.

The time to be bold is now. The time for the European Federation is now.