The dangers of unstoppable code

With real-time, interconnected, self-executing systems, sometimes when things wrong, they go really wrong.  I wrote about this general idea previously here.

Yesterday, while I was writing my post on Trusted Brands, I was doing a little searching through my blog archives, so as to link back to all the posts categorized under "Trust".  In the process of doing that, I went back and actually re-categorized some older posts that fell under that category, but weren't appropriately marked.  In the process of doing that, I came across a whole bunch of posts from 2013 that I had imported from my old Tumblr blog, but were still just saved as drafts rather than published posts.

So, I did a little test with one of them -- hit Publish and checked that it looked right.  Then, after that, I did a bulk-edit with about 15 posts, selecting all of them and changing the status from "draft" to "published".

This did not have the intended effect.

Rather than those posts showing up in the archives under 2013, they were published as of yesterday.  So now I have 15 posts from 2013 showing up at the top of the blog as if I wrote them yesterday.  

That would not have been a real problem on its own -- the real problem stemmed that because of our automated "content system" (that I built, mind you) within the USV team, those posts didn't just show up on my blog, they showed up on the USV Team Posts widget (displayed here, on Fred's blog and on Albert's blog), they showed (via the widget) in Fred's RSS feed, which feeds his daily newsletter, and blast notifications were sent out via the USV network slack.  Further, some elements of the system (namely, the consolidated USV team RSS feed, which is powered by Zapier) is not easily changeable.  

Because of the way this happens to be set up, all of those triggers happen automatically and in real-time.  As Jamie Wilkinson remarked to me this morning, it is unclear whether this is a feature of a bug.   

Of course, as all of this happened, I was on a plane to SF with spotty internet, and was left trying to undo the mess, restore things to a previous point, monkey patch where needed, etc.  

Point is: real-time automation is really nice, when it works as intended.  Every day for the past few years, posts have been flowing through this same system, and it's been great. No fuss, no muss, everything flowing where it should.

But as this (admittedly very minor) incident shows, real-time, automatic, interconnected systems carry a certain type of failure risk.  In this particular case, there are a few common sense safeguards we could build in to protect against something like this (namely: a delay in the consolidated RSS feed in picking up posts, and/or an easy way to edit it post-hoc) -- maybe we will get to those.

But I also think about this in the world of crypto/blockchain and smart contracts, where a key feature of the code is that it is automatic and "unstoppable".  We have already seen some high-profile cases where unstoppable code-as-law can lead to some tough situations (DAO hack, ETH/ETC hard fork, etc), and will surely see more.

There is a lot of power and value in automated, unstoppable, autonomous code. But it does absolutely bring with it a new kind of risk, and will require both a new programming approach (less iterative, more careful) and also new tools for governance, security, etc (along the lines of what the teams at Zeppelin, Aragon and Decred are building).

Trusted Brands

Today is election day.  I'm on a plane today, so I voted early, a few days ago.  I cast my vote and it felt good. I marked my paper ballot with a marker (for optical scanning) glued it shut into a sealed envelope, and handed it to a volunteer who placed it in a secure case.  I left with a sense of confidence that my vote had been placed, and would be counted fairly.

Unfortunately, many Americans do not have the same confidence, often for good reason.  As Zeynep Tufecki wrote in the NYT last weekend in The Election Has Already Been Hacked:

"the legitimacy of an election depends on the electorate accepting that it was fair, that everyone who tried to vote got to vote and that every vote counted. Lose that, and your voting system might as well have suffered a devastating technological attack.

Unfortunately, in much of the United States, we are no longer able to assure people that none of those things has happened. A recent poll shows that 46 percent of the American electorate do not think their votes will be counted fairly, and about a third think it is likely that a foreign country will tamper with the results."

Put more generally, America's most important product, our democracy, is developing a trust problem.

I have been writing about trust for a long time. It is an essential ingredient for institutions (civic, corporate, etc) and also for technology platforms.  It is famously hard to earn and easy to lose.

In the most recent iteration of the USV investment thesis, we explicitly call out the idea of "Trusted Brands" as a key component.  This has been somewhat controversial, because -- obviously -- it's premature to declare that any early-stage startup is a trusted brand.  But what we mean by this is not that it's already a widely trusted brand, but that it's architected to be a trusted brand.

“Trusted Brands” is also controversial because in startups, things change. For example, while for a long time many considered Google (or really, insert nearly any other large tech company here) to be a trusted brand, initially because of the super useful/helpful services it provides, and then through it's mantra of "don't be evil" -- it's safe to say that today, many people (both consumers and b2b users) may not feel the same way.  

So, "Trusted Brands" is both aspirational, and potentially fleeting. 

By "architecting for trust“ we mean two things: 1) a business model where the platform and its stakeholders are aligned; and/or 2) a technical architecture that makes tampering/hacking/meddling difficult or impossible, even for the creators of the system.

On business model alignment: this one gets trickier the more stakeholders you have, and typically as tech platforms grow and expand, so does the breadth and variety of stakeholders.  But core decisions can be made, early on, that align interests as much as possible -- for example, see Andy's post about our recent investment in Scroll, and how they are attempting to align interests in the publishing industry.

On technical architecture: most of the work in the crypto/blockchain space is based on this concept, and many have taken to a mantra of a “can’t be evil” as a counterpoint to the arbitrary decisionmaking that is possible in traditional platforms -- this is at the heart of the "centralized vs. decentralized" debate.  But even in "regular" apps, technical design decisions, particularly around how data is stored/processed/shared/accessed, can go a long way towards building trust.

What we can safely say is that we are suffering from trust problems on a number of fronts.  On the tech platforms side, it's things like data breaches, abuses of market power, shady privacy practices, harassment and abuse, etc etc.   On the institutional side, it's relevance, capacity, efficiency, technological capability, corruption, and fundamental integrity.  Not a small or simple list of challenges to face.

The technology we build is really the foundation of our lives, and even the two "sides" I discuss here (traditional institutions and tech platforms) are getting blurrier by the day.  Given all that, what we mean by "Trusted Brands" is that establishing and sustaining trusted products / platforms / systems / institutions / brands is more important than ever, and we believe that the efforts that do manage to earn our trust over the long term will be the most important and valuable.

Suffering, self, and service

The massacre in Pittsburgh is heartbreaking and awful, and another example of the extent to which society seems to be fraying.

The Pittsburgh attacker spent a lot of time on social media sites that stoked his fear, isolation and anger.  I think about the internet a lot, and while the internet has the ability to help us form a better understanding of "we" (global humanity), it can also cultivate a strong sense of "them" (the dangerous other), as this case demonstrates.

In other words, we are simultaneously increasing our capacity to understand one another through connectedness and information, and fracturing along tribal lines, increasing the sense of distance and disconnectedness.

I am no scholar of Buddhism, but have been interested recently in the Buddhist notion of the relationship between "suffering" and the "self".  In a nutshell, the concept is: suffering is an essential human condition, and it is primarily brought about by our sense of self and how events impact us as individuals (jealousy, greed, wanting, disappointment, etc).  Meanwhile, there actually is no "self", as everything in the universe is connected.  Therefore, if you can release your focus on the self, you can dissolve the suffering. (Here is a good overview of these concepts.)

I think about these concepts in the day-to-day: for me they manifest in all the little moments of going about my work and getting things done.  Often times, I feel a resistance welling up, often manifested as fear, which I have written about, but more generally I think the culprit is the self-centered thinking.  When this happens, an idea that works for me is actively seeking to replace thoughts of the self with thoughts of service: take the suffering that comes from seeing things through the lens of your individual self, and redirect it to the service of others.  When this happens, I can physically feel the "suffering" melt away.

My own examples are of course trivial compared to the broader environment of fear, suffering, and violence.  But I would like to think that we have the potential as humans to re-knit the ties that bind us together, somehow.

The Slow Hunch by Nick Grossman

Written by

Investing @ USV. Student of cities and the internet.

Subscribe