Staying with it
A reflection on Andrew Kelley’s ‘Don’t take the black pill’
I recently listened to Andrew Kelley’s talk Don’t take the black pill from the inaugural Software Should Work conference and had a few loose reflections.
If you are reading this and haven’t watched it perhaps go do that, or read the written version.
Firstly, I haven’t seen The Matrix. So I would’ve been one of the hands raised at the beginning when this question was asked. That may discredit any opinions shared here. Fair enough, so be it.
Now for anyone still reading, the second thing I’d say is that I have a lot of respect and admiration for Kelley and enjoyed his talk very much. From an outsider’s perspective, Kelley seems to have a remarkable authenticity and capacity to operate according to his values. The Zig Software Foundation itself is a beautiful thing in the world, produced with will and imagination.
In the talk, Kelley describes incel culture and its negative feedback loop. Groups of people arranged, at least in part, around a grievance. Members who have their grievances assuaged eventually leave, the group becomes denser in apathy and spirals into self-fulfilling despair.
We find a similar sort of grouping in an audience of technologists gathering around the idea that ‘Software Should Work’. With the implication that it doesn’t. These are loose reflections on a conference talk, not the building of an argument or the knocking of one down. But very roughly I’d say the premise is that we’ve long had the ability to make software that works, in theory, and at times in the recent past had the ability to make it work in practice due to an arbitrary or incidental distribution of power in the hands of people that make it.
It was possible for programmers to smuggle in quality in spite of an environment that largely had other priorities, because demand for development outstripped supply.
I think there are some interesting murky complications here.
First of all the idea of software ‘working’ is an interpretative idea. Work for who? For whose benefit?
Here, we take it as being for the user. With the dichotomy that it is either that, or work for the organisation or business that develops it largely for the purpose of extracting value from the user.
Ignoring a plethora of asides on how we could think about what it means for software to ‘work’. The idea that we have long known how to make software that functions well, on whatever definition of effectiveness, is itself interesting. We have shown that given the right set of trade-offs and instrumental interests we are capable of wrangling our soft and hardware to do things like go to the moon or create the internet.
But there are those who, perhaps in a vein similar to John Backus in his 1977 ACM Turing award lecture Can programming be liberated from the Von Neumann style, might read this history less as the attainment of a knowledge potentially now lost, but rather like getting immediately fused into one way of doing computation that solved particular needs at the time.
But again I don’t want to continue with that idea further here.
Aside from interesting digressions on the point of the state of our knowledge of computation and its implementations, the talk goes on to speak about power and the ability to put this knowledge to work.
As mentioned, that in the recent past an incidental distribution of power helped the people that make software have more agency in dictating how it was produced. But that now that incidental power arrangement might be being disrupted.
So we have a description of what’s happening in terms of trades of power and control. And it is beautifully captured in the image of the transistor, which accepts a tiny current of power or voltage in a base terminal to control where a relatively large current of power will be directed.
That today, the control over how this literal and figurative power flows has now veered far to one side of the spectrum to the benefit of large corporations. This slow drift of the tacitly agreed notion of what counts as a legitimate control flow has now produced an abundance of ‘coercionware’; software as a kind of exploit attack on the human mind.
A/B testing as a feedback loop used purely with bad intentions. Or using it as cover from stating higher order intentions altogether, being able then to merely point at the numbers with innocence.
Kelley isn’t trying to provide a complete description of A/B testing and that is fair enough. We can say it’s true that it ‘prioritizes outcome over intentionality’. But, we might also say that A/B testing could also prioritise ‘discovery over top-down decision making’. If only we could have the culture to support such an approach in your average team.
But thanks to all this coercionware, we are like creators whose content drifts in response to algorithms to the point where we come to actually believe what was initially just effective for us to say.
Or consumers finding themselves ‘hijacked’ by infinite scrolling, the software is acting on us. And we are more algorithmic in our thought due to our interaction with it.
What does it say about us if we are fundamentally hijackable into doomscrolling by these methods? The philosopher and youtuber Vlad Vexler approached this point asking, to what extent does doomscrolling say something about our nature?
Does it describe us?
We can come to the idea that we are in some sense just physiological beings that move through the natural world responding to stimuli. And respond reliably to those designed to elicit such responses. But being what Charles Taylor calls ‘self-interpreting animals’, we also have the capability to transcend this aspect of nature.
So to just say that we need to firm up and stop scrolling (I’m not implying Kelley suggests this) is in contempt to at least one part of our nature.
This point of transcendence is important because without it, I think we can take the metaphor of our brains being like exploited computers too far. This metaphor can come to over or under describe things.
Under describe some aspects of how we move through the world and its stimuli, and over describe certain effects of coercionware such as its direct effect on election results, paradoxically here under describing its indirect effects on the same election results. I’ll come back to this point.
So Kelley describes a politically realist account of how programmers made good software via a negotiating position, with a supply and a demand, and via various tactics and benevolent incompliance.
We then turn to faith. And a call that we need to believe humanity to be good because the alternative would be unbearable. Or in other words, take the political situation we have just sketched and turn it into a form of applied morality.
This transformation is one way to not have to go further, or, one kind of disengagement.
So like we earlier had the black pill disengagement, the turn towards nihilism and apathy. Here we have another disengagement, the turn towards moralism. Dissolve the political predicament — but with voluntarist optimism.
What if there was a third way that didn’t require turning away so much in either direction. What resources might we need if we were to stay with the account we described, and stay engaged, in some way, in the conversation around that account? We might have to hold that engagement more fluidly, with a broader conception of how things are, ought or might be.
I think an engagement of this kind might be helpful.
While the moral remedy may work for some people — those that can and will pay the cost upfront. Like Kelley does, or a particular engineer on a particular team might be willing to do. It is probably less realistic for many.
While, yes, people have choices, costs are not distributed evenly, and a remedy that ignores that is a kind of curiously inhumane or quietly misanthropic moralism. Or a moralism that contains a secret wish that humans were different.
So, again, what would happen if we stayed with the realism a little longer?
We said earlier that a fundamental point was: who or by what mechanism do we decide how the power in the transistor gets routed. Or in what way is the control of power deemed legitimate?
Our sense of political legitimacy and technology is generally culturally confused right now. Lines are blurred between how legitimate a thing a market is generally in its relation to power, and technology has a blurred place within that market. We tacitly treat technology platforms as if they are more legitimate political actors than other random companies.
Their legitimacy should not be sustained by their power or interference, by being intermingled with media and capital. That is a bizarre inversion of a legitimacy demand.
But this doesn’t just boil down to an argument about capitalism. Deeper than that, it is related to core parts of our humanity and how they are expressed outwith any particular ideology. How we navigate the modern world socially, politically and phenomenologically.
Doomscrolling is one particularly salient expression of the manner in which technology is interwoven in social and political life. Technology’s impact in such local-personal ways, and social media’s disruption of the wider informational environment is one of many factors that is causing an erosion of trust in our society today. But that isn’t to say it caused this erosion as such. The causal arrows go both ways, to again echo Charles Taylor.
Narratives like ‘social media is being used for interference in our democracies’ are actually underselling the problem. While there is some of that going on, it is not the central issue, we treat it as the central issue because it is an easier thing to think. It is easier to pretend that there are external forces doing bad things, and if only we could stop them then everything would be okay.
The erosion of trust affecting the state of our democracies is more importantly understood as a somewhat more disturbing internal problem. Trust, and thus legitimacy, is a determining factor in the routes of power in our societies. Lack of trust destabilises our social and political institutions, with technology playing a multifaceted role both in changing our informational environment more broadly, affecting trust, and in affecting our capacities to trust in personal, private ways.
Replacing political thought and action with different types of depoliticised thought and action is this erosion happening in real time, from the bottom up rather than top down. Turning away from society into pure nihilism through lack of trust is one example, generally doing politics as a form of self expression rather than as a way of trying to cause political effects in the world is another. Making even optimistic purely moral choices in a political situation would be a third.
Having stayed with the political aspect of the situation a little longer, can we still find grounds for hope?
I think so.
Kelley gestures towards the power of small acts. And I think that is right.
We are usually well aware of how small differences in the way that people act towards us can have an effect on us. We tend to underestimate the degree to which it is true that small changes in the way we act have similarly large effects on others. This is a hopeful fact, and I agree with Kelley that we should use it to proceed with care in how we interact with others.
Then there is the hope that we have already touched on, of choosing to believe in good, described by Kelley and embodied by his reading of Stanislav Petrov, Russian lieutenant colonel of the Soviet Air Defence Forces, who contrary to protocol didn’t notify his seniors of a potential nuclear attack that might have resulted in catastrophic devastation. Later he said that he thought the odds of it being a false alarm were 50-50. Kelley says that faced with uncertainty Petrov made his decision by choosing faith in humanity, choosing to believe mankind is Good.
As mentioned I might have reservations about this as such, and also whether Petrov does indeed embody it. Perhaps we could see Petrov as someone who made a judgement about reliability within a system he generally distrusted. Someone who faced an absurd decision and acted humanely rather than goodly. Someone whose decision was as much about realism under pressure as about faith. I don’t know, and wouldn’t want to offer anything beyond saying I doubt that that moment is described either by calculation or by belief.
Two other kinds of hope.
One kind of hope might be found in looking at the general idea of progress through the lens of someone like John Gray. Feeling like one has ‘lost ground’ and that things that were possible previously are now not, and the general idea that we should be ‘accelerating humanity along its current trajectory’ all assume some kind of direction built into history. Something we might call progress. From monotonic growth to Obama describing history as moving three-steps-forward-two-steps back. They all share some generally teleological view that is only acceptable with very narrow conceptions of history.
As Gray might say there is no such thing as progress, as we understand it. That progress is an illusion given to us by a simplified view of history and the present. Some things get better, some things get worse, and the continual conversation of conflicts proceeds. Implicitly accepting that things generally proceed on an incline, and then feeling that things are declining, causes us to fall into dissonance and worry. It is not necessarily easy, but one can have a slow adjustment to a different understanding of history.
Letting go of this conception isn’t to turn to apathy, but rather to let go of something no longer serving you, and to be freer for it.
Another type of hope would be one produced not by climbing down from apparent commitments to ideas of progress, but instead a climbing down from implicit ideas of humans and what we are like.
Earlier we said that we need to decide humans are good because the alternative is unbearable. But another option is that we don’t require humans to be ‘good’ at all in order to be fond of them anyway.
Doing technology badly is just kind of human, voting for particular things or people is likewise just humans doing politics. This isn’t to blindly say everything is okay, which would be another example of moralistic disengagement.
It is to say that it is possible to see ourselves as these very fallible things, without that being so threatening. Evidence of our fallibility doesn’t mean we have to rush to harsh dichotomies of goodness or badness and subsequent necessities to choose good.
Evidence of fallibility could, rather than be a potential trigger for nihilism or a potential trigger for a secret wish that we were not what we are — we could see ourselves a bit more gently, fluidly as just being what we are. Perhaps this might be one way of allowing ourselves the extra flexibility to stay engaged, even as things get darker.
So, an invitation.
To consider whether a ‘let it all go to hell’ kind of apathy and a ‘just firm it’ kind of optimistic moralism are two different ways of exiting a conversation.
And what it might be like to instead, stay with it.
Also — go watch that Ninajirachi Infohazard video if you haven’t already.