Superintelligence
The fundamental problem of politics is how to transform societies composed of self-interested individuals and sectional interests into societies composed of citizens who, in acting for themselves, act also for others and thereby preserve and advance the public good.
The nature of the relation between truth, politics, and people has baffled, frustrated, and disgusted people of all persuasions since the death of Socrates at least. The politicians should ‘get on with it,’ they should ‘all get together and agree,’ and do all the good things the good people want, even if said people can’t precisely name what these good things are. ‘Something should be done,’ and the politicians are the ones charged with doing that ill-defined and very possibly impossible ‘something,’ to be excoriated in short order as useless and worse.
We should move quickly past the basic truth that politicians are supposed to argue and dispute, even if the public demanding unity prefer to scorn it as squabbling and bickering. The basic point that is always at issue and will always be at issue so long as we have a politics worthy of the name is what, in precise terms, is the ‘it’ politicians are supposed to be ‘getting on with.’ As for the view that politicians should ‘all get together’ and do this ‘something’ that should be done, I agree wholeheartedly. I agree that everyone should unite behind me and do exactly what I say should be done. For the sake of unity and harmony, you understand. Problem solved. Of course, the people who demand such common agreement and conformity on the part of each and all presume that the world and its wife will be united behind their particular view as to what should be done. Whenever I hear that call, I immediately demand that those making the call abandon every single one of their demands and join in agreement with their political opponents. For the sake of ending the squabbling and achieving unity. The communitarians and Unitarians become most fractious and dissenting in response to that demand.
This may be childish nonsense. Unfortunately, an awful lot of people have got it into their heads that their particular view of democracy is the one that overrides all other views in politics, democratic or otherwise. They are merely giving expression to Plato’s Social Beast. An aggregate of self-choosing individuals can never constitute a public realm, and will ensure that democracy fractures and implodes from within.
So let us start from the premise that human beings are utterly flawed creatures, armed with self-assertive stupidities that, when institutionally and morally untempered and unguided, are certain to sum to one great collective stupidity. And the politicians trying to mediate between truth and people in these circumstances will be the ones to be blamed for the mess. Here we have Plato’s Social Beast on the rampage, demanding the impossible, and then blaming the politicians for their having failed to deliver it.
I want, however, to look at the problem from the other end. Those armed with truth, I would suggest, are also demanding the impossible when they suppose that there is, or ought to be, a direct uni-linear translation of truth into government, politics, and policy. The emphasis here is upon the education of the people and, that failing, the authoritarian direction of the people. Truth, after all, is non-negotiable, so there really is no reason to waste time with dialogue, debate, and deliberation with and between individuals who just don’t know stuff. Since we already know what the truth is, we are just wasting time trying to get sufficient people to know it, recognize it, and act on it. It would be so much easier if we just accepted that people are stupid, follow their noses, mistake shadows for realities, and thereby send the public realm into the ditch.
The attitude seems to be that democracy is a recipe for chaos. I would be inclined to say that there is plenty to commend that view, were it not for the fact the elites have proven to be at least as stupid, self-interested, and blinkered as the people, in foreign policy, war and the arms trade, in finance and economics, on the environment. It’s an ancient wisdom. The blind lead the blind, and all end up in the ditch. In an age when truth has been reduced to value, itself long reduced to mere personal choice, then leaders cannot lead, for the reason that people are no longer willing to be led by anything other than their own god of personal choice. Leadership presupposes people who are willing to be led, on the assumption that there are greater truths than personal likes and preferences. In an age which lives by purely human values, we find that these values themselves reduce to the individuals choosing these values, with democracy becoming merely the aggregate of wants. This is Plato's Social Beat. The result is the chaos we have now, characterised by the lack of a properly public realm, with people and society split and demanding that politicians unify the whole terrain. It can’t be done. And so the squabbling and bickering people turn and blame the squabbling and bickering politicians. And enlightened evolutionary vanguards, in turn, blame the people for being stupid and selfish.
Unity is possible in this fractured terrain, but only as a dead collection of atoms without internal connection and cohesion. That unity is a grotesque parody of the ideal solution, which refers to diverse voices combining in concordance with truth. If we look at the problem from the other direction, from the standpoint of truth rather than people, we see all the squabbles and stupidities coming to be excised once arrangements are in place so that only the right conclusions can be drawn. If truth trumps all things, then there can be no objection to the creation of a new society in which it is impossible for prejudice, ignorance, and stupidity to obstruct and distort the decision making processes. Committed to a cold, logical, and non-negotiable truth, those who consider themselves to already know this truth demand that all individuals become flawless reasoning machines or be deemed unworthy of account. This entails a demand that human beings be rational calculating machines that always make the right decisions. There is no recognition of the creativity of citizen agency, no sense that human beings are co-agents in making history and reality, and no sense of the universe as a participatory universe. Instead, there is only a disclosure of a truth which human agency has no part in making or unfolding. That view amounts to a total denigration of politics and of creative human agency, and flies in the face of the evidence, however negative it may be in many respects, of the human impact on the environment. The fact of climate change may be a negative example that somewhat proves the point of the need to conform to truth and reality, but that is the point of agency set at extremes. Short of negative impact, the human-nature interaction may well be complementary rather than contradictory, and it is this interactive dynamic that goes missing in views of pure disclosure. The rightness of choices and decisions in this view is determined by the extent of conformity to truth and reality. Since we know the truth, of course, then decision-making can never be a true making, merely an acceptance and assent. That view is not true to nature, to human beings, and to politics. I see in these views a continuation of the ‘men like gods’ fantasy that H.G. Wells wrote about. Or a variant of this view. On the one hand, there are those who emphasise that human beings, through their technical power, could still create the Earthly Paradise, if only human beings could be more rational. This is the ‘men as gods’ delusion. On the other hand, there are those who claim such deification to be a delusion whichever way it goes, and instead affirm only a nature that could care less one way or the other. Human beings are alone, without purpose and meaning in a cold and indifferent universe. This is the reality whose truth human beings are enjoined to conform to. To which I say that if this is the only reality, then it is no wonder that human beings make up alternative truth. Human beings are cultural creatives by nature, and create a reality that is every bit as real as nature, and a sight more meaningful. Human beings are immersed in culture. They may die by it, too, comes the rejoinder from people who take delight in constantly stating that physics trumps politics and nature defeats people. All material things pass. People die. If that is your truth, then it is trite and of no great import. That’s not where the great issues are.
The fact that so many make such an issue of this tells me that there are still people who aspire to perfection, immortality and eternity through human self-creation. They may yet get their wish. The ‘men as gods’ fantasists are still at work. Some of them are currently obsessed with creating machines that are more intelligent than human beings. I’m curious to know what motivates the scientific and technological mind to pursue such an objective. Part of the explanation, I would suggest, is a fear and loathing of human beings for being somewhat less than worshipful in face of truth, for being somewhat flawed, and for being somewhat slow in recognising the facts and figures wielded by the smarter folk among us.
Four hundred years B.C. Plato argued that there would be no end to our troubles until philosophers become kings. Plato’s philosopher-king is a ruler who possesses a love of wisdom, intelligence, reliability, a disinterested commitment to the general good, and a willingness to live a simple life. These are the rulers of the utopian city of Kallipolis. For such a rational polity to exist, "philosophers [must] become kings…or those now called kings [must]…genuinely and adequately philosophize" (Plato The Republic, 5.473d). Karl Popper saw in the figure of the philosopher-king the origins of totalitarianism, achieved through the combination of an “idealism” detached from real individuals and lived realities and "social engineering." Popper misunderstands Plato’s intent here. Plato was not writing a blueprint but elaborating the principles and conditions of good government in a well-tempered polity. Plato’s argument on the need to relate philosophy, politics, and people still stands. That said, Popper’s critique of a blueprintism also still stands. Oddly, whilst Popper criticizes both Hegel and Marx as totalitarians in descent from Plato, these two were themselves vehement critics of a blueprintism that imposed an a priori rationality on reality (and both were Aristotelians rather than Platonists. I would say that Rousseau and Kant were the Platonists here, both displaying an antipathy to Aristotle).
The combination of a detached idealism and social engineering in the cause of the perfect polity, or, in the first instance, in pursuit of the goal of creating perfect reasoning and calculating machines, is well under way in the contemporary world. Whilst most of us wrestle with the age-old dilemma of making politics and people more truthful, some very clever people have been working to circumvent the whole problem. If people can’t be made to see, respect, and live in truth, then the solution is to just create new beings who will do so, and will do so quickly. The most intelligent human beings among us are creating intelligent machines that will in time supersede and supplant the slow and the stupid. The only problem is that these new beings may well be so quick and clever as to supplant the most intelligent human beings too. In comparison with Superintelligent Machines, all human beings are slow and stupid. Creating machines which are quicker and smarter than we are may well be the peak in the ‘men as gods’ delusion, the first event in achieving god-like status. It will also be the last.
Humanity has had an awful long time to crack the problem of collective action, and we are still no further than the dream and delusion of the philosopher king as blueprint. Those who want a people united behind the truth and a political class who just ‘get on with’ enacting ‘it’ may well see their wish fulfilled. The only problem is that human beings are about to supplant themselves in cultural evolution.
Be careful what you wish for. People need to take time and thought to consider what, precisely, it is that they seek and strive. With every wish there comes a curse. We need to consider how the world will change should we succeed in realizing our ambitions. That is, if we fail to understand the nature of what it is we are really seeking, we may well realize the opposite of what we need. Truth is not merely an objective ‘thing,’ it is a process. The journey to truth is a process of self-discovery, the coming to be of humans as moral and rational beings. The view that human beings keep flunking their exams is leading to an impatience on the part of those who consider we have the scientific knowledge and technological know-how to do much better. There is an itch to circumvent the all-important process which brings truth to life in the human act. The process is therefore being eliminated as too messy and too tardy, as all-too human. Human beings are being circumvented in favour of intelligent machines who know, reason, evaluate, calculate and make the right choices and decisions. Isn’t that precisely what those who say that truth is non-negotiable and trumps all things really want? If human beings do not grasp the truth immediately, or as quickly as an intelligent machine can, then so much the worse for human beings.
The view may be seen as the logical extension of confinement within a Megamechanical civilisation based on imperatives and priorities which proceed without regard to persons. Human beings are being robotized and, to the extent that they fail to perform their pre-determined functions and fulfil their roles, are dispensable. Only the perfect are worthy of survival. Imperfect human beings are to be eliminated by the next phase in cultural evolution.
In Pythagoras’ Trousers, Margaret Wertheim argues for a more embodied science. She writes:
‘At the beginning of Mathematical Man's story in the time of Pythagoras, he was primarily an ethical being. One of the reasons he has lost his ethical grounding is that he has been largely without female company for so long.’
I would suggest that researchers secluded away in their laboratories and departments have been largely without human company as such, developing a tunnel vision that is self-validating and self-confirming. It is a self-idolatry which begins by making an idol of truth and ends in a worship of one’s own mediations of the truth.
Wertheim comments:
‘I believe we need a culture of physics that embraces, rather than tries to escape from, nature; a culture that values the embodied, and is not so obsessed with escaping into disembodied abstraction. As I noted at the start of this book, the problem with physics is not that its practitioners use mathematics to describe the world but how they use it and to what goals they apply it. There is nothing in a mathematical approach to nature that demands a fixation on a unified theory or on "transcendent" abstractions.’
I argue for the transcendent, but insist on an embodied truth, just as I argue for an incarnate spirituality. The obsession with a remote, disembodied abstraction is leading us to an ever further remove from our bio-ecological matrix. Wertheim writes that ‘Physicists today who equate a theory of everything with "the mind of God" not only ground their "theology" in science, they see themselves as some sort of priesthood in their own right.’ It is one short step from Wells' ‘men like gods’ to the ‘men as gods’ delusion, one well within technological reach. The supplanting of religion – and of flawed human beings – by scientific and technological man has long been within sight. As Einstein noted, ‘A contemporary has said, not unjustly, that in this materialistic age of ours the serious scientific workers are the only profoundly religious people.’ Those who consider religion to be a danger to health and sanity should try the absence of religion or a false religion. Human beings are difficult creatures balanced precariously between dream and deed. That’s the predicament and the challenge. It seems many are tired of it. Instead of cracking the problem, the ambition seems to be to remove it, removing people in the process.
It’s not only the ‘boffins’ who are hankering after the supplanting of imperfect human beings by perfect reasoning machines. A lot of supposed nature lovers seem to be less than impressed with human nature. I saw this article being shared across social media among ecologists of a certain hue. “Empty half the Earth of its humans. It's the only way to save the planet” by Kim Stanley Robinson. The article is based on E.O. Wilson’s book and plan Half Earth. The plan is one I can agree with. As one who has argued for an urban public sphere and a city planet, I find the argument and its logic cogent and compelling. I live in a small terraced house in the middle of rows of streets of terraced housing. We are snug, compact, and cosy. I like it and I like that it frees land from human habitation. It can be done.
My animus is not against the article and its idea, but against how many supposed ecologists didn’t even read the article but looked at the title, concluded that it was a demand to remove half the human population from the planet as such, and expressed their whole-hearted approval. One, a prominent author, even added ‘at least half.’
Well, stick around, we may yet remove humanity from the Earth, as an unintended consequence of our own brainy brilliance. In the past, these unintended consequences have been messes and occasional disasters for ‘society’ to clean up. The problem is that as our creations have gotten bigger in time, so the messes have also gotten bigger and the disasters for frequent and pervasive: financial and economic crises, pollution, climate change. The problems are so many and so wide that there has been a tendency to forget all about the threat of nuclear Armageddon, the previous preoccupation of all those who live daily in fear of the end of the world.
Human beings in a competitive society are forever in a race to be the first to do something before anyone else, monopolize the gains, go quicker, higher, better. Whatever can be done must be done, and those who advocate caution are dismissed on account of having an unreasonable fear.
In 1995, Stuart Russell and Peter Norvig wrote the book Artificial Intelligence: A Modern Approach. In the book’s final section, the authors finally posed the question that should have been posed at first, the question which needs to be pondered at length today: “What If We Do Succeed?” The section weighed the possibility of good and bad outcomes. Their answer was hardly a ringing endorsement of AI: “The trends seem not to be too terribly negative.” That answer does at least indicate that all may not be well with AI, even if it flags up the absence of the necessary critical reflection. The book, however, drew no firm conclusions either way. In Human Compatible, however, Russell does reflect deeply on the issue and this time expresses his concerns at the darker potentials of AI in the clearest of terms.
In his poem from 1967, All Watched Over by Machines of Loving Grace, Richard Brautigan envisaged a benign existence in a technological utopia in which machines improve the conditions of life and look after human beings. Instead of being in antagonistic relation, humanity and technology work in partnership to bring about the greater good. Brautigan writes about "mammals and computers liv[ing] together in mutually programming harmony", with technology acting as caretakers while "we are free of our labors and joined back to nature.” I read the poem as being in keeping with the humanisation of technology, as against the rejection of technology as necessarily anti-human and ecologically destructive. That represents a communitarian and humanist socialist alternative to the Megamechanical visions of the Cold War-era. Brautigan's publisher, Claude Hayward, said it "caught me with its magical references to benign machines keeping order ... [which] fit right in with our optimism over the promise of the computer." The magic of an order established and maintained by benign machines sounds more like a delusion and fantasy to me, investing machines with an existential power and responsibility that properly belongs to human beings. Re-enchanting the world and recovering the sense of the sacred is quite different from deifying our inventions and creations. The attainment of the good life depends upon the attainment of right relationships and the restoration of transcendent vision, human beings united in solidary structures and watched over by the loving grace of God.
According to Russell, a professor at the University of California, Berkeley, the AI promises not a world in which machines watch over us in loving attention and care, the very opposite in fact. Russell warns of the dangers of creating machines that surpass the human intelligence. In one respect, the greatest ever human achievement would also prove to be the last. Evolution is call and response. The scale of the crises confronting humanity have grown with cultural evolution until they now exist on a global scale. Humanity, in the shape of its nations, governments, international organisations, business leaders, and various associations, have so far proven incapable of scaling up to the size of the challenge. There is too much information to collate and evaluate, too many vested interests and stakes for unbiased judgement, too much institutional and psychological inertia to crack the problems and complexities of collective action. Marx thought that human beings would develop the collective wit and will to resolve the crisis tendencies and contradictory dynamics of capital in a socialist society based upon the cooperative mode of production. Every time we have moved in that direction, the result has been the realienation of power in a dead and abstract collectivism. Not only has the ‘global’ problem of the economy not be solved, it has been compounded by the ‘global’ problem of climate change. The institutional inadequacies of human civilisation are becoming all too apparent under the resulting stresses.
Finally, at last, humanity has developed machines with the capacity to gather in all the facts, evaluate and assimilate them without any problems of biases and stakes, and devise effective courses of collective action, without delay. In power, speed, and effectiveness, AI and its intelligence machines beat humanity hands down.
Russell notes that by 2010, “many people had finally begun to consider the possibility that superhuman AI might not be a good thing,” commenting that such people were mainly outsiders rather than being in the mainstream. It was about that time that I expressed my reservations in Of Gods and Gaia. But I am, unquestionably, an outsider. By 2013, Russell “became convinced that the issue not only belonged in the mainstream but was possibly the most important question facing humanity.”
If humanity is unable to control AI, then the danger is that AI will come to control humanity. In fact, I would say that that eventuality is much more a likelihood than it is a danger. If the language appears alarmist, then we need only consider Marx’s critical notion of alienation and note the extent to which human creations have already escaped the control and comprehension of their human creators, acquired an existential significance of their own, and control humanity through the imposition of their own imperatives over human choices and ends. I am thinking specifically of capital here, which few seem able to even name, let alone challenge, still less overcome. And now with AI there is a deliberate attempt to invest human inventions with intelligence and consciousness.
In November 2013, Russell gave a talk at the Dulwich Picture Gallery. After giving an explanation as to what AI was, he nominated five events for the “biggest event in the future of humanity”:
We all die (asteroid impact, climate catastrophe, pandemic etc).
We all live forever (medical solution to aging).
We invent faster-than-light travel and conquer the universe.
We are visited by a superior alien civilisation.
We invent superintelligent AI
These options very concisely summarise the problems, solutions, promises, potentials, and challenges confronting humanity in its social and cultural evolution.
Option one states the ‘global’ problem facing humanity, the converging crises threatening catastrophe and collapse if they are not resolved in time. Here is the need for collaborative structures and decision making in order to address the collective forces threatening to undermine civilisation.
Option two I see as the subconscious underpinning of mechanical progress, the idea that machines contain the promise not merely of Earthly Paradise but most of all of immortality and eternal life. The flesh corrupts, decays, and dies. The machine promises a world beyond mortality.
Option three tells us we can have the final conquest and mastery of nature and not have to relinquish control.
Option four is a hankering after a humanity that comes to respond to the scale of challenge it has set itself through its social and technological development. In the Preface of 1859, Marx wrote that: ‘mankind always sets itself only such tasks as it can solve; since, looking at the matter more closely, we will always find that the task itself arises only when the material conditions necessary for its solution already exist or are at least in the process of formation.’ That’s the challenge and response of evolution. The problems before humankind are capable of resolution. The problem is that humankind has been an awful long time in resolving them. Following Feuerbach, Marx thought God a projection of all the best qualities of human beings upon a supernatural being. These qualities were denied within prevailing social relations. The solution was to transform these social relations to enable human beings to reintegrate and exercise all their best qualities. Option four is a continued search for God, in the form of an alien superintelligence, but is really, in Marx’s terms, a call to human beings to grow into their full intelligence. ‘Reason has always existed,’ wrote Marx, ‘but not always in rational forms.’ Option four is a demand for this rational form on the part of humanity.
Option five puts all of the above together.
Russell therefore ‘suggested that the fifth candidate, superintelligent AI, would be the winner, because it would help us avoid physical catastrophes and achieve eternal life and faster-than-light travel, if those were indeed possible. It would represent a huge leap – a discontinuity – in our civilisation. The arrival of superintelligent AI is in many ways analogous to the arrival of a superior alien civilisation but much more likely to occur. Perhaps most important, AI, unlike aliens, is something over which we have some say.’
The questions, though, are many and deep: do humans have some or any say over AI? If so, how much say and in what way? And which humans have the most say and which the least? AI promises the alienation, exploitation, and iniquity of the capital system writ large. For Hegel and Marx, alienation was a positive and progressive process to the extent that it indicated an initial loss that could later be redeemed at a higher stage of development. AI seems to be an irrevocable process entirely lacking in redemptive possibilities. Russell notes that whilst the arrival of a superior alien intelligence on Earth would cause pandemonium, the concern with respect to the arrival of superintelligent AI has been … ‘well, underwhelming begins to describe it.’ Russell makes up for the muted response by describing the significance of superintelligent AI as follows:
‘Success would be the biggest event in human history … and perhaps the last event in human history.’ (Stuart Russell, Human Compatible: AI and the Problem of Control).
Russell now declares himself ‘publicly committed to the view that my own field of research posed a potential risk to my own species.’
Raised on Karl Marx and alienation, Lewis Mumford and the Megamachine, and Jacques Ellul and technological terrorism, I don’t need to be warned as to the threat that uncontrolled and uncomprehended human inventiveness poses to human life. A succession of Popes have warned of the dangers of technical capacities coming to run ahead of and outstrip moral capacities, and have suffered condemnation as reactionaries for their concern. All of the above were wise in a way that those with a naïve faith in technology as delivering automatic progress were not. I see the biggest threat, though, in a strange combination of two contradictory things in the contemporary age. There are those who still express that boundless faith in technological progress, cleaving to the old delusion of ‘men as gods’ creating an Earthly Paradise through the technological mastery of the world. Such people deify humanity through technology. Then there are those who are just plain exhausted and seem to welcome the coming eclipse of humanity and civilisation. ‘The sooner the better,’ one put it to me. ‘The human species is generally despicable,’ said another. Such people denigrate humanity and welcome its demise grace of its own technological invention.
We are living in a strange age indeed, an age of imbalance and extremes, with optimists and pessimists drifting out to places beyond recall. It is an age of self-estrangement.
There are now tens of thousands of the smartest people in the world now at work building AIs. Most of these are working on the “narrow” AIs that focus on specific tasks, such as processing speech, translating languages, recognizing faces in crowds, and such like. The ultimate goal of the field, the real breakthrough which threatens to eclipse humanity, pertain to those general purpose AIs that first match and then supersede the broad-based brainpower of the human species. In effect, the most intelligent members of the human species are applying their intelligence to the goal of creating an AI superintelligence that will overtake human intelligence, with whatever consequences follow as a result. That sounds like a species at the end of its tether and seeking a rational way to end it all.
It’s not only possible, it has all been identified as a conscious goal and ambition. DeepMind, the AI group owned by Alphabet, declared its goal from the start as being to “solve intelligence” and, on that basis, solve everything else. Vladimir Putin declared that whoever becomes the leader in AI ‘will become the ruler of the world.’ That is a remarkably complacent view that fails to appreciate how the invention of superintelligent AI changes all the rules of the human world. AI superintelligence will be the new ruler of the world.
Reassuringly, that nightmare vision doesn’t seem to be too close. Russell lists the formidable problems confronting computer engineers as they seek to create human-level AI. It’s quite a challenge to be human, knowing how to turn words into coherent, reliable knowledge, learning new actions and ordering them appropriately, manage cognitive resources to reason, evaluate, and make good decisions fast. Russell speculates that such challenges and more will occupy the best minds for at least another 80 years, but states that it is impossible to be precise on the timing.
I take Russell’s book as a warning to human beings to get their skates on and make the breakthrough to the next stage in social and cultural evolution, that of a collaboration and cooperation in which all things are ordered to their true ends.
Russell is in no doubt that, whatever the various apocalypses competing to end the world at present, the creation of superintelligence is a threat to the long-term survival of humanity. He writes:
‘A system that can both discover new high-level actions … and manage its computational activity to focus on units of computation that quickly deliver significant improvements in decision quality would be a formidable decision maker in the real world. Like those of humans, its deliberations would be “cognitively efficient,” but it would not suffer from the tiny short-term memory and slow hardward that severely limit our ability to look far into the future, handle a large number of contingencies, and consider a large number of alternative plans.’
And to think that someone, referring to the interlocking financial, economic and environmental crises, once asked me ‘what is your strategy?’ It was as if he expected me to be superintelligent AI machine. Ridiculous, I know. I’m not quite that good, and need a little help on one or two things. But that question revealed to me something about the character of many of those seeking solutions to environmental problems, by way of their technocratic mentality. It revealed to me that, instead of looking at all the areas I emphasise - politics, ethics, emotional intelligence, imagination, narrativity, vision - they look for something hard and quantifiable, something they can systematize, something operational. They are looking for the things that computers can already do easily, whilst neglecting the distinguishing features of humanity, which computers currently struggle with. One day, the computers will catch up. And if humanity continues to fail to integrate all its faculties within a cooperative collective framework, it will be eclipsed.
Russell asks how the system would work and behave should we put together everything we know how to do with all the potential new developments. The resulting system would
‘plow through time, absorbing vast quantities of information and keeping track of the state of the world on a massive scale by observation and inference. It would gradually improve its models of the world (which include models of humans, of course). It would use these models to solve complex problems and it would encapsulate and reuse its solution processes to make its deliberations more efficient and to enable the solution of still more complex problems. It would discover new concepts and actions, and these would allow it to improve its rate of discovery. It would make effective plans over increasingly long time scales.
In summary, it’s not obvious that anything else of great significance is missing, from the point of view of systems that are effective in achieving their objectives.’
I would suggest to those who - without, mark you, ever actually troubling themselves to read let alone understand the texts I have spent years slaving over - have harangued me for plans of action and strategies that this is strategy enough. That Superintelligent Machine fits the demands for rational calculation and action precisely, without any need to bother with the disappointingly recalcitrant human material. It’s neither me nor any human being that such people need. They want a perfectly calculating and reasoning machine that makes all problems go away and removes the need for personal moral, intellectual, and political effort. It’s a form of hygiene. As one of life’s awkward squad, on the outside of perfectly functioning systems looking in, I bid these people goodbye and good riddance. To think that I was still in cardiac rehab when a number of these demands were being pressed on me, by people who knew my history and my work, and ignored both to press their misguided mindsets on me. It strikes me as very plain and obvious that such folk, hankering after clever, clinical strategies, are merely mirroring the inhumanism of the system they claim to criticize, or the nature they have turned into an idol. Deep down in the subconscious, they have given up on humanity and really want a Superintelligent Machine that absorbs vast amounts of information from all sources, can access it in an instant, evaluate it, select it according to requirements, and obtain a far more sophisticated understanding of the world and a far more effective power of decision than any human being could hope to achieve, not even me.
With AI personal assistants, Russell writes, “you would, in effect, have a high-powered lawyer, accountant, and political advisor on call at any time.” I would very much expect assistants of this calibre to become managers in short order.
And all of this is the optimistic assessment of the possibilities before we come to the potential misuses of AI. With respect to misuses, most obviously, advanced AI would invest the extraordinary powers of surveillance, persuasion and control in government as to make the Stasi ‘look like amateurs.’ Entitling this section ‘the automated Stasi,’ Russell begins with the description of the Stasi as ‘one of the most effective and repressive intelligence and secret police agencies to have ever existed.’
He notes that the intelligence agencies have spotted the potential for using AI in their work, with respect to voice recognition and identification of key words and phrases in speech, but these are mere simple forms compared to other potential uses. Increasingly, AI systems are able to understand the content of what people are saying and doing, in whatever medium. ‘In regimes where this technology is adopted for the purposes of control, it will be as if every citizen had their own personal Stasi operative watching over them twenty-four hours a day.’
Again, what Russell presents as a warning can seem a promising development to those who seek from politics and people no more than that they conform to ‘truth’ and ‘reality,’ with no creative input from those sources other than assent. How else to ensure ‘the masses’ conform to climate truth than by the micro-managing of every aspect of their behaviour? All human actions have an environmental impact. If these are to be controlled and monitored for the general ecological good, then what better than a Superintelligent Machine, imposing upon humans the behaviours they cannot be trusted to follow by their own free choice? The technology for such surveillance is already available:
‘Even in the civilian sphere, in relatively free countries, we are subject to increasingly effective surveillance. Corporations collect and sell information about our purchases, Internet and social network usage, electrical appliance usage, calling and texting records, employment, and health. Our locations can be tracked through our cell phones and our Internet-connected cars. Cameras recognize our faces on the street. All this data, and much more, can be pieced together by intelligent information integration systems to produce a fairly complete picture of what each of us is doing, how we live our lives, who we like and dislike, and how we will vote. The Stasi will look like amateurs by comparison.’
Not surprisingly, Russell’s next section is entitled ‘Controlling your behaviour.’ And again, I have to ask, isn’t this precisely the task that humanity is being charged with undertaking? I ask this question because my work offers a deep, systematic, and consistent criticism of a liberal society characterised by a libertarianism in politics, society, ethics, and culture. Human beings are free to pursue the good as they see fit. Such a society generates collective problems that it is incapable of resolving. The collective consequences of these uncoordinated choices are accumulating and taken overwhelming and seemingly unresolvable form. Every attempt to achieve concerted and comprehensive action about at the governmental realm and self-control and rational restraint in personal relations is resisted and rejected as an infringement on individual liberty. In every issue and every area, attempts to inculcate and institute ‘rational freedom’ are countered with the assertion that the individual is free to do as s/he pleases. The result is the chronic failure to develop the collective wit and will necessary to address and resolve the ‘global’ issues confronting humanity. I have argued that point for years now. I have had some influence over on Academia, where I have a strong and loyal readership, but for the rest I may as well have written in hieroglyphs. The smarter people want a calculating machine, the rest are engulfed in lamentation. All is see is a despair in humanity. In the section on ‘Controlling your behaviour,’ Russell writes that ‘once surveillance capabilities are in place, the next step is to modify your behaviour to suit those who are deploying this technology.’ He goes on to state that once the system finds something, it will enter into correspondence with you to ‘coerce behaviour,’ ‘if the goal is political control.’ I think this is the crudest end of control. With respect to employment, whilst we can no longer supply routine physical or mental labour, we can still supply our humanity. “We will need to become good at being human,” writes Russell.
I had always thought that being good at being human, becoming human, was the aim anyway. I have pursued this through a religious ethic, through Aristotle’s flourishing, and Marx’s commitment to a truly human society. Russell quotes Keynes here:
‘It will be those peoples, who can keep alive, and cultivate into fuller perfection, the art of life itself and do not sell themselves for the means of life, who will be able to enjoy the abundance when it comes.’
Russell argues that ‘all of us need help in learning “the art of life itself.”’ Quite. See my comments above and my work in ‘rational freedom’ in general.
The only thing worse than a society-destroying AI is a society-destroying AI that won’t switch off. Russell spends a lot of time on the problem. I don’t doubt that many AI researchers will abuse the good name of the Luddites to dismiss such claims. The much abused Luddites are the smartest people of all in these controversies, seeking a society in which technology was used for social ends and the human good.
I’ve never been remotely impressed by threats of a sudden and dramatic end of the world. I remember watching Kenneth Clark’s Civilisation when I was young, and its lessons stayed with me. I heard Clark repeatedly say that the thing that makes a civilisation most of all is confidence, and the thing which brings a civilisation down most of all is the loss of internal confidence:
‘Of course, civilisation requires a modicum of material prosperity— What civilization needs: confidence in the society in which one lives, belief in its philosophy, belief in its laws, and confidence in one’s own mental powers. The way in which the stones of the Pont du Gard are laid is not only a triumph of technical skill, but shows a vigorous belief in law and discipline. Vigour, energy, vitality: all the civilisations—or civilising epochs—have had a weight of energy behind them.’
Civilisations decline and fall from within. They lose internal coherence and confidence. Kenneth Clark thus warned that cynicism and disillusion were as great a threat to civilisation as war and invasion. It is the gradual loss of confidence that kills a civilisation. The greatest danger facing humanity and civilisation is a loss of meaning, direction, and purpose, a loss of vitality, a progressive enfeeblement and enervation and loss of effort and ambition. The foundations of civilisation slowly corrode, with human beings lacking the drive, the energy, the imagination, the courage, and, indeed, the confidence to strengthen them and rebuild anew.
Russell refers to the ‘one trillion person-years learning and teaching’ that the more than one hundred billion people who have lived on Earth have spent to ensure the continuation of civilisation. Up until the present day, the only possibility for this continuation ‘has been through re-creation in the minds of new generations.’ This, he states, is now changing: ‘increasingly, it is possible to place our knowledge into machines that, by themselves, can run our civilisation for us.’
‘Once the practical incentive to pass our civilisation on to the next generation disappears, it will be very hard to reverse the process. One trillion years of cumulative learning would, in a real sense, be lost. We would become passengers in a cruise ship run by machines, on a cruise that goes on forever.’
Russell mentions the film WALL-E here. The fate he describes here, however, is the one exactly envisaged for humanity by E. M. Forster in his 1909 novel The Machine Stops.
In this book, Forster describes a world in which humanity has lost the ability to live on the surface of the Earth, each individual now living below ground in isolation in a standard room. All bodily needs met by the omnipotent, global Machine, with communication proceeding via a kind of instant messaging/video conferencing machine. There is a world outside of the Machine, but the Machine threatens those seeking escape with 'Homelessness': expulsion from the mechanical environment and, therefore, a presumed death. As with Marx’s alienation thesis, human beings forget that they are their creators of the Machine and instead come to serve and, through the religion of "Technopoly", worship it as a mystical object whose demands take priority over human needs.
"The Machine," they exclaimed, "feeds us and clothes us and houses us; through it we speak to one another, through it we see one another, in it we have our being. The Machine is the friend of ideas and the enemy of superstition: the Machine is omnipotent, eternal; blessed is the Machine."
Those who refuse to accept the deity of the Machine are condemned as 'unmechanical' and threatened with Homelessness. In time, the Mending Apparatus, which is charged with repairing defects that appear in the Machine, comes to fail. The problem is that belief in the omnipotence of the Machine is now so ingrained that concerns with system failure are dismissed. And the inhabitants of the passive, sterile, mechanical environment have become so used to the Machine delivering everything they need that they have lost the natural ability, spontaneity, creativity, and curiosity to identify problems and act to resolve them. The human inhabitants are so entirely compliant within the operation of the Machine that they see the initial defects as mere passing quirks, of no fundamental significance. But as the Machine continues to malfunction and the defects accumulate, the worsening situation becomes impossible to ignore. The problem is that human beings have lost the knowledge of how to repair the Machine and therefore live in the mechanical environment below. They have also forgotten how to live on Earth above. So subservient are human beings that when the Machine collapses, 'civilization' collapses with it too. Too late it is realised that the source of collapse lies in the ever greater removal of humanity from its natural conditions of existence, and that the responsibility now lies with those few who have remained dwellers on Earth to build anew on firmer foundations, guarding against the error of mechanarchy and megamechanical abstraction.
"They wept for humanity, those two, not for themselves. They could not bear that this should be the end. Ere silence was completed their hearts were opened, and they knew what had been important on the earth. Man, the flower of all flesh, the noblest of all creatures visible, man who had once made god in his image, and had mirrored his strength on the constellations, beautiful naked man was dying, strangled in the garments that he had woven. Century after century had he toiled, and here was his reward."
Forster, The Machine Stops
The objection that Russell puts in the mouth of a good consequentialist sounds very much like the disbelief that the inhabitants of Forster’s Machine expressed at the first signs of deterioration: ‘Obviously this is an undesirable consequence of the overuse of automation! Suitably designed machines would never do this!’
It is the reformist cry heard throughout the decades. ‘The system’ can never fail, the problems being experienced are merely accidental features that can be reformed and regulated away. That response gives evidence of the idolatrous hold that The Machine has over human thought and initiative.
Russell asks us to think as to what this means. Machines may well understand that autonomy and competence are important aspects of how human beings seek to conduct their lives, and insist that humans retain control and responsibility for their own well-being. But a tragedy of the the commons comes into play here:
‘for any individual human, it may seem pointless to engage in years of arduous learning to acquire knowledge and skills that machines already have; but if everyone thinks that way, the human race will, collectively, lose its autonomy.’
Humanity is therefore charged with resolving the fundamental problem of politics upon which I premised this piece. Humanity develops the institutional, moral, and social conditions to support a collective wit and will, or it fractures through a prevailing libertarianism. Russell’s conclusion brings into the ‘rational freedom’ I have spent the past quarter of a century writing on:
‘The solution to this problem seems to be cultural, not technical. We will breed a cultural movement to reshape our ideals and preferences towards autonomy, agency, and ability and away from self-indulgence and dependency – if you like, a modern cultural version of ancient Sparta’s military ethos.’
Russell is on the right lines, but there are much better ways of conceiving the rational polity than this. In my doctoral work on ‘rational freedom,’ I argued for this polity in terms of a ‘modern polis democracy.’ As for the rest of this passage, I can only refer people to my work as an attempt to integrate autonomy, authenticity, and authority so as to reconcile individuality and sociality as two aspects of the same human nature. The main point here is that Russell is right, the solution is cultural, to the extent that it takes place in the field of practical reason (politics and ethics, of which economics is conceived once more to be a branch).
Russell argues that this requires ‘human preference engineering on a global scale’ along with radical changes in how our society works. That latter transformation is key, but receives no treatment here. I would also argue for Marx’s notion of praxis in which social transformation and self-transformation coincide, setting this within the context of appropriate modes of conduct, character formation, cultivation of the virtues, and communities of practice, so as to offer an internal alternative to the technical and manipulative implications of ‘human preference engineering on a global scale.’ Moral reason educates desire, moderates behaviour, and achieves self-control from the inside, not the outside. Russell’s concluding passage presents the case for the internalisation of the rational ideal and its assimilation into everyday relations, character, and practice that I make with reference to the society of ‘rational freedom.’ The morally educative purpose of the rational ideal succeeds to the extent that, in time, external constraint gives way to an internal self-restraint:
‘Any parent of a small child is familiar with this process. Once the child is beyond the helpless stage, parenting requires an ever-evolving balance between doing everything for the child and leaving the child entirely to his or her own devices. At a certain stage, the child comes to understand that the parent is perfectly capable of tying the child’s shoelaces but is choosing not to. Is that the future for the human race – to be treated like a child, forever, by far superior machines?’
I would have Kant answer here:
Enlightenment is man’s emergence from his self-incurred immaturity. Immaturity is the inability to use one's own understanding without the guidance of another. This immaturity is self-incurred if its cause is not lack of understanding, but lack of resolution and courage to use it without the guidance of another. The motto of enlightenment is therefore: Sapere aude! Have courage to use your own understanding!
Kant, What is Enlightenment?
My various texts on Kant elaborate on the Kantian vision of ‘rational freedom.’
Russell ends by asserting confidently that humans will not be pets or zoo animals, whilst admitting that ‘there is really no analog in our present world to the relationship we will have with beneficial intelligent machines in the future.’ The reference to ‘beneficial’ rather begs all the questions raised above again.
I answer them at length in my substantive writings.