A new arms race: the battle for attention

For years, it has been primarily social media algorithms that determine what we talk about, what we see and what we think. But in the battle for our attention, a new front has recently emerged: artificial intelligence. What does this mean for the reliability of our information? And can we still turn the tide?

British researcher Carl Miller was taken aback when he found himself embroiled in a discussion with a friend in a café. His friend made some rather remarkable statements about the war in Ukraine and reached for his phone to prove his point. Not to Google anything, but to consult ChatGPT.

To Miller’s astonishment, the chatbot appeared to agree with his friend on several points. For instance, it claimed that Ukrainian President Zelensky was involved in illegal cryptomining and was therefore responsible for his country’s energy problems. As a source, ChatGPT cited the Foundation to Battle Injustice, a website that presents itself as a human rights organisation but which, according to Miller, is a notorious Russian tool for information warfare.

How is it possible that an AI system takes such a source seriously, wondered Miller, who has been researching digital influence and information warfare for fifteen years? And what happens if more and more people worldwide use AI’s answers to form their opinions?

Information as a weapon

The use of information as a weapon of war is nothing new. For centuries, regimes have sought to mislead their own populations and opponents through propaganda and misleading reports about troop movements. But according to Miller, something fundamental changed at the turn of the century: information was no longer merely a tool for waging war, but a battlefield in its own right. Alongside land, sea, air and space, a new domain emerged: the information environment.

And within it, human attention is the most precious asset. After all, producing disinformation is one thing; ensuring that people actually see it is quite another. Most influence operations fail because they fail to capture the attention of their target audience.

With the advent of social media, that battle for attention has already changed dramatically. This was particularly true once algorithms began determining which posts we see and which we do not. Truth and relevance are secondary in that process: what matters is how much emotion certain content evokes. Posts that further stoke existing frustrations, fears and mistrust are particularly lucrative: the more attention, the more money for Big Tech.

Alongside land, sea, air and space, a new domain emerged: the information environment.

A new gold rush

With generative AI, this information war is now taking on a new dimension. More and more people are using chatbots as sources of information, shopping assistants or even as digital romantic partners. This has opened up a whole new arena for influencing people and their views.

Miller and his team investigated how Russian disinformation finds its way into AI responses. They collected fifty claims from the aforementioned ‘human rights organisation’, the Foundation to Battle Injustice, formulated various questions based on them, and put these to different AI models.

The outcome was worrying: around one in five of the AI responses examined confirmed or reinforced the Russian claims. Only 17 per cent completely debunked them. Whilst this study focused solely on these specifically selected claims, the results do demonstrate how vulnerable AI systems can be to deliberate manipulation.

A new market has now emerged in the tech industry: Generative Engine Optimisation (GEO). Companies use these techniques to ensure their products appear as frequently and favourably as possible in chatbot responses, just as they used to improve their search engine rankings with SEO. But according to Miller, the same techniques are now also being used to promote political messages and disinformation.

A nuclear arms race

According to Miller, no solution to this problem is to be expected from the technology companies themselves in the short term. They are caught up in a commercial arms race: no company wants to be the first to lose users’ attention whilst the competition continues to optimise its systems to retain users for as long as possible. Miller compares this to nuclear disarmament. Unilaterally giving up one’s weapons is unattractive as long as the other side continues to arm itself.

Paul Verhagen, a researcher specialising in technology and geopolitics at the Hague Centre for Strategic Studies (HCSS), points out a crucial difference compared with the nuclear arms race. Nuclear weapons are primarily in the hands of states, whilst the digital systems that inform us on a daily basis are largely developed by commercial companies. Their main motivation is not the welfare of citizens or the protection of our democracy. They want to make a profit.

And we are still ill-equipped for a world in which commercial companies determine a large part of our information supply. According to Verhagen, we are operating with political ideas from the eighteenth century, an economic system from the nineteenth century and political institutions from the twentieth century, whilst power over information is shifting at breakneck speed in the twenty-first century. We have learnt to limit state power. But who limits the power of commercial platforms that are increasingly determining what people see, think or believe?

Who holds the power?

In his book The Techtonic Shift, Verhagen describes how technology has transformed from a unifying force into an instrument of geopolitical power struggles. He also outlines various ways of dealing with this. The United States is focusing primarily on commercial innovation in AI and aims to develop this as quickly as possible. China regards information more as something that should be state-controlled and grants the government significant control over the information environment. And Europe is still seeking a way to protect citizens from manipulation without infringing on freedom of information.

We are operating with political ideas from the eighteenth century, an economic system from the nineteenth century and political institutions from the twentieth century, whilst power over information is shifting at breakneck speed in the twenty-first century.

According to Verhagen, Europe should in any case become less dependent on foreign technology companies. In the defence sector and other vital infrastructure, digital dependence is already seen as a security risk. But perhaps our information systems are so important that Europe needs to gain more control over them, argues Verhagen. Miller also advocates regulation: clear rules for all technology companies. He compares it to disarmament agreements: these only work if they apply to everyone.

It would help if more scientific evidence were to emerge soon regarding the social harm that algorithms can cause. Miller therefore calls on researchers to get to work on this. And quickly, please, because time is running out.

The battle for the truth

According to Verhagen and Miller, science therefore has an important task ahead of it. Research into how algorithms and AI systems work is indispensable, but so too is research into the consequences for human behaviour, political power and democratic institutions. It is precisely by combining technical knowledge with insights from the social sciences and humanities that researchers can contribute to solutions.

In addition, Miller wants reputable knowledge institutions to start working with GEO themselves as soon as possible. GEO for Public Good must become a counter-movement in which universities, journalistic organisations, museums and other public institutions ensure that their verified information is more easily discoverable by AI systems. They must, however, get involved in this new battle for attention. If they remain on the sidelines, they will leave the playing field to parties for whom the truth does not matter, but who have a different agenda.