Colophon

Editing: Michiel Boreel, Evert de Vries and Annelies Gallagher
Cover and interior design: Frank August
Typesetting: Mirjam Kroondijk
Production: Peter ten Hoor, LINE UP boek en media bv, the Netherlands

ISBN 9789493170681
ISBN 9789493170698 (ePub)

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. The full licence terms are available at creativecommons.org/licenses/by-nc-nd/4.0/legalcode.

Introduction by Andrew Keen

Real Fake, a book written by a trio of distinguished Dutch techno-futurists, asks three big questions about ‘reality’ in the age of artificial intelligence. Firstly, how does ‘synthetic media’, the latest technology to come out of Silicon Valley, enable the faking of reality? Secondly, what is the future of an increasingly smart digital technology that empowers internet users to create their own personalized version of ‘reality’? And thirdly, what are the dangers of synthetic media that allows us to play with online content to the extent that we can no longer distinguish between what is real and what is fake?

But Real Fake will also ask why, or at least, why now? As a serial tech entrepreneur and writer of non-fiction, I know one thing for sure: timing is everything. And in the age of TikTok, Synthesia, Epic Games, VR glasses and the myriad number of synthetic media tools that allow us to make our own digital versions of reality, this book is as timely as anything you’ll read this year. Real Fake is anything but fake. For all its elegant flights into techno-futurism, it is a reassuringly realistic text that will make sense of right- and left-wing conspiracy theories, Covid-19 hoaxes, purported sightings of alien spacecraft and the other memetic imaginations of early-twenty-first-century ‘conspirituality’.

In the manner of a timely start-up, Menno van Doorn, Sander Duivestein and Thijs Pepping have written a book that is just ahead of the curve; neither too early nor too late, Real Fake is the right work by the right team at the right time. Their probing questions have been on the minds of many of the most prescient writers in our technology-saturated world – from techno-novelists like Kazuo Ishiguro and techno-memoirists like Sherry Turkle to techno-philosophers like Jonathan Rauch. The questions concerning the danger of synthetic media addressed in Real Fake are the three great existential issues in today’s age of increasingly sophisticated artificial intelligence.

Describing these concerns as ‘existential’ – meaning that they might endanger our existence, both as individuals and as a species – isn’t an exaggeration. These really are painfully existential questions for those of us who fear that the undermining of epistemological certainty might represent a crisis for twenty-first-century humanity. After all, if we don’t know what’s ‘real’, if we can’t separate digital fakery from digital truth, what can we know for sure about anything?

Indeed, if our online ontologies – the word philosophers use to describe our perception of reality – are as malleable as a TikTok video or a virtual human created by Synthesia, then aren’t we all being sucked into a philosophical black hole, in which everything is concerned with appearance and facts are unimportant? If online fakes are presented as the truth, Western philosophy has returned to where it began in Athens two and half thousand years ago – to the metaphorical cave in Plato’s Republic, and we have once again become illusion-addicted troglodytes.

Imagine what would happen if we were unable to distinguish between real and fake human beings in this updated version of Plato’s epistemological prison. Kazuo Ishiguro’s 2021 novel Klara and the Sun imagines a world in which humans have outsourced their empathetic intelligence to smart machines. In Ishiguro’s not-too-distant future, machines seem to know us better than we know ourselves – or maybe we know our smart machines better than they know themselves. In any event, the existential crisis that Ishiguro sees on the horizon is the struggle to distinguish between ‘fake’ and ‘real’ humans. Playing with reality and outsourcing our humanity to machines, the Nobel-prize-winning author warns, could be a deadly game for our species; it might even be our ‘final invention’.

Machines replacing humans is an old trope, of course – at least in science fiction – but it’s a scenario that appears less and less speculative as we invent smart machines that can learn, labor, think, drive and talk. The most intriguing question is what robots won’t be able to do in today’s age of artificial intelligence.

Sherry Turkle, professor of social psychology at the Massachusetts of Technology and author of books including Life on the Screen, Alone Together and Reclaiming Conversation, has spent her life studying the impact and limits of artificial intelligence on humans. In her 2021 memoir The Empathy Diaries, Turkle suggests that while we may be able to teach digital humans to learn, labor, think, drive and even talk, these robots cannot be programmed to learn to be intuitively empathetic. For Turkle, it is the power of empathy that distinguishes real humans from their smart artificial fakes. It is this, the MIT sociologist and psychologist asserts, that will maintain our humanity in an age of artificial intelligence.

Menno van Doorn, Sander Duivestein and Thijs Pepping share Turkle’s skepticism about the ability of robots to display empathy, and I suspect they would add two other qualities to her list of human qualities that even the smartest robots cannot fake. The two characteristics that are unique to Homo Sapiens, the authors of Real Fake would say, is our ability to tell stories and to play games, and these are the twin anthropological truths that drive the book’s narrative. We are storytellers and game-players, Van Doorn, Duivestein and Pepping tell us. That is our essence.

In today’s age of artificial intelligence, a time when our thinking machines threaten to make us redundant, there is a pressing need that we tell stories about who we really are. In order to get to the hows and whats of our synthetic media age, we need a compelling narrative about mankind, and the first part of Real Fake is an anthropological exploration of the human condition. Weaving together the ideas of thinkers as seemingly unrelated as the French sociologist Jean Baudrillard, the Israeli technology historian Yuval Noah Harari, the Canadian social theorist Erwin Goffman and the Dutch historian of play Johan Huizinga, our authors identify their own ideal version of mankind, which they dub Homo Syntheticus. In our age of artificial intelligence, they argue, we are all storytellers and game-players.

In their 1846 essay titled The German Ideology, Karl Marx and Friedrich Engels invented the anthropocentric concept they called ‘species-being’ – a uniquely human characteristic, they believed, which would be realized through the disruptive technology of industrialization. Van Doorn, Duivestein and Pepping’s Homo Syntheticus continues this tradition of viewing technological progress as being both a cause of and affected by human improvement, and our authors quote the twentieth-century American media scholar John Culkin. ‘ We become what we behold. We shape our tools and then our tools shape us.’

In the age of TikTok and VR glasses, our authors assert, we simultaneously shape our digital tools as well as having those synthetic tools shape us. This age of synthetic media has the potential, they explain, to enable us to realize our real – or perhaps more accurately, our fake – natures. When we own the tools to invent our online realities, they argue, we are simultaneously becoming Yuval Noah Harari’s ‘post-reality species’ and Johan Huizinga’s Homo Ludens.

As ‘reality builders’, everyone can now be both a storyteller and a game-player. Technology democratizes the narrative and gaming arts, unlocking our potential to fully realize our innately human creativity. Empowered by the newest technologies coming out of Silicon Valley which enable us to, quite literally, Be John Malkovich or to Bend It Like Beckham, we can all be both playwrights and players in what Erving Goffman regards as life as a perpetual theatrical production. Synthetic media, then, is unlocking a new synthetic form in humanity: the ever-malleable, hyper-real Homo Syntheticus. On our new digital stage, anyone can be anything.

However, our authors warn that this radically democratized spectacle in which we all simultaneously produce and star might be regarded by tech critics as an Agatha Christie-style ‘whodunnit’. The crime at the heart of our conspiritual age is the death of truth – the murder of the Enlightenment ideal of a single scientifically verifiable reality. It was that bright ontological truth that Plato contrasted with the flickering images inside his dark cave.

In our conspiratorial age, there are, of course, no lack of Hercule Poirots out there, philosophical sleuths investigating what appears to be the tragic death of truth. In his 2021 book The Constitution of Knowledge: A Defense of Truth, the American political philosopher Jonathan Rauch argues that the preservation of both representative democracy and individual liberty rely upon a common respect for the conception of truth. And Rauch warns that American democracy can’t survive today’s digital epistemological disruption. When everything is regarded as equally conceivable, Rauch warns, there is a danger that our democratic edifice will collapse in on itself.

Jonathan Rauch might say that some things, such as American democracy, are more important than game-playing or storytelling. But I wonder whether Rauch – and many other critics of the apparent hostility for empirical facts on both the left and right of American politics – might learn something from reading Real Fake. As Van Doorn, Duivestein and Pepping remind us, we are Homo Syntheticus, and our power depends as much on creating and believing fictions as on ontological certainty. So, for example, when more than half of Republican voters tell pollsters that they believe the 2020 US presidential election was ‘stolen’, I find myself wondering if they really believe this story to be empirically ‘true’. Instead, by bending events to conform to their own idealized narrative, they are, instead, playing games with reality.

Technology does offer an intricate mirror on our socio-economic and political circumstances, and I wonder if there is a deeper historical reality that mirrors today’s reinvention of reality. Jonathan Rauch is correct to connect the Enlightenment ideals of scientific truth and representative democracy, but it’s no coincidence that both these pinnacles of modernity have been achieved under the aegis of the nation-state – a singular political structure designed, perhaps, as both a cause and an effect of Rauch’s notion of a singular scientific truth. In the digital twenty-first century, however, as both the myriad forces of globalization and localization gnaw away at the nation-state, that enlightenment ideal of truth is becoming detached from its moorings. It shouldn’t be surprising that – in today’s multi-cultural, multi-ethnic, multi-sexual and even multi-gendered realities – truth itself is being simultaneously fragmented and enriched.

It would also be wrong to assume that all roads involving Homo Syntheticus inevitably lead to hateful, anti-social fantasies. As Van Doorn, Duivestein and Pepping argue, there are many ‘restoration stories’, in which synthetic media improves the twenty-first-century world. Perhaps its greatest quality, they suggest – borrowing ideas from progressive thinkers as diverse as the Italian economist Mariana Mazzucato, the crusading British environmental journalist George Monbiot and the Serbian performance artist Marina Abramović – is that it allows us to escape reality and rethink the very nature of things. For example, virtual reality technology that liberates us from our own bodies and allows us to float above the physical world will empower us to imagine what Mazzucato calls ‘moonshots’ for confronting our dual existential crises of global warming and economic inequity.

It’s not just the world that can be improved by synthetic media. Our new digital tools – what the authors of Real Fake playfully describe as ‘reality as a platform’ – can also train us to be better people. You’ll remember Sherry Turkle’s argument that while smart machines can be taught to learn, labor, think, drive and even talk like humans, these robots can never be programmed to be empathetic and to sympathize with others. But here lies another of the great uses of synthetic media. As Van Doorn, Duivestein and Pepping remind us, the use of virtual reality technology allows us to be put in somebody else’s shoes. For example, they cite the VR movie Travelling While Black, which enables us to experience what it feels like to be black in a historically racist society, thereby enabling us to empathize with oppressed people.

Finally, let me add one speculative idea of my own. I began this introduction by noting that Real Fake was a multi-levelled triptych: a narrative written by three authors asking three questions about the fate of reality in the age of artificial intelligence. While this is a book of trios and trinities rather than twins and doubles, I also noted that, in their definition of Homo Syntheticus, Van Doorn, Duivestein and Pepping suggest that we humans, as storytellers and game-players, have two essential qualities.

In conclusion, let me add a third characteristic to Homo Syntheticus. Yes, our authors may be correct that we are natural storytellers and game-players, but there’s something else too, a condition that may be antithetical to the first two qualities. You see, for all our love of stories and games, we also seek certainty and authenticity; we long for the truth and want to know ‘reality’. That accounts for the birth of Western philosophy in Plato’s metaphorical cave in ancient Athens. And that quest for the truth is also the foundation for the world’s three monotheistic faiths which, even as our early-twenty-first-century networking tools make us more malleable, remain powerful creeds among billions of our fellow humans.

So how will this longing for the genuine article affect our digital world of real fakes? I think we are going to see increasingly creative attempts to manufacture reality out of the intrinsic fakery of our digital revolution. Think of it as the analog squaring of the digital circle. The authors of Real Fake touch on this in their discussion of the philosopher Luciano Floridi’s concept of ‘ectypes’. And, as Van Doorn, Duivestein and Pepping note, we are seeing the birth of the authentic digital copy. Known technically at a ‘non-fungible token’, an NFT creates an original version of online content, a kind of digitalized Platonic form, a singularly scarce product with potentially significant monetary value.

As I write this, Tim Berners-Lee, the British physicist who invented the hyperlinked World Wide Web while working at the CERN research facility in Geneva in 1989, has just sold the Web’s original source code as an NFT for $5.4 million. In exchange for this significant amount of cash, the buyer received the time-stamped files of Berners-Lee’s source code, the original digital copy of a massively important historical document. While the digital version of Berners-Lee’s files looks no different from its copies, this ‘unique’ NFT is worth $5.4 million because of our appetite for the authentic, our longing for truth and the value we place on a single reality.

That, in a single paradox, captures the complex fate of ‘reality’, which you are about to read about in this timely book by a trio of Dutch techno-masters. So, Homo Syntheticus – all you storytellers, game-players and truth-seekers – enjoy this rich triptych. I have no doubt that your enjoyment will be real rather than fake.

Preface

We are increasingly experiencing reality through our screens. We have business meetings with our colleagues over Zoom every day, while our iPads and mobile phones constantly remain within easy reach. Trend watchers talk about the ‘screenification’ of society, and with good reason; it is hard to imagine a significant event in which screens would not play a key role. If we recall the storming of Washington DC’s Capitol in January 2021, we will remember seeing rioters armed with smartphones taking selfies and streaming video footage on social media as they rushed up the stairs and into the heart of American democracy.¹ We followed the shocking events on television, perhaps while sitting on our sofa and indignantly texting our friends or reacting furiously on Twitter. At the same time, the far-right Proud Boys and other groups sympathetic to the cause were rejoicing on Parler and demanding that the coup continue.² While we were watching open-mouthed, our children might have been pleading to be allowed to purchase digital Gucci sneakers that they would then ‘unbox’ on Roblox or Fortnite. And then, when the last Zoom meeting of the day is over, we will sit down and binge-watch the latest drama series on Netflix or Amazon Prime. With our second screen always at hand, we will be live-tweeting or browsing our Instagram feed while we watch.

Our screen also provides support when we are searching for answers to essential life questions. Not surprisingly, in early 2021, Google announced that the word ‘why’ had been searched more than ever in 2020.³ It is little wonder that in a year when the entire world’s population was constantly being tested by a global pandemic, people found themselves consulting the search engine more often than ever. After all, nothing is more human than the desire to know ‘why’. ‘Why is Bitcoin worth so much?’ ‘Why are people demonstrating?’ ‘Why is it called Covid-19?’

We might type the last question into Google and click return. On the first page of results, we will see ten suggestions, one of which will be a video of know-it-all Professor of virology Jaap Goudsmit⁴. At the start of the Covid-19 pandemic, we saw this respected scientist make a sober statement in which he gravely said that we knew very little about this new virus. It was almost like we were listening to the Spanish waiter Manuel from ‘Fawlty Towers’ say, ‘I know nothing.’

Dissatisfied with this answer, you might scroll down Google’s list of suggestions and click on the home page for Wion, a media outlet based in New Delhi that claims to ‘deliver information that is not biased’. That sounds more promising. A pulsing red button is on the screen, enticing us to click on it. When we do, we will find ourselves watching a live news broadcast. After following this apparently ‘unbiased’ news coverage for a while, we might return to our Google search. Advertisements for mask manufacturers are scattered among links for articles with headlines like ‘Why Covid-19 is here to stay’ and ‘Why Covid-19 is our equity check’. The World Health Organization suggests that we sign up to a messaging service about coronavirus that will send us updates about the pandemic via WhatsApp, while other links take us to Facebook and encourage us to ‘like’ them. We end up watching part of a TED Talk in which someone explains that she scores an eight out of ten on a list of pandemic experts before telling her story. She’s saying you can trust her, although you might wonder if that score is sufficient.

Anyone who wants to form an informed opinion about current events will undoubtedly end up in this turbulent storyworld, perusing a random mix of links to a TEDTalk video, plain text on a website, live news footage or a WhatsApp interaction. Reality is to be found in the overlap between the actual events, their presentation in the media and what our brain makes of it, but what we do not always realize is that the facts are presented to us in the form of stories. Behind each screen a world of stories, in raw, exaggerated or spectacular form is hidden. Life does not take place in objective reality but rather in these storyworlds. And in these storyworlds, organizations, institutions and citizens all play an important role. It is here – not in so-called reality – where brand reputations are made and broken and opinions about important world events as well as trivial matters are formed.

The screenification of reality

In this already turbulent storyworld things are about to change. Thanks to the development of cutting-edge artificial intelligence, any image, sound or text can be molded to suit our desire. These emerging ‘synthetic media’ will guarantee more spectacle, more imagination and more seduction. As a result, the storyworlds in which we spend our lives will look different, and this will have important consequences for organizations, institutions, politics and our personal lives. Media revolutions (like any revolution) have the tendency to turn the world upside down. This new ‘media revolution’ comes soon after the previous social media revolution that turned the simple ‘Tech’ into ‘Big Tech’, and has already progressed much further than many people realize. The Silicon Valley venture capital company founded by the internet pioneer Marc Andreessen and Ben Horowitz, a16z, is betting heavily on this new development. They call this new media revolution ‘social+’, a combination of social media with the additional use of artificial intelligence.⁵ With that, the computer becomes able to generate manipulated video and voice of celebrities at will, a face of a non-existent person, or convincing footage of events that never took place. In short, the technology emerges that is allowing us to play with reality and shape it according to our wishes. Examples are science-fiction-like virtual worlds⁶ where popstars perform, apps that allow you to manipulate images and video at comparable quality level of a Hollywood studio and deepfake videos that are indistinguishable from the real thing.

Andreessen and Horowitz are smart investors who want to take advantage of the possibilities that exist in what is commonly referred to as the ‘synthetic media revolution’. In this book we share our conviction that synthetic media and the possibilities that come with them, offer a golden future. Opportunities abound for meaningful or entertaining new applications, including some that can help us heal the shortcomings of the revolution and recover from doubt and pessimism. Today, the concern that these changes have a negative effect on how we perceive reality is frequently heard – technology is so advanced that fake can no longer always be distinguished from real. Post-truth is evolving into post-reality. As the new media develop in a way that allows everyone to play with reality, can we still trust the images that we see on our screens?

Our four questions for Google

If you typed the question whether we can trust our screens to Google, the search engine would suggest an infinite number of web pages with all sorts of content. But if you instead let us be your search engine for a moment, we would like to look at the four questions we asked ourselves in writing this book and give some preliminary answers.

We should start with the most fundamental question: Is the distinction between ‘real’ and ‘fake’ fading? Our answer is that it is not so much that the distinction is fading, but a continuum of nuances is emerging between the two. As human beings we are constantly playing with reality and value judgements like ‘real’ or ‘fake’ change during the game. We will immersive ourselves in the motivations that drive human behavior and the cognitive and emotional strategies that lay beneath. Notions like ‘reality’, ‘playing with’ and ‘real & fake’ need to be clearly defined before we dive in.

Dear Google, is the distinction between ‘real’ and ‘fake’ disappearing?

Part 1 of this book attempts to answer this question. Our conclusion is that the desire to play with reality has nothing to do with modern developments in technology. Deep within all of us is a playful human being, a so-called ‘Homo Ludens’. And it is this innate playfulness that make these synthetic media so suited to us. They allow us to do what we do naturally: play with the facts. They present super-powers to our playful nature, which convinces us of the golden future for these technologies.

Humans are generally skilled in distinguishing between fake and real and over time we will understand the new nuances that are emerging in between. Most important conclusion however is that for us as a post-reality species, synthetic media offer a glorious candy store of opportunity.

The other side of the medal is that sharing a common truth becomes less obvious. The many conspiracy theories that have emerged during the Corona pandemic make this perfectly clear once again. That is why a more fundamental question arises from Part 1:

Dear Google, why are facts so often defeated by fiction?

Part 2 points out that filter bubbles are part of the broader phenomenon of reality bubbles where any interaction with someone from a different bubble seems doomed to fail. Conspiracy theories mingle with New Age ideologies creating Conspirituality. Knowledge resistance and survival heuristics give many the idea that our brain is constantly failing, but it turns out it does exactly what it’s supposed to do: find arguments and open the dialogue. We are social, story-telling creatures, and we need each other to survive. The tools we use for this are changing dramatically as artificial intelligence moves into the realm of text, sound, image, and game creation. Not surprisingly, the follow-up question is future-oriented:

Dear Google, how are synthetic media being used to tell new stories?

Part 3 opens with the economic impact of these new stories. According to narrative economics, a concept by Nobel laureate and economist Robert Shiller, new ways of storytelling will also create new dynamics in the economy. In many ways, the synthetic media stories penetrate the reality bubble in which we live. Therein also lie their qualities: attractive new ways of storytelling that are perfectly attuned to our way of perceiving. From stories in which you can completely immerse yourself to lightning-fast memes, and from the democratization of stories to digital people who act as storytellers. The question then arises whether we will still look at the world realistically in the future? Our last question is therefore about the danger, and about desirable paths for the future:

Dear Google, how do we restore the relationship with reality?

Part 4 assesses the dystopian perspectives of the future and offers utopian remedies. Countless books, documentaries and internet speeches have warned us about the dangers posed by synthetic media, and here we will examine these opinions. Deepfakes will be an inevitable result of technological development, but there’s a great deal more at stake. By examining the situation from different angles, we will shine an uncomfortable light on our relationship with the media, concluding that the most crucial thing is that we are aware of how these technologies are advancing and how to govern them. If they are guided by the right framework, whether that be reality ethics, digital happiness or something else, synthetic media are a powerful tool that can have a positive impact. Like technology more generally, they should serve people – and if they do, they can contribute to a better world. With the digital happiness of humanity in mind and aware of the complex issues facing us, we will end this book on a positive note and look forward to a brighter future.

Contents

Cover

Colophon

Introduction by Andrew Keen

Preface

Part 1 Playing with Reality

Intro Part 1

Chapter 1 Reality

The reality game

Synthetic media versus reality

From Hollywood to the home

Reality is malleable

The objective reality and the subjective perception

Synthetic reality stimulates imagination

The game has begun

Conclusion

Chapter 2 The Birth of the Game

The Shed at Dulwich

Humans are a post-reality species

Homo Ludens, Sapiens, Deus, Syntheticus

The life game

Two kinds of games

Conclusion

Chapter 3 Fake, Real and New Nuances

A real fake copy

Fake news does not exist

Six types of false information

Synthetic media as a curse and blessing for democracy

Conclusion

Part 2 The eternal battle between fact and fiction

Intro Part 2

Chapter 4 Conspirituality and Alternate Reality Games

The second man on the moon

‘Conspirituality’ in a disenchanted world

Knowledge resistance

Alternate Reality Games

The truth as a game

Conclusion

Chapter 5 The Origin of a Conspiracy

How the ball keeps rolling

From LARP to ARG to MOAC

The Plandemic and the Infodemic

Understandable and healthy distrust

Conclusion

Chapter 6 The Reality Bubble and Common Sense

The brain does not fail, but does exactly what it is supposed to do

Distressing cases and disastrous effects

The reality bubble and our storyworlds

The role of synthetic media

Conclusion

Part 3 The Future of Synthetic Media

Intro Part 3

Chapter 7 The Narrative Economy

The dematerialization of the economy

Digital scarcity

New revenue models

The Creator Economy

The Metaverse

Conclusion

Chapter 8 Synthetic Media as Meme Machines

We see reality through our own virtual reality glasses

Synthetic stimuli and the game played with us

Meme Machines

Synthetic storytellers

Reality as a platform

Conclusion

Chapter 9 Synthetic People Who Tell the Story

From human-machine to human-human interaction

Digital Humanization

The existential factor

The many faces of digital storytellers

The six virtual humans that have crossed the uncanny valley

The living brand

The next step in humanization

In closure

Conclusion

Chapter 10 Hollywood Productions from Your Living Room

Hollywood through a filter

Beat Hollywood with deepfakes

How do you learn the deepfake trade?

AI as ‘creator’ of fakes

A never-ending story

The toolkit continues to expand

Distinguishing real from fake

In closure

Conclusion

Chapter 11 In Synthetic Media You Are The Story

Creating new memories

The emotion+ experience

The You-ser experience

The customer journey will be a media production

Sharing synthetic experiences in games

Sharing synthetic experiences in the office

In closure

Conclusion

Part 4 Restoration Stories and Media Awareness

Intro Part 4

Chapter 12 Our Reality Tools in Media-Rich Life

Looking through the lens of technology

Our built-in viewer

Media literacy and awareness

The story as an instrument

Finally: the brief history of our reality

Conclusion

Chapter 13 Real-Fake: The Murder of Reality

Media literacy and our armed brain

Reality is dead and buried

Amused to death?

Media shapes culture

The collapse of the information ecosystem

We are a post-reality species – and that matters

The murder of reality is a call for morality

Conclusion

Chapter 14 Big Tech, Small People

The reality business

Jaron Lanier’s Cyber-Amageddon

The restoration story

Conclusion

Chapter 15 Happiness- and Wellbeing-Based Reality Ethics

The story of digital happiness and well-being

Digital happiness and well-being

Reality ethics

Restoration stories of hope: More moonshots, please!

Notes

Epilogue

Thanks to these experts

About the authors

Part 1 Playing with Reality

Intro Part 1

Dear Google, is the distinction between ‘real’ and ‘fake’ disappearing?

Chapter 1 Reality

In 1902, the photographer Levin Corbin Handy published a photograph of Ulysses S. Grant, commanding general of the Union forces during the American Civil War (1861 – 1865) and later the eighteenth President of the United States. In the photo, Grant is on horseback outside the Union army headquarters at City Point, about 125 miles south of Washington, DC. But what isn’t clear from the picture to the casual viewer is the way in which Handy lived up to his name. Because, in fact, the background is from a different photo altogether, as is the horse and the rider’s body, with Grant superimposed on top. In short, the iconic photo is actually a combination of three different photos, all of which were taken at around the same time, in 1864. In reality, the moment that is depicted never happened. The Library of Congress, where the photo is kept, notes that ‘this photo is a composite of several images and does not actually show General Ulysses S. Grant in City Point.’ It represents an important moment in photographic history, as the first photo that we know for sure to have been edited, over a hundred years before Photoshop. Of course, such editing now seems to be a standard procedure, and one which might even be done unknowingly with an AI tool that automatically improves such properties as the level of color saturation. There might also be more conscious manipulation nowadays – ‘digital surgery’ might add a smile, remove wrinkles or adjust features. Or, as Handy did, other elements might be added to the original image.

Photo: General Ulysses S. Grant at City Point, the first Photoshop.

More than a century after this first manipulation of a photograph, editing of images, film and sound is the latest trend in the rapidly unfolding new media revolution.

The reality game

The birth of social media is a relatively recent development – we might look at it as a sixteen-year-old adolescent, barely mature and certainly not yet fully grown.¹ At the beginning of the media revolution we could not have imagined that it would lead us ‘playing creatively with reality’. Initially, we simply thought that a level playing field would be created offering equal chance to all. Media power which had until then always been concentrated in the hands of the mass-media companies, was suddenly shared with users. The new game revolved around conversations. This first phase began with the emergence of the first social media companies – Friendster in 2002, MySpace in 2003, LinkedIn in 2003, Facebook in 2004 and Hyves in 2004 – and continued until Apple’s US launch of their iPhone on 29 June 2007. There was, at this point, a deep-rooted belief that these internet conversations would be a primarily democratizing force. After all, everybody had the chance to participate and be heard.

In 2008, we published a book titled Me the Media. At that time, the conversations about social media were typically optimistic but tinged with disbelief. We constantly vacillated between asking ourselves the vulgar ‘What good does it do for me to know what my former high school friend ate last night?’ and loftily thinking that ‘Markets become conversations’, as the Cluetrain Manifesto led us to belief. During the symposium that we organized to coincide with the book’s publication, when we asked the audience who was on Twitter, only three out of the hundreds of participants raised their hands.

Me the Media was taught as part of university curricula and was runner-up in a competition for the Marketing Book of the Year. We were writing about the transition from a mass media to a ‘media-mass’ where everyone has a voice and can make themselves heard. At the time we were already amazed that there were 100 million people on Facebook, and social media like Hyves and Second Life were also booming. Today, the online game Fortnite alone is played by 350 million people, with the global domination of social media going from strength to strength. In 2020, Facebook had 2.4 billion users, YouTube had 2 billion, WhatsApp 1.6 billion, TikTok 800 million, Instagram 1 billion and Reddit 382 million. Worldwide, people spend an average of almost seven hours on the internet every day, of which 2.5 hours is taken up by social media. 500 hours of video content is uploaded to YouTube every minute, compared to just ten hours in 2008. And this information revolution is not just leaving a mark on how we live our lives, but also on the financial markets. The estimated value of YouTube is 150 times higher than it was in 2008 ($300 billion now compared to $2 billion then), Facebook is worth fifty times more than it was in 2008 ($752 billion now versus $15 billion then) following investment by Microsoft. And ever since the Chinese doctor Li Wenliang gave a warning on 30 December 2019 about the threat of a global pandemic, the value of the Big Tech companies has exploded even further, as the change in their share price indicates. In addition, new companies that are at the basis of the so-called Creator Economy are emerging as valuable players. Coinbase, Patreon, Cameo, OnlyFans, and OpenSea have amassed billions of market value.

| June 30 2020 | June 30 2021 | Growth Percentage
Alphabet | 0.962 | 1.673 | +74%
Facebook | 0.647 | 0.982 | +53%
Apple | 1.563 | 2.268 | +45%
Microsoft | 1.541 | 2.037 | +32%
Amazon | 1.382 | 1.741 | +31%

Market value of Big Tech in trillions $ (1 trillion is 10¹² = 1,000,000,000,000)

The first smartphones rolled off the production line in 2007, with 122 million being shipped that year and 139 million more manufactured in 2008. In 2020, that number will have risen to more than 1.5 billion a year. In 2008, we would have been surprised by the suggestion that when people woke up in the morning, the first thing they did was reach for the phone to check their social media. Now most people do that without shame. In 2010, we published The App Effect, a book that examined the addictive nature of endless timelines, notifications and social media-related ‘FOMO’. With the introduction of the iPhone, our eyes turned inwards and looked at ourselves, both literally and figuratively. Before we knew it, selfies seemed to have taken over, with visual culture slowly but surely overtaking the written word. In 2012 – the same year that Kodak, regarded as the most innovative company of the last century, forever closed its doors – Instagram was bought by Facebook. It quickly became the chosen app of a new profession: the influencers where it is all about gathering as much attention as possible. With the advent of the smartphone, the speed with which we were bombarded with information exploded. It always has our attention – everything appears on the screen and we always have the device close at hand. A year later we wrote The Dark Side of Social Media, a report about the adverse side effects of social media, of which addiction to mobile phones was a major factor.

In recent years, the tide has turned, and the consequences of social media are becoming increasingly well known. Whether in our personal lives, in terms of our reliance on our smartphones, in politics, where platforms such as Facebook have been seen to influence elections or in Big Tech’s increasing influence on our economies, it is increasingly clear that decisions regarding technology have an ethical dimension. We have, in recent years, focused on such issues. We wrote four reports with the title Digital Happiness, in which we suggested how we can harness technology for the well-being and happiness of mankind. Human happiness is playing an increasingly big role when we think about technology, from ‘positive’ and sustainable technology to ethical AI. This book is the sequel to Me the Media, but also to Digital Happiness. Technology in general – including the technologies that are subject of this book – is not something that just ‘happens’ to us. We can actively steer it in directions that we desire and govern its effects and consequences. Technology offers the possibilities, but it is us as a society that makes the choices.

In the last twelve years, there have been developments of unprecedented magnitude that have, in many areas, changed how we look at the world and how we look at technology. The world has been completely taken over by new media, with the end of that march not yet in sight. In fact, the Corona pandemic has only accelerated this trend. What was foreseen for 2030 was forced to be realized in 2020. The ‘real’ economy of airlines and oil companies are beginning to lose out to the new media organizations, the ones that are behind Instagram stories, the TikTok videos and the funny memes that go viral and spread around the world like wildfire. In twelve years, a radical different economic and social media reality has emerged, with new conventions and associated human behaviors.

To illustrate: the stock values of KLM-Airfrance (€2.66 billion on June 30, 2021) and Easyjet (€4.84 billion are dwarfed by the estimated value of $28.7 billion of Epic Games, maker of the internet game Fortnite. It is clear that media has come to control society in an unbelievably short period of time. At the end of August 2020, the composition of the Dow Jones Industrial Average was changed,² with companies that had been included since time immemorial, making way for relative newcomers. The oil giant ExxonMobil, for example, part of the index since 1928, was among the companies that were removed, with companies including the cloud software provider Salesforce.com being added. The Carlyle Group, a global investment company, has calculated that the appreciation of the S&P 500, an index that measures the performance of 500 large American companies, comes mainly from digital platforms, software and intangible assets. This contribution is five times bigger than that of material assets and traditional relationships, according to A Wall Street Journal article with the headline ‘Pandemic Hastens Shift to Asset-Light Economy’.³

This so-called ‘asset-light economy’ relies heavily on the information ecosystem that has been built up rapidly in recent years. The starting point for the transformation of this system occurred in 2017, when the word ‘deepfake’ was used on the Reddit internet forum for the first time, to describe the technique of pasting someone else’s head into a video. This marked the start of a new era, and numerous other synthetic media options have since emerged. This is the ‘+’ that has been added to social media. Social+ is all about malleable realities.

Synthetic media versus reality

As soon as artificial intelligence is used in either creating or editing images, video, sound or text, we call it synthetic media. And within this category, the creation of deepfakes is drawing most attention. Video manipulation is especially growing in popularity; several apps offer users the opportunity to appear as a leading actor in a Hollywood blockbuster. Although the deepfake apps might often seem to be little more than funny gimmicks, deepfakes are increasingly used by advertising agencies – for example, to avoid having to fly celebrities halfway across the world, with celebrities’ faces being superimposed on the bodies of local actors. A deepfake of Bruce Willis has been used for a series of commercials for Russian phone carrier and internet provider MegaFon. Willis reprises his role as John McClane from the Die Hard movies while speaking perfect Russian. In return, Willis doesn’t need to travel and receives a compensation of probably around $1-2 million.⁴ Another sign of synthetic media becoming a serious business is the use of AI-generated influencers being employed for marketing purposes. Lifelike holographic models serve as, for example, mannequins on e-commerce platforms, and the application of AI is also exploding, in virtual reality, video gaming and the film industry. Artificial intelligence is also used in the music industry, whether that involves writing music from scratch or completing songs that a musician has started. Other examples of synthetic media have less honest intentions; in some of them, there is a conscious effort to fool the viewer, and the consequences can be very harmful. One striking example of this is the use of deepfakes in pornography⁵, the so-called ‘deepnudes’ where somebody’s face is superimposed on the body of a pornstar.

Synthetic media are defined as digital informational artifacts produced through manipulation or creation of data and media by artificial intelligence, such as images, sound, text, video and virtual experiences.

In practice, synthetic media are already ubiquitous. The cameras on our smartphones use artificial intelligence to focus on people’s faces, to automatically adjust the color saturation of our photos and to stabilize video footage. On Zoom video conferencing you can smooth out your wrinkles with the click of a button, and in a Microsoft Teams meeting, AI can adjust your background. The audio feed of these meetings is automatically filtered so that background noise or the rustling of your colleague eating a bag of chips is no longer audible, while her voice is.⁶ We might call such things the ‘light’ type of synthetic media. Deepfakes that are indistinguishable from the real thing, computer-generated art, virtual reality worlds and interactive virtual humans are the ‘heavy’ variants.

From Hollywood to the home

If we zoom out and take a broader view, we might regard the development of synthetic media as the fourth media revolution. The invention of the printing press was the first, followed by the invention of electronic media – radio and television – and then the digital revolution with the web and mobile devices. Now we witness a new type of media emerging, with automated creation and manipulation as its trademarks. We are entering a new era, in which the creation of content is no longer merely controlled by the film industry and editing studios – with the help of artificial intelligence, anyone can get creative. Synthetic media editing is the standard on social media platforms such as TikTok and Snapchat as well as in the professional Adobe software suite; the result is that creativity has moved from Hollywood to the home, giving many more people the opportunity to create and share new realities. Every media revolution affects the zeitgeist. The previous three revolutions were dominated by the democratization of knowledge and the acceleration of information distribution. This fourth revolution, by contrast, revolves around the democratization of creativity and malleability, with everyone now able to adjust reality to their liking. And this has created new ‘storyworlds’, such as TikTok, Fortnite and virtual humans on Instagram. We can even see the beginning of a fifth media revolution which revolves around so-called crypto media. By using blockchain technologies, people can buy, sell and own a piece of internet culture. Dogecoin, Cryptopunks, EtherRocks and the Bored Ape Yacht Club (a NFT community based on avatars of bored apes with unique expressions, outfits and accessories⁷) are all examples of this new type of media. By building communities (fandoms) around these digital assets, people start creating their own narratives, their own storyworlds and even complete transmedial universes⁸ that have the potential of becoming a ‘decentralized version of Disney’.

Reality is malleable

Before the invention of the camera, we had to make do with other types of illustrations like drawings or text descriptions; if we go far enough back, we get to clay tablets and hieroglyphs. We see synthetic media as representing a fundamental change in terms of what we can believe, but the relationship between what we read in the media and what actually happened has always been at odds with each other. Anyone who studies the hieroglyphs of the Battle of Kadesh, for example, would conclude that the Egyptian pharaoh Ramses II, who ruled between 1290 and 1224 BC, achieved a glorious victory. In reality, he narrowly escaped being captured by the Hittites, however tourists still marvel at his glorious acts of heroism. Or take the ‘Great Moon Hoax’. In 1835, readers of The New York Sun were told about the existence of aliens that lived on the moon. Detailed black and white drawings showed strange creatures, a cross bred between humans and bats, bathing, flying and talking near a lunar riverbed. The newspaper falsely attributed the discovery of this extraterrestrial life to Sir John Herschel, one of the most famous astronomers of the time. This ‘fake news’ went down a storm, and did no harm to the newspaper’s sales figures. It was not discovered to be a hoax for several weeks after its publication and, even then, the newspaper did not issue a retraction.

The proof that an event actually happened and guidance concerning how we should interpret it often comes to us through the media, whether that’s a hieroglyph, a drawing, a photo or a tweet. But as we have already seen, reality can be malleable, whether it involves airbrushing your face or printing misleading stories to encourage newspaper sales; this is all part of media history. The trend for fake news that reached a climax at the end of the nineteenth century was known as ‘Yellow Journalism’,⁹ and was followed by the revival of newspapers that kept fact and fiction as separate as possible. People had had enough of fantasy stories such as the ‘Great Moon Hoax’ and longed for more realistic news reports. We saw a similar phenomenon during the presidency of Donald Trump. The subscriptions of newspapers like The New York Times and The Washington Post skyrocketed as they set up entire fact-checking departments. Apparently, man longs for more realistic news coverage to be able to keep a better view of reality. But this prompts the question: what is ‘reality’?

The objective reality and the subjective perception

The subject of ‘reality’ has caused endless philosophers to engage in intellectual debate over the years, and there are various theories regarding what it is. These vary from solipsism, in which it is believed that there is only one consciousness (that would be yours, dear reader), to quantum physics, in which whether reality is observed or not is the most crucial thing, to theories in which all forms of matter ultimately are forms of consciousness. Anyone who wants to contribute something significant to our understanding of these issues should be quite modest – it is, after all, a very crowded field – but the reason for our involvement in the discussion is the increasing influence that the ‘world of pixels’ has on our perception of reality. We see this, for example, in the relationship between social media and the belief in conspiracy theories or alternate truths. The culmination of this came during the storming of the Capitol in January 2021, when a large crowd of demonstrators was so convinced that President Trump had been deprived of victory in the presidential election that they invaded the building while screaming for the heads of Nancy Pelosi, Speaker of the House, and Mike Pence, Trump’s vice president. But other new phenomena such as the ‘snapchat dysmorphia’ – the desire to have your appearance surgically adjusted to better resemble your filtered social media image – put a different spin on the question of reality. It is a constant quest for words to express our changing perspective of reality. Terms like ‘mixed’ and ‘extended reality’ have emerged, blurring the boundaries between physical and synthetic. Augmented and virtual reality have, on the other hand, been around for a bit longer. The concept that we want to introduce here is that of synthetic – malleable – reality.

In our definition of this new term, we allow ourselves to be guided by the classic dichotomy: that of objective and subjective reality. To put it bluntly, objective reality is what is actual, meaning that it is there even when we are not, while subjective reality is what happens in our brain. Physicists speak of objective reality, which is also known as ‘time-space’ and refers to everything that takes place in a space within a certain period of time. In this physical world, objective reality is everything of substance, from mountains, rivers and bridges to people, trees and animals.

It seems helpful to approach the digital world in a similar way, treating synthetic reality as a second type of objective reality. It also concerns space and time that exist beyond the subjective, at least in some sense. We ‘place’ documents in Microsoft Teams and we ‘meet’ in Zoom, with our actions continuing to have an effect over a period of time. It also doesn’t matter if we forget about them or stop believing in them; the virtual environments will continue to exist. We focus on the artificial malleability of this new form of reality. Hence the term ‘synthetic’: artificially made. In synthetic reality, the ‘malleability’ of public space is much more straightforward. Anyone who wants to construct a cycle path in the physical space, for example, must take into account the various laws of nature, from gravity to erosion. They might also, if they are not careful, get into trouble with the authorities. However, in synthetic reality, the laws of nature do not apply, and it is much easier to ‘build’ something or to add or adjust something that someone else has built. Synthetic reality runs on computer processing capacity and bandwidth and is made up of pixels. Though there are no laws of nature, there are still ‘things’, artifacts made by humans, such as music, videos, text, images, and collaboration platforms. All those things are virtual. You can fly through the air or, if you select the right Snapchat filter, look like Johnny Depp. And you can always return and experience the exact same thing all over again. We see that the laws of physics are being defied.

Just as the physical environment shapes us – whether you grew up in the countryside or a big city – we are also shaped by our synthetic environment. This applies more than ever to the younger generations, the Millennials and Generation Z, who perceive reality through the lens of the new media from an early age. Generation Alpha, the generation that is still being born, will be the first generation for whom there is no longer a distinction between the virtual and the real world; between the fake and the real; it will all be equally real to them. They are the product of visual culture, receive news differently, enter into relationships differently, such as via Tinder, and they are used to collaborate in a distributed manner. The things we build shape us. Or as John Culkin, a professor at Fordham University and a close friend of the Canadian philosopher Marshall McLuhan, once said,

‘We become what we behold. We shape our tools and then our tools shape us.’

Subjective reality