The Governance of Privacy

Privacy as Process:
The Need for Resilient Governance

Hans de Bruijn

Amsterdam University Press

Cover design: Gijs Mathijs Ontwerpers, Amsterdam

Lay-out: Crius Group, Hulshout

ISBN 978 94 6372 967 3

e-ISBN 978 90 4855 612 0

NUR 740 | 980

© H. de Bruijn / Amsterdam University Press B.V., Amsterdam 2022

All rights reserved. Without limiting the rights under copyright reserved above, no part of this book may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the written permission of both the copyright owner and the author of the book.

Table of Contents

1 The Radical Transformation of Privacy

1.1 Introduction

1.2 Privacy as a process (I)

1.3 Type 1: privacy invasions as disclosure – continuous, ubiquitous, and emergent

1.4 Type 2: privacy invasions as profiling

1.5 Type 3: privacy invasion as manipulation

1.6 Type 4: privacy invasion as a collective problem

1.7 Privacy as a process (II): comparing the four types of privacy invasions

2 The Complexity of the Governance Challenge

2.1 Introduction

2.2 Victims and their many cost–benefit analyses

2.3 The object: data and the many uncertainties concerning data use

2.4 The spaghetti-like network of data-producing devices

2.5 The villain comes in many guises

2.6 Distribution cannot be distinguished from production

2.7 The addressee as victim and villain

2.8 The essence of the differences between then and now

2.9 The complexity of the governance challenge, summarized

3 An Introduction to Governance

3.1 Introduction

3.2 State, market, and society

3.3 The context and the inevitable unintended effects

3.4 The dynamics of underlying norms

3.5 When to use instruments: upstream or downstream?

3.6 Resilient governance

4 The Power of the State

4.1 Introduction

4.2 From prescriptive- to goal-based regulation

4.3 From substantive regulation to procedural regulation

4.4 From imposed to negotiated regulation

4.5 From direct to indirect regulation

4.6 From instrumental to institutional regulation

4.7 The essence of resilience-based regulation

4.8 Conditions for regulation based on resilience—capacity and infrastructure building

5 The Power of the Market

5.1 Introduction

5.2 Competing on privacy

5.3 The pricing and taxing of data

5.4 Challenging the business proposition and, therefore, the business model

5.5 Creating barriers in the data-journey—breaking up companies

5.6 The power of the market and resilience

6 The Power of Society

6.1 Introduction

6.2 Empowerment upstream

6.3 Patterns: locking-in and locking-out

6.4 Empowerment downstream: how can a user weaponize against locking-in and locking-out?

6.5 Resilience and the role of government

7 Reflections

References

1 The Radical Transformation of Privacy

1.1 Introduction

Let me start this book with a confession: I have had my doubts concerning the title The Governance of Privacy. Why?

In our data-driven world privacy is, on the one hand, a cardinal theme. On a daily basis, the data we share enables others to keep track of what we are doing, to discover our preferences and beliefs, and to understand the essence of who we are. A familiar example is the number of Facebook “likes”: the more likes a user gives, the better Facebook knows you—perhaps better than your colleague, better than your roommate, even better than your parents or children, and, in the event of 300 likes, maybe even better than your partner.1 The next step seems only logical: the algorithm knows us better than we know ourselves. The same happens with data trails we leave on search engines: they allow others to fathom our identity.2 Once others are knowledgeable about who we are, they may use this knowledge in either a benevolent or malicious manner. Hence, privacy matters, and so does the governance of privacy.

On the other hand, the concept of privacy already existed in the old, pre-Web world, though had a more particular meaning: ‘the claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to others’ (I cite this definition from a 1967 discourse on privacy).3

Pre-Web definitions of privacy often contain three core elements: (1) individuals, (2) the control of their personal information, and (3) the right to be left alone.4 However, in a data-driven world, privacy and the violation thereof have gained an entirely different meaning:

– Invasions of privacy entail more than just disclosure. In the world of data, algorithms, and artificial intelligence, data can be used to profile, intimidate or manipulate individual users. Profiling may also concern groups, which means that privacy violations do not only lead to individual damage, but also to collective damage. I will elaborate on this phenomenon later in this chapter.

– Privacy breaches are often data-based, while in the pre-Web world they mainly concerned information. A single data point often comes without any meaning5—data characterized as ‘know-nothing’.6 This may have a host of effects that are relevant to governance—for example, the mere fact that users perceive data as meaningless dots can make them less alert to possible privacy invasions.

– When data is being collected, a service is often rendered in return, such as medical care, online education, tailored search results. These services are often free of charge, and users pay with their data. The more that users value these services, the more privacy becomes part of a complicated trade-off, an essential difference from the pre-Web world. In the pre-Web world, privacy was about restricting the flow of information; in a data-driven world privacy is about ensuring that the information ‘flows appropriately’.7

– Privacy is an all-encompassing concept. It involves everybody, and it affects almost anything—nearly everything in life has an online aspect. In the old world, the problem of violations of privacy often had a more limited scope. The issue of privacy has taken such a quantum leap that, at least from a governance perspective, it has become fundamentally different from what it once was.

These observations explain my initial discomfort with the title of this book. Using an old concept to describe a new phenomenon might be misleading. Privacy has developed beyond the pre-Web meaning of ‘individual’, ‘control’, and ‘leaving alone’. However, the concept of privacy has become so commonplace that I cannot evade using it. But let us be clear: privacy is a much more layered concept than it was in pre-Web times. In this chapter, I will peel back these layers. Section 2 contains a few introductory remarks about privacy. In Sections 3–6 I will introduce four types of privacy invasions, which will then be compared in Section 7.

1.2 Privacy as a process (I)

The traditional concept of privacy can be visualized with the image of “my-home-is-my-castle”: there is a territory which, in principle, no other person is allowed to enter.8 The residents of the castle have a right to secrecy and control with secrecy having one clear meaning: what happens in my castle stays in my castle. Accordingly, control has an unambiguous meaning: the residents decide which information may leave the castle. When information stays behind the walls, there is privacy. When information leaves the castle without consent, there is a breach of privacy.

Privacy as a dynamic and fluid concept

The castle metaphor is static—it is the image of an enclosed environment. Current privacy challenges are much more dynamic and can be visualized through the metaphor of the flow, the stream, or the journey—I will use these metaphors interchangeably in this book.

After data leaves an actor, data travels to other data and other actors. In some cases, the journey is very short; in other cases, the journey is long and unpredictable—it is not known in advance which data will be connected and whether these connections will have any effect on privacy. Privacy is therefore a process—a privacy invasion arises and builds up during this process. Privacy protection is less about fencing off an imaginary space (the castle walls) and more about protection against the excesses of the data flow.

The data-journey may start in several ways: with commercial transactions (transaction data), with visiting websites, with the use of devices that are linked to the Internet, by entering a public space with cameras (engagement data), through social media or by answering website feedback questions (attitudinal data), and, of course, users often have to leave personal data if they want to use Internet services.

Solove summarizes the process in four steps: data is (1) collected, (2) processed, (3) disseminated, and these three steps might eventually result in (4) a privacy invasion.9 Privacy is, therefore, not a static, binary concept (information is inside or outside the castle)—it is a dynamic, fluid concept.

Continuous, non-linear, and unpredictable processes

The four steps, from data collection to privacy invasion, may suggest that privacy invasions are linear processes, which, of course, is not the case. Privacy as process can have several important additional features.

First, the process is a continuous one: data is continuously harvested and, continually, new data travels from the user, is processed and disseminated, and needs to be protected. That is why the metaphor of the stream is a powerful one: a stream continues to flow, albeit with a different intensity and in different directions.

Second, both at the individual and collective level, there are many data-journeys and these journeys take place simultaneously. The data-journeys are interconnected, which evokes the image of a spaghetti-like system. There are many spaghetti strands (the data-journeys), some of which are stuck together, so that data can jump from one strand to the other. There is an important consequence of this “spaghetti”: the data-journeys are non-linear, no one can oversee them, and no one knows how they interfere

Third, the process of data collection itself can result in a privacy invasion, but that is not always the case. Data collection, processing, and dissemination can also lead to results that are positively valued: better service delivery, better education, or better health care. As a result of this, a privacy invasion is a process with uncertainty and unpredictability as a main feature. At the start of the journey, it is not always clear whether data collection will finally result in an invasion—not only because we do not know how the data dots will be connected, but also because the trade-off between privacy and other values may change during the process. If we learn during the journey that certain data make a much greater contribution to health than we initially thought, this can influence our views considering the trade-off between privacy and health.

In short, privacy is a process and privacy invasion is a process. This means that privacy invasion is in many cases a wicked problem:

– The factual question, “how does data travel; how does the journey go?” is often a difficult question to answer due to the spaghetti-like structure of the data-journeys.

– The moral question, is data being misused for a privacy violation or is it being used well for a valuable service? does not always have an unambiguous answer.

When a problem comes with both factual and moral uncertainties, it is called ‘wicked’.10 If the facts are clear and there is consensus about the moral dimension, there is a ‘structured problem’. A privacy invasion can be a very structured problem. Posting revenge porn, not properly protecting credit card data, careless handling of patient data—we all agree that these data and privacy breaches are morally wrong. If the facts are clear (we know who made the mistake or who is responsible for the mistake) and we agree about the values and norms (the mistake is morally wrong) then the invasion is a structured problem. However, in this book I want to cover the full complexity of the problem of privacy invasions, which are largely wicked.

Principles of data-protection

There are a number of important principles and rules to protect users’ data and privacy. These principles can, for instance, be found in Article 5.1 of the European Union’s General Data Protection Regulation (GDPR); see the box below.

Personal data shall be:

(a) processed lawfully, fairly and in a transparent manner in relation to individuals (‘lawfulness, fairness, and transparency’);

(b) collected for specified, explicit, and legitimate purposes and not further processed in a manner that is incompatible with those purposes; further processing for archiving purposes in the public interest, scientific or historical research purposes, or statistical purposes shall […] not be considered to be incompatible with the initial purposes (‘purpose limitation’);

(c) adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimization’);

(d) accurate and, where necessary, kept up to date; every reasonable step must be taken to ensure that personal data that are inaccurate, having regard to the purposes for which they are processed, are erased or rectified without delay (‘accuracy’);

(e) kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the personal data are processed; personal data may be stored for longer periods insofar as the personal data will be processed solely for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes […] subject to the implementation of the appropriate technical and organizational measures required by the GDPR to safeguard the rights and freedoms of individuals (‘storage limitation’);

(f) processed in a manner that ensures appropriate security of the personal data, including protection against unauthorized or unlawful processing and accidental loss, destruction or damage, using appropriate technical or organizational measures (‘integrity and confidentiality’).

Source: Regulation (EU) 2016/679 of the European Parliament and the Council, General Data Protection regulation

Regulations such as the GDPR offer strong protection of data and privacy. The GDPR’s principles bind data collection and without these principles there is no data protection. However, the principles are always related to a purpose, and therefore require a trade-off between data collection and that purpose. If data collection purposes are fixed and unambiguous at the start of the data-journey, a singular trade-off can be made. If the purpose changes, or the purpose is ambiguous or perceptions of the purpose change, the trade-off might change over time. Privacy is a process and the application of the principles is often a process too.

Data use and privacy

There are two possible views on the relationship between data use and privacy violations.11 The first implies that a data breach is equivalent to a privacy violation. When someone’s data falls into the hands of a third party in an improper manner, that person’s privacy is violated; therefore, a data breach=privacy breach. The second view is that processing data is permitted—there is only a privacy breach when data is misused later on, during the data-journey.

Without taking a position in this discussion, the distinction between data breaches and privacy breaches is relevant from a governance perspective. There are at least two reasons for this:

– A privacy breach is a process: a data breach takes place and later has certain consequences. From a process perspective, the distinction between data breach and privacy breach matters. Why? Because this process is often non-linear and can be very unpredictable: (I) the major privacy breach can be the result of a series of minor data breaches; (II) normative views on privacy versus other values either will or will not change during the process—what appeared to be a privacy breach may not prove to be a privacy breach after all, and vice versa; (III) it is even conceivable that a privacy breach takes place without being preceded by a formal data breach. There are other examples, but what matters here is that these different types of processes, from data breach to privacy breach, can require different types of governance. So, from a governance perspective, the difference between data breach and privacy breach is very relevant.

– There is always the issue of capacity. Suppose that data breach A leads to a low-impact privacy breach and data breach B leads to a high-impact privacy breach. Suppose that a regulator belongs to the school of “data breach=privacy breach”. Therefore, according to this regulator, there is a privacy breach in both cases. Nevertheless, this regulator will consider the difference between A and B: every regulator has limited capacity and must select those breaches that will be dealt with. Low-impact cases will be given a low priority. A regulator may define a data breach as a privacy breach but will nevertheless use the distinction between data and privacy breach when making decisions regarding priorities.

There is, undoubtedly, more to mention, but first I will further peel back the concept of privacy. The starting point is the traditional concept of privacy: “my home is my castle”. In the world of data and the Internet, privacy is a layered phenomenon and at least four types of privacy can be added to pre-Web privacy concept.

1.3 Type 1: privacy invasions as disclosure – continuous, ubiquitous, and emergent

A type 1 privacy invasion is, in large part, similar to traditional, my-home-is-my-castle invasions. But it is also different. It is similar because it also concerns the values of disclosure and secrecy. Information that is supposed to be private is disclosed to third parties. It is different because, on the Internet, private information is ubiquitous and is eternally retrievable. Two metaphors are helpful here: the earthquake and the dormant volcano—again, we will learn that privacy is a process.

The earthquake with aftershocks – privacy as an ongoing process

The well-known example here is revenge porn. A member of the US Congress has become the victim of revenge porn.12 The pictures have been published by a website and a tabloid and immediately go viral. Social media such as Facebook and YouTube deploy their anti-revenge porn policies and remove the photos, but this is in no way sufficient to prevent further distribution. These photos would end up in a tabloid in the pre-Web world and would only be seen by a limited readership—today, they are always retrievable, and can be retrieved by anyone.

As such, a privacy violation is no longer a one-time shock—the making public of photos that will slowly disappear from collective memory—and may be a lifelong process, with many painful aftershocks. Time and time again the images can re-emerge. Just like an earthquake, a shock can be followed by aftershocks, which may be even heavier than the actual earthquake itself. The victim does not know when these aftershocks will present themselves, and the fear of these aftershocks may sometimes be greater than the aftershock itself. In Section 2, I refer to Solove’s summary of privacy as a process: data collection, processing, and dissemination may eventually result in a privacy invasion. The metaphor of the earthquake makes clear that a privacy invasion is not just the endpoint of the data-journey, it is also a process—an ongoing process, not a one-time affair.

The dormant volcano – privacy as an emergent process

Violations of privacy can present themselves in a wholly unexpected manner: what is harmless today may prove to be a privacy violation tomorrow.

– Information may be re-adapted. In 2019 it was discovered that a ‘softcore paedophile ring’ was copying and editing innocuous YouTube clips of children.13 Members used a timestamp to indicate when children unintentionally adopt a particular pose. The pieces of these videos were then pasted together and distributed among a network of YouTube commenters. Initially innocent pictures—children who are engaging in sports or dancing—are suddenly abused by third parties. An innocent picture of today may result in a severe privacy violation tomorrow.

– Subsequently, there is the phenomenon of re-contextualization. Twenty years ago, a person may have appeared in a video that was posted online by someone else; it is just one of the many videos on the Web. Suppose this person is a visible public figure today and the video suddenly attracts attention—maybe the person said a few things in a private context that are problematic in today’s public realm. In the current context, the twenty year old post has suddenly become a privacy breach. Another example can be seen in “sharenting”, where parents share information about their children on social media. Today, it is nice and cute, but tomorrow it might be used for building digital dossiers on the children.14

– Technological dynamics may lead to privacy violations. For many video makers, TikTok clips are a harmless form of leisure. Often, the makers are fully visual, with their faces being visible. Consider a thought experiment: imagine that, with the development of facial recognition, TikTok clips could be used to gather biometric data of individuals. That would be significantly less harmless.

In essence: information that is currently more or less harmless may become a privacy breach in the future. Here, the metaphor that can be used is that of a volcanic eruption: the volcano lies dormant, but the dangerous eruption may manifest at a moment’s notice. Likewise, privacy invasion is not just a process, but an emergent process—the invasion can suddenly pop up.

1.4 Type 2: privacy invasions as profiling

In the transition from type 1 to type 2, the concept of privacy changes fundamentally. Information is not disclosed, instead, users are profiled.

Data-based profiling

Primarily: using services is relinquishing data—and this data can then be used to profile individuals. In many cases, booking hotel rooms on a website is the same as building a profile, based on preferences for locations, timeframes, types of hotels, or group size. This is, again, relatively harmless, but things may get out of hand. In computer science literature, one research is published after the other, showing how, with a limited dataset, quite detailed user profiles can be created.15

Often, the message of the research is that with (1) ‘easily accessible digital records of behaviour’ (2), analysts can give an accurate prediction of (3) ‘highly sensitive personal attributes including: sexual orientation, ethnicity, religious and political views, personality traits, intelligence, happiness, use of addictive substances, parental separation, age, and gender.’16

Examples of data or meta-data that may be used to build profiles are:

– Data concerning search behaviour—keywords may provide an insight into a person’s age, gender, and political and religious identity;17

– Mobile phone meta-data—conclusions may be drawn about a person’s wealth from the volume, mobility, and structure of their contact network;18

– Data about keystroke dynamics— the rhythm of typing patterns on a standard keyboard can be used to determine user emotion;19

– Instagram pictures—researchers claim that, based on Instagram pictures, they are able to make a reasonable prediction as to whether an individual has depression;20

– Smartphone meta-data—the number of calls a person makes per day or night, and their daily smartphone use can be used to predict a person’s personality traits.21

All kinds of data may be used to procure a profile: input data (data that users have given about themselves), observed data (data based on observations about what users occupy themselves with), and reputation data (data concerning other people’s perception of a person, such as their client rating or likes), as well as inferred data, which is constructed by analysts. With a considerable volume of this data, the focus lies in content-driven data footprints (e.g., likes), but meta-data may also lead to profound profiling of individual users.

Profiling using other people’s data

In the examples above, users provide data dots, but, as stated above, a user’s profile can be created based on data shared by their fellow users. Users have data and information on other users—contact lists, the results of online competitions, all kinds of interactions—and can share that information with third parties.

Co-users may exchange data, allowing analysts to make group profiles. For instance, a group of users may be characterized by A, B, C, D, and E. Suppose that several users in this group fanatically shield their data about E, and in no way relinquish these data, but that analysts do possess their data for A, B, C, and D. Under certain conditions, an analyst may predict what E entails for these users with a near-certain probability; they can do this because the analyst has the data about E that concerns other members of the group. The data dots are like a trail of breadcrumbs—they lead to information about an individual user. The fact that a single crumb is missing is of no consequence whatsoever.

Profile amplification

There is also the phenomenon of profile amplification, where data from a limited group of users is harvested in order to profile big groups of other users. I will give an example that is partly based on the Cambridge Analytica scandal in 2018.

– Researchers invite a group of respondents to participate in, for example, a personality test, which results in an OCEAN score—a score related to the dimensions Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.

– Thereafter, the results of the personality test are linked to specific data—for instance, respondents’ likes on Facebook or smartphone usage logs.

– The researchers will look for patterns and subsequently build algorithms, which can predict characteristics based on, for instance, a person’s likes.

– Subsequently, the likes of the second group of respondents are collected and, with the help of the algorithm, the respondents’ OCEAN scores are predicted. These respondents are also asked to fill in the personality test to check whether or not the likes predicted their scores correctly. The outcomes are used to perfect the algorithm.

– Of course, this process can be repeated several times. When successful, it means that a user “only” leaves data in the form of likes but that an algorithm exists that may, with the support of these likes, predict that user’s personality.

The example shows how relatively easy it is to create an amplifier; a small group of respondents is thoroughly researched to give meaning to the data they relinquish. Subsequently, analysts can make far-reaching statements about large groups of other users based on limited datasets.

Furthermore, we need to bear in mind that individuals can be profiled incorrectly or linked to the wrong group. For example, a person may be allocated to a specific user category that is given a profile, but that person belongs within the 10% where the profile is not applicable. However, an incorrect profile can also invade privacy—I will return to this in the next section.

Data collection and profiling as sequential or parallel processes

Finally, it is essential to distinguish between data collection and profiling as sequential processes and data collection and profiling as parallel processes.

Sequential. Suppose a person regularly books a hotel via a website. In doing so, the user leaves a data trail, which the service provider may subsequently use for profiling.

– There is, based on the latter, a clear division between (a) providing services (booking the hotel), (b) data gathering (collecting the most vital data about the hotel bookings), and (c) subsequent profiling (using the collection of data to create a user profile).

– Providing services, data gathering, and profiling are sequential activities—initially, there is the provision of services, then there is data gathering, and, after that, a profile is determined. Of course, this is a continuous process, but this does not diminish its sequential character.

Conversely, there are types of online services, where,

– providing services, gathering data, and creating a profile are fully integrated;

– thus, service provision, collecting data, and profiling are parallel activities—they are performed simultaneously.

A few examples of these services:

– Advice about nutritional supplements. Advice about nutritional supplements will be based on data about a person’s lifestyle, medical information, goals, and personal values. An algorithm assembles a package of nutritional supplements, users report their progression (muscle growth, sleeping patterns), and, subsequently, the composition of the package can be further refined.

– Styling advice. An online shop may provide styling advice based on data the user shares. Subsequently, an algorithm suggests suitable clothing to the user. The user is required to rate suggestions and, after that, a new suggestion—this continues until the user is happy.

– Advice about information. Users are offered tailored online magazines with themes within their particular interest. Again, a user relinquishes data and an algorithm creates a magazine that is in line with the interests of the user. Data about reading behaviour may subsequently be used to fine-tune the magazine.

In cases such as these, profiling is a sine qua non for the service provision—without profiling, there can be no provision of services. From the perspective of governance, this is an important observation. A hotel booking site can also perform without profiling and so, from the perspective of governance, there are possibilities to rein in data collection. When profiling is a sine qua non for service provision, this may be much more difficult, and data protection may lead to termination of the service.

To summarize: how does a type 2 privacy invasion relate to the traditional concept of privacy?

– As opposed to the traditional violation of privacy, data and information are used to profile and not to disclose. Companies who create a profile of a user often benefit from keeping this information hidden since it is strategic business information. This profile may be extraordinarily detailed, and may also concern users who have given no or limited data.

– With a traditional privacy violation, the focus lies on information rather than data—a type 2 breach involves data that has been translated into information.

– The traditional privacy violation is, generally, a momentary phenomenon; a type 2 privacy violation is a continuous process—a profile is created, and profiles are refined or adjusted with new data. Life is a journey, and our profile will travel along with us as a digital shadow. When we change, our digital shadow changes accordingly.

1.5 Type 3: privacy invasion as manipulation

Profiles may be used to influence individual users;22 here, users receive particular information about specific products or services in line with their profile, which might be in the interest of both the service provider and the user.

Information – selective information – manipulation

But there is more. Informing may grow into selective informing and, subsequently, into intimidation or manipulation. The more precise the profile, the more precisely this intimidation or manipulation can be designed.

– In the world of products and services, profiling may imply that some users are to receive an offer that is not presented to other users. This is efficient and effective, but it may also develop into manipulation: users with a particular profile are not offered houses in specific areas or may have a harder time getting a loan.23

– A similar phenomenon can occur in the world of ideas. For example, one user is shown a political advert, another user does not receive the advert. Since this microtargeting enables attuning the advert to a certain profile, manipulation looms. During the Brexit referendum, the Leave campaign bombarded constituents with specific profiles with adverts that often contained controversial claims.24 Voters who care about animal rights were given an advert with the following text: ‘The EU is supporting commercial whaling by forcing us to allow ships carrying whale meat to dock in our ports!’—while, in reality, the EU does not support commercial whaling.

– In addition to this, users have certain beliefs, including underlying preconceptions and biases. These beliefs can be part of their profile, which means that they constantly receive information confirming those beliefs, preconceptions, and biases. Again, this may grow into manipulation. I will address this more extensively in Chapter 6.

– The more data a malicious party has that concerns an individual, the better that individual can be profiled—and the more ingeniously a phishing-email can be constructed. Furthermore, think of “doxing”—private data is collected to create a profile (a “doc”). Subsequently, the doxed person can be intimidated in all kinds of ways—through anonymous threats, via a social media call to threaten that person, or by buying goods in online stores on behalf of the person who has been doxed.

The digital identity overrules the “real” identity

What we see here is, again, privacy invasion as a process: harvesting data—profiling users—offering tailored information to users—influencing users—manipulating and intimidating them. The next step may be the digital profile overruling the user themselves, thereby superseding who a person truly is.

Analytically, we can distinguish between a digital identity or profile and a “real identity”. A person’s real identity can never be fully objectified, allows different interpretations, and is ambiguous and fluid. A digital identity is often much more conclusive—the algorithm provides a particular personal credit score or awards a person with certain characteristics. Such scores may differ, greatly or not, from someone’s “real identity”.

If (1) a digital identity strongly deviates from a real identity and (2) many decisions are based on digital profiles, there is a significant risk: someone’s digital identity will rule over the real identity or, in other words, one becomes the captive of their digital identity.25

This process may be harmless or even fun. For example, you are traveling and end up in a hideous village in the middle of nowhere. The algorithm thinks that this is the place to be for you, incorporates this in your profile, and subsequently bombards you with offers of hotels in the village. But it can also be quite serious: credit may be denied or a person may have no access to particular information since their underlying digital profile is not in line with their real profile. It seems to be good news when data collection and processing results in an incorrect profile: the data did not reveal a user’s identity. But it is also bad news: an incorrectly profiled user may be intimidated or manipulated because of this incorrect profile.

Profiling and manipulation as parallel processes