The Fourth Digital Divide – Divide by Algorithm?

(On algorithms-mediated society and what to do next)

The future is here but it's not evenly distributed

The more digitally mediated benefits we have, the fewer opportunities there are for humans to exercise control and authority[1].


Show me your (meta)data and I’ll tell you what you can(not) do!

September 2017: Facebook discloses that more than 3000 ads ($100,000 in ad spending) were produced by Russian bots during the USA presidential elections[2]. The influence and its scope on the outcome of the elections is yet to be assessed. In the meantime, Facebook removed the alleged posts, disabling independent researchers to investigate the issue[3].

September 2017: ProPublica discovers that Facebook’s ad algorithm enables marketers (or anyone who wants to use its ad-buying platform) to target people who have expressed interest in “How to burn jews”, “History of ‘why Jews ruin the world’” and “Jew hater”[4]. According to Facebook, they removed the categories. It remains unknown how many additional discriminatory ad-categories are out there “in the wild”.

October 2017: ProPublica reveals that Facebook’s ad-buying systems allows advertisers to exclude black, Hispanic, and other “ethnic affinities” from seeing (housing-related) ads[5].

June 2015: Google Photo using AI software for image recognition auto-tagged two black people as “gorillas,” as a result of the program having been under-trained to recognize dark-skinned faces[6]

These are just few of the recent examples revealed and made visible by researchers, journalists and activists,  showing the scope of influence the algorithms and machine learning technology have on not just our everyday living, but on society at large as well. From which information is available to us (information asymmetry through, for example, Facebook’s filter bubbles and Google’s PageRank) to whose ads (and ideology, when it comes to political advertising) will we be exposed to, to how much we will have to pay for insurance depending on our geolocation (IP addresses and post codes as proxies for race/ethnic group), to our eligibility for a bank loan (“today, insurance companies analyse how many exclamation points we use in social media posts to determine whether we are a safe driver”[7]) and our recidivism likelihood. We are living in an era where even more and more decisions concerning our choices and our lives are left to the “mercy” of algorithms. With the decrease of human involvement in many decision-making processes and the automation of these very processes, artificial intelligence is used to make determinations and predictions in vast areas of our lives. And this (as research shows so far) can have devastating consequences on people’s (and entire communities’) lives, as it includes decisions concerning housing, education, hiring, criminal justice etc.  As the AI Now Initiative observes – “they have the potential to impact our basic rights and liberties in profound ways”, and seriously undermine them[8].

The promises made by technology development

In her paper “The Mediation of Hope: Communication Technologies and Inequality in Perspective”[9] Robin Mansell points out that when it comes to the debates around the advantages of the rapid development of new media technologies, the following premise is predominant: “the rapid commercialization of advanced digital technologies works as ‘a powerful catalyst and a driver of inclusiveness’ (Wyckoff, 2016, para. 13), enabling countries to rise up the global value chain, expanding markets, offering greater choice to consumers, creating employment, and leading to sustained prosperity.” (Mansell, 2017, 6)[10]. The evidence that the advancement of digital technology does not eradicate different types of inequality, but additionally strengthen and perpetuate systemic and global inequality, is often overlooked.  Mansell further questions the premise held for a long time by the neoclassical economy that technology advancement is good for everyone asking “whether the extension of calculative (intelligent) machines throughout society is consistent with an inclusive and more equitable society” (ibid, p.2).

One of the ways to overcome this dominant paradigm, she argues, is to use the SCOTS tradition, where both technical and non-techical aspects of technology, its infrastructure and impact are taken into account and analyzed, by not just scraping below the surface of what is inherently technological (the infrastructural components), but also to investigate the “the motivations and actions of individuals, “relevant social groups,” “system builders,” or actors and “actants” (Bijker, Hughes, & Pinch, 2012)[11] (infrastructural relations) and that way, by introducing critical theories of power, to investigate the power relations that influence technology.

This is extremely important, and a raising issue in the latest debates, because, as the research of Sara Wachter-Boettcher shows, one of the reasons why technology is not working for everyone equally, is beacause of inherently biased algorithms and AI. Most of the biases encoded in the technology are as a result of “an insular, mostly-white, tech industry that has built its own biases into the foundations of the technology we use and depend on”[12]. If we look at the percentage of minorities (women, race, ethnicities etc) working in the tech industry, this might not come as a surprise. If not overseen or regulated by an organizational policy, power relations, stereotypes and biases can easily be inscribed into codes, training data sets and algorithms.

According to an article in MIT Technology Review[13], biased algorithms are everywhere. And we are just starting to be aware of the social implications artificial intelligence has not just on our individual lives, but also on society/world in general (just think of Trump’s win in the USА). Informational asymmetry, machine bias, algorithmic bias, algorithmic injustice are just some of the terms, emerging these past few years, trying to describe and analyze the state of algorithmic divide we’re in.  As it is further stated in the article, “Algorithms that may conceal hidden biases are already routinely used to make vital financial and legal decisions. Proprietary algorithms are used to decide, for instance, who gets a job interview, who gets granted parole, and who gets a loan.[14]


Infrastructural Inversion as a (research) method

All these issues previously discussed were made visible through an “infrastructural inversion”. As Liah Levrouw pointed out during the DCLead Summer Seminar presentation[15], infrastructural inversion occurs when breakdown occurs – the revelation of the anti-Semitism and hate speech categories on Facebook’s ad-buying platform, the possibility for Russian bot farms to influence the USA presidential elections, the exclusion of black communities and minorities of the educational system and the workspace etc&etc. The breakdown episodes made the infrastructural elements and the inner working of the system (algorithms) visible.  In their paper “Toward Information Structure Study: Ways of Knowing in a Networked Environment” Bowker et all, point out that

“Infrastructure typically exists in the background, it is invisible, and it is frequently taken for granted (Star & Ruhleder, 1994) […] In such a marginalized state its consequences become difficult to trace and politics are easily buried in technical encodings (Hanseth, Monteiro, & Hatling, 1996; Monteiro & Hanseth, 1997). The design of infrastructure itself can make its effects more or less visible (Bowker, Timmermans, & Star, 1995). Calls to study infrastructure in STS have engendered methods for making it and associated, emergent roles visible (Edwards, 2003; Karasti & Baker, 2004; Ribes & Baker, 2007): practical methods such as observing during moments of breakdown (Star, 1999) or conceptual methods such as “infrastructural inversion” (Bowker, 1994) (Bowker et all, 2010, 98)[16].

According to the authors, if we want to understand any given infrastructure, we need to unfold both the political and ethical, as well as the social choices that were done throughout its development. When we talk about the work of an infrastructure, we should always talk about “relations’, not just components of a network. If we translate this to how AI and machine learning work, we will need to include the affordances of the infrastructure, its practices, but also the human factor behind them – the code designer, the programmers, the system builders and the company’s policy.

To rephrase Bowker et all, the workings of an infrastructure are always a “technical” and a “social” problem. As they also point out, “social, ethical and political values are being built into these self-effacing, self-perpetuating infrastructures” [17](Bowker et all, 2010, 106). And this is also the case with the AI and machine learning technologies and their underlying infrastructure. Both of them are developed by humans and both of them are perpetuating biases inscribed in them by their creators – humans. As Lievrouw pointed out, with the infrastructure inversion we’re able to defamiliarize, investigate and reveal “relations among articulated infrastructure elements: ‘politics, voice and authorship embedded in the systems’”. If we follow the premises of the infrastructural inversion as a method, when analyzing the inner workings of a given infrastructure, in all these cases mentioned earlier (AI and machine learning) we will have to look at the decisions about encoding, code design, but as well as policies of the companies behind the technology (for example, Facebook, Google).

In line with this is Mansell’s call for use of critical interdisciplinary engagement with digital technology innovation[18]. Quoting Kitchin (2017), she proposes the “use of research methods such as genealogies of code, auto-ethnographies of coding practices, interviews with coding teams, and examinations of the tasks that algorithms perform together with reverse engineering of algorithm computations to understand the implications of their further development” (Mansell, 2017, p.9).

And this is what many researchers did so far. That is why, as Privacy International stated in their Submission of evidence to the House of Lords Select Committee on Artificial Intelligence, we can now grasp and draw conclusions of how algorithms work and how they contribute to, further strengthen and create additional societal inequalities. “AI-driven applications sort, score, categorise, assess, and rank people, often without their knowledge or consent”[19] This is most often done through machine-learning using algorithms, trained with vast amounts of data in order to complete tasks, that range from recognizing patterns, to making predictions and making decisions. How machines are trained and what data are they instructed to use, as well what data they are feed into the training sets (research shows data is predominantly based on white people features, not taking sufficiently into account the various demographic and ethnographic feature, for example) and later when they compute the tasks, depends on the engineers creating the system. And as various recent researches show, as humans are intrinsically biased, they inscript those biases in the algorithms and codes they produce. It might look benign, but with the level of automation nowadays for profiling as a way to make decisions and predictions, this is a serious concern regarding the enjoyment of human rights, both economic, social, cultural and digital rights as well.

The opacity of algorithms

Algorithms have a transparency problem. They are the perfect definition of a “black-box” – no one knows for sure what’s inside, how they operate and what can go wrong. Especially when it comes to machine learning, when the outcomes are not predictable by the designer either. I will borrow here the classification outlined by Privacy International, as it sums up well the transparency problems of AI: there can be distinguished three forms of opacity “(1) opacity as intentional corporate or state secrecy; (2) opacity as technical illiteracy; and (3) an opacity that arises from the characteristics of machine learning algorithms and the scale required to apply them usefully”[20] (Privacy International, 2017, p.6).

Because much of the algorithms running our daily lives are privately owned by companies and corporations, they are treated as private ownership/commercial products and they are not open to the public eye. Meaning, fellow-coders, researchers and analysts are not being able to investigate them, see how they operate, which mathematical models they use, thus disabling them to find the flaws, offer corrections and make them “work better”. The call for companies (and governments) to open their AI systems to the public is gaining momentum the past year(s). As Privacy International outline, “Black boxing should not be permissible wherever AI systems are used to make or inform consequential decisions about individuals or their environment; in such instances, a lack of transparency is highly problematic. Consequential decisions are decisions that produce irreversible effects, or effects that can significantly affect an individual’s life or infringe on their fundamental and human rights (Privacy International, 2017, p.7)”[21]. So far, both companies and governments are not interested in opening up the codes, and in monitoring and thus limiting algorithmic bias[22].  But as some researchers state – it is of utmost importance for the public to know “how an algorithm was chosen, developed, and tested—for example how sensitive it is to false positives and negatives.[23]

Digital (i)literacy

Additional issue in this complex system is that most of the people are not “code literate”, they do not know how algorithms and AI work, what data is gather about them, how that data is used in the automated decision-making processes, how they influence their lives and what decisions they are making about their everyday living and to which extend and scope. Actually, probably most people don’t even know that is not a human behind the decision if they should get their loan request approved. So this is an additional axis in the new digital divide – if people are not aware, they are not in a position or have the power to demand transparency and accountability and to influence the policy-making processes. And this, to use Mansell’s words, leads to “erosion of the capacity of humans to exercise control over their digital environment” (Mansell, 2017, p. 10).

Many times, the code designers and engineers aren’t aware of their biases themselves either. Let alone of their inscription in the algorithms and systems they are building. And this is something that needs to be tackled as well. As a tweet from Twitter user Brian Mastebrook says: “Not a crisis: not everybody can code. Actually a crisis: programmers don’t know ethics, history, sociology, psychology or the law[24]” – todays coders and programmers need not necessary to be educated in social sciences, but at least they should be familiar with social and power dynamics, so they, via their “products”, do not perpetuate the injustices and additionally marginalize vulnerable communities.

How to (and can we?) regulate algorithms?

From everything stated above, can we say that algorithms are increasingly contributing to the rise of a new, Fourth Digital Divide? Algorithms are governing our lives without transparency, accountability or our knowledge. They profile us, gather data about us, make decisions about us, and by doing so, we are afforded or denied information, access, and services.  With all the examples of “algorithms and technology went wrong”, the urgent question that comes to mind is – should we try to regulate AI? Should a policy be developed? Regulations introduced? If yes, how? To what extend and when?

These are some of the questions different stakeholders problematize at the moment. From academics, researchers, policy-makers, activists, journalists – to the industry itself. Robin Mansell is also one of the academics working on this issue. According to her, it is highly unlikely that the big players (the companies) that benefit the most from the artificial intelligence and data gathering will willingly agree to be imposed on policies and standards that are consistent with citizen rights and the values of social justice and inclusion.

That is why the most common case when regulating digital technologies (in case there is some sort of regulation) are the ex post policies – policies developed after the technologies have been released in the marketplace and after the “damage was done”[25]. One of the reasons for this is the firm believe that if we try to impose ex ante (precautionary) policies on the digital technology innovation – “market intervention via government policy”[26], we will restrict the further development of innovative technologies. Asking herself if we need to adopt a “wait and see approach”, Mansell suggests a “middle” path – being aware that ex ante policies are unlikely to be proposed and/or adopted, she goes for introducing ex post policies in the hope that “digital economy policy relying mainly on ex post adaptive policy responses could start to be coupled with the greater use of ex ante policy measures” (Mansell, 2017, p.14).[27]

But it’s not just academics who demand, if not strict regulation, than more transparency on AI and algorithms and more research on the scope of their influence. These calls come from the civil society, researchers, practitioners and industry itself. Privacy International, for example, calls the British Government and all other relevant actors to regulate/investigate the following: “the data that feeds into AI systems; the data (and insights) that AI systems generate; as well as how and whether AI systems should be used to make or inform consequential decisions about individuals and groups, especially if these systems are highly complex and opaque” (Privacy International, 2017, p.7).[28]

The AI Now Institute, comprising of industry representatives, researchers and academics, is dedicated to doing an interdisciplinary research on the social implications of AI, focused on rights and liberties, labor and automation, bias and inclusion, and safety and critical infrastructure.[29] The latest initiative from the industry actors (“the industry shapers”) comes in the form of The Copenhagen Letter[30], calling for accountability and transparency from all involved into shaping the society and the future to come through codes and algorithms.

In order to build societies where social values of equality, justice and access to human rights for all are respected and enjoyed equally, we need to 1. build transparent AI and technological systems, which we could hold accountable and 2. educate the public and make the “data subjects” literate on how their data is gathered, processed and used for decision-making processes. How do we do that? Firstly, by making the code-producers release their code – by opening their code design to the public eye, they will allow outside researchers to investigate the systems, code and tools used. That way, they could challenge the algorithms, can work on removing the inscribed biases, improve the systems and build systems that will serve all, not just the few. And ex post (maybe even ex ante) policies could be developed as well, governing the current and the future digital technologies development.

And let’s not forget – we need digital citizens, literate and aware, who can control their data and who will exercise full control and authority over their (digital) lives again. To borrow The Copenhagen Letter’s maxima: “Tech is not above us”. And it should not be. Ever.

For the PDF version, click here.


[1] Robin Mansell, during her presentation at the DCLead Summer School, October 2nd 2017, at Strobl, Austria

[2] An Update On Information Operations On Facebook By Alex Stamos, Chief Security Officer at Facebook ( last accessed on October 14th 2017

[3] See more here:

[4] Facebook Enabled Advertisers to Reach ‘Jew Haters’ by Julia Angwin, Madeleine Varner and Ariana Tobin, published September 14th, 2017 ( last accessed on October 14th 2017)

[5] Facebook Lets Advertisers Exclude Users by Race, published October 28th 2017 ( last accessed October 16th 2017)

[6] See and the original tweet

[7] Privacy International (2017), Submission of evidence to the House of Lords Select Committee on Artificial Intelligence, p. 3

[8] AI Now Institute, landing page ( , last accessed October 14th 2017)

[9] Mansell, Robin. The Mediation of Hope: Communication Technologies and Inequality in Perspective in: International Journal of Communication 11(2017), pp. 1-20

[10] Ibid.

[11] Ibid, p.6

[12] As stated in Silicon Valley Is Inserting Its Biases Into Nearly Every Technology We Use, published October 10th 2017 (, last accessed October 15th 2017)

[13] Biased Algorithms are everywhere, and No One seems to care, by Will Knight, published July 12th 2017 (, last accessed October 14th 2017)

[14] Ibid.

[15] Held on October 2nd 2017, in Strobl, Austria

[16] “Toward Information Structure Study: Ways of Knowing in a Networked Environment” in International Handbook of Internet Research, J. Hunsinger et al. (eds.), 2010

[17] Ibid.

[18] Mansell, Robin. The Mediation of Hope: Communication Technologies and Inequality in Perspective in: International Journal of Communication 11(2017), pp. 1-20

[19] Privacy International (2017), Submission of evidence to the House of Lords Select Committee on Artificial Intelligence, p.6

[20] Privacy International (2017), Submission of evidence to the House of Lords Select Committee on Artificial Intelligence

[21] Ibid.

[22] Biased Algorithms are everywhere, and No One seems to care, by Will Knight, published July 12th 2017 (, last accessed October 14th 2017)

[23] When Government Rules by Software, Citizens Are Left in the Dark, published August 17th 2017 ( , last accessed October 15th 2017)

[24] Tweet accessible on the following link: (last accessed October 15th 2017)

[25] See for example the regulations Facebook will change after the revelation of the Russian botfarms during presidential election in 2016: and

[26] Mansell, Robin. The Mediation of Hope: Communication Technologies and Inequality in Perspective in: International Journal of Communication 11(2017), pp. 1-20

[27] Ibid.

[28] Privacy International (2017), Submission of evidence to the House of Lords Select Committee on Artificial Intelligence

[29] See more at: (last accessed October 16th 2017)

[30] See more at: : (last accessed October 16th 2017)

Leave a Reply

Your email address will not be published. Required fields are marked *