Now everything will change as EU creates a new AI landscape

This is a summary of the EU decision regarding the interactions between artificial intelligence (AI) and data protection, focusing on the 2016 EU General Data Protection Regulation (GDPR).

Data protection is at the forefront of the relationship between AI and the law, as many AI applications involve the massive processing of personal data, including the targeting and personalized treatment of individuals on the basis of such data. This explains why data protection has been the area of the law that has most engaged with AI and, despite the fact that AI is not explicitly mentioned in the General Data Protection Regulation (GPDR), many provisions of the GDPR are not only relevant to AI, but are also challenged by the new ways of processing personal data that are enabled by AI. This new STOA study addresses the relation between the GDPR and AI and analyses how EU data protection rules will apply in this technological domain and thus impact both its development and deployment.

Regarding STOA

This study addresses the relation between the EU General Data Protection Regulation (GDPR) and artificial intelligence (AI). It considers challenges and opportunities for individuals and society, and the ways in which risks can be countered and opportunities enabled through law and technology. The study discusses the tensions and proximities between AI and data protection principles, such as in particular purpose limitation and data minimization. It makes a thorough analysis of automated decision-making, considering the extent to which it is admissible, the safeguard measures to be adopted, and whether data subjects have a right to individual explanations. The study then considers the extent to which the GDPR provides for a preventive risk-based approach, focused on data protection by design and by default.

The EU General Data Protection Regulation(GDPR) provides significant and purposeful guidance for data protection in the context of AI applications; no major changes to the GDPR are needed in order to address AI. However, a number of AI-related data-protection issues are not explicitly answered in the GDPR. This may lead to uncertainties and costs, and may needlessly hamper the development of AI applications. Indeed, the GDPR abounds in vague clauses and open standards, such as: ‘personal data concern identified or identifiable natural persons’ (Article 4(1)); ‘consent must be freely given’ (Article (4)(11); ‘further processing must be non-incompatible with the original processing’ (Article 5(1)(b)); ‘the data must be necessary for the purposes for which they are processed’ (Article 5 (1)(c)); ‘controllers must pursue legitimate interests that are non-overridden by the interests or fundamental rights and freedoms of the data subject’ (Article 6(1)(f)); ‘the information about the logic involved in automated decision-making must be meaningful’ (Articles 13(2)(f) and 14 (2)(g)); ‘suitable safeguard measures should be adopted for automated decision-making’ (Article 22 (2)); ‘technical and organizational measures for data protection by design and by default must be appropriate’ (Article 25). It may be difficult for controllers to determine whether the processing they envisage satisfies these open standards.

Although artificial intelligence (AI) is not explicitly mentioned in the EU General Data Protection Regulation (GPDR), many of its provisions are relevant to the use of AI, and some indeed face challenges posed by the new ways of processing personal data that are enabled by AI. A tension exists between traditional data protection principles — purpose limitation, data minimization, special treatment of ‘sensitive data’, limitations on automated decisions — and the full deployment of the power of AI. However, it is possible to interpret, apply and develop those data protection principles that are consistent with beneficial uses of AI. A number of AI-related data-protection issues are not explicitly answered in the GDPR, where provisions are often vague and open-ended. Controllers and data subjects should be provided with guidance on how AI can be applied to personal data in conformity with the GDPR, and on the available technologies for doing so. A broad social, political and legal debate is needed on what standards should apply to processing of personal data using AI, particularly to ensure the explanation, acceptability, fairness and reasonableness of decisions about individuals. The debate should also address the question of which applications are to be barred unconditionally, and which ones may instead be admitted only under specific circumstances and controls.

Ethics

Similarly, the effective, explicit and free exercise of data subjects’ consent (or denial of consent) to AI-based processing is to be ensured whenever consent may provide a legal basis.

Artificial intelligence has become hungry for data, and this hunger has spurred data collection, in a self-reinforcing spiral: the development of AI systems based on machine learning presupposes and fosters the creation of vast data sets, i.e., big data

AI has gone through rapid development. It has acquired a solid scientific basis and has produced many successful applications. It provides opportunities for economic, social, and cultural development; energy sustainability; better health care; and the spread of knowledge. These opportunities are accompanied by serious risks, including unemployment, inequality, discrimination, social exclusion, surveillance, and manipulation.

Artificial intelligence systems are populating the human and social world in multiple varieties: industrial robots in factories, service robots in houses and healthcare facilities, autonomous vehicles and unmanned aircraft in transportation, autonomous electronic agents in e-commerce and finance, autonomous weapons in the military, intelligent communicating devices embedded in every environment. AI has come to be one of the most powerful drivers of social transformation: it is changing the economy, affecting politics, and reshaping citizens’ lives and interactions. Developing appropriate policies and regulations for AI is a priority for Europe, since AI increases opportunities and risks in ways that are of the greatest social and legal importance. AI may enhance human abilities, improve security and efficiency, and enable the universal provision of knowledge and skills. On the other hand, it may increase opportunities for control, manipulation, and discrimination; disrupt social interactions; and expose humans to harm resulting from technological failures or disregard for individual rights and social values. A number of concrete ethical and legal issues have already emerged in connection with AI in several domains, such as civil liability, insurance, data protection, safety, contracts and crimes. Such issues acquire greater significance as more and more intelligent systems leave the controlled and limited environments of laboratories and factories and share the same physical and virtual spaces with humans (internet services, roads, skies, trading on the stock exchange, other markets, etc.). Data protection is at the forefront of the relationship between AI and the law, as many AI applications involve the massive processing of personal data, including the targeting and personalized treatment of individuals on the basis of such data. This explains why data protection has been the area of the law that has most engaged with AI, although other domains of the law are involved as well, such as consumer protection law, competition law, antidiscrimination law, and labor law. This study will adopt an interdisciplinary perspective. Artificial intelligence technologies will be examined and assessed on the basis of most recent scientific and technological research, and their social impacts will be considered by taking account of an array of approaches, from sociology to economics and psychology. A normative perspective will be provided by works in sociology and ethics, and in particular information, computer, and machine ethics. Legal aspects will be analyzed by reference to the principles and rules of European law, as well as to their application in national contexts. The report will focus on data protection and the GDPR, though it will also consider how data protection shares with other domains of the law the task of addressing the opportunities and risks that come with AI.

Two different legal perspective, complementary rather than incompatible, may inspire data protection law, a right based and a risk-based approach. Though the focus of the GDPR is on the right-based approach, there are abundant references to the risk prevention in the GDPR that can be used to address AI-related risks.

In conclusion, it seems that issues that have just been presented should not lead us to exclude categorically the use of automated decision-making. The alternative to automated decision-making is not perfect decisions but human decisions with all their flaws: a biased algorithmic system can still be fairer than an even more biased human decision-maker. In many cases, the best solution consists in integrating human and automated judgements, by enabling the affected individuals to request a human review of an automated decision as well as by favoring transparency and developing methods and technologies that enable human experts to analyze and review automated decision making. In fact, AI systems have demonstrated an ability to successfully also act in domains traditionally entrusted the trained intuition and analysis of humans, such as medical diagnosis, financial investment, the granting of loans, etc. The future challenge will consist in finding the best combination between human and automated intelligence, taking into account the capacities and the limitations of both.

In the case of processing by private controllers, the right to rectify the data should be balanced with the respect for autonomy of private assessments and decisions.76. According to the Article 29 Working Party data subjects have a right to rectification of inferred information not only when the inferred information is ‘verifiable’ (its correctness can be objectively determined), but also when it is the outcome of unverifiable or probabilistic inferences (e.g., the likelihood of developing heart disease in the future). In the latter case, rectification may be needed not only when the statistical inference was mistaken, but also when the data subject provides specific additional data that support a different, more specific, statistical conclusion

The Cambridge Analytica case

First of all, people being registered as voters in the USA were invited to take a detailed personality/political test (about 120 questions), available online. The individuals taking the test would be rewarded with a small amount of money (from two to five dollars). They were told that their data would only be used for the academic research. About 320 000 voters took the test. In order to be receive the reward each individual taking the test had to provide access to his or her Facebook page (step 1). This allowed the system to connect each individual’s answers to the information included in his or her Facebook page. When accessing a test taker’s page, Cambridge Analytica collected not only the Facebook page of test takers, but also the Facebook pages of their friends, between 30 and 50 million people altogether (step 2). Facebook data was also collected from other sources. After this data collection phase, Cambridge Analytica had at is disposition two sets of personal data to be processed (step 3): the data about the test takers, consisting in the information on their Facebook pages, paired with their answers to the questionnaire, and the data about their friends, consisting only in the information on their Facebook pages. Cambridge Analytica used the data about test-takers as a training set for building a model to profile their friends and other people. More precisely, the data about the test-takers constituted a vast training set, where the information on an individual’s Facebook pages (likes, posts, links, etc.) provided values for predictors (features) and the answers to the questionnaire (and psychological and political attitudes expressed by such answers) provided values the targets. Thanks to its machine leaning algorithms Cambridge Analytica could use this data to build a model correlating the information in people’s Facebook pages to predictions about psychology and political preferences. At this point Cambridge Analytica engaged in massive profiling, namely, in expanding the data available on the people who did not take the test (their Facebook data, and any further data that was available on them), with the predictions provided by the model. For instance, if test takers having a certain pattern of Facebook likes and posts were classified as having a neurotic personality, the same assessment could be extended also to non-test-takers having similar patterns in their Facebook data. This shows the connection between identified and de-identified data.

Ethics

Legal scholars have argued that data subjects should be granted a general right to ‘reasonable inference’ namely, the right that any assessment of decision affecting them is obtained through automated inferences that are reasonable, respecting both ethical and epistemic standards

Therefore, data subjects should be allowed to give their consent to certain areas of scientific research when in keeping with recognized ethical standards for scientific research

Data subjects should have the opportunity to give their consent only to certain areas of research or parts of research projects to the extent allowed by the intended purpose

Finally, consent should be invalid when refusal or withdrawal of consent is linked to a detriment that is unrelated to the availability of the personal data for which consent was refused (e.g., a patients are told that in order to obtain a medical treatment they must consent that their medical data are used for purposes that are not needed for that treatment)

According to Recital (77) the Board is supposed to provide guidance on the implementation of the GDPR through guidelines: Guidance on the implementation of appropriate measures and on the demonstration of compliance by the controller or the processor, especially as regards the identification of the risk related to the processing, their assessment in terms of origin, nature, likelihood and severity, and the identification of best practices to mitigate the risk, could be provided in particular by means of approved codes of conduct, approved

References

· AI-HLEG, High-Level Expert Group on Artificial Intelligence (2019). A definition of AI: Main capabilities and scientific disciplines.

· AI-HLEG, High-Level Expert Group on Artificial Intelligence (2019). Ethics guidelines for trustworthy AI. Ashley, K. D. (2017). Artificial Intel ligence and Legal Analytics. Cambridge University Press.

· Balkin, J. M. (2008). The constitution in the national surveillance state. Minnesota Law Review 93, 1–

· 25. Balkin, J. M. (2017). The three laws of robotics in the age of big data. Ohio State Journal Law Journal 78, 1217–241. Barocas, S. and A. D. Selbst (2016). Big data’s disparate impact. California Law Re view 104, 671–732.

· Bayer, J., Bitiukova, N., Bard, P., Szakacs, J., Alemanno, A., and Uszkiewicz, E. (2019). Disinformation and propaganda — impact on the functioning of the rule of law in the EU and its member states. Study, Policy Department for Citizens’ Rights and Constitutional Affairs, European Parliament.

· Bhuta, N., S. Beck, R. Geiss, C. Kress, and H. Y. Liu (2015). Autonomous Weapons Systems: Law, Ethics, Policy. Cambridge University Press.

· Bostrom, N. (2014). Superintelligence. Oxford University Press.

· Bosco, F., Creemers, N., Ferraris, V., Guagnin, D., & Koops, B. J. (2015). Profiling technologies and fundamental rights and values: regulatory challenges and perspectives from European Data Protection Authorities. In Reforming European data protection law (pp. 3–33). Springer, Dordrecht.

· Brynjolfsson, E. and A. McAfee (2011). Race Against the Machine. Digital Frontier Press.

· Burr, C. and Cristianini, N. (2019). Can machines read our minds? Minds and Machines 29:461–494.

· Calo, M. R. (2012). Against notice skepticism in privacy (and elsewhere). Notre Dame Law Review, 87:1027– 72.

· Cate, F. H., P. Cullen, and V. Mayer-Schönberger (2014). Data Protection Principles for the 21st Century: Revising the 1980 OECD Guidelines. Oxford Internet Institute.

· Cath, C., Wachter, S., Mittelstadt, B., Taddeo, M., and Floridi, L. (2018). Artificial intelligence and the ‘good society’: the US, EU, and UK approach. Science and Engineering Ethics 24:505–528.

· Cohen, J. D. (2019). Between Truth and Power. The Legal Constructions of Informational Capitalism. Oxford University Press.

· Cristianini, N. (2016a, 23 November). Intelligence rethought: AIs know us, but don’t think like us. New Scientist. Cristianini, N. (2016b, 26 October). The road to artificial intelligence: A case of data over theory. New Scientist.

· Cristianini, N. and T. Scantamburlo (2019). On social machines for algorithmic regulation. AI and Society.

· De Hert, P. and Gutwirth, S. (2009). Data protection in the case law of Strasbourg and Luxemburg: Constitutionalisation in action. In Gutwirth, S., Poullet, Y., De Hert, P., de Terwangne, C., and Nouwt, S., editors, Reinventing Data Protection? 3–44. Springer.

· Edwards, L. and Veale, M. (2019). Slave to the algorithm? Why a ‘right to an explanation’ is probably not the remedy you are looking for. Duke Law and Technology Review, 16–84.

· Floridi, L., J. Cowls, M. Beltrametti, R. Chatila, P. Chazerand, V. Dignum, C. Luetge, R. Madelin, U. Pagallo, F. Rossi, B. Schafer, P. Valcke, and E. Vayena (2018). Ai4people– an ethical framework for a good ai society: Opportunities, risks, principles, and recommendations. Minds and Machines 28, 689–707.

· Guidotti, R., A. Monreale, F. Turini, D. Pedreschi, and F. Giannotti (2018). A survey of methods for explaining black box models. ACM Computer Surveys 51 (5) Article 93, 1–4.

· Halpern, J. Y. and Hitchcock, C. (2013). Graded causation and defaults. The British Journal for the Philosophy of Science, 1–45.

· Harel, D. and Y. Feldman (2004). Algorithmics: The Spirit of Computing. Addison- Wesley.

· Hildebrandt, M. (2009). Profiling and AML.

· In Rannenberg, K., Royer, D., and Deuker, A., editors, The Future of Identity in the Information Society. Challenges and Opportunities. Springer.

· Hildebrandt, M. (2014). Location data, purpose binding and contextual integrity: What’s the message? In

· Hildebrandt, M. (2015). Smart Technologies and the End(s) of Law: Novel Entanglements of Law and

· Jobin, A., Ienca, M., and Vayena, E. (2019). Artificial intelligence: the global landscape of ethics guidelines.

· Kahneman, D. (2011). Thinking: fast and slow. Allen Lane. Kamara, I. and De Hert, P. (2019). Understanding the balancing act behind the legitimate interest of the controller ground: A pragmatic approach. In Seligner, E., Polonetsky, J., and Tene, O., editors, The Cambridge Handbook of Consumer Privacy. Cambridge University Press.

· Kaplow, L. (1992). Rule vs standards: An economical analysis. Duke Law Journal, 42: 557–629.

· Kleinberg, J., J. Ludwig, S. Mullainathan, and C. R. Sunstein (2018). Discrimination in the age of algorithm. Journal of Legal Analysis 10, 113–174.

· Kurzweil, R. (1990). The Age of Intelligent Machines. MIT.

· Kurzweil, R. (2012). How to Create a Mind. Viking.

· Licklider, J. C. R. (1960). Man-computer symbiosis. IRE Transactions on Human Factors in Electronics HFE-1 (March), 4–11.

· Lippi, M., P. Palka, G. Contissa, F. Lagioia, H.-W. Micklitz, Y. Panagis, G. Sartor, and P. Torroni (2019). Claudette: an automated detector of potentially unfair clauses in online terms of service. Artificial Intelligence and Law. Lippi, M., Contissa, G., Jablonowska, A., Lagioia, F., Micklitz, H.-W., Palka, P., Sartor, G., and Torroni, P. (2020). The force awakens: Artificial intelligence for consumer law. The journal of Artificial Intelligence Research 67:169–190.

· Mantelero, A. (2017). Regulating Big Data. The guidelines of the Council of Europe in the context of the European data protection framework. Computer Law and Security Review 33, 584–602.

· Mayer-Schönberger, V. and K. Cukier (2013). Big Data. Harcourt.

· Mayer-Schönberger, V. and Y. Padova (2016). Regime change? enabling Big Data through Europe’s new data protection regulation. Columbia Science and Technology Law Review 17, 315–35.

· McAfee, A. and E. Brynjolfsson (2019). Machine, Platform, Crowd. Norton.

· Marcus, G. and Davis, E. (2019). Rebooting AI: building artificial intelligence we can trust. Pantheon Books.

· Mindell, D. A. (2015). Our Robots, Ourselves: Robotics and the Myths of Autonomy. Penguin.

· Nilsson, N. (2010). The Quest for Artificial Intelligence. Cambridge University Press.

· O’Neil, C. (2016). Weapons of math destruction: how Big Data increases inequality and threatens democracy. Crown Business.

· Pariser, E. (2011). The Filter Bubble. Penguin.

· O’Neil, C. (2016). Weapons of math destruction: how big data increases inequality and threatens democracy. Crown Business.

· Parkin, S. (14 June 2015). Science fiction no more? channel 4’s humans and our rogue ai obsessions. The Guardian.

· Pasquale, F. (2019). The second wave of algorithmicaccountability. Law and PoliticalEconomy.

· Pentland, A. (2015). Social Physics: How Social Networks Can Make Us Smarter. Penguin.

· Polanyi, K. ([1944] 2001). The Great Transformatio n. Beacon Press.

· Powles, J. and Nissenbaum, H. (2018). The seductive diversion of ‘solving’ bias in artificial intelligence. Medium.

· Prakken, H. and G. Sartor (2015). Law and logic: A review from an argumentation perspective. Artificial Intelligence 227, 214–45.

· Rawls, J. ([1971] 1999). A Theory of Justice. Oxford University Press.

· Ruggeri, S., D. Pedreschi, and F. Turini (2010). Integrating induction and deduction for finding evidence of discrimination. Artificial Intelligence and Law 18, 1–43.

· Russell, S. J. and P. Norvig (2016). Artificial Intel ligence. A Modern Approach (3 ed.). Prentice Hall. Sartor, G. (2017). Human rights and information technologies. In R. Brownsword, E. Scotford, and K.

· Yeung (Eds.), The Oxford Handbook on the Law and Regulation of Technology, pp. 424–450. Oxford University Press.

· Stiglitz, J. (2019). People, Power, and Profits. Progressive Capitalism for an Age of Discontent. Norton.

· Sunstein, C. R. (2007). Republic.com 2.0. Princeton University Press.

· Turing, A. M. ([1951] 1996). Intelligent machinery, a heretical theory. Philosophia Mathematica 4, 256– 60.

· van Harmelen, F., V. Lifschitz, and B. Porter (2008). Handbook of Knowledge Representation. Elsevier.

· Varian, H. R. (2010). Computer mediated transactions. American Economic Review (2): 100, 1–10.

· Varian, H. R. (2014). Beyond Big Data. Business Economics (49), 27–31.

· Wachter, S. and B. Mittelstadt (2017). A right to reasonable inferences: Re-thinking data protection law in the age of Big Data and AI. Columbia Business Law Review, 1–130.

· Wachter, S., B. Mittelstadt, and L. Floridi (2016). Why a right to explanation of automated decisionmaking does not exist in the General Data Protection Regulation. Internatio nal Data Privacy Law 7, 76–99.

· Yeung, K. (2018). ‘Hypernudge’: Big data as a mode of regulation by design. Communication and Society 20, 118–36.

· Zarsky, T. Z. (2017). Incompatible: The GDPR in the age of Big Data. Seton Hall Law Review, 47:995–1020.

· Zuboff, S. (2019). The Age of Surveillance Capitalism. Hachette.