“Faulty data, invalid assumptions and defective models can’t be corrected when they are hidden”
In his book Black Box Society, Frank Pasquale sets out to shed light on the complex, obfuscated, and secretive institutions that are profiling humans and arbitrarily utilizing the gaps in the legal system where they can unilaterally obtain huge amounts of information about virtually everyone, while they themselves conceal their business behind “one way mirrors”; trade-secrecy laws, and national security. Frank Pasquale shows the dangers associated with data and algorithms which are pulling the strings and make decisions about human beings. This book is important for STS students because it demonstrates how reducing social issues to mere technological problems as technocrats tend to do is unlikely to produce good results, and that interdisciplinary collaboration is necessary to overcome the complex matters at hand. At times when we are facing a new data breach scandal which has purportedly been utilized in manipulating the US elections, and new laws are being passed on in the European Union, it is illustrative to look at a book of this sort which argues how legal action needs to be taken in the newly emerging conditions of the information society.
As a law professor he approaches the subject from the legal perspective mostly, with a lot of cases to clarify and support his stance. Breaches happen regularly, but with firms such as Facebook getting larger and larger and methods of data analysis improving, it is becoming a more and more encroaching threat to civil liberties, democracy and the conduct of our lives.
The entire book could be read as a call for transparency against black box systems in an age of information technologies where regulations are amiss and there are no checks on the companies that describe and shape our lives in all kinds of hard to perceive if not imperceptible ways. Frank Pasquale is calling for a reform in laws to scrutinize how these businesses operate and to bring in accountability rather than obfuscated systems where decisions that influence people’s lives remain unanswered.
From the financial crisis of 2008 where automated and occluded financial transactions has led to less supervision and high risk-taking behavior and deliberate manipulation, to insurance companies and banks who set out to give people credits based on unaccountable scores through black-box systems as though from a Black Mirror episode, to job applications and health this data is then used to give critical judgments about people without them even realizing it, and search engine results that are prone to display unwelcome and uncalled for results, the book presents a plethora of cases and situations where big data analysis has gone awry. It also argues that tech companies have become giants that are doing what they can to dwindle competition. The author argues that they were lucky to find themselves in the right time and right place and has then went on to use that early-comer advantage to dominate the sector. Indeed, who has now the capacity to out-compete Google? The obtaining and possession of data about the audience is then garnered and sold to marketers who want to be able to target specific audiences which in turn drive in huge revenues. There is no free lunch, and our attention is what these companies are hoarding and selling.
The author depicts how automation of job application procedures such as application screening software and personality tests can lead to inexplicable results and lack of accountability. Important facets of people’s life such as credit scores and job applications are increasingly decided by black-box algorithms that are guarded from scrutiny. For example a job applicant may find it difficult to land on a job interview despite having all the skills and qualifications for not having the right kind of curriculum vitae which is screened by computer algorithms. Or insurance may be denied to certain people with a predisposition to diseases in order to cut down costs and increase profits all of which may be obtained through big data analyses and profiling.
Albeit technology evangelists may invoke the benefits of the system as objective and agnostic tools, it is far from clear that biases are not somehow embedded in these systems and as long as they remain closed, it can only be disputed without end. Frank Pasquale talks about how easy it may be to be misled into believing that these systems which can conjure up results so speedily can lead you to have a bias towards automation that these devices can regulate the financial sector without the human prejudices and ineffectiveness. However, this book contains many case studies where such automation in social spheres go wrong and he warns us against the use of automation in human affairs.
From a political perspective, this black boxing of decision making are empowering those at the top and disenfranchising those at the bottom by giving them excessive power in remolding society, and re-instigating racial and other prejudices which would have been punished if it was done deliberately.
The matter at core is checks and balances. Avoiding companies to grow too big and become monopolies that shape our lives with not many alternatives to turn to, and avoiding a black box opaque system the workings of which are not clear. Decentralization and opening up of the boxes is what Frank Pasquale is advocating as a professor of law.
He criticizes no-liberal economists whose arguments such as the availability of alternatives and competition and consent of the web-consumers conceal the rapacious hold of big tech firms on the markets and over us. Neo-liberal arguments want to take regulation out of the picture and they want to have their own thing without the pesky government peeping in.
Indeed as can be seen from the many examples presented in the book natural sciences should not be applied on society, as society is dynamic and shifting. There is no brute data. As a shocking search suggestion draws attention, the link between the search and the object of the search become stronger although that might not be the wish of those who click that suggestion. Automation brings all sorts of problems with it. The financial sector’s reliance on computer aided transactions have also cause autonomy bias, which is to think that computers will bring prejudice-free results by taking out the intermediary human counterparts. Today, we are relying so much on these technologies and they are shaping us.
We hear and read about technology mostly insofar as it is marketed to us as something that will bring benefits. We hear about it as the path towards utopia, but our lives are increasingly more like the Black Mirror series. Big data is mostly heard through a business perspective and a new hype and re-branding is generated. This book depicts the dark side of the technological paths we are subscribing to. Reading this book people working in the related fields will find themselves questioning their choices. The author suggests anti-trust laws. Closed systems can lead to another global economical crisis and perhaps other more grave dangers may be proliferating in the darkness.
There is more to big data then just bringing in better health results through matching up correlations without necessarily finding out causes as for example Kenneth Cukier says, and it is not just the benefits of what such data analyses can bring, but also what it means for political freedom, financial security, and individual liberty.
In conclusion, this book is a convincing piece of writing that shows more attention needs to be paid on the business models of such tech firms and the social and legal aspects of their use of data especially in the wake of GDPR and new regulations. In my opinion scholars should keep a firm eye on this and establish consortiums to work together in dealing with all this complexity.
“Faulty data, invalid assumptions and defective models can’t be corrected when they are hidden”
anlasanız da anlamasanız da sorumlusunuz (!)