How Big Data is Changing Democracy

Humans now create an extraordinary amount of information every day.1 Unique content, regardless of veracity, is distributed with unprecedented velocity to an ever-growing audience. Individual citizens’ patterns of life are now collected by everyone from commercial organizations to malicious actors. Methods to collect, collate, and analyze data become more agile and widely available every day. Institutions in democracies must adapt to their citizens having more of a voice, and being more vulnerable, than ever before. This means that many organizations need to modify their behavior.

Governments need to assess and regulate an environment where they may no longer have the most data on individuals. Commercial organizations must accept some responsibility for the data and information they gather, and what it can be used for. The press must adapt to a new paradigm of information sharing and act to protect their audience. Finally, all these things must happen in concert, in an environment of public-private partnership that somehow protects personal freedoms, organizational integrity, and private revenue.

Data and Democratic Government

In governments, data management is usually subordinate to the larger information technology operation. In many cases, governments recognized the need for a concerted technology management offering in the 1990s, began delivering it in the early 2000s, and largely completed implementation by the 2010s. Over the course of that long timeline, the service architecture identified and created in the late 1990s and early 2000s became inadequate in a cloud and mobile-based world. Monolithic government IT departments, designed for the centralized physical management of the early 2000s, now scramble to modify their services in time for their customers to use them.

The Seattle Times recently highlighted an auditor’s report that castigated the Washington state technology agency for poor service delivery and operating at a deficit.2 However, the agency is shackled by the $300 million data center that thestate built in 2009. The state’s data, along with its technology services, are trapped in a massive capital artifact. This is not an isolated example, either in the United States or the larger democratic world.

The organizations that underpin democracy are unwieldy. Changing institutional and organizational processes takes a long time. Often it takes too long. Technology and even public sentiment moves on before changes are made. An issue or threat that requires vital attention becomes obsolete, but morphs into new requirements. An amusing report from the July 1922 United States Naval Institute proceedings bemoans how long it takes for a “minority that ‘knows’ to instruct the vast majority until sufficient public sentiment is created to force the change.”3 The Institute was talking about using Navy yards as industrial establishments—the 1922 version of requiring large government agencies to be nimble in their tech processes.

This is most apparent in the municipal and national governments of relatively established democracies. Elected legislatures with appointed functional officers, like most democracies, tend to be risk averse with what they buy and how they direct that it be used. Directing change in a democracy creates risk at a very personal level for those who administer it. Professional democratic officers tend to not be familiar or comfortable with the processes, education, or technologies that allow effective data sharing and security, and move slowly when acquiring or directing it. The Information Technology and Innovation Foundation described this slow and obstinate policy environment in a 2016 report advocating for digital infrastructure improvements.4 They warn that “outdated and costly policies designed for the infrastructure of the 20th Century” put modern and effective policy at risk.

"Changing institutional and organizational processes takes a long time. Often it takes too long. Technology and even public sentiment moves on before changes are made.”

New Vulnerabilities and New Obligations

This sloth, however, is not acceptable in the current environment. The same report warns how cyberattacks on supervisory control and data acquisition (SCADA) systems have increased from 90,000 worldwide in 2012 to 675,000 in 2014. These are the systems that collect on and control industrial processes like air coordination, electric power generation and transmission, and wastewater management. In 2016, according to Security Week, attackers took control of a regional US water utility and acquired control of the flow of toxic chemicals.5 Another group took control of the transit system in San Francisco. This year, unknown hackers released malware targeting systems in the Middle East and Europe. That software not only wiped systems, but it provided attackers access to systems and was used for “espionage against an unknown number of targets.”6 The 2017 Equifax breach compromised and distributed personal information from millions of people in at least three countries.7

Most information that used to reside in wallets and safe deposit boxes is now digital, and data that supports it grows every day. This is a good thing. More people than ever before have greater access to education, banking, legal services, and identity verification. However, the current regulatory environment lacks a global commitment to collection and protection standards. Most critically, there is a bewildering array of acts, guidance, and agreements across dozens of countries that attempt to address the new size and shareability of citizen data. While almost everyone agrees that personal data should be protected, there is little consensus as to how and who should be responsible.

Estonia’s digital-first government is widely hailed as the best way to deliver services to citizens in a modern world.8 But even Estonia was subject to Web War I, and had to rapidly retool its security and alliances after a massive cyberattack in 2007.9 Democratic governments around the world have responded to these threats by treating them like familiar military or law enforcement challenges. The attacks on Estonia led to the creation of NATO’s Cooperative Cyber Defence Centre of Excellence.10 Similarly, many governments have implemented cyber commands, or other military structures, in order to wage “warfare.” Cyber warfare, while nebulous, is something that legislators can at least conceptualize. There are attackers and defenders, and fortifications to build.

However, this dramatically overlooks the amount of private data that is created and shared every day by private citizens.

Sweden, the first country to “enact a comprehensive statute to protect the privacy of personal data on computers,” serves as an example.11 They are clearly proactive in protecting its citizens, going so far as to provide their personal data constitutional protection. But they are “much less concerned with...voluntary use and submission [of data].” This ignores the current reality of massive and ever increasing user generated data.

A Swedish citizen with credit in the United States is technically choosing to give organizations like Equifax their personal data. However, the choice is between taking the risk of not being protected by their government, or losing whatever personal opportunity requires US credit. Similarly, a citizen exposed to a malware infected public system has made a personal choice, but possibly a choice of necessity. Most commonly, a citizen who experiences personal loss or harm due to voluntary information sharing has little legal redress. In 2016, many of Facebook’s users were worried by the suggestion that the social media service used location data to identify potential connections.12 This may or may not have been covered by terms of use, but customers were taken aback at being tracked, arguably, without their consent.

The ubiquity and even necessity of social media and internet-based services mandates a re-evaluation of citizen protections. Governments must recognize that their citizens spend a significant portion of their time in cyberspace, exposed to all of the risks discussed above, and actively engaging with many of them. It is not enough to excuse negligent or malicious action simply because a citizen has voluntarily used a service. It is certainly not enough to limit protection or at least benevolent counsel for a citizen concerned about their data who still wants to engage with the digital world and all its opportunities.

Responsibilities for Private and Commercial Organizations

Even if the government regulation pivots to establish more global standards and more fully protect citizens’ activity online, compliance with regulation is not enough to ensure that private companies are protecting their customers. As discussed above, government can be slow and can get left behind by technology. That vulnerability is exacerbated by the fact that citizens are creating and sharing information faster and in greater volume every day.

Private organizations can act quickly to protect their revenues and customers but must use that agility to shift away from planning for possible liability to an active focus on data protection. The 2013 data breach at Target compromised millions of personal records despite the company largely being in compliance. Attackers gained access by exploiting a vendor who did not have appropriate safeguards.13 Expert researchers determined that the records lost in this attack will continue to be at risk until “changes are made to the technology behind payment cards.”14 Assessments like this created a new urgency to shift to chip-enabled cards, which was a positive development, but customers could have been protected earlier. Compliance with government regulations neither identified the systemic vulnerability nor the need to change payment cards.15

Protecting citizens across the entire data supply chain is an important part of doing business in the 21st century. The information security functions at large organizations understandably must try to balance cost with compliance but have some obligation to use their agility to protect their customers better. Massive repositories of personal data like social media and search services cannot claim that compliance with possibly outdated government regulations is enough if their customers are exposed by breaches in their service. They can, however, actively engage with digital policymakers and forge partnerships that support both private agility and protective regulation.

Linking Information Organizations in Government and Industry

Fundamentally, commercial organizations must realize that all information collected from their users, whether loyalty program applications, a payment, or social media posts, is protected first by their own competence. Given that, any organization that collects data from their customers must prioritize protecting it and build those protections into their organizational processes.

Part of creating sustainable processes to protect data is working with both legislative and law enforcement agencies within governments. As discussed above, it is unlikely that regulation will keep up with risks in the private market. Further, layering regulation and compliance obligations is probably not an effective way forward if governments and societies wish to maintain the current pace of innovation and development.

One solution is the previously mentioned focus on active data protection, coupled with injecting expert staff into government regulatory and law enforcement agencies to judge the efforts. Rather than creating hard and fast rules that become obsolete quickly, governments must create a bench of expert knowledge that can judge compliance by the unique active steps to protect data taken by commercial organizations. An organization is compliant if experts come to the consensus that they made all possible efforts to protect their data. This gives each organization the freedom to exercise their agility and even share techniques. Then, should a breach occur, companies would not be judged by compliance to outdated regulations, but rather, by an expert evaluation of their current, proactive security measures.

This is, of course, a brief description of a complex and paradigm-changing public-private partnership. Such an arrangement would depend on leaders in multiple industries agreeing to let their security efforts—and liability—be judged with an element of subjectivity. It would likely not be enough for a legislative or executive committee comprised of non-experts to make decisions based on investigative reports. Democratic governments would likely have to recruit and maintain significant new expert talent in their regulatory agencies to serve as arbiters, and these new personnel would have to be widely respected across multiple industries.

Such an effort would have far-reaching effects, especially in legal and judicial environments. It would also probably take years to set up and require commitment to socialize and sustain the processes through early challenges. There would be further time involved in selecting the government personnel who would be making judgements on organizations’ performances, as these individuals’ personal judgement would exercise a significant amount of power over commercial organizations.

However, there is precedent for an industry-spanning data management partnership between a democratic government and commercial organizations. This even included special government employees with wide leeway to evaluate and direct the actions of commercial organizations. PRISM, the US government program to extract intelligence on individuals, used judgements from a specially established regulatory agency and widespread cooperation from a broad swathe of the internet services industry. It even facilitated a similar program in the UK.16

While this is a rather sinister example, it does speak to the feasibility of regulating private data security through expert judgement and private cooperation. It would be a larger scope program than simply intelligence gathering, and would encompass more of the commercial world, but the concept is proven.

Velocity, Veracity, and Unique Vulnerability

One of the traits that defines big data is velocity: how quickly data changes or new data is created. With people now able to create and share content with almost no cost in money or time, the velocity of unique content available to everyone is unprecedented. Also, importantly, the velocity with which this content is shared is also unprecedented.

For example, consider a social media post made by a single person with a relatively small, connected audience of 100 people. If that post is seen and shared by just five of those people, who can be assumed to have the same modest, connected audience, the original content has reached 600 people. Given that most of this happens using mobile devices on cellular networks, there is relatively little time lag. If the content is broadly interesting, there is almost no lag between creation and widespread sharing. Literally hundreds of thousands of people created and shared content on the death of Osama Bin Laden before the US president had even finished his official remarks.17

An easy experiment is to post any content, even nonsense, along with the top trending hashtag at the time on any social media service. Depending on the popularity of the service, that post will be seen by at least thousands, and often tens of thousands, of people within an hour.

This seems to describe an exciting new world where every citizen has a voice. Content creation and internet publishing have given literally millions of people a platform and an audience. This can be a great thing, connecting creators, thinkers, and doers all over the world. It also ensures that every citizen has at least the opportunity to make their wishes or complaints heard outside of potentially stifling bureaucratic processes.

However, the internet does not discriminate between well-intentioned informative content and maliciously created propaganda. Further, the rapidity with which content is shared makes it difficult for the average information consumer to judge what is good content. Veracity, another trait of big data, is often obscured by the velocity with which internet content is shared. Some of the more harmless malicious content sites (fake news) garnered 1 million views in just a few weeks, and stories were propagated instantly over multiple social media sites.18 The average viewer, when sent a story in multiple formats, often paraphrased by commenters or posters, has a hard time recognizing that it is dubious, single-source information rather than accepted as true.

It is easy to recognize poor or malicious journalism in traditional outlets. There is a professional organization that maintains standards, and reliable outlets support each other and broadcast their reliability.19 However, there is not yet a concerted effort to react to incorrect or malicious, shared, online content. Journalists and free-speech advocates are concerned and want help from major internet companies like Google and Facebook, but they do not yet operate aggressively against fake news.20

In the past, journalists’ ability to share good information shaped government and society, mostly for the better. Now, however, everyone can share information. Everyone with an internet connection can create content, unbound by any professional ethic or societal expectation, and share it with the world. The average information consumer is not trained to evaluate sources or verify information and is now vulnerable to malicious information being delivered at a velocity that masks its nature.

The modern free press is uniquely positioned as an expert, well-regulated, and relatively ubiquitous population in democratic countries that can actively identify and combat fake news. Identifying and debunking hoaxes and false claims need not be limited to boutique or specialty outlets, but rather can become a familiar part of traditional reporting.

Protecting Citizens

This writing is a relatively glib discussion of large-scope issues facing democratic institutions in the modern information environment. Benevolent but effec- tive government, innovative but responsible private enterprise, and an honest and free press are mutually supporting pillars of effective democracy that must adapt to how much data we now create and share.

Democratic governments cannot deny their citizens protection, or at least efforts at security, for voluntarily participating in the information sharing that is now a part of daily life. Most fundamentally, all governments need to clean up the layers of legislation and executive policy that deal with protecting their citizens’ data and information. They must commit to actively protecting their citizens online in the same way they do in the physical world. Further, they need to move away from military-style approaches to information security and commit to at least some policy adaptation that allows for agility.

All of that can be facilitated by broad partnerships with private industry. Almost all Internet and data services are provided by commercial organizations, and entities across industries now maintain granular intelligence about their customers. These organizations are motivated to protect their revenues and have seen reverses related to consumer data breaches. Their efforts, necessarily more agile than legislated regulation, can help inform government standards. Further, a regulatory environment that allows leeway and incentivizes proactive data security would protect citizens’ data better. However, both government and private entities would have to commit to mutual effort and sustainment to create such an environment.

Finally, the press can position itself to protect citizens from malicious information sharing. Journalists around the globe have taken steps to diminish the reach of false information sharing by advocating for action within the Internet services. However, as information evaluation and sharing experts, they are the best qualified to identify and counter fake news as it is created. Active measures to weed out such content, along with efforts to guide internet services, are real ways the free press can help protect citizens in the current information environment.

Venkat Motupalli is the Chief Information Officer of the New York City Department of Veterans’ Services (DVS) and a lecturer in Columbia University School of International and Public Affairs, teaching in the Technology, Media, and Communications (TMaC) specialization. Most recently, he led a project to map the veteran population in New York City, a project that received Government Technology’s 2017 Best of NYC Award for Data Analytics and Business Intelligence.