This article appeared in The Cyber Issue in Winter 2016.
Attribution of malicious cyber activities is a deep issue about which confusion and disquiet can be found in abundance. Attribution has many aspects—technical, political, legal, policy, and so on. A number of well-researched and executed papers cover one or more of these aspects, but integration of these aspects is usually left as an exercise for the analyst. This paper distinguishes between attribution of malicious cyber activity to a machine, to a specific perpetrator (often a human being pressing the keys) initiating that activity, and to an adversary that is deemed ultimately responsible for that activity. Which type of attribution is relevant depends on the goals of the relevant decisionmaker. Further, attribution is a multi-dimensional issue that draws on all sources of information available, including technical forensics, human intelligence, signals intelligence, history, and geopolitics, among others. From the perspective of the victim, some degree of factual uncertainty attaches to any of these types of attribution, although the last type—attribution to an ultimately responsible party—also implicates to a very large degree legal, policy, and political questions. But from the perspective of the adversary, the ability to conceal its identity from the victim with high confidence is also uncertain. It is the very existence of such risk that underpins the possibility of deterring hostile actions in cyberspace.
Attribution of malicious cyber activities is a deep issue, about which confusion and disquiet can be found in abundance. Attribution has many aspects, and a variety of well-researched and well-executed papers cover one or more of these aspects; these papers are referenced in the body of the paper and are called out again in the acknowledgements section. This paper tries to synthesize the best aspects of these works with some original thoughts of the author’s own into a coherent picture of how attribution works, why it is both important and difficult, and how the entire process relates to policymaking.
The primary takeaway messages of this paper are that (1) attribution has a different meaning depending on what a relevant decisionmaker wants to do (i.e., attribution of malicious cyber activity can be to a machine, to a specific perpetrator (often a human being pressing the keys) initiating that activity, or to an adversary that is deemed ultimately responsible for that activity); (2) attribution is a multi-dimensional issue that draws on all sources of information available, including forensics, human intelligence, signals intelligence, history, and geopolitics, among others; (3) all attribution judgments are necessarily accompanied by some measure of uncertainty; and (4) an adversary cannot be fully confident of its ability to conceal its identity from the victim.
What Is Attribution About?
Every parent who has ever broken up a fight between two children and tried to figure out what happened has asked, “Who started this?” The question expresses our very basic concerns about responsibility for actions that lead to conflict or harm.
Concerns about responsibility for actions or for events are embedded in domestic law. A person is found on the street with a bullet through his head, and we want to know who fired the shot. Much of our criminal justice system is devoted to “fair” processes that we believe can determine the identity of that person with sufficient certainty to mete out an appropriate punishment. International law is concerned with questions of responsibility as well, especially as it relates to matters involving conflict. With a number of important (and controversial) exceptions, states are usually regarded as accountable for actions that emanate from within their borders.
Similar concerns about responsibility are also present in cyberspace, but just how they play out is often quite different, for reasons both technical and historical. Usually captured under the rubric of attribution, concerns about responsibility generally arise when a malicious cyber activity or incident is known to have happened. “Who (or what) is responsible?” is then often the question of interest.
If this question cannot be answered, it may be hard for victims to mitigate ongoing harm; to do so would require the victim to be able to quickly and correctly identify the instrument or mechanism causing the harm and find a way to stop its malicious activities. Further, it would be impossible to punish the parties responsible for causing the incident. And, if punishment is impossible, deterrence of malicious activity in the future is also difficult to achieve.
We begin with a working definition of a cyber incident. We recognize a cyber incident when something “bad” happens to an information technology-based system. In this context, badness involves errant behavior of the victim’s computer (or a system involving a computer)—that is, the computer or system behaves in a way that it should not behave. Examples abound: the computer freezes; commands given to the computer do not have the expected result; the printer spews out paper with gibberish. More serious examples of badness include: a drive-by-wire car does not slow down when the driver presses the brake pedal; the computer-controlled missile misses a target when it should have hit it; or the ATM machine at the corner bank dispenses hundreds of $20 bills onto the street.
Investigations are usually (but alas, not always) triggered by errant computer (or system) behavior. But apart from routine inspections, investigations will not occur if the errant behavior leaves behind no clues that it has occurred. Similarly, clues may be noticed only long after the precipitating actions or events have occurred, making investigations much more difficult.
The first part of the investigation is determining that something “errant” has happened at all. In all of the examples above, it is pretty clear that an undesirable outcome has occurred, and the undesirability demonstrates or at least suggests a breakdown in the program’s functionality. But consider the case in which a computer system (and anything that is controlled or affected by that system) produces an undesirable result or outcome that is what would be expected given the inputs. (Most people who have tried to balance a checkbook by hand, or even with a calculator, can speak to such an experience.) In such cases, it is far more likely that the result—though undesirable—is correct and inevitable because the user has provided bad inputs than it is that the program used to calculate that result is in error.
Similarly, if the missile misses its target or the car does not slow down when the driver presses the brake pedal, it is possible that a human operator aimed the missile at a shadow or the driver pressed the accelerator when he thought he pressed the brake. In such cases, it is hard to associate “errant” behavior to the computer or system per se, since the system was given the wrong input. It is also possible that the errant behavior is the result of a flaw in the program, introduced by accident rather than intentionally.
Errant behavior resulting from factors other than foul play does not usually play a part in traditional attribution concerns. Attribution usually arises as a concern when an incident is determined to have resulted from foul play (i.e., intentional harm). When the determination is made that foul play was involved, what was previously a cyber incident involving errant system behavior becomes a malicious cyber incident (or, equivalently, an intrusion)—and attribution is the process by which it is determined who or what is responsible for the intrusion.
Attribution sometimes goes hand in hand with determining if a cyber incident is malicious. That is, an investigation regarding the cause of errant system behavior may (or may not) reveal it to be the deliberate and intentional action of an actor. But identification of the specific actor is not necessarily required to infer bad intention—in many cases, a particular behavior of the system is so likely to be the result of an intentional bad action that investigators presume maliciousness.
Suppose that Bill is the legitimate user of a computer in the human resources department of a large defense contracting firm. He has been putting together a spreadsheet with all of the names, addresses, email addresses, and salaries of the other employees of this firm. One day, he opens his computer to discover that the spreadsheet has been deleted from his hard drive. He reports this to IT support, which then begins an investigation. What happened? How did the file get deleted?
The IT support staff may begin by examining who had access to the file. Susan, Bill’s direct supervisor, also had access to the file. Network records demonstrate that Susan’s computer did access and delete the file the evening before Bill reported it missing. Susan, however, claims that she did nothing to the file. Is Susan forgetful or lying? Or was she somehow tricked into deleting the file? Or did someone else access Bill’s file, pretending to be Susan?
Perhaps the investigators determine—or make an educated guess—that Susan is indeed telling the truth, and that she inadvertently deleted the file without knowing it. Who set this action in motion? In this case, misdirection is involved: on the surface, Susan appears responsible, but she did not wish for the file to be deleted and does not actually bear any meaningful responsibility for ill intent.
But the IT support staff may determine that an intruder engineered this attack through Susan’s computer. Attribution has two goals: to distinguish between errant behavior that is malicious and deliberate and errant behavior that is accidental, and if the former, to distinguish between intentional, real, and meaningful responsibility on one hand and apparent responsibility on the other. The latter goal focuses on the question of who set this event in motion. However, determining “real” responsibility is much more difficult than it may initially seem. This paper explores different ways of understanding attribution and, subsequently, responsibility for a malicious cyber incident.
What Does Attribution Mean?
Ascertaining responsibility for malicious cyber activity can be understood in a variety of different ways because the term “responsibility” has a number of possible meanings, any or all of which may (or may not) be relevant in any given situation.
Working through a concrete scenario helps to unpack the meaning of “responsibility.” The following scenario, as seen from a God’s-eye perspective, involves Tony, the systems administrator for a Department of Defense (DOD) computer system in San Francisco. This computer system is attacked (and in this instance, has been the subject of a remote-access attack in which an unauthorized party—George—took direct control of it as if he were sitting at the keyboard in San Francisco). The attack traffic came from a computer based in Arkansas, owned by Karen, an eighty-four-year-old woman. The computer in Arkansas, however, was compromised through a computer in Greece. George sat at the keyboard in Greece and pressed the keys that set into the motion the attack against the DOD computer in San Francisco. George is a citizen of China. However, he is also a member of a Russian organized crime group. The head of that crime group, Sergey, is a close personal friend of a senior operative named Ivan in the Federal Security Service (FSB) in Russia. Ivan and Sergey had dinner two weeks ago, and while Ivan and Sergey did not talk about computers or hacking, Ivan did tell his close friend that he was having problems with some activity happening at a DOD facility in San Francisco.
To whom or what should the attack on the U.S. computer in San Francisco be attributed?
Three Meanings of Attribution
In principle, such a question can be answered in three ways, which are not mutually exclusive. The possible types of answers are a machine (or set of machines), a party whose conduct or behavior set the intrusion into motion (often a specific human being), and an ultimately responsible or accountable party. To distinguish between the party behaving badly and the ultimately responsible party, the reader should understand the term “intruder” (or, equivalently and interchangeably, “perpetrator”) to mean the former and the term “adversary” to mean the latter. It is the hostile behavior of an intruder that makes the intruder’s identity interesting to the victim—the victim may wish to prevent or deter future hostile behavior. It is the hostile intent of the adversary that makes the adversary’s identity interesting to the victim—the victim may wish to hold the adversary accountable. Some degree of uncertainty attaches to any specific answers. Which possible type of answer should be sought depends on the goal of the relevant decisionmaker.
Attributing malicious cyber activity to a machine (or machines)
In the above example, attributing the intrusion to a machine would require identifying the computers used to perpetrate it on the DOD computer in San Francisco. The easiest machine to identify is Karen’s computer, since that computer is proximate (in cyberspace) to the DOD computer. Any other computers through which the intrusion was routed are also of interest because each computer in the path points to one or more additional links. The trail will eventually stop somewhere, either at George’s computer because the evidence collected along the way suggests that George’s computer is in fact the originating point of the attack (a good outcome) or somewhere else because the trail peters out (a bad outcome). Following Clark and Landau, an intrusion in which multiple computers are used in a chain to reach the intended target is called a “multi-stage” intrusion.
Ascertaining the machines associated with a malicious cyber incident usually involves technical forensics—the art and science of looking for technical clues left behind in an intrusion. In tracing the origin of the activity, it may be necessary to gain access to Karen’s computer to obtain any relevant information it might contain. Technical forensics could also be performed at the network level without needing direct access to Karen’s computer, e.g., by examining various logfiles that document activity on the servers in the network. (In general, technical forensics at the network level must examine large volumes of mostly irrelevant information to find the few, if any, relevant entries.)
For example, technical forensics applied to the DOD computer may reveal the IP address of Karen’s computer, which was the one most immediately and proximately connected to the DOD computer in San Francisco. By consulting a service that provides geocoded IP addresses, investigators may learn that this computer is in Arkansas. Internet address assignment authorities will show the name of the Internet service provider (ISP) associated with that specific IP address—call the ISP in question Castcom. Using a subpoena, investigators may then ask Castcom to reveal the name of the subscriber using that IP address at that time. Castcom may or may not be able to provide that information. For example, the logs containing a dynamic assignment of their customers to IP addresses may only be retained by them for a brief time, or they may be using a technology called Carrier Grade NAT (Network Address Translation) that shares a single IPv4 address among a multiplicity of customers. Should Castcom reveal that the name of the subscriber is Karen, and that her address is 132 Main Street in Little Rock, Arkansas, Karen may receive a visit from investigators armed with a search warrant who demand access to her computer to gather further information.
On the other hand, if Karen’s computer is found in another country rather than in the United States, it is likely that a different set of procedures would obtain. Under some circumstances, investigators may ask law enforcement authorities in that country for assistance. Under other circumstances (such as the refusal of that country’s authorities to cooperate), they may simply find a technical way to gain access (e.g., hack into it by sending an authorized user of the computer an email that grants access when the victim clicks on a link in the email).
In either case, the proximate computer may well hold additional clues that help to identify the next link in the chain. For example, they may find malware on Karen’s computer that periodically contacts a particular IP address in Greece.
Technical forensics can be challenging, especially in an environment in which multi-stage cyber intrusions are conducted. Complicating the technical forensics job even more is the use of anonymity-enhancing tools; such tools obscure technical information that might be used for forensics. Impeding technical forensics may serve a socially desirable goal when it protects people who engage in politically controversial dialogue, but anonymity-enhancing tools can also be problematic when they help malicious cyber actors to evade responsibility for their actions and get in the way of identifying the actual machines involved in perpetrating an intrusion.
TOR is a good example of an anonymity-enhancing tool. TOR is a system that enables users to communicate more anonymously across the Internet with ease. TOR traffic is automatically encrypted and routed through many different nodes around the world rather than being routed directly. A list of anonymity enhancing tools is maintained by the Electronic Privacy Information Center, and the proper use of such tools increases the difficulty of performing technical forensics.
At the same time, anonymity-enhancing tools are only one side of the coin. Efforts to improve technical forensics are also underway. A contemporary example is the DARPA Enhanced Attribution Program (Box 1).
A second source of information that can contribute to an attribution judgment is honeypots. A honeypot is in essence a decoy, configured to look attractive to an intruder but instrumented so that the intruder’s behavior can be clandestinely observed and monitored. If and when the same intruder returns to the targeted installation, his behavior can be recognized more easily.
A third source of information useful for machine attribution consists of pre-positioned instrumentation. In some cases, pre-positioning of instrumentation occurs in systems and networks that an adversary might use to launch an intrusion. Thus, if that adversary initiates an intrusion, the pre-positioned instrumentation can record data streams that, when properly interpreted, indicate the nature and source of malicious activity underway. Such instrumentation was reportedly part of the attribution to North Korea of the attack against Sony Pictures Entertainment in 2014. Use of pre-positioned instrumentation obviously presumes a prior policy decision that a particular adversary may launch future intrusions and that an investment in anticipatory emplacement of such instrumentation is therefore justified.
In other cases, instrumentation is pre-positioned as a matter of good security practice on the part of others or even good luck. In the first instance, consider the possibility that an intruder is able to successfully launch an intrusion that appears to be coming from Institution A. If Institution A has installed instrumentation that monitors traffic in and out of its networks (a good security practice), Institution A may be able to show that it was not in fact the source of the intrusion. That fact may in turn provide information on the techniques used by the intruder. Good luck may contribute if the intruder unwittingly reveals actions that may be preparatory to the intrusion. In both cases, information potentially relevant to attribution is uncovered, and if shared among the relevant parties, that information may actually be relevant.
Two observations about this process are noteworthy. First, attributing to a machine or an IP address is not the same as identifying the perpetrator. Technical information can point to a computer located at IP address 22.214.171.124 and note that this particular IP address is associated with someone calling himself George. While that piece of information is suggestive, it does not imply that George was necessarily the individual who pressed the keys initiating the attack.
Second, as Clark and Landau point out, the use of one or more intermediaries (in this case, Karen’s computer) through which to route an intrusion greatly complicates the technical forensics task. Investigators start with information found on the DOD computer, and this information points to Karen’s computer. They need information from Karen’s computer, but their access rights to that privately owned computer in Arkansas are more limited than if they had full control over it (which they would have if it were a DOD computer). In addition to their technical tasks, they now also face tasks based on law and policy about how and to what extent, if any, they may access Karen’s computer. If the law and policy are clear in any given instance, those tasks may be relatively easy to complete. But if they are not (e.g., if Karen’s computer is in Brazil rather than Arkansas, and the investigators need Brazilian permission to access Karen’s computer due to a bilateral agreement between the two nations), carrying out the full range of technical forensics needed may be much more difficult.
Attributing malicious cyber activity to a specific perpetrator
Attributing malicious cyber activity to a specific perpetrator means ascertaining the identity of the person or persons or organizational entity directly involved in perpetrating it. In the example above, attributing the activity to its human intruder means identifying George as the person who pressed the keys on the keyboard located in Greece needed to launch the intrusion. In other cases, the organizational entity might be a “hacking for hire” company operating openly in a nation whose domestic laws do not prohibit such activity or operating underground in a nation that ignores such activity even if it is illegal under that nation’s domestic laws.
Technical forensics alone cannot definitely determine the identity of the person sitting at that keyboard in Greece, because technical forensics usually look only at information that may have been left behind on the various computers in the wake of an intrusion. However, someone else may have stolen George’s login credentials to pretend that she is George, and the identity of the credentials thief may not be discoverable using only technical forensics. (A non-cyber analogy is that John Doe’s car may have been the car that killed a pedestrian, but this does not mean that John Doe was the one driving the car.)
In the example above, investigators might consult historical records and find that this particular Greek IP address has been identified many times in the past as an originating point for a variety of Chinese and Romanian hackers. But the particular malware found on Karen’s computer has been used primarily by Chinese hackers in the past, thus suggesting that Chinese rather than Romanian involvement in this attack is more likely.
Yet another clue might be found in a Chinese online discussion forum that is ostensibly private but that has been secretly infiltrated by a U.S. intelligence agency for a number of years. In this forum is a question from George asking for the most recent information about security measures taken at the DOD computer facility in San Francisco—and the date on which this question was posted is eight days before the attack on the San Francisco computer.
If enough such clues can be accumulated, the investigators may have sufficient confidence to point to George as the most likely perpetrator of the intrusion on the DOD computer in San Francisco. Of course, how many and what kinds of clues are “enough” is an important question and is the focus of the next section of this paper. Another important question is the strength of these clues, since no one clue is likely to be definitive (i.e., investigators of such incidents rarely, if ever, find a “smoking gun”). For purposes of attribution, investigators may require a large number of clues that point only weakly to a given person or a fewer number of clues that point strongly to that person.
There are many instances in which technology can help facilitate attribution to a human intruder. Authentication is the process through which specific individuals can be better tied to technical online activities and actions. Most people are familiar with the ritual of entering a login name followed by a secret password. If the login process is successful (and the user’s login credentials have not been compromised), the user is granted access to a variety of privileges on the relevant computer system, and many of that user’s actions on the system can be associated with him or her personally.
If the user goes beyond the local computer system onto the Internet, an ISP will have provided Internet access. That ISP will often have information on file about the individual in order to provide access (e.g., where the individual is) and to receive payment (e.g., through the individual’s credit card), and so the ISP may have some insight into the Internet activities of its subscriber as individuals. (The ISP may not have complete insight into activities carried out on its networks. For example, if the individual sends emails with attachments encrypted locally, the ISP will know about their recipients, but not know about their contents. But complete information might not be necessary for an attribution judgment, depending on the particular pattern of facts and circumstances that obtain at the time.) Using the ISP’s records on its subscribers, an investigator would be in a better position to attribute some activity carried on its network to a particular individual.
And technical means do sometimes point directly to specific individuals. For example, the way an individual types on a keyboard may be sufficient to specify that individual uniquely—that is, no other person in the world would type a particular passage of text with the same timing of keystrokes. If the human intruder is using a remote access tool to explore the victim’s computer system or network, a keystroke monitor may be able to capture such data. (Indeed, the DARPA program on Enhanced Attribution described above uses keyboard dynamics as one aspect of identifying virtual personas of intruders.) Similarly, hacking into the computer in Greece to turn on its camera and capture a picture of the person at the keyboard would also yield useful information.
Such means can indeed provide useful information about an individual’s identity, just as a DNA signature (e.g., a specific genomic sequence belonging to an individual) or fingerprints can point to specific individuals. But none of these signatures—keyboard, pictures, DNA, or fingerprints—are of any value in identifying the individual unless there is some database against which the given signature can be compared and an identity uncovered. That database is the essential link between specifying an individual and identifying that individual, and technical forensics applied to any one incident, cyber or otherwise, cannot populate that database. In the absence of such a database, the most that can be said is that the same individual perpetrated two or more intrusions, but this individual will not be identifiable.
Compromising this link is also inherent in the possibility of stolen credentials. Someone may have used George’s credentials to gain access to the computer in Greece, but how do we know if that someone was actually George? Two-factor authentication is a stronger form of authentication that calls for the user to present something he or she knows (e.g., a password) with something he or she has (e.g., a token or a smart phone). The use of two-factor authentication reduces the likelihood that an attempt to impersonate George will succeed. But two-factor authentication is not foolproof, as a gun held to George’s head will also probably serve the same purpose for someone determined to use George’s credentials.
More generally, even if George can be identified as the perpetrator of the intrusion, it is often important to know why George did it and who asked him to do it. For many purposes, the identity of the party responsible for setting the intrusion into motion is quite important. Who is the party that is ultimately responsible for the intrusion, that is, who is the adversary?
Attributing malicious cyber activity to the ultimately responsible adversary
At whose behest was George acting? George may be acting on his own—that is, he alone chose to carry out the intrusion and acted accordingly. But in the most general case, George acts at the behest of another party—usually an organization, such as his employer, his gang, or his government. Attributing malicious cyber activity to a specific adversary as the ultimately responsible party answers the question “who is to blame?” rather than “who did it?” (the latter being the focus of attributing an intrusion to its perpetrator).
Considered in this light, it is clear that the party at whose behest George is acting cannot be determined by technical forensics alone. Indeed, in some cases, it is possible that technical forensics play only a minimal role in making this determination.
A non-cyber example is a good place to start. If a missile fired from an Elbonian navy ship caused damage to a U.S. Navy ship during peacetime in the Atlantic Ocean, the United States would hold Elbonia responsible. If Elbonia asserted that the ship’s captain was a rogue actor and not acting on orders from the Elbonian government, it would be up to the Elbonian government to demonstrate that this claim is true. For example, in no particular order, the Elbonian government could prosecute and punish the captain; allow the United States to interview the captain and members of the crew; pay reparations; formally apologize; show the United States the orders under which the captain was operating; or share the message traffic to and from the ship to Elbonian authorities before and after the incident or recordings made on the bridge of the Elbonian ship during the incident.
Some combination of these (and/or other) steps might suffice to persuade the United States that the missile firing was the act of a rogue captain and that the Elbonian government should not be held responsible for what would otherwise be an illegal use of force. But the reason that the Elbonian government would be required to demonstrate its lack of culpability in such an incident is the international convention that in general, states are responsible for the acts of their armed forces. Units of these forces are clearly marked with national insignias, partly for this reason. The rationale for this presumption is that historically, only states have had the wherewithal to build and use weapons capable of threatening national security.
But it is unclear how to apply present conventions for state responsibility to cyber incidents and the extent to which cyber-specific rules would be needed for such application. Is Greece the responsible party because George launched the attack from Greece? Is China the responsible party because George is a Chinese citizen? Is the Russian organized crime group the responsible party because of George’s involvement with the group? Is Russia the responsible party because of ties between the FSB and the organized crime group of which George is a part? In principle, a plausible case could be made for any of these possibilities, but in the absence of a broad political agreement or convention that argues for one over the other, the determination of “the responsible party” is necessarily based on policy and political judgments that take into account the relevant facts known from all sources.
The relationship among the three types of attribution
As noted above, the question of “who is responsible?” can be answered by pointing to a specific machine (or machines), a specific perpetrator acting to set the intrusion into motion, and a specific adversary as the ultimately responsible party. But the discussion above should make clear that the last kind of attribution is different from the first two in that the notion of a party that is “ultimately responsible” implicates legal, policy, or political issues to a much greater degree. The reason is that the first two involve behavior—who or what is acting in a way that causes harm—whereas the last involves ascertaining a party’s motivation and intent, a much more complicated task. Subsequent sections will build on this point.
There is not necessarily a direct connection between these different types of attribution. Knowing the machine responsible does not necessarily provide the identity of the perpetrator, and knowing the identity of the perpetrator does not necessarily reveal the party that is ultimately responsible.
Nevertheless, although these three types of attribution are conceptually distinct, they are often related in practice. Knowing the machine from which the intrusion initially emanated may provide some clues that can help uncover the identity of the perpetrator, and knowing the perpetrator may provide some clues that can help identify the specific adversary that should be held ultimately responsible.
For example, if the machine originating an intrusion is definitively located in Nation A, it suggests that the perpetrator has access to machines in Nation A. If Nation A is a country in which only a small segment of the population has easy access to computers, the search for the perpetrator’s identity may entail examining fewer possible suspects than if Nation A made it easy for large segments of the population to access computers. A common clue picked up by technical forensics is the language setting for the keyboard of a particular computer. Despite the fact that many people in the world are multilingual, such a clue is nevertheless suggestive and raises the likelihood that a human perpetrator is from a nation in which that language is used.
It may also be the case that responsibility cannot be allocated cleanly to a specific party. For example, in decentralized organizations, it is common for the leader to express his or her intent and then leave it to subordinates to execute in accordance with that intent. A subordinate operator may well do something that he believes is consistent with that intent but in fact may be “too much” from the perspective of the leader. In such a situation, responsibility is diffused among the individuals involved in an unclear manner.
A worked example of attribution
In 2013, Mandiant released a report called “APT1: Exposing One of China’s Cyber Espionage Units,” identifying a group it called “APT1” as a single organization of operators that conducted a cyber espionage campaign against a broad range of victims between 2006 and 2013. Mandiant concluded that APT1 was most likely sponsored by the Chinese government. Mandiant was also able to develop profiles (“personas”) on several individuals within APT1, though it was not able to determine with any certainty their real names or identities.
The attribution process in which Mandiant engaged touched on all three meanings of attribution: specific machines, specific human beings (perpetrators) pressing the keys, and an ultimately responsible party.
For example, the Mandiant report notes that:
[C]yber intruders leave behind various digital “fingerprints.” They may send spear-phishing emails [in this case, emails to specific individuals within the targeted company containing malicious links or files] from a specific IP address or email address. Their emails may contain certain patterns of subject lines. Their files have specific names, MD5 hashes, timestamps, custom functions, and encryption algorithms. Their backdoors may have command and control IP addresses or domain names embedded.
All of these indicators were used by Mandiant in their identification of the specific machines used by APT1 in their intrusions.
Mandiant used a variety of other information to associate these machines with Chinese actors. For example, they noted large volumes of intrusion traffic associated with blocks of IP addresses known to be assigned to Chinese ISP’s operating in Shanghai. APT1 hackers also used a Remote Desktop client from Microsoft to manage their remote access to targeted systems, and in the majority of such cases, the keyboard language setting was “simplified Chinese.”
Public domain registration information (e.g., who is the registered owner of the domain example.com) also helped to identify specific individuals; such information includes names, addresses, phone numbers, and email addresses. Of course, an intruder may provide false registration information when asked, but systematic errors (e.g., misspellings) can provide valuable clues as well.
To identify individuals, Mandiant searched the Web for various email addresses uncovered through domain registration and other sources. In many cases, these email addresses were also found on other sites providing additional information about the individual, and in many cases apparently supplied by the individual. Mandiant was confident in its identification of personas, but far less certain about the actual names associated with those personas.
As for an ultimately responsible party, Mandiant pointed to a specific unit of the People’s Liberation Army (PLA). Mandiant first identified a group of operators who perpetrated a large number of intrusions, resulting in the exfiltration of large volumes of information. It found that the industries targeted matched industries that China has identified as strategic to their growth. Mandiant then identified a unit of the PLA (Unit 61398) that was similar to this group in its mission, capabilities, and resources, and was also located in the same geographical area from which many APT1 activities appeared to have originated. Mandiant identified individuals with a connection to Unit 61398, which appears to be actively soliciting and training English-speaking personnel specializing in a wide variety of cyber topics, such as covert communications, operating system internals, digital signal processing, and network security. Unit 61398 also recruits new talent from the science and engineering departments of Chinese universities and associates various “profession codes” describing positions within Unit 61398 with competence in highly technical computer skills. Lastly, Mandiant found a memo describing a special fiber optic communication infrastructure provided by the state-owned enterprise China Telecom in the name of national defense.
In sum, Mandiant asserted high confidence that APT1 should be associated with Unit 61398 of the PLA. But it also acknowledged the possibility that “a secret, resourced organization full of mainland Chinese speakers with direct access to Shanghai-based telecommunications infrastructure [had] engaged in a multi-year, enterprise scale computer espionage campaign right outside of Unit 61398’s gates, performing tasks similar to Unit 61398’s known mission.”
Attribution for different types of intrusion
For simplicity of discussion, the discussion of this section uses a particular scenario involving an intrusion on a DOD computer in San Francisco to illustrate some aspects of the attribution process. That scenario is based on a multi-stage intrusion in which intermediate computers are used to mask the computer from which a remote-access intrusion originated, but of course other types of intrusion are possible, and indeed as common or even more common than the one depicted. In practice, the attribution process unfolds differently with different types of intrusion.
For example, an intrusion may result from the sending of an email to a user, who then clicks on a malicious link or attachment and inadvertently launches malware that takes destructive action in his computer. That email may be sent from a Gmail address, and it is well known that a Gmail address can be created from anywhere (e.g., a Wi-Fi-equipped coffee shop) with near total anonymity. In this case, there are no intermediate stepping stones that will lead back to an originating computer. Technical forensics may thus focus out of necessity more on characteristics of the malware being used.
Intrusions can occur when a user merely surfs the Web on ostensibly safe sites. Because many sites display advertisements, the content that a user sees on his or her screen is not entirely under the control of the operator of the Web site. Ad content can be poisoned so that when the image from the ad is displayed, malware is downloaded to the user’s computer. Investigation in this instance may require approaching the party who obtained ad display rights on that Web site.
Other types of intrusion may not involve the Internet at all. An adversary may be able to compromise the hardware supply chain, leading to the delivery to the intended victim of a clandestinely modified computer. Even if this computer is never connected to the Internet, the modification might cause the computer to destroy itself on a specific date. In this case, technical forensics would focus on the characteristics of the compromised hardware that was delivered to the user, which is not the focus in investigations involving malware. Another scenario involves inducing a user to insert into his computer a USB key that is contaminated with malware and runs upon insertion. In such cases, technical forensics directed at Internet activity may not reveal useful information, for example, if the malware destroyed files without accessing Internet services, but the manufacturer of the USB key may be able to provide insight. In such scenarios, technical forensics coupled with other investigations may yield useful information about the perpetrator immediately responsible for the intrusion.
Legal Authorities for Gathering Information Related to Attribution
Gathering information that might be used to make an attribution judgment does not take place in a vacuum. U.S. law and policy govern the information-gathering activities of U.S. law enforcement and intelligence agencies, and in particular recognize several key distinctions. These include information gathering undertaken domestically versus that undertaken on foreign soil; information gathering undertaken to investigate domestic criminal activity versus that undertaken for national security versus foreign intelligence purposes; and information gathering involving U.S. citizens versus foreigners.
In today’s security environment, the activities of adversaries often blur these lines. The 11 September 2001 terrorist attacks on the World Trade Center and the Pentagon were clearly matters of national security, but they were criminal acts as well. Terrorists may seek to fund their operations by engaging in criminal activity such as human or drug trafficking. Foreign terrorists may operate from U.S. territory, thereby gaining some of the default protections afforded to U.S. citizens on U.S. soil. And U.S. citizens may undertake criminal activity on behalf of foreign governments or terrorist movements. Operating in cyberspace further complicates these distinctions, as communications traffic (and intrusions) freely transit national borders while jurisdiction and legal authorities to gather information do not.
Describing existing law relevant to gathering information useful for attribution, John Carlin, assistant attorney general for national security in the Obama administration, writes in a recent article that “online” investigations are in fact conducted mostly offline and thus use investigative tools such as physical examination of servers, conversations with network users, and requests or compelled production of copies of records from service providers. (He also notes, somewhat cryptically and no doubt constrained by classification, “the important (and sensitive) tools that the IC [Intelligence Community], beyond just the FBI, brings to the effort to attribute...cyber activity.”) Carlin points to several legal instruments governing the domestic use of these tools by the law enforcement community, including:
- The Stored Communications Act (SCA), which sets out the procedures for law enforcement agencies to obtain voluntary or compelled disclosure of stored communications from domestic communications service providers (e.g., whether a search warrant or a subpoena is necessary in a given instance).
- The Foreign Intelligence Surveillance Act (FISA) of 1978, which allows electronic surveillance conducted in the United States for national security or foreign intelligence investigations. In such instances, the target of surveillance must be a foreign power or an agent of a foreign power; the facilities or places at which the electronic surveillance is directed must be used, or must be about to be used, by a foreign power or an agent of a foreign power; and a significant purpose of the surveillance must be to obtain foreign intelligence information.
- Search warrants (or in the case of national security and foreign intelligence investigations, FISA orders) for the search and seizure of physical devices (e.g., phones, computers, or servers).
Outside the United States, activities of the intelligence community (as opposed to the law enforcement community) are governed by Executive Order 12333, which is intended to “provide for the effective conduct of United States intelligence activities and the protection of constitutional rights.” Because constitutional rights do not attach at all to foreigners unless they are within the United States, intelligence collection activities directed against foreigners are largely unconstrained by U.S. law and policy except to the extent that U.S. persons may be involved. (When U.S. persons are involved, the Executive Order and other law, notably the Foreign Intelligence Surveillance Act, does place some constraints on U.S. intelligence agencies.) International law has traditionally placed no constraints on intelligence collection activities (i.e., espionage), though such activities against foreigners abroad may violate the domestic law of other nations.
U.S. law enforcement agencies also operate outside the United States in cooperation with their counterparts abroad by “exchanging information, investigating attacks or crimes, preventing or stopping harmful conduct, providing evidence, and even arranging for the rendition of individuals from a foreign state to the United States.” Sometimes such cooperation is governed by treaty (e.g., extradition treaties or Mutual Legal Assistance Treaties (MLATs) that generally apply to a list of agreed crimes). MLATs also require “state parties to assist one another by providing information, evidence, and other forms of cooperation when requested to do so in such situations.”
The Budapest Convention, also known as the Council of Europe Convention on Cybercrime, is an international agreement that seeks to harmonize national laws explicating offenses that constitute cyber crimes, to improve national capabilities for investigating such crimes, and to increase international cooperation among the signatories on investigations. The convention’s provisions on cooperation are a rough substitute for pairs of signatory nations that do not have a MLAT in place, but existing MLATs between other pairs of nations supersede the convention’s provisions. Increased international cooperation on investigations may well increase the amount and quality of useful information available for attribution judgments.
Nation-States as the Ultimately Responsible Party?
As noted earlier, the consensus that exists for the presumed responsibility of states for the acts of their armed forces does not necessarily apply to when a state is associated in some way with malicious cyber activity.
Identifying a particular nation-state as the party ultimately responsible for a cyber intrusion hinges on what it means to be “responsible.” A variety of different forms of state responsibility can be imagined. The following hierarchy of national involvement as it corresponds to responsibility is largely based on Jason Healey’s taxonomy in “Beyond Attribution: Seeking National Responsibility for Cyber Attacks.”
- A state could be incapable of detecting hacking activities within its borders.
- A state could prohibit hacking activities (defined here as conducting cyber intrusions of various kinds), but"This appeared in The Cyber Issue in Winter 2016." have no ability to enforce this prohibition against third-party actors.
- A state could tolerate hacking activities. States could decide not to outlaw these actions, or not to prosecute those who launch attacks.
- A state could encourage hacking activities. In this scenario, a state may provide under-the-table support (intelligence, operational guidance, or “suggestions”), or simply promote a culture whereby these actions are lauded.
- A state could direct hacking activities. For example, a state could ask organizations within its jurisdictional reach or contract with non-state organizations to conduct specific hacking activities.
- A state could conduct hacking activities. A state uses its military or intelligence assets to conduct offensive cyber operations, perhaps integrated with third-party hackers.
A refinement on the above list is that these different types of responsibility might vary by the specific kind of hacking activity involved. For example, a state might conduct cyber-enabled espionage but prohibit destructive cyber attacks.
A second related dimension along which to characterize state responsibility is the actor conducting any of the hacking activities described above. Responsibility could in principle also attach to hacking activities initiated by parties within the state’s geographic borders and/or by parties who owe some form of allegiance or loyalty to the state (e.g., citizens of that state).
With respect to the case of responsibility attaching to activities initiated by parties within the state’s geographic borders, a body of international law related to terrorism may be relevant. Prior to the 11 September 2001 attacks on the United States, a nation-state was responsible for the acts of private groups inside its territory over which it exercised “effective control.” In the aftermath of those attacks, the United States took the position that the mere harboring of these actors, even in the absence of control over them, suffices to make the state where the terrorists are located responsible for their actions, and many parts of the international community, including the UN Security Council, concurred with this position. How and to what extent, if any, such law applies to subnational or transnational groups perpetrating acts of cyber intrusion is uncertain, but the law as it relates to its original context of terrorism is at least suggestive.
In addition, the International Law Commission (ILC) submitted in 2001 to the UN General Assembly its report on “Draft Articles on Responsibility of States for Internationally Wrongful Acts.” International adoption of these articles has not yet progressed beyond the submission of these draft articles, although the International Court of Justice (ICJ) has cited them at least once since their submission. In general, these draft articles lay out what the ILC believes to be “the general conditions under international law for the State to be considered responsible for wrongful actions or omissions.” For an action or omission to be considered wrongful, it must at least constitute a breach of an international obligation. How and to what extent, if any, malicious cyber activities conducted across international borders constitute such breaches is contested and open to question.
At the same time, elements of the draft articles do speak to some part of the Healey taxonomy. For example, the draft articles embody a general rule “that the only conduct attributed to the State at the international level is that of its organs of government, or of others who have acted under the direction, instigation or control of those organs, i.e. as agents of the State.” On the other hand, the articles also indicate that “a State may be responsible for the effects of the conduct of private parties, if it failed to take necessary measures to prevent those effects.”
To the best of this author’s knowledge, there is no body of international law that holds a nation accountable for the actions of its citizens per se. On the other hand, various nations can and do assert jurisdiction over their own citizens in many instances even when these citizens are abroad; in such cases, a citizen of Nation A is subject to the domestic law of Nation A even if he or she is located in Nation B. Moreover, various Nation Bs have from time to time sought, using diplomatic and other means, to influence or persuade Nation A to exert more control or influence over A’s citizens when A’s citizens are responsible for harm to B.
This paper does not seek to resolve the “proper” definition for state responsibility, but three observations are pertinent.
- Technology has very little to say about the proper definition for state responsibility. No amount of technical forensic information will point to the proper definition.
- For all practical purposes, the definition that a nation-state will adopt in any given instance will almost certainly depend on the facts and circumstances of that instance. It may be that over time, an international consensus or norm may develop for the level in the Healey hierarchy that corresponds to the minimum level of involvement needed to declare that a state is “responsible.” But we are not there yet.
- Multiple parties could be responsible depending on how norms for assigning responsibility evolve. For example, if citizenship and the geographic location from which an intrusion was initiated both become important norms in determining a responsible state party, then perhaps China and Greece would both bear some responsibility for the intrusion in our hypothetical example.
Subnational Entities as the Ultimately Responsible Party?
As a general rule, nations are the subject of international law. However, from time to time, the UN Security Council has identified particular subnational entities engaged in international terrorism as threats to the maintenance of international peace and security. For example, UN Security Council resolution 1267 called out Osama bin Laden and others associated with him as terrorists that were being protected by the Taliban, and called upon member nations to deny permission for Taliban-operated aircraft to take off from or land in their territory and to freeze Taliban funds and other financial resources.
Such actions suggest that under international law, subnational entities could at some point be recognized as the ultimately responsible party for serious cyber intrusions in the way that certain subnational entities are held responsible for terrorism. But there is no history that is directly on point regarding this matter.
Arguments have also been made that individuals could even be responsible under international law for cyber “war crimes.” For example, Fidler has argued that the videos showing the killing of human beings by the Islamic State are themselves violations of international humanitarian law (IHL) and constitute war crimes. Under the Rome Statute (which establishes the International Criminal Court and gives it jurisdiction over individuals charged with war crimes), Fidler argues that “those making and posting the Islamic State’s videos are criminally accountable” under IHL.
How Attribution Judgments Are Made
In a 2014 paper on attribution, Rid and Buchanan argue that thinking about attribution is currently based on three assumptions, two of which are relevant to the discussion of this section—first, that attribution is a largely intractable problem because of the technical characteristics and geography of the Internet (Box 2) and second, that attribution is either possible or not possible in any given case of interest. The third assumption—that the main challenge in attribution is finding the evidence itself and not in interpreting or using it—is relevant in a subsequent section.
In short, the conventional wisdom holds that one cannot attribute a malicious cyber activity to its perpetrator with high confidence. As the saying goes, “electrons don’t wear uniforms”—there’s no inherent binding of any given IT activity to specific actors. Anyone could be at the computer in Greece that launched the attack against the DOD computer in San Francisco, evidence could have been planted to mislead investigators, and the perpetrator could even have been a computer program, set by someone to run autonomously.
The conventional wisdom has a grain of truth to it—technical forensics alone cannot lead to high-confidence attribution. Caloyannides goes so far as to assert that “forensics’ presumed usefulness against anyone with computer savvy is minimal because such persons can readily defeat forensics techniques. Because computer forensics can’t show who put the data where forensics found it, it can be evidence of nothing.”
At the same time, that grain of truth does not come close to the full story of how attribution judgments can be and are made. One important point to consider is that while an intruder may have many counter-forensics measures at his or her disposal, he or she may not take all of the necessary measures; we return to this point below. Most importantly, only when the goal is attribution—to a machine—are technical forensics the primary source of evidence.
In trying to attribute an intrusion to a specific perpetrator or ultimately responsible adversary, technical forensics alone are generally inconclusive, and the information they provide must often be combined with other sources to be genuinely useful.
For example, a given intrusion may be similar or even identical to a previous intrusion—the same code could be executed, the same IP addresses used, the same technical signatures found. Such similarity would suggest that the same party could be behind the intrusion at hand. If that party had been previously identified, that identification might be carried over to the present case—or perhaps allies or associates of that other party might be implicated. Is such similarity conclusive or dispositive? Absolutely not. But neither should the clue it provides be thrown away.
Behavioral information can also contribute to attribution judgments. For example, Carlin notes that useful clues may be found in the kinds of malware that intruders use and in the way they communicate with their victims. Behavioral patterns have been used in criminal investigations for a wide variety of offenses, and many of the analytical techniques employed to understand these patterns have proven useful in attribution.
In the case of the 2014 Sony hack, the perpetrators left a “splash screen” on infected Sony computers with the name “Guardians of Peace” and various logos. Carlin points out that the perpetrators behaved in ways that were similar to the behavior of criminals like serial killers who “stage” the crime scene, arranging it to send a message or conceal involvement. Such stagings go beyond what is necessary to commit the crime, and they thus provide extra information that can be helpful in attribution.
An intruder can also make errors of tradecraft. For example, text strings can sometimes be extracted from the binaries used in an intrusion. When an investigator examines the binary used in the intrusion on the DOD computer in San Francisco, she finds the text string Linsong9862. An Internet search reveals that this string is also the user name associated with a dating profile of a Chinese computer scientist who says he lives in Greece. Another indicator may be the time of day that certain malicious cyber incidents occur—a time, possibly, that correlates with working hours in Greece. In neither case is such evidence conclusive, but that evidence constitutes additional data points that may point to the human intruder.
Sometimes intruders make mistakes of operational security. For example, an intruder may discuss his or her plans on insecure channels that are monitored. A hacker may look to others for advice, or seek recognition for his or her bravado and skill in perpetrating a successful intrusion, or upload or download files to or from known, previously used locations. Because intelligence agencies collect information from a variety of different sources in different parts of the world, sometimes such information is available; if so, such information could prove useful in identifying the human intruder.
The style and methodology of an intrusion may also be helpful to attribution. For example, a cyber attack aimed at destroying or disrupting cyber physical systems that are part of a nation’s physical infrastructure is likely to require significantly more expertise than one directed at deleting files on computer systems; while both require expertise in penetration techniques, only the former requires expertise regarding the specific cyber physical systems involved. One reason the Stuxnet attack was attributed to state actors was the sophistication of the attack in precisely targeting particular configurations of Siemens controllers (and leaving others alone), in concealing from centrifuge operators what was happening to the targeted centrifuges, and in the profligate use of zero-day vulnerabilities, which are usually regarded as a resource to be conserved and used sparingly.
Other intelligence and information-gathering activities may also provide information useful for attribution. According to the CIA, human intelligence (HUMINT)—information that can be gathered from human sources—is collected through “clandestine acquisition of photography, documents, and other material, overt collection by people overseas, debriefing of foreign nationals and U.S. citizens who travel abroad, and official contacts with foreign governments.” For example, a spy in the office of a senior political leader in another nation could provide information that the intrusion was ordered by that nation’s leadership—such information could well be conclusive when coupled with technical forensics. Information about adversary plans and capabilities for cyber operations may be found in a dumpster and used later to investigate an intrusion.
HUMINT is not necessarily clandestine. As suggested earlier, informal conversations or formal interviews with operators, service providers, and other users can also generate useful information. Debriefing a U.S. citizen who had conversations with foreign network operators on a recent trip abroad can provide useful tips. Interviews with victims of cyber intrusions can provide valuable context for an intrusion, as investigators might learn more about why the intruders wanted to do what they did when they did it. For example, investigators might learn of demands that the intruder made of the victim in connection with the intrusion. Sharing information about similar intrusions might be useful as well; one victim might have one part of the information necessary to attribute an intrusion and a second victim might have another part.
Pre-positioned implants for cyber-enabled intelligence collection may provide useful information regarding the connection between the intrusion and agencies of a nation’s government—for example, these implants may reveal communications regarding an intrusion between decisionmakers in that government’s military department. Such implants were mentioned above.
Geopolitical circumstances could provide clues as to who would want to launch a particular intrusion. Who would benefit most from gaining access to the DOD computer in San Francisco? Are there particular tensions between a company and a state, or between the United States and another international actor? Is another international actor making demands of the United States that are serious enough to warrant the use of force or cyber force? This information could provide a helpful lens for determining who would be most motivated to launch a certain attack.
Finally, historical relationships help to frame the attribution process. It is less likely that a non-adversarial nation would conduct, support, or tolerate malicious cyber activity against the United States as compared to an adversarial nation.
None of these methods or sources of evidence alone can be used to determine the responsible party. However, together, these pieces of data could pull together into a compelling analysis. A useful analogy is that of big data analytics, in which no individual datum is by itself significant, but instead large volumes of data are analyzed to draw conclusions.
In short, attribution is an all-source issue—no one method or source of information can be used to point fingers, but multiple sources taken as a whole may paint a convincing picture. Box 3 illustrates how the all-source intelligence process can be applied to attributing putatively anonymous non-cyber incidents.
The fact that attribution judgments draw on many different sources of information has one major temporal implication: early judgments made with less information are generally less believable than later judgments made with more information. That is, more investigation may reveal additional useful information, which may (or may not) reinforce attribution judgments made earlier.
One important reason for the improvement in capabilities for attribution over the past several years is that as the importance of cybersecurity has grown, more people are paying attention. Given the likelihood of malicious cyber activity in the future, they are more willing to make investments in intelligence and to build investigative capacity that will pay off in the future. Put differently, capabilities for attribution are partly a function of the investment a nation (or indeed third parties, such as private cybersecurity companies) is willing to make in those capabilities, both in infrastructure and in the effort that any given case demands.
Lastly, it is important to understand that the all-source intelligence process described in this section has a different focus than the discussion of the ultimate responsibility of states and non-state actors in previous sections. The all-source intelligence process seeks to approximate the God’s-eye understanding of an intrusion, whereas the discussions in earlier sections are legal and policy discussions. In short, understanding who did what (the focus of the intelligence process) is different from, though relevant to, understanding who is to blame.
Government Views on Attribution
U.S. government views of attribution have evolved over the past half-dozen years.
In 2010, then-deputy secretary William Lynn emphasized the difficulties of attribution in cyberspace. He said that “whereas a missile comes with a return address, a computer virus generally does not. The forensic work necessary to identify an attacker may take months, if identification is possible at all.”
In 2012, then-secretary of defense Leon Panetta said that the DOD “has made significant advances in solving a problem that makes deterring cyber adversaries more complex: the difficulty of identifying the origins of an attack. Over the last two years, DOD has made significant investments in forensics to address this problem of attribution and we're seeing the returns on that investment. Potential aggressors should be aware that the United States has the capacity to locate them and to hold them accountable for their actions that may try to harm America.”
In 2015, the DOD Cyber Strategy stated that attribution is a “fundamental part of an effective cyber deterrence strategy as anonymity enables malicious cyber activity by state and non-state groups. On matters of intelligence, attribution, and warning, DOD and the intelligence community have invested significantly in all source collection, analysis, and dissemination capabilities, all of which reduce the anonymity of state and non-state actor activity in cyberspace. Intelligence and attribution capabilities help to unmask an actor’s cyber persona, identify the attack’s point of origin, and determine tactics, techniques, and procedures. Attribution enables the Defense Department or other agencies to conduct response and denial operations against an incoming cyberattack.” The 2015 articulation is thus more measured and moderate in tone than Panetta’s comments of 2012.
Also in 2015, Director of National Intelligence (DNI) James Clapper testified that although “cyber operators can infiltrate or disrupt targeted ICT [information and communications technology] networks, most can no longer assume that their activities will remain undetected. Nor can they assume that if detected, they will be able to conceal their identities. Governmental and private sector security professionals have made significant advances in detecting and attributing cyber intrusions.” He testified in 2016 that information security professionals “will continue to make progress in attributing cyber operations and tying events to previously identified infrastructure or tools that might enable rapid attribution in some cases. However, improvements in offensive tradecraft, the use of proxies, and the creation of cover organizations will hinder timely, high-confidence attribution of responsibility for state-sponsored cyber operations.”
One significant development in the attribution landscape over the past several years is the increasing involvement by private sector firms in rendering attribution judgments. Regarding the value of private-sector attribution, the DOD Cyber Strategy of 2015 notes that private-sector parties (e.g., security firms) reporting on attribution “can play a significant role in dissuading cyber actors from conducting attacks in the first place” and states that the “Defense Department will continue to collaborate closely with the private sector and other agencies of the U.S. government to strengthen attribution. This work will be especially important for deterrence as activist groups, criminal organizations, and other actors acquire advanced cyber capabilities over time.”
In addition to the Mandiant APT1 report described above, some other examples of private-sector involvement in attribution include:
- FireEye’s report, “APT28 - A Window Into Russia's Cyber Espionage Operations,” indicating Russian involvement in a variety of espionage activities against private-sector and government actors.
- Novetta’s report, “Operation SNM: Axiom Threat Actor Group Report,” indicating Chinese government involvement in cyber espionage against a variety of private companies, governments, journalists, and pro-democracy groups.
- CrowdStrike’s report, “CrowdStrike Intelligence Report: Putter Panda,” identifying Unit 61486 in the Chinese PLA as responsible for the cyber-enabled theft of corporate trade secrets primarily relating to the satellite, aerospace, and communication industries.
Private-sector involvement in attribution has advantages and disadvantages. Among the advantages are:
- The unclassified nature of such reports. Because such reports are unclassified in their entirety, they can be used by government officials in responding to questions about the attribution of any given cyber incident. They also make available to independent analysts substantial information that would not otherwise be available and thus contribute to a more informed public debate about such matters.
- The potential increase in analytical and collection resources that can be brought to bear on tracing the origin of hostile cyber operations. Additional resources will be necessary as the volume of hostile cyber operations conducted by parties with advanced cyber capabilities increases.
- Continuing concealment of sensitive sources and methods of government intelligence, which are not revealed in private-sector attribution reports.
- High level of technical access to information of victimized organizations. Private sector firms producing attribution reports do so at the invitation of victimized organizations, and are thus privy to technical information that is controlled by those organizations. That is, they have technical visibility—perhaps even in real time under some circumstances—into the information technology infrastructure of the victimized organization. Such high visibility is often unavailable to government investigators.
- The attenuation of government responsibility for an attribution judgment. When the actual judgment is associated with a private party, government officials can distance themselves from it, even if they point unofficially to that analysis when questioned about a given incident. The resulting ambiguity may have diplomatic benefits.
Some of the disadvantages include the following:
- The marketing aspect of private-sector attribution reports. Such reports often gain considerable media attention, especially if government officials have not been particularly forthcoming about cyber incidents. These reports are thus valuable marketing tools that elevate the authoring firms in the public eye, and the incentives motivating these firms to produce such reports quickly and ahead of their competitors may degrade the quality of their research and analysis.
- Lack of independent quality control and independent oversight. Authoritative government reports are usually subject to an interagency process that challenges evidence and conclusions. The private-sector security market is robust enough to provide some independent scrutiny, and since each firm has its professional reputation to uphold, they all have incentives to produce high-quality work. Whether market forces are sufficient to uphold quality in such reports remains to be seen.
- The possible lack of true independence of the private-sector report. Given the semi-permeable membrane between private-sector security firms and government authorities, it would not be surprising if from time to time, government officials talking to their colleagues in the private sector suggest that looking for X rather than Y in their investigative efforts could prove more fruitful. That is, such reports may be produced with some measure of government input, even if such input is not apparent.
- Finally, nations other than the United States often do not appreciate fully the separation between public and private sector that operates in the United States. In particular, more authoritarian regimes that exert a high degree of control and influence over civil society may well regard private-sector entities as being willing to speak or act in accordance with U.S. government wishes under many or most circumstances.
Some other nations may be following the same path that the United States followed regarding attribution, but with some degree of lag time. That is, nations implicated by the United States in various malicious cyber activities say that it is impossible to attribute such activities to any particular actor and that there is no definitive evidence suggesting their involvement; hence, they say, all accusations amount to nothing more than speculation.
It is obvious that a nation engaged in such activities (call it Nation A) but also wishing to deny its involvement has a strong motive for making such a claim. But apart from showmanship and politics, is it plausible that Nation A’s officials “really” believe the claim?
This author believes the answer may be yes, at least to some extent, and the following narrative—based on his own discussions with cyber experts from a major cyber power—may capture some of the logic behind the claim. The United States publicly blames (or hints at blaming) Nation A for cyber intrusions against the United States. A’s leaders ask their technical cyber experts for a response. Because these experts are technical experts, they provide the right technical answer—that the evidence available from a single incident, viewed as a single, isolated incident, is never definitive regarding the perpetrator. And that is the answer that is sent back up the chain of command to the leadership. That leadership, already politically and strategically motivated to deny the possibility of attribution, has no particular interest in breaking down stovepipes, an action that might lead them to a different conclusion. And why should it be expected that the bureaucratic stovepipes of other nations are any less rigid and impenetrable than those of the United States, which has taken many years to evolve past this position to see attribution as an all-source issue? Confident in the judgment of their experts, A’s leaders repeat the technical answer to the world, and, in fact, that is what we have heard in the multiple denials from various actors that “there is no conclusive evidence” pointing to A’s complicity or involvement.
This narrative is obviously not the whole story, or even most of it. But to the extent that it is valued, it helps to provide a colorable patina for Nation A’s public statements.
How Attribution Relates to Policy
The discussion up to this point has presumed that the attribution task is to determine as best as possible the machine, human intruder, and/or ultimately responsible parties that are behind a given malicious cyber incident. In this context, the word “determine” is relative to a Gods-eye perspective—to determine the machine, intruder, and/or party actually involved in and responsible for the undertaking the intrusion. As noted earlier, attribution to a machine or a perpetrator turns on factual issues, whereas attribution to an ultimately responsible party strongly depends on the legal, policy, and political definition of “ultimately responsible.”
Determining factual reality—important as it is—is only the beginning of the attribution process from a policy perspective. Three key points need to be made.
- A “determination” is rarely definitive. God may know who “really” did it, but our determinations of who did it will be associated with some degree of uncertainty—and it is very hard to be 100 percent confident about a determination. The use of the word “judgment” underscores this point.
- The necessary degree of confidence in an attribution judgment depends on the nature of the malicious activity being attributed and the action that is contemplated in its aftermath.
- The audience that an attribution judgment seeks to persuade has a significant impact on how subsequent aspects of the attribution process unfold.
These points are fundamentally policy points rather than technical ones and are at the heart of the political challenges of attribution.
Confidence in Attribution
An attribution judgment is a statement with an inherent degree of uncertainty. Different professions use different sets of words to convey such uncertainty. For example, in the U.S. legal community, the following words are used regarding the persuasiveness of evidence that a given person is in fact responsible for an event.
- Reasonable suspicion: There is sufficient evidence for reasonable suspicion that John Doe robbed the bank.
- Probable cause: The police officer had sufficient evidence for probable cause to believe that John Doe robbed the bank.
- Substantial evidence: There is substantial evidence that John Doe robbed the bank.
- Preponderance of the evidence: The preponderance of the evidence indicates that John Doe robbed the bank.
- Clear and convincing evidence: There is clear and convincing evidence that John Doe robbed the bank.
- Beyond reasonable doubt: The evidence indicates beyond a reasonable doubt that John Doe robbed the bank.
The audience in question is an impartial and unbiased judge or jury, and advocates for each side try to persuade this audience to draw some conclusion about the responsibility of the alleged perpetrator of some event that happened in the past. The relevant standard of evidence that the judge or jury applies depends on the nature of the case. If the event in question is a criminal matter, the judge or jury must be convinced beyond a reasonable doubt about the party responsible, whereas in a civil matter the judge or jury need only be convinced by a preponderance of the evidence.
The legal process of ascertaining responsibility is also intended to be fair. In the legal process of ascertaining responsibility, due process requirements seek to ensure that state action occurs only in accordance with law and that justice is administered fairly (i.e., that prejudicial or unequal treatment does not occur). Due process also protects the rights of an accused party (e.g., by excluding improperly gathered evidence from a trial).
In short, if a malicious cyber incident is regarded as a matter for domestic law enforcement authorities to address, then legal requirements for due process, standards of evidence, and degrees of certainty about attribution obtain. Outside this context, there is much less clarity.
Consider, for example, the attribution issue from the standpoint of international law. International law operates in an environment of sovereign nations. Nations sometimes have interests in using international bodies such as the ICJ or the United Nations to adjudicate their political and diplomatic positions with respect to other nations, and thus they grant these bodies jurisdiction in certain contexts. But few if any of these nations are willing to subordinate important national interests to the judgments of such bodies. Moreover, unlike domestic courts that are backed by police forces, these bodies generally lack the enforcement authorities associated with the use of force. (It is true in principle that the UN Security Council may authorize the use of force to enforce a judgment, but it is exceedingly rare in practice, since any one of the permanent five can veto a resolution containing such authorization.)
An important legal lacuna in the ability of an international tribunal to make attribution judgments is underscored by Tsagourias, who argues that the nations involved may for security reasons be unwilling to make relevant information available or only willing to make it available in truncated or abstracted form. For example, in the 1986 Nicaragua case, the ICJ noted:
One of the Court's chief difficulties in the present case has been the determination of the facts relevant to the dispute. First of all, there is marked disagreement between the Parties not only on the interpretation of the facts, but even on the existence or nature of at least some of them.... Thirdly, there is the secrecy in which some of the conduct attributed to one or other of the Parties has been carried on. This makes it more difficult for the Court not only to decide on the imputability of the facts, but also to establish what are the facts (emphasis added). Sometimes there is no question...that an act was done, but there are conflicting reports, or a lack of evidence, as to who did it. The problem is then...the prior process of tracing material proof of the identity of the perpetrator.
Tsagourias also argues that international law “does not lay down any specific standards of evidence with regard to issues involving the use of force or self-defense.” He suggests (but does not defend) a generic threshold that “claims against a State involving charges of exceptional gravity must be proved by evidence that is fully conclusive. The same standard applies to the proof of attribution for such acts,” but notes that this standard is less strict than the standard of “beyond a reasonable doubt” and is higher than that of “balance of evidence.”
Tsagourias concludes that “standards concerning the availability and probity of evidence in cases involving armed attacks, uses of force or interventions are rather lax.” Nevertheless, he argues, “even if the standard of proof is not the same as the one required for the criminal prosecution of individuals and even if ‘a more political approach to attribution...might accept less exacting standards,’ it should be stressed that a State should not resort to self-defense on the basis of casual evidence or wild political inferences.”
No national policymaker would agree that any action of theirs, let alone actions related to self-defense, is, can, or should ever be justified “on the basis of casual evidence or wild political inferences.” Nevertheless, if the malicious cyber incident in question is regarded as a national security matter, determining the necessary degree of certainty is more complex. When national security is at stake, policymakers may have to make decisions that have a wide range of potentially significant consequences. But unlike the unbiased judge or jury that is the linchpin of decisionmaking in the legal community, national security policymakers are highly biased in the sense that they are predisposed to making decisions that they believe best protect and advance national interests. Nor does national security decisionmaking recognize good analogs to “rights of the accused” or “due process.” To take one obvious example, information is not excluded from consideration if it has been gathered “improperly.”
The intelligence community provides information, often in the form of assessments, to support national security decisionmaking. For example, the national intelligence estimate for Iran’s nuclear intentions and capabilities states that:
We judge with high confidence that in fall 2003, Tehran halted its nuclear weapons program; we also assess with moderate-to-high confidence that Tehran at a minimum is keeping open the option to develop nuclear weapons.... We assess with moderate confidence Tehran had not restarted its nuclear weapons program as of mid-2007, but we do not know whether it currently intends to develop nuclear weapons.... We continue to assess with low confidence that Iran probably has imported at least some weapons-usable fissile material, but still judge with moderate-to-high confidence it has not obtained enough for a nuclear weapon. We cannot rule out that Iran has acquired from abroad—or will acquire in the future—a nuclear weapon or enough fissile material for a weapon.
The words in bold above are words of estimative probability intended to convey the degree of uncertainty (or, conversely, the degree of confidence) in various assessments and judgments made by analysts. Assessment guidelines call for ascribing high, moderate, or low levels of confidence to assessment as follows:
- “High confidence generally indicates that our judgments are based on high-quality information, and/or that the nature of the issue makes it possible to render a solid judgment. A ‘high confidence’ judgment is not a fact or a certainty, however, and such judgments still carry a risk of being wrong.
- Moderate confidence generally means that the information is credibly sourced and plausible but not of sufficient quality or corroborated sufficiently to warrant a higher level of confidence.
- Low confidence generally means that the information’s credibility and/or plausibility is questionable, or that the information is too fragmented or poorly corroborated to make solid analytic inferences, or that we have significant concerns or problems with the sources.”
This background on how the intelligence community operates is important because it frames how the policymaker approaches attribution judgments in a national security context. Given that national security decisions are a matter of sovereignty (i.e., there is no world government body that serves the role of impartial judge or jury and there are no due process requirements on national decisionmaking imposed by international law), the standard that governs national security decisionmaking is not controlled by legal terms such as “beyond a reasonable doubt” or “preponderance of the evidence” but is rather one of reasonableness—taking everything that is known into account, is the decision a reasonable one?
Policymakers are also quite often in the position of having to take a responsive action, even when only low or moderate confidence assessments are available. And a further complicating factor is that the degree of confidence required to take any given action depends on the nature of the action—and the putative actor—involved. This point is discussed further below.
The Persuasiveness of Attribution Judgments
Based on intelligence information and shaped by their own biases and judgments about what is best for the national interest, policymakers need to satisfy themselves about attribution. But it is a different—and often more difficult—task to persuade others who may be skeptical about official U.S. positions.
One major reason for this difficulty is that much of the public believes that legal standards of evidence are applicable to national security decisionmaking. These individuals thus conclude that because publicly offered evidence (which in practice cannot include all sources of information) would not “stand up in a court of law,” the U.S. government does not have a legitimate basis for action. For example, in the wake of the Sony hack in December 2014, public critics of the U.S. government, which had attributed the hack to North Korea, asserted that the evidence presented in favor of the attribution to North Korea was weak and that the available evidence pointed instead to a disgruntled insider at Sony. In a telling commentary, one security expert said that:
[C]alling out a foreign nation over a cybercrime of this magnitude should never have been undertaken on such weak evidence. The evidence used to attribute a nation state in such a case should be solid enough that it would be both admissible and effective in a court of law. As it stands, I do not believe we are anywhere close to meeting that standard.
This stance is somewhat ironic, given that even international courts have ruled that the standards for evidence in disputes between nations may not be as stringent as the standards applied to disputes aired in domestic courts. For example, in the 1949 Corfu Channel case, even the ICJ recognized the difficulties of providing evidence if that evidence had to be obtained from territory under the control of another state that was unwilling to cooperate. The court wrote:
By reason of this exclusive control, the other State, the victim of a breach of international law, is often unable to furnish direct proof of facts giving rise to responsibility. Such a State should be allowed a more liberal recourse to inferences of fact and circumstantial evidence. This indirect evidence...must be regarded as of special weight when it is based on a series of facts linked together and leading logically to a single conclusion.
Other nations are also a potential audience for an attribution judgment. In the wake of a malicious cyber incident, a state may want to persuade allies and unaligned nations that it has been wronged. To do so, the victimized state may not follow legally prescribed procedures, but instead may use tools of diplomacy and persuasion to convince necessary actors that a particular event occurred. Individual states may require different levels of evidence before siding with the supposed victim state.
In this context, it is worth recalling that during the Cuban missile crisis in 1963, president Kennedy asked former secretary of state Dean Acheson to seek French support for the U.S. position. The secretary of state traveled to Paris and offered to show French president Charles de Gaulle the CIA's surveillance photos of the Cuban missiles. According to Theodore Sorenson, then counselor to president Kennedy, President de Gaulle declined to view the photographs, saying the “word of the president of the United States is good enough for me.” Today, in the wake of the Snowden disclosures and the U.S. government’s history of public failure regarding claims of yellow rain in Southeast Asia and weapons of mass destruction in Iraq, a similar scenario of trust, either between the U.S. government and other nations—even friendly nations—or even between the U.S. government and its citizens, seems unlikely in the future. Yet, diplomatic dealings often necessitate a different interpretation of trust and evidence.
Against this backdrop, it is fair to say that regardless of whether the public mistakenly applies domestic legal standards to the national security decisionmaking process, skepticism about attribution judgments increases pressures on policymakers to make public more evidence for attribution judgments than they might otherwise prefer. Jack Goldsmith said it well on Lawfare:
[E]ven if the attribution problem is solved in the basement of Ft. Meade and in other dark places in the government, that does not mean the attribution problem is solved as far as public justification – and defense of legality – is concerned.
Policymakers are not legally constrained in their freedom of action by such considerations, but politically they may very well be—and in the long run, they will almost certainly have to reveal some amount of hitherto secret information relating sources and methods for gathering evidence used in attribution judgments. Goldsmith notes further that we will almost certainly see in the future an increase “in the demand for publicly verifiable attribution before countermeasures (or other responses) are deemed legitimate. In this small but significant sense, the United States has lost a battle in the early days of cyber conflict.” Similarly, Paul Rosenzweig argued that in the “post-Watergate post-Snowden world, the USG can no longer simply say ‘trust us.’ Not with the U.S. public and not with other countries. Though the skepticism may not be warranted, it is real.”
In this context, it is not without irony that private-sector entities such as Google and Facebook are also sensitive to the need to protect sources and methods of information used to attribute compromises of user accounts to nation-states. These entities warn users if they believe a nation-state compromise has occurred, but also do not provide the evidence underlying such a judgment. For example, Google tells compromised users that you “might ask how we know this activity is state-sponsored [but] we can’t go into the details without giving away information that would be helpful to these bad actors.” Facebook tells compromised users that to “protect the integrity of our methods and processes, we often won’t be able to explain how we attribute certain attacks to suspected attackers. That said, we plan to use this warning only in situations where the evidence strongly supports our conclusion.”
Lastly, it is highly unlikely that any amount of evidence made public would persuade a nation to publicly acknowledge its own responsibility for an untoward event, cyber or otherwise, if such an acknowledgement would not be in its interest. Demands for such public acknowledgement are common, but are unrealistic and are not a matter of “sufficient evidence” in any case. These demands are again rooted in an expectation derived from a legal system in which an impartial court standing in judgment of an individual can require such acknowledgment from a party found responsible for some misdeed. (On the possibility of such a court, see Box 4.)
Note that even if an adversary has openly claimed responsibility for an incident, decisionmakers would still have to ascertain the scope and nature of that claim—and intelligence analysts would go through exactly the same process of gathering and sifting evidence to arrive at a judgment with low, medium, or high confidence. This point is addressed further below.
The Relationship Between Attribution and Action
Attribution is a key element of taking responsive action, but attribution and responsive action are not independent variables. As noted earlier, even the type of attribution at issue in any given instance—that is, whether attribution should be to a specific machine, to a specific perpetrator, or to a specific adversary—depends on the goal of the relevant decisionmaker.
The section on the meaning of attribution began with a specific scenario. If the goal of the decisionmaker faced with that scenario is action to stop or mitigate the pain being caused by the intrusion as soon as possible, then what is most relevant is machine attribution: to find the machine causing the pain as quickly as possible and to take action against it. If Tony—the operator of the targeted computer—discovers that files are being deleted from his computer mid-attack, his immediate concern may be to simply stop this from happening. In this moment, he may not care that Karen—the owner of the attacking computer in Arkansas—is not truly responsible for initiating the attack. Instead, Tony simply is concerned that a computer in Arkansas is deleting files from his computer and intends to disrupt further infiltration by that computer. The perpetrator or the specific adversary ultimately responsible are not important.
If the goal of the decisionmaker is action to prosecute someone for an attack that has occurred, then he will care about ascertaining the identity of the perpetrator as the first step in taking the relevant person or persons into custody. In this case, identifying George as the perpetrator is crucial; as the actor who set the attack in motion, he is the person who can be charged with committing an actual crime. Of course, the ability to prosecute someone depends on the relevant legal regime that governs his or her actions—and the ultimately responsible party may have some influence over the specifics of that legal regime. Note also that Tony is most likely not the one who will decide that prosecution is the appropriate path to take. Someone else, higher in the chain of command, will almost certainly make that decision.
If the goal of the decisionmaker is action to deter malicious cyber activity in the future from being perpetrated against the entities for which the decisionmaker is responsible, then he cares most about the party that is ultimately responsible for motivating and initiating the activity. Identification of the responsible party is a prerequisite for administering the punishment required to dissuade it from conducting similar actions in the future. Identification of the responsible party is also a prerequisite in convincing an adversary that not undertaking the action to be deterred results in an acceptable outcome to him. The perpetrator is not the most relevant party in deterring future malicious activity, because anyone with sufficient technical skill can be hired, persuaded, or amused enough to press the right keys—that is, an individual person or “hacking for hire” organization is likely to simply be one cog in the machine. Because the ultimately responsible party could easily act through other would-be perpetrators or machines in the future, only the ultimately responsible party—the adversary—can actually be meaningfully deterred from initiating further malicious activity. Moreover, a decision to pursue deterrence rather than prosecution will be made at an even higher level up the chain of command—very much removed from Tony, the person operating the computer that suffered the attack.
Regardless of the type of attribution involved, the confidence required for an attribution judgment depends on the nature and target of that action. For example, policymakers would usually require a higher degree of confidence if the action contemplated were a kinetically destructive action than if the action were a diplomatic démarche—in general and all else being equal, the more “severe” or “serious” the action, the higher the confidence in an attribution judgment would have to be. Under some circumstances, the response action may simply be a public announcement pointing the finger at an ultimately responsive party—public “naming and shaming” may be effective in deterring future action, especially if the ultimately responsible party conducted its actions believing it could do so anonymously.
Similarly, and again all else being equal, policymakers would usually require a higher degree of confidence if the putative actor involved were a powerful nation or one with whom the United States had a relationship with multiple important threads than if it were a relatively weak nation or one that were relatively isolated.
The connection between attribution and action also has a temporal dimension. As noted above, attribution judgments are made on the basis of multiple sources of information, and integrating multiple sources of information takes time. Filtering through technical forensic details, comparing a given incident to previous incidents, extracting information obtained from human and signals intelligence sources, and so on are not easy tasks, and attributing a cyber incident may take weeks or months even when the analytical skills are available. Put differently, what is hard is prompt high-confidence attribution.
What is the significance of the difference between prompt and delayed attribution? For what purposes and under what circumstances is prompt attribution necessary (and by implication delayed attribution inadequate)? The answer depends on the nature of the response at issue for policymakers.
Consider first the tactical response to a malicious cyber incident. As noted above, machine attribution will be needed to mitigate the immediate harm being caused by the intrusion; the malicious operation of the machines involved in the intrusion must be blocked or disrupted. (Mitigation may well only be temporary if other machines are available to the adversary.) Choosing which courses of action would be most appropriate is another matter.
If the desired response is to arrest the perpetrator(s) or hold them criminally responsible for the incident, the conventions and rules of law enforcement hold sway. Because we hold individuals responsible for criminal acts, attribution to specific individual human beings is needed. Under these circumstances, rapid response may be desirable, but law enforcement authorities may work for years to identify, pursue, and take into custody individuals believed to be responsible for criminal acts.
If the desired response is to impose costs on a nation-state ultimately responsible for an intrusion, the conventions and rules of national decisionmaking are relevant, especially those of making such decisions in a security context. In the aftermath of a cyber attack, national security decisionmakers may respond by punishing or retaliating against an adversary’s attack. There are limits on such responses—retaliation or punishment for a hostile act once the act has stopped is prohibited under the UN Charter Section 2(4) if it rises to the level of a use of force. Nevertheless, forceful actions are allowable under Article 51 of the UN Charter if they can be regarded as acts of self-defense in the face of an armed attack, and such actions are often justified as acts of self-defense that deter future attacks. It is a matter of stated U.S. policy that a sufficiently severe cyber attack would indeed qualify as an armed attack under the UN Charter.
Note also that responses even to an armed attack may not entail the use of military force. As noted in the International Strategy for Cyberspace, the United States reserves the right to use “all necessary means—diplomatic, informational, military, and economic—as appropriate and consistent with applicable international law, in order to defend our Nation, our allies, our partners, and our interests” in response to hostile acts in cyberspace.
Appropriate responses are a central element of deterrence, but what makes a response appropriate? U.S. Strategic Command identifies three important factors for achieving deterrent effects; one was mentioned above, and the other two factors are credibility of a threat to impose costs on the adversary and costs that the adversary regards as too painful to incur. (Credibility is equivalent to certainty—a more credible response is one that an adversary regards as more certain. Painful costs are equivalent to severity of response.) These two factors are also identified in the traditional deterrence literature in international relations. If these conditions are met, an adversary faced with a credible threat to impose too-painful costs should the adversary act in a certain way will choose not to act in that way (i.e., will be deterred from that action).
By definition, an action that has already happened cannot be deterred. But future actions can be deterred, and an appropriate response to an action that has already happened serves to reinforce the credibility of a deterrent threat in the future. Thus, when faced with a decision about how to respond to a given hostile action, decisionmakers must identify the party against which to respond (i.e., they must attribute the hostile action correctly) and then respond in a sufficiently painful way so that the adversary will be deterred from similar actions in the future.
Curiously, the temporal element is missing from this calculus. Traditional theories of deterrence in international relations as well as U.S. Strategic Command’s construct for deterrence are silent on the impact on deterrence, if any, of the elapsed time between the hostile action and the response. It is intuitively plausible that long delays between hostile action and response will change the deterrent effect of a response, but whether this intuition is in fact true is not at all clear. For example, consider that an attribution effort that spans many months may cover the transition from one political administration to another, and a second administration may well have different policy preferences, some of which might drive different responses with different costs. A “tougher” administration might choose to impose costs that are even more painful than a “softer” one, or vice versa.
Delays in attribution may implicate international law as well. An extended period of time passing after an intrusion likely weakens the case for forceful responsive actions to be regarded as legitimately acting in self-defense, since actions taken in self-defense are supposed to be only the minimum necessary to restore the status quo. A similar argument holds true for countermeasures, which are acts that would be forbidden under international law except for the fact that they are taken in response to a prior illegal act by another nation and are intended to induce the cessation of that illegal act. For a sufficiently extended period of time (imagine in the limit a decade or two), a forceful “response” would likely be regarded as a new (and illegal) use of force in its own right.
Perhaps of greatest significance are the political dimensions. In some cases, the speed of a response—such as publicly calling out an adversary—is important for geopolitical reasons, since other events in the world will continue to play out and silence regarding an important intrusion will have negative consequences. Under such circumstances, policymakers are likely to accept a higher degree of uncertainty in an attribution judgment than they would prefer, especially if history suggests that a suspected adversary would benefit from silence. An overt signal to that adversary (or perhaps to an influential ally) sent promptly could help to forestall those negative consequences.
In other cases, policymakers may have an attribution of a malicious cyber incident in hand (indeed, perhaps a high-confidence attribution) and choose not to make it public. One obvious reason for not “going public” is the reality that a public attribution will generate demands for public evidence, a point discussed earlier. But another reason for not “going public” is that the relationships between many nations that act against each other in cyberspace are complex and multidimensional. “Going public” may result in demands to take retaliatory action that, in the view of senior policymakers, may be unwise given the range of interests at stake. The consequences of any retaliatory action (i.e., the significance of a possible adversary response) must be taken into account, and before policymakers decide to retaliate, they must be willing to face the consequences of any such action.
Lastly, the effects of the intrusion may manifest themselves quickly and force political leaders to face public pressures to “do something” even in the face of incomplete information. If one accepts that active cyber defense is likely to be technically ineffective, pressures for rapid response are in the end political in nature. Under these circumstances, the consequence of this conclusion is unpleasant for political leaders—they must be prepared to resist public pressures until the necessary judgments are in hand and to communicate to the public their rationales for waiting (Box 5).
Attribution from the Standpoint of the Adversary
Up to this point, this paper has focused on the victim’s perspective in attribution. But it is also necessary to consider the adversary’s perspective on attribution. For example, most discussions of attribution (including this one) assume that the adversary wishes to conceal its involvement in an intrusion. This assumption may not always be valid—an adversary (Nation A) may conduct an intrusion and deliberately engage in sloppy tradecraft to signal to the victim (Nation B) that it has the ability to conduct such an intrusion. A may send such a signal to B in the hope that knowledge of A’s capabilities would deter B from taking some action that would be undesirable to A.
Assuming the adversary wishes to conceal its involvement in an intrusion, it is important to consider any given intrusion in a larger context. Specifically, any given intrusion may be only one in a set of intrusions, and an adversary may well change its approach to later intrusions depending on the defending victim’s actions in attempting to attribute and/or thwart earlier intrusions. That is, the adversary’s techniques, tactics, and procedures may be adaptive to the defense’s actions.
Thus, if the adversary’s personnel make mistakes of tradecraft that give the victim enough information to attribute the intrusion publicly, they will try not to make those mistakes again. They may use different tools to conduct future intrusions to frustrate historical comparisons. Such actions may make the attribution judgment more difficult for the victim.
On the other hand, the adversary may not know what mistakes were made that revealed useful information to the victim. New tools may be unfamiliar to the adversary’s perpetrators, thus increasing the likelihood of making a mistake in using them. Such actions may increase the likelihood that an attribution judgment will be successful.
In short, while the victim faces a number of uncertainties in reaching an attribution judgment, the adversary faces a number of uncertainties in seeking to mask its responsibility. It is true that the victim cannot always be highly confident in the success of its attribution process, but although the cyber terrain favors the adversary under many circumstances, the adversary still cannot always be confident that it will remain anonymous. Put differently, even if the victim cannot always have high confidence in its ability to attribute an intrusion to a specific adversary, the adversary always runs some risk that the victim will be able to attribute hostile intrusions successfully. It is the very existence of such risk that underpins the possibility of deterring hostile actions in cyberspace.
If an adversary affirmatively wants its use of cyber weapons to be attributed to it for some reason, a somewhat different set of considerations applies. In this scenario, Nation A uses its cyber weapons against Nation B, but also wants B to know that A is responsible. In this context, one would usually speak of A taking credit for the cyber attack.
A could persuasively take credit simply by informing B that it was responsible for the cyber attack on target X belonging to B on a particular time and date, and providing B with details that only A would know about that particular attack. In this case, B would almost certainly want to verify A’s claims, and B would have to go through the all-source intelligence process described above to confirm A’s involvement. However, seeking to confirm A’s involvement is an easier task than determining A’s involvement, because in the former case, A has provided information that would not be available in the latter case.
In principle, it is also possible for A to use “loud” cyber weapons that self-attribute, much like nationality markings on aircraft assert that an airplane using the U.S. nationality marking is in fact a U.S. military airplane and national uniforms worn by soldiers assert that a soldier wearing a U.S. military uniform is in fact a member of the U.S. armed forces. But even if such cyber weapons are used (and U.S. Cyber Command has expressed an interest in obtaining such weapons), B might still have to go through the process of determining A was indeed responsible, even if the weapon was eminently traceable to A. (The technical challenge for self-attributing cyber weapons is two-fold. First, the self-attributing characteristic must not enable an adversary’s defenses to identify the weapon as hostile before it acts. Second, the self-attributing characteristic must not be usable by another Nation C.)
This paper began with the observation that attribution is a deep issue. In 2009, the National Research Council wrote that the “bottom line [on attribution] is that it is too strong a statement to say that plausible attribution of an adversary’s cyberattack is impossible, but it is also too strong to say that definitive and certain attribution of an adversary’s cyberattack will always be possible.” Fast forwarding to 2016, DNI Clapper’s earlier observation is consistent with that view—in some ways attribution is becoming easier, and in other ways it is becoming harder.
On one hand, attribution capabilities are increasing because more attention and resources are being devoted to the topic. Indeed, attribution capabilities are better than they were a decade ago in large part because nations are more attentive to the possibility of malicious cyber activity. They are thus more likely than before to collect data that might be useful in the investigation of a present—or a future—intrusion, and collection efforts have resulted in a decade’s worth of data, providing a historical corpus against which to compare future cyber intrusions. The tools for attribution are better and analysts are more experienced. Put differently—given the likelihood of malicious cyber activity in the future, many nations are more willing to make investments in intelligence and to build investigative capacity that will pay off in the future, and capabilities for attribution are in large part a function of the investment a nation is willing to make in those capabilities, both in infrastructure and in the effort that any given case demands.
On the other hand, adversaries are more aware than ever that they are being tracked, and given the ease with which false clues can be planted and false-flag operations conducted, they may well be more likely to carry out countermeasures to throw investigators off the attribution trail, especially as the stakes grow larger. And the number of skilled adversaries is growing. Adversaries that are identified can also exploit the uncertainty inherent in an attribution judgment. An adversary can deny its activities outright, secure in the knowledge that even if the information underlying the judgment is publicly revealed, that information is highly unlikely to contain any “smoking guns” pointing to its involvement. It can discredit each individual inference and piece of circumstantial evidence by pointing to alternative story lines. Such an approach to discrediting an attribution judgment may be especially valuable in the court of public opinion, in which individuals have little expertise on which to base their own judgments.
Policymakers are accustomed to making decisions about what to do or not to do under conditions of uncertainty—this is the reality of their daily lives. But the reality of some degree of irreducible uncertainty about attribution judgments has important political ramifications. If policymakers are forced to “go public” with an attribution judgment, skeptics and adversaries alike will pounce on any expressed uncertainty to dispute it and to set forth alternative theories and conclusions. Thus, they may be forced to assume a public posture that appears to be more certain than the actual evidence warrants.
The center of gravity of informed judgment seems to indicate greater overall confidence in attribution today compared to a decade ago, but the future remains cloudy as intruders and attributers advance their respective capabilities. Regardless of how these competing factors compare in the future, a number of fundamental propositions will remain. To be successful, attribution will always entail an all-source proposition, and technical forensics will be only one part of an attribution judgment. Attribution judgments will always have some degree of uncertainty associated with them, and the significance of such uncertainty is a political and policy matter rather than a technical one. Victims will have to live with the possibility that they will not be able to arrive at accurate attribution judgments with high confidence, and adversaries will have to live with the possibility that their victims will be able to attribute their malicious cyber activities to them.
Dr. Herbert Lin is senior research scholar for cyber policy and security at the Center for International Security and Cooperation and research fellow at the Hoover Institution, both of Stanford University.
An earlier version of this article was first published as Herbert Lin, “Attribution of Malicious Cyber Incidents: From Soup to Nuts,” Hoover Institution Aegis Paper Series on National Security, Technology, and Law, 5 September 2016.
I am grateful to Taylor Grossman’s services as research associate for this paper. Steven Bellovin, Eileen Donahoe, Kristen Eichensehr, David Elliott, John Gerth, Jack Goldsmith, Chris Jacobi, Alex Keller, Susan Landau, Hal Murray, Joseph Nye, Mark Seiden, Ashwin Sreenivas, Eli Sugarman, Wesley Tiu, and Benjamin Wittes provided valuable comments on an early version of this paper that helped to improve it.
Furthermore, this paper drew heavily on and built on work by W. Earl Boebert, John Carlin, David Clark and Susan Landau, Clement Guitton and Elaine Korzak, Jason Healey, Thomas Rid and Ben Buchanan, Jon Lindsay, Nicholas Tsagourias, and David Wheeler and Gregory Larsen.
 Malicious cyber activities or incidents are also sometimes known as “intrusions”; these terms are meant to include both what are called cyber attacks and cyber exploitations in much of the literature. Attacks are intended to destroy, degrade, damage, disrupt, manipulate, usurp, or reduce the availability of information and/or the computer and communications systems handling such information. Exploitations are intended to surreptitiously exfiltrate information that is meant to be kept confidential by the owners or operators of the system or network storing or transmitting such information. For more discussion of the difference between these two, see William Owens, Kenneth Dam, and Herbert Lin, eds., Technology, Policy, Law, and Ethics Regarding U.S. Acquisition and Use of Cyberattack Capabilities (Washington, DC: National Academies Press, 2009), Chapter 1.
 Many conceptualizations of deterrence include deterrence by denial: a strategy that seeks to deny an adversary the benefits it may realize by conducting malicious or hostile activities. According to the logic of deterrence by denial, an adversary will refrain from malicious actions if he knows he will not gain the benefits of those actions. In cyberspace, this approach is essentially equivalent to having cyber defenses that are sufficient to make it not worth the adversary’s while to act maliciously. The problem today is that we don’t know how to design, build, or operate cyber defenses that are sufficiently effective to deter.
 Sometimes, the misbehavior or badness is not apparent. A computer can be compromised in a way that allows it to be misused without causing any change in the computer’s behavior that is apparent to the user—that is, a machine can be compromised and still be fully and properly functional from the user’s standpoint. Such a compromise can nevertheless cause the machine to behave in a way that the user would not like if he or she knew about it. For example, a machine may be compromised to serve as a clandestine sender of spam or a proxy in an attack on another machine. While the user would not experience direct harm, his or her machine would be being used for nefarious purposes without his or her knowledge.
 For example, when a long time elapses between intrusion and the manifestation of a clue that something is wrong, many more system log entries may need to be examined to find the two or three useful entries that relate to the initial intrusion. Or multiple system updates performed during this time may have destroyed information that could have been useful.
 A complementary point of view is that computer or computer-based systems that allow the user to do the wrong thing are in fact defective in some sense themselves, even if the computers per se worked properly. Further, as such systems become more sophisticated, knowing whether a “bad outcome” is the result of human error or computer error becomes harder. And if the problem is “computer error,” we won’t know what the cause of the error is—in particular whether it’s due to a malicious actor or some unanticipated quirk from a big data analysis or something similar. This point, for which the author has considerable sympathy, will not be further addressed in this paper because it is not usually regarded as falling within the ambit of attribution as a security concern.
 Whether she bears responsibility for being careless in her security precautions is a different question. If she does, it would be fair to call her carelessness an indirect cause of or a contributing factor to the incident. (On the other hand, a system that makes it easy to inadvertently delete a file and not know it is poorly designed, and thus a deletion of a file could arguably reflect a system design problem rather than foul play.)
 A God’s-eye perspective describes what actually happened. The attribution process is intended to reveal to investigators as much of that perspective as possible.
 This particular way of formulating answers to this question owes much to a discussion found in David Clark and Susan Landau, “Untangling Attribution,” Harvard National Security Journal 2 (2011), 323-352.
 Ibid.; The term “stepping stones” is also used in the literature. See Yin Zhang and Vern Paxson, “Detecting Stepping Stones” (proceedings of the 9th USENIX Security Symposium, Berkeley, CA: 2000), 171-184.
 A good, if dated, treatment of technical means that can yield information useful for attribution can be found in David Wheeler and Gregory Larsen, “Techniques for Cyber Attack Attribution” (report, Institute for Defense Analyses, 2003). This report presages a number of the conclusions drawn in the present paper.
 See, for example, W. Earl Boebert, “A Survey of Challenges in Attribution” (Proceedings of a Workshop on Deterring Cyberattacks: Informing Strategies and Developing Options for U.S. Policy, National Academies Press, Washington, DC: 2010), 41-52.
 “Tor Project,” https://www.torproject.org/.
 “Electronic Privacy Information Center,” https://www.epic.org/privacy/tools.html.
 David E. Sanger and Martin Fackler, “N.S.A. Breached North Korean Networks Before Sony Attack,” The New York Times, 18 January 2015.
 Given the capability to pre-position instrumentation to surveil traffic in a potential adversary’s network, an interesting question is why one could not also pre-position other tools to shut down an intrusion by that adversary as it is being launched. A full answer to this question is beyond the scope of this paper. For now, two observations must suffice. First, it may not be possible to immediately recognize traffic associated with the start of an intrusion as such, especially if that information is collected and analyzed without knowledge of what is about to happen. Second, even if it were possible to do so, the scope and nature of the intrusion’s negative effects may not warrant exposing the intelligence capability in place, and weighing those equities (preventing the presumed negative effects of the intrusion vs. maintaining the secrecy of the intelligence capability in place) is not something that policymakers would do quickly or leave to an automated system to decide.
 And at worst, an adversary may be able to “hjjack” an IP address so that intrusion traffic appears to originate from that address, making the IP address much less useful as evidence for attribution. A similar outcome may result under circumstances in which IP addresses are assigned dynamically.
 Similar issues arise even in a purely domestic context that crosses state lines. For example, a 2013 decision of the Fifth Circuit Court of Appeals found that federal district judges may not authorize wiretaps of cell phones outside of their jurisdiction (United States v. North, 735 F.3d 212 (5th Cir. 2013)). This ruling conflicted with a 1997 decision of the Seventh Circuit Court of Appeals stating that district judges did have some authority to do so under certain circumstances (United States v. Ramirez, 112 F.3d 849 (7th Cir. 1997)). For a newspaper account of the Fifth Circuit case, see Joe Palazzolo, “Court Curbs Authority to Issue Wiretap Warrants,” Wall Street Journal, 27 August 2013. More recently, controversy has arisen over a proposed change to Rule 41 that some analysts believe grant judges anywhere, regardless of jurisdiction, the authority to “issue a search warrant to remotely access, seize, or copy data relevant to a crime when a computer was using privacy-protective tools to safeguard one’s location.” See Rainey Reitman, “With Rule 41, Little-Known Committee Proposes to Grant New Hacking Powers to the Government” (blog, Electronic Frontier Foundation, 30 April 2016).
 Whether actions such as turning on a Web camera to capture a picture of the person sitting at the keyboard should count as technical forensics are interesting edge cases.
 It has been observed that the same “neuro-physiological factors that make written signatures unique, are also exhibited in a user’s typing pattern,” and thus “when a person types, the latencies between successive keystrokes, keystroke durations, finger placement and applied pressure on the keys can be used to construct a unique signature (i.e., profile) for that individual. For well-known, regularly typed strings, such signatures can be quite consistent.” See Fabian Monrose and Aviel D. Rubin, “Keystroke dynamics as a biometric for authentication,” Future Generation Computer Systems 16 (2000): 351-359.
 This formulation (“who is to blame” vs. “who did it”) is due to Jason Healey, “Beyond Attribution: Seeking National Responsibility for Cyber Attacks” (issue brief, Atlantic Council, 2012).
 Article 4 of the International Law Commission’s Draft Articles on State Responsibility states that the “conduct of any State organ shall be considered an act of that State under international law, whether the organ exercises legislative, executive, judicial or any other functions, whatever position it holds in the organization of the State, and whatever its character as an organ of the central Government or of a territorial unit of the State.” The Draft Articles are a UN-sponsored attempt to codify international law in this area, but although a UN General Assembly Resolution in January 2002 (UN General Assembly Resolution 67/97, Responsibility of States for internationally wrongful acts, A/RES/56/83) took note of the Draft Articles and commended them to the attention of governments without prejudice to the question of their future adoption or other appropriate action, no further action has been taken on these articles.
 The Diamond Model proposed by Caltagirone et al. provides a useful model for integrating information (mostly but not exclusively technical information) related to intrusion activity. See Sergio Caltagirone, Andrew Pendergast, and Christopher Betz, “The Diamond Model of Intrusion Analysis” (paper, Center for Cyber Intelligence Analysis and Threat Research, 2013).
 “APT1: Exposing One of China’s Cyber Espionage Units” (report, Mandiant, 2013).
 Ibid., 62.
 John P. Carlin, “Detect, Disrupt, Deter: A Whole-of-Government Approach to National Security Cyber Threats,” Harvard National Security Journal 7 (2016), 391-436.
 The Federal Bureau of Investigation and the Drug Enforcement Agency are both federal law enforcement agencies and members of the intelligence community. See “Members of the IC,” Office of the Director of National Intelligence; Executive Order (EO) 12333 of 4 December 1981, United States Intelligence Activities, Code of Federal Regulations (1983).
 EO 12333 defines U.S. persons as U.S. citizens, U.S. permanent resident aliens, an unincorporated association substantially composed of U.S. citizens or permanent resident aliens, or a corporation incorporated in the United States, except for a corporation directed and controlled by a foreign government or governments; The only known public and explicit constraint on U.S. intelligence activities regarding foreigners is contained in PPD-28, which states, “To the maximum extent feasible consistent with the national security, these policies and procedures [in this PPD] are to be applied equally to the personal information of all persons, regardless of nationality.” In other words, PPD-28 states that foreigners do have some legitimate privacy interests against U.S. intelligence agencies, and that these agencies will treat that data (in the absence of national security concerns) as it treats data about U.S. citizens. For more on this point, see Benjamin Wittes, “The President’s Speech and PPD-28: A Guide for the Perplexed,” Lawfare, 20 January 2014. On the other hand, both U.S. law and policy do forbid other activities (e.g., EO 12333 forbids assassinations) or require that U.S. intelligence agencies may now engage in activities that violate the U.S. Constitution or U.S. statutes. To the extent that intelligence collection activities might run afoul of U.S. law, the U.S. Constitution, or Executive Order, they may not be undertaken. Also, other international law not specifically related to intelligence collection could prohibit certain collection activities—for example, torture is prohibited as a matter of international law, and U.S. intelligence agencies are prohibited from torturing individuals to collect intelligence information. This paper does not address the distinction between torture and enhanced interrogation techniques, but for more information on this point, see Anne Daugherty Miles, “Perspectives on Enhanced Interrogation Techniques” (report, Congressional Research Service, 2016).
 A May 2015 blog post on Lawfare by Ashley Deeks, “The Increasing State Practice and Opinio Juris on Spying,” notes that in the wake of the Snowden revelations, many states have expressed views on the relationship between surveillance and international law and that these expressions are “an important development in the process of understanding how intelligence activities are and should be regulated by international law.” See Ashley Deeks, “The Increasing State Practice and Opinio Juris on Spying,” Lawfare, 6 May 2015. In August 2015, Deeks also argues that “adopting a number of procedural norms to regulate foreign surveillance would help states and their citizens begin to balance the competing equities of privacy and security in concrete and observable ways.” See Ashley Deeks, “An International Law Framework for Surveillance," Virginia Journal of International Law 55 (2015), 291-368.
 Abraham Sofaer, “Cyber Security and International Agreements” (Proceedings of a Workshop on Deterring Cyberattacks: Informing Strategies and Developing Options for U.S. Policy, National Academies Press, Washington, DC: 2010), 179-207.
 Michael Vatis, “The Council of Europe Convention on Cybercrime” (Proceedings of a Workshop on Deterring Cyberattacks: Informing Strategies and Developing Options for U.S. Policy, National Academies Press, Washington, DC: 2010), 207-223.
 Healey, “Beyond Attribution.”
 I am indebted to David Aitel for this addition to the Healey taxonomy.
 I am indebted to Chris Jacoby for this point.
 The discussion of this paragraph is taken from Owens, Dam, and Lin, eds., Technology, Policy, Law, and Ethics, 186.
 See, for example, Article 8 of the International Law Commission (ILC) State Responsibility Articles, 47, and the ICJ Nicaragua decision (arguing for “effective control”) and the ICTY (International Criminal Tribunal for Yugoslavia) Tadic decision (arguing for “overall control”).
 UN Security Council, “Letter Dated 7 October 2001 From the Permanent Representative of the United States of America to the United Nations Addressed to the President of the Security Council,” UN Doc. No. S/2001/946 (2001); Derek Jinks, “State Responsibility for the Acts of Private Armed Groups,” Chicago Journal of International Law 4 (2003), 83-96.
 Clarke and Knake propose an explicit treaty assigns responsibility to nations for cyber activities emanating from their territories. See Richard Clarke and Robert Knake, Cyber War: The Next Threat to National Security and What to Do About It (Harper Collins: 2010).
 UN General Assembly, “Report of the International Law Commission: Fifty-Third Session” (New York, NY: 23 April to 10 August 2001).
 UN Security Council, Resolution 1267, 15 October 1999.
 David Fidler, “Cyber War Crimes: Islamic State Atrocity Videos Violate the Laws of War,” Net Politics, Council on Foreign Relations, 8 April 2015.
 “How the Court Works,” International Criminal Court.
 Thomas Rid and Ben Buchanan, “Attributing Cyber Attacks,” Journal of Strategic Studies 38 (2014), 4-37. Many of the points made in this section are also addressed in this paper, and in the view of the author, the Rid and Buchanan paper was the most complete paper on attribution when it was first published.
 For a canonical expression of this perspective, see Jeffrey Carr, “Responsible Attribution: A Prerequisite For Accountability” (Tallinn paper no. 6, NATO Cooperative Cyber Defence Centre of Excellence, 2014).
 A view often heard in the technical community and presented here in oversimplified form holds that definitive attribution is essentially impossible. Many in the technical community believe that only technical evidence speaks for itself, and that it is somehow “purer” and “less tainted” than information gained from some source whose motives were suspect and who could lie. They further assert that “a mountain of weak or poor-quality evidence” is inherently unpersuasive and non-authoritative. As someone who once held this view, I (the author of this paper) would assess each piece of weak evidence on its own (“weak” in this context meant “not bulletproof”), and because it was weak, I would throw it away. At the end of the process, because I insisted on throwing away every piece of weak evidence, and only weak evidence was available, I was left with no evidence at all. And of course, with no evidence, attribution is impossible. This is not to say that conclusions that emerge from analyzing weak evidence are necessarily reliable. An important caveat is that pieces of weak evidence collectively point to a stronger conclusion only when they are independent. For example, an intruder determined to mislead forensic investigators will plant a variety of false clues. Thus, at the moment of collection, the investigator cannot presume the independence of any given clue, and he or she must take into account the probability that a newly gathered clue is not in fact independent. On the other hand, that probability is not unity, and it would have to be probability 1.0 to discard the new clue entirely. In general, the higher the probability of non-independence, the greater the necessity of obtaining other corroborating sources that are not technical in nature.
 Michael Caloyannides, “Forensics is so ‘yesterday,’” IEEE Security & Privacy 7 (2009), 18-25. Some empirical work undertaken by Nunes et al. found that in an exercise where ground truth about identities was known, the majority of misidentifications of an intruder resulted from deceptive activities. See Eric Nunes et al., “Cyber-Deception and Attribution in Capture-the-Flag Exercises” (Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, Association for Computing Machinery, New York, NY: 2015).
 Similarities between the malware used in the 2014 hack on Sony Pictures Entertainment and malware used in other cyber intrusions previously attributed to North Korea were in part responsible for the FBI’s attribution of the Sony hack to North Korea. See James B. Comey, “Addressing the Cyber Security Threat” (remarks, International Conference on Cyber Security, Fordham University, NY: 7 January 2015).
 Carlin, “Detect, Disrupt, Deter: A Whole-of-Government Approach to National Security Cyber Threats,” 391-436.
 Guitton and Korzak elaborate on this point, arguing that the correlation between “sophistication” and likelihood of a nation-state actor being involved is not perfect, at least in part because “the lack of clarity and inconsistency around the term ‘sophistication’” means that sophistication is context-dependent and is therefore an unreliable guide to associating a nation-state with any given intrusion. See Clement Guitton and Elaine Korzak, “The Sophistication Criterion for Attribution: Identifying the Perpetrators of Cyber-Attack,” The RUSI Journal 158 (2013), 62-68.
 “INTelligence: Human Intelligence” (news, Central Intelligence Agency, 30 April 2013).
 For more on this point, see Jon R. Lindsay, “Tipping the scales: The attribution problem and the feasibility of deterrence against cyberattack,” Journal of Cybersecurity 1 (2015), 1-15.
 William Lynn III, “Defending a New Domain: The Pentagon’s Cyberstrategy,” Foreign Affairs 89 (2010), 97-108.
 Leon Panetta, “Defending the Nation from Cyber Attack” (remarks, Business Executives for National Security, U.S. Department of Defense, New York, NY: 12 October 2012).
 James Clapper, Worldwide Threat Assessment of the US Intelligence Community (testimony to the Senate Armed Services Committee, 26 February 2015).
 James Clapper, Worldwide Threat Assessment of the US Intelligence Community (testimony to the Senate Armed Services Committee, 9 February 2016).
 U.S. Department of Defense, The DOD Cyber Strategy (Washington, DC: 2015).
 The examples described here are taken from Kristen Eichensehr, “The Private Frontline in Cybersecurity Offense and Defense,” Just Security, 30 October 2014.
 “APT28: A Window Into Russia’s Cyber Espionage Operations?” (complimentary report, FireEye, 2014).
 “Operation SNM: Axiom Threat Actor Group Report” (report, Novetta, 2014).
 “CrowdStrike Intelligence Report: Putter Panda” (report, CrowdStrike, 2014).
 The discussion below of private sector attribution is derived from Herbert Lin, “Reflections on the New DOD Cyber Strategy: What It Says, What It Doesn’t Say,” Georgetown Journal of International Relations (forthcoming 2016).
 This point relates only to process, and should not be read to imply that private-sector analyses are necessarily less accurate or rigorous than government analyses.
 In the annals of intelligence, these words are called “words of estimative probability.” See Sherman Kent, “Words of Estimative Probability” (Central Intelligence Agency, 1964).
 Peter Strauss, “Due Process” (Legal Information Institute, Cornell University Law School). See also “due process of law” at Law.com (ALM Network of Legal Publications).
 Nicholas Tsagourias, “Cyber attacks, self-defence and the problem of attribution,” Journal of Conflict and Security Law 17 (2012), 229-244.
 International Court of Justice, Case Concerning Military and Paramilitary Activities In and Against Nicaragua (Nicaragua v. United States of America), Merits (1986), 28 (emphasis added); cited in Tsagourias.
 Separate Opinion of Judge Higgins in Case Concerning Oil Platforms (Islamic Republic of Iran v. United States of America), Merits (2003), ICJ Rep 161, paragraph 30; cited in Tsagourias.
 “Iran: Nuclear Intentions and Capabilities” (report, National Intelligence Council, 2007).
 Kent, “Words of Estimative Probability.”
 “Iran: Nuclear Intentions and Capabilities.”
 Ellen Nakashima, “Why the Sony hack drew an unprecedented U.S. response against North Korea,” Washington Post, 15 January 2015.; Even assuming that a disgruntled insider at Sony was involved, there is no reason in principle that government operatives from North Korea might not have compromised such an individual. Indeed, when asked whether other individuals may have assisted North Korea or been involved in the assault on Sony without being ultimately responsible for the damage, an FBI spokesperson said, “We’re not making the distinction that you’re making about the responsible party and others being involved.” See Shane Harris, “FBI Won’t Stop Blaming North Korea for Sony Hack – Despite New Evidence,” Daily Beast, 30 December 2014.
 Marc Rogers, “No, North Korea Didn’t Hack Sony,” Daily Beast, 24 December 2014.
 International Court of Justice, Corfu Channel Case, Merits (1949), 18.
 Theodore Sorenson, Counselor: A Life at the Edge of History (New York, NY: HarperCollins, 2008), 291.
 Jack Goldsmith, “The Sony Hack: Attribution Problems, and the Connection to Domestic Surveillance,” Lawfare, 19 December 2014.
 Jack Goldsmith, “The Consequences of Credible Doubt About the USG Attribution in the Sony Hack,” Lawfare, 30 December 2014.
 Paul Rosenzweig, “Was it North Korea?” Lawfare, 24 December 2014.
 I am grateful to a blog post by Kristen Eichensehr on this point; see Kristen Eichensehr, “’Your Account May Have Been Targeted by State-Sponsored Actors’: Attribution and Evidence of State-Sponsored Cyberattacks,” Just Security, 11 January 2016. The examples in subsequent endnotes are from her blog post as well.
 Eric Gross, “Security warnings for suspected state-sponsored attacks,” Google Security Blog, 5 June 2012.
 Facebook, “Notification for targeted attacks,” 16 October 2015.
 See, for example, Bill Gertz, “China Says OPM Hack Was Not State-Sponsored,” Washington Free Beacon, 2 December 2015.
 It is not coincidental that the same process occurs when various groups claim credit for an act of terrorism.
 This is one of three factors in U.S. Strategic Command’s formulation of the requirements for deterrence. See U.S. Strategic Command, Deterrence Operations: Joint Operating Concept, Version 2.0 (2006).
 Certain types of active cyber defense call for just such action, and a number of analyses have asserted the value of such action. This particular author is skeptical about the actual value of such action, but this point will not be addressed in this paper.
 Harold Hongju Koh, “International Law in Cyberspace” (remarks, USCYBERCOM Inter-Agency Legal Conference, Ft. Meade, MD: 18 September 2012).
 White House, International Strategy for Cyberspace (Washington, DC: 2011).
 U.S. Strategic Command, Deterrence Operations: Joint Operating Concept, Version 2.0 (2006).
 See, for example, Thomas Schelling, Arms and Influence (New Haven, CT: Yale University Press, 2008 (originally 1966)).
 The criminal deterrence literature does address the impact of celerity (or swiftness of punishment) on the deterrence of crime, but here too, the outcome is mixed. In a review of the criminal deterrence literature, Paternoster (Raymond Paternoster, “How Much Do We Really Know about Criminal Deterrence,” The Journal of Criminal Law and Criminology 100 (2010), 765-824) cites early theories of criminal deterrence (where early refers to theories of 1764!) arguing that punishment must be swift in order to be effective and more recent experimental work (1987) suggesting that “given the choice, people would like to get their punishment over as quickly as possible and that punishment delayed is seen as more costly than if given immediately”—that is, dread induced by delay increases rather than decreases the perceived cost of punishment. Paternoster concludes that the criminal deterrence literature has no real knowledge base about the celerity of punishment. Moreover, it is unclear how and to what extent, if any, the psychological mechanisms of would-be criminals driving their cost estimates are applicable to how nations account for potential costs. As an example of a criminal investigation running a very long time, consider that in the case of Benjamin Arellano-Felix, the leader of the Tijuana drug cartel, a period of fifteen years elapsed between his indictment (See “Under New Law, Mexico Extradites Suspect to U.S.,” New York Times, 5 May 2001) in 1997 and his incarceration in 2012 (Richard Marosi, “Former Drug Kingpin Arellano Felix Gets 25-Year Prison Term,” Los Angeles Times, 3 April 2012. (Both of these citations are from Carlin.)
 I am grateful to Kristen Eichensehr for this point.
 Joseph S. Nye, Jr., “Deterrence and Dissuasion in Cyberspace,” International Security 41 (2016/17).
 Nye, Jr., “Deterrence and Dissuasion in Cyberspace.”
 It is often the case that one intrusion is conducted to establish continuing access, thus facilitating later intrusions.
 Chris Bing, “U.S. Cyber Command director: We want ‘loud,’ offensive cyber tools,” fedscoop, 30 August 2016.
 Owens, Dam, and Lin, eds., Technology, Policy, Law, and Ethics.
 Lindsay, “Tipping the scales.”
 For example, North Korea explicitly denied that it was responsible for the Sony hack. See Bill Trott and Jonathan Oatis, “North Korea denies involvement in Sony cyber attack,” Reuters, 4 December 2014.
 Boebert, “A Survey of Challenges in Attribution.”
 Carlin, “Detect, Disrupt, Deter.”
 Clark and Landau, “Untangling Attribution.”
 Guitton and Korzak, “The Sophistication Criterion for Attribution.”
 Healey, “Beyond Attribution.”
 Rid and Buchanan, “Attributing Cyber Attacks.”
 Lindsay, “Tipping the scales.”
 Tsagourias, “Cyber attacks, self-defence and the problem of attribution.”
 Wheeler and Larsen, “Techniques for Cyber Attack Attribution.”