Cyber Threat Susceptibility Assessment

Definition: Cyber Threat Susceptibility Assessment (TSA) is a methodology for evaluating the susceptibility of a system to cyber-attack. TSA quantitatively assesses a system's [in]ability to resist cyber-attack over a range of cataloged attack Tactics, Techniques, and Procedures (TTPs) associated with the Advanced Persistent Threat (APT). In the Mission Assurance Engineering (MAE) methodology, cyber TSA is a follow-on to Crown Jewel Analysis (CJA), and a prerequisite to cyber Risk Remediation Analysis (RRA).

Keywords: Advanced persistent threat, APT, cyber-attack, MAE, mission assurance engineering, risk remediation, Threat Susceptibility Matrix, TTP

MITRE SE Roles & Expectations: MITRE systems engineers (SEs) are expected to understand the purpose and role of Mission Assurance Engineering (MAE) in the systems acquisition life cycle and its constituent activities, including cyber Threat Susceptibility Analysis (TSA). The MAE methodology has application throughout the system life cycle. MITRE SEs are expected to know the context(s) in which this methodology can be applied. MITRE SEs are also expected to help establish the scope of the assessment and set sponsor expectations regarding deliverables and schedules.

Introduction and Background

The MAE process framework is depicted in Figure 1. This framework provides an analytical approach to:

  • Identify the cyber assets most critical to mission accomplishment (the "crown jewels" of a Crown Jewels Analysis).
  • Understand the threats and associated risks to those assets (accomplished via a subsequent cyber Threat Susceptibility Analysis (TSA)).
  • Select mitigation measures to prevent and/or fight through attacks (cyber Risk Remediation Analysis (RRA) is used to identify recommended mitigation measures).

The MAE process framework provides a common repeatable risk management process that is part of building secure and resilient systems [1].

Figure 1. The Mission Assurance Engineering (MAE) Process Framework

Cyber threat susceptibility analysis (TSA) is an MAE activity that quantitatively assesses a system's [in]ability to resist cyber-attack over a range of adversary Tactics, Techniques, and Procedures (TTPs). A TSA assessment produces a Threat Susceptibility Matrix, which provides a ranked list of TTPs that cyber assets are susceptible to. This matrix is used in a follow-on MAE activity called cyber Risk Remediation Analysis, which develops recommendations for how to mitigate cyber TTP risk.

This article focuses on what we know today about cyber TSA, a fast-moving discipline. The concepts and methods described are evolving and will continue to mature as more experience is gained in applying this discipline. Please revisit this article for additional insights as the community's collective knowledge builds.

The first step in a cyber TSA assessment is to establish the scope of the evaluation. Assessment scope is characterized in terms of:

  • The set of assets being evaluated
  • The range of attack TTPs being considered
  • The types of adversaries

When TSA is conducted as a follow-on to a Crown Jewel Analysis (CJA), the set of system assets within the scope of the assessment may include identified crown jewel cyber assets, i.e., cyber assets whose compromise would seriously impair mission capability or readiness. If the TSA is being conducted independently or in the absence of the CJA, the list of cyber assets may be arbitrarily selected or may include a presumptive list of crown jewel cyber assets.

The range of attacks considered in a TSA assessment may include but is not limited to cyber, electronic warfare (EW), and supply chain. A cyber-attack targets an enterprise's use of cyberspace for the purpose of disrupting, disabling, destroying, or maliciously controlling a computing environment/infrastructure; destroying the integrity of the data; or stealing controlled information. Electronic warfare refers to military action involving the use of electromagnetic and directed energy to control the electromagnetic spectrum or to attack the enemy. Supply chain attacks allow the adversary to use implants or other vulnerabilities inserted prior to installation in order to infiltrate or exfiltrate data or to manipulate information technology hardware, software, operating systems, peripherals (information technology products) or services at any point during the life cycle. The Advanced Persistent Threat (APT) refers to adversaries, typically nation states, capable of mounting sophisticated attacks in each of these areas.

Types of adversaries considered in a TSA assessment may include external adversaries, insiders, and trusted insiders. The distinctions among the types are fuzzy, but relate to the adversary's proximity to the targeted system. A security perimeter separates an external adversary from an internal adversary, i.e., an insider. This perimeter can take the form of a firewall, a DMZ, a locked door, and so on. Once the security perimeter is breached, however, the external adversary has gained insider access. Similarly, an insider is distinguished from a trusted insider by the level of access granted, i.e., a trusted insider may have physical or administrative access that an unprivileged user does not. Enforcing the principle of least privilege separates insiders from trusted insiders, who may have opportunities to apply a wider range of attack TTPs than insiders or external adversaries. The scope of a TSA assessment may include or exclude each of these types of adversaries.

Once the scope of the TSA assessment is established, the next step is to evaluate the cyber asset's architecture, technology, and security capabilities against a cataloged set of TTPs. Unclassified, open source TTP catalogs used in TSA assessments include MITRE-hosted resources such as:

  • Common Attack Pattern Enumeration and Classification (CAPEC)—A compilation of cyber-attack patterns that describe common methods for exploiting software derived from specific real-world incidents [2]. In this context, the terms "attack TTP" and "attack pattern" are synonymous.
  • Common Weakness Enumeration (CWE)—A catalog of defined software weaknesses that attack TTPs may exploit [3].
  • Common Vulnerabilities and Exposures (CVE)—An open-source dictionary of publicly known information security vulnerabilities and exposures [4].  

This initial set of TTPs undergoes a narrowing process to eliminate TTPs considered implausible. Several factors can make a TTP an implausible method of cyber attack. Many TTPs have prerequisites or conditions that must hold true in order for that TTP to be effective. A prerequisite for a structured query language (SQL) injection attack, for example, is that the system must include a SQL database. Weak passwords is one condition that must hold true in order for an adversary to successfully conduct brute force password attacks. Many candidate attack TTPs may be eliminated because of missing prerequisites.

It is also possible to eliminate candidate attack TTPs by making assumptions about the system's security posture. For example, DoD systems undergo the Defense Information Assurance Certification and Accreditation (DIACAP) process to verify that all required security controls are implemented. One set of security controls requires that the system's configuration be hardened using Defense Information Systems Agency published Security Technical Implementation Guides (STIGs). Certain attack TTPs may not be plausible for systems that have been hardened in accordance with these STIGs.

Candidate attack TTPs that cannot be eliminated may be ranked using a scoring model that assesses the risk associated with each TTP relative to other plausible TTPs considered in the assessment. This ranking helps set priorities on where to apply security measures to reduce the system's susceptibility to cyber-attack. The default TTP scoring model spreadsheet is illustrated in Figure 2.

Figure 2. Default TTP Risk Scoring Model

The default TTP scoring model assesses TTP risk based on twelve criteria, including impact, restoration costs, down time, level of sophistication, likelihood for attribution, and so on. This list of criteria, which has evolved over time, may be tailored for use in a given assessment. Use of the same scoring model provides a common context for comparing and ranking TTPs based on relative risk. TTP risk scores derived using different scoring models are not comparable.

A uniform range of values [1..5] is assigned to each criteria. For criteria such as impact, a higher value results in a higher TTP risk score. These criteria appear in blue in the scoring model spreadsheet. For adversary level of sophistication criteria, such as required skills and required resources, a higher value results in a lower TTP risk score. These criteria appear in red in the scoring model spreadsheet. In the threat model from which this scoring model derives, it is assumed that a high degree of adversary sophistication required to successfully execute a TTP reduces the overall likelihood of occurrence, leading to a lower overall risk score.

In the default TTP scoring model, different criteria can have different weightings. Some criteria may be more significant to the overall risk score than others. For a system that processes classified data, for example, a higher weighting is assigned to loss of confidentiality than for a system that processes unclassified data. TTP risk scores are calculated based on the criteria value assignments and associated criteria weightings. In the default scoring model, this calculation yields a TTP risk score in the range [1..5], with the value 5 signifying a TTP that poses the greatest risk.

A TSA assessment produces a Threat Susceptibility Matrix, which lists plausible attack TTPs ranked by decreasing risk score, and their mapping to cyber assets as a function of adversary type. The Threat Susceptibility Matrix also tabulates TTP risk scores to provide an overall assessment of aggregate susceptibility to cyber-attack for each cyber asset considered in the assessment. This matrix is used in the follow-on cyber risk remediation analysis (RRA) to identify countermeasures that effectively mitigate TTP susceptibilities. For further information on cyber RRA, see the companion article under this same topic. A sample Threat Susceptibility Matrix is illustrated in Figure 3.

Figure 3. Sample Threat Susceptibility Matrix

The sample Threat Susceptibility Matrix in Figure 3 evaluates two cyber assets over a range of sixteen attack TTPs that are scored using the default TTP scoring model from Figure 2. If a cyber asset is susceptible to a TTP, its risk score is transferred to that cyber asset. At the bottom of the matrix, aggregate susceptibility is tabulated for each cyber asset and adversary type. In this example, Cyber Asset #2 is more susceptible than Cyber Asset #1.

TTPs are "binned" into risk categories based on risk score, as follows:

  • TTPs in the range [4.0 . . . 5.0] pose serious risk and appear in red.
  • TTPs in the range [2.5 . . . 3.9] pose moderate risk and appear in yellow.
  • TTPs in the range [1.0 . . . 2.4] pose minimal risk and appear in blue.

Government Interest and Use

TSA has been applied to sponsor systems in various forms for a number of years. Before 2010, TSA assessments used a loosely defined, non-rigorous, and undocumented methodology. In 2010, a formal methodology for conducting TSA assessments was developed by MITRE, which has subsequently been applied to Army, Navy, and Air Force programs [5]. The methodology outlined above reflects this TSA approach.

Best Practices and Lessons Learned

Timing is critical. TSA may not be well suited to all phases of acquisition programs. For example, the Threat Susceptibility Matrix cannot be constructed without knowledge of the cyber assets that make up the system. The identification of cyber assets is derived from the system's allocated baseline, which may not be fully defined prior to PDR.

Assume the adversary can gain access. Mission Assurance Engineering (MAE) is based on the assumption that APT adversaries are able to successfully penetrate a system's security perimeter and gain persistent access. TSA's focus on the Insider or Trusted Insider relates to the notion of adversary proximity and in no way reflects on the loyalty or ability of IT staff.

TSA of non–crown jewel assets—The value proposition. Although a Crown Jewel Analysis (CJA) identifies cyber assets of greatest importance to mission success, it does not identify the cyber assets that are most susceptible to attack. There is value in scoping a TSA assessment to evaluate non–crown jewel cyber assets, especially those whose compromise would give an attacker a path to any crown jewel assets.

Importance of documented rationale as context for future efforts. It is important to record the rationale for eliminating candidate attack TTPs from consideration, including assumptions made regarding the system's security posture or Security Technical Implementation Guide (STIG) compliance. The rationale provides context in follow-on MAE activities such as Threat Remediation Engineering (TRE).

More than one cyber risk assessment methodology. Several methodologies similar to TSA assess risk to cyber assets, including Microsoft's DREAD [6] and the National Security Agency's MORDA [7]. Each methodology functions by assessing cyber assets using a defined set of evaluation criteria. The criteria used in this article's default TTP scoring model are representative of criteria used by these other methodologies and can be tailored to meet the needs of the program.

Pathological scores and what to do about them. Certain "pathological" TTP scoring modes may reflect situations where more information about a cyber asset is required, evaluation criteria need to be revised, or the assigned range of values is either too narrow or too wide. When tailoring the scoring model to address this, it is necessary to go back and rescore all TTPs using the updated model. Otherwise, a single scoring model is not being used and there is no basis for comparing TTP risk scores in the assessment.

Variation on the TTP risk scoring theme. One variation on the TTP risk score calculation is to compute low and high TTP risk scores based on a range of values for each evaluation criteria. This approach produces risk scoring that reflects best case and worst case assumptions.

The need for remediation. A cyber TSA provides no recommendations on how to respond to cyber risk.  Cyber Risk Remediation Analysis (RRA) is the core MAE portfolio activity used to identify risk mitigation alternatives. Threat Assessment and Remediation Analysis (TARA) [8] is the MAE portfolio practice that combines Cyber TSA with RRA. Sponsors seeking to identify, assess, and mitigate cyber risks are encouraged to consider a TARA assessment.

References & Resources

  1. Goldman, H., Building Secure, Resilient Architectures for Cyber Mission Assurance, The MITRE Corporation, October 2010.
  2. Common Attack Pattern Enumeration and Classification (CAPEC).
  3. Common Weakness Enumeration (CWE).
  4. Common Vulnerabilities and Exposures (CVE).
  5. Wynn, J., and Montella, L., "Cyber Threat Susceptibility Analysis (TSA) Methodology," Version 2.0, MITRE Technical Report (MTR) 100379, October 2010.
  6. Meier, J. D., Mackman, A., et al., Microsoft Patterns and Practices, Chapter 3, Threat Modeling, June 2003.
  7.  Buckshaw, D., Parnell, G., et al., "Mission Oriented Risk and Design Analysis of Critical Information Systems (MORDA)," Military Operations Research, vol. 10, no. 2, 2005, pp. 19–38.
  8. Wynn, J., et al., "Threat Assessment & Remediation Analysis (TARA)", Version 1.4, MITRE Technical Report 110176, MITRE Corporation, October 2011.


Download the SEG

MITRE's Systems Engineering Guide

Download for EPUB
Download for Amazon Kindle
Download a PDF

Contact the SEG Team