You may not be aware, but a group composed of some of the most famous Threat Modeling practicians has joined forces and recently published the Threat Modeling Manifesto, a document summarizing some important principles around Threat Modeling.
I applaud this effort and consider the content as both interesting and useful. There are various levels to read the Threat Modeling Manifesto; thus, it may help to make an in-depth analysis of its content.
I have to anticipate that while I fully agree with the Threat Modeling Manifesto, besides some minor concerns I discuss in my analysis, I do not necessarily agree with all the explanations I am reading from its authors.
Of course, I am not so arrogant to think that my interpretation is the correct one, but it may still contain valuable insights. Hopefully, it could help to further improve the Manifesto.
I guess the fact that the Manifesto is still very valuable and correct is yet another demonstration of how great is the work its authors did.
What is Threat Modeling?
Threat modeling is analyzing representations of a system to highlight concerns about security and privacy characteristics.
Threat Modeling Manifesto, What is Threat Modeling.
At the highest levels, when we threat model, we ask four key questions:
– What are we working on?
– What can go wrong?
– What are we going to do about it?
– Did we do a good enough job?
Those four questions cover all the various major phases of the Threat Model:
- Understanding the system in scope.
- Identifying the potential attacks, that is the Threats.
- Listing actions that could be done to reduce the risk, that is the Mitigations.
- Evaluate the potential impact of the Threat Model activity on the risk, and also identify potential improvements to the process.
In my blog, I have written at length on those steps. And there is a considerable space for improvement in any given organization from the learnings obtained from a full Threat Modeling experience.
It is important to note that those questions are partial by design because they represent the absolute minimum aspects of a Threat Model. For instance, after we have gained a good understanding of the system, we may want to identify its weaknesses (what issues do we have?) and use this knowledge to determine how a malicious actor may exploit them, which would be addressed by the “what can go wrong” question. That would represent an intermediate step between the first two.
Why Threat Model?
When you perform threat modeling, you begin to recognize what can go wrong in a system. It also allows you to pinpoint design and implementation issues that require mitigation, whether it is early in or throughout the lifetime of the system. The output of the threat model, which are known as threats, informs decisions that you might make in subsequent design, development, testing, and post-deployment phases.
Threat Modeling Manifesto, Why threat model?
I partially agree with this one.
I know I am very picky, but I need to highlight a discrepancy between the previous questions and this description. As noticed previously, the basic set of questions lacks one related to the identification of “design and implementation issues” (see the proposed, “what issues do we have?”).
Talking about “what may go wrong” is not the same than talking about the causes. Matter of fact, there is some confusion and many approaches do not really discriminate between weaknesses and threats. Microsoft STRIDE approach definitely doesn’t, as the current approach we have with Threats Manager Studio – but this is something that will be fixed soon, in fact Threats Manager Platform already has the concept of Weaknesses and Vulnerabilities.
What I do not like in this definition is that the focus is on the Threats. I know, Threat Modeling is by definition about modeling Threats, but this is not useful per se, because it does not tell anything about what we could do to mitigate those Threats. In other words, the Threats are essential to understand the need and the priority of the Mitigations, but the real focus should be on the Mitigations. If you think about that, the same Threat Modeling Manifesto asserts the Mitigations’ centrality with the third question: “What are we going to do about it?”.
On the same line of thought, I would add another important concept related to Threats we are missing in the Threat Modeling Manifesto: the Risk, defined as potential loss. In fact, you may accept compromise if the damage is limited. Acceptance of the risk is totally fine and even expected as part of the Threat Modeling process. As Threat Modelers, our role is to give an awareness of this Risk through different means: it could be through the adoption of a Bug Bar or something more sophisticated like FAIR, but still it would a necessary step to allow the solution Stakeholder to make a decision.
The alternative definition I would propose, is:
Threat Modeling is an ever-evolving process to understand potential security attacks to a system (also known as Threats), determine potential losses (the Risks) deriving from them, and establish appropriate mitigations.
Simone’s definition to Threat Modeling.
Why is it better? Because it aligns all those elements we have discussed earlier:
- Threat Modeling is an ever-evolving process –> “Did we do a good enough job?”
- to understand potential security attacks to a system –> covers both “What are we working on?” and “What can go wrong?”.
- determine potential losses –> again, this is crucial, because represents our key measure and is linked with our final goal, that is to make losses acceptable.
- establish appropriate mitigations –> the main tool to achieve our goal, aligned with question “What are we going to do about it?”.
Who should threat model?
You. Everyone. Anyone who is concerned about the privacy, safety, and security of their system.
Threat Modeling Manifesto, Who should threat model?
I agree with that, but I do not agree with the consequence some of the Manifesto authors are getting out of this. In fact, some of them imply that being Threat Modeling everyone’s responsibility, and considering that we do not have many experienced Threat Modelers, it would become Developers’ responsibility to perform Threat Models for their solutions.
This is a great idea, on paper. And it may work in the future, perhaps ten years from now, but it doesn’t as of today. Why? For three reasons:
- Threat Modeling is about the Security of the design. Developers are focused on code.
- Developers rarely see the whole picture. Threat Modeling is built on the holistic approach.
- Most often than not, developers are slammed by the need to implement functional requirements. No way they can focus the required time to do a threat model.
Most developers simply lack the most relevant skills to be good Threat Modelers. I have collected various articles explaining what the critical skills required to be a great Threat Modeler are:
- Being a better Threat Modeler « Simone On Security
- Knowing Risks and Mitigations « Simone On Security
- The importance of the holistic approach (simoneonsecurity.com)
- The Critical Mindset « Simone On Security
Those skills can be learned: there is nothing requiring special genes or talents. Still, people lacking any of them would do a poor job compared with those who don’t.
The only way I know to ensure you get similar results from both beginners and experts is to use automation. Typically automation has the primary goal of somewhat improving the quality provided by the first category. Unfortunately, in too many cases automation has been implemented without considering the needs of the experienced Threat Modelers, with the result of forfeiting the possibility to get excellent results. That’s not where we should go. What we need is to lower the complexity of the process, making easier for everyone to become an expert, without forfeiting Quality.
A different matter altogether is when Threat Modeling is performed as an activity done by a Team led by an expert. I agree with this view wholeheartedly.
How should I use the Threat Modeling Manifesto?
Use the Manifesto as a guide to develop or refine a methodology that best fits your needs. We believe that following the guidance in the Manifesto will result in more effective and more productive threat modeling. In turn, this will help you to successfully develop more secure applications, systems, and organizations and protect them from threats to your data and services. The Manifesto contains ideas, but is not a how-to, and is methodology-agnostic.
Threat Modeling Manifesto, How should I use the Threat Modeling Manifesto?
The Threat Modeling Manifesto follows a similar format to that of the Agile Manifesto by identifying the two following guidelines:
– Values: A value in threat modeling is something that has relative worth, merit, or importance. That is, while there is value in the items on the right, we value the items on the left more.
– Principles: A principle describes the fundamental truths of threat modeling. There are three types of principles: (i) fundamental, primary, or general truths that enable successful threat modeling, (ii) patterns that are highly recommended, and (iii) anti-patterns that should be avoided.
Nothing to comment, here.
Values
We have come to value:
Threat Modeling Manifesto, Values
A culture of finding and fixing design issues over checkbox compliance.
People and collaboration over processes, methodologies, and tools.
A journey of understanding over a security or privacy snapshot.
Doing threat modeling over talking about it.
Continuous refinement over a single delivery.
I am in compete agreement with those values.
To them, I would add:
- Awareness and options over forcing the view of the Expert.
The idea of this value is that our goal as Threat Modelers should be to provide an unbiased view of the risks – as unbiased as possible, at least – and then to identify multiple mitigations for each Threat, with the intent of giving choices. Another angle to that is related to the consideration that not all Mitigations are born equal: not only we do have multiple Security Controls, to get Defense in Depth. Ultimately, we shall not lose the focus on the goal of reducing the Residual Risk to an acceptable level, not to force anyone to implement the Mitigations we think are necessary.
This new Value is also aligned with the Threat Modeling Hero Anti Pattern.
Principles
We follow these principles:
Threat Modeling Manifesto, Principles
– The best use of threat modeling is to improve the security and privacy of a system through early and frequent analysis.
– Threat modeling must align with an organization’s development practices and follow design changes in iterations that are each scoped to manageable portions of the system.
– The outcomes of threat modeling are meaningful when they are of value to stakeholders.
– Dialog is key to establishing the common understandings that lead to value, while documents record those understandings, and enable measurement.
I agree with those principles. Again, I think the “Awareness and Options” principle should be added to increase the alignment between Values and Principles.
Patterns
Achieve thoroughness and reproducibility by applying security and privacy knowledge in a structured manner.
Threat Modeling Manifesto, Systematic Approach Pattern
This requires “security and privacy knowledge” and also to know about the right structured approach. Agreed, but again this knowledge may not be easy to apply to those not focused on Security Design, which means Architects may be the best candidates, indeed not the Developers.
Allow for creativity by including both craft and science.
Threat Modeling Manifesto, Informed Creativity Pattern
Agreed. This means that automation is an option, but it shall not be the only option. Expert ingenuity should get a prominent place in the Threat Modeling practice.
Assemble a diverse team with appropriate subject matter experts and cross-functional collaboration.
Threat Modeling Manifesto, Varied Viewpoints Pattern
It is essential to involve not only Security Experts and Architects, who would be the first targets of the Threat Modeling activity, but also Business Stakeholders and other roles, including those who work with Operations. A great pattern I agree with.
Support your approach with tools that allow you to increase your productivity, enhance your workflows, enable repeatability and provide measurability.
Threat Modeling Manifesto, Useful Toolkit Pattern
<cough> Threats <cough> Manager <cough> Studio <cough> 🙂
Seriously, this point is about Continuous Improvement of the Threat Modeling practice. It also means: do not consider Threat Modeling as a fixed process but continuously improve it.
Use successfully field-tested techniques aligned to local needs, and that are informed by the latest thinking on the benefits and limits of those techniques.
Threat Modeling Manifesto, Theory into Practice Pattern
Ever-evolving process and adaptability. Those are definitely two great principles.
Additional Patterns
Focus on Residual Risk Acceptability
The goal is to make the Residual Risk acceptable, not to make it zero. This means two things. First of all, that it is the responsibility of the System Owner to decide what to do for a Threat: to accept the risk, mitigate it (and eventually, how), to avoid it by removing the affected functionality, or to transfer it.
Secondarily, this means that we need to measure in some way the Residual Risk. I would love to have a quantitative measure of the risk, for example adopting FAIR, but for now, I fear we have to accept sub-optimal qualitative approaches.
Anti Patterns
Threat modeling does not depend on one’s innate ability or unique mindset; everyone can and should do it.
Threat Modeling Manifesto, Hero Threat Modeler Anti Pattern
Innate being the keyword here. If this implies that an inexperienced child should be able to Threat Model, then I would say that I disagree with this point.
Are we thinking anyone attending high-school could design skyscrapers? Of course not! Do we believe that only a few blessed persons can design an industrial implant? Neither, of course. This is what training and experience are there for. We need to simplify the process to make it more easily adopted. We also need to reduce the cost and increase the quality to give Threat Modeling more appeal and entice more organizations to invest in this practice. But we cannot take shortcuts like trying to make Threat Modelers out of the average developer.
Go beyond just analyzing the problem; reach for practical and relevant solutions.
Threat Modeling Manifesto, Admiration of the Problem Anti Pattern
Agreed, but this looks obvious. When I see recommendations like this one, I cannot refrain from thinking about the situation that has caused to think about adding it. And I guess it was not an isolated case, even, or they would not have added it to the list of anti-patterns. Interesting.
Do not lose sight of the big picture, as parts of a model may be interdependent. Avoid exaggerating attention on adversaries, assets, or techniques.
Threat Modeling Manifesto, Tendency to Overfocus Anti Pattern
Agreed. This is also the “original sin” shared by all tools that automatically generate Threats and associate Mitigations – including Threats Manager Studio. The way automation can be implemented nowadays, that is through rules considering tiny fragments of the Threat Model, is why this functionality is separated from the main others in TMS.
The point here is that Threat Modeling must stop at a certain point because further analysis would tend to increase the cost of the Threat Modeling experience more than linearly.
It is better to create multiple threat modeling representations because there is no single ideal view, and additional representations may illuminate different problems.
Threat Modeling Manifesto, Perfect Representation Anti Pattern
Agreed. Ideally, the Threat Model represents a unique truth that should be read from different angles, based on your role and needs. And each role should have multiple views.
Additional Anti Patterns
Force your agenda
The Threat Modeling experience should be as unbiased as possible. Never use it to force the implementation of some Mitigation for personal reasons. Never add or remove recommendations because you are somewhat forced to. Never accept to change the severity of some Threat simply because someone wants you to. The Threat Model should provide a faithful representation of the risks and possible mitigations, to the best of your knowledge. Mistakes are possible, of course, but only if done in good faith. Professional integrity is a must.
This also means that it is perfectly fine for the Owner of the solution to accept the risk. Your responsibility as a Threat Modeler is only to provide awareness about risks and options, not to decide what to do with them.
Conclusions
Threat Modeling Manifesto is an excellent contribution to the Security Community, and we all shall be grateful to those who have had the initiative. It contains a lot of good material, and I hope it will foster many conversations around Threat Modeling and Security, hopefully creating even more improvement opportunities.
It still has space for improvements, like everything else.
I hope with my small contribution to have given further strength to this wonderful initiative, to let it be known a little more and, why not, to make it even better.
Simone On Security 