Thinking what can go wrong? Introduction to Threat Modeling.

This series will be divided into two articles answering the following questions:

1. Why integrate threat modeling into your SDLC process?
2. How to properly use the results of threat modeling?

In this part we will focus on the first point.

What is threat modeling?

Threat modeling is an activity to help understand what you are building and to verify if you have done a good job – focused on security and privacy concerns. In order to answer a question of “Is this software secure?” or “What security mechanisms should we apply?” it is necessary to identify potential threats. This allows to optimise the cost and effort of security programme.

You are already doing it

Threat modeling may sound like a complicated process performed by corporate risk departments but, believe me, each and every one of you performs it every day intuitively. You analyse the current risk, potential consequences and apply some mitigations to avoid the threats. I will give you a few examples:

• You choose a taxi over walking through a dangerous neighborhood at night.
• You would not take all your cards and money when going surfing and leaving your stuff on the beach.
• When travelling abroad, you don’t keep a lot of cash in a back pocket or you invest in a hidden wallet or a pouch.

Intuition does not always choose the best decisions. Common counterexamples for perceived vs actual risk include:

• Widespread fear of flying, while the actual risk of dying in an aviation accident is much lower than a death in a car crash on the way to the airport.
• The West Nile virus outbreak in 2002 killing 284 people in the US vs. 3.1 million deaths annually (worldwide) because of AIDS.

In some industries, especially those where human’s life is at risk, responsibility for actions is much more formalised. Imagine a simplified process of building a bridge that should not collapse:

The investor transfers the risk of a collapse to the construction company
⬇️
An architect or urban planner estimates the potential threats to the bridge (weather conditions, required load that needs to be handled) 
⬇️ 
A constructor calculates what kind of beams need to be applied to support such a load
⬇️
The builders apply the required mitigations and construct the bridge
⬇️
The bridge can be tested in a controlled environment
⬇️
Regular maintenance – the bridge is regularly checked for cracks which may affect its construction

Now, there are two questions I will try to answer: Can you build a bridge in agile methodology? And: Can you apply the above threat modeling in software development?

Agile bridge development 

What could go wrong with an agile development of a bridge? Well, first, if no formal (architectural) threat modeling is made to answer what could happen to a bridge in the future, at some point there will be a product that is ready to use, at least to some level. Welcome to an MVP of bridges (Minimum Viable Product). It is not even a bridge yet – more like a pedestrian overpass. You can put a ‘no entry” road sign for vehicles and it works pretty well. However, the moment it turns out it should also support bigger vehicles, huge changes in construction are required. Usually, it would be much cheaper to put stronger beams in the very beginning but this, on the contrary, would have required a planner to estimate the target load for such a bridge. There is only one scenario in which the bridge does not collapse and does not have a threat model. The expert builder looks at the construction and says “hey boss, this will collapse, we need stronger beams”. This however, requires an expert builder working on each key part of the construction, which usually is not the case. Also, the fact that each part works properly does not mean that the full integration of all parts will work properly. 

Software development security

The difference between building a bridge and releasing software is that you cannot put a ‘no entry for hackers’ sign in front of your application. And of course, similarly, re-designing software or building new mitigations at the end of the product development process is much more expensive than planning them from the very beginning. Therefore, threat modeling is not only an option, but a requirement for enterprise software development. What would that look like?

The bigger question is how to integrate threat modeling into the existing processes – waterfall or agile. And even a bigger one, how to do it at scale for tens or hundreds of projects. As we now have a business reason for security in the software development project, the further part will describe how to integrate it into the current processes and orchestrate a change in the corporate practices.

Introducing another S to SDLC – Secure

Can you build secure software without formal threat modeling? Sure you can, but it is hard. Imagine a standard software development with just one modification – a security test (penetration test) at the end of the development. Using my bridge analogy you hire an external company to test your ‘bridge’. The problem is that either:

• The breakers are better at breaking than builders are at building,
• or the opposite.
  

In the first case, usually, the builders are unaware of what will happen during the tests. The bridge (or software) will collapse, and it requires a lot of effort to rebuild it in a way it will survive the same tests again. You can add ‘patches’ to the bridge but at some point maintenance of the patches itself will be a huge cost.

In the second case, the bridge will stay solid, but there is always an uncertainty of how the trusted breakers (testers) compare to the real-world breakers (hackers). Therefore, in both cases, just a penetration testing requirement does not solve the real issue – answering how secure the software is and if it would survive the attacks.

How to measure software security?

In measuring software security, we want to answer the following questions: 

• Who are the potential attackers?
• What do they want to achieve?
• How can they achieve it? 

Only then we can decide whether a potential threat is something we are really afraid of, and where to implement necessary mitigations. Afterall, the penetration test then finally gets another meaning and answers the question “how many of the potential attacks were possible?”.

Example modifications of SDLC process to support the Secure part would be to include :

• Threat modeling during the design phase.
• Risk analysis and decision to accept or mitigate potential threats.
• Definition of security requirements (within the existing non-functional requirements).
• Implementation of mitigations supporting the requirements.
• Designing the security test scenarios.
• Confirming the assumptions with a penetration test.
• Regularly update the threat model with each software change and with each threat landscape change (e.g. new exploits).

Secure By Design

With such a design, it is quite easy to orchestrate an organisational process. Nothing changes in the development process itself, we just need to add a few checkpoints and make sure that everything happens in the right order. Starting with the last link in the chain (software release), we can see that each phase requires the previous one and in the result, we need to have a threat model in the very beginning:

Once we have a high-level policy and process, let’s get into implementation which will be the topic of the next article. 

Quick Summary

I hope you got a better understanding of threat modeling and why it is worth implementing in your organization. You can watch example threat modeling sessions on our YT channel where we regularly publish the Instant Threat Modeling series. 

Keep in mind that threat modeling is a relatively new approach to IT security. By putting security issues at the very beginning of the development process, you can avoid mistakes that require much more work in the future. 

Based on many years of experience, I believe that such an approach is really a way to go and achieves optimal results in combination with security tests that verify your assumptions at the end. This way you can double-check and be more sure about our security. Otherwise: accept the risk of not knowing.

Jakub Kałużny Principal IT Security Consultant
Head of Secure Development