As a Security Engineer in the NDR Development team, I work on creating and maintaining the service for detecting network intrusions and stopping them before they do damage.

In this article, I’ll take you behind the scenes of how we approached building our service from defining what “good” prevention looks like, to solving the engineering challenges of making it fast, accurate, and reliable in the real world.

The best incident is the one that never happens

Running Security Operations means spending countless hours chasing alerts, investigating incidents, and cleaning up after attacks. So my team’s mindset is to stop threats before they even land in a system. Which is why the Network Detection and Response (NDR) service contains:

• Intrusion Detection System (IDS) components that detect an intrusion in the network. For this, a signature-based system called SNORT is used. It performs real-time traffic analysis and packet captureon IP networks. (A signature is a string, for example a word, that can match a network packet.)
• Intrusion Prevention System (IPS) components that prevent the detected intrusion from harming the network and the assets in it. The IPS is built in-house with professional third-party data sources to ascertain whether traffic is malicious. Zeek is the open-source software that drives it and which provides deep insights into network activity. It operates on a sensor and analyzes network traffic in real time. Zeek captures high-fidelity transaction logs, file content and customizable data outputs.

IDS and IPS each serve a distinct but complementary role in defending network environments. An IDS continuously monitors network traffic to identify suspicious activity or known malicious patterns, and marks alerts for further investigation, but does not take action. In contrast, an IPS picks up alerts and operates in line with network traffic, not only detecting threats but also actively enforcing security policies by blocking or dropping malicious packets in real time.

In other words, IDS provides visibility and forensic insight into potential intrusions, while IPS adds an enforcement layer that can disrupt attacks as they occur. Together, IDS and IPS form the analytical and preventive backbone of NDR, where we need to balance detection fidelity and response speed to minimize risk and operational impact.

The NDR service monitors the network in order to match indicators of compromise (IoCs) on traffic. For example, these can be:

• IP address
• URL or domain name
• File hash, also known as a cryptographic hash: A fixed-size string of alphanumeric characters that uniquely identifies files.
• Certificate fingerprint: This is a unique identifier for a digital certificate which proves that a valid public key belongs to the identity of the holder.
• Self-signed certificates: These digital certificates are not issued by a certificate authority (CA) and are easy to make. They do not provide any trust value. It’s like when the person creating a corporate budget is also the one approving it.

To illustrate how traffic is monitored, I sometimes use an analogy of physical package traffic at a post office.

Suppose a postal worker sees a random package and needs to assess if it contains a weapon based on some “signature criteria” which is really just a description of something suspicious. The process goes something like this:

• First check if the sender’s address belongs to a registered store or supplier.
• Then check whether the package is small and heavy.
• Thirdly, x-ray the package to see if it shows the outline of a shape resembling a weapon.

If all three checks match, the postal worker decides to conclude that this package is suspicious. And it travels no further.

In networking terms, such suspicious traffic will be blocked.

Of false positives and IoCs

A typical challenge in network security is a high false-positive rate. This can lead to blocking legitimate business traffic when there is really no need.

My team’s goal is to get the IoCs as precise as possible. We call IoCs context-driven “intelligence data” that covers a broader area than signatures do. We only want to block the traffic when there is a high degree of confidence that it is malicious. To use my postal package analogy, “Just because the outline looks like a weapon, doesn’t mean it actually is.”

For example, certain traffic might be legitimate if a customer has a firewall solution that potentially produces a traffic pattern which the NDR service has deemed suspicious.

We want to avoid false positives, and it’s not trivial – the main characteristics of IoCs being that they change often and quickly. A malicious domain may be registered in the morning, weaponized by noon, blocked by evening, and abandoned the next day. That’s why the NDR sensors are set up to use the latest available IoCs from professional third-party feeds at any moment in time. The IPS pulls these IoCs and matches them to the network traffic. A further component takes NDR rules into consideration. They are the core of the IPS and they specify which network traffic to block, for example: “If XYZ certificate fingerprint matches XZY in a network packet, then block the destination IP address.” The connection is interrupted and stops causing damage.

An IPS is highly configurable. It is flexible and adjusts itself to customer environments. You can decide to block the traffic for a certain length of time, or you can choose to block only the source or only the destination IP. Or you can define that a Mission Control ticket is created with high severity and that a notification is sent to your key IT admin personnel. You can also have it create tickets without blocking the traffic. It is really up to your company policy and what is more effective for your particular business environment.

The Technical Account Managers at Open Systems work with company admins to decide questions like:

• If the source of the network traffic is external and so is its destination, what do you want to do? Do you care? After all, it’s external. So do you let it pass through? Or do you block?
• If the suspicious traffic originates from your network behind the Security Gateway and is going to the internet, do you assume that it is a malicious connection and block it?

Three core requirements for an IPS

I think it’s unique that developers at Open Systems not only design systems but also operate them for customers as part of the Mission Control operations model. That way, we have a “front-row view” of what’s happening in the field and can take action. We see that modern threats are getting trickier and they change very, very fast. On top of that, the sheer volume and variety of alerts can make it difficult to separate real threats from noise. It’s precisely the challenge of achieving high-fidelity detection without drowning in false positives that guided how we designed our system.

We defined three requirements for the IPS:

• Fast: Threats move in milliseconds, so the IPS must detect and block malicious traffic instantly, preventing the spread of potential compromise. Connections are usually initiated very quickly, and data transfers happen rapidly these days. The sooner malicious traffic is recognized and acted upon, the less damage it can do. Also, being aware of potential bottlenecks is necessary to ensure that the IPS remains fast.
• Accurate: The rate of false positives must be very low. The IPS must reliably distinguish real attacks from legitimate business traffic, ensuring that normal operations aren’t disrupted. A false positive can be very frustrating for customers when legitimate traffic is blocked, because suddenly something no longer works and it is very difficult to debug.
• Reliable: A misconfigured drop rule can bring down an entire service. Our IPS must maintain consistent performance and stability, even under heavy traffic or during a complex attack. The IPS must also provide the option to unblock a false-positively blocked connection as soon as possible.

Complementary security

Several customers asked us if we were aware of a specific threat. They needed to report to their SOC department of the guardrails that were in place for it. There was, indeed, data in the IoCs that would have detected such a threat if that customer had been under attack. It’s a bit like a general practice doctor asking a specialist for an opinion.

Sometimes customers cannot, or don’t want to, use a Secure Web Gateway (also known as a proxy), for example when apps don’t support it. In such situations, it is impossible for them to monitor or know which websites their internal network is connecting to. The NDR service provides an alternative solution in that case, as well as an additional level of security to those who have a proxy in place. The subtle difference is that the IPS does not block a connection before it happens – it blocks it as it is happening, right upon detection – and can therefore match on more elements than IP, URL, or domain.

Then there are situations where edge devices are not technically capable of using a Secure Web Gateway, for example robots in operational technology. But a robot might occasionally need to have its drivers updated and for that, it would need to initiate an external connection. When Technical Account Managers are given a new customer environment to review, they don’t yet know much about it and must take into consideration that robots may have been (unknowingly) delivered with malware.

As the IPS is capable of also monitoring the internal network traffic, it can be considered as complementary to the Secure Web Gateway, like an extra layer of security that goes hand in hand with it. In the long term, the NDR service is set to detect IoCs that go beyond the proxy.

Challenges we overcame

We’ve had to deal with three complex constructs:

• To build a framework for the configuration of NDR rules – that type of thing gets difficult really fast. Now we have built-in support for it, and we can also use several firewall objects such as networks or IP groups in the NDR rules if we so choose.
  Here we collaborated with the SD-WAN Cloud team and had discussions about defining strategies that would most help our customers.
• To update the IoCs and distribute them globally for the Open Systems Security Gateways to use. Here my team is constantly monitoring and optimizing the process in the background.
• To truly understand the software we use for pattern matching and how it needs to be configured to be effective, and to run it reliably. This is a hands-on continuous development approach combined with a learning mindset.

We’ve experienced that the NDR service gives greater security because it can run in combination with other services such as Network Analytics, Secure Web Gateway, and Firewall. It can also be helpful with incident handling and post-incident analysis and forensics.

On the horizon

One area I’m particularly excited about is improving transparency around blocked events, through clearer logging and better UI, so that prevention is not only effective, but also understandable. And of course, I’m looking forward to that moment in testing: connecting to a malicious domain, watching the IPS react instantly, and knowing the attack was stopped before it had a chance to begin. By the way, we’ve had some very promising results so far in real-life customer environments.

We are extending the service to include malware scanning. The idea is to enable file scanning on internal or external traffic. For example, if someone sends a file to a robot, then that file can be blocked if it is determined to be malicious.

It will also become easier for customers to more precisely configure the filtering in one place. They know their environment best and will have even more options to filter out cases that they know are benign. Such fine-grained NDR rules are usable by several NDR components whose main task is to filter. Configuring NDR rules is designed to be similar to the way firewall rules are set up, so that it is easier to understand how they work and save time by being consistent. I can definitely say we put in the hard work to make things simple for customers.

By being able to include IoCs that detect truly malicious traffic, we can prevent malware and other cyberattacks from making a huge business impact on our customers.

Because no matter how strong your analytics are, a threat prevented is a threat you never have to respond to.