Advanced security (DPI, IDS/IPS)
Defense in depth: what a firewall can’t see, stateful inspection, DPI and TLS, IDS vs IPS, real attacks (port scan, SQLi, C2) and layered defense.
Lesson 1: What a basic firewall can’t see
A firewall sees only addresses and ports
In the Network security course we built a firewall that decides by zones, IP addresses and ports (layers L3/L4). That stops a lot – but the firewall can’t see inside the packet. It knows "TCP to port 443 from outside to the DMZ", but not WHAT is in that traffic. And the content is exactly where most attacks hide today. This course adds a layer that looks deeper: stateful inspection, DPI and IDS/IPS.
Step by step
- The firewall sees "TCP 443 from the internet to the DMZ" – addresses and a port. By those it permits or drops.
- But what’s inside (which HTTP request, what data) the firewall can’t see. An attack hidden in an allowed port slips through.
- So we add an element that looks into the content – DPI and IDS/IPS. That’s this whole course.
A packet has content too
Every packet has two parts: headers (who to whom, which port – that’s what the firewall sees) and the payload (the actual data – the request content, a chunk of a file, a command). An attack is rarely recognizable from the header; it’s recognizable from the content: the payload ' OR '1'='1 is SQL injection (SQL = the query language for databases; this trick fools the query into returning everything or letting you in without a password), the string /etc/passwd is an attempt to read system files, a certain byte pattern is known malware. To catch attacks like that you must read the payload – and that’s what DPI does.
Step by step
- Header: "TCP 443 to the DMZ". The firewall reads it and permits – looks harmless.
- But the payload carries SQL injection. From the header alone you can’t tell – you must read the content.
- DPI peeks into the payload and compares it against attack patterns (signatures). That’s how it spots an attack inside an allowed port.
Tunneling in an allowed port
Attackers know the firewall allows a few ports (80, 443, 53…). So they hide their traffic in exactly those – this is called tunneling. Malware can talk to its command server (C2) over DNS queries or wrap data in HTTPS that looks like ordinary browsing. To the firewall (L3/L4) it’s "allowed port 443", and it lets it through. Spotting it is only possible from behavior and content – another reason for deep inspection and IDS/IPS.
Step by step
- An infected PC inside wants out to its C2 server. A direct channel the firewall would cut.
- So it hides in DNS queries or HTTPS. The firewall sees only "an allowed port" and lets it through.
- Spotting it needs deep inspection and behavior (oddly many DNS queries, a suspicious pattern). Hence IDS/IPS.
Lesson 2: Stateful inspection in depth
The connection table
Back to the stateful firewall from the Network security course – but deeper. A stateful firewall keeps a connection table: for each established connection it remembers who started it, to which port, and what state it’s in. Thanks to that it recognizes the reply to an allowed connection and lets it back automatically – without you writing a rule for the return direction. And conversely: a packet that doesn’t belong to any entry in the table is suspicious.
Step by step
- A PC inside opens a connection out (e.g. web on 443). The firewall records it in the table.
- The server’s reply is found in the table → the firewall lets it back automatically, no manual return rule.
- A packet that matches nothing in the table (unsolicited from outside) is suspicious and gets dropped.
TCP states and the handshake
To know a connection’s "state", the firewall watches the TCP handshake (covered in the ISO/OSI and TCP/IP course). A new TCP connection starts with the trio SYN → SYN-ACK → ACK (SYN = synchronize, ACK = acknowledge; like "I call – you answer – I confirm"). Only after it is the connection established. A stateful firewall watches this: a packet in the middle that pretends to be an established connection without a handshake is a fake. Tracking state is the foundation that attack detection like SYN flood builds on.
Step by step
- The client sends SYN – "I want a connection". The firewall notes a half-open connection.
- The server replies SYN-ACK, the client confirms ACK. The three-way setup is done.
- Now the connection is established. A packet without this handshake posing as established the firewall spots.
SYN flood
Now an attack that abuses state: SYN flood. The attacker sends a flood of SYN packets but never completes the handshake (no ACK). The server holds many half-open connections, waits for confirmation, its queue of half-open connections fills up and it stops accepting legitimate clients – that’s a DoS (denial of service). A stateful firewall / IPS spots it by the pattern (unusually many SYNs without a completed handshake) and starts dropping or rate-limiting them. The server can also defend itself with SYN cookies: it keeps no state for the half-open connection at all – it encrypts the needed info straight into the number it has to send anyway, and reads it back from the client’s reply.
Step by step
- The attacker spews SYN after SYN… but never sends ACK. The connections stay half-open.
- The server runs out of its half-open table and stops accepting even legitimate clients. That’s a DoS.
- The IPS spots the pattern (too many SYN without ACK) and starts dropping/limiting them. The server keeps breathing.
Lesson 3: DPI – deep inspection
What DPI is
DPI (Deep Packet Inspection) = deep inspection: the device unwraps the packet down to the payload and reads the content too, not just headers. Instead of "port 443 allowed" it asks "what exactly is that HTTP request doing?". This catches attacks hidden in allowed traffic (SQL injection, file reads, known malware) and recognizes the application regardless of port. The price is performance (reading content costs more than just a header) and with encrypted traffic a problem – next module.
Step by step
- Traffic reaches the element with DPI. It doesn’t just pass it – first it looks inside.
- DPI unwraps the payload and compares it against signatures (attack patterns). It looks for "/etc/passwd", SQLi patterns, malware.
- On a match it flags (IDS) or blocks (IPS). In the simulator you write your own signature in the ips-custom task.
Application signatures
How does DPI recognize an attack? By signatures – exact patterns in the content. A signature can be a string (the payload contains /etc/passwd), a regular expression (an SQL injection pattern) or a fingerprint of known malware. Vendors ship large signature sets that update continuously (new threats arrive daily). The strength of signatures is precision on known attacks; the weakness is that a new, unknown attack with no signature slips through – which is why anomaly detection is added (next lesson).
Step by step
- A signature = a pattern: the string "/etc/passwd", a regex for SQL injection, a malware fingerprint.
- When the payload matches a pattern, it’s a hit = a known attack. DPI catches it.
- Weakness: a new attack with no signature matches nothing → it slips by. Anomaly detection fills that gap.
TLS inspection and MITM
A big catch: most traffic today is encrypted (HTTPS/TLS). But DPI needs to see the content – and it’s hidden inside the encryption. The enterprise solution is TLS inspection: the security device acts as a controlled man-in-the-middle (MITM) – unlike an attacker, though, the company does this on its own device, with your consent (in the employment/policy terms) and its own certificate, which is what makes it legitimate rather than an attack. It decrypts the connection, inspects it, and re-encrypts it onward. It only works because the company installs its own root (CA) certificate on its devices – with it the device can mint a certificate for each site that the endpoints trust (because that pre-installed company certificate signed it). It’s powerful but sensitive: it touches privacy, is demanding, and done badly it weakens security. So it’s deployed thoughtfully. Even without decrypting, the network still sees at least metadata – e.g. the server name and the volume of data.
Step by step
- The traffic is encrypted. DPI would love to read the content, but sees only ciphertext. Now what?
- TLS inspection: the device decrypts (as a controlled MITM), inspects and re-encrypts. It works thanks to the company certificate on the devices.
- Powerful but sensitive: it touches privacy, is demanding, and done badly weakens security. Deploy thoughtfully.
Lesson 4: IDS – detection
IDS as a camera
IDS (Intrusion Detection System) = intrusion detection. Think of it as a security camera: it sits off the traffic path (gets a copy via TAP/SPAN), watches the traffic, and when it sees an attack it raises an alarm (alert) – but doesn’t intervene, doesn’t block traffic. Upside: it slows nothing and never blocks anything by mistake. Downside: it only records the attack, doesn’t stop it – someone has to see it and react. In the simulator you switch the sensor to IDS mode and see "ALERT" without blocking.
Step by step
- The IDS gets only a copy of the traffic (TAP/SPAN). Traffic flows on with or without it.
- It sees an attack → raises an alarm (ALERT to the central log). But it doesn’t stop the traffic – only records it.
- The attack still reached the target. IDS is a witness, not a guard. If no one acts on the alert, nothing’s solved.
Signatures vs anomalies
IDS/IPS recognizes threats in two ways. Signature-based (by patterns) is precise on known attacks but misses unknown ones. Anomaly-based first learns what normal traffic looks like (a baseline) and then reports deviations – say a server that suddenly sends gigabytes out at night, or ten times the usual DNS queries. Upside: it catches new attacks with no signature. Downside: more false alarms (unusual ≠ always bad). In practice the two approaches are combined.
Step by step
- Signature-based: the payload matches a known pattern → a sure hit. But it won’t catch an unknown attack.
- Anomaly-based: first it learns the normal traffic (a baseline).
- Then it reports deviations – a sudden data exfil, oddly many DNS. It catches new attacks, but at the cost of false alarms. Hence the approaches are combined.
Alert fatigue
A big practical problem with detection: alert fatigue. When a system spews thousands of alerts a day and most are false alarms, the team stops taking them seriously – and the one real attack drowns in the noise. So alerts are tuned (reduce noise), triaged by severity and funneled into a SIEM, where they’re correlated. Detection without response is useless – and you can’t respond if you can’t make sense of the alerts. Now in the ids-to-ips task you’ll try the difference between "just alert" and "block outright".
Step by step
- Detection spews thousands of alerts, most false. The team stops watching them.
- And in that noise the one real attack drowns. That’s alert fatigue.
- The fix: tune alerts, triage, correlate in a SIEM – and let critical things block outright (IPS). You’ll try that in the ids-to-ips task.
Lesson 5: IPS – prevention
IPS as a guard (inline)
IPS (Intrusion Prevention System) = intrusion prevention. Unlike IDS (a camera off the path), an IPS sits directly in the traffic path (inline) – it’s a guard everything must pass through. When it sees an attack, it doesn’t stop at an alarm – it drops/blocks it outright, so it never reaches the target. The price: because it’s in the path, its failure or slowness affects traffic, and a false alarm blocks something legitimate. So an IPS is carefully tuned. In the simulator the sensor in IPS mode is exactly this – inline blocking.
Step by step
- The IPS sits directly in the path. All traffic must pass through it – it’s a guard, not a camera.
- It sees an attack → drops it outright. It never reaches the web server. That’s the difference from IDS.
- The target stayed clean. The cost: an IPS in the path – its failure or a false alarm affects traffic. So it’s tuned.
Detect vs block and tuning
The key decision: detect or block? IDS (detection) is safe for traffic but won’t stop the attack. IPS (prevention) stops the attack, but a false alarm hurts – it blocks something legitimate. So a new signature is often first deployed in alert-only mode, watched to see if it hits benign traffic, and only once tuned switched to blocking. Good tuning balances false positives (blocking the good) and false negatives (letting the bad through). That’s the security team’s daily work.
Step by step
- You first deploy a new signature as alert only (like IDS) and watch what it hits.
- Is it hitting legitimate traffic (false positives)? You tune it to be precise.
- Only once tuned do you switch it to blocking (IPS). A balance between "block the good" and "let the bad through".
A custom signature
Sometimes no off-the-shelf signature catches an attack – say a targeted attack on your application. Then you write a custom DPI signature: a pattern (string or regex) searched in the payload. The key is to write it so it catches the attack but doesn’t block benign traffic (no false alarms). You’ll try this now in the ips-custom task (advanced mode): a payload with /etc/passwd targets the web, no category catches it – you write a signature that catches that string and verify a benign request passes.
Step by step
- A targeted attack with /etc/passwd hits the web. No off-the-shelf category catches it.
- In the IPS (advanced mode) you write a custom signature for that string.
- The attack is blocked, a benign request (GET /index.html) passes. That’s exactly the ips-custom task.
Lesson 6: Real attacks
Port scan (recon)
Real attacks rarely start "head-on". First comes reconnaissance – typically a port scan: the attacker systematically tries which ports are open on the target to find a way in. From the network’s view it looks like many connections to different ports from one address in a short time – a distinct pattern that IDS/IPS recognizes. A port scan itself does no harm, but it’s a precursor: someone is mapping your surface before attacking.
Step by step
- The attacker tries port after port – 22? 80? 443? 3389? Looking for a way in.
- Many connections to different ports from one address = a clear port-scan pattern. IDS/IPS spots it.
- The scan itself is harmless, but it’s a precursor. You can watch the source or block it before it attacks.
SQLi over HTTPS and lateral movement
When the attacker finds a web app, they try to break it – e.g. SQL injection (SQLi): slipping a piece of SQL (' OR '1'='1) into a form/URL to pull data from the database or bypass login. And because the web runs over HTTPS, the payload is encrypted – an ordinary firewall doesn’t see it; only DPI after TLS inspection catches it (the simulator simplifies encryption – the IPS sees the payload directly; in reality TLS inspection would have to sit in front of it). Once the attacker lands on one server, they try lateral movement: spreading from it to other machines on the network. Here segmentation (VLANs/zones from the Switching and VLANs and Network security courses) pays off – it limits how far they get.
Step by step
- The attacker slips SQL injection to the web server. Over HTTPS the payload is encrypted – only DPI after TLS inspection catches it (simplified in the simulator: the IPS sees the payload directly).
- If it works, they try lateral movement: spreading from the web to other servers and hosts.
- Segmentation (VLANs/zones) and IPS limit the attack’s reach – it can’t get everywhere. Defense is layered.
C2 over DNS
The last piece of the chain: the compromised host needs to talk to the attacker (download more tools, send stolen data). To hide that channel from the firewall, it tucks it into a commonly allowed protocol – often DNS (few block DNS) or HTTPS. Data is chopped into DNS queries for odd subdomains. The firewall sees "normal DNS" and lets it pass. It’s revealed mainly by behavior (anomaly: unusually many DNS, long random names) and DPI over DNS – exactly what anomaly-based detection and deep inspection do.
Step by step
- The compromised PC sends data out – hidden in DNS queries to odd subdomains.
- The firewall sees only allowed DNS and lets it pass. A classic L3/L4 firewall doesn’t spot it.
- Behavior reveals it: unusually many DNS, long random names. That’s the job of anomaly detection and DPI. You can now try stopping one of this lesson’s attacks yourself – SQLi against a web server – in the task below.
Lesson 7: Layered defense
Defense in depth
No single layer is enough. Security is defense in depth: if one layer fails, the next catches it. An attack from the internet hits, in order, the firewall (zones, ports – the Network security course), the IDS/IPS with DPI (content – this course), inside it segmentation by VLANs and zones (the Switching and VLANs and Network security courses), ACLs between networks (the same courses) and finally the servers’ own hardening. No wall is perfect – but stacked layers make an attack far more expensive and noisy.
Step by step
- Layer 1: the firewall filters by zones and ports. What passes moves on.
- Layer 2: IDS/IPS with DPI checks content and catches an attack hidden in an allowed port.
- Layer 3+: segmentation, ACLs, server hardening. If one layer fails, the next catches it. That’s defense in depth.
The complete C3–C8 network
Let’s tie it together across all the courses. We’ve built: VLANs and switching (C3), routing and IP (C4), firewall and zones (C5), redundancy and STP (C6), OSPF (C7) and now deep security (C8). Together they make a real enterprise network: divided (VLANs), reliably connected (routing/OSPF), resilient to failure (redundancy) and protected in multiple layers (firewall + IDS/IPS + segmentation). Each course was one layer of this puzzle.
Step by step
- Foundation: a divided network (C3), connected by routing (C4), protected by a firewall (C5).
- Reliability: redundancy and STP (C6) and dynamic OSPF routing (C7). The network survives failures.
- And now layered defense (C8): content, IDS/IPS, real attacks. A complete enterprise network together.
What’s next
You’ve managed to build and secure a real network from the ground up. Where do the next steps lead? Security is a never-ending field: SIEM and a SOC (central event collection and analysis, a team that responds), Zero Trust ("trust nothing, verify everything" – the end of trust based on network location), cloud and SD-WAN (network and security as software across sites), and automation/threat intelligence. That’s beyond this simulator – but thanks to courses C1–C8 you now understand these topics in context. Congratulations.
Step by step
- A built, connected, resilient and multi-layer protected network. That’s the core of networking.
- Next come SIEM/SOC, Zero Trust, cloud and SD-WAN, automation. A never-ending field.
- These topics are beyond the simulator – but thanks to C1–C8 you now understand them in context. Congratulations!