This post discusses new algorithms we at SIDN Labs, in collaboration with NBIP, have developed to classify the traffic associated with multi-vector Distributed Denial of Service (DDoS) attacks. We have implemented the algorithms in the DDoS Clearing House, a system that enables operations teams across different organizations to collaboratively prepare for DDoS attacks by automatically sharing DDoS characteristics in the form of so-called ‘DDoS fingerprints’. We’re developing the DDoS Clearing House as part of the CONCORDIA project and it will be used by the Dutch National Anti-DDoS Coalition.
Capturing DDoS characteristics with the Dissector
Our DDoS classification algorithms are part of the Dissector, a core component of the DDoS Clearing House. The Dissector’s task is to process DDoS network traffic (for instance, in the form of a PCAP file), identify attack characteristics (such as protocols, ports, source IPs, and packet payload lengths), and generate a fingerprint that captures these characteristics. The Dissector is modular in the sense that it supports different classification heuristics, each capable of generating a fingerprint for a specific type of attack. Figure 1 depicts the overall design of the Dissector. It receives DDoS network traffic as an input — either packet data or flow data — and provides as output, a fingerprint file in JSON format. The network traffic is typically a sample of the total amount of traffic involved in a DDoS attack, allowing the Dissector to generate fingerprints quickly with relatively limited resources (in terms of memory and CPU requirements).
The processing pipeline of the Dissector consists of two components:
- Attack profiler: Creates profiles from the sampled attack traffic. It determines the protocols and services involved in the attack and identifies a set of attributes relevant for classifying the attack. Attributes are either types of packet header (such as HTTP headers, DNS requests, packet payloads, source, and destination IPs) or ‘higher-level’ attributes such as the suspected nature of the attack (such as amplification or ransomware-based DDoS).
- Attack classifier: Calculates the values for each of the attributes in the profile, aiming to identify patterns and similarities. Our classifiers use simple heuristics based on the number of occurrences combined with more elaborate algorithms to determine the values of higher-level attributes (such as a classifier that can assess whether an attack is an amplification attack).
To develop and improve our implementation of the Dissector, we rely mainly on PCAPs of real attacks kindly provided by NBIP, one of the partners in the Dutch Anti-DDoS Coalition. NBIP runs the Dutch National DDoS Scrubber (NaWas), which handled 1,610 DDoS attacks last year. Clients that are under attack send their incoming traffic to the NaWas, which removes (‘scrubs’) the DDoS traffic and sends the clean traffic back to the client over a separate VLAN.
Example: Multi-vector attack
The use of multiple attack vectors to perform DDoS attacks is becoming popular. Attackers are targeting different services and mixing protocols (such as DNS, NTP, LDAP and SNMP) to overload the target. Using multiple attack vectors makes the mitigation process more complex for operations teams and, as a result, increases the chances of success for attackers.
Let’s use, as an example, a DDoS amplification attack that uses the LDAP and DNS protocols. This is a real attack that NBIP handled. It took place on 30 September 2020 and involved 690 source IPs. The attackers used legitimate LDAP and DNS servers to amplify the volume of traffic targeting the victim. The use of legitimate servers makes the attack even more difficult to handle because the plain filtering of source IP addresses could also block legitimate user requests.
The Dissector analyses both attack vectors (LDAP and DNS) in order to identify packet characteristics that operations teams could use to filter that traffic. The Dissector accomplishes this by first splitting the DDoS traffic into two vectors (DNS and LDAP attack vectors) and then further analyses each vector to identify packet values that could identify the DDoS attack. Figure 2 shows this process in more detail.
In Figure 2, our classifier has identified a set of attributes and their values that together represent the LDAP-DNS amplification attack. For the DNS vector, the Dissector identified two specific queries evil.com and bad.com, and other parameters (packet length, source IP address and IP TTL). Likewise, for LDAP we’ve got a set of IPs and a specific packet length. This set of characteristics is essential for generating the attack fingerprint.
DDoS classifiers
We implemented the attack classifier using lightweight statistical methods to keep the Dissector software simple and less dependable on external resources. We used z-score and frequency analysis. Z-score is a statistical value that represents the number of standard deviations from the mean, making it useful for finding outliers. Combining z-score and frequencies (number of packets with certain protocol fields) have been shown to be sufficient for building fingerprints of the DDoS attacks we have evaluated so far. The Dissector does not yet implement a classifier for every vector and attack type. So, when the Dissector does not have a classifier for a specific attack, it runs a generic heuristic-based default classifier on the number of occurrences of packet headers. For our dataset, this approach secured good results in most cases. However, we can improve the fingerprint generation process by using DDoS type-specific classifiers that consider the protocol fields and attack type (amplification, ransom-based DDoS, and flood).
We recently extended the Dissector by adding new classifiers, as shown in Table 1. For example, the LDAP Amplification with Fragmentation classifier assesses whether an attack vector is of this type by analysing the source addresses of LDAP traffic in a PCAP file and calculating what percentage of the packets are fragmented. This enables us to correlate different attacks and attack vectors. For example: is the same set of source IP addresses performing LDAP and IP fragmentation attacks?
DDoS attack classifiers implemented:
- DNS Amplification
- Multiprotocol Amplification
- LDAP Amplification
- GRE Flood
- DNS Water Torture
- SNMP Amplification
- Multi-protocol Flood
- LDAP Amplification with Fragmentation
New types of DDoS attack appear all the time, and so it is crucial to continually develop new classifiers. Existing classifiers need to be improved regularly as well, so that the fingerprints they generate are as specific as possible, to avoid false positives.
Fingerprint generation
The Dissector captures the output of the classifiers in a JSON-formatted fingerprint file. It also adds a set of metadata fields to the fingerprint file, such as generic statistics regarding the attack (average bits per second, number of packets, total number of IP addresses). The Dissector’s classification algorithms also add a tag that labels the attack, such as ‘Amplification’, ‘NTP’, ‘Volumetric Attack’, or ‘Layer 7 Attack’. Figure 3 shows a fragment of a fingerprint generated from a DNS-based DDoS attack.
{
"attack_vector": [
{
"src_ips": [
"ommited"
],
"attack_vector_key": "66f2e83fde0e6351d3f5ad967c6230aa3b60dbc498ad13b074296cb5f84c7734",
"one_line_fingerprint": "{'dns_qry_type': 1, 'ip_proto': 'UDP', 'highest_protocol': 'DNS', 'dns_qry_name': 'a.packetdevil.com', 'frame_len': 1514, 'udp_length': 4103, 'srcport': 53, 'fragmentation': True, 'src_ips': 'omitted'}"
}
],
"start_time": "2013-08-14 23:04:00",
"duration_sec": 0.16,
"total_dst_ports": 4649,
"avg_bps": 143426993,
"total_packets": 16471,
"ddos_attack_key": "44518107642b9ac7098174a16cbf220395c862bf26389c734e0b109b318e9291",
"key": "44518107642b9ac",
"total_ips": 2065,
"file_type": "PCAP",
"tags": [
"AMPLIFICATION",
"DNS_QUERY",
"FRAGMENTATION",
"SINGLE_VECTOR_ATTACK",
"UDP_SUSPECT_LENGTH",
"DNS"
]
}Fingerprint sample of a DNS-based DDoS attack (JSON).
The fingerprint format is flexible. We have defined the mandatory fields in the schema (protocols and source IP addresses) and other information can easily be added. This is important because new attack types may require fields that are not present in the current fingerprint schema. This flexibility is also important for developing mitigation rules (such as a snort or IP tables rule that filters out specific address ranges), since the DDoS Clearing House also contains the Converter, which automatically creates mitigation rules based on fingerprints.
Next steps
Our Dissector is able to process multiple DDoS attack types because of its modular architecture, which can be flexibly extended by adding new attack classifiers. This is especially important for dealing with DDoS attacks of the kind that have recently become popular, where multiple attack vectors are used, and for adapting to the continually changing DDoS attack landscape. Moreover, the classifiers can be improved in isolation without affecting the other core software components of the DDoS Clearing House.
Our next aim is to develop more challenging classifiers, for example for spoofed attacks and DNS water torture attacks. Furthermore, we will be continually improving our existing classifiers to reduce the probability of false positives and maximize the accuracy of fingerprints.
We’ll also be deploying the Dissector in the networks of a few of the Dutch Anti-DDoS Coalition’s members in the next few months as a pilot.
This work was partly funded through the PAADDoS and CONCORDIA projects. Visit the project website.
Pim van Stam (NBIP), Gerald Schaapman (NBIP), Cristian Hesselman (SIDN Labs) contributed to this work.
This post was originally published at SIDN Labs Blog.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.