Like all critical infrastructure sectors, the energy sector has become reliant on automation techniques to monitor and control its networks. However, not much is publicly known about these techniques or the networks that tend to use proprietary protocols and operate in closed settings.
To address this knowledge gap, we at the Brandenburg University of Technology, recently conducted a project to study the design of industrial control networks that run power plants. Below are some of our key takeaways that we presented recently at PAM 2022.
Industrial control networks 101
Industrial control systems can be broken down into Supervisory Control and Data Acquisition (SCADA) and Distributed Control Systems (DCS).
While SCADA networks control the interaction of distributed assets (for example, to enable power and water supply), DCS is used to control local core processes in power plants. In this post, we will focus on DCS.
While such networks differ in their physical extension and associated network characteristics, they are quite similar in terms of network organization. Yet, the concrete network design including its configuration (for example, addressing schemes) is vendor-specific.
Typically they use a layered design as shown in Figure 1:
There are multiple layers, each with a ring topology using switches. For robustness, the network is statically configured and thus DHCP or DNS are not needed.
The lowest network part, the Field Network, consists of the physical level, including sensors and actuators measuring and adjusting physical parameters (for example, temperature and pressure). They are connected to programmable logic controllers (PLCs). Each PLC controls actuators and/or monitors sensors to realize a low-level application-specific control loop.
In the Control Network, inputs from various PLCs are collected and evaluated by subsystem-specific servers to aggregate all activities corresponding to a subprocess. For power plants, an example subprocess is monitoring the boiler temperature.
Automation servers (Auto) enable clients to automate control procedures. Migration servers (Mig) aggregate data from the lower-layer field network to the higher layer Control Network. Application servers prepare the process data for visualization on the Human Machine Interfaces (HMI).
Monitoring and manual adjustment of the physical process are realized in the Supervisory Network. Here the operator interacts with the subsystems via HMI that incorporate the data from the Control Network.
Another characteristic is that these networks are rarely upgraded. In the case of power plants, operators only plan for one network upgrade during the entire lifetime; once installed, networks run for decades without major changes. This is in stark contrast to Internet networks that are upgraded frequently.
What kind of traffic do these networks carry?
To study this, we captured traffic traces at three operational power plants. Let’s dive into the application mix of each network level starting at the bottom, the Field Network.
We refer to an ‘application protocol’ as a protocol used to transmit application payload, regardless of the transport protocol. At the field layer, we saw only PLC to PLC communication over one single protocol dissected by Wireshark as COTP. This is the typical understanding of an industrial network — only a single protocol is used. Yet, this assumption of a single protocol does not reflect all industrial networks.
In the case of power plants, we found a rich application mix that differs by layer (Figure 2). Beyond the field level, we found a jungle of binary proprietary communication for which no public specification exists. That is, normally we can refer to IETF RFCs to understand Internet protocols, but no equivalent exists for industrial networks. As such, understanding industrial traffic requires hard manual reverse engineering.
To ease this process, we clustered protocols by their payload similarity. Check our paper for more details. As a result, we identified three protocol clusters, which we referred to as ICS 1, ICS 2 and SPPA in the figure below. Beyond that, there are also known protocols like NTP or Java RMI in the trace.
At the supervisory level, we do not see any proprietary or ICS-typical protocols. Java RMI (76.6%) is dominating here (Figure 2, far right) followed by HTTPS and SAMBA. Java RMI based RPC communication is used for graphical representation by browser-based clients at the supervisory level, which interacts with application servers.
To learn more about our study, check out our PAM 2022 paper: Lights on Power Plant Control Networks.
Stefan Mehner is currently a Research Assistant at the Computer Networks Group at the Brandenburg University of Technology in Cottbus, Germany.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.