It’s the Process That Counts, Not the Data
Introduction
Control system cybersecurity is about keeping lights on, water flowing, etc. It is not simply a matter of maintaining network availability. If control systems are affected by a cyber incident, whether it’s an unintentional incident or a deliberate attack, critical infrastructure reliability, availability and safety may be affected. Industrial, manufacturing, transportation and other industries rely on operational technology, internet protocol-based networks to bring significant productivity improvements. But those improvements also bring significant cyber vulnerabilities.
The fallacy about critical infrastructure cybersecurity is the assumption that IP networks are needed to keep lights on, water flowing, etc. For more than 80 years, the grid operated without an IP network. Control systems in power systems are designed to work in coordination with each other so the equipment associated with control systems can work without SCADA and the SCADA network. As an example, following the 2015 cyberattack of the Ukrainian power grid, the Ukrainians continued to operate the grid manually for months without the IP networks, which couldn’t be trusted. But the grid could not have been operated if the critical hardware, including the process sensors monitoring and controlling the grid, had been compromised or damaged.
On July 28, 2021, the Biden Administration made an announcement about further actions to protect U.S. critical infrastructure. President Biden’s Industrial Control System Cybersecurity Initiative is a voluntary, collaborative effort between the federal government and the critical infrastructure community to facilitate the deployment of technology and systems that provide threat visibility, indicators, detections and warnings. To date, this is a network-based approach specific to cyberthreats. Control system field devices, such as pressure, level, flow, temperature and voltage sensors – which are often not considered part of OT – are inherently insecure and generally not designed to be connected to IP networks. The president’s ICS Initiative does not address this problem.
Background
Prior to 9/11, cybersecurity was simply one of the risks that had to be considered when designing and implementing control systems, along with seismic risk, environmental risk, fire risk, reliability risk, etc. Those risks were regarded as engineering considerations, and managing them was considered an engineering function. The intent was to ensure that the engineering basis of the design would be met, regardless of the risk. Consequently, the engineering organization was responsible, and this included cybersecurity. It was a bottom-up approach of process anomaly detection, performed in the interest of mission assurance. In fact, this was the basis of the Electric Power Research Institute’s control system cybersecurity program that I helped start in 2000, which unfortunately is now, like others, concerned with securing the networks.
Some time after 9/11, cybersecurity became national security. Around the same time, however, cybersecurity for control systems was moved to the IT, now OT, network monitoring organizations, and engineering was no longer involved. As a result, control system cybersecurity went from mission assurance to information assurance.
The focus on networks rather than on the process can also be seen by having the CISO and not the vice president of engineering/operations responsible for the cybersecurity of engineering systems. Consequently, cybersecurity monitoring and mitigation tended to move to the IP network layer, and network anomaly detection tended to replace process anomaly detection.
Control system devices, such as protective relays, work on instructions entered into registers within the hardware of the device. These instructions reference other instructions and raw process sensor input data to perform desired commands. This means that devices such as protective relays have little to do with traditional higher-level networks but depend on the integrity of the measurement.
A Timeline of Avoidance
The reticence of the U.S. government and industry to move away from a traditional network-based approach can be seen from the following examples:
- The July 2021 version 2.0 of the Cybersecurity Capability Maturity Model does not address the process sensors and process anomaly detection. How mature can the process be if it doesn’t address what keeps the lights on and water flowing?
- The electric industry’s NERC Critical Infrastructure Protection cybersecurity standards consider process sensors out of scope for cybersecurity considerations.
- The recent podcast by Idaho National Laboratory personnel supports the network approach.
- Presidential Executive Order 13920 was issued following discovery of hardware backdoors in large Chinese-made electric transformers. The attack focused exclusively on hardware and control systems, but the government and industry response was to turn this hardware attack into a software supply chain problem.
Use of Sensor Monitoring
Process sensor monitoring has been used for many years for process anomaly detection. I was using it in the late 1970s to identify flow-induced vibration issues in nuclear plants and in the early 1990s while managing the EPRI Nuclear Instrumentation and Diagnostic Program to detect a major supply chain common-cause process sensor problem.
Legacy engineering field devices – such as process sensors, actuators, drives, positioners and analyzers – have no cybersecurity, authentication or cyber logging, and they cannot be easily upgraded for cybersecurity. Yet process sensors deliver the inputs to the OT network where the OT network monitoring providers assume the sensor input is uncompromised, authenticated and correct. Because the sensor input is not authenticated, however, it is not clear that the apparent sensor data is actually coming from the sensors and not from spoofed signals.
The actuators, drives, controllers, etc., receiving the sensor signals have no way to authenticate the origin of the sensor signals and therefore automatically accept the sensor and respond accordingly. This could be the approach the Chinese are using with the hardware backdoors in the large electric transformer to take control of the transformer without having to hack the networks. Therefore, the intractable network monitoring approach needs to be made into a tractable engineering program.
Modern machine learning enables pattern detection of the raw process sensor signals that wasn’t previously possible. This additional capability enables sensor monitoring to identify process anomalies regardless of cause and independent of IP networks and their associated cyber vulnerabilities. As a result, the Israel Water Authority recently took that engineering approach, approving offline process sensor monitoring technology to secure the country’s water systems.
Unlike the prevalent U.S. practice of monitoring IT and OT networks for cybersecurity – that is, for network anomaly detection – the Israeli approach is based on monitoring the electrical characteristics of the process sensors – process anomaly detection – and not just relying on network monitoring as in the U.S.
Benefits of Offline Sensor Monitoring
To understand why the process sensor approach can be so valuable, consider a car moving at 70 mph when one of the tires is flat. You pull off the road and replace the flat tire with the small spare tire. You can then continue to drive the car, albeit at a reduced speed, until you can get the regular tire replaced. Now consider the IT and OT networks that provide productivity. If they are lost because of ransomware or any other type of malware, the offline monitoring of the sensors, which is not sensitive to the IT malware, allows the facility to continue operating, albeit in reduced efficiency, until the IP networks are restored.
Specifically, the benefits of the Israeli approach are:
- Raw process sensor signals provide ground truth about the physical operation of the system.
- The process sensor monitoring system is not susceptible to IT or OT unintentional network issues; network attacks, including ransomware; or vulnerabilities induced by patch management oversights.
- In process anomaly detection, the system detects any anomaly regardless of cause, not just malicious cyberattacks, which means even sophisticated attacks that look like equipment malfunctions, such as Stuxnet, will be identified.
- Process sensor monitoring can identify incidents as process anomalies. I have amassed a database of almost 12 million control system cyber incidents that have resulted in more than 1,500 deaths and more than $90 billion in direct damage, but network monitoring did not identify the vast majority of them as being cyber-related.
- By monitoring in real-time, the system is essentially a sensor health monitoring system and so also functions as a predictive maintenance system that can be used to extend maintenance intervals.
- Process sensor monitoring systems have detected equipment impacts that were not identified by the Windows-based OT monitoring system.
- Monitoring the sensors requires the involvement of the engineers responsible for the process.
- Monitoring the process sensors provides authentication, which otherwise would not exist.
- The process sensor monitoring system is applicable to any critical infrastructure and has been installed in water, power, chemicals and building controls.
- Monitoring of process sensors applies to all infrastructures, as they all use process sensing. This approach of addressing multiple industries meets the intent of the president’s ICS Initiative. For example, the new Transportation Security Administration cybersecurity requirements do not address potential pipe failures because they are network-based and don’t address the process sensors.
As of July 27, critical pipeline operators have reported more than 220 cybersecurity incidents since the TSA implemented emergency measures in the wake of the crippling ransomware attack on one of America’s most important pipelines. I am not aware, however, of any recent reports of pipeline ruptures or pipeline outages, meaning the 220 cybersecurity incidents were IT incidents not affecting the operation of the pipelines. Yet the two cyber-related pipeline ruptures that have killed people and destroyed structures would not have been addressed by the TSA cybersecurity guidelines.
- Sensor monitoring can be applied to certain supply chain situations, such as the hardware backdoors in the Chinese-made electric transformers, to ensure the sensing input going to the transformer devices are not spoofed signals coming from elsewhere, as the hardware backdoors bypassed all cybersecurity protections.
- Given the recent JBS meat processing plant shutdowns, the sensor monitoring approach can help the food industry justify continued operation, as there continues to be a view of the plant processes.
The Limits of Network Security
The disadvantages of the U.S. approach include:
- Neither IT nor OT networks provide ground truth about the process and assume the sensor input is uncompromised, authenticated and correct.
- Network monitoring is a never-ending game of whack-a-mole in which defenders come up with a solution and attackers come up with a bypass.
- Even the best network cybersecurity can be defeated; consider SolarWinds.
- OT networks are susceptible to unsophisticated as well as sophisticated network vulnerabilities.
- OT cybersecurity organizations tend to exclude the engineers responsible for the design and operation of the control systems.
Keeping the Lights On
With the never-ending and too often successful attacks on critical infrastructure networks, there needs to be a better way to protect control systems and the processes they monitor and control. The existing approach of securing critical infrastructures by securing the networks is not working. The Israel Water Authority recognized this and is monitoring the electrical characteristics of the process sensors as the raw process sensor signals are ground truth and not susceptible to network attacks. Hopefully, the U.S. government, insurance companies, credit rating agencies and others will recognize what is really needed to be secured: the field control system equipment that keeps lights on and water flowing.