Detecting a Linux Backdoor
Spotting the Ebury credential stealing botnet in the wild.
Date: 2024-05-25 | Author: Julian De Los Prados
Recently I was forwarded an email from a friend: 
They asked if I could look at the server and confirm if it was infected. I began researching the now 15 year old Ebury backdoor and Operation Windigo - the "massive, sophisticated and organized" malware campaign first launched in 2011 but brought to light by researchers at ESET in 2014.
In that initial report on Operation Windigo, researchers detail a series of exploits that showed "out of the ordinary knowledge of operating systems and programming". Ebury is just one of the tools used in that malware campaign. Recent reporting estimates that 400,000 Linux servers have been compromised across the globe since the inception of "one of the most advanced server-side malware campaigns". The goal is simple: monetize victms by stealing their credit card information, emptying their cryptocurrency wallets and intercepting their login credentials, all while redirecting their traffic and sending spam from their server.
Eight years later, Ebury shows no signs of abatement and has been consistently updated to enhance its features and better cover its tracks. Its latest version, 1.82 was discovered in 2024 and is discussed more in ESET's latest whitepaper.
Ebury Characteristics and Risk
Ebury is a SSH rootkit/backdoor trojan designed to steal OpenSSH credentials and maintain access to compromised servers. It allows the Windigo malware campaign to expand by stealing SSH credentials of newly infected targets. It completeley bypasses the built-in password validation system in sshd, the OpenSSH server process, allowing hackers to intercept credentials and configure a persistent backdoor. Once established, the backdoor allows the user to logon to any legitimate user by passing an 11-digit backdoor password to OpenSSH.
Ebury and the attackers behind it are known to take intensive measures to cover their tracks, including deleting evidence in log files while the backdoor is in use, avoiding downloading files onto the filesystem, and injecting code at runtime. I used two characteristics in particular to confirm the presence of Ebury:
- Unix pipes designed to look like legitimate process
- payloads disguised as a legitimate library called `libkeyutils.so`
Host-based indicators
Starting with version 1.5, Ebury runs a continuous process for managing data and stealing credentials. It uses abstract UNIX sockets to enable communication between the infected SSH client or server and this ongoing process. These sockets create an IPC (Inter-Process Communication) channel, allowing parts of the malware to coordinate secretly and avoid detection by not using the file system.
I searched the server and found an abstract UNIX socket named @UDEV-4kAmkRW3 that was associated with a legitimate application - anacron. 'UDEV-4kAmkRW3' is a common name known to be used by Ebury, and '@' differentiates it as an abstract UNIX socket, meaning that it is independent of the filesystem and will not connect to it directly.
Ebury favors these types of UNIX sockets because they avoid the use of the file system, enhancing its ability to evade detection. With a secure channel established, the different moving parts of the malware can circulate data without alerting the system's antibodies.
Using @UDEV-4kAmkRW3, the malware listens for an opportunity to load the Ebury payload into a legitimate executable, in this case it seems to have picked anacron - a Linux daemon that is used to schedule tasks, similar to cron. Other legitimate Linux processes that Ebury has been known to target are: hostname, sync, auditd, rsyslogd, udevd, systemd-udevd, acpid, arpd, atd, and crond.
Tied to the suspicious UNIX socket was a process with PID 616. Further investigating the process I examined:
- the files that the process was using, including directories and network connections (sockets), and other types of file descriptors.
- information about the process including the user running it, CPU and memory usage, the start time of the process and the command that started it.
- any external and internal network connections the process is involved with.
Of concern was the fact that the socket `@UDEV-4kAmkRW3` is in a listening state and associated with `anacron`. This is unusual because anacron typically does not need to create network sockets; it should only be dealing with time-based scheduling of tasks.
Finding the Ebury payload
Another indicator of compromise I searched for was the Ebury payload, often disguised as a trojanized version of a legitimate library. Some libraries that Ebury has been known to trojanize are: libkeystats.so, libkeyctl.so, and librwctl.so.
Running a ClamAV scan in the shared libraries directory on the server revealed a compromised libkeystats.so library:
Running a detection script released by ESET also confirmed the Ebury infection:
How did it happen?
There are a few known attack vectors for Ebury:
- supply chain attacks on hosting providers.
- credential stuffing.
- vulnerabilities such as CVE-2021-45467 affecting Centos Web Portal.
There aren't any reports online of DigitalOcean being compromised by Ebury, and I was unable to find leaked server passwords in pwn databases (my search was not exhaustive). This server did host a CWP control panel though, which warranted a further look.
If you are not familiar with CWP, it's a popular control panel used by server administrators. CVE-2021-45467 describes a vulnerability in which an attacker can use %00 bytes to cause /usr/loader/php to register an API key on their behalf. It wasn't hard to find evidence of similar attacks in the server's logs:
I was unable to confirm wether any of these attacks where successful and I confirmed that the CWP control panel was up to date with the patch for this vulnerability. Still, it's possible that the server was compromised before the last update.
Next steps
My advice to my friend was threefold:
- back up all their critical files.
- re-create the impacted Droplet, fully patched.
- ensure backups aren't compromised as to not reintroduce the malware.
Thank you for sticking with me through this exploration of the Ebury backdoor, staying informed is our best defense against cyber threats.