VestaCP compromised in a new supply-chain attack

Customers see their admin credentials stolen and their servers infected with Linux/ChachaDDoS

In recent months, numerous users of VestaCP, a hosting control panel solution, have received warnings from their service providers that their servers were using an abnormal amount of bandwidth. We know now that these servers were in fact used to launch DDoS attacks. Analysis of a compromised server has shown that malware we call Linux/ChachaDDoS was installed on the system. At the same time this week, we found out that the official VestaCP distribution was compromised, resulting in a supply-chain attack on new installations of VestaCP since at least May 2018. Linux/ChachaDDoS has some similarity to Linux/Xor.DDoS (aka Xor.DDOS), but unlike this older family, it has multiple stages and uses Lua for its second- and third-stage components.

Infection vector

According to user “Razza” on the VestaCP forum, the attacker tried launching Linux/ChachaDDoS via SSH. It is not clear how the payload was dropped in the /var/tmp directory, but assuming the attacker already has the admin password, it would have been a trivial task. During the installation, VestaCP creates a user named “admin” that has sudo privileges. How could the attacker have known the password for this admin user?

There are multiple hypotheses as to how the credentials were obtained in the first place. We first suspected a vulnerability in the web interface of VestaCP. While looking at the code, we found that the unencrypted password is kept in /root/.my.cnfbut accessing the content of this file would still require the attacker to exploit a local file inclusion and privilege escalation vulnerability. User “Falzo” also dug in the code and found out something even more interesting: some versions of the installation script were leaking the admin password and server name to vestacp.com, the official website of VestaCP.

As “L4ky” pointed out, it is all in the Git history of the vst-install-ubuntu.sh file. From May 31 18:15:53 2018 (UTC+3) (a3f0fa1) to Jun 13 17:08:36 2018 (ee03eff) the $codename variable contained the base64-encoded password and server domain name sent to http://vestacp.com/notify/. Falzo says he found the hack in line 809 of the Debian installer, but unlike for the Ubuntu installer, we couldn’t find a reference to it in the Git history. Perhaps the installer on VestaCP differed from what is visible on Github?

Given this major password leak, we urge all VestaCP administrators to change the admin password and harden access to their servers. Serious admins should consider an audit of the VestaCP code.

While this finding is shocking, there is no evidence that this password leakage is how Linux/ChachaDDoS was distributed in the first place. It could have been through another hole.

VestaCP maintainers stated they were compromised. How the malicious code ended up in their Git tree is still unclear. Perhaps the perpetrator modified the installation scripts on the server and this version was used to create the next version of the file in Git, but only for the Ubuntu target. This would mean they have been compromised since at least May 2018.

Linux/ChachaDDoS analysis

The malware dropped on the compromised servers is a variant of a new strain of DDoS malware that we call ChachaDDoS. It seems like an evolution of multiple existing DDoS malware. The first and second stages set their process names to [kworker/1:1]. That is what would appear in the output of ps.

First stage

Persistence mechanism and link to Xor.DDoS

The persistence mechanism used in Linux/ChachaDDoS is actually the same as in Linux/XorDDos except for the filename, dhcprenew. It consists of the following steps:

  1. It copies itself to /usr/bin/dhcprenew
  2. If any persistence mechanism related to the malware is already set on the host, it is removed
  3. A new service is added to /etc/init.d/dhcprenew

  1. A symlink to this service is created in /etc/rc[1-5].d/S90dhcprenew and /etc/rc.d/rc[1-5].d/S90dhcprenew
  2. It runs the command lines chkconfig --add dhcprenew and update-rc.d dhcprenew defaults to enable the service

Download and decryption of the second stage

Once the persistence is set, the second stage is periodically downloaded from a hardcoded URL. Interestingly, from the different samples we analysed, we observed some similar characteristics concerning the structure of the URL:

  • The use of port 8852
  • All the IP addresses used belong to the 193.201.224.0/24 subnet (AS25092, OPATELECOM PE Tetyana Mysyk, Ukraine)
  • The resource name of the second stage looks pseudorandom but is always a 6-to-8 character uppercase string (such as JHKDSAG or ASDFRE)

The URL follows the pattern http://{C&C}:8852/{campaign}/{arch}. We found that second stage binaries are available for multiple architectures including x86, ARM, MIPS, PowerPC and even s390x. After downloading the ELF file corresponding to the architecture of the victim host, it is decrypted with ChaCha encryption algorithm. ChaCha is the successor of the Salsa20 stream cipher. Both ciphers use the same constant expand 32-byte k to set the initial state; the following image shows the beginning of the decryption function:

The differences between the two algorithms are a rearrangement of the initial state and a modification of the quarter-round, which is the core operation performed by the ciphers. Thanks to the specific rotations used in its quarter-round, we could identify the use of ChaCha as shown in the following snippet:

The key size used for ChaCha decryption is 256 bits and among all the samples we collected, we observed the use of the same key. In order to avoid the pain of reimplementing the decryption algorithm, we developed a script based on Miasm to emulate the decryption function.

Once we decrypted the second stage, it appeared the output is LZMA compressed, so we just extracted the binary using  lzma -d < output > second_stage.elf.

Second stage

The binary itself is much larger than the first stage and this is mainly because of the embedded Lua interpreter. Malware in Lua is something we have seen before with Linux/Shishiga. The purpose of the second stage is to execute a hardcoded Lua payload that downloads tasks periodically. We consider the task as a third stage because a task is basically Lua code to be interpreted. In all variants we observed, the second stage uses the same C&C server as its first stage. This second stage embeds numerous Lua libraries (such as LuaSocket) to communicate with the hardcoded C&C server.

Some native functions of the binary are exposed so they can be called from Lua code; the following screenshot shows how they are exported, like the ChaCha encryption function, for example.

The task downloaded by the Lua payload is ChaCha decrypted (with a different encryption key) and executed by the Lua interpreter. As for the second stage, the URL used to download the task seems to follow a specific pattern, as one can observe from the following snippet of code:

Also, the payload should send some statistics using the URL specified in the screenshot above regarding the task usage; however, in practice, it sends only the MAC address and some other information:

Third stage (tasks)

From the tasks we were able to collect, we observed only the implementation of a DDoS function. The code is pretty explicit and consists mainly of a call to a function to perform a SYN DDoS attack against a given target:

The IP address of the DDoS target, 144.0.2.180, belongs to a Chinese ISP. We couldn’t find any obvious reason for this IP address to be a target of the DDoS attack, as no services seem to be hosted on that IP address.

The Last-Modified HTTP response header of the task file response indicates that this target was the same since September 24th 2018. It should be reliable since the malware use the If-Modified-Since HTTP request  header to avoid downloading payloads again.

The ASDFREM campaign is the only other one with an active task. It is similar but targets yet another IP address in China: 61.133.6.150.

Conclusion

It’s obvious ChachaDDoS shares code with Xor.DDoS for its persistence mechanism. But is it from the same author or did the ChachaDDoS authors simply steal it? ChachaDDoS got our attention because it was caught on VestaCP instances, but the existence of binaries for multiple architectures suggests that other devices, including embedded devices, are targeted by this threat.

This incident is also a reminder that just because software is open source, it is not necessarily 100% safe. Malware can still make its way in. The malicious credential-stealing code was right there, for everyone to see on GitHub, for multiple months before it was spotted. We do agree it does help find vulnerabilities — post-mortem in this case — but it doesn’t mean we should blindly trust a product simply on the basis that it is open source.

ESET products detect this threat as Linux/Xorddos.Q, Linux/Xorddos.R and Linux/ChachaDDoS.

Thanks to Hugo Porcher for his help in this analysis and write-up.

Indicators of Compromise (IoCs)

First stage

Hash (SHA-1)ESET detection nameArchSecond stage URLs
bd5d0093bba318a77fd4e24b34ced85348e43960Linux/Xorddos.Qx86_64hxxp://193.201.224.238:8852/RTEGFN01
0413f832d8161187172aef7a769586515f969479Linux/Xorddos.Rx86_64hxxp://zxcvbmnnfjjfwq.com:8852/RTEGFN01
hxxp://efbthmoiuykmkjkjgt.com:8852/RTEGFN01
0328fa49058e7c5a63b836026925385aac76b221Linux/ChachaDDoS.Bmipshxxp://9fdmasaxsssaqrk.com:8852/YTRFDA
hxxp://10afdmasaxsssaqrk.com:8852/YTRFDA
334ad99a11a0c9dd29171a81821be7e3f3848305Linux/ChachaDDoS.Bmipshxxp://193.201.224.238:8852/DAAADF
4e46630b98f0a920cf983a3d3833f2ed44fa4751Linux/ChachaDDoS.Barmhxxp://193.201.224.233:8852/DAAADF
3caf7036aa2de31e296beae40f47e082a96254ccLinux/ChachaDDoS.Bmipshxxp://8masaxsssaqrk.com:8852/JHKDSAG
hxxp://7mfsdfasdmkgmrk.com:8852/JHKDSAG
0ab55b573703e20ac99492e5954c1db91b83aa55Linux/ChachaDDoS.Barmhxxp://193.201.224.202:8852/ASDFREM
hxxp://193.201.224.202:8852/ASDFRE
ChaCha key
fa408855304ca199f680b494b69ef473dd9c5a5e0e78baa444048b82a8bd97a9

Second stage

Hash (SHA-1)ESET detection nameArchSecond stage URLs
1b6a8ab3337fc811e790593aa059bc41710f3651Linux/ChachaDDoS.Apowerpc64hxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
4ca3b06c76f369565689e1d6bd2ffb3cc952925dLinux/ChachaDDoS.Aarmhxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
6a536b3d58f16bbf4333da7af492289a30709e77Linux/ChachaDDoS.Apowerpchxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
72651454d59c2d9e0afdd927ab6eb5aea18879ceLinux/ChachaDDoS.Ai486hxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
a42e131efc5697a7db70fc5f166bae8dfb3afde2Linux/ChachaDDoS.As390xhxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
abea9166dad7febce8995215f09794f6b71da83bLinux/ChachaDDoS.Aarm64hxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
bb999f0096ba495889171ad2d5388f36a18125f4Linux/ChachaDDoS.Ax86_64hxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
d3af11dbfc5f03fd9c10ac73ec4a1cfb791e8225Linux/ChachaDDoS.Amips64hxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
d7109d4dfb862eb9f924d88a3af9727e4d21fd66Linux/ChachaDDoS.Amipshxxp://193.201.224.238:8852/RTEGFN01/RTEGFN01.dat
56ac7c2c89350924e55ea89a1d9119a42902596eLinux/ChachaDDoS.Amipshxxp://193.201.224.238:8852/DAAADF/DAAADF.dat
Chacha key
000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f

References

  1. http://blog.malwaremustdie.org/2014/09/mmd-0028-2014-fuzzy-reversing-new-china.html
  2. https://www.fireeye.com/blog/threat-research/2015/02/anatomy_of_a_brutef.html
  3. https://otx.alienvault.com/indicator/file/0177aa7826f5239cb53613cc90e247b710800ddf
  4. https://forum.vestacp.com/viewtopic.php?f=10&t=16556
  5. https://carolinafernandez.github.io/security/2015/03/16/IptabLeX-XOR-DDoS
  6. https://blog.checkpoint.com/wp-content/uploads/2015/10/sb-report-threat-intelligence-groundhog.pdf
  7. https://grehack.fr/data/2017/slides/GreHack17_Down_The_Rabbit_Hole:_How_Hackers_Exploit_Weak_SSH_Credentials_To_Build_DDoS_Botnets.pdf

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Top
%d bloggers like this: