docker容器磁盘使用率超过100%怎么办
2019-01-25 本文已影响0人
开始懂了90
docker容器默认大小是10G 如果磁盘满了 如何进行清理 如下：
Filesystem Size Used Avail Use% Mounted on
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 20G 5.9G 13G 32% /
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 16G 1.6G 15G 11% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
tmpfs 16G 12K 16G 1% /var/lib/kubelet/pods/390a8d0e-e174-11e8-b14e-5254416bb222/volumes/kubernetes.io~secret/default-token-phghf
shm 64M 0 64M 0% /var/lib/docker/containers/69618ee0322b2a62b586b8f8a546ee189517cc111d6ec720db70b19ceebfed27/shm
/dev/dm-34 10G 10G 20K 100% /var/lib/docker/devicemapper/mnt/71f36d95c98684ecd86f73f342c440e7bc08b9bb6a552e7bbd348bc431b1df21
## /dev/dm-34 这块磁盘使用率超过了100%，如何清理这个磁盘 释放空间
找出写满磁盘的进程
[root@node-23-44 ~]# lsof /var/lib/docker/devicemapper/mnt/71f36d95c98684ecd86f73f342c440e7bc08b9bb6a552e7bbd348bc431b1df21
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sh 18108 root cwd DIR 252,34 69 20972553 /opt
sh 18108 root rtd DIR 252,34 293 1027 /
sh 18108 root txt REG 252,34 960376 10486805 /usr/bin/bash
sh 18108 root mem REG 252,34 31459441 /usr/lib64/libnss_files-2.17.so (path dev=253,1, inode=788547)
sh 18108 root mem REG 252,34 31458406 /usr/lib64/libc-2.17.so (path dev=253,1, inode=788165)
sh 18108 root mem REG 252,34 31459329 /usr/lib64/libdl-2.17.so (path dev=253,1, inode=788538)
sh 18108 root mem REG 252,34 31459548 /usr/lib64/libtinfo.so.5.9 (path dev=253,1, inode=788497)
sh 18108 root mem REG 252,34 31458382 /usr/lib64/ld-2.17.so (path dev=253,1, inode=788131)
java 18146 root cwd DIR 252,34 69 20972553 /opt
java 18146 root rtd DIR 252,34 293 1027 /
java 18146 root txt REG 252,34 7734 6295582 /jdk1.8.0_131/bin/java
java 18146 root mem REG 252,34 31459566 /usr/lib64/libz.so.1.2.7 (path dev=253,1, inode=788132)
java 18146 root mem REG 252,34 31458405 /usr/lib64/libbz2.so.1.0.6 (path dev=253,1, inode=788591)
java 18146 root mem REG 252,34 31459411 /usr/lib64/liblzma.so.5.0.99 (stat: No such file or directory)
java 18146 root mem REG 252,34 31459333 /usr/lib64/libelf-0.163.so (stat: No such file or directory)
java 18146 root mem REG 252,34 31458396 /usr/lib64/libattr.so.1.1.0 (path dev=253,1, inode=788761)
java 18146 root mem REG 252,34 31459347 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 (path dev=253,1, inode=786448)
java 18146 root mem REG 252,34 31459331 /usr/lib64/libdw-0.163.so (stat: No such file or directory)
java 18146 root mem REG 252,34 31458411 /usr/lib64/libcap.so.2.22 (path dev=253,1, inode=788766)
java 18146 root mem REG 252,34 31459498 /usr/lib64/libresolv-2.17.so (path dev=253,1, inode=788552)
java 18146 root mem REG 252,34 31459439 /usr/lib64/libnss_dns-2.17.so (path dev=253,1, inode=788181)
java 18146 root mem REG 252,34 31459445 /usr/lib64/libnss_myhostname.so.2 (path dev=253,1, inode=789230)
java 18146 root mem REG 252,34 23073833 /jdk1.8.0_131/jre/lib/jsse.jar (stat: No such file or directory)
java 18146 root mem REG 252,34 23073846 /jdk1.8.0_131/jre/lib/resources.jar (stat: No such file or directory)
java 18146 root mem REG 252,34 6299672 /jdk1.8.0_131/jre/lib/ext/sunpkcs11.jar (stat: No such file or directory)
java 18146 root mem REG 252,34 6299671 /jdk1.8.0_131/jre/lib/ext/sunjce_provider.jar (stat: No such file or directory)
java 18146 root mem REG 252,34 6299666 /jdk1.8.0_131/jre/lib/ext/jfxrt.jar (stat: No such file or directory) /jdk1.8.0_131/jre/lib/ext/jaccess.jar (stat: No such file or directory)
java 18146 root 1w REG 252,34 4720398336 21008433 /opt/info.log
java 18146 root 3r REG 252,34 66159478 23073847 /jdk1.8.0_131/jre/lib/rt.jar
java 18146 root 4r REG 252,34 75018036 21008432 /opt/e-share-service-content.jar
java 18146 root 5r REG 252,34 75018036 21008432 /opt/e-share-service-content.jar
java 18146 root 6w REG 252,34 9707 8430632 /opt/logs/log_error.log
java 18146 root 7w REG 252,34 217 8430610 /opt/logs/log_warn.log
java 18146 root 8w REG 252,34 124 8430613 /opt/logs/log_info.log
java 18146 root 10r REG 252,34 3505330 23073846 /jdk1.8.0_131/jre/lib/resources.jar
java 18146 root 11r REG 252,34 3135391 23073803 /jdk1.8.0_131/jre/lib/charsets.jar
java 18146 root 12r REG 252,34 18490072 6299666 /jdk1.8.0_131/jre/lib/ext/jfxrt.jar
java 18146 root 13r REG 252,34 3860502 6299663 /jdk1.8.0_131/jre/lib/ext/cldrdata.jar
java 18146 root 14r REG 252,34 8286 6299664 /jdk1.8.0_131/jre/lib/ext/dnsns.jar
java 18146 root 15r REG 252,34 44516 6299665 /jdk1.8.0_131/jre/lib/ext/jaccess.jar
java 18146 root 16r REG 252,34 1179093 6299667 /jdk1.8.0_131/jre/lib/ext/localedata.jar
java 18146 root 17r REG 252,34 2022531 6299669 /jdk1.8.0_131/jre/lib/ext/nashorn.jar
java 18146 root 18r REG 252,34 42154 6299670 /jdk1.8.0_131/jre/lib/ext/sunec.jar
java 18146 root 19r REG 252,34 280161 6299671 /jdk1.8.0_131/jre/lib/ext/sunjce_provider.jar
java 18146 root 20r REG 252,34 251327 6299672 /jdk1.8.0_131/jre/lib/ext/sunpkcs11.jar
java 18146 root 21r REG 252,34 68924 6299673 /jdk1.8.0_131/jre/lib/ext/zipfs.jar
java 18146 root 23r REG 252,34 647516 23073833 /jdk1.8.0_131/jre/lib/jsse.jar
java 18146 root 234w REG 252,34 4001792 8430641 /opt/logs/log_debug.log
java 18146 root 236r REG 252,34 75018036 21008432 /opt/e-share-service-content.jar
通过查看发现 只有18108 18146 这两个进程使用过这个磁盘文件 让我们来看看这两个什么样的进程
[root@node-23-44 ~]# ps -ef |grep 18108
root 18108 18091 0 2018 ? 00:00:00 /bin/sh -c java -jar e-share-service-content.jar > info.log
root 18146 18108 1 2018 ? 08:53:03 java -jar e-share-service-content.jar
root 20782 13686 0 14:06 pts/0 00:00:00 grep --color=auto 18108
[root@node-23-44 ~]# ps -ef |grep 18146
root 18146 18108 1 2018 ? 08:53:03 java -jar e-share-service-content.jar
root 20880 13686 0 14:06 pts/0 00:00:00 grep --color=auto 18146
## 由上可知 18108 是18146的父进程 所以我们只要查看一下18146跑的是那个容器即可
[root@node-23-44 ~]# docker inspect -f '{{.State.Pid}} {{.Id}}' $(docker ps -q) | grep 18108
18108 2ded174d0bb5b353fbb73f9bca980a2d8d163b17d7c5ff4bbeb4961bb9f7f364
进入容器查看
[root@node-23-44 ~]# docker exec -it 2ded174d0bb5 sh
sh-4.2# du -sh ./*
72M ./e-share-service-content.jar
4.4G ./info.log
4.1G ./logs
## 容器日志已经超过了8.5G
由上分析 是日志打满了磁盘 如果需要释放空间 清理日志即可
