Crash 分析 kdump文件
2021-08-27 本文已影响0人
偷油考拉
Kernel crash dump guide Red Hat Enterprise Linux 7
Analyzing Linux kernel crash dumps with crash - The one tutorial that has it all (dedoimedo.com)
运行Crash
wget http://debuginfo.centos.org/7/x86_64/kernel-debug-debuginfo-3.10.0-693.el7.x86_64.rpm
rpm -ivh kernel-debug-debuginfo-3.10.0-693.el7.x86_64.rpm
缺失 /usr/lib/debug/lib/modules/ 会导致运行不了
解决办法：
安装对应版本的kernel-debug
[sysadmin@prod-proxy ~]$ cd /var/crash/127.0.0.1-2021-08-20-00\:09\:50/
[sysadmin@prod-proxy 127.0.0.1-2021-08-20-00:09:50]$ sudo crash /usr/lib/debug/lib/modules/$(uname -r)/vmlinux vmcore
[sudo] password for sysadmin:
crash 7.2.3-11.el7_9.1
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/lib/modules/3.10.0-693.el7.x86_64/vmlinux
DUMPFILE: vmcore [PARTIAL DUMP]
CPUS: 2
DATE: Fri Aug 20 00:09:46 2021
UPTIME: 1 days, 00:52:45
LOAD AVERAGE: 0.05, 0.03, 0.05
TASKS: 267
NODENAME: prod-proxy
RELEASE: 3.10.0-693.el7.x86_64
VERSION: #1 SMP Tue Aug 22 21:09:27 UTC 2017
MACHINE: x86_64 (1695 Mhz)
MEMORY: 4 GB
PANIC: "SysRq : Trigger a crash"
PID: 9151
COMMAND: "bash"
TASK: ffff8800b8c5af70 [THREAD_INFO: ffff88013717c000]
CPU: 0
STATE: TASK_RUNNING (SYSRQ)
crash>
分析内存
set scroll off
set scroll on
crash> task | grep uid
uid = 3369,
euid = 3369,
suid = 3369,
fsuid = 3369,
crash> foreach bt > bt.all
crash> ps >> process.data
crash> kmem -i | grep SLAB > slab.pages
crash>
-
bt命令。Display a kernel stack backtrace。
crash> bt
PID: 9151 TASK: ffff8800b8c5af70 CPU: 0 COMMAND: "bash"
#0 [ffff88013717fb28] machine_kexec at ffffffff8105c4cb
#1 [ffff88013717fb88] __crash_kexec at ffffffff81104a32
#2 [ffff88013717fc58] crash_kexec at ffffffff81104b20
#3 [ffff88013717fc70] oops_end at ffffffff816ad278
#4 [ffff88013717fc98] no_context at ffffffff8169d29a
#5 [ffff88013717fce8] __bad_area_nosemaphore at ffffffff8169d330
#6 [ffff88013717fd30] bad_area_nosemaphore at ffffffff8169d49a
#7 [ffff88013717fd40] __do_page_fault at ffffffff816b013e
#8 [ffff88013717fda0] do_page_fault at ffffffff816b02e5
#9 [ffff88013717fdd0] page_fault at ffffffff816ac508
[exception RIP: sysrq_handle_crash+22]
RIP: ffffffff813fe3f6 RSP: ffff88013717fe88 RFLAGS: 00010246
RAX: 000000000000000f RBX: ffffffff81ab8100 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff88013fc0f8b8 RDI: 0000000000000063
RBP: ffff88013717fe88 R8: ffffffff81d86dfc R9: ffff8800baa6c640
R10: 00000000000005f8 R11: 00000000000005f7 R12: 0000000000000063
R13: 0000000000000000 R14: 0000000000000004 R15: 0000000000000000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#10 [ffff88013717fe90] __handle_sysrq at ffffffff813fec17
#11 [ffff88013717fec0] write_sysrq_trigger at ffffffff813ff08f
#12 [ffff88013717fed8] proc_reg_write at ffffffff8127018d
#13 [ffff88013717fef8] vfs_write at ffffffff81200d2d
#14 [ffff88013717ff38] sys_write at ffffffff81201b3f
#15 [ffff88013717ff80] system_call_fastpath at ffffffff816b4fc9
RIP: 00007fbe8a31c840 RSP: 00007ffdfda17cf0 RFLAGS: 00000246
RAX: 0000000000000001 RBX: ffffffff816b4fc9 RCX: 0000000000000000
RDX: 0000000000000002 RSI: 00007fbe8ac43000 RDI: 0000000000000001
RBP: 00007fbe8ac43000 R8: 000000000000000a R9: 00007fbe8ac3c740
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000001
R13: 0000000000000002 R14: 00007fbe8a5f0400 R15: 0000000000000002
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
-
ps命令。This command displays process status for selected, or all, processes
in the system。
-
kmemThis command displays information about the use of kernel memory.
kmem -i
kmem -f
kmem -F
kmem -p
kmem -m lru
kmem -m lru | grep ffffea0001dafb20
kmem -s kmalloc-8192
kmem -m slab_cache | grep ffff880215802e00
kmem -f c40425b0
kmem -p c25a9c00
kmem -p 1fe0000
kmem -v
kmem -V
kmem -h
kmem -c
kmem -C
kmem -s
kmem -S arp_cache
kmem -n
kmem -g 4080
crash> help kmem
NAME
kmem - kernel memory
SYNOPSIS
kmem [-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]
[[-s|-S] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]
DESCRIPTION
This command displays information about the use of kernel memory.
-f displays the contents of the system free memory headers.
also verifies that the page count equals nr_free_pages.
-F same as -f, but also dumps all pages linked to that header.
-c walks through the page_hash_table and verifies page_cache_size.
-C same as -c, but also dumps all pages in the page_hash_table.
-i displays general memory usage information
-v displays the mapped virtual memory regions allocated by vmalloc().
-V displays the kernel vm_stat table if it exists, the cumulative
page_states counter values if they exist, and/or the cumulative
vm_event_states counter values if they exist.
-n display memory node data (if supported).
-z displays per-zone memory statistics.
-o displays each cpu's offset value that is added to per-cpu symbol
values to translate them into kernel virtual addresses.
-h display the address of hugepage hstate array entries, along with
their hugepage size, total and free counts, and name.
-p displays basic information about each page structure in the system
mem_map[] array, made up of the page struct address, its associated
physical address, the page.mapping, page.index, page._count and
page.flags fields.
-m member similar to -p, but displays page structure contents specified by
a comma-separated list of one or more struct page members. The
"flags" member will always be expressed in hexadecimal format, and
the "_count" and "_mapcount" members will always be expressed
in decimal format. Otherwise, all other members will be displayed
in hexadecimal format unless the output radix is 10 and the member
is a signed/unsigned integer. Members that are data structures may
be specified either by the data structure's member name, or expanded
to specify a member of the data structure. For example, "-m lru"
refers to a list_head data structure, and both the list_head.next
and list_head.prev pointer values will be displayed, whereas if
"-m lru.next" is specified, just the list_head.next value will
be displayed.
-s displays basic kmalloc() slab data.
-S displays all kmalloc() slab data, including all slab objects,
and whether each object is in use or is free. If CONFIG_SLUB,
slab data for each per-cpu slab is displayed, along with the
address of each kmem_cache_node, its count of full and partial
slabs, and a list of all tracked slabs.
slab when used with -s or -S, limits the command to only the slab cache
of name "slab". If the slab argument is "list", then
all slab cache names and addresses are listed.
-I slab when used with -s or -S, one or more slab cache names in a
comma-separated list may be specified as slab caches to ignore.
their hugepage size, total and free counts, and name.
-g displays the enumerator value of all bits in the page structure's
"flags" field.
flags when used with -g, translates all bits in this hexadecimal page
structure flags value into its enumerator values.
-P declares that the following address argument is a physical address.
address when used without any flag, the address can be a kernel virtual,
or physical address; a search is made through the symbol table,
the kmalloc() slab subsystem, the free list, the page_hash_table,
the vmalloc() region subsystem, the current set of task_structs
and kernel stacks, and the mem_map array. If found in any of
those areas, the information will be dumped in the same manner as
if the location-specific flags were used; if contained within a
curent task_struct or kernel stack, that task's context will be
displayed.
address when used with -s or -S, searches the kmalloc() slab subsystem
for the slab containing of this virtual address, showing whether
it is in use or free.
address when used with -f, the address can be either a page pointer,
a physical address, or a kernel virtual address; the free_area
header containing the page (if any) is displayed.
address when used with -p, the address can be either a page pointer, a
physical address, or a kernel virtual address; its basic mem_map
page information is displayed.
address when used with -m, the address can be either a page pointer, a
physical address, or a kernel virtual address; the specified
members of the associated page struct are displayed.
address when used with -c, the address must be a page pointer address;
the page_hash_table entry containing the page is displayed.
address when used with -l, the address must be a page pointer address;
the page address is displayed if it is contained with the list.
address when used with -v, the address can be a mapped kernel virtual
address or physical address; the mapped region containing the
address is displayed.
All address arguments above must be expressed in hexadecimal format.
查看内存总体使用情况
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 970517 3.7 GB ----
FREE 44818 175.1 MB 4% of TOTAL MEM
USED 925699 3.5 GB 95% of TOTAL MEM
SHARED 21531 84.1 MB 2% of TOTAL MEM
BUFFERS 527 2.1 MB 0% of TOTAL MEM
CACHED 79502 310.6 MB 8% of TOTAL MEM
SLAB 11424 44.6 MB 1% of TOTAL MEM
TOTAL HUGE 0 0 ----
HUGE FREE 0 0 0% of TOTAL HUGE
TOTAL SWAP 1015807 3.9 GB ----
SWAP USED 0 0 0% of TOTAL SWAP
SWAP FREE 1015807 3.9 GB 100% of TOTAL SWAP
COMMIT LIMIT 1501065 5.7 GB ----
COMMITTED 296049 1.1 GB 19% of TOTAL LIMIT
crash> kmem -v
VMAP_AREA VM_STRUCT ADDRESS RANGE SIZE
ffff88013b03d000 ffff88013b005080 ffffc90000000000 - ffffc90000002000 8192
ffff88013b03d180 ffff88013b005140 ffffc90000002000 - ffffc90000403000 4198400
ffff88013b03d200 ffff88013b005180 ffffc90000403000 - ffffc90000406000 12288
ffff88013b03d280 ffff88013b0051c0 ffffc90000406000 - ffffc90000607000 2101248
ffff88013b03d300 ffff88013b005200 ffffc90000607000 - ffffc90000618000 69632
ffff88013b03d400 ffff88013b005240 ffffc90000618000 - ffffc90000629000 69632
ffff88013b03da80 ffff88013b0055c0 ffffc9000062a000 - ffffc9000062c000 8192
ffff88013959de00 ffff8801395d68c0 ffffc9000062c000 - ffffc9000062e000 8192
ffff880135c3cb80 ffff880135c9c100 ffffc9000062e000 - ffffc90000630000 8192
ffff88013b03da00 ffff88013b005500 ffffc90000630000 - ffffc90000640000 65536
ffff880035ca7300 ffff880036726240 ffffc90000640000 - ffffc90000642000 8192
ffff88013959df80 ffff8801395d6dc0 ffffc90000642000 - ffffc90000644000 8192
ffff880035ca7380 ffff880036726280 ffffc90000644000 - ffffc90000646000 8192
ffff880035ca7400 ffff880036726140 ffffc90000646000 - ffffc90000648000 8192
ffff8801390e1b00 ffff8801390de600 ffffc90000648000 - ffffc90000689000 266240
ffff8801390e1b80 ffff8801390de640 ffffc90000689000 - ffffc9000070a000 528384
ffff8801390e1c80 ffff8801390de680 ffffc9000070a000 - ffffc9000071b000 69632
ffff8801390e1d80 ffff8801390de6c0 ffffc9000071b000 - ffffc9000072c000 69632
ffff880139336200 ffff880137f6a100 ffffc9000072c000 - ffffc90000735000 36864
ffff880135c3ca00 ffff880135c44280 ffffc90000735000 - ffffc9000073a000 20480
ffff880135c3c880 ffff880135c44200 ffffc9000073a000 - ffffc9000075b000 135168
ffff880035ca7580 ffff880036726380 ffffc9000075c000 - ffffc9000075e000 8192
ffff880036728600 ffff880135e973c0 ffffc9000075e000 - ffffc90000760000 8192
ffff880035a1cb00 ffff880139222f80 ffffc90000760000 - ffffc90000762000 8192
ffff880138282400 ffff880138329b00 ffffc90000762000 - ffffc90000764000 8192
ffff880138282780 ffff8800ba61ba00 ffffc90000764000 - ffffc90000766000 8192
ffff8800ba6cb280 ffff88013573ef40 ffffc90000766000 - ffffc90000768000 8192
ffff8800367d0600 ffff880035c098c0 ffffc90000769000 - ffffc9000076b000 8192
ffff8800367d0800 ffff880035c09180 ffffc9000076b000 - ffffc9000076d000 8192
ffff8800ba7fa800 ffff88013830cb40 ffffc9000076d000 - ffffc9000076f000 8192
ffff8800366b9580 ffff8800b8575fc0 ffffc9000076f000 - ffffc90000771000 8192
ffff8800366b9700 ffff8800b8575c80 ffffc90000771000 - ffffc90000773000 8192
ffff880135caaa80 ffff880139222180 ffffc90000780000 - ffffc90000792000 73728
ffff880035a1cc00 ffff880135832340 ffffc900007c6000 - ffffc900007cf000 36864
ffff88003651dc80 ffff880135e975c0 ffffc900007d0000 - ffffc900007d9000 36864
ffff880035a1cf00 ffff880035c01a80 ffffc900007d9000 - ffffc900007e2000 36864
ffff8800bb8a4200 ffff880035c01840 ffffc900007e2000 - ffffc900007e5000 12288
ffff8800bb8a4300 ffff8801358324c0 ffffc900007e5000 - ffffc900007ed000 32768
ffff8800bb8a4f80 ffff880135832280 ffffc900007ed000 - ffffc900007f0000 12288
ffff880035a1ce80 ffff880035c01640 ffffc90000800000 - ffffc90000841000 266240
ffff8800bb8a4000 ffff880035c01e80 ffffc90000841000 - ffffc90000942000 1052672
ffff8800bb8a4280 ffff8801358323c0 ffffc90000942000 - ffffc900015c3000 13111296
ffff8801390e1580 ffff880135832640 ffffc900015c3000 - ffffc90001984000 3936256
ffff880036728900 ffff880138329e80 ffffc9000199e000 - ffffc900019be000 131072
ffff880138282300 ffff880138329b40 ffffc90001b00000 - ffffc90001bf9000 1019904
ffff880139534c00 ffff8801395370c0 ffffc90010000000 - ffffc90018001000 134221824
ffff880135c26e00 ffff8801394b89c0 ffffe8ffffc00000 - ffffe8ffffe00000 2097152
ffff880036461800 ffff88003650f240 ffffffffc0000000 - ffffffffc0020000 131072
ffff8801390d4d00 ffff880036414540 ffffffffc0020000 - ffffffffc0027000 28672
ffff880036728a00 ffff88003670e000 ffffffffc0027000 - ffffffffc002c000 20480
ffff880036461d00 ffff880036414600 ffffffffc002d000 - ffffffffc0033000 24576
ffff8801390d4d80 ffff880036414580 ffffffffc0033000 - ffffffffc003a000 28672
ffff88003651db00 ffff880135f0eac0 ffffffffc003e000 - ffffffffc0049000 45056
ffff880035ca7800 ffff88003670e280 ffffffffc0049000 - ffffffffc004e000 20480
ffff88003651db80 ffff880135f0ed40 ffffffffc0050000 - ffffffffc0057000 28672
ffff880036642c80 ffff880135efd0c0 ffffffffc0057000 - ffffffffc0093000 245760
ffff880035ca7100 ffff880135efdd00 ffffffffc0093000 - ffffffffc00a2000 61440
ffff880035ca7200 ffff880036726200 ffffffffc00a2000 - ffffffffc00a9000 28672
ffff8801390d4e80 ffff880135efdb80 ffffffffc00ab000 - ffffffffc00b5000 40960
ffff880035ca7d00 ffff880036726100 ffffffffc00b5000 - ffffffffc00ba000 20480
ffff880035ca7f00 ffff880036726640 ffffffffc00ba000 - ffffffffc00bf000 20480
ffff880036728b00 ffff880035c01dc0 ffffffffc00bf000 - ffffffffc00c4000 20480
ffff880036728300 ffff880135f0e140 ffffffffc00c4000 - ffffffffc00c9000 20480
ffff880036728c80 ffff880035c01d80 ffffffffc00c9000 - ffffffffc00ce000 20480
ffff880036728e00 ffff880035c01fc0 ffffffffc00ce000 - ffffffffc00d3000 20480
ffff880036728480 ffff880135f0e200 ffffffffc00d4000 - ffffffffc00de000 40960
ffff880036728500 ffff8800364b6900 ffffffffc00e1000 - ffffffffc00e6000 20480
ffff880036728400 ffff880035c01980 ffffffffc00e7000 - ffffffffc00ec000 20480
ffff880035ca7e00 ffff880135efda40 ffffffffc00ec000 - ffffffffc0114000 163840
ffff88003651da80 ffff8800364b6c40 ffffffffc0114000 - ffffffffc0119000 20480
ffff880036728000 ffff880135f0e240 ffffffffc011d000 - ffffffffc0126000 36864
ffff880036728280 ffff880135e97700 ffffffffc0126000 - ffffffffc0182000 376832
ffff880036728380 ffff880035c01e00 ffffffffc0182000 - ffffffffc019c000 106496
ffff8800baade900 ffff8800367266c0 ffffffffc019c000 - ffffffffc01a1000 20480
ffff880036728700 ffff8800364b6ac0 ffffffffc01a1000 - ffffffffc01ae000 53248
ffff880035a1ca80 ffff880135832540 ffffffffc01b0000 - ffffffffc01eb000 241664
ffff8800bab99980 ffff880135789a80 ffffffffc01eb000 - ffffffffc01f0000 20480
ffff8800367d0000 ffff8800364b6440 ffffffffc01f0000 - ffffffffc01f7000 28672
ffff8800367d0500 ffff88013b0052c0 ffffffffc01f7000 - ffffffffc01ff000 32768
ffff88003651d600 ffff8800364765c0 ffffffffc0202000 - ffffffffc020e000 49152
ffff8800baade500 ffff880036726ec0 ffffffffc020e000 - ffffffffc02fe000 983040
ffff880135d6e980 ffff88003650f480 ffffffffc02fe000 - ffffffffc0305000 28672
ffff88003651d000 ffff8800ba5bc200 ffffffffc0305000 - ffffffffc030d000 32768
ffff8800ba7fa700 ffff880135df3c80 ffffffffc030d000 - ffffffffc0313000 24576
ffff88003651d500 ffff8800ba5bc080 ffffffffc0313000 - ffffffffc031f000 49152
ffff880135caae80 ffff880135819740 ffffffffc031f000 - ffffffffc0331000 73728
ffff8800ba5cca00 ffff8800ba5b0900 ffffffffc0343000 - ffffffffc034e000 45056
ffff8800367d0380 ffff88003667ba80 ffffffffc0355000 - ffffffffc0360000 45056
ffff88003651d680 ffff8800ba5a3900 ffffffffc0360000 - ffffffffc036e000 57344
ffff880135caae00 ffff88013830c800 ffffffffc036e000 - ffffffffc0374000 24576
ffff88003651d580 ffff8800ba5bc5c0 ffffffffc0375000 - ffffffffc0397000 139264
ffff8800bab99500 ffff880136deca40 ffffffffc0397000 - ffffffffc039d000 24576
ffff8800ba7fa100 ffff88013830cf80 ffffffffc039d000 - ffffffffc03a2000 20480
ffff8800ba7fa380 ffff88013830cb80 ffffffffc03a2000 - ffffffffc03a7000 20480
ffff8800bab99580 ffff880136decac0 ffffffffc03ab000 - ffffffffc03b0000 20480
ffff8800bab99e00 ffff880139222e40 ffffffffc03b0000 - ffffffffc03b5000 20480
ffff8800bab99180 ffff88013573e680 ffffffffc03b5000 - ffffffffc03ba000 20480
ffff8800ba7fa280 ffff8800ba65fc80 ffffffffc03bf000 - ffffffffc03d2000 77824
ffff8800ba6c1500 ffff8800ba653f40 ffffffffc03d2000 - ffffffffc03d7000 20480
ffff880138317880 ffff8800b8590fc0 ffffffffc03d7000 - ffffffffc03e1000 40960
ffff880138317380 ffff880137894240 ffffffffc03e1000 - ffffffffc03ea000 36864
ffff8800bab99d00 ffff8800ba7ab0c0 ffffffffc03eb000 - ffffffffc03f1000 24576
ffff8800ba5cce00 ffff8800ba653cc0 ffffffffc03f1000 - ffffffffc03f6000 20480
ffff880036728080 ffff880138329ec0 ffffffffc03f6000 - ffffffffc0406000 65536
NODE: 0 ZONE: 0 ADDR: ffff88013ffd7000 NAME: "DMA"
SIZE: 4095 PRESENT: 3998 MIN/LOW/HIGH: 69/86/103
VM_STAT:
NR_FREE_PAGES: 3859
NR_ALLOC_BATCH: 17
NR_INACTIVE_ANON: 18
NR_ACTIVE_ANON: 3
NR_INACTIVE_FILE: 23
NR_ACTIVE_FILE: 11
NR_UNEVICTABLE: 0
NR_MLOCK: 0
NR_ANON_PAGES: 1
NR_FILE_MAPPED: 38
NR_FILE_PAGES: 54
NR_FILE_DIRTY: 0
NR_WRITEBACK: 0
NR_SLAB_RECLAIMABLE: 3
NR_SLAB_UNRECLAIMABLE: 13
NR_PAGETABLE: 2
NR_KERNEL_STACK: 1
NR_UNSTABLE_NFS: 0
NR_BOUNCE: 0
NR_VMSCAN_WRITE: 0
NR_VMSCAN_IMMEDIATE: 0
NR_WRITEBACK_TEMP: 0
NR_ISOLATED_ANON: 0
NR_ISOLATED_FILE: 0
NR_SHMEM: 20
NR_DIRTIED: 0
NR_WRITTEN: 0
NUMA_HIT: 500
NUMA_MISS: 0
NUMA_FOREIGN: 0
NUMA_INTERLEAVE_HIT: 0
NUMA_LOCAL: 500
NUMA_OTHER: 0
WORKINGSET_REFAULT: 0
WORKINGSET_ACTIVATE: 0
WORKINGSET_NODERECLAIM: 0
NR_ANON_TRANSPARENT_HUGEPAGES: 0
NR_FREE_CMA_PAGES: 0
ALL_UNRECLAIMABLE: no PAGES_SCANNED: 0
NODE: 0 ZONE: 1 ADDR: ffff88013ffd7800 NAME: "DMA32"
SIZE: 1044480 PRESENT: 782320 MIN/LOW/HIGH: 12551/15688/18826
VM_STAT:
NR_FREE_PAGES: 25799
NR_ALLOC_BATCH: 689
NR_INACTIVE_ANON: 3351
NR_ACTIVE_ANON: 128561
NR_INACTIVE_FILE: 44125
NR_ACTIVE_FILE: 15028
NR_UNEVICTABLE: 0
NR_MLOCK: 0
NR_ANON_PAGES: 16547
NR_FILE_MAPPED: 11923
NR_FILE_PAGES: 60342
NR_FILE_DIRTY: 9
NR_WRITEBACK: 0
NR_SLAB_RECLAIMABLE: 3432
NR_SLAB_UNRECLAIMABLE: 2453
NR_PAGETABLE: 1362
NR_KERNEL_STACK: 163
NR_UNSTABLE_NFS: 0
NR_BOUNCE: 0
NR_VMSCAN_WRITE: 0
NR_VMSCAN_IMMEDIATE: 0
NR_WRITEBACK_TEMP: 0
NR_ISOLATED_ANON: 0
NR_ISOLATED_FILE: 0
NR_SHMEM: 1189
NR_DIRTIED: 32633
NR_WRITTEN: 24962
NUMA_HIT: 4652757
NUMA_MISS: 0
NUMA_FOREIGN: 0
NUMA_INTERLEAVE_HIT: 0
NUMA_LOCAL: 4652757
NUMA_OTHER: 0
WORKINGSET_REFAULT: 247
WORKINGSET_ACTIVATE: 247
WORKINGSET_NODERECLAIM: 0
NR_ANON_TRANSPARENT_HUGEPAGES: 223
NR_FREE_CMA_PAGES: 0
ALL_UNRECLAIMABLE: no PAGES_SCANNED: 0
NODE: 0 ZONE: 2 ADDR: ffff88013ffd8000 NAME: "Normal"
SIZE: 262144 MIN/LOW/HIGH: 4275/5343/6412
VM_STAT:
NR_FREE_PAGES: 15160
NR_ALLOC_BATCH: -7
NR_INACTIVE_ANON: 1176
NR_ACTIVE_ANON: 49711
NR_INACTIVE_FILE: 14297
NR_ACTIVE_FILE: 4295
NR_UNEVICTABLE: 0
NR_MLOCK: 0
NR_ANON_PAGES: 5814
NR_FILE_MAPPED: 4618
NR_FILE_PAGES: 19633
NR_FILE_DIRTY: 0
NR_WRITEBACK: 0
NR_SLAB_RECLAIMABLE: 2752
NR_SLAB_UNRECLAIMABLE: 2771
NR_PAGETABLE: 474
NR_KERNEL_STACK: 102
NR_UNSTABLE_NFS: 0
NR_BOUNCE: 0
NR_VMSCAN_WRITE: 0
NR_VMSCAN_IMMEDIATE: 0
NR_WRITEBACK_TEMP: 0
NR_ISOLATED_ANON: 0
NR_ISOLATED_FILE: 0
NR_SHMEM: 1041
NR_DIRTIED: 5096
NR_WRITTEN: 3678
NUMA_HIT: 1618569
NUMA_MISS: 0
NUMA_FOREIGN: 0
NUMA_INTERLEAVE_HIT: 15350
NUMA_LOCAL: 1618569
NUMA_OTHER: 0
WORKINGSET_REFAULT: 0
WORKINGSET_ACTIVATE: 0
WORKINGSET_NODERECLAIM: 0
NR_ANON_TRANSPARENT_HUGEPAGES: 86
NR_FREE_CMA_PAGES: 0
ALL_UNRECLAIMABLE: no PAGES_SCANNED: 0
NODE: 0 ZONE: 3 ADDR: ffff88013ffd8800 NAME: "Movable"
[unpopulated]
