KVM制作image
2021-01-06 本文已影响0人
TEYmL
[toc]
使用kvm创建cloud image给OpenStack集群
制作image
环境：Ubuntu18.04 server版Workstations虚拟机
关机状态在设置虚拟机支持虚拟化
image.png
kvm环境准备
查看是否已经支持虚拟化
root@ubuntu:/home/k8snode2# egrep -o '(vmx|svm)' /proc/cpuinfo
vmx
vmx
vmx
vmx
有输出代表支持虚拟化
安装kvm
apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils libguestfs-tools
验证kvm是否可用
root@ubuntu:/home/k8snode2# kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used
root@ubuntu:/home/k8snode2# lsmod | grep kvm
kvm_intel 217088 0
kvm 610304 1 kvm_intel
irqbypass 16384 1 kvm
root@ubuntu:/home/k8snode2# virsh -c qemu:///system list
Id Name State
----------------------------------------------------
root@ubuntu:/home/k8snode2#
image制作
创建qcow2文件
root@ubuntu:/home/k8snode2# chown libvirt-qemu:kvm mini.iso
root@ubuntu:/home/k8snode2# qemu-img create -f qcow2 bionic.qcow2 5G
Formatting 'bionic.qcow2', fmt=qcow2 size=5368709120 cluster_size=65536 lazy_refcounts=off refcount_bits=16
root@ubuntu:/home/k8snode2# chown libvirt-qemu:kvm bionic.qcow2
启动实例
命令
virt-install --virt-type kvm --name bionic --ram 1024 --cdrom=mini.iso --disk bionic.qcow2,bus=virtio,size=10,format=qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole --os-type=linux --os-variant=ubuntu18.04
执行结果
root@ubuntu:/home/k8snode2# virt-install --virt-type kvm --name bionic --ram 1024 --cdrom=mini.iso --disk bionic.qcow2,bus=virtio,size=10,format=qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole --os-type=linux --os-variant=ubuntu18.04
Starting install...
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
root@ubuntu:/home/k8snode2#
安装系统
使用vnc连接
image.png
安装完成之后可以正常登陆
image.png
安装软件
下载fio，vim，lsscsi等软件
将虚拟机image从kvm导出
克隆kvm虚拟机
root@ubuntu:/home/k8snode2# virt-clone -o bionic -n bionic2 -f bionic2.qcow2
Allocating 'bionic2.qcow2' | 5.0 GB 00:00:39
Clone 'bionic2' created successfully.
清理虚拟机MAC地址
root@ubuntu:/home/k8snode2# virt-sysprep -d bionic
[ 0.0] Examining the guest ...
virt-sysprep: error: libguestfs error: guestfs_launch failed.
This usually means the libguestfs appliance failed to start or crashed.
Do:
export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again. For further information, read:
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
If reporting bugs, run virt-sysprep with debugging enabled and include the
complete output:
virt-sysprep -v -x [...]
有error，暂时跳过
undefined kvm虚拟机
root@ubuntu:/home/k8snode2# virsh undefine bionic
Domain bionic has been undefined
root@ubuntu:/home/k8snode2#
将image交给OpenStack使用
将image上传到OpenStack controller节点
root@ubuntu:/home/k8snode2# scp bionic.qcow2 openstcontroller@10.203.1.99:/home/openstcontroller
The authenticity of host '10.203.1.99 (10.203.1.99)' can't be established.
ECDSA key fingerprint is SHA256:uPpPUyTFBbgsOZ7P5J7twCYn+x9ADHtOw99YS+1a4mQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.203.1.99' (ECDSA) to the list of known hosts.
openstcontroller@10.203.1.99's password:
bionic.qcow2 100% 2735MB 21.3MB/s 02:08
root@ubuntu:/home/k8snode2#
制作OpenStack image
openstcontroller@openstcontroller:~$ openstack image create "bionic" --file bionic.qcow2 --public --disk-format qcow2
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | fac88859bf9590af56d25480cfe10592 |
| container_format | bare |
| created_at | 2020-09-29T01:59:15Z |
| disk_format | qcow2 |
| file | /v2/images/bccef1fe-15e0-42d2-a2c7-bfd14c652d25/file |
| id | bccef1fe-15e0-42d2-a2c7-bfd14c652d25 |
| min_disk | 0 |
| min_ram | 0 |
| name | bionic |
| owner | a4998d69c0434c0cb903e40c477519c9 |
| properties | os_hash_algo='sha512', os_hash_value='4cf166d08cba9d141f232c0731eb6930faebaca47c12f427cff6b9390bed927eb46b07db1b377208f84703309405527d2540132e7d8270a90f30772aaed8582f', os_hidden='False' |
| protected | False |
| schema | /v2/schemas/image |
| size | 2868183040 |
| status | active |
| tags | |
| updated_at | 2020-09-29T02:01:42Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
创建实例
attach LINSTOR volume并进行I/O test
创建Windows image
使用kvm制作image
iso准备
除了Windows ISO之外需要下载VirtIO driver，因为win不支持virtio驱动，而openstack管理虚拟机是需要virtio相关驱动
下载链接
https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso
创建qcow2文件
root@ubuntu:/home/k8snode2# qemu-img create -f qcow2 ws2012.qcow2 15G
Formatting 'ws2012.qcow2', fmt=qcow2 size=16106127360 cluster_size=65536 lazy_refcounts=off refcount_bits=16
root@ubuntu:/home/k8snode2# ls
bionic2.qcow2 bionic.qcow2 cn_windows_server_2012_vl_x64_dvd_917962.iso mini.iso ubuntuserver18.img virtio-win.iso ws2012.qcow2
root@ubuntu:/home/k8snode2# chown libvirt-qemu:kvm ws2012.qcow2
启动kvm虚拟机
加载驱动
安装好的虚拟机缺少设备驱动，右击计算机，点击管理，找到设备管理器
image.png
点击选择查找驱动，使用VirtIO driver进行安装
image.png
使用iometer进行测试
将image上传到OpenStack集群
OpenStack创建Windows instance
创建image
openstcontroller@openstcontroller:~$ openstack image create "ws2012" --file ws2012.qcow2 --public --disk-format qcow2
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | 7845608fa7d1ce7beb67097c3a06d0ba |
| container_format | bare |
| created_at | 2020-10-10T05:59:35Z |
| disk_format | qcow2 |
| file | /v2/images/4b9449d5-e044-4691-84bf-8f57d445644d/file |
| id | 4b9449d5-e044-4691-84bf-8f57d445644d |
| min_disk | 0 |
| min_ram | 0 |
| name | ws2012 |
| owner | a4998d69c0434c0cb903e40c477519c9 |
| properties | os_hash_algo='sha512', os_hash_value='f6a04e4254fe38b064256802244a67c0324d09da132ba2b6dcb15f48785eade594fceeccde3e22197617cd9564b46b1f7fd35bfb986981f33055dcdd35448df9', os_hidden='False' |
| protected | False |
| schema | /v2/schemas/image |
| size | 10044440576 |
| status | active |
| tags | |
| updated_at | 2020-10-10T06:09:12Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
openstcontroller@openstcontroller:~$
创建flavor
openstcontroller@openstcontroller:~$ openstack flavor create --id 4 --vcpus 2 --ram 2048 --disk 15 win
+----------------------------+-------+
| Field | Value |
+----------------------------+-------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 15 |
| id | 4 |
| name | win |
| os-flavor-access:is_public | True |
| properties | |
| ram | 2048 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 2 |
+----------------------------+-------+
创建volume
image.png
创建instance
使用Windows instance
查看当前磁盘信息
系统盘15G
image.png
创建 LINSTOR 资源并attach
image.png
重新扫描之后能够看到新attach的磁盘
image.png
使用iometer进行测试
新建卷
image.png
image.png
image.png
测试
image.png
系统空间不够的问题
问题描述
在create OpenStack image的时候，提示空间不足
openstcontroller@openstcontroller:~$ openstack image create "ws2012" --file ws2012.qcow2 --public --disk-format qcow2
HTTP 413 Request Entity Too Large: Image storage media is full: There is not enough disk space on the image storage media.
解决步骤
查看系统disk使用情况，发现系统盘/dev/sda1的空间才25G，而且已经使用了92%
openstcontroller@openstcontroller:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 796M 1.2M 795M 1% /run
/dev/sda1 25G 22G 1.9G 92% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs 796M 0 796M 0% /run/user/1000
此时有两种方法可以解决这个问题，第一是给系统盘扩容，第二是增加新的disk，挂载在OpenStack glance 存储 image的路径，尝试发现VMware无法直接改变虚拟机系统盘的大小，扩容的话也是需要增加新的disk来进行，后续经过考虑，新增disk并挂载在OpenStack glance 存储 image的路径
将新加的disk创建ext4文件系统
root@openstcontroller:~# mkfs.ext4 /dev/sdb
mke2fs 1.44.1 (24-Mar-2018)
Creating filesystem with 5242880 4k blocks and 1310720 inodes
Filesystem UUID: aadcfabc-ee61-4c60-a7d2-de6d12356102
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
将image路径的文件先同步到tmp
root@openstcontroller:~# rsync -avp /var/lib/glance/images /tmp/
sending incremental file list
images/
images/bccef1fe-15e0-42d2-a2c7-bfd14c652d25
sent 2,868,883,451 bytes received 39 bytes 23,419,457.06 bytes/sec
total size is 2,868,183,040 speedup is 1.00
将新创建的文件系统挂载到image路径
root@openstcontroller:~# mount /dev/sdb /var/lib/glance/images
将tmp路径下的image同步回来
root@openstcontroller:~# rsync -avp /tmp/images/ /var/lib/glance/images/
sending incremental file list
./
bccef1fe-15e0-42d2-a2c7-bfd14c652d25
sent 2,868,883,436 bytes received 38 bytes 16,535,351.44 bytes/sec
total size is 2,868,183,040 speedup is 1.00
