虚拟化运维--Docker Machine(十八)
2019-10-29 本文已影响0人
无剑_君
一、Docker Machine简介
Docker Machine
Docker Machine 是Docker官方编排项目之一，使用go语言编写的，使用不同引擎在多种平台上快速的安装Docker环境，Docker Machine 是一个工具，它允许你在虚拟宿主机上安装Docker，并使用docker-machine命令管理这个宿主机，可以使用Docker Machine在本地的MAC或者windows box、公司网络，数据中心或者AWS这样的云提供商上创建docker。
使用docker-machine命令，可以启动、审查、停止、重启托管的docker 也可以升级Docker客户端和守护程序并配置docker客户端和宿主机通信。
Docker Machine 也可以集中管理所以得docker主机。
开源地址:https://github.com/docker/machine。
Docker Machine 使你能够在各种 Linux 上配置多个远程 Docker 宿主机。
此外，Machine 允许你在较早的 Mac 或 Windows 系统上运行 Docker。
Docker Engine Linux 系统上原生地运行。如果你有一个 Linux 作为你的主系统，并且想要运行 docker 命令，所有你需要做的就是下载并安装 Docker Engine 。然而，如果你想要在网络上、云中甚至本地配置多个 Docker 宿主机有一个有效的方式，你需要 Docker Machine。
无论你的主系统是 Mac、Windows 还是 Linux，你都可以在其上安装 Docker Machine，并使用 docker-machine 命令来配置和管理大量的 Docker 宿主机。它会自动创建宿主机、在其上安装 Docker Engine 、然后配置 docker 客户端。每个被管理的宿主机（“machine”）是 Docker 宿主机和配置好的客户端的结合。
二、Docker和Docker Machine之间的区别
当人们说“Docker”时，他们通常是指 Docker Engine，它是一个客户端 - 服务器应用程序，由 Docker 守护进程、一个REST API指定与守护进程交互的接口、和一个命令行接口（CLI）与守护进程通信（通过封装REST API）。Docker Engine 从 CLI 中接受docker 命令，例如 docker run <image>、docker ps 来列出正在运行的容器、docker images 来列出镜像，等等。
Docker Machine 是一个用于配置和管理你的宿主机（上面具有 Docker Engine 的主机）的工具。通常，你在你的本地系统上安装 Docker Machine。Docker Machine有自己的命令行客户端 docker-machine 和 Docker Engine 客户端 docker。你可以使用 Machine 在一个或多个虚拟系统上安装 Docker Engine。
这些虚拟系统可以是本地的（就像你在 Mac 或 Windows 上使用 Machine 在 VirtualBox 中安装和运行 Docker Engine 一样）或远程的（就像你使用 Machine 在云提供商上 provision Dockerized 宿主机一样）。Dockerized 宿主机本身可以认为是，且有时就称为，被管理的“machines”。
Docker Machine 支持多种后端驱动：
物理机
虚拟机
云平台
三、主机安装
- 安装Docker Machine前需先安装Docker（主机）
安装文档：https://docs.docker.com/machine/install-machine/
[root@localhost ~]# vi /lib/systemd/system/docker.service
# 将 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock修改为以下内容
# ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock -H tcp://0.0.0.0:7654
- 主机安装
# 下载文件
[root@master ~]# base=https://github.com/docker/machine/releases/download/v0.16.0
[root@master ~]# curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 617 0 617 0 0 496 0 --:--:-- 0:00:01 --:--:-- 497
100 26.8M 100 26.8M 0 0 4880 0 1:36:11 1:36:10 0:00:01 3498
# 移动文件
[root@master ~]# sudo mv /tmp/docker-machine /usr/local/bin/docker-machine
# 添加权限
[root@master ~]# chmod +x /usr/local/bin/docker-machine
# 获取版本信息
[root@master ~]# docker-machine version
docker-machine version 0.16.0, build 702c267f
- 服务器节点规划
| 主机名 | 操作系统 | IP地址 |
|---|---|---|
| master | Centos 7.4-x86_64 | 192.168.247.130 |
| node1 | Centos 7.4-x86_64 | 192.168.247.131 |
| node2 | Centos 7.4-x86_64 | 192.168.247.132 |
- 配置主机间的SSH免密
# 生成keys并配置可以免密登录主机
[root@master ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:a3XywBhjYNEZCmUPmpsiXvTu1KBGuHoYK7bAlzq/m+4 root@master
The key's randomart image is:
+---[RSA 2048]----+
| ..Bo.o |
| * =o |
| .o . = |
| o .o . = |
|...ooo S = . |
|+.+.+ o o = |
|.B = o .o . |
|*o= + . |
|++*Eo. |
+----[SHA256]-----+
# 将keys拷贝到 node1上去
[root@master ~]# ssh-copy-id root@192.168.247.131
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.247.131 (192.168.247.131)' can't be established.
ECDSA key fingerprint is SHA256:SoQsbaxpRbTyA/WQ6jE7Z7y89wYYtyjyCDKN4PvE58M.
ECDSA key fingerprint is MD5:05:56:09:87:03:18:df:dc:a7:7f:29:c3:52:f4:91:b7.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.247.131's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.247.131'"
and check to make sure that only the key(s) you wanted were added.
# 测试免密登录
[root@master ~]# ssh root@192.168.247.131
Last login: Mon Oct 28 00:03:35 2019 from 192.168.247.1
[root@node1 ~]#
# 查看machine
[root@master ~]# docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
- 用docker machine创建docker host(远程主机没有创建Docker)
# 创建machine （主机名为：node1-machine） 因为我们是往普通的 Linux 中部署 docker，所以使用 generic driver。
# generic:创建一个machines通过SSH使用已经存在的虚拟机或是主机
[root@master ~]# docker-machine create --driver generic --generic-ip-address=192.168.247.131 --generic-ssh-key=/root/.ssh/id_rsa --generic-ssh-user=root --generic-ssh-port=22 node1-machine
Creating CA: /root/.docker/machine/certs/ca.pem
Creating client certificate: /root/.docker/machine/certs/cert.pem
Running pre-create checks...
Creating machine...
(node1-machine) No SSH key specified. Assuming an existing key at the default location.
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with centos...
l^HCopying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.247.131:2376": dial tcp 192.168.247.131:2376: connect: no route to host
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.
Machine drivers：
docker-machine 创建虚拟机需要驱动，创建方式及常用驱动如下:
docker-machine create -d 驱动名称 虚拟机名称 // -d也可以改成--driver，两者等价
微软hyper-v驱动名称：hyperv
VirtualBox驱动名称：virtualbox
VMware vSphere驱动名称：vmwarevsphere
VMware Fusion驱动名称：vmwarefusion
VMware vCloud Air驱动名称： vmwarevcloudair
注意：这里会出现Error creating machine: Error running provisioning: error installing docker: 这样的错误
原因就是因为网络的原因，没有安装docker
# 登录到client查看配置项
[root@master ~]# ssh root@192.168.247.131
Last login: Mon Oct 28 06:51:56 2019 from 192.168.247.1
[root@node1-machine ~]# cat /etc/systemd/system/docker.service.d/10-machine.conf
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver overlay2 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=generic
Environment=
# 关闭远程机防火墙
[root@node1-machine ~]# systemctl stop firewalld
[root@node1-machine ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
# 创建成功执行ls查看
[root@master ~]# docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
node1-machine - generic Running tcp://192.168.247.131:2376 Unknown Unable to query docker version: Get https://192.168.247.131:2376/v1.15/version: dial tcp 192.168.247.131:2376: connect: no route to host
# 显示环境
[root@master ~]# docker-machine env node1-machine
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.247.131:2376"
export DOCKER_CERT_PATH="/root/.docker/machine/machines/node1-machine"
export DOCKER_MACHINE_NAME="node1-machine"
# Run this command to configure your shell:
# eval $(docker-machine env node1-machine)
注：-H tcp://0.0.0.0:2376 使docker daemon接受远程连接
--tlscert 对远程连接启用安全认证和加密
注：主机名hostname也会被更改
# 查看node1-machine的环境变量
[root@master ~]# docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
node1-machine - generic Running tcp://192.168.247.131:2376 v19.03.4
# 重启运程机
[root@master ~]# docker-machine restart node1-machine
# ssh远程连接
[root@master ~]# docker-machine ssh node1-machine
# 卸载node1-machine
[root@master ~]# docker-machine rm node1-machine
说明：
generic:创建一个machines通过SSH使用已经存在的虚拟机或是主机
如果docker没有运行在主机上，它将被自动安装.
它会更新主机的软件包（apt-get update, yum update）.
为了确保docker daemon的安全它会生成证书.
docker daemon 将会重启，因此所有正在运行的容器将会停止.
主机的hostname将被更改为machine name.
- 用docker machine创建docker host(远程主机已安装Docker)
卸载node1-machine，进行添加（必须）。
# 创建虚拟机(已安装docker)
[root@master ~]# docker-machine create --driver none --url=tcp://192.168.247.131:2376 node1-machine
四、VirtualBox 中运行Docker
- 安装 VirtualBox 虚拟机
# 安装virtualbox
[root@localhost ~]# cd /etc/yum.repos.d/
# 配置yum的源
[root@localhost yum.repos.d]# wget http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo
# 更新源
[root@localhost yum.repos.d]# yum upgrade
# 安装virtualbox 虚拟机
[root@localhost ~]#yum install -y VirtualBox-6.0
- 创建虚拟机Docker
# 创建 Docker Machine
[root@master ~]# docker-machine create --driver virtualbox linux
# 按提示运行：/sbin/vboxconfig,安装内核
[root@localhost ~]# rpm -qa |grep kernel
kernel-tools-3.10.0-1062.4.1.el7.x86_64
kernel-3.10.0-1062.4.1.el7.x86_64
kernel-3.10.0-957.el7.x86_64
kernel-tools-libs-3.10.0-1062.4.1.el7.x86_64
# 安装内核
[root@localhost ~]# sudo yum install kernel-devel
# 安装内核
[root@localhost ~]# sudo yum install kernel-headers
make-3.82-24.el7.x86_64
[root@localhost ~]# rpm -qa perl
perl-5.16.3-294.el7_6.x86_64
[root@localhost ~]# sudo yum install gcc
# 重启系统，查看内核
[root@localhost ~]# uname -sr
Linux 3.10.0-1062.4.1.el7.x86_64
# 重新加载virtualbox服务
[root@localhost ~]# sudo /sbin/vboxconfig
vboxdrv.sh: Stopping VirtualBox services.
vboxdrv.sh: Starting VirtualBox services.
vboxdrv.sh: Building VirtualBox kernel modules.
[root@localhost ~]# docker-machine create --driver virtualbox host1
Running pre-create checks...
Creating machine...
(host1) Copying /root/.docker/machine/cache/boot2docker.iso to /root/.docker/machine/machines/host1/boot2docker.iso...
(host1) Creating VirtualBox VM...
(host1) Creating SSH key...
(host1) Starting the VM...
(host1) Check network to re-create if needed...
(host1) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env host1
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
host1 - virtualbox Running tcp://192.168.99.102:2376 v19.03.4
# 用本机docker客户端指向host1的docker
# docker 与 docker-machine 通信，运行 docker-machine env <虚拟机名称> 来实现
[root@localhost ~]# eval $(docker-machine env host1)
[root@localhost ~]# docker info
Client:
Debug Mode: false
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 19.03.4
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.150-boot2docker
Operating System: Boot2Docker 19.03.4 (TCL 10.1)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 989.5MiB
Name: host1
ID: NLVA:WYHR:TW2W:7F6M:JDFZ:FJAF:G7PM:MSZG:SJOR:UVPC:YXNI:OVZU
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
provider=virtualbox
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
使用VirtualBox在本地创建计算机。此驱动程序要求在主机上安装VirtualBox 5+。使用VirtualBox 4.3+应该可以工作，但会发出警告。旧版本不起作用。
--driver virtualbox 选项表示我们要在 VirtualBox 的虚拟机里面部署 docker，最后的参数“linux” 是虚拟机的名称。这个命令会下载 boot2docker iso，它是个基于 Tiny Core Linux 的轻量级发行版，自带 Docker 程序，然后 docker-machine 命令会创建一个 VirtualBox 虚拟机（我们也可以选择其他的虚拟机软件）来运行这个 boot2docker 系统。
虚拟机磁盘的大小可以通过以下方式配置：
docker-machine create -d virtualbox --virtualbox-disk-size "100000" large
你也可以在创建时加上如下参数，来配置主机或者主机上的 Docker。
--engine-opt dns=114.114.114.114 配置 Docker 的默认 DNS
--engine-registry-mirror https://dockerhub.azk8s.cn 配置 Docker 的仓库镜像
--virtualbox-memory 2048 配置主机内存
--virtualbox-cpu-count 2 配置主机 CPU
- 运行 Docker 容器
完成配置后我们就可以在 VirtualBox 上运行 docker 容器了。
# host1主机IP
[root@localhost ~]# docker-machine ip host1
192.168.99.102
# 停止主机
[root@localhost ~]# docker-machine stop host1
# 启动主机
[root@localhost ~]# docker-machine start host1
# config命令
[root@localhost ~]# docker-machine config host1
--tlsverify
--tlscacert="/root/.docker/machine/machines/host1/ca.pem"
--tlscert="/root/.docker/machine/machines/host1/cert.pem"
--tlskey="/root/.docker/machine/machines/host1/key.pem"
-H=tcp://192.168.99.102:2376
# inspect命令
[root@localhost ~]# docker-machine inspect host1
{
"ConfigVersion": 3,
"Driver": {
"IPAddress": "192.168.99.102",
"MachineName": "host1",
"SSHUser": "docker",
"SSHPort": 45590,
"SSHKeyPath": "/root/.docker/machine/machines/host1/id_rsa",
"StorePath": "/root/.docker/machine",
"SwarmMaster": false,
"SwarmHost": "tcp://0.0.0.0:3376",
"SwarmDiscovery": "",
"VBoxManager": {},
"HostInterfaces": {},
"CPU": 1,
"Memory": 1024,
"DiskSize": 20000,
"NatNicType": "82540EM",
"Boot2DockerURL": "",
"Boot2DockerImportVM": "",
"HostDNSResolver": false,
"HostOnlyCIDR": "192.168.99.1/24",
"HostOnlyNicType": "82540EM",
"HostOnlyPromiscMode": "deny",
"UIType": "headless",
"HostOnlyNoDHCP": false,
"NoShare": false,
"DNSProxy": true,
"NoVTXCheck": false,
"ShareFolder": ""
},
"DriverName": "virtualbox",
"HostOptions": {
"Driver": "",
"Memory": 0,
"Disk": 0,
"EngineOptions": {
"ArbitraryFlags": [],
"Dns": null,
"GraphDir": "",
"Env": [],
"Ipv6": false,
"InsecureRegistry": [],
"Labels": [],
"LogLevel": "",
"StorageDriver": "",
"SelinuxEnabled": false,
"TlsVerify": true,
"RegistryMirror": [],
"InstallURL": "https://get.docker.com"
},
"SwarmOptions": {
"IsSwarm": false,
"Address": "",
"Discovery": "",
"Agent": false,
"Master": false,
"Host": "tcp://0.0.0.0:3376",
"Image": "swarm:latest",
"Strategy": "spread",
"Heartbeat": 0,
"Overcommit": 0,
"ArbitraryFlags": [],
"ArbitraryJoinFlags": [],
"Env": null,
"IsExperimental": false
},
"AuthOptions": {
"CertDir": "/root/.docker/machine/certs",
"CaCertPath": "/root/.docker/machine/certs/ca.pem",
"CaPrivateKeyPath": "/root/.docker/machine/certs/ca-key.pem",
"CaCertRemotePath": "",
"ServerCertPath": "/root/.docker/machine/machines/host1/server.pem",
"ServerKeyPath": "/root/.docker/machine/machines/host1/server-key.pem",
"ClientKeyPath": "/root/.docker/machine/certs/key.pem",
"ServerCertRemotePath": "",
"ServerKeyRemotePath": "",
"ClientCertPath": "/root/.docker/machine/certs/cert.pem",
"ServerCertSANs": [],
"StorePath": "/root/.docker/machine/machines/host1"
}
},
"Name": "host1"
}
# ssh命令
[root@localhost ~]# docker-machine ssh host1 docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
# url命令
[root@localhost ~]# docker-machine url host1
tcp://192.168.99.102:2376
# status命令
[root@localhost ~]# docker-machine status host1
Running
# version命令
[root@localhost ~]# docker-machine version host1
19.03.4
# 查看环境变量
[root@localhost ~]# docker-machine env host1
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.102:2376"
export DOCKER_CERT_PATH="/root/.docker/machine/machines/host1"
export DOCKER_MACHINE_NAME="host1"
# Run this command to configure your shell:
# eval $(docker-machine env host1)
# 运行 Docker 容器 运行虚拟机 docker run busybox ，并在里面里执行 echo hello world 命令，我们可以看到容器的输出信息。
[root@localhost ~]# docker run busybox echo hello world
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
7c9d20b9b6cd: Pull complete
Digest: sha256:fe301db49df08c384001ed752dff6d52b4305a73a7f608f21528048e8a08b51e
Status: Downloaded newer image for busybox:latest
hello world
[root@localhost ~]# docker-machine ssh host1 docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
busybox latest 19485c79a9bb 7 weeks ago 1.22MB
五、常见问题：
- You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
# 重新生成证书
[root@master ~]# docker-machine regenerate-certs node1-machine
Regenerate TLS machine certs? Warning: this is irreversible. (y/n): y
Regenerating TLS certificates
Waiting for SSH to be available...
Detecting the provisioner...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
# 重启Docker主机
[root@master ~]# docker-machine restart node1-machine
- Error starting host: Error creating. Error with pre-create check: "We support Virtualbox starting with version 5.
解决:
[root@master ~]# sudo /sbin/vboxconfig
-
Error creating machine: Error in driver during machine creation: unexpected EOF
虚拟机内存过小，请增加。
增加内存
-
Error with pre-create check: "This computer doesn't have VT-X/AMD-v enabled. Enabling it in the BIOS is mandatory"
不要用Oracle VM virtaulbox创建虚拟机，Oracle VM virtaulbox不支持嵌套虚拟机。本文用的是VMware Workstation，所以是在 虚拟机->设置 里设置处理器的虚拟化引擎首选模式为“Intel VT-x/EPT 或AMD-V/RVI”及“虚拟化Intel VT-x/EPT 或AMD-V/RVI(V)”
设置虚拟化
六、常用命令：
docker-machine active
显示当前的活动主机
docker-machine config
显示连接主机的配置
docker-machine create
创建一个主机
docker-machine env
设置当前的环境与哪个主机通信
docker-machine inspect
查看主机的详细信息
docker-machine ip
查看主机的IP
docker-machine kill
强制关闭一个主机
docker-machine ls
查看所有的主机信息
docker-machine provision
重新配置现在主机
docker-machine regenerate-certs
为主机重新生成证书
docker-machine restart
重启主机
docker-machine rm
删除主机
docker-machine ssh
以SSH的方式连接到主机上
docker-machine scp
远程复制
docker-machine status
查看主机的状态
docker-machine stop
停止一个正在运行的主机
docker-machine upgrade
升级主机的docker服务到最新版本
docker-machine version
查看docker-machine版本
附：
Machine drivers：
通过 -d选项可以选择支持的驱动类型
Amazon Web Services
Microsoft Azure
Digital Ocean
Exoscale
Google Compute Engine
Generic
Microsoft Hyper-V
OpenStack
Rackspace
IBM Softlayer
Oracle VirtualBox
VMware vCloud Air
VMware Fusion
VMware vSphere
VMware Workstation (unofficial plugin, not supported by Docker)
Grid 5000 (unofficial plugin, not supported by Docker)
Scaleway (unofficial plugin, not supported by Docker)
