kubernetes基础命令使用
2018-07-03 本文已影响117人
9c46ece5b7bd
kubectl命令使用帮助
获取相关资源对象的文档kubectl explain resources
# kubectl explain pods|deployments|statefulset|configmap
$ kubectl explain deployments
KIND: Deployment
VERSION: extensions/v1beta1
DESCRIPTION:
DEPRECATED - This group version of Deployment is deprecated by
apps/v1beta2/Deployment. See the release notes for more information.
Deployment enables declarative updates for Pods and ReplicaSets.
FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https:/git.k8s.io/community/contributors/devel/api-conventions.md#resources
kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https:/git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds
metadata <Object>
Standard object metadata.
spec <Object>
Specification of the desired behavior of the Deployment.
status <Object>
Most recently observed status of the Deployment.
资源管理
注意:虽然kubectl可以支持使用命令式创建相关资源，但是还是建议使用声明式创建资源，便于后期的问题的溯源
# 声明式创建资源
$ kubectl create -f deployment.yaml
# 命令式创建资源
# 也是默认使用的deployment创建的对象
$ kubectl run nginx --image=nginx
# 资源查找 kubectl get resource
# 查看当前namespace下的资源
$ kubectl get services|deployments|statefulset|pods
# 全部namespace
$ kubectl get services --all-namespaces
# 指定namespace
$ kubectl get services -n default
# 当前namespace下的service详细信息
$ kubectl get services -o wide
# 资源详细信息管理 kubectl describe resourcetype resourceinstance
$ kubectl describe nodes node-ip
$ kubectl describe pods my-pod
# 编辑资源 kubectl edit resourcetype/resourceinstance
# 资源编辑保存后立即生效
$ kubectl edit svc/test-go-web
$ kubectl edit deploy/test-go-web
# 资源扩展 kubectl scale --replicas=3
# 使用资源声明文件更新或直接更新控制器资源
# 默认是扩展到几个
$ kubectl scale --replicas=3 -f test-Deployments.yaml
$ kubectl scale --replicas=3 deployments/test-go-web
# 当前实例为2个扩展到3个
$ kubectl scale --current-replicas=2 --replicas=3 -f test-Deployments.yaml
pod调试和交互
# 查看pod日志
$ kubectl logs -f --tail=2 pod-example
Hello World
# 在pod容器内部执行debug [kubectl exec -it podname -c container -- command]
$ kubectl exec -it test-go-web-78d795c5cc-cgdsp -- date
Thu Jul 19 06:25:03 EDT 2018
集群运维管理
# 标记节点是否可调度
$ kubectl cordon node-ip
$ kubectl nocordon node-ip
# 清空node以待维护
$ kubectl drain node-ip
# 显示集群信息
$ kubectl cluster-info
发布管理
# 滚动更新kubectl rolling-update
# 指定pod更新相关镜像
kubectl rolling-update OLD_CONTROLLER_NAME ([NEW_CONTROLLER_NAME] --image=NEW_CONTAINER_IMAGE
$ kubectl rolling-update frontend-v1 -f frontend-v2.json
# 指定镜像更新容器v1到v2
$ kubectl rolling-update frontend-v1 frontend-v2 --image=image:v2
# 自动扩展容器实例
# kubectl autoscale (-f FILENAME | TYPE NAME | TYPE/NAME) [--min=MINPODS] --max=MAXPODS [--cpu-percent=CPU] [options]
$ kubectl autoscale deployment foo --min=2 --max=10
$ kubectl autoscale rc foo --max=5 --cpu-percent=80
# 回滚 kubectl rollout
节点信息查询
# 查看集群工作节点
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.0.47.202 Ready <none> 5d v1.10.4
10.0.51.121 Ready <none> 11d v1.10.4
10.0.51.122 Ready <none> 5d v1.10.4
# 查看工作节点详情(使用ps格式)
$ kubectl get nodes -o wide
# 查看节点标签
$ kubectl get nodes --show-labels=true
NAME STATUS ROLES AGE VERSION LABELS
10.0.47.202 Ready <none> 5d v1.10.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=10.0.47.202
10.0.51.121 Ready <none> 11d v1.10.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=10.0.51.121
10.0.51.122 Ready <none> 5d v1.10.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=10.0.51.122
# 获取指定标签的node节点
$ kubectl get nodes -l net=all
NAME STATUS ROLES AGE VERSION
10.0.47.202 Ready <none> 5d v1.10.4
节点信息更新
# 更新node节点的label标签
# 标记节点网络为全网通
$ kubectl label nodes 10.0.47.202 net=all
node "10.0.47.202" labeled
kubernetes的资源控制器使用示例
任务更新和查看
1. 使用k8s的Job控制器来构建conda包和dockerimage
# 创建job任务自动构建image
$ cat image-build-service.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: image-build-service
spec:
template:
metadata:
name: image-build-service
spec:
volumes:
- name: hosts
hostPath:
path: /etc/hosts
containers:
- name: image-build-service
image: myregistry.com/deeplearning/centos7.3.1611-image-build-service
env:
- name: dockerhost
value: "10.0.60.148:5256"
- name: branch
value: "master"
- name: giturl
value: "http:/git.jd.com/JFCCP/aidockerfiles.git"
- name: app
value: "jrapp_antispam"
- name: dockerfiledir
value: "/export/Data/"
volumeMounts:
- name: hosts
mountPath: /etc/hosts
restartPolicy: Never
nodeSelector:
net: all
# 创建job任务自动构建conda包
# 注意容器内部解析外部域名，暂时使用hosts方式
$ cat conda-build-service.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: conda-build-service
spec:
template:
metadata:
name: conda-build-service
spec:
volumes:
- name: hosts
hostPath:
path: /etc/hosts
containers:
- name: conda-build-service
image: myregistry.com/deeplearning/centos7.3.1611-conda-build-service
env:
- name: giturl
value: "http:/git.jd.com/xuxuebiao/conda-recipes.git"
- name: branch
value: "py2.7"
- name: package
value: "lightgbm"
- name: pyenv
value: "2.7"
volumeMounts:
- name: hosts
mountPath: /etc/hosts
restartPolicy: Never
nodeSelector:
net: all
# 创建kubernetes job任务
$ kubectl create -f image-build-service.yaml
$ kubectl create -f conda-build-service.yaml
# 查看job运行情况(期望1个并成功1个)
$ kubectl get jobs
NAME DESIRED SUCCESSFUL AGE
conda-build-service 1 1 24m
image-build-service 1 1 16h
# 查看承载job任务的pod详情
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
conda-build-service-fw9b4 0/1 Completed 0 27m
image-build-service-gpg85 0/1 Completed 0 16h
# 查看job详情
$ kubectl describe jobs image-build-service
Name: image-build-service
Namespace: default
Selector: controller-uid=bcf72759-7ea8-11e8-b6aa-f000ac1941ca
Labels: controller-uid=bcf72759-7ea8-11e8-b6aa-f000ac1941ca
job-name=image-build-service
Annotations: <none>
Parallelism: 1
Completions: 1
Start Time: Tue, 03 Jul 2018 18:06:22 +0800
Pods Statuses: 0 Running / 1 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=bcf72759-7ea8-11e8-b6aa-f000ac1941ca
job-name=image-build-service
Containers:
image-build-service:
Image: myregistry.com/deeplearning/centos7.3.1611-image-build-service
Port: <none>
Host Port: <none>
Environment:
dockerhost: 10.0.60.148:5256
branch: master
giturl: http:/git.jd.com/JFCCP/aidockerfiles.git
app: jrapp_antispam
dockerfiledir: /export/Data/
Mounts:
/etc/hosts from hosts (rw)
Volumes:
hosts:
Type: HostPath (bare host directory volume)
Path: /etc/hosts
HostPathType:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 5m job-controller Created pod: image-build-service-gpg85
# 查看承载job的pod详情，通常情况下，需要通过pods来查看容器的整体执行过程
$ kubectl describe pod image-build-service-gpg85
Name: image-build-service-gpg85
Namespace: default
Node: 10.0.47.202/10.0.47.202
Start Time: Tue, 03 Jul 2018 18:06:21 +0800
Labels: controller-uid=bcf72759-7ea8-11e8-b6aa-f000ac1941ca
job-name=image-build-service
Annotations: <none>
Status: Succeeded
IP: 172.17.0.3
Controlled By: Job/image-build-service
Containers:
image-build-service:
Container ID: docker:/6aab561d62e5157959f9045d3fca180fb7e4225490c38f7b32f7321678268e46
Image: myregistry.com/deeplearning/centos7.3.1611-image-build-service
Image ID: docker-pullable:/myregistry.com/deeplearning/centos7.3.1611-image-build-service@sha256:a6adcd64290009ada5f563e659ce7258e9af16cfc9cea00b04f1fba4d26faa25
Port: <none>
Host Port: <none>
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 03 Jul 2018 18:06:23 +0800
Finished: Tue, 03 Jul 2018 18:08:50 +0800
Ready: False
Restart Count: 0
Environment:
dockerhost: 10.0.60.148:5256
branch: master
giturl: http:/git.jd.com/JFCCP/aidockerfiles.git
app: jrapp_antispam
dockerfiledir: /export/Data/
Mounts:
/etc/hosts from hosts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-hkslc (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
hosts:
Type: HostPath (bare host directory volume)
Path: /etc/hosts
HostPathType:
default-token-hkslc:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-hkslc
Optional: false
QoS Class: BestEffort
Node-Selectors: net=all
Tolerations: <none>
Events: <none>
# 查看job容器详情
# 因为使用了nodeSelector，会调度到有net=all标签的node节点
$ docker logs --tail 10 k8s_image-build-service_image-build-service-gpg85_default_bcf94afa-7ea8-11e8-b6aa-f000ac1941ca_0
Step 5 : CMD ['/bin/bash']
---> Running in 1bd774a78016
---> ed7eeba8e50a
Removing intermediate container 1bd774a78016
Successfully built ed7eeba8e50a
########################End building########################
########################Begin pushing########################
The push refers to a repository [myregistry.com/deeplearning/jrapp_antispam]
20180703T180643: digest: sha256:25d0894f0f0c7131ccc54381a42e2e3ba21b0d4f1ba2eca9de055a44fa281c9b size: 1580
########################End pushing########################
2. 使用DaemonSet控制器创建node节点守护进程
注意: DaemonSet控制器一般用于宿主机的守护lang time进程，比如log-agent,monitor-agent等
$ cat test-DaemonSet.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: daemonset-example
spec:
template:
metadata:
labels:
app: daemonset-example
spec:
containers:
- name: daemonset-example
image: xxbandy123/centos6.5:base
command:
- /bin/sh
args:
- -c
# This script is run through `sh -c <script>`
- >-
while [ true ]; do
echo "DaemonSet running on $(hostname)" ;
sleep 10 ;
done
$ kubectl create -f test-DaemonSet.yaml
daemonset.extensions "daemonset-example" created
# 注意: daemonset类型应用一般如果没有指定Selector的时候会在所有的node节点运行
$ kubectl get daemonset
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset-example 5 5 5 5 5 <none> 7s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
daemonset-example-99md2 1/1 Running 0 38s
daemonset-example-cwpbw 1/1 Running 0 38s
daemonset-example-dh5n6 1/1 Running 0 38s
daemonset-example-h4fpt 1/1 Running 0 38s
daemonset-example-nbfjf 1/1 Running 0 38s
$ kubectl logs --tail 3 daemonset-example-99md2
DaemonSet running on daemonset-example-99md2
DaemonSet running on daemonset-example-99md2
DaemonSet running on daemonset-example-99md2
# 使用exec金融容器内部
$ kubectl exec -it daemonset-example-99md2 -- bash
[root@daemonset-example-99md2 ~]# hostname
daemonset-example-99md2
3. 使用Deployment控制器快速创建无状态应用
使用deployment创建一个无状态应用集群，一般来说无状态应用分为4层RPC方式调用和7层HTTP 方式调用，前者需要通过其他方式实现服务注册和服务发现，比如zk，eruka等；后者一般会使用负载均衡相关的技术进行对外暴露服务，比如nginx,lvs,F5等，而在k8s中，该种情景需要使用Service进行对外暴露服务。Service默认使用iptables规则进行路由转发。
# 使用deployment快速创建go-web的无状态应用,这是一个简单用来打印container hostname和container time 的简易web程序
$ cat test-Deployments.yaml
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: test-go-web
spec:
replicas: 5
template:
metadata:
labels:
app: test-go-web
spec:
containers:
- name: test-go-web
image: xxbandy123/go-web
ports:
- containerPort: 9090
$ kubectl create -f test-Deployments.yaml
deployment.apps "test-go-web" created
$ kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
test-go-web 5 5 5 5 16s
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
test-go-web-78d795c5cc-6czv5 1/1 Running 0 54s 172.30.55.2 10.0.47.138
test-go-web-78d795c5cc-cgdsp 1/1 Running 0 54s 172.30.56.2 10.0.47.75
test-go-web-78d795c5cc-dpqr9 1/1 Running 0 54s 172.30.65.2 10.0.47.202
test-go-web-78d795c5cc-p8p96 1/1 Running 0 54s 172.30.32.2 10.0.51.122
test-go-web-78d795c5cc-zwnzw 1/1 Running 0 54s 172.30.68.2 10.0.51.121
# 在k8s集群内部可以访问每个pod ip来进行访问服务
$ for i in 55 56 65 32 68 ;do curl 172.30.$i.2:9090 ;done
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-6czv5
Currently Date:2018-07-19 03:58:44Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-cgdsp
Currently Date:2018-07-19 03:58:44Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-dpqr9
Currently Date:2018-07-19 03:58:44Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-p8p96
Currently Date:2018-07-19 03:58:44Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-zwnzw
Currently Date:2018-07-19 03:58:43
# 但是如果test-go-web这个服务作为一个统一对外的服务，是需要使用统一入口的，这个时候我们可以创建一个service
# 需要注意的是service是通过.spec.selector 来识别具有相关label的pod的，仔细观察的话会发现上面deployment中我们定义了'app: test-go-web',所以才能将service和上面的deployment关联起来
$ cat test-Deployments-Service.yaml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-go-web
name: test-go-web
namespace: default
spec:
type: NodePort
selector:
app: test-go-web
ports:
- port: 9090
$ kubectl create -f test-Deployments-Service.yaml
service "test-go-web" created
$ kubectl get svc test-go-web -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
test-go-web NodePort 10.254.175.92 <none> 9090:31054/TCP 4s app=test-go-web
# 接下来用户就可以在k8s集群内部直接访问cluster-ip来访问上面test-go-web的5个实例了
# 可以发现直接访问这个service-ip的9090端口就可以访问到test-go-web的5个容器实例了，其实这里类似于一层简单的7层负载代理
$ for i in `seq 1 5`;do echo $i; curl 10.254.175.92:9090 ;echo "\n";done
1
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-p8p96
Currently Date:2018-07-19 04:12:25\n
2
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-zwnzw
Currently Date:2018-07-19 04:12:25\n
3
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-zwnzw
Currently Date:2018-07-19 04:12:25\n
4
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-dpqr9
Currently Date:2018-07-19 04:12:26\n
5
Hello ,I'm biaoge.
My Container name is test-go-web-78d795c5cc-cgdsp
Currently Date:2018-07-19 04:12:25\n
4. 使用pod来创建测试容器
因为我们使用k8s来进行集群管理的，而pod只是一个应用的承载器，单独创建并没有多大意义，一般场景中还是会使用一些更高级的控制器来代为创建pod，以便对整个业务应用的整体状态进行管理。
这里可以使用pod来简单看看k8s底层最基础的业务承载器是如何工作的。
$ cat apiVersion: v1
kind: Pod
metadata:
name: pod-example
spec:
containers:
- name: pod-example
image: xxbandy123/centos6.5:base
command: ["echo"]
args: ["Hello World"]
restartPolicy: Never
$ kubectl create -f test-Pod.yaml
pod "pod-example" created
# 由于该pod内部只echo一个输出后即退出，因此很短暂，状态为completed
$ kubectl get pods pod-example -o wide
NAME READY STATUS RESTARTS AGE IP NODE
pod-example 0/1 Completed 1 23s 172.30.56.3 10.0.47.75
$ kubectl logs --tail=2 pod-example
Hello World
注意:当然无状态服务控制器除了Depoyment之外还有z之前的ReplicaSet和ReplicaController,不过Deployment的功能更多更高级一些,推荐使用Deployment
5. 使用StatefulSet来创建有状态应用集群
StatefulSet是一个可以给pod提供唯一标识的控制器，另外它可以保证部署和扩展的顺序。
一般情况下，如果有如下需求时，StatefulSet的价值就体现出来了。
- 稳定的、唯一的网络标识
- 稳定的、持久化的存储
- 有序的、优雅的部署和扩展。
- 有序的、优雅的删除和停止
一般情况下有状态服务会要求固定的网络标识、持久化的存储以及有序的部署顺序,这个时候使用StatefulSet+Headless Service+volumeClaimTemplates来进行有状态服务的管理。
注意:clusterIP为None的service就是headless service
$ cat test-statefulset.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx # has to match .spec.template.metadata.labels
serviceName: "nginx"
replicas: 3 # by default is 1
template:
metadata:
labels:
app: nginx # has to match .spec.selector.matchLabels
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: xxbandy123/nginx:latest
ports:
- containerPort: 80
name: web
$ kubectl create -f test-statefulset.yaml
$ kubectl get statefulset -o wide
NAME DESIRED CURRENT AGE CONTAINERS IMAGES
web 3 3 1m nginx xxbandy123/nginx:latest
$ kubectl get pods
NAME READY STATUS RESTARTS AGE IP NODE
web-0 1/1 Running 0 3m 172.30.32.3 10.0.51.122
web-1 1/1 Running 0 3m 172.30.65.3 10.0.47.202
web-2 1/1 Running 0 2m 172.30.55.3 10.0.47.138
6. service的创建
k8s中使用service来抽象pod的集合，作为集群的外部入口。service分为三种模式分别为userspace、iptables、lvs三种模式，在k8s1.6之后默认使用iptables模式，由kube-proxy组件进行统一管理
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-go-web
name: test-go-web
namespace: default
spec:
# NodePort ClusterIP[default] LoadBalancer
type: NodePort
selector:
app: test-go-web
ports:
- port: 9090
targetPort: 9090
7. secret的使用
secret的使用
$ kubectl create secret docker-registry my-docker --docker-server=10.0.60.148 --docker-username=username --docker-password=password --docker-email=371990778@qq.com
# 查看secret的内容：
$ kubectl get secret -o yaml
apiVersion: v1
items:
- apiVersion: v1
data:
.dockercfg: eyIxNzIuMjUuNjAuMTQ4Ijp7InVzZXJuYW1lIjoiamRqciIsInBhc3N3b3JkIjoiSkRqci53eXBlMTIzIiwiZW1haWwiOiJ4dXh1ZWJpYW9AamQuY29tIiwiYXV0aCI6ImFtUnFjanBLUkdweUxuZDVjR1V4TWpNPSJ9fQ==
kind: Secret
metadata:
creationTimestamp: 2017-08-25T08:33:46Z
name: my-docker
namespace: default
resourceVersion: "1096287"
selfLink: /api/v1/namespaces/default/secrets/my-docker
uid: 1cac3aa5-8970-11e7-ac55-f000ac192d4d
type: kubernetes.io/dockercfg
kind: List
metadata: {}
resourceVersion: ""
selfLink: ""
# 生成的.dockercfg的值是可以使用base64进行解码的
# 使用docker
$ cat pod-harbor.yml
apiVersion: v1
kind: Pod
metadata:
name: biaoge-registry
spec:
containers:
- name: biaoge-registry
image: xxbandy123/r2m-zk:latest
# 使用上面创建的secret
imagePullSecrets:
- name: my-docker
相关链接文档
kubectl-cheatsheet
k8s-api-docs
k8s-kubectl-commands
redis-cluster-on-k8s
redis-cluster
