istio流量管理
2019-07-16 本文已影响0人
潘猛_9f76
image.png
1、部署bookinfo应用
bookinfo应用包含4个微服务：
productpage
details
reviews
ratings
其中ratings微服务包含3个版本：
Version v1 无显示
Version v2 显示黑色1-5星
Version v3 显示红色1-5星
应用的逻辑结构如下：
image.png
下载并解压istio软件包
$ wget https://github.com/istio/istio/releases/download/1.2.2/istio-1.2.2-linux.tar.gz
$ tar zxvf istio-1.2.2-linux.tar.gz
安装参考helm安装istio
为namespace default添加istio-injection=enabled标签，缺省namespace default已经打了这个标签
$ kubectl label namespace default istio-injection=enabled
namespace/default labeled
如果安装时没有开启自动注入，需要手动注入sidecar
$ cd istio-1.2.2/
$ kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml)
部署并查看bookinfo的微服务
$ cd istio-1.2.2/
$ kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
$ kubectl get services
#已经删除不相关的结果
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
details ClusterIP 10.106.182.81 <none> 9080/TCP 26s
productpage ClusterIP 10.101.215.246 <none> 9080/TCP 25s
ratings ClusterIP 10.107.154.27 <none> 9080/TCP 25s
reviews ClusterIP 10.96.202.59 <none> 9080/TCP 25s
$ kubectl get pods
#已经删除不相关的结果
NAME READY STATUS RESTARTS AGE
details-v1-5544dc4896-dv98q 0/2 Init:0/1 0 83s
productpage-v1-7868c48878-m6xlk 0/2 Init:0/1 0 82s
ratings-v1-858fb7569b-mzskn 0/2 Init:0/1 0 83s
reviews-v1-796d4c54d7-qzj8j 0/2 Init:0/1 0 83s
reviews-v2-5d5d57db85-mlg4d 0/2 Init:0/1 0 84s
reviews-v3-77c6b4bdff-xszww 0/2 Init:0/1 0 84s
验证部署结果
#需要等待上面的pod状态正常后验证
$ kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>
定义gateway
$ kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
$ kubectl get gateway
NAME AGE
bookinfo-gateway 4m27s
查看gateway相关ingress配置
$ kubectl -n istio-system get svc istio-ingressgateway
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway NodePort 10.102.175.166 <none> 15020:30694/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:30128/TCP,15030:32065/TCP,15031:31074/TCP,15032:30039/TCP,15443:30552/TCP 4d21h
测试集群内访问
$ export GATEWAY_URL=10.102.175.166:80
$ kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl ${GATEWAY_URL}/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>
测试集群外通过nodePort访问
$ export GATEWAY_URL=22.22.3.243:31380
$ curl -s http://${GATEWAY_URL}/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>
配置destination-rule，也就是virtual-service中的subset
$ kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created
2、请求路由
如何将请求动态的路由到不同版本的微服务中
新建virtual-service，包含productpage，reviews，ratings，details 4个virtual-service
$ kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml
virtualservice.networking.istio.io/productpage created
virtualservice.networking.istio.io/reviews created
virtualservice.networking.istio.io/ratings created
virtualservice.networking.istio.io/details created
$ kubectl get virtualservices
NAME GATEWAYS HOSTS AGE
bookinfo [bookinfo-gateway] [*] 3h26m
details [details] 4m59s
productpage [productpage] 4m59s
ratings [ratings] 4m59s
reviews [reviews] 4m59s
$ kubectl get destinationrules
NAME HOST AGE
details details 117s
productpage productpage 117s
ratings ratings 117s
reviews reviews 117s
2.1、基于用户ID的路由
修改virtualservice reviews
$ kubectl apply -f samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
virtualservice.networking.istio.io/reviews configured
查看配置
$ kubectl get virtualservices.networking.istio.io reviews -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"reviews","namespace":"default"},"spec":{"hosts":["reviews"],"http":[{"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"reviews","subset":"v2"}}]},{"route":[{"destination":{"host":"reviews","subset":"v1"}}]}]}}
creationTimestamp: "2019-07-16T07:39:02Z"
generation: 2
name: reviews
namespace: default
resourceVersion: "10076327"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/reviews
uid: c80f4e8d-a79c-11e9-bd89-005056966598
spec:
hosts:
- reviews
http:
- match:
- headers:
end-user:
exact: jason
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
查看destinationrules，可以看到subsets的配置
$ kubectl get destinationrules.networking.istio.io reviews -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"DestinationRule","metadata":{"annotations":{},"name":"reviews","namespace":"default"},"spec":{"host":"reviews","subsets":[{"labels":{"version":"v1"},"name":"v1"},{"labels":{"version":"v2"},"name":"v2"},{"labels":{"version":"v3"},"name":"v3"}]}}
creationTimestamp: "2019-07-16T07:42:18Z"
generation: 1
name: reviews
namespace: default
resourceVersion: "10074760"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/destinationrules/reviews
uid: 3ca2209e-a79d-11e9-bd89-005056966598
spec:
host: reviews
subsets:
- labels:
version: v1
name: v1
- labels:
version: v2
name: v2
- labels:
version: v3
name: v3
浏览器使用jason登录可以黑色的星，说明路由到了reviewer v2
jason
使用其他用户登录则没有星，说明路由到了reviewer v1
image.png
可以看到服务器响应的报文头中并没有jason的标识，通过base64解码cookie，可以看到cookie携带了jason用户id的信息
image.png
echo eyJ1c2VyIjoiamFzb24ifQ.EA8YMw.2S1RsN51L2hyBFIPY8JxtUyHeEU | base64 -d
{"user":"jason"}base64: invalid input
3、故障注入
3.1、注入延时
在ratings微服务上注入7s的延时
$ kubectl apply -f samples/bookinfo/networking/virtual-service-ratings-test-delay.yaml
$ kubectl get virtualservices.networking.istio.io ratings -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"ratings","namespace":"default"},"spec":{"hosts":["ratings"],"http":[{"fault":{"delay":{"fixedDelay":"7s","percentage":{"value":100}}},"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"ratings","subset":"v1"}}]},{"route":[{"destination":{"host":"ratings","subset":"v1"}}]}]}}
creationTimestamp: "2019-07-16T07:39:02Z"
generation: 2
name: ratings
namespace: default
resourceVersion: "10084138"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/ratings
uid: c810e3ff-a79c-11e9-bd89-005056966598
spec:
hosts:
- ratings
http:
- fault:
delay:
fixedDelay: 7s
percentage:
value: 100
match:
- headers:
end-user:
exact: jason
route:
- destination:
host: ratings
subset: v1
- route:
- destination:
host: ratings
subset: v1
只有jason用户使用到review-v2，才会使用到ratings，所以采用jason用户来测试。可以看到网页在6s多加载完成，但是有报错
image.png
image.png
出现这个问题的原因是productpage和review之间的超时时间为6s，而review到rating之间延时达到7s，所以会出现这种情况
3.1、注入HTTP错误
在rating微服务前注入http 500错误
$ kubectl get virtualservices.networking.istio.io ratings -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"ratings","namespace":"default"},"spec":{"hosts":["ratings"],"http":[{"fault":{"abort":{"httpStatus":500,"percentage":{"value":100}}},"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"ratings","subset":"v1"}}]},{"route":[{"destination":{"host":"ratings","subset":"v1"}}]}]}}
creationTimestamp: "2019-07-16T07:39:02Z"
generation: 6
name: ratings
namespace: default
resourceVersion: "10088194"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/ratings
uid: c810e3ff-a79c-11e9-bd89-005056966598
spec:
hosts:
- ratings
http:
- fault:
abort:
httpStatus: 500
percentage:
value: 100
match:
- headers:
end-user:
exact: jason
route:
- destination:
host: ratings
subset: v1
- route:
- destination:
host: ratings
subset: v1
使用jason登录，可以看到“Ratings service is currently unavailable ”
image.png
流量转移
流量转移主要是在微服务的多个版本之间转移流量
恢复virtual-service的配置
$ kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml
修改virtual-service-reviews的配置，将v1和v2的权重都改为50
$ kubectl apply -f samples/bookinfo/networking/virtual-service-reviews-50-v3.yaml
$ kubectl get virtualservices.networking.istio.io reviews -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"reviews","namespace":"default"},"spec":{"hosts":["reviews"],"http":[{"route":[{"destination":{"host":"reviews","subset":"v1"},"weight":50},{"destination":{"host":"reviews","subset":"v3"},"weight":50}]}]}}
creationTimestamp: "2019-07-16T07:39:02Z"
generation: 4
name: reviews
namespace: default
resourceVersion: "10089206"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/reviews
uid: c80f4e8d-a79c-11e9-bd89-005056966598
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
weight: 50
- destination:
host: reviews
subset: v3
weight: 50
通过批量测试（testpod-96zq5是基于busybox启动的测试pod），可以看到有50个请求分发到了v3上，另外50个请求分发到了v1上
$ kubectl exec -it testpod-96zq5 /bin/sh
/ # for i in `seq 100`; do wget http://10.102.175.166/productpage -q -O - | grep -o "stars" | wc -l; done > result.txt
# cat result.txt | grep 4 | wc -l
50
/ # cat result.txt | grep 0 | wc -l
50
4、tcp流量转移
新建测试tcp用微服务
$ kubectl apply -f samples/tcp-echo/tcp-echo-services.yaml
service/tcp-echo created
deployment.extensions/tcp-echo-v1 created
deployment.extensions/tcp-echo-v2 created
配置路由策略，将所有流量路由到v1版本的pod
$ kubectl apply -f samples/tcp-echo/tcp-echo-all-v1.yaml
gateway.networking.istio.io/tcp-echo-gateway created
destinationrule.networking.istio.io/tcp-echo-destination created
virtualservice.networking.istio.io/tcp-echo created
查看istio-ingressgateway service，确认微服务使用的端口
$ kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].port}'
31400
发起10次tcp连接，测试路由情况. 可以 看到流量全部转发到了v1上
$ for i in {1..10}; do sh -c "(date; sleep 1) | nc 22.22.3.242 31400"; done
one Wed Jul 17 10:47:45 CST 2019
one Wed Jul 17 10:47:46 CST 2019
one Wed Jul 17 10:47:47 CST 2019
one Wed Jul 17 10:47:48 CST 2019
one Wed Jul 17 10:47:49 CST 2019
one Wed Jul 17 10:47:50 CST 2019
one Wed Jul 17 10:47:51 CST 2019
one Wed Jul 17 10:47:52 CST 2019
one Wed Jul 17 10:47:53 CST 2019
one Wed Jul 17 10:47:54 CST 2019
将20%的流量转移到v2上
$ kubectl apply -f samples/tcp-echo/tcp-echo-20-v2.yaml
$ kubectl get virtualservices.networking.istio.io tcp-echo -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"tcp-echo","namespace":"default"},"spec":{"gateways":["tcp-echo-gateway"],"hosts":["*"],"tcp":[{"match":[{"port":31400}],"route":[{"destination":{"host":"tcp-echo","port":{"number":9000},"subset":"v1"},"weight":80},{"destination":{"host":"tcp-echo","port":{"number":9000},"subset":"v2"},"weight":20}]}]}}
creationTimestamp: "2019-07-17T02:04:15Z"
generation: 2
name: tcp-echo
namespace: default
resourceVersion: "10252542"
selfLink: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/tcp-echo
uid: 2d6bc10f-a837-11e9-bd89-005056966598
spec:
gateways:
- tcp-echo-gateway
hosts:
- '*'
tcp:
- match:
- port: 31400
route:
- destination:
host: tcp-echo
port:
number: 9000
subset: v1
weight: 80
- destination:
host: tcp-echo
port:
number: 9000
subset: v2
weight: 20
再次测试10个连接
for i in {1..10}; do sh -c "(date; sleep 1) | nc 22.22.3.242 31400"; done
one Wed Jul 17 10:51:58 CST 2019
one Wed Jul 17 10:51:59 CST 2019
one Wed Jul 17 10:52:00 CST 2019
one Wed Jul 17 10:52:01 CST 2019
one Wed Jul 17 10:52:02 CST 2019
one Wed Jul 17 10:52:03 CST 2019
one Wed Jul 17 10:52:04 CST 2019
two Wed Jul 17 10:52:05 CST 2019
one Wed Jul 17 10:52:06 CST 2019
two Wed Jul 17 10:52:07 CST 2019
并不是每次测试都可以看到two，需要多次测试
删除相关配置
$ kubectl delete -f samples/tcp-echo/tcp-echo-all-v1.yaml
$ kubectl delete -f samples/tcp-echo/tcp-echo-services.yaml
5、请求超时
还原实验环境
kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml
实验环境如下拓扑：
client----productpage----reviews v2----ratings
将reviews的请求都路由到v2
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v2
EOF
在reviews v2----ratings之间增加2s的延时
client----productpage----reviews v2--2s--ratings
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ratings
spec:
hosts:
- ratings
http:
- fault:
delay:
percent: 100
fixedDelay: 2s
route:
- destination:
host: ratings
subset: v1
EOF
image.png
在productpage----reviews v2之间增加请求超时0.5s
client----productpage--0.5s(timeout)--reviews v2--2s(delay)--ratings
当productpage重试2次，也就是1s后，productpage会认为reviews v2超时
image.png
6、短路
短路主要是在DestinationRule中定义
新建实验环境基础服务httpbin
$ kubectl apply -f samples/httpbin/httpbin.yaml
service/httpbin created
deployment.extensions/httpbin created
新建DestinationRule，限制最大连接数为1，最大Pending请求1，每连接最大请求1
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin
spec:
host: httpbin
trafficPolicy:
connectionPool:
tcp:
maxConnections: 1
http:
http1MaxPendingRequests: 1
maxRequestsPerConnection: 1
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 3m
maxEjectionPercent: 100
EOF
创建客户端deployment
$ kubectl apply -f samples/httpbin/sample-client/fortio-deploy.yaml
service/fortio created
deployment.apps/fortio-deploy created
获取客户端pod name
$ FORTIO_POD=$(kubectl get pod | grep fortio | awk '{ print $1 }')
在客户端上测试连接正常, -curl表示取一次
注意等待pod状态正常后
$ kubectl exec -it $FORTIO_POD -c fortio /usr/bin/fortio -- load -curl http://httpbin:8000/get
HTTP/1.1 200 OK
server: envoy
date: Wed, 17 Jul 2019 07:05:54 GMT
content-type: application/json
content-length: 371
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 14
{
"args": {},
"headers": {
"Content-Length": "0",
"Host": "httpbin:8000",
"User-Agent": "fortio.org/fortio-1.3.1",
"X-B3-Parentspanid": "1810664d6a5d4b64",
"X-B3-Sampled": "0",
"X-B3-Spanid": "542ed0312282a174",
"X-B3-Traceid": "a16e92e48b6b9b261810664d6a5d4b64"
},
"origin": "127.0.0.1",
"url": "http://httpbin:8000/get"
}
在客户端上发起2个并发连接，20个request，-qps 0表示不等待
kubectl exec -it $FORTIO_POD -c fortio /usr/bin/fortio -- load -c 2 -qps 0 -n 20 -loglevel Warning http://httpbin:8000/get
Sockets used: 3 (for perfect keepalive, would be 2)
Code 200 : 19 (95.0 %)
Code 503 : 1 (5.0 %)
#省略了大部分输出
$ kubectl exec -it $FORTIO_POD -c istio-proxy -- sh -c 'curl localhost:15000/stats' | grep httpbin | grep pending
cluster.outbound|8000||httpbin.default.svc.cluster.local.circuit_breakers.default.rq_pending_open: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.circuit_breakers.high.rq_pending_open: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_active: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_failure_eject: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_overflow: 1
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_total: 19
#成功了19次，失败了1次
在客户端上发起3个并发连接，30个request
$ kubectl exec -it $FORTIO_POD -c fortio /usr/bin/fortio -- load -c 3 -qps 0 -n 30 -loglevel Warning http://httpbin:8000/get
#省略了大部分输出
Sockets used: 18 (for perfect keepalive, would be 3)
Code 200 : 14 (46.7 %)
Code 503 : 16 (53.3 %)
$ kubectl exec -it $FORTIO_POD -c istio-proxy -- sh -c 'curl localhost:15000/stats' | grep httpbin | grep pending
cluster.outbound|8000||httpbin.default.svc.cluster.local.circuit_breakers.default.rq_pending_open: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.circuit_breakers.high.rq_pending_open: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_active: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_failure_eject: 0
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_overflow: 17
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_pending_total: 33
#成功了33-19=14次，失败了17-1=16次
清理实验环境
$ kubectl delete destinationrule httpbin
destinationrule.networking.istio.io "httpbin" deleted
$ kubectl delete deploy httpbin fortio-deploy
deployment.extensions "httpbin" deleted
deployment.extensions "fortio-deploy" deleted
$ kubectl delete svc httpbin
service "httpbin" deleted
$ kubectl delete svc fortio
service "fortio" deleted
7、镜像
创建两个版本的deployment，httpbin-v1和httpbin-v2
$ cat <<EOF | bin/istioctl kube-inject -f - | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: httpbin-v1
spec:
replicas: 1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
containers:
- image: docker.io/kennethreitz/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"]
ports:
- containerPort: 80
EOF
$ cat <<EOF | bin/istioctl kube-inject -f - | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: httpbin-v2
spec:
replicas: 1
template:
metadata:
labels:
app: httpbin
version: v2
spec:
containers:
- image: docker.io/kennethreitz/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"]
ports:
- containerPort: 80
EOF
创建kubernetes服务httpbin
$ kubectl create -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: httpbin
labels:
app: httpbin
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
EOF
创建Deployment sleep，作为客户端来访问httpbin
$ cat <<EOF | bin/istioctl kube-inject -f - | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
template:
metadata:
labels:
app: sleep
spec:
containers:
- name: sleep
image: tutum/curl
command: ["/bin/sleep","infinity"]
imagePullPolicy: IfNotPresent
EOF
创建VirtualService httpbin及相关DestinationRule，将所有请求路由到httpbin-v1
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin
http:
- route:
- destination:
host: httpbin
subset: v1
weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin
spec:
host: httpbin
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
测试client是否可以正常访问到httpbin-v1
$ export SLEEP_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
$ kubectl exec -it $SLEEP_POD -c sleep -- sh -c 'curl http://httpbin:8000/headers' | python -m json.tool
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin:8000",
"User-Agent": "curl/7.35.0",
"X-B3-Parentspanid": "1f7a88f4dbd67b9a",
"X-B3-Sampled": "0",
"X-B3-Spanid": "5794ab2613eaa9ac",
"X-B3-Traceid": "40ab50914721544a1f7a88f4dbd67b9a"
}
}
分别查看两个pod的日志,可以看到日志中的源地址是127.0.0.1，看不到实际的客户端地址
$ export V1_POD=$(kubectl get pod -l app=httpbin,version=v1 -o jsonpath={.items..metadata.name})
$ kubectl logs -f $V1_POD -c httpbin
#省略部分日志
127.0.0.1 - - [17/Jul/2019:09:20:06 +0000] "GET /headers HTTP/1.1" 200 303 "-" "curl/7.35.0"
127.0.0.1 - - [17/Jul/2019:09:20:21 +0000] "GET /headers HTTP/1.1" 200 303 "-" "curl/7.35.0"
127.0.0.1 - - [17/Jul/2019:09:20:24 +0000] "GET /headers HTTP/1.1" 200 303 "-" "curl/7.35.0"
$ export V2_POD=$(kubectl get pod -l app=httpbin,version=v2 -o jsonpath={.items..metadata.name})
$ kubectl logs -f $V2_POD -c httpbin
#省略部分日志
修改VirtualService httpbin， 所有流量路由到v1，并镜像到v2，这个时候v2会收到请求，但是不会响应
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin
http:
- route:
- destination:
host: httpbin
subset: v1
weight: 100
mirror:
host: httpbin
subset: v2
EOF
在客户端启动tcpdump抓包
$ export SLEEP_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
$ export V1_POD_IP=$(kubectl get pod -l app=httpbin,version=v1 -o jsonpath={.items..status.podIP})
$ export V2_POD_IP=$(kubectl get pod -l app=httpbin,version=v2 -o jsonpath={.items..status.podIP})
$ kubectl exec -it $SLEEP_POD -c istio-proxy -- sudo tcpdump -A -s 0 host $V1_POD_IP or host $V2_POD_IP
从客户端再次访问httpbin
$ kubectl exec -it $SLEEP_POD -c sleep -- sh -c 'curl http://httpbin:8000/headers' | python -m json.tool
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin:8000",
"User-Agent": "curl/7.35.0",
"X-B3-Parentspanid": "1a72fb1ed1ea7e67",
"X-B3-Sampled": "0",
"X-B3-Spanid": "4274bec949807bdc",
"X-B3-Traceid": "b3aadfd8a6ea56091a72fb1ed1ea7e67"
}
}
查看httpbin pod上的日志,可以看到v1和v2都有访问记录
#只记录增加的日志
$ kubectl logs -f $V1_POD -c httpbin
127.0.0.1 - - [17/Jul/2019:09:30:50 +0000] "GET /headers HTTP/1.1" 200 303 "-" "curl/7.35.0"
$kubectl logs -f $V2_POD -c httpbin
127.0.0.1 - - [17/Jul/2019:09:30:50 +0000] "GET /headers HTTP/1.1" 200 343 "-" "curl/7.35.0"
查看在client上抓取的报文，v1和v2响应的报文都可以看到，但是由于v2响应的host头不对，会被丢弃
09:37:35.518095 IP sleep-58d7459fb4-hsxcz.59626 > 10-244-186-217.httpbin.default.svc.cluster.local.80: Flags [P.], seq 2708436932:2708437664, ack 2031609492, win 253, options [nop,nop,TS val 134578745 ecr 134818527], length 732: HTTP: GET /headers HTTP/1.1
#路由到v1的GET报文
GET /headers HTTP/1.1
host: httpbin:8000
user-agent: curl/7.35.0
accept: */*
x-forwarded-proto: http
x-request-id: d9af174b-9cfa-4846-93a1-5558a1e7edae
x-envoy-decorator-operation: httpbin.default.svc.cluster.local:8000/*
x-istio-attributes: Cj0KF2Rlc3RpbmF0aW9uLnNlcnZpY2UudWlkEiISIGlzdGlvOi8vZGVmYXVsdC9zZXJ2aWNlcy9odHRwYmluCj8KGGRlc3RpbmF0aW9uLnNlcnZpY2UuaG9zdBIjEiFodHRwYmluLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwKJQoYZGVzdGluYXRpb24uc2VydmljZS5uYW1lEgkSB2h0dHBiaW4KKgodZGVzdGluYXRpb24uc2VydmljZS5uYW1lc3BhY2USCRIHZGVmYXVsdAo7Cgpzb3VyY2UudWlkEi0SK2t1YmVybmV0ZXM6Ly9zbGVlcC01OGQ3NDU5ZmI0LWhzeGN6LmRlZmF1bHQ=
x-b3-traceid: 580b124d18c31e00294265984a47cadf
x-b3-spanid: 294265984a47cadf
x-b3-sampled: 0
content-length: 0
09:37:35.518167 IP sleep-58d7459fb4-hsxcz.59252 > 10-244-241-222.httpbin.default.svc.cluster.local.80: Flags [P.], seq 3931781914:3931782710, ack 1759932701, win 228, options [nop,nop,TS val 134578745 ecr 134493370], length 796: HTTP: GET /headers HTTP/1.1
#mirror到v2的GET报文
GET /headers HTTP/1.1
host: httpbin-shadow:8000
#httpbin-shadow代表是mirror的请求
user-agent: curl/7.35.0
accept: */*
x-forwarded-proto: http
x-request-id: d9af174b-9cfa-4846-93a1-5558a1e7edae
x-envoy-decorator-operation: httpbin.default.svc.cluster.local:8000/*
x-istio-attributes: Cj0KF2Rlc3RpbmF0aW9uLnNlcnZpY2UudWlkEiISIGlzdGlvOi8vZGVmYXVsdC9zZXJ2aWNlcy9odHRwYmluCj8KGGRlc3RpbmF0aW9uLnNlcnZpY2UuaG9zdBIjEiFodHRwYmluLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwKJQoYZGVzdGluYXRpb24uc2VydmljZS5uYW1lEgkSB2h0dHBiaW4KKgodZGVzdGluYXRpb24uc2VydmljZS5uYW1lc3BhY2USCRIHZGVmYXVsdAo7Cgpzb3VyY2UudWlkEi0SK2t1YmVybmV0ZXM6Ly9zbGVlcC01OGQ3NDU5ZmI0LWhzeGN6LmRlZmF1bHQ=
x-b3-traceid: 580b124d18c31e00294265984a47cadf
x-b3-spanid: 294265984a47cadf
x-b3-sampled: 0
x-envoy-internal: true
x-forwarded-for: 10.244.196.153
content-length: 0
09:37:35.523399 IP 10-244-186-217.httpbin.default.svc.cluster.local.80 > sleep-58d7459fb4-hsxcz.59626: Flags [P.], seq 1:540, ack 732, win 263, options [nop,nop,TS val 135223822 ecr 134578745], length 539: HTTP: HTTP/1.1 200 OK
#v1响应的http 200
HTTP/1.1 200 OK
server: istio-envoy
date: Wed, 17 Jul 2019 09:37:35 GMT
content-type: application/json
content-length: 303
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 3
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin:8000",
"User-Agent": "curl/7.35.0",
"X-B3-Parentspanid": "294265984a47cadf",
"X-B3-Sampled": "0",
"X-B3-Spanid": "2592a40ed3df24b2",
"X-B3-Traceid": "580b124d18c31e00294265984a47cadf"
}
}
09:37:35.523673 IP 10-244-241-222.httpbin.default.svc.cluster.local.80 > sleep-58d7459fb4-hsxcz.59252: Flags [P.], seq 1:580, ack 796, win 242, options [nop,nop,TS val 134898660 ecr 134578745], length 579: HTTP: HTTP/1.1 200 OK
#v2响应的http 200
HTTP/1.1 200 OK
server: istio-envoy
date: Wed, 17 Jul 2019 09:37:35 GMT
content-type: application/json
content-length: 343
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 4
{
"headers": {
"Accept": "*/*",
"Content-Length": "0",
"Host": "httpbin-shadow:8000",
"User-Agent": "curl/7.35.0",
"X-B3-Parentspanid": "294265984a47cadf",
"X-B3-Sampled": "0",
"X-B3-Spanid": "74dae7c4eefec1dc",
"X-B3-Traceid": "580b124d18c31e00294265984a47cadf",
"X-Envoy-Internal": "true"
}
