istio熔断
2020-09-28 本文已影响0人
OPS_Joy
熔断，是创建弹性微服务应用程序的重要模式。熔断能够使您的应用程序具备应对来自故障、潜在峰值和其他 未知网络因素影响的能力。
创建熔断目标规则
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: myapprule
spec:
host: myappv2
trafficPolicy:
connectionPool:
tcp:
maxConnections: 3
http:
http1MaxPendingRequests: 3
maxRequestsPerConnection: 1
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 3m
maxEjectionPercent: 100
一些参数说明
maxConnections：到目标主机的HTTP1/TCP最大连接数量，只作用于http1.1，不作用于http2，因为后者只建立一次连接。
http1MaxPendingRequests：http请求pending状态的最大请求数，从应用容器发来的HTTP请求的最大等待转发数，默认是1024。
maxRequestsPerConnection：在一定时间内限制对后端服务发起的最大请求数，如果超过了这个限制，就会开启限流。如果将这一参数设置为 1 则会禁止 keepalive 特性；
创建一个Fortio负载压力测试客户端
apiVersion: v1
kind: Service
metadata:
name: fortio
labels:
app: fortio
spec:
ports:
- port: 8080
name: http
selector:
app: fortio
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: fortio-deploy
spec:
replicas: 1
selector:
matchLabels:
app: fortio
template:
metadata:
annotations:
sidecar.istio.io/statsInclusionPrefixes: cluster.outbound,cluster_manager,listener_manager,http_mixer_filter,tcp_mixer_filter,server,cluster.xds-grpc
labels:
app: fortio
spec:
containers:
- name: fortio
image: fortio/fortio:latest_release
imagePullPolicy: Always
ports:
- containerPort: 8080
name: http-fortio
- containerPort: 8079
name: grpc-ping
查看运行状况
[root@harbor myapp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
fortio-deploy-6dc9b4d7d9-57jqg 2/2 Running 0 4m31s
istio-demo-75d8848475-ddrqd 2/2 Running 0 2d23h
myappv2-5c98456b77-g54ss 2/2 Running 0 5m58s
nginx-client-689b9b7f98-9sjdc 2/2 Running 0 6d17h
触发熔断器
[root@harbor myapp]# kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380
07:03:54 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
07:03:54 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
07:03:54 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
...........忽略..............
# target 50% 0.00182857
# target 75% 0.00366667
# target 90% 0.02
# target 99% 0.092646
# target 99.9% 0.0962182
Sockets used: 54 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 6 (10.0 %)
Code 503 : 54 (90.0 %)
Response Header Sizes : count 60 avg 24.5 +/- 73.5 min 0 max 245 sum 1470
Response Body/Total Sizes : count 60 avg 178.5 +/- 47.82 min 159 max 310 sum 10710
All done 60 calls (plus 0 warmup) 7.449 ms avg, 175.7 qps
Code 200 : 6 (10.0 %)
Code 503 : 54 (90.0 %)
可以看到有90%被熔断拦截，istio-proxy允许存在一些误差，所以每次执行并不一定会一样
[root@harbor myapp]# kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380
06:54:29 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
06:54:29 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
06:54:29 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
........忽略..........
> 0.012 <= 0.014 , 0.013 , 98.33, 1
> 0.04 <= 0.0412401 , 0.0406201 , 100.00, 1
# target 50% 0.0016087
# target 75% 0.00193478
# target 90% 0.004
# target 99% 0.040496
# target 99.9% 0.0411657
Sockets used: 58 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 2 (3.3 %)
Code 503 : 58 (96.7 %)
请求没有被熔断的，都是正常的
[root@harbor myapp]# kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380/admin/
07:10:05 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380/admin/
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
Ended after 552.357911ms : 60 calls. qps=108.63
Aggregated Function Time : count 60 avg 0.054438912 +/- 0.1574 min 0.000856591 max 0.526561805 sum 3.2663347
# range, mid point, percentile, count
>= 0.000856591 <= 0.001 , 0.000928295 , 11.67, 7
> 0.001 <= 0.002 , 0.0015 , 85.00, 44
> 0.014 <= 0.016 , 0.015 , 90.00, 3
> 0.5 <= 0.526562 , 0.513281 , 100.00, 6
# target 50% 0.00152273
# target 75% 0.00186364
# target 90% 0.016
# target 99% 0.523906
# target 99.9% 0.526296
Sockets used: 6 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 60 (100.0 %)
