k8s service
2021-12-07 本文已影响0人
Wu杰语
在基础网络架构方案上，k8s是怎么构建网络的呢，主要是service和ingress两个抽象，本文先探讨service。
service实例
首先创建一个后端service程序，就是一个简单的go语言的http服务器，DockerFile创建本地镜像，Dockerfile如下
FROM golang:latest
WORKDIR /home/goapp
ADD ./dockertest .
ADD start.sh .
RUN go build .
EXPOSE 8080
ENTRYPOINT ["./start.sh"]
mkdir /tmp
touch /tmp/healthy
./server
Dockerfile启动点是start.sh，主要是要执行多条语句，其中后面两句就是创建一个/tmp/healthy文件，这个文件是为健康探活使用的，k8s 探针会定期检查是否有这个文件，以确定Pod是否存活。
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-server
spec:
selector:
matchLabels:
app: my-server
replicas: 1
template:
metadata:
labels:
app: my-server
spec:
containers:
- name: my-server
image: server
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
resources:
requests:
cpu: "0.5"
memory: 200Mi
limits:
cpu: "0.5"
memory: 200Mi
livenessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 5
periodSeconds: 30
terminationGracePeriodSeconds: 60
这个文件用于创建后端服务Pod，其中有几个注意点：
- imagePullPolicy: IfNotPresent，因为想使用本地编译好的镜像，所以设置这个参数为IfNotPresent，意思是如果存在本地镜像就是用本地镜像，否则就从远程下载
- resources的request和limits设置程一样的，这样Pod的Qos就为Guaranteed
- 设置了livenessProbe，用于Pod的监控检查，检查方式为每30秒检查pod中是否有/tmp/healthy目录
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
type: NodePort
selector:
app: my-server
ports:
- protocol: TCP
port: 80
targetPort: 8080
nodePort: 30000
这个是个service定义，这个service代理的后端就是上面的Pod，可以通过命令
kubectl describe service
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations: <none>
Selector: <none>
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.96.0.1
IPs: 10.96.0.1
Port: https 443/TCP
TargetPort: 8443/TCP
Endpoints: 192.168.49.2:8443
Session Affinity: None
Events: <none>
Name: my-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=my-server
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.106.27.125
IPs: 10.106.27.125
Port: <unset> 80/TCP
TargetPort: 8080/TCP
NodePort: <unset> 30000/TCP
Endpoints: 172.17.0.4:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
可以看到EndPoints，这个EndPoint就是后端Pod，service通过轮询的方式来发送命令到EndPoints。
踩过的小坑
通过minikube使用本地镜像，一开始使用docker编译了镜像，但是镜像怎么都使用不了本地镜像，后来找到一篇文章，才搞明白，k8s使用的docker环境和本机的docker环境并非同一个，需要通过如下命令进行切换，然后在k8s的docker环境下编译镜像，这样才能让k8s看到本地镜像
sudo minikube start --driver=docker
eval $(minikube -p minikube docker-env)
service原理
service的访问如下图
service
理解这个我们需要明白service的目的，service的目的有两个，一个是供k8s集群内服务访问，一个是供k8s集群外服务访问。所以，service有几种类型(参见https://www.cnblogs.com/zhoushiya/p/12259886.html)：
-
cluster ip，这种类型就是一个集群ip，供k8s集群内服务访问
cluster ip
-
NodePort，这种类型在cluster ip的情况下，在服务所在的主机上开启了一个端口，通过这个端口可以从外部直接访问service，然后service再访问到主机
NodePort
但是这样一来，这个类型每多一个service，就要在相应的Node上开一个端口，资源消耗很大
-
LoadBalance
LoadBlance
在NodePort的基础上增加了一个负载均衡层，这个负载均衡层由各个云厂家提供。
-
ExternalName
再外部访问的基础上，再封装一次，提供域名访问。
可以看到，除了Cluster ip仅供内部服务访问，其它三种情况都可以提供外部访问。
代码实现上的思考
service不同于其它Department等对象，这个对象并不是从Pod继承
type Service struct {
metav1.TypeMeta `json:",inline"`
// Standard object's metadata.
// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
// +optional
metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
// Spec defines the behavior of a service.
// https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
// +optional
Spec ServiceSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`
// Most recently observed status of the service.
// Populated by the system.
// Read-only.
// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
// +optional
Status ServiceStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}
// ServiceSpec describes the attributes that a user creates on a service.
type ServiceSpec struct {
// The list of ports that are exposed by this service.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
// +patchMergeKey=port
// +patchStrategy=merge
// +listType=map
// +listMapKey=port
// +listMapKey=protocol
Ports []ServicePort `json:"ports,omitempty" patchStrategy:"merge" patchMergeKey:"port" protobuf:"bytes,1,rep,name=ports"`
// Route service traffic to pods with label keys and values matching this
// selector. If empty or not present, the service is assumed to have an
// external process managing its endpoints, which Kubernetes will not
// modify. Only applies to types ClusterIP, NodePort, and LoadBalancer.
// Ignored if type is ExternalName.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/
// +optional
// +mapType=atomic
Selector map[string]string `json:"selector,omitempty" protobuf:"bytes,2,rep,name=selector"`
// clusterIP is the IP address of the service and is usually assigned
// randomly. If an address is specified manually, is in-range (as per
// system configuration), and is not in use, it will be allocated to the
// service; otherwise creation of the service will fail. This field may not
// be changed through updates unless the type field is also being changed
// to ExternalName (which requires this field to be blank) or the type
// field is being changed from ExternalName (in which case this field may
// optionally be specified, as describe above). Valid values are "None",
// empty string (""), or a valid IP address. Setting this to "None" makes a
// "headless service" (no virtual IP), which is useful when direct endpoint
// connections are preferred and proxying is not required. Only applies to
// types ClusterIP, NodePort, and LoadBalancer. If this field is specified
// when creating a Service of type ExternalName, creation will fail. This
// field will be wiped when updating a Service to type ExternalName.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
// +optional
ClusterIP string `json:"clusterIP,omitempty" protobuf:"bytes,3,opt,name=clusterIP"`
// ClusterIPs is a list of IP addresses assigned to this service, and are
// usually assigned randomly. If an address is specified manually, is
// in-range (as per system configuration), and is not in use, it will be
// allocated to the service; otherwise creation of the service will fail.
// This field may not be changed through updates unless the type field is
// also being changed to ExternalName (which requires this field to be
// empty) or the type field is being changed from ExternalName (in which
// case this field may optionally be specified, as describe above). Valid
// values are "None", empty string (""), or a valid IP address. Setting
// this to "None" makes a "headless service" (no virtual IP), which is
// useful when direct endpoint connections are preferred and proxying is
// not required. Only applies to types ClusterIP, NodePort, and
// LoadBalancer. If this field is specified when creating a Service of type
// ExternalName, creation will fail. This field will be wiped when updating
// a Service to type ExternalName. If this field is not specified, it will
// be initialized from the clusterIP field. If this field is specified,
// clients must ensure that clusterIPs[0] and clusterIP have the same
// value.
//
// Unless the "IPv6DualStack" feature gate is enabled, this field is
// limited to one value, which must be the same as the clusterIP field. If
// the feature gate is enabled, this field may hold a maximum of two
// entries (dual-stack IPs, in either order). These IPs must correspond to
// the values of the ipFamilies field. Both clusterIPs and ipFamilies are
// governed by the ipFamilyPolicy field.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
// +listType=atomic
// +optional
ClusterIPs []string `json:"clusterIPs,omitempty" protobuf:"bytes,18,opt,name=clusterIPs"`
// type determines how the Service is exposed. Defaults to ClusterIP. Valid
// options are ExternalName, ClusterIP, NodePort, and LoadBalancer.
// "ClusterIP" allocates a cluster-internal IP address for load-balancing
// to endpoints. Endpoints are determined by the selector or if that is not
// specified, by manual construction of an Endpoints object or
// EndpointSlice objects. If clusterIP is "None", no virtual IP is
// allocated and the endpoints are published as a set of endpoints rather
// than a virtual IP.
// "NodePort" builds on ClusterIP and allocates a port on every node which
// routes to the same endpoints as the clusterIP.
// "LoadBalancer" builds on NodePort and creates an external load-balancer
// (if supported in the current cloud) which routes to the same endpoints
// as the clusterIP.
// "ExternalName" aliases this service to the specified externalName.
// Several other fields do not apply to ExternalName services.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
// +optional
Type ServiceType `json:"type,omitempty" protobuf:"bytes,4,opt,name=type,casttype=ServiceType"`
// externalIPs is a list of IP addresses for which nodes in the cluster
// will also accept traffic for this service. These IPs are not managed by
// Kubernetes. The user is responsible for ensuring that traffic arrives
// at a node with this IP. A common example is external load-balancers
// that are not part of the Kubernetes system.
// +optional
ExternalIPs []string `json:"externalIPs,omitempty" protobuf:"bytes,5,rep,name=externalIPs"`
// Supports "ClientIP" and "None". Used to maintain session affinity.
// Enable client IP based session affinity.
// Must be ClientIP or None.
// Defaults to None.
// More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies
// +optional
SessionAffinity ServiceAffinity `json:"sessionAffinity,omitempty" protobuf:"bytes,7,opt,name=sessionAffinity,casttype=ServiceAffinity"`
// Only applies to Service Type: LoadBalancer
// LoadBalancer will get created with the IP specified in this field.
// This feature depends on whether the underlying cloud-provider supports specifying
// the loadBalancerIP when a load balancer is created.
// This field will be ignored if the cloud-provider does not support the feature.
// +optional
LoadBalancerIP string `json:"loadBalancerIP,omitempty" protobuf:"bytes,8,opt,name=loadBalancerIP"`
// If specified and supported by the platform, this will restrict traffic through the cloud-provider
// load-balancer will be restricted to the specified client IPs. This field will be ignored if the
// cloud-provider does not support the feature."
// More info: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/
// +optional
LoadBalancerSourceRanges []string `json:"loadBalancerSourceRanges,omitempty" protobuf:"bytes,9,opt,name=loadBalancerSourceRanges"`
// externalName is the external reference that discovery mechanisms will
// return as an alias for this service (e.g. a DNS CNAME record). No
// proxying will be involved. Must be a lowercase RFC-1123 hostname
// (https://tools.ietf.org/html/rfc1123) and requires `type` to be "ExternalName".
// +optional
ExternalName string `json:"externalName,omitempty" protobuf:"bytes,10,opt,name=externalName"`
// externalTrafficPolicy denotes if this Service desires to route external
// traffic to node-local or cluster-wide endpoints. "Local" preserves the
// client source IP and avoids a second hop for LoadBalancer and Nodeport
// type services, but risks potentially imbalanced traffic spreading.
// "Cluster" obscures the client source IP and may cause a second hop to
// another node, but should have good overall load-spreading.
// +optional
ExternalTrafficPolicy ServiceExternalTrafficPolicyType `json:"externalTrafficPolicy,omitempty" protobuf:"bytes,11,opt,name=externalTrafficPolicy"`
// healthCheckNodePort specifies the healthcheck nodePort for the service.
// This only applies when type is set to LoadBalancer and
// externalTrafficPolicy is set to Local. If a value is specified, is
// in-range, and is not in use, it will be used. If not specified, a value
// will be automatically allocated. External systems (e.g. load-balancers)
// can use this port to determine if a given node holds endpoints for this
// service or not. If this field is specified when creating a Service
// which does not need it, creation will fail. This field will be wiped
// when updating a Service to no longer need it (e.g. changing type).
// +optional
HealthCheckNodePort int32 `json:"healthCheckNodePort,omitempty" protobuf:"bytes,12,opt,name=healthCheckNodePort"`
// publishNotReadyAddresses indicates that any agent which deals with endpoints for this
// Service should disregard any indications of ready/not-ready.
// The primary use case for setting this field is for a StatefulSet's Headless Service to
// propagate SRV DNS records for its Pods for the purpose of peer discovery.
// The Kubernetes controllers that generate Endpoints and EndpointSlice resources for
// Services interpret this to mean that all endpoints are considered "ready" even if the
// Pods themselves are not. Agents which consume only Kubernetes generated endpoints
// through the Endpoints or EndpointSlice resources can safely assume this behavior.
// +optional
PublishNotReadyAddresses bool `json:"publishNotReadyAddresses,omitempty" protobuf:"varint,13,opt,name=publishNotReadyAddresses"`
// sessionAffinityConfig contains the configurations of session affinity.
// +optional
SessionAffinityConfig *SessionAffinityConfig `json:"sessionAffinityConfig,omitempty" protobuf:"bytes,14,opt,name=sessionAffinityConfig"`
// TopologyKeys is tombstoned to show why 16 is reserved protobuf tag.
//TopologyKeys []string `json:"topologyKeys,omitempty" protobuf:"bytes,16,opt,name=topologyKeys"`
// IPFamily is tombstoned to show why 15 is a reserved protobuf tag.
// IPFamily *IPFamily `json:"ipFamily,omitempty" protobuf:"bytes,15,opt,name=ipFamily,Configcasttype=IPFamily"`
// IPFamilies is a list of IP families (e.g. IPv4, IPv6) assigned to this
// service, and is gated by the "IPv6DualStack" feature gate. This field
// is usually assigned automatically based on cluster configuration and the
// ipFamilyPolicy field. If this field is specified manually, the requested
// family is available in the cluster, and ipFamilyPolicy allows it, it
// will be used; otherwise creation of the service will fail. This field
// is conditionally mutable: it allows for adding or removing a secondary
// IP family, but it does not allow changing the primary IP family of the
// Service. Valid values are "IPv4" and "IPv6". This field only applies
// to Services of types ClusterIP, NodePort, and LoadBalancer, and does
// apply to "headless" services. This field will be wiped when updating a
// Service to type ExternalName.
//
// This field may hold a maximum of two entries (dual-stack families, in
// either order). These families must correspond to the values of the
// clusterIPs field, if specified. Both clusterIPs and ipFamilies are
// governed by the ipFamilyPolicy field.
// +listType=atomic
// +optional
IPFamilies []IPFamily `json:"ipFamilies,omitempty" protobuf:"bytes,19,opt,name=ipFamilies,casttype=IPFamily"`
// IPFamilyPolicy represents the dual-stack-ness requested or required by
// this Service, and is gated by the "IPv6DualStack" feature gate. If
// there is no value provided, then this field will be set to SingleStack.
// Services can be "SingleStack" (a single IP family), "PreferDualStack"
// (two IP families on dual-stack configured clusters or a single IP family
// on single-stack clusters), or "RequireDualStack" (two IP families on
// dual-stack configured clusters, otherwise fail). The ipFamilies and
// clusterIPs fields depend on the value of this field. This field will be
// wiped when updating a service to type ExternalName.
// +optional
IPFamilyPolicy *IPFamilyPolicyType `json:"ipFamilyPolicy,omitempty" protobuf:"bytes,17,opt,name=ipFamilyPolicy,casttype=IPFamilyPolicyType"`
// allocateLoadBalancerNodePorts defines if NodePorts will be automatically
// allocated for services with type LoadBalancer. Default is "true". It
// may be set to "false" if the cluster load-balancer does not rely on
// NodePorts. If the caller requests specific NodePorts (by specifying a
// value), those requests will be respected, regardless of this field.
// This field may only be set for services with type LoadBalancer and will
// be cleared if the type is changed to any other type.
// This field is beta-level and is only honored by servers that enable the ServiceLBNodePortControl feature.
// +featureGate=ServiceLBNodePortControl
// +optional
AllocateLoadBalancerNodePorts *bool `json:"allocateLoadBalancerNodePorts,omitempty" protobuf:"bytes,20,opt,name=allocateLoadBalancerNodePorts"`
// loadBalancerClass is the class of the load balancer implementation this Service belongs to.
// If specified, the value of this field must be a label-style identifier, with an optional prefix,
// e.g. "internal-vip" or "example.com/internal-vip". Unprefixed names are reserved for end-users.
// This field can only be set when the Service type is 'LoadBalancer'. If not set, the default load
// balancer implementation is used, today this is typically done through the cloud provider integration,
// but should apply for any default implementation. If set, it is assumed that a load balancer
// implementation is watching for Services with a matching class. Any default load balancer
// implementation (e.g. cloud providers) should ignore Services that set this field.
// This field can only be set when creating or updating a Service to type 'LoadBalancer'.
// Once set, it can not be changed. This field will be wiped when a service is updated to a non 'LoadBalancer' type.
// +featureGate=LoadBalancerClass
// +optional
LoadBalancerClass *string `json:"loadBalancerClass,omitempty" protobuf:"bytes,21,opt,name=loadBalancerClass"`
// InternalTrafficPolicy specifies if the cluster internal traffic
// should be routed to all endpoints or node-local endpoints only.
// "Cluster" routes internal traffic to a Service to all endpoints.
// "Local" routes traffic to node-local endpoints only, traffic is
// dropped if no node-local endpoints are ready.
// The default value is "Cluster".
// +featureGate=ServiceInternalTrafficPolicy
// +optional
InternalTrafficPolicy *ServiceInternalTrafficPolicyType `json:"internalTrafficPolicy,omitempty" protobuf:"bytes,22,opt,name=internalTrafficPolicy"`
}
它本身并不创建Pod对象，而是创建网络资源，存储上和其它对象一样，存储再etcd，通过controller watch变化。所以对于cluster ip对象，因为这个ip要在etcd上获取对那个的映射，因此只能供k8s集群内部访问，在集群外无法直接访问cluster ip类型的service。
而service对象背后的原理，还是k8s基础网络方案，如Flannel、Callico，这些基础镜像背后的原理，还是操作系统路由、iptable等基本原理。
小结
k8s service 是个有别于Depoloyment、Job等的对象，它不从Pod继承，只创建网络，并且提供了四种类型，其中cluster ip只提供集群内部访问，NodePort、LoadBalance、ExternalName提供集群外部访问。
