K8S学习笔记0522

发布时间：2024-02-25 08:30

K8S中资源限制

如果运行的容器没有定义资源(memory、CPU)等限制，但是在namespace定义了LimitRange限制，那么该容器会继承LimitRange中的默认限制。

如果namespace没有定义LimitRange限制，那么该容器可以只要宿主机的最大可用资源，直到无资源可用而触发宿主机(OOMKiller)。

CPU以核心为单位进行限制，单位可以是整核、浮点核心数或毫核(m/milli)。2=2核心=200%0.5=500m=50%1.2=1200m=120%

memory以字节为单位,单位可以是E、P、T、G、M、K、 Ei、Pi、Ti、Gi.Mi、 Ki。如1536Mi=1.5Gi

requests(请求)为kubernetes scheduler执行pod调度时node节点至少需要拥有的资源。

limits(限制)为pod运行成功后最多可以使用的资源上限。

备注：request不建议比limits小太多，如果request内存设置100m，limits内存设置512m，node节点内存只有300m，慢慢使用过程中会超过300m内存从而OOM

实例内存/CPU限制

创建pod

apiVersion: apps/v1
kind: Deployment
metadata:
  name: limit-test-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: limit-test-pod
  template:
    metadata:
      labels:
        app: limit-test-pod
    spec:
      containers:
      - name: limit-test-container
        image: k8s-harbor.com/public/docker-stress-ng:v1#压测镜像
        resources:
          limits:
            memory: "200Mi"
          requests:
            memory: "100Mi"
        args: ["--vm", "2", "--vm-bytes", "100M"]#2线程，每个线程占100m内存

添加cpu限制

apiVersion: apps/v1
kind: Deployment
metadata:
  name: limit-test-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: limit-test-pod
  template:
    metadata:
      labels:
        app: limit-test-pod
    spec:
      containers:
      - name: limit-test-container
        image: k8s-harbor.com/public/docker-stress-ng:v1
        resources:
          limits:
            cpu: "1"
            memory: "200Mi"
          requests:
            cpu: "0.5"
            memory: "100Mi"
        args: ["--vm", "2", "--vm-bytes", "100M"]

Pod的资源限制

Limit Range是对具体某个Pod或容器的资源使用进行限制：https://kubernetes.io/zh/docs/concepts/policy/limit-rangel
1、限制namespace中每个Pod或容器的最小与最大计算资源
2、限制namespace中每个Pod或容器计算资源request、 limit之间的比例
3、限制namespace中每个存储卷声明(PersistentVolumeClaim）可使用的最小与最大存储空间
4、设置namespace中容器默认计算资源的request. limit，并在运行时自动注入到容器中

root@k8s-master1 request-limit]# cat pod-limit.yaml 
apiVersion: v1
kind: LimitRange
metadata:
  name: limitrange-pod
  namespace: magedu
spec:
  limits:
  - type: Container     #限制资源类型
    max:
      cpu: "2"          #限制单个容器最大的CPU
      memory: "2Gi"     #限制单个容器最大的内存
    min:
      cpu: "500m"       #限制单个容器最小的CPU
      memory: "256Mi"   #限制单个容器最小的内存
    default:
      cpu: "500m"       #默认单个容器的CPU限制
      memory: "256Mi"   #默认单个容器的内存限制
    defaultRequest:
      cpu: "500m"       #默认单个容器的CPU创建请求
      memory: "256Mi"   #默认单个容器的内存创建请求
    maxLimitRequestRatio:
      cpu: "2"          #限制CPU limit/request比值最大为2
      memory: "2"       #限制内存 limit/request比值最大为2
  - type: Pod
    max:
      cpu: "2"          #限制单个Pod的最大CPU
      memory: "2Gi"     #限制单个Pod的最大内存
  - type: PersistentVolumeClaim
    max:
      storage: 50Gi     #限制PVC最大的storage
    min:
      storage: 50Gi     #限制PVC最小的storage

[root@k8s-master1 request-limit]# kubectl get limitranges -n magedu
NAME             CREATED AT
limitrange-pod   2022-05-24T14:41:49Z
[root@k8s-master1 request-limit]# kubectl describe limitranges limitrange-pod -n magedu
Name:                  limitrange-pod
Namespace:             magedu
Type                   Resource  Min    Max   Default Request  Default Limit  Max Limit/Request Ratio
----                   --------  ---    ---   ---------------  -------------  -----------------------
Container              cpu       500m   2     500m             500m           2
Container              memory    256Mi  2Gi   256Mi            256Mi          2
Pod                    cpu       -      2     -                -              -
Pod                    memory    -      2Gi   -                -              -
PersistentVolumeClaim  storage   50Gi   50Gi  -                -              -

Namespace资源限制

kubernetes对整个namespace的CPU及memory实现资源限制
https://kubernetes.io/zh/docs/concepts/policy/resource-quotas/

限定某个对象类型(如Pod. service)可创建对象的总数;
限定某个对象类型可消耗的计算资源(CPU、内存)与存储资源（存储卷声明)总数

apiVersion: v1
kind: ResourceQuota
metadata:
  name: quota-magedu
  namespace: magedu
spec:
  hard:
    requests.cpu: "8"   #一般等于宿主机最大CPU
    limits.cpu: "8"     #一般等于宿主机最大CPU
    requests.memory: 4Gi #一般等于宿主机最大内存 
    limits.memory: 4Gi   #一般等于宿主机最大内存
    requests.nvidia.com/gpu: 4
    pods: "20"  #pod数量，已经创建的不受影响
    services: "6"       #service数量

[root@k8s-master1 request-limit]# kubectl get deployments  -n magedu
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
magedu-nginx-deployment   2/5     2            2           2m17s


kubectl get deployments magedu-nginx-deployment   -n magedu -o json

K8S的亲和和反亲和

nodeSelector

简介

nodeSelector基于node标签选择器，将pod调度的指定的目的节点上.
https://kubernetes.io/zh/docs/concepts/scheduling-eviction/assign-pod-node/

可用于基于服务类型干预Pod调度结果，如对磁盘I/O要求高的pod调度到SSD节点，对内存要求比较高的pod调度的内存较高的节点。

也可以用于区分不同项目的pod，如将node添加不同项目的标签，然后区分调度。

查看默认标签

[root@k8s-master1 request-limit]# kubectl describe node 192.168.226.145
Name:               192.168.226.145
Roles:              node
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=192.168.226.145
                    kubernetes.io/os=linux
                    kubernetes.io/role=node

给节点打标签

[root@k8s-master1 request-limit]# kubectl label node 192.168.226.145 project=magedu
node/192.168.226.145 labeled
[root@k8s-master1 request-limit]# kubectl label node 192.168.226.145 disktype=ssd
node/192.168.226.145 labeled

给节点删除标签

[root@k8s-master1 request-limit]# kubectl label node 192.168.226.145 disktype-
node/192.168.226.145 unlabeled

pod调度指定node标签

[root@k8s-master1 nodesel]# cat case1-nodeSelector.yaml 
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine 
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
        resources:
          limits:
            cpu: 1
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "512Mi"
      nodeSelector: #pod全局
        project: magedu    #指定项目
        disktype: ssd    #指定磁盘类型

pod调度指定node名称

[root@k8s-master1 nodesel]# cat case2-nodename.yaml 
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      nodeName: 192.168.226.146
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine 
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
        resources:
          limits:
            cpu: 1
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "512Mi"

node affinty 节点亲和与反亲和

affinity是Kubernetes 1.2版本后引入的新特性，类似于nodeSelector，允许使用者指定一些Pod在Node间调度的约鬼，目前支持两种形式：

requiredDuringSchedulinglgnoredDuringExecution(不匹配调度失败)：#必须满足pod调度匹配条件，如果不满足则不进行调度
preferredDuringSchedulinglgnoredDuringExecution(不匹配但最终还会调度成功)：#倾向满足pod调度匹配条件，不满足的情况下会调度的不符合条件的Node上

lgnoreDuringExecution：表示如果在Pod运行期间Node的标签发生变化，致亲和性策略不能满足，也会继续运行当前的Pod。

Afinity与anti-affinity的目的也是控制pod的调度结果，但是相对于nodeSelector，亲和与反亲和功能更加强大

1、标签选择器不仅仅支持and，还支持In、NotIn、Exists、DoesNotExist、Gt、Lt；
2、可以设置软匹配和硬匹配，在软匹配下，如果调度器无法匹配节点，仍然将pod调度到其它不符合条件的节点；
3、还可以对pod定义亲和策略，比如允许哪些pod可以或者不可以被调度至同一台node.


In:标签的值存在匹配列表中(匹配成功就调度到目的node，实现node亲和)
Notln:标签的值不存在指定的匹配列表中(不会调度到目的node，实现反亲和)
Gt:标签的值大于某个值(字符串)
Lt:标签的值小于某个值(字符串)
Exists:指定的标签存在

硬亲和案例一

如果定义一个nodeSelectorTerms(条件)中通过一个matchExpressions基于列表指定了多个operator条件，则只要满足其中一个条件，就会被调度到相应的节点上，即or的关系，即如果nodeSelectorTerms下面有多个条件的话，只要满足任何一个条件就可以

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions: #匹配条件1,多个values可以调度
              - key: disktype
                operator: In
                values:
                - ssd # 只有一个value是匹配成功也可以调度
                - xxx
            - matchExpressions: #匹配条件2,多个matchExpressions加上以及每个matchExpressions values只有其中一个value匹配成功就可以调度
              - key: project
                operator: In
                values:
                - mmm #即使这俩条件2的都匹配不上也可以调度
                - nnn

硬亲和案例二

如果定义一个nodeSelectorTerms中都通过一个matchExpressions(匹配表达式)指定key匹配多个条件，则所有的目的条件都必须满足才会调度到对应的节点，即and的关系，即果matchExpressions有多个选项的话，则必须同时满足所有这些条件才能正常调度

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions: #硬亲和匹配条件1
              - key: disktype
                operator: In
                values:
                - ssd
                - xxx #同个key的多个value只有有一个匹配成功就行
              - key: project #硬亲和条件1和条件2必须同时满足,否则不调度
                operator: In
                values:
                - magedu

软亲和案例一

软亲和匹配不成功也会调度

    affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 80 #匹配成功就调度，不在往下面走
            preference:
              matchExpressions:
              - key: project
                operator: In
                values:
                  - mageduxx
          - weight: 60    #权重
            preference:
              matchExpressions:
              - key: disktype
                operator: In
                values:
                  - ssdxx

案例三软亲和与硬亲和结合使用

kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution: #硬亲和
            nodeSelectorTerms:
            - matchExpressions: #硬匹配条件1
              - key: "kubernetes.io/role" 
                operator: NotIn
                values:
                - "master" #硬性匹配key 的值kubernetes.io/role不包含master的节点,即绝对不会调度到master节点(node反亲和)
          preferredDuringSchedulingIgnoredDuringExecution: #软亲和
          - weight: 80 
            preference: 
              matchExpressions: 
              - key: project 
                operator: In 
                values: 
                  - magedu
          - weight: 60 
            preference: 
              matchExpressions: 
              - key: disktype
                operator: In 
                values: 
                  - ssd

node节点反亲和

[root@k8s-master1 nodesel]# cat case3-3.1-nodeantiaffinity.yaml
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions: #匹配条件1
              - key: disktype
                operator: NotIn #调度的目的节点没有key为disktype且值为hdd的标签
                values:
                - ssd #绝对不会调度到含有label的key为disktype且值为ssd的hdd的节点,即会调度到没有key为disktype且值为hdd的hdd的节点

pod级别亲和与反亲和

Pod亲和性与反亲和性可以基于已经在node节点上运行的Pod的标签来约束新创建的Pod可以调度到的目的节点，注意不是基于node上的标签而是使用的已经运行在node上的pod标签匹配。

其规则的格式为如果node节点A已经运行了一个或多个满足调度新创建的Pod B的规则，那么新的Pod B在亲和的条件下会调度到A节点之上，而在反亲和性的情况下则不会调度到A节点至上.

其中规则表示一个具有可选的关联命名空间列表的LabelSelector，只所以Pod亲和与反亲和需可以通过LabelSelecto选择namespace，是因为Pod是命名空间限定的而node不属于任何nemespace所以node的亲和与反亲和不需要namespace,因此作用于Pod标签的标签选择算符必须指定选择算符应用在哪个命名空间.

从概念上讲, node节点是一个拓扑域(具有拓扑结构的域)，比如k8s集群中的单台node节点、一个机架、云供应商可用区云供应商地理区域等，可以使用topologyKey来定义亲和或者分亲和的颗粒度是node级别还是可用区级别，以便K8S调度系统用来识别并选择正确的目的拓扑域。

pod亲和的写法

Pod亲和性与反亲和性的合法操作符(operator)有ln、NotIn、Exists、DoesNotExist.
在Pod亲和性配置中，在requiredDuringSchedulinglgnoredDuringExecution和preferredDuringSchedulinglgnoredDuringExecution中，topologyKey不元诗为空炬opty topologykey is not allowed.)。
在Pod反亲和性中配置中，requiredDuringSchedulinglgnoredDuringExecution和preferredDuringschedulinglgnoredDuringExecution中,topologyKey也不可以为空(Empty topologyKey is not allowed.).
对于requiredDuring5chedulinglanoredDuringExecution要求的Pod反亲和性，准入控制器LimitPodHardAntiAffinityTopology被引入以确保topologyKey只能是kubernetes.io/hostname，如果希望topologyKey也可用于其他定制拓扑逻辑，可以更改准入控制器或者禁用.除上述情况外, topologyKey可以是任何合法的标签键.

案例 pod软亲和

kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: python-nginx-deployment-label
  name: python-nginx-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: python-nginx-selector
  template:
    metadata:
      labels:
        app: python-nginx-selector
        project: python
    spec:
      containers:
      - name: python-nginx-container
        image: nginx:1.20.2-alpine
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        #imagePullPolicy: IfNotPresent
        imagePullPolicy: Always
        ports:
        - containerPort: 80
          protocol: TCP
          name: http
        - containerPort: 443
          protocol: TCP
          name: https
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
#        resources:
#          limits:
#            cpu: 2
#            memory: 2Gi
#          requests:
#            cpu: 500m
#            memory: 1Gi


---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: python-nginx-service-label
  name: python-nginx-service
  namespace: magedu
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30014
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
    nodePort: 30453
  selector:
    app: python-nginx-selector
    project: python #一个或多个selector，至少能匹配目标pod的一个标签

[root@k8s-master1 nodesel]# cat case4-4.2-podaffinity-preferredDuring.yaml 
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        podAffinity:
          #requiredDuringSchedulingIgnoredDuringExecution:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: project 
                  operator: In
                  values:
                    - python
              topologyKey: kubernetes.io/hostname 
              namespaces: 
                - magedu

案例 pod硬亲和

[root@k8s-master1 nodesel]# cat case4-4.3-podaffinity-requiredDuring.yaml 
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 3
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        podAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: project
                operator: In
                values:
                  - python1
            topologyKey: "kubernetes.io/hostname"
            namespaces:
              - magedu

pod 硬反亲和

kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: project
                operator: In
                values:
                  - python
            topologyKey: "kubernetes.io/hostname"
            namespaces:
              - magedu

pod软反亲和

#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app2-deployment-label
  name: magedu-tomcat-app2-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-tomcat-app2-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app2-selector
    spec:
      containers:
      - name: magedu-tomcat-app2-container
        image: tomcat:7.0.94-alpine
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: project 
                  operator: In
                  values:
                    - python
              topologyKey: kubernetes.io/hostname 
              namespaces: 
                - magedu

K8S中污点与容忍

污点(taints),用于node节点排斥Pod调度，与亲和的作用是完全相反的,即taint的node和pod是排斥调度关系,

容忍(pleration),用于Pod容忍node节点的污点信息，即node有污点信息也会将新的pod调度到node。

污点

污点三种类型

NoSchedule：表示不会将pid调度到具有该污点的节点上(硬限制)

kubectl taint node 192.168.226.146 key1=value1:NoSchedule    #设置污点

[root@k8s-master1 nodesel]# kubectl describe node 192.168.226.146    #查看污点
Name:               192.168.226.146
Roles:              node
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=192.168.226.146
                    kubernetes.io/os=linux
                    kubernetes.io/role=node
Annotations:        node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sun, 17 Apr 2022 19:27:24 +0800
Taints:             key1=value1:NoSchedule

kubectl taint node 192.168.226.146 key1:NoSchedule-    #取消污点

PrefeNoSchedule:表示k8s将尽量避免将Pod调度到具有该污点的Node上

NoExecute:表示k8s将不会将Pod调度到具有该污点的Node上，同时会将Node上已经存在的Pod强制驱逐出去

kubectl taint nodes 192.168.226.146 key1=value1:NoExecute

容忍

定义Pod的容忍度,可以调度至含有污点的node.

基于operator的污点匹配:
如果operator是Exists，侧容忍度不需要value而是直接匹配污点类型。
如果operator是Equal，则需要指定value并且value的值需要等于tolerations的key.

案例容忍pod

[root@k8s-master1 nodesel]# cat case5.1-taint-tolerations.yaml 
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app1-deployment-label
  name: magedu-tomcat-app1-deployment
  namespace: magedu
spec:
  replicas: 10
  selector:
    matchLabels:
      app: magedu-tomcat-app1-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app1-selector
    spec:
      containers:
      - name: magedu-tomcat-app1-container
        #image: harbor.magedu.local/magedu/tomcat-app1:v7
        image: tomcat:7.0.93-alpine 
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
#        env:
#        - name: "password"
#          value: "123456"
#        - name: "age"
#          value: "18"
#        resources:
#          limits:
#            cpu: 1
#            memory: "512Mi"
#          requests:
#            cpu: 500m
#            memory: "512Mi"

      tolerations: 
      - key: "key1"
        operator: "Equal"
        value: "value1"
        effect: "NoSchedule"

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: magedu-tomcat-app1-service-label
  name: magedu-tomcat-app1-service
  namespace: magedu
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
    #nodePort: 40003
  selector:
    app: magedu-tomcat-app1-selector

[root@k8s-master1 nodesel]# kubectl taint node 192.168.226.146 key1=value1:NoSchedule
node/192.168.226.146 tainted
[root@k8s-master1 nodesel]# kubectl taint node 192.168.226.145 key1=value1:NoSchedule
node/192.168.226.145 tainted

驱逐

节点压力驱逐是由各kubelet进程主动终止Pod，以回收节点上的内存、磁盘空间等资源的过程, kubelet监控当前node节点的CPU、内存、磁盘空间和文件系统的inode等资源，当这些资源中的一个或者多个达到特定的消耗水平kubelet就会主动地将节点上一个或者多个Pod强制驱逐，以防止当前node节点资源无法正常分配而引发的OOM.

taint 的 effect 值 NoExecute，它会影响已经在节点上运行的 pod：

如果 pod 不能容忍 effect 值为 NoExecute 的 taint，那么 pod 将马上被驱逐
如果 pod 能够容忍 effect 值为 NoExecute 的 taint，且在 toleration 定义中没有指定 tolerationSeconds，则 pod 会一直在这个节点上运行。
如果 pod 能够容忍 effect 值为 NoExecute 的 taint，但是在toleration定义中指定了 tolerationSeconds，则表示 pod 还能在这个节点上继续运行的时间长度

kubelet配置的K8S硬驱逐

/var/lib/kubelet/config.yaml

evictionHard:
  imagefs.available: 15%
  memory.available: 300Mi
  nodefs.available: 10%
  nodefs.inodesFree: 5%

驱逐(eviction，节点驱逐)，用于当node节点资源不足的时候自动将pod进行强制驱逐，以保证当前node节点的正常运行。

Kubernetes基于是QoS(服务质量等级)驱逐Pod , Qos等级包括目前包括以下三个：

Guaranteed: limits和request的值相等，等级最高、最后被驱逐
Burstable: limit和request不相等，等级折中、中间被驱逐
BestEffort:没有限制,即resources为空，等级最低、最先被驱逐

驱逐条件

eviction-signal: kubelet捕获node节点驱逐触发信号，进行判断是否驱逐，比如通过cgroupfs获取memory.available的值来进行下一步匹配.
operator:操作符，通过操作符对比条件是否匹配资源量是否触发驱逐.
quantity:使用量，即基于指定的资源使用值进行判断，如memory.available:30OMi、 nodefs.available:10%等.

比如: nodefs.available<10%
以上公式为当node节点磁盘空间可用率低于10%就会触发驱逐信号

软驱逐

软驱逐不会立即驱逐pod,可以自定义宽限期，在条件持续到宽限期还没有恢复,kubelet再强制杀死pod并触发驱逐

软驱逐条件:

eviction-soft:软驱逐触发条件，比如memory.available < 1.5Gi，如果驱逐条件持续时长超过指定的宽限期，可触发pod驱逐
eviction-soft-grace-period:软驱逐宽限期, 如memory.available=1m30s，定义软驱逐条件在触发Pod驱逐之前必须保持多久
eviction-max-pod-grace-period:终止pod的宽限期，即在满足软驱逐条件而终止Pod时使用的最大允许宽限期(以秒为单位)

K8S的准入控制

/root/.kube/config 证书文件

鉴权流程

K8S中鉴权类型分为node鉴权和RBAC(考试会考)

node(节点鉴权):针对kubelet发出的API请求进行鉴权。

授予node节点的kubelet读取services、 endpoints、 secrets、 configmaps等事件状态，并向APl server更新pod与node状态

[root@k8s-master1 ~]# vim /etc/systemd/system/kube-apiserver.service 

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
  --allow-privileged=true \
  --anonymous-auth=false \
  --api-audiences=api,istio-ca \
  --authorization-mode=Node,RBAC \    #默认类型

Webhook：一个HTTP回调，发生某些事情时调用的HTTP调用。如镜像扫描器

ABAC(Attribute-based access control ):基于属性的访问控制，1.6之前使用，将属性与账户直接绑定。

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1","kind" : "Policy ", "spec": {"user": "user1", "namespace :"*",resource :"*",apiGroup" : "*"}}#用户user1对所有namespace所有API版本的所有资源拥有所有权限((没有设置" readonly': true).

--authorization-mode=....RBAC,ABAC --authorization-policy-file=mypolicy.json: #开启ABAC参数

RBAC(Role-Based Access Control)：基于角色的访问控制,将权限与角色(role)先进行关联,然后将角色与用户进行绑定(Binding)从而继承角色中的权限。角色分为clusterrole和role，clusterrole是将集群中所有资源对象分配给一个用户，而role需要指定namespace/deployment

RBAC

RBAC简介

RBAC API声明了四种Kubernetes对象: Role、ClusterRole、RoleBinding和ClusterRoleBinding.Role:定义一组规则，用于访问命名空间中的Kubernetes资源。
RoleBinding:定义用户和角色(Role)的绑定关系。
ClusterRole:定义了一组访问集群中Kubernetes资源(包括所有命名空间)的规则。
ClusterRoleBinding:定义了用户和集群角色(ClusterRole)的绑定关系。