当前位置：首页 > article >正文

Linux——K8s pod调度

article 2024/11/20 18:29:49

rc/rs
deployment
statefulset
daemonset
job | cronjob

访问pod中的应用：在pod已经处于running状态之下，客户端的请求如何到达pod中的应用？

K8S 平台本身的网络架构设计：
1. coredns 属于K8S核心组件，提供K8S集群内的名称解析，集群内所有有命名的资源都可以通过名称访问，实际名称和资源直接的对应关系由COREDNS解决。每一个pod在创建的时候，都会进行命名和划分IP地址，名称和IP地址都会在coredns进行注册。因为在K8S集群内部的访问，基本上都会使用名称来进行访问。
2. CNI（容器网络插件）：不同网络插件，提供不同的网络功能，部分插件的功能的重叠，不同插件彼此之间只要配置不冲突的话，就可以同时部署多个网络插件。在目前的实验环境下，使用的网络插件为flannel的插件。
3. 前两点都是关于集群内部的pod如何通信，而真实客户端的请求不会直接从集群内发起，因此K8S还需要解决如何让集群外的客户访问到集群内的应用

为了符合K8S的网络架构，一般会出现三种不同的IP地址：

nodeIP 节点的IP地址
podIP pod对应的IP地址
ClusterIP 集群IP地址。这个IP地址配合必要的路由策略，就可以让客户端的请求给到K8S集群内的pod中。

简单的K8S服务访问模型

service 暴露pod 相关实验过程:
[root@control ~]# docker pull quay.io/rnoushi/busyboxplus:curl
curl: Pulling from rnoushi/busyboxplus
a3ed95caeb02: Pull complete
72d86f26813c: Pull complete
f45cff1e8e73: Pull complete
Digest: sha256:4cd8ccdc346a1ccf22228f18e3a6bc2d21f81cfa6600023b3a3669ab3f432e88
Status: Downloaded newer image for quay.io/rnoushi/busyboxplus:curl
quay.io/rnoushi/busyboxplus:curl
[root@control ~]# kubectl run curl --image=radial/busyboxplus:curl --image-pull-policy=IfNotPresent -i --tty --rm
[root@control ~]docker save -o bsp.tar quay.io/rnoushi/busyboxplus
[root@control ~]# scp bsp.tar root@node1:/root
root@node1's password:
bsp.tar                                                                                                                                   100% 4645KB  50.6MB/s   00:00
[root@control ~]# scp bsp.tar root@node2:/root
root@node2's password:
bsp.tar                                                                    


[root@node1 ~]# ctr -n k8s.io   image import bsp.tar
unpacking quay.io/rnoushi/busyboxplus:curl (sha256:5ecd23315d7624d62020e3d3478127692446599944702b2be4e1f4                                                                                                              b5584af3a8)...done
[root@node1 ~]# ctr -n k8s.io   image tag quay.io/rnoushi/busyboxplus:curl docker.io/radial/busyboxplus:curl
docker.io/radial/busyboxplus:curl


[root@node2 ~]# ctr -n k8s.io   image import bsp.tar
unpacking quay.io/rnoushi/busyboxplus:curl (sha256:5ecd23315d7624d62020e3d3478127692446599944702b2be4e1f4                                                                                                              b5584af3a8)...done
[root@node2 ~]# ctr -n k8s.io   image tag quay.io/rnoushi/busyboxplus:curl docker.io/radial/busyboxplus:curl
docker.io/radial/busyboxplus:curl


[root@control ~]# kubectl delete deployments.apps my-nginx nginx-deployment
deployment.apps "my-nginx" deleted
deployment.apps "nginx-deployment" deleted
[root@control ~]# kubectl delete statefulsets.apps web
statefulset.apps "web" deleted
[root@control ~]# kubectl delete service
serviceaccounts  services
[root@control ~]# kubectl delete service nginx 
service "nginx" deleted
[root@control ~]#
[root@control ~]# kubectl get pods
NAME                   READY   STATUS      RESTARTS   AGE
hello-28790283-zvqdh   0/1     Completed   0          2m54s
hello-28790284-b7c9r   0/1     Completed   0          114s
hello-28790285-thfbv   0/1     Completed   0          54s
[root@control ~]# kubectl delete cronjobs.batch hello
cronjob.batch "hello" deleted

[root@control ~]# vim app1.yml
[root@control ~]# cat app1.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 2
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80

[root@control ~]# kubectl apply -f app1.yml
deployment.apps/my-nginx created
[root@control ~]# kubectl get deployments.apps my-nginx
NAME       READY   UP-TO-DATE   AVAILABLE   AGE
my-nginx   2/2     2            2           17s
[root@control ~]# kubectl get pods -l run=my-nginx -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED NODE   READINESS GATES
my-nginx-74485854c4-4rt28   1/1     Running   0          85s   10.244.1.44   node1   <none>           <none>
my-nginx-74485854c4-7jx8t   1/1     Running   0          85s   10.244.2.55   node2   <none>           <none>
[root@control ~]# kubectl get pods -l run=my-nginx -o custom-columns=POD_IP:.status.podIPs
POD_IP
[map[ip:10.244.1.44]]
[map[ip:10.244.2.55]]
[root@control ~]# curl 10.244.1.44
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@control ~]# curl 10.244.2.55
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@control ~]# kubectl logs my-nginx-74485854c4-4rt28
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
10.244.0.0 - - [27/Sep/2024:06:12:05 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.76.1" "-"
[root@control ~]# kubectl logs my-nginx-74485854c4-7jx8t
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
10.244.0.0 - - [27/Sep/2024:06:12:10 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.76.1" "-"
[root@control ~]#
[root@control ~]# kubectl expose deployment/my-nginx
service/my-nginx exposed
[root@control ~]# kubectl get service
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   10m
my-nginx     ClusterIP   10.101.191.194   <none>        80/TCP    8s
[root@control ~]# kubectl describe service my-nginx
Name:                     my-nginx
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 run=my-nginx
Type:                     ClusterIP
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.101.191.194
IPs:                      10.101.191.194
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
Endpoints:                10.244.2.55:80,10.244.1.44:80
Session Affinity:         None
Internal Traffic Policy:  Cluster
Events:                   <none>
[root@control ~]# curl 10.101.191.194
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@control ~]# curl 10.101.191.194
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@control ~]# kubectl logs my-nginx-74485854c4-7jx8t
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
10.244.0.0 - - [27/Sep/2024:06:12:10 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.76.1" "-"
10.244.0.0 - - [27/Sep/2024:06:18:43 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.76.1" "-"
[root@control ~]#
[root@control ~]# kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
my-nginx-74485854c4-4rt28   1/1     Running   0          13m
my-nginx-74485854c4-7jx8t   1/1     Running   0          13m
[root@control ~]# kubectl exec my-nginx-74485854c4-4rt28 -- printenv | grep SERVICE
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_HOST=10.96.0.1
KUBERNETES_SERVICE_PORT=443
[root@control ~]# kubectl scale deployment my-nginx --replicas=0
deployment.apps/my-nginx scaled
[root@control ~]# kubectl scale deployment my-nginx --replicas=2
deployment.apps/my-nginx scaled
[root@control ~]# kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
my-nginx-74485854c4-87vhn   1/1     Running   0          7s
my-nginx-74485854c4-c7pgb   1/1     Running   0          7s
[root@control ~]# kubectl exec my-nginx-74485854c4-4rt28 -- printenv | grep SERVICE
Error from server (NotFound): pods "my-nginx-74485854c4-4rt28" not found
[root@control ~]# kubectl exec my-nginx-74485854c4-87vhn -- printenv | grep SERVICE
MY_NGINX_SERVICE_PORT=80
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_HOST=10.96.0.1
KUBERNETES_SERVICE_PORT_HTTPS=443
MY_NGINX_SERVICE_HOST=10.101.191.194
[root@control ~]# kubectl run curl --image=radial/busyboxplus:curl -i --tty --rm
If you don't see a command prompt, try pressing enter.
[ root@curl:/ ]$ nslookup my-nginx
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      my-nginx
Address 1: 10.101.191.194 my-nginx.default.svc.cluster.local
[ root@curl:/ ]$ nslookup my-nginx.my-nginx
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

nslookup: can't resolve 'my-nginx.my-nginx'
[ root@curl:/ ]$ nslookup my-nginx
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      my-nginx
Address 1: 10.101.191.194 my-nginx.default.svc.cluster.local
[ root@curl:/ ]$ curl my-nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[ root@curl:/ ]$ curl my-nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[ root@curl:/ ]$ exit
Session ended, resume using 'kubectl attach curl -c curl -i -t' command when the pod is running
pod "curl" deleted

设置节点externalIP

此时为空

[root@control ~]# kubectl edit node node1

node/node1 edited

[root@control ~]# kubectl edit svc my-nginx

service/my-nginx edited
[root@control ~]# kubectl describe svc my-nginx
Name:                     my-nginx
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 run=my-nginx
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.101.191.194
IPs:                      10.101.191.194
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  32394/TCP   // 自动定义port
Endpoints:                10.244.2.56:80,10.244.1.45:80
Session Affinity:         None
External Traffic Policy:  Cluster
Internal Traffic Policy:  Cluster
Events:                   <none>

就可以从指定的32394端口访问到服务了

需要注意受限于实验环境，此时使用的IP依旧是内部IP而给外部IP ，尽管设置了。同事内部IP和外部IP一致也有可能导致网络错误，请避免这样配置。

[root@control ~]# curl 192.168.110.11:32394
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

日志记录：