1984 字
10 分钟
Kubernetes 容器云学习手册

1. 容器云组件速查表#

组件说明
Kubernetes容器编排平台,自动化部署、扩展和管理容器化应用
KubeVirt为 K8S 提供虚拟机管理能力,实现 VM 与容器无缝集成
Istio服务网格,连接、管理和保护微服务
Grafana数据可视化和监控平台
Prometheus系统监控和警报工具,多维数据收集
Jaeger UI分布式追踪系统,监控微服务调用链
KialiIstio 服务网格可视化平台
Harbor企业级容器镜像仓库,镜像存储、签名和扫描
Docker应用容器引擎
Docker Compose多容器 Docker 应用定义和运行工具
DjangoPython Web 框架
JenkinsCI/CD 自动化工具
GitLabGit 仓库管理和 CI/CD 平台

2. Kubernetes 平台部署#

2.1 环境准备#

Terminal window
mount -o loop chinaskills_cloud_paas_v2.0.1.iso /mnt/
mv /opt/kubeeasy /usr/bin/kubeeasy
# 安装依赖
kubeeasy install depend \
--host 10.24.2.10,10.24.2.11 \
--user root --password Abc@1234 \
--offline-file /opt/dependencies/base-rpms.tar.gz
# 检查 SSH
kubeeasy check ssh \
--host 10.24.2.10,10.24.2.11 \
--user root --password Abc@1234
# 创建 SSH 密钥
kubeeasy create ssh-keygen \
--master 10.24.2.10 --worker 10.24.2.11 \
--user root --password Abc@1234

2.2 部署集群#

Terminal window
kubeeasy install kubernetes \
--master 10.24.2.10 --worker 10.24.2.11 \
--user root --password Abc@1234 \
--version 1.22.1 \
--offline-file /opt/kubernetes.tar.gz
kubectl cluster-info
kubectl top nodes --use-protocol-buffers

Dashboard:http://master_IP:30080(admin/Abc@1234)

2.3 部署示例 Pod#

apiVersion: v1
kind: Pod
metadata:
name: exam
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
env:
- name: "exam"
value: "2022"
Terminal window
kubectl apply -f pod.yaml
kubectl exec exam -- printenv

2.4 KubeVirt 虚拟机管理#

Terminal window
kubeeasy add --virt kubevirt
kubectl -n kubevirt get pods
# VMI 操作
kubectl create -f vmi.yaml
kubectl get vmis
kubectl delete vmis <vmi-name>
# VM 操作
virtctl start <vmi-name>
virtctl stop <vmi-name>
virtctl restart <vmi-name>

2.5 Istio 服务网格#

Terminal window
kubeeasy add --istio istio
kubectl -n istio-system get pods
istioctl version

访问地址:

  • Grafana:http://master_IP:33000
  • Prometheus:http://master_IP:30090
  • Jaeger:http://master_IP:30686
  • Kiali:http://master_IP:20001
Terminal window
istioctl profile list
istioctl profile dump demo
istioctl proxy-status
# 启用命名空间注入
kubectl create namespace exam
kubectl label namespace exam istio-injection=enabled

2.6 Harbor 镜像仓库#

Terminal window
kubeeasy add --registry harbor
# 访问 http://master_ip(admin/Harbor12345)

2.7 集群运维#

Terminal window
# 重置集群
kubeeasy reset
# 添加 Worker 节点
kubeeasy install depend --host 10.24.2.12 --user root --password Abc@1234 \
--offline-file /opt/dependencies/base-rpms.tar.gz
kubeeasy add --worker 10.24.2.12 --user root --password Abc@1234 \
--offline-file /opt/kubernetes.tar.gz

3. Helm 包管理#

3.1 Helm 简介#

Helm 是 Kubernetes 的包管理器,类似于 Linux 的 yum/apt。通过 Chart(包)来定义、安装和升级复杂的 K8S 应用。

概念说明
ChartHelm 包,包含一组 K8S 资源定义的文件集合
RepositoryChart 仓库,用于存储和分发 Chart
ReleaseChart 的运行实例,一个 Chart 可多次安装产生不同 Release
values.yamlChart 的默认配置值,安装时可覆盖

3.2 Chart 目录结构#

mychart/
├── Chart.yaml # Chart 元数据(名称、版本等)
├── values.yaml # 默认配置值
├── charts/ # 依赖的子 Chart
├── templates/ # K8S 资源模板
│ ├── deployment.yaml
│ ├── service.yaml
│ ├── ingress.yaml
│ └── _helpers.tpl # 模板辅助函数
└── templates/tests/ # 测试模板

3.3 常用命令#

Terminal window
# 创建 Chart
helm create mychart
rm -rf mychart/templates/*
# 编写 templates/deployment.yaml
# 安装和查看
helm install web mychart
helm status web
# 常用命令
helm version # 查看版本
helm list # 查看已安装 Release
helm search repo <chart-name> # 搜索仓库中的 Chart
helm search hub <keyword> # 搜索 ArtifactHub
helm status redis # 查看 Release 状态
helm uninstall web # 卸载 Release
helm delete --purge <chart-name> # 删除(Helm 2)
# Chart 开发
helm create helm_charts # 创建 Chart 骨架
helm lint # 语法检查
helm template helm_charts # 渲染模板查看生成的 YAML
helm package ./ # 打包 Chart
helm install web ./mychart -f custom-values.yaml # 使用自定义配置安装
helm install web ./mychart --set replicaCount=3 # 命令行覆盖配置

3.4 values.yaml 示例#

replicaCount: 2
image:
repository: nginx
tag: "1.21"
pullPolicy: IfNotPresent
service:
type: ClusterIP
port: 80
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 250m
memory: 64Mi

4. Docker Compose + DjangoBlog 部署#

4.1 构建四个镜像#

Memcached:

FROM centos:centos7.9.2009
RUN rm -rf /etc/yum.repos.d/*
COPY local.repo /etc/yum.repos.d/
COPY yum /root/yum
RUN yum install libevent libevent-devel -y && yum install memcached -y
EXPOSE 11211
ENTRYPOINT /usr/bin/memcached -u root

MariaDB:

FROM centos:centos7.9.2009
RUN rm -rf /etc/yum.repos.d/*
COPY local.repo /etc/yum.repos.d/
COPY yum /root/yum
ENV LC_ALL en_US.UTF-8
RUN yum -y install mariadb-server
COPY mysql_init.sh /opt/
COPY sqlfile.sql /opt
RUN bash /opt/mysql_init.sh
EXPOSE 3306
CMD ["mysqld_safe","--user=root"]

Nginx:

FROM centos:centos7.9.2009
RUN rm -rf /etc/yum.repos.d/*
COPY local.repo /etc/yum.repos.d/
COPY yum /root/yum
RUN yum -y install nginx
ADD nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx","-g","daemon off;"]

DjangoBlog:

FROM centos:centos7.9.2009
RUN rm -rfv /etc/yum.repos.d/*
COPY local.repo /etc/yum.repos.d/
COPY yum /root/yum
RUN yum install -y make openssl-devel bzip2-devel expat-devel gdbm-devel readline-devel sqlite-devel gcc gcc-devel python-devel mysql-devel
COPY Python-3.6.5.tgz /opt
RUN tar -zxvf /opt/Python-3.6.5.tgz && mv Python-3.6.5 /usr/local
RUN cd /usr/local/Python-3.6.5/ && ./configure && make && make install
RUN ln -s /usr/local/python3/bin/python3 /usr/bin/python3
RUN ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
ADD requirements.txt requirements.txt
RUN pip3 install --upgrade pip --no-index --find-links=/opt
RUN pip3 install -r requirements.txt --no-index --find-links=/opt
RUN pip3 install gunicorn[gevent] --no-index --find-links=/opt
RUN mkdir -p /code/djangoBlog
ADD . /code/djangoBlog/
RUN chmod +x /code/djangoBlog/bin/docker_start.sh
ENTRYPOINT ["/code/djangoBlog/bin/docker_start.sh"]

4.2 Compose 编排#

version: '3'
services:
memcached:
restart: always
image: blog-memcached:v1.0
container_name: blog-memcached
ports: ["11211:11211"]
db:
image: blog-mysql:v1.0
restart: always
environment: [MYSQL_DATABASE=djangoblog, MYSQL_ROOT_PASSWORD=root]
ports: ["3306:3306"]
depends_on: [memcached]
container_name: blog-mysql
djangoblog:
image: blog-service:v1.0
restart: always
ports: ["8000:8000"]
environment:
- DJANGO_MYSQL_HOST=db
- DJANGO_MEMCACHED_LOCATION=memcached:11211
links: [db, memcached]
depends_on: [db]
container_name: blog-service
nginx:
restart: always
image: blog-nginx:v1.0
ports: ["8888:80", "443:443"]
links: [djangoblog:djangoblog]
container_name: blog-nginx
Terminal window
docker-compose up -d
# 访问 http://IP:8888(admin/123456)

5. 基于 K8S 构建 CI/CD#

5.1 部署 Harbor 和导入镜像#

Terminal window
tar -zxf BlueOcean.tar.gz
cp BlueOcean/tools/docker-compose-Linux-x86_64 /usr/bin/docker-compose
tar -zxf BlueOcean/harbor-offline-installer.tar.gz -C /opt/
sh /opt/harbor/install.sh
# 新建 springcloud 项目(公开)
docker load -i BlueOcean/images/maven_latest.tar
docker tag maven 10.26.15.244/library/maven
docker push 10.26.15.244/library/maven

5.2 部署 Jenkins#

Terminal window
kubectl create ns devops
kubectl -n devops apply -f jenkins-deploy.yaml
# 访问 http://master:30880
kubectl -n devops exec deploy/jenkins -- cat /var/jenkins_home/secrets/initialAdminPassword
kubectl -n devops cp BlueOcean/plugins/ <pod>:/var/jenkins_home
kubectl -n devops rollout restart deployment jenkins

5.3 部署 GitLab#

Terminal window
kubectl -n devops apply -f gitlab-deploy.yaml
# 访问 http://master:30888(root/admin@123)

5.4 Jenkinsfile 流水线#

pipeline{
agent none
stages{
stage('mvn-build'){
agent { docker { image '10.26.15.244/library/maven' } }
steps{
sh 'mvn package -DskipTests'
archiveArtifacts artifacts: '**/target/*.jar'
}
}
stage('image-build'){
agent any
steps{
sh 'docker build -t 10.26.15.244/springcloud/gateway -f Dockerfile .'
sh 'docker login 10.26.15.244 -u=admin -p=Harbor12345'
sh 'docker push 10.26.15.244/springcloud/gateway'
}
}
stage('cloud-deploy'){
agent any
steps{
sh 'kubectl apply -f yaml/deployment/gateway-deployment.yaml'
sh 'kubectl apply -f yaml/svc/gateway-svc.yaml'
}
}
}
}

5.5 Webhook 触发#

  1. GitLab:Settings → Network → 允许本地网络请求
  2. GitLab:创建 Access Token
  3. Jenkins:系统配置 → 添加 GitLab 连接
  4. Jenkins:全局安全配置 → 任何用户可以做任何事
  5. GitLab:Settings → Webhooks → 添加 Jenkins URL
  6. 测试 Push events → HTTP 200 表示成功

6. GitLab 构建持续集成#

6.1 部署 GitLab 到 K8S#

Terminal window
kubectl create ns gitlab-ci
# 编写 gitlab-deploy.yaml 和 gitlab-svc.yaml
kubectl apply -f gitlab-deploy.yaml
kubectl apply -f gitlab-svc.yaml

6.2 配置 CoreDNS#

Terminal window
kubectl edit configmap coredns -n kube-system
# 添加 hosts 配置,删除 forward 相关行
kubectl -n kube-system rollout restart deploy coredns

6.3 部署 GitLab Runner#

Terminal window
# 创建 ServiceAccount 和 RBAC
kubectl apply -f runner-sa.yaml runner-role.yaml runner-rb.yaml default.yaml
# Helm 安装 Runner
helm -n gitlab-ci install gitlab-runner gitlab-runner

6.4 编写 .gitlab-ci.yml#

stages: [build, release, review]
maven_build:
image: maven:3.6-jdk-8
stage: build
script:
- mvn clean install -DskipTests=true
image_build:
image: docker:18.09.7
stage: release
script:
- docker build -t demo:latest .
- docker push 172.129.20.38/demo/demo:latest
deploy_review:
image: kubectl:1.22
stage: review
script:
- kubectl apply -f template/

7. 容器云竞赛真题精选#

7.1 容器化部署监控系统#

Dockerfile 模板(Node Exporter):

FROM centos:centos7.9.2009
WORKDIR /root
COPY node_exporter-0.18.1.linux-amd64.tar.gz /root/
RUN tar -zxvf node_exporter-0.18.1.linux-amd64.tar.gz && \
mv node_exporter-0.18.1.linux-amd64/* /usr/local/bin/
EXPOSE 9100
CMD ["/usr/local/bin/node_exporter"]

Docker Compose 编排:

version: '3'
services:
monitor-node:
container_name: monitor-node
image: monitor-exporter:v1.0
ports: ["9100:9100"]
monitor-alertmanager:
container_name: monitor-alertmanager
image: monitor-alert:v1.0
ports: ["9093:9093", "9094:9094"]
monitor-grafana:
container_name: monitor-grafana
image: monitor-grafana:v1.0
ports: ["3000:3000"]
monitor-prometheus:
container_name: monitor-prometheus
image: monitor-prometheus:v1.0
ports: ["9090:9090"]

7.2 Istio Ingress Gateway#

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: bookinfo-gateway
spec:
selector:
istio: ingressgateway
servers:
- hosts: ["*"]
port:
number: 80
name: http
protocol: HTTP
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: bookinfo
spec:
hosts: ["*"]
gateways: [bookinfo-gateway]
http:
- match:
- uri: {exact: /productpage}
route:
- destination:
host: productpage
port: {number: 9080}

7.3 KubeVirt 创建 VM#

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: vm-fedora
spec:
running: true
template:
spec:
domain:
devices:
disks:
- disk: {bus: virtio}
name: containerdisk
resources:
requests:
memory: 1Gi
volumes:
- name: containerdisk
containerDisk:
image: fedora-virt:v1.0

7.4 QoS Guaranteed Pod#

apiVersion: v1
kind: Pod
metadata:
name: qos-demo
spec:
containers:
- name: nginx
image: nginx:latest
resources:
limits: {memory: "100Mi", cpu: "500m"}
requests: {memory: "100Mi", cpu: "500m"}
restartPolicy: Always

7.5 Pod 安全上下文#

apiVersion: v1
kind: Pod
metadata:
name: context-demo
spec:
containers:
- name: context-demo
image: busybox:latest
command: ["sh", "-c", "sleep 1h"]
securityContext:
runAsUser: 1000
runAsGroup: 3000

7.6 CRD 自定义资源#

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: students.stable.example.com
spec:
group: stable.example.com
versions:
- name: v1
served: true
storage: true
scope: Namespaced
names:
plural: students
singular: student
shortNames: [stu]
kind: Student

7.7 HPA 自动扩缩容#

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: web
spec:
maxReplicas: 1000
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: web
behavior:
scaleUp:
policies:
- periodSeconds: 5
type: Percent
value: 900
metrics:
- type: Pods
pods:
metric: {name: cpu}
target: {averageValue: 80, type: AverageValue}

7.8 HostAliases#

apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx:latest
hostAliases:
- ip: "127.0.0.1"
hostnames: [chinaskill]

7.9 Python K8S SDK — Job 管理#

import time, yaml
from kubernetes.client import BatchV1Api
from kubernetes.config import load_kube_config
load_kube_config()
api = BatchV1Api()
try:
api.delete_namespaced_job(name='pi', namespace='default')
except: pass
time.sleep(15)
with open("spec-pi-job.yaml") as f:
data = yaml.safe_load(f)
api.create_namespaced_job(body=data, namespace="default")
get = api.read_namespaced_job(name="pi", namespace="default")
print(get)

7.10 Python 自定义调度器#

from kubernetes import client
import kubernetes as k8s
import random, json
def getUseNode(api):
nodes = api.list_node()
return [n.metadata.name for n in nodes.items
if n.status.conditions[-1].status == "True"]
def getScheduledPod(api, ns):
pods = api.list_namespaced_pod(ns)
return [p.metadata.name for p in pods.items
if p.status.phase == 'Pending' and p.spec.node_name is None]
def podBinding(api, pod, node, ns):
target = client.V1ObjectReference(kind="Node", api_version="v1", name=node)
meta = client.V1ObjectMeta(name=pod)
body = client.V1Binding(target=target, metadata=meta)
api.create_namespaced_binding(ns, body)
k8s.config.load_kube_config()
api = client.CoreV1Api()
nodes = getUseNode(api)
pods = getScheduledPod(api, "default")
node = random.choice(nodes)
for pod in pods:
podBinding(api, pod, node, "default")

7.11 Python REST API 封装 Pod 管理#

from http.server import HTTPServer, BaseHTTPRequestHandler
import requests, json, yaml
headers = {"Content-Type": "application/json", "Authorization": "bearer token"}
url = "https://master_IP:6443/api/v1/namespaces/default/pods"
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.startswith('/pod/'):
name = self.path.split('/')[-1]
r = requests.get(f"{url}/{name}", headers=headers, verify=False)
self.send_response(200)
self.send_header("Content-type", "application/json")
self.end_headers()
self.wfile.write(json.dumps(r.json()).encode())
def do_POST(self):
if self.path.startswith('/pod/'):
filename = self.path.split('/')[-1]
with open(filename) as f:
data = yaml.safe_load(f)
r = requests.post(url, headers=headers, data=json.dumps(data), verify=False)
self.send_response(200)
self.send_header("Content-type", "application/json")
self.end_headers()
self.wfile.write(json.dumps(r.json()).encode())
HTTPServer(("localhost", 8889), MyHandler).serve_forever()
Kubernetes 容器云学习手册
https://blog.xeu.asia/posts/kubernetes-guide/
作者
Xeu
发布于
2024-06-18
许可协议
CC BY-NC-SA 4.0