Kubernetes 高可用集群与 CI/CD 流水线部署
发布时间：2026/9/7 21:00:06 作者：尧图编辑部 阅读量：1,286
一、环境概述在 VMware Workstation 上部署 4 台 RHEL 9 虚拟机构建 K3s 三节点控制平面高可用集群并在此基础上落地 GitHub Actions ArgoCD 的 CI/CD 流水线。二、技术栈组件选型操作系统RHEL 9容器运行时containerdK3s 内置集群K3s3 server 节点内置 etcd代码托管GitHub持续集成GitHub Actions自托管 Runner镜像仓库Harbor本地内网持续部署ArgoCD / kubectl规划中三、节点规划主机名角色内存IPk8s-master1server2G192.168.100.11k8s-master2server2G192.168.100.12k8s-master3server2G192.168.100.13k8s-node1agent2G192.168.100.21harbor镜像仓库4G192.168.100.31四、系统初始化1. 操作系统安装安装 RHEL 9Minimal首台安装完成后克隆其余 3 台。2. 网络与主机名hostnamectl set-hostname k8s-master1 nmcli con mod ens160 ipv4.addresses 192.168.100.11/24 nmcli con mod ens160 ipv4.gateway 192.168.100.2 nmcli con mod ens160 ipv4.dns 114.114.114.114 nmcli con mod ens160 ipv4.method manual nmcli con up ens160/etc/hosts 解析192.168.100.11 k8s-master1 192.168.100.12 k8s-master2 192.168.100.13 k8s-master3 192.168.100.21 k8s-node13. 订阅与系统配置subscription-manager register subscription-manager attach --auto systemctl disable --now firewalld sed -i s/^SELINUX.*/SELINUXdisabled/ /etc/selinux/config setenforce 0五、镜像加速K3s 默认从 Docker Hub 拉取镜像通过 registry mirror 配置加速# /etc/rancher/k3s/registries.yaml mirrors: docker.io: endpoint: - https://example.mirror.example.com六、集群部署1. 初始化第一个 server 节点curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRRORcn sh -s - server --cluster-init获取 node-tokencat /var/lib/rancher/k3s/server/node-token2. 加入其余 server 节点curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRRORcn sh -s - server --server https://192.168.100.11:6443 --token TOKEN3. 加入 agent 节点curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRRORcn K3S_URLhttps://192.168.100.11:6443 K3S_TOKENTOKEN sh -4. 验证k3s kubectl get nodes4 个节点 Ready3 个 server 组成高可用控制平面。七、Harbor 镜像仓库部署1. 安装 Dockerharbor 节点安装 Docker CE 及 Compose 插件dnf install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin systemctl enable --now docker2. 下载离线安装包Harbor 提供离线安装包内置全部组件镜像无需联网拉取下载 v2.15.2 离线包并上传至 harbor 节点tar -xzf harbor-offline-installer-v2.15.2.tgz3. 生成配置基于模板生成配置修改关键项cd harbor cp harbor.yml.tmpl harbor.yml修改内容hostname设为 harbor 节点 IP注释https段内网环境走 HTTP设置harbor_admin_password。4. 执行安装cd harbor ./install.sh安装脚本加载内置组件镜像registry、core、portal、jobservice、数据库、redis并以 Compose 方式启动全部容器。5. 验证docker psHarbor 各组件容器处于 Up 状态浏览器访问管理界面http://192.168.100.31默认管理员账号admin登录后创建项目demo访问级别公开。6. 推送镜像构建演示应用镜像mkdir demo-app vim Dockerfile FROM nginx:alpine COPY index.html /usr/share/nginx/html/index.html vim index.html html body h1Hello Kubernetes/h1 pCI/CD demo v1/p /body /html docker build -t 192.168.100.31/demo/demo-app:v1 .配置 insecure-registriesHarbor 为 HTTP 仓库:vim /etc/docker/daemon.json { insecure-registries: [192.168.100.31] } docker login 192.168.100.31 docker push 192.168.100.31/demo/demo-app:v17. K3s 节点信任 Harbor在 4 台 K3s 节点的 registries.yaml 中加入 Harbor# /etc/rancher/k3s/registries.yaml mirrors: docker.io: endpoint: - https://example.mirror.example.com 192.168.100.31: endpoint: - http://192.168.100.31修改后逐台重启 k3s / k3s-agent 使配置生效。八、应用部署与验证1. 部署 demo-appapiVersion: apps/v1 kind: Deployment metadata: name: demo-app spec: replicas: 1 selector: matchLabels: app: demo-app template: metadata: labels: app: demo-app spec: containers: - name: demo-app image: 192.168.100.31/demo/demo-app:v1 ports: - containerPort: 80 k3s kubectl apply -f demo-app.yaml2. 暴露服务apiVersion: v1 kind: Service metadata: name: demo-app spec: type: NodePort selector: app: demo-app ports: - port: 80 targetPort: 80 nodePort: 300803. 验证k3s kubectl get pods curl http://192.168.100.11:30080demo-app 容器 Running页面正常返回。九、CI/CD 流水线代码托管于 GitHubCI 采用 GitHub Actions自托管 Runner镜像仓库为本地 Harbor实现代码提交到集群部署的全自动闭环。1. 架构git push → GitHub Actions 触发 → 自托管 RunnerHarbor 节点本地执行 → docker build → 推送 Harbor → kubectl set image 更新 K3s Deployment2. 自托管 Runner 部署GitHub Actions 默认在云端虚拟机执行无法访问内网 Harbor 与 K3s 集群因此部署自托管 Runner 于 Harbor 节点具备 Docker 环境由 GitHub 调度本地执行构建与部署。2.1 安装 Runner在 GitHub 仓库页面 Settings → Actions → Runners → New self-hosted runner 获取下载地址与注册令牌。于 Harbor 节点执行mkdir -p /opt/actions-runner cd /opt/actions-runner curl -o actions-runner-linux-x64-2.337.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.337.0/actions-runner-linux-x64-2.337.0.tar.gz tar xzf actions-runner-linux-x64-2.337.0.tar.gz2.2 注册 RunnerRunner 不允许以 root 运行创建专用用户并加入 docker 组构建需调用 Dockeruseradd -m runner usermod -aG docker runner chown -R runner:runner /opt/actions-runner su - runner cd /opt/actions-runner ./config.sh --url https://github.com/用户名/k8s-cicd-demo --token 注册令牌config.sh 交互提示均回车使用默认值注册完成后显示 Runner successfully added。前台运行测试连接./run.sh输出 Listening for Jobs 表示已成功连接 GitHub等待任务调度。2.3 Runner 环境配置Runner 需具备 Docker、kubectl 及集群访问凭据DockerHarbor 节点已安装runner 用户已加入 docker 组kubectl安装与集群版本一致的客户端curl -x http://192.168.100.1:10808 -LO https://dl.k8s.io/release/v1.36.4/bin/linux/amd64/kubectl chmod x kubectl mv kubectl /usr/local/bin/kubeconfig将 K3s 生成的 k3s.yaml控制平面节点上 /etc/rancher/k3s/k3s.yaml放置于 runner 用户并将 server 地址由 127.0.0.1 改为控制平面节点地址mkdir -p ~/.kube cp k3s.yaml ~/.kube/config sed -i s/127.0.0.1:6443/192.168.100.11:6443/ ~/.kube/config chmod 600 ~/.kube/config验证集群访问kubectl get nodes2.4 Runner 服务化run.sh 前台运行占用终端注册为系统服务常驻sudo ./svc.sh install sudo ./svc.sh start服务化后 Runner 随系统自启GitHub 仓库 Runner 列表显示为绿色 Idle 状态。3. GitHub Secrets 配置推送镜像至 Harbor 需要认证Harbor 密码以 Secret 存储避免明文写入 workflow仓库 Settings → Secrets and variables → Actions → New repository secretNameHARBOR_PASSWORDValueHarbor 管理员密码。4. 流水线配置workflow 文件.github/workflows/ci.ymlname: Build and Deploy to K3s on: push: branches: [ main ] jobs: build-deploy: runs-on: self-hosted steps: - name: Checkout uses: actions/checkoutv4 - name: Build image run: | docker build -t 192.168.100.31/demo/demo-app:${{ github.sha }} . - name: Push to Harbor run: | echo ${{ secrets.HARBOR_PASSWORD }} | docker login 192.168.100.31 -u admin --password-stdin docker push 192.168.100.31/demo/demo-app:${{ github.sha }} - name: Deploy to K3s run: | kubectl set image deployment/demo-app demo-app192.168.100.31/demo/demo-app:${{ github.sha }} -n default配置说明runs-on: self-hosted任务调度至本地 Runner 执行镜像标签使用github.shacommit SHA每次提交生成唯一镜像HARBOR_PASSWORDSecret 通过 stdin 传入 docker login不落盘推送后kubectl set image更新 Deployment 镜像触发滚动更新。5. 效果验证修改应用代码并推送至 main 分支后workflow 自动执行拉取代码、构建镜像、推送 Harbor、更新 Deployment。全程无需人工干预。浏览器访问应用页面即可看到新版本生效完成 CI/CD 全自动闭环验证。
尧图网站开发 